RAG Q&A System

Ask natural-language questions over the document corpus in corpus/ and get grounded, cited answers. Design rationale: see DESIGN.md.

Setup

# with uv
uv sync

# or with pip
pip install -r requirements.txt

Set the API key (either put OPENAI_API_KEY=... in .env, or):

export OPENAI_API_KEY="$SCALER_LLM_API_KEY"

Run

uv run uvicorn main:app --port 8000    # or: uvicorn main:app --port 8000

The index builds automatically at startup. To pick up files added to corpus/ while the server is running (incremental — only new files are embedded):

curl -X POST localhost:8000/index

Ask a question:

curl -X POST localhost:8000/ask \
  -H 'Content-Type: application/json' \
  -d '{"question": "What is the refund policy?"}'

Every query appends a structured JSON trace (scores, prompt, latency split) to trace.log.

Eval

Runs 10 test cases through the pipeline and scores them with an LLM judge (faithfulness + context precision):

uv run python -m eval.run_eval

Layout

main.py            FastAPI app: POST /index, POST /ask
src/ingest.py      loaders + type-aware chunking (prose / FAQ / records)
src/index.py       OpenAI embeddings -> normalized in-memory numpy matrix
src/retrieve.py    cosine top-k + similarity floor
src/generate.py    grounded generation (gpt-4o), citations, no-info fallback
src/trace.py       one JSON trace line per query
eval/              test cases + LLM-judge eval script
scripts/           similarity-floor tuning helper