Recall

Your Personal Semantic Memory

A local-first RAG system where you upload documents and ask questions in natural language.

By default, retrieval runs entirely on your machine. Optional API providers (Gemini embeddings, Jina reranking) trade privacy for speed on weak hardware.

Quick Start · Learn AI Concepts · API Reference · Experiments

Recall is a Retrieval-Augmented Generation (RAG) application built as an AI Engineering learning project. You upload .txt, .md, or .pdf documents, and Recall indexes them so you can search and ask questions about their content using natural language.

How it works

You ask:  "How does authentication work?"
              ↓
Recall searches your documents for relevant passages
              ↓
Gemini generates a grounded answer with citations
              ↓
You get:  "Authentication uses JWT tokens [1]. Configure
           the secret via JWT_SECRET [2]."

Embeddings	Vector Search	BM25	RRF	Reranking	Generation
Local / API	Local	Local	Local	Local/API	Gemini

Part 1 - Project & Setup

1. What Is Recall?

Recall is a personal knowledge application. You upload documents, and Recall indexes them so you can search and ask questions about their content.

When you ask a question like:

"How does authentication work in this project?"

Recall:

Searches your indexed documents for relevant passages
Ranks the most relevant passages
Passes those passages to Gemini as context
Generates an answer based on that context
Cites its sources so you can verify

Design principle: The LLM should not be responsible for remembering your documents. Retrieval supplies the knowledge; the LLM performs reasoning over that knowledge.

2. Quick Start

Prerequisites

Node.js 18+ (recommended: 20+)
npm (or pnpm)
A Gemini API key (free tier available)
~2GB free disk space (for model downloads on first run)

Install & Run

git clone <repo-url>
cd recall
npm install

Set up your environment:

cp .env.example .env

Edit .env and add your Gemini API key:

GEMINI_API_KEY=your_key_here

That's the only required key. Everything else runs locally.

Start the dev server:

npm run dev

Open http://localhost:3000, upload a document, and ask a question.

Docker

docker compose up -d --build   # build & start on http://localhost:3001
docker compose down            # stop

The compose file reads your .env, mounts ./data and ./.cache as volumes (indexes and models persist across rebuilds), and bakes the local embedding model into the image via the WARM_EMBEDDINGS build arg.

Deploy to Render (free tier, no credit card)

The repo includes a Dockerfile. Render's Docker runtime builds it automatically — docker-compose.yml is local-only and ignored.

Push the repo to GitHub, then in dashboard.render.com create a Web Service from the repo (language: Docker)
Add environment variables:
- GEMINI_API_KEY
- EMBED_PROVIDER=gemini (fast uploads on the free tier's 0.1 vCPU)
- RERANK_PROVIDER=jina + JINA_RERANKING_API_KEY (avoids loading local models)
Deploy

Because the dashboard cannot pass build args, Render builds a slim image with no ML models baked in (WARM_EMBEDDINGS=false, WARM_RERANKER=false by default) — correct, since API providers need no model files.

Free-tier caveats:

Caveat	Impact
Spins down after 15 min idle	~1 min cold start on the next request
Ephemeral disk	`data/lancedb` is wiped on restart/redeploy — re-upload documents
512 MB RAM / 0.1 vCPU	Fine with API providers; local inference is very slow here

Switching embedding providers or dimensions? Vectors are not compatible across providers/dimensions. Delete data/lancedb/ and re-index all documents after changing EMBED_PROVIDER or GEMINI_EMBED_DIMENSION.

First Run Notes

On the first upload or query, Recall downloads ML models to the .cache/ directory (~180MB total). This only happens once, models are cached for subsequent runs.

Component	Behavior
Embedding model	With `EMBED_PROVIDER=local` (default), downloads `all-MiniLM-L6-v2` (~90MB) on first use. `gemini` needs no download.
Reranking model	If `RERANK_PROVIDER=local` (default), downloads `ms-marco-MiniLM-L-6-v2` (~90MB). Set `RERANK_PROVIDER=jina` to skip.
Vector database	LanceDB persists to `data/lancedb/`
Keyword index	MiniSearch persists to `data/minisearch.json`

Troubleshooting

"GEMINI_API_KEY not found" - Check that .env exists and contains your key. Restart the dev server after editing.

Model download fails - The first query requires internet access to download models from HuggingFace. Check your connection and retry.

Search returns "I don't have that information in your documents." - This is expected when no relevant chunk is found. Upload a document with content matching your question, or ask something different.

PDF parsing returns no text - Some PDFs are image-based (scanned). Recall requires text-based PDFs. Try a .txt or .md file instead.

3. Project Structure

recall/
├── app/
│   ├── api/
│   │   ├── upload/route.ts        # POST /api/upload — document ingestion
│   │   ├── search/route.ts        # POST /api/search — query with SSE streaming
│   │   └── documents/route.ts     # GET/DELETE /api/documents — list & delete
│   │
│   ├── components/                # UI components
│   │   ├── FileUpload.tsx         # File picker with upload logic
│   │   ├── DocumentManager.tsx    # Lists indexed documents, delete capability
│   │   ├── SearchBar.tsx          # Query input form
│   │   ├── Results.tsx            # Displays retrieved chunks with scores
│   │   └── Answer.tsx             # Streamed markdown answer with citations
│   │
│   └── lib/                       # Core AI pipeline (the interesting part)
│       ├── config.ts              # Pipeline constants (chunk size, top-k, etc.)
│       ├── types.ts               # Shared TypeScript types
│       ├── parsers.ts             # Text extraction (.txt/.md/.pdf)
│       ├── chunker.ts             # Overlapping word-window chunking
│       ├── embeddings.ts          # On-device MiniLM embeddings
│       ├── vectordb.ts            # LanceDB vector store
│       ├── minisearch.ts          # MiniSearch BM25 index
│       ├── search.ts              # Hybrid search + Reciprocal Rank Fusion
│       ├── rerank.ts              # Cross-encoder reranking (local or Jina API)
│       └── generate.ts            # Gemini grounded prompt + streaming
│
├── data/                          # Persisted indexes (auto-created, gitignored)
│   ├── lancedb/                   # Vector database
│   └── minisearch.json            # BM25 index
│
├── eval/                          # Offline retrieval evaluation
│   ├── sample-docs/               # 6 benchmark documents
│   └── qa.jsonl                   # 10 gold-standard queries
│
├── docs/                          # Part 2 — Learn AI Concepts guide
│
├── scripts/
│   ├── evaluate.ts                # Benchmark script (npm run eval)
│   └── warm-models.mjs            # Pre-downloads ML models for Docker builds
│
├── Dockerfile                     # Production image (Render-ready)
├── docker-compose.yml             # Local containerized run
├── .env.example                   # Environment variables template
├── .env                           # Your local configuration (gitignored)
└── package.json

The `app/lib/` Pipeline

Each file in app/lib/ maps to one stage of the AI pipeline, making it easy to study one concept at a time:

parsers.ts       →  Extract text from files
      ↓
chunker.ts       →  Split text into smaller pieces
      ↓
embeddings.ts    →  Convert text chunks into vectors
      ↓
vectordb.ts      →  Store vectors for semantic search
minisearch.ts    →  Store text for keyword search
      ↓
search.ts        →  Find relevant chunks using both methods
      ↓
rerank.ts        →  Reorder chunks by relevance
      ↓
generate.ts      →  Generate an answer using the best chunks

4. Technology Stack & Packages

Core Framework

Package	Version	Purpose
`next`	16.3.0	Full-stack React framework (App Router, API routes)
`react` / `react-dom`	19.2.8	UI library
`typescript`	^5	Type safety (strict mode enabled)

AI & ML

Package	Version	Purpose
`@google/genai`	^2.17.1	Google Gemini SDK — answer generation + optional embeddings
`@huggingface/transformers`	^4.2.0	Runs ML models on-device (embeddings + reranking)
`Xenova/all-MiniLM-L6-v2`	(model)	Local embedding model — converts text to 384-dim vectors
`gemini-embedding-001`	(model)	API embedding alternative — 768/1536/3072-dim vectors
`Xenova/ms-marco-MiniLM-L-6-v2`	(model)	Cross-encoder reranker — scores query-passage relevance

Data & Search

Package	Version	Purpose
`@lancedb/lancedb`	^0.37.1	Local vector database for semantic search
`minisearch`	^7.2.0	Lightweight BM25/keyword search library
`pdf-parse`	^2.4.5	Extracts text from PDF files

UI & Styling

Package	Version	Purpose
`tailwindcss`	^4	Utility-first CSS framework
`react-markdown`	^10.1.0	Renders LLM answers as Markdown in the UI
`remark-gfm`	^4.0.1	GitHub-Flavored Markdown support

Dev Tools

Package	Version	Purpose
`eslint` / `eslint-config-next`	^9 / 16.3.0	Linting
`tsx`	^4.23.12	TypeScript execution for the eval script
`dotenv`	^17.4.2	Environment variable loading

5. Configuration

All options live in .env. Copy .env.example to .env and adjust as needed.

Variable	Required	Default	Purpose
`GEMINI_API_KEY`	Yes (for generation)	—	Google AI Studio API key. Local retrieval works without it.
`GEMINI_MODEL`	No	`gemini-2.5-flash`	Generation model override
`EMBED_PROVIDER`	No	`local`	`local` = on-device MiniLM; `gemini` = Gemini Embedding API
`GEMINI_EMBED_MODEL`	No	`gemini-embedding-001`	Embedding model when `EMBED_PROVIDER=gemini`
`GEMINI_EMBED_DIMENSION`	No	`768`	Gemini embedding dimensions (768, 1536, or 3072)
`RERANK_PROVIDER`	No	`local`	`local` = on-device cross-encoder; `jina` = API reranker
`JINA_RERANKING_API_KEY`	Only if `RERANK_PROVIDER=jina`	—	Jina API key
`JINA_RERANK_MODEL`	No	`jina-reranker-v2-base-multilingual`	Jina model to use

Warning: vectors are not compatible across embedding providers or dimensions. After changing EMBED_PROVIDER or GEMINI_EMBED_DIMENSION, delete data/lancedb/ and re-index all documents.

Choosing an Embedding Provider

	`local` (default)	`gemini`
Model	`Xenova/all-MiniLM-L6-v2`	`gemini-embedding-001`
Runs on	Your machine	Google's servers
Privacy	Fully local, no data leaves your machine	Document text is sent to the API
Speed	CPU-bound; slow on weak hardware	Near-instant, even on small cloud hosts
Dimensions	384	768 (default), 1536, or 3072
Cost	Free	Free tier: 1,000 requests/day
Best for	Privacy-first local use	Deployments on small/weak hosts

Documents are embedded with RETRIEVAL_DOCUMENT and queries with RETRIEVAL_QUERY, so Gemini optimizes each vector for its role. All Gemini embeddings are L2-normalized (including truncated dimensions), so cosine similarity works out of the box.

Choosing a Reranker Provider

	`local` (default)	`jina`
Model	`Xenova/ms-marco-MiniLM-L-6-v2`	`jina-reranker-v2-base-multilingual`
Runs on	Your machine	Jina's servers
Privacy	Fully local, no data leaves your machine	Query + chunks sent to Jina
Latency	No network, but uses CPU/GPU	Network round-trip
Setup	Model auto-downloads to `.cache/` (~90MB)	Requires API key
Best for	Privacy-first local use	Serverless or weak hardware

6. API Reference

`POST /api/upload`

Uploads and indexes a document.

Request: multipart/form-data with a file field.

curl -X POST http://localhost:3000/api/upload \
  -F "file=@my-notes.md"

Pipeline: Upload → Parse → Chunk → Embed → Store (LanceDB + MiniSearch)

Response:

{
  "success": true,
  "fileName": "my-notes.md",
  "chunksIndexed": 42
}

Supported formats: .txt, .md, .pdf (text-based only) Max file size: 10 MB Re-uploading the same filename replaces the old document

`POST /api/search`

Searches indexed documents and streams a grounded answer.

Request:

{
  "query": "How does authentication work?"
}

Response: Server-Sent Events (SSE) stream.

Event	Payload	Description
`sources`	`Chunk[]`	Top retrieved chunks (sent first)
`token`	`string`	Answer text deltas as they're generated
`error`	`{ message: string }`	Generation failure message
`done`	—	Stream complete

Pipeline: Query → Embed → Hybrid Search (RRF) → Rerank → Gemini → Grounded Answer

Example SSE stream:

event: sources
data: [{"id":"chunk-1","text":"...","docId":"a1b2c3","score":0.92}, ...]

event: token
data: The

event: token
data:  application

event: done

Retrieval happens entirely on-device. Only the final generation calls Gemini. If reranking fails, the system falls back to the RRF ranking.

`GET /api/documents`

Lists all indexed documents.

[
  {
    "docId": "a1b2c3d4",
    "source": "my-notes.md",
    "chunkCount": 42
  }
]

`DELETE /api/documents?docId=...`

Deletes a document and all its chunks from both indexes.

curl -X DELETE "http://localhost:3000/api/documents?docId=a1b2c3d4"

7. Evaluation Harness

Recall ships with an offline retrieval benchmark plus an optional LLM-as-a-judge phase for the generation layer, covering the full six-metric report card:

Layer	Metrics	When
Retrieval	Precision@K, Recall@K, MRR	Always runs
Generation	Faithfulness, answer relevance, correctness	`--judge` flag

Run It

npm run eval                  # Index sample docs, then evaluate retrieval
npm run eval -- --fresh       # Wipe indexes first for an isolated benchmark
npm run eval -- --no-ingest   # Evaluate against already-indexed documents
npm run eval -- --judge       # Also grade generation with an LLM-as-a-judge (needs GEMINI_API_KEY)
npm run eval -- --judge --limit 5    # Judge phase on the first 5 questions only (cheap smoke test)

What It Measures

Dataset: 6 sample documents + 10 gold-standard queries, each with an expected text snippet (for retrieval relevance) and a human-written ground-truth answer (for generation correctness)
Relevance: A retrieved chunk is "relevant" if it contains the expected snippet
Retrieval metrics: Precision@K, Recall@K, MRR — reported for both pre-rerank and post-rerank results
Generation metrics: --judge runs each question's final top-K context through the real generation prompt, then three independent judges grade the answer: faithfulness (answer vs. retrieved context), answer relevance (answer vs. question), and correctness (answer vs. ground truth)

Each judge returns a score in [0, 1] with temperature: 0. Judge failures are reported as skipped rather than silently scored 0.

Sample Output

Retrieval evaluation (higher is better)

metric          | RRF (pre-rerank) | Reranked
------------------------------------------------------
Precision@5     | 18.7%            | 21.7%
Recall@5        | 91.3%            | 95.7%
MRR             | 0.768            | 0.842

Generation evaluation (LLM-as-a-judge, 0.0–1.0, higher is better)

metric           | avg   | judged
----------------------------------------
Faithfulness     | 0.950 | 10/10
Answer relevance | 0.911 | 10/10
Correctness      | 0.870 | 10/10

Note: on this dataset each question has one gold chunk, so a Recall@5 of ~95% means the gold chunk made the top-5 context ~~95% of the time; Precision@5 is low by construction (~~Recall@5/5) since each top-5 window can contain at most one gold chunk. Precision is more informative on real corpora where a question has multiple answerable chunks. The values above are illustrative.

Why Evaluate Retrieval and Generation Separately?

An incorrect final answer doesn't tell you where the system failed:

Failure Type	What Happens	Investigate
Retrieval failure	Wrong chunks found	Chunking, embeddings, BM25, RRF, reranking
Generation failure	Right chunks, wrong answer	Prompt construction, LLM behavior

The six metrics triangulate the failure:

Low precision → the retriever is pulling in noise → lower top_k or raise the similarity threshold
Low recall → relevant chunks exist but weren't returned → raise top_k
Low MRR → the right chunk ranks too low → better embeddings or reranking
Low faithfulness → the answer drifted from the context → tighten the grounding prompt
Low answer relevance → the answer dodged the question → tighten the answer-the-question instruction
Low correctness → check whether retrieval failed first, then the knowledge base itself

Reproducibility note: retrieval scores use whatever embedding/reranking providers are configured. The generation metrics add two more variables: the answer model (GEMINI_MODEL) and the judge, which uses that same model to grade. Provider defaults produce the sample numbers above; switching providers changes them — which is exactly what makes the harness useful for comparing them.

Part 2 - Learn AI Concepts

The full concept guide now lives in docs/. Each document explains one AI engineering concept and maps it directly to the file in app/lib/ that implements it.

#	Document	Concept
8	What Is RAG?	Retrieval-Augmented Generation fundamentals
9	The Recall Architecture	Ingestion + query phases, file map
10	Phase 1 — Document Ingestion	Parsing, chunking, embeddings, vector storage
11	Phase 2 — Retrieval	Semantic search, BM25, hybrid search, RRF
12	Phase 3 — Reranking	Bi-encoders vs cross-encoders, two-stage retrieval
13	Phase 4 — Generation	Prompt construction, grounding, citations, abstention
14	Streaming AI Responses	Server-Sent Events, perceived latency
15	End-to-End Request Flow	The complete system in one diagram
16	Real Query Walkthrough	A single query traced through every step
17	Important Trade-offs	Quality vs latency, local vs API, context size
18	RAG Failure Modes	Six ways a RAG system fails
19	Evaluating a RAG System	Precision, Recall, MRR with worked examples
20	How to Study This Project	Suggested reading order
21	Experiment Lab	Hands-on experiments with expected observations
22	Final Mental Model	The one-paragraph summary

Project status: Local-first RAG demo. Retrieval runs on-device by default; optional API providers (Gemini embeddings, Jina reranking) are available for deployments on weak hardware. Generation always calls Gemini.

Name		Name	Last commit message	Last commit date
app		app
docs		docs
eval		eval
public		public
scripts		scripts
.dockerignore		.dockerignore
.env.example		.env.example
.gitignore		.gitignore
AGENTS.md		AGENTS.md
CLAUDE.md		CLAUDE.md
Dockerfile		Dockerfile
README.md		README.md
docker-compose.yml		docker-compose.yml
eslint.config.mjs		eslint.config.mjs
next.config.ts		next.config.ts
package-lock.json		package-lock.json
package.json		package.json
postcss.config.mjs		postcss.config.mjs
tsconfig.json		tsconfig.json

Latest commit

History

Folders and files

Repository files navigation

Recall

Your Personal Semantic Memory

How it works

Table of Contents

Part 1 - Project & Setup

1. What Is Recall?

2. Quick Start

Prerequisites

Install & Run

Docker

Deploy to Render (free tier, no credit card)

First Run Notes

3. Project Structure

The app/lib/ Pipeline

4. Technology Stack & Packages

Core Framework

AI & ML

Data & Search

UI & Styling

Dev Tools

5. Configuration

Choosing an Embedding Provider

Choosing a Reranker Provider

6. API Reference

POST /api/upload

POST /api/search

GET /api/documents

DELETE /api/documents?docId=...

7. Evaluation Harness

Run It

What It Measures

Sample Output

Why Evaluate Retrieval and Generation Separately?

Part 2 - Learn AI Concepts

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

The `app/lib/` Pipeline

`POST /api/upload`

`POST /api/search`

`GET /api/documents`

`DELETE /api/documents?docId=...`