Skip to content

memory-and-rag.md

Latest commit

 

History

History
144 lines (111 loc) · 5.03 KB

memory-and-rag.md

File metadata and controls

144 lines (111 loc) · 5.03 KB
Raw

Memory and retrieval

Two different things that are easy to confuse.

Memory is what the agent chooses to remember about you and your work. Small, durable, and injected into every system prompt.

Retrieval is search over a body of documents you indexed. Large, on demand, and only what a query pulls back.

Memory

The agent decides what is worth keeping and writes it with the memory tool. Good memories are the things that stay true: which database this project uses, that you prefer no semicolons, that the staging box is srv-2.

memory:
  memory_enabled: true
  user_profile_enabled: true
  memory_char_limit: 4000
  search_limit: 10
  nudge_interval: 20

memory_char_limit bounds what goes into the prompt. Past it, the most recently used survive.

Managing it

/memory                    recent memories
/memory postgres           search
/remember staging is srv-2
/remember db: postgres 16 on srv-2
/forget db

key: value sets an explicit key, which is what /forget takes. The dashboard's Memory page lists everything with a delete on each.

What not to store

Anything that changes — today's date, the current branch, what you are working on this afternoon. A stale memory is worse than no memory, because the agent believes it.

Retrieval

Point it at documentation, a codebase, or a pile of notes, and the agent can search meaning rather than exact words.

rag:
  enabled: true
  embed_model: text-embedding-3-small
  chunk_size: 1200
  chunk_overlap: 150
  recall: 40          # candidates pulled before rerank/dedup narrow to top_k
  top_k: 8
  hybrid: true
  rerank_mode: llm    # llm | api | off
  compress: true      # drop near-duplicate results
  auto_context: true  # index conversations and auto-recall into every turn

Indexing

antares rag index ~/projects/myapp
antares rag index ~/notes --collection notes

Or from the dashboard's Memory & RAG page, or with the rag_index tool during a conversation.

Collections keep bodies separate so a search can be scoped.

How it works (native, in-process)

Retrieval is built in — no external daemon, and no database extension. Vectors live in the Antares database, embedded with your configured model. A query runs a four-stage pipeline:

  1. Recall — hybrid search pulls recall candidates (default 40). The dense side runs against a per-collection HNSW graph cached in-process; the graph is built lazily on the first search and invalidated by a persisted revision counter, so an out-of-process writer stays visible without a manual refresh. The lexical side uses the same full-text index the rest of the app uses — FTS5 on SQLite, tsvector/GIN on Postgres — so no extension is needed on either backend, in particular not pgvector. When hybrid is on, dense and lexical hits are fused with reciprocal-rank fusion.
  2. Rerank — the candidates are reordered by relevance to the query. rerank_mode: llm (default) has an auxiliary model score them; api calls an external reranker (rerank_url + rerank_api_key, Voyage/Jina/Cohere-shaped); off keeps retrieval order. Rerank is separate from embedding.
  3. Compress — with compress: true, near-duplicate results are collapsed.
  4. Top-K — the best top_k (default 8) are returned.

The graph trades a one-time build cost and some resident memory for a per-query cost that no longer scales with the collection size — a rebuild is a full scan of the collection's embeddings and the cache holds one graph per active collection. On very small collections a straight SQL scan can beat the graph, and the crossover depends on both size and embedding dimensionality; the intent is that larger, higher-dimensional corpora — the ones where a scan is actually painful — are the ones the graph helps. Benchmarks live next to the code (go test ./internal/store -run TestVectorIndex -bench BenchmarkVectorIndex).

Auto-context (living memory)

With auto_context: true, retrieval is wired into every chat turn: each finished exchange is indexed into a conversations collection, and relevant indexed knowledge plus past conversation is pulled back into the system prompt automatically. It is best-effort and never blocks a turn.

Chunking

chunk_size is characters, not tokens. 1200 with 150 overlap suits prose and code alike. Larger chunks give more context per hit and fewer hits; smaller chunks are more precise and more numerous.

Re-index after changing it — existing chunks keep the old size.

Session search

Separate from both, and needs no setup: session_search is full-text search across every past conversation, backed by FTS5 on SQLite and tsvector on Postgres.

"What did we decide about the schema last week" is a session search, not a retrieval query.

Which to use

You want Use
A fact about you or the project, always available Memory
Something said in a past conversation Session search
Something in a document or codebase you indexed Retrieval
Something on the web web_search or browser