Improve memory storage — needs proper scoping/grooming before implementation #136

What this is

A scoping/grooming ticket, not an implementation ticket. Memory storage has accreted — store, reflex, consolidation, snapshot, TUI surface — and nobody has groomed it as one feature. Before any code changes, this issue needs to be scoped: what memory storage is supposed to be, what is in and out, and which of the known gaps below are real problems vs. acceptable tradeoffs.

Do not start implementing from this issue. The first deliverable is a groomed scope written back here (or in a linked doc), broken into engineering tasks.

Current shape (what exists today)

• internal/store/memory.go (~1.2k lines) — event-sourced SQLite brain: memories table + memories_fts, five event kinds (add/update/supersede/forget/restore), five memory types (fact/preference/decision/correction/project_state), three scopes (user/project/env), tombstones instead of deletes, ranking counters (use_count/miss_count) journaled as periodic snapshots.
• internal/store/memory_snapshot.go — census/snapshot reads.
• internal/reflex/ — the per-turn router: shortlist of 8 candidates via RRF (lexical bm25 + importance + recency), model rejects rather than searches; post-turn extractor + decider (add/refine/replace/skip).
• internal/session/memory.go (~1.1k lines) — the four-moment arrangement (pre-turn block, post-turn extraction, consolidation gate, remember verb), fails-open on every reflex error.
• internal/session/memory_consolidate.go (~680 lines) — idle "dreaming" pass over the 50 most recently touched lines; merges duplicates, retires superseded lines, never deletes, never decays by age.
• internal/tui3/memory.go + memoryplace.go — the /memories, /memory <query>, /remember, /forget surface.

Known gaps / open questions to groom

1. Scope semantics are thin. user/project/env exist as a column and a filter, but nothing governs which scope a write lands in, whether project-scoped memories follow a repo across machines, or how env-scoped truth is validated. Is scope a real blast-radius boundary or just a label?
2. Consolidation coverage. The dreaming pass touches only the 50 most recently touched lines. What happens to lines 51+? No answer on whether that cap is principled or accidental.
3. Ranking counters have no lifecycle. use_count/miss_count grow forever; snapshots preserve them across rebuilds but nothing ages or normalizes them. Is that fine, or does importance need a horizon?
4. No cross-machine story. The store is one SQLite file per machine. Sync, merge, and conflict resolution across machines are unaddressed.
5. Text cap is 512 runes. One line of substance per memory. Is that the right unit, or do we need longer structured entries (e.g. typed claims with evidence, per LatticeMind-style conflict-aware memory)?
6. Retrieval is lexical-only. bm25 + recency + use counters; no embeddings, no semantic recall. Decide whether that's a gap or a deliberate bound.
7. Provenance is partial. source_session/source_seq exist but old rows carry unknown provenance; no story for auditing what a memory was before it was replaced beyond MemoryRecord.

Acceptance for the grooming itself

• A written scope: in/out list, prioritized gaps, and at least one measurable acceptance criterion per in-scope item.
• Gaps ruled out are ruled out with a reason, not silently dropped.
• Follow-up engineering tasks filed (or one umbrella task) with owners.

/assign @santoshkumarradha

Drafted with CodeAF · reviewed and owned by the author