Skip to content

architecture.md

Latest commit

 

History

History
260 lines (175 loc) · 7.03 KB

architecture.md

File metadata and controls

260 lines (175 loc) · 7.03 KB

Local Memory System Architecture

1. Problem framing

LLMs are stateless by design. Long projects (writing, research, startup building) need memory that is:

  • Local-first: your data stays on disk, not in a SaaS database
  • Structured: facts + summaries you can query / diff / review
  • Human-in-the-loop: important facts and scores can be curated
  • Composable: outputs are plain JSON + Markdown

This system implements a local-first memory layer that converts raw dialog into structured artifacts:

Facts → Topic Memory → Global Memory

2. Core concepts

Fact

A Fact is an atomic, reviewable memory unit with:

  • subject
  • relation
  • object
  • type: event | trait | identity | commitment | preference | constraint
  • time_span: a coarse time window such as short_term | medium_term | current | long_term | future
  • confidence: 0–1 (model-estimated)
  • user_fact_score: 0–10 (human override / curation signal)
  • entry_id and entry_ts: source entry reference

Topic Memory

Aggregated state for one topic:

  • state.summary: compressed topic summary
  • state.bounds: applicability bounds / assumptions
  • state.open_questions: what is still unknown
  • state.next_actions: actionable follow-ups
  • facts: curated facts for the topic

Global Memory

Cross-topic index that surfaces:

  • active_topics[]: active topics with their latest state snapshot
  • do_not_forget[]: a tiered shortlist of facts grouped by:
    • tier: core then secondary
    • type: identity, commitment, constraint, preference, trait, event

3. Data contracts

All memory artifacts include explicit schema versions.

Storage layout (local-first)

Write layer outputs (append-only):

  • data/topics/<topic>/<entry_id>/raw.md
  • data/topics/<topic>/<entry_id>/meta.json
  • data/topics/<topic>/<entry_id>/summary.json
  • data/index/audit.log

Rebuildable outputs:

  • data/topics/<topic>/topic_memory.json
  • data/topics/<topic>/topic_memory.md
  • data/GLOBAL_MEMORY.json
  • data/GLOBAL_MEMORY.md

Rebuild semantics

  • Lower-layer artifacts are append-only.
  • Higher layers are safe to regenerate and can always be rebuilt from stored artifacts.
  • Higher layers never modify lower-layer artifacts.

4. System architecture

flowchart LR

%% ---------- Styles ----------
classDef pipeline fill:#2b2b2b,color:#ffffff,stroke:#666,stroke-width:1px
classDef storage fill:#f5f5f5,stroke:#999
classDef builder fill:#eef3ff,stroke:#6b8cff
classDef artifact fill:#fafafa,stroke:#bbb
classDef external fill:#fff6e6,stroke:#e6a700

%% ---------- Flow ----------
A[Raw Dialog] --> B[WritePipeline]

subgraph Storage
C[raw.md]
D[meta.json]
E[summary.json]
L[audit.log]
end

B --> C
B --> D
B --> E
B --> L

subgraph Topic Layer
F[TopicMemoryBuilder]
G[topic_memory.json]
H[topic_memory.md]
end

C --> F
D --> F
E --> F

F --> G
F --> H

subgraph Global Layer
I[GlobalMemoryBuilder]
J[GLOBAL_MEMORY.json]
K[GLOBAL_MEMORY.md]
end

G --> I
I --> J
I --> K

LLM[(LLM Provider)] -. enrich .-> B

%% ---------- Class assignment ----------
class B pipeline
class C,D,E,L,G,H,J,K artifact
class F,I builder
class L storage
class LLM external
Loading

Layered architecture

  • Write layer: raw persistence + enrich pipeline
  • Topic layer: topic-local aggregation and summarization
  • Global layer: cross-topic indexing and tier selection
  • Audit layer: append-only write events

Key invariant:

Higher layers never modify lower-layer artifacts.

5. Operational model

v1.0 operates under a strict single-user, local-first assumption:

  • Only one writer is assumed at a time
  • File-system persistence is sufficient under this constraint
  • GlobalMemoryBuilder is deterministic and idempotent: it only applies rule-based selection over persisted topic_memory.json artifacts.
  • TopicMemoryBuilder produces two kinds of outputs:
    • Facts table is deterministic/idempotent (pure aggregation over persisted entry summary.json facts).
    • Topic state summary (state.summary/bounds/open_questions/next_actions) is non-deterministic by design because it is LLM-generated. This is acceptable since it is a human-facing compression layer, not the source of truth.

System invariant:

lower-layer artifacts are append-only; higher layers are safe to regenerate and can always be rebuilt from stored artifacts.

6. Failure modes & safeguards

Potential issues:

  • Hallucinated facts
  • Incorrect fact typing
  • Duplicate facts across entries
  • Topic drift over time

Safeguards:

  • Human override via user_fact_score
  • Tiered do-not-forget selection (core/secondary)
  • Append-only audit log
  • Stable JSON contracts (schema-versioned output objects)

7. Scaling characteristics

The system is designed for personal-scale memory growth with predictable rebuild costs. As entry volume increases, the system scales through layered aggregation.

Topic layer

TopicMemoryBuilder aggregates entry summaries within a topic.

  • Rebuild cost grows linearly with the number of entries in that topic.
  • This is acceptable because topics are expected to remain bounded in size.

Global layer

GlobalMemoryBuilder operates only on topic_memory.json artifacts.

  • Global rebuild cost scales with the number of topics, not entries.
  • This avoids global scans over raw dialog history.

Memory surface

Global memory remains bounded through selection rules:

  • active_topics maintains only current topic snapshots.
  • do_not_forget is capped by tier and type limits.

This ensures the size of GLOBAL_MEMORY remains stable even as historical entries accumulate.

Historical entries accumulate indefinitely, while higher memory layers remain bounded and rebuildable.

8. Schema evolution strategy

Schema evolution is handled through immutable lower-layer artifacts and rebuildable higher layers.

Immutable source artifacts

The write layer produces append-only artifacts:

  • raw.md
  • meta.json
  • summary.json

These files are never rewritten and serve as the source of truth for memory reconstruction.

Rebuildable derived layers

Higher-level memory artifacts are derived from persisted entries:

  • topic_memory.json
  • GLOBAL_MEMORY.json

These artifacts can be safely regenerated when schema upgrades occur.

Schema versioning

All structured artifacts include explicit schema identifiers:

{ "schema": "topic_memory.v1" }
{ "schema": "global_memory.v1" }

Readers validate schema versions at load time and apply compatibility logic when necessary.

Upgrade workflow

When schema changes occur:

  1. Existing persisted artifacts remain untouched.
  2. Builders are updated to emit the new schema version.
  3. Derived artifacts are regenerated from stored entry summaries.

This guarantees schema upgrades never corrupt historical memory.

Historical entries remain stable even as summarization prompts and memory schemas evolve.

9. Extension points

Possible extension points:

  • Replace DeepSeek with other LLM providers
  • Swap do-not-forget tiering rules without changing storage contract
  • Add embedding-based recall as an optional layer
  • Expose a REST API wrapper
  • Add multi-user namespace isolation