Learned context cartridges via self-study distillation (parametric extension of #924 memento compaction)

[tcconnally]

Summary

Add a learned context cartridge tier to Perseus's compact-context surface: an offline-trained, loadable memory object that compresses a corpus (project history, decision log, codebase) into a small reusable representation loaded at inference, extending #924's memento-style artifacts with the parametric approach from the Cartridges paper (arXiv:2506.06266, Eyuboglu/Re et al., Stanford, June 2025).

Source motivation (competitive intelligence)

Engram (engram.com) launched from stealth 2026-06-23 with a $98M seed at ~$600M valuation (General Catalyst, Kleiner Perkins, Sequoia; angel: Andrej Karpathy) on exactly this thesis — a "learned memory" layer claiming up to 100× fewer tokens vs. frontier models, with Microsoft, Notion, and Harvey as early customers (CNBC 2026-06-23). The public technical core is the Cartridges method: train a small KV cache offline on each corpus, load it at inference, decode against it. Perseus already owns the corpus problem (the Vault); the missing piece is the distillation step. This is the single most borrowable idea from that research line.

Problem

#924 defines compact derivative text artifacts (task objective, constraints, evidence anchors). Those are still rendered text. Cartridges go further: a trained representation that (a) amortizes corpus processing across all queries touching the corpus, (b) compresses memory footprint at serving time, and (c) can be composed with other cartridges at inference without retraining.

Proposal

Add a learned cartridge pipeline alongside #924's artifact tier:

• Self-study distillation: generate synthetic QA conversations over a bounded corpus (Perseus has ideal source material: past session transcripts, decision records, journal events) and train the cartridge with a context-distillation objective — not naive next-token prediction, which the paper shows is not competitive with in-context learning (ICL).
• Loadable at inference: cartridge binds to a corpus identity + source entity hashes; context assembly can load it instead of raw entity dumps.
• Composable: support loading multiple cartridges (project + org + policy) without retraining.
• Bounded scope: start with a fixture corpus + one live projection surface (e.g. task-scoped projection, docs(brand): canonicalize public MCP card product prose #859).

Why now

• External validation: an 8-month-old company raised ~$100M on this exact method, so the approach is de-risked commercially.
• Perseus's session/journal corpora are the perfect self-study source — synthetic QA can be derived from real usage rather than generated cold.
• DoD/NSF proposal angle: measured inference-cost reduction (see Add a reproducible code-context benchmark that separates retrieval overhead from delivered prompt tokens #922's accounting discipline) is concrete, auditable evidence.

Success criteria

• A cartridge for a fixture corpus trains fully offline and loads deterministically.
• Loaded-cartridge task quality matches the full-context baseline within a declared tolerance on the Add a retrieval policy engine that maps task shapes to structured truth, targeted fetch, broad search, or synthesis #862 harness.
• Measured memory/throughput improvement over raw context (target order-of-magnitude, per paper: 38.6× less memory, 26.4× higher throughput, 128k→484k effective context on MTOB).
• Provenance: every cartridge records the source entity hashes it was distilled from (feeds Add a selective recall budget controller to prefer concise explanation over bulk memory injection #863 / perseus-vault Render conventions, corrections, and scoped rules explicitly #876 (governed distillation)).

Grounding

• Cartridges paper: https://arxiv.org/abs/2506.06266 (self-study recipe; 38.6× less memory, 26.4× higher throughput at ICL-matching quality; composition at inference)
• CNBC announcement: https://www.cnbc.com/2026/06/23/ai-memory-startup-focused-on-cutting-token-costs-raises-98-million.html
• Related: Add memento-style compact context artifacts for token-efficient task projections #924 (memento artifacts), Add a retrieval policy engine that maps task shapes to structured truth, targeted fetch, broad search, or synthesis #862 (evaluation harness), Add a reproducible code-context benchmark that separates retrieval overhead from delivered prompt tokens #922 (token accounting), perseus-vault docs(brand): canonicalize public MCP card product prose #859 (task-scoped projections)