Implement Codex-style memory system

openspec/changes/archive/2026-06-01-implement-codex-style-memory-system/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-06-01

openspec/changes/archive/2026-06-01-implement-codex-style-memory-system/design.md

@@ -0,0 +1,88 @@
+## Context
+
+Parent change `abstract-memory-entry-lifecycle` added a pure
+`memory_entry` module for generation and use decisions. Runtime code still has
+no `[memory]` config, no generated memory file layout, no prompt loading, and
+no end-of-turn generation pass. Existing session trajectories are saved under
+`RA_HOME/sessions/<cwd-hash>/`; generated memories should live under the same
+Ra home/state root as inspectable generated state, but they must not become the
+primary place for required project guidance.
+
+## Goals / Non-Goals
+
+**Goals:**
+
+- Wire a disabled-by-default memory feature through config and generated JSON
+  schema.
+- Persist, list, and load durable local memories from the Ra home/state root.
+- Add per-session controls for using existing memories and contributing future
+  memories without mutating global config.
+- Inject eligible durable memories into system prompt context while respecting
+  external-context suppression.
+- Run a deterministic background-style generation pass after saved sessions,
+  gated by the existing `memory_entry` lifecycle decisions.
+- Redact likely secrets from generated memory fields.
+
+**Non-Goals:**
+
+- Implement high-quality model-backed extraction in this change.
+- Make memory files the authoritative control surface for team rules.
+- Add UI for browsing or editing memories.
+- Add cross-machine sync or remote memory services.
+
+## Decisions
+
+1. Add a new `memory` runtime module rather than expanding
+   `memory_entry.rs`.
+
+   The lifecycle module remains pure policy. The new module owns storage,
+   redaction, prompt rendering, and generation orchestration, and calls
+   `decide_generation` / `decide_use` for the policy decision. Alternative:
+   fold runtime behavior into `memory_entry.rs`; rejected because it would
+   blur the parent abstraction and make lifecycle unit tests depend on IO.
+
+2. Store generated files as JSON under `<RA_HOME>/memories/<cwd-hash>/`.
+
+   Reusing the cwd bucket shape keeps local project memories separated like
+   trajectories while avoiding path leakage. Each file is inspectable and
+   atomic-write persisted. Alternative: append to a single TOML/Markdown file;
+   rejected because generated state should not look like the user's primary
+   hand-edited config surface.
+
+3. Use a narrow `MemoryExtractor` trait with a deterministic local extractor.
+
+   The trait leaves room for model-backed extraction later. The default
+   extractor creates conservative facts from stable-looking user turns and is
+   sufficient to test the generation pipeline. Alternative: call the LLM during
+   save; rejected for this scoped change because model extraction quality is
+   explicitly allowed to be stubbed.
+
+4. Attach memory runtime options to `Session` and generation to
+   `RunnerHost::save_session`.
+
+   Existing ACP/A2A/TUI paths already save through `SessionRunner`, so adding a
+   hook near save keeps generation tied to completed turns. CLI print/resume
+   paths save inline and will call the same helper after persistence.
+   Alternative: spawn an always-on scheduler; rejected because active-session
+   tracking and shutdown semantics would be larger than necessary.
+
+5. Treat external context suppression as a caller-provided boolean with config
+   aliases.
+
+   This gives ACP/A2A/TUI/CLI a simple control point and maps existing
+   suppression aliases into `MemoryPolicy::disable_on_external_context`.
+   Current callers default to no external context unless they explicitly know
+   otherwise.
+
+## Risks / Trade-offs
+
+- Deterministic extraction may generate fewer memories than a model-backed
+  extractor -> keep the interface narrow and the pipeline testable so model
+  extraction can replace it later.
+- End-of-turn generation is synchronous enough to run during session save ->
+  keep it lightweight, skip when rate-limit headroom is too low, and avoid LLM
+  calls in this change.
+- Secret redaction is heuristic -> redact common key/token/password shapes and
+  record `redaction_applied`; do not claim perfect data-loss prevention.
+- Memory context may compete with other system prompt resources -> render a
+  small bounded section and allow thread-level/global suppression.

openspec/changes/archive/2026-06-01-implement-codex-style-memory-system/proposal.md

@@ -0,0 +1,41 @@
+## Why
+
+Ra already models Codex-style memoryEntry lifecycle decisions, but it does not
+yet persist memories, load them into prompts, or run the session-level gates
+that make the policy useful. This change turns the existing abstraction into a
+small runtime memory system while keeping generated memories local and
+non-authoritative.
+
+## What Changes
+
+- Add config wiring for a globally disabled-by-default `[memory]` section and
+  per-thread/session use and generation controls.
+- Persist generated memory artifacts under the Ra home/state directory and load
+  durable memories into the system prompt when eligible.
+- Add a generation pipeline that evaluates idle, active-session, short-lived
+  session, rate-limit, external-context, and redaction gates through the
+  existing `memory_entry` lifecycle abstraction.
+- Keep model extraction quality behind a narrow interface with a deterministic
+  local extractor suitable for tests and future model-backed extraction.
+- Document that required team guidance belongs in `AGENTS.md` or checked-in
+  docs, not only generated memory files.
+
+## Capabilities
+
+### New Capabilities
+
+- `codex-style-memory-system`: runtime storage, loading, thread controls, and
+  generation behavior for Codex-style local memories.
+
+### Modified Capabilities
+
+- None.
+
+## Impact
+
+- Affected code: config parsing/schema, a new memory runtime module, system
+  prompt composition, session runner end-of-turn integration, CLI/TUI/ACP/A2A
+  session construction, and tests.
+- Affected state: generated memory JSON files under the Ra home/state root.
+- Affected docs: PRD/issue notes and generated-state guidance.
+- No new external dependency is expected.

openspec/changes/archive/2026-06-01-implement-codex-style-memory-system/specs/codex-style-memory-system/spec.md

@@ -0,0 +1,106 @@
+## ADDED Requirements
+
+### Requirement: Memory Configuration
+
+Ra SHALL expose a disabled-by-default memory configuration that can enable local
+memory use and generation without requiring code changes.
+
+#### Scenario: Memories disabled by default
+
+- **WHEN** Ra loads a minimal config
+- **THEN** the memory policy disables generated memories globally
+
+#### Scenario: Config enables memories and thresholds
+
+- **WHEN** Ra loads `[memory]` settings for use, generation, idle delay,
+  minimum session duration, rate-limit threshold, storage path, and external
+  context suppression
+- **THEN** Ra maps those settings into the runtime memory policy
+
+#### Scenario: External context suppression alias
+
+- **WHEN** Ra loads the alias for disabling memory when external context is
+  present
+- **THEN** Ra treats it the same as the canonical external-context suppression
+  setting
+
+### Requirement: Local Memory Storage
+
+Ra SHALL persist generated durable memories as inspectable generated JSON state
+under the Ra home/state directory or a configured memory directory.
+
+#### Scenario: Default storage path
+
+- **WHEN** memories are stored for a working directory and no explicit memory
+  directory is configured
+- **THEN** Ra writes them under `<RA_HOME>/memories/<cwd-hash>/`
+
+#### Scenario: Persist and load durable memories
+
+- **WHEN** Ra persists a generated durable memory artifact
+- **THEN** a later runtime can load the same artifact and preserve its metadata,
+  redaction flag, and content fields
+
+#### Scenario: Generated state guidance
+
+- **WHEN** Ra exposes or renders memory guidance
+- **THEN** it states that memory files are generated local state and that
+  required team guidance belongs in `AGENTS.md` or checked-in documentation
+
+### Requirement: Thread Memory Controls
+
+Ra SHALL allow each session/thread to control whether it can use existing
+memories and whether it can contribute future memories without changing global
+settings.
+
+#### Scenario: Thread disables memory use
+
+- **WHEN** global memories are enabled and a session disables memory use
+- **THEN** Ra suppresses prompt memory context for that session
+
+#### Scenario: Thread disables memory generation
+
+- **WHEN** global memories are enabled and a session disables memory generation
+- **THEN** Ra skips future memory generation for that session but can still use
+  existing memories if use is enabled
+
+### Requirement: Prompt Memory Context
+
+Ra SHALL load eligible durable memories into prompt context when memory use is
+enabled and suppression settings allow it.
+
+#### Scenario: Durable memories render into system prompt
+
+- **WHEN** memory use is globally enabled, thread use is enabled, and durable
+  memories exist for the current working directory
+- **THEN** Ra appends a bounded local memory context section to the system prompt
+
+#### Scenario: External context suppresses prompt memories
+
+- **WHEN** external context is present and external-context suppression is
+  enabled
+- **THEN** Ra omits memory context from the system prompt
+
+### Requirement: Memory Generation Pipeline
+
+Ra SHALL evaluate completed prior sessions for generated memories using the
+existing memoryEntry lifecycle policy before writing artifacts.
+
+#### Scenario: Eligible completed session generates memory
+
+- **WHEN** a completed session is long enough, idle long enough, inactive, above
+  the rate-limit threshold, and allowed by global and thread policy
+- **THEN** Ra writes a generated durable memory artifact
+
+#### Scenario: Active, short-lived, idle-pending, and rate-limited sessions skip
+
+- **WHEN** a session is active, too short-lived, still within the idle delay, or
+  below the configured rate-limit percentage
+- **THEN** Ra does not write a memory artifact and reports the lifecycle
+  decision reason
+
+#### Scenario: Secrets are redacted before storage
+
+- **WHEN** generated memory fields contain likely secrets
+- **THEN** Ra redacts those fields before writing and records that redaction was
+  applied

openspec/changes/archive/2026-06-01-implement-codex-style-memory-system/tasks.md

@@ -0,0 +1,18 @@
+## 1. Config And Storage
+
+- [x] 1.1 Add `[memory]` config fields, defaults, aliases, and schema/example coverage.
+- [x] 1.2 Implement local memory artifact types, cwd-bucketed storage paths, atomic persistence, and loading.
+- [x] 1.3 Implement redaction and generated-state guidance helpers.
+
+## 2. Runtime Integration
+
+- [x] 2.1 Map config and thread controls into `MemoryPolicy` using the existing lifecycle abstraction.
+- [x] 2.2 Load eligible durable memories into system prompt context with global, thread, and external-context suppression.
+- [x] 2.3 Add the generation pipeline with idle, active, short-lived, rate-limit, and redaction gates.
+- [x] 2.4 Wire memory runtime into CLI, resume, TUI, ACP, and A2A session save/load paths.
+
+## 3. Tests And Validation
+
+- [x] 3.1 Add focused tests for config gating, aliases, storage paths, persistence, and prompt loading.
+- [x] 3.2 Add focused tests for thread use/generation controls, idle/rate-limit skipping, active/short-lived skipping, and redaction.
+- [x] 3.3 Run formatting, targeted tests, OpenSpec validation, and full available validation.

openspec/specs/codex-style-memory-system/spec.md

@@ -0,0 +1,110 @@
+# codex-style-memory-system Specification
+
+## Purpose
+TBD - created by archiving change implement-codex-style-memory-system. Update Purpose after archive.
+## Requirements
+### Requirement: Memory Configuration
+
+Ra SHALL expose a disabled-by-default memory configuration that can enable local
+memory use and generation without requiring code changes.
+
+#### Scenario: Memories disabled by default
+
+- **WHEN** Ra loads a minimal config
+- **THEN** the memory policy disables generated memories globally
+
+#### Scenario: Config enables memories and thresholds
+
+- **WHEN** Ra loads `[memory]` settings for use, generation, idle delay,
+  minimum session duration, rate-limit threshold, storage path, and external
+  context suppression
+- **THEN** Ra maps those settings into the runtime memory policy
+
+#### Scenario: External context suppression alias
+
+- **WHEN** Ra loads the alias for disabling memory when external context is
+  present
+- **THEN** Ra treats it the same as the canonical external-context suppression
+  setting
+
+### Requirement: Local Memory Storage
+
+Ra SHALL persist generated durable memories as inspectable generated JSON state
+under the Ra home/state directory or a configured memory directory.
+
+#### Scenario: Default storage path
+
+- **WHEN** memories are stored for a working directory and no explicit memory
+  directory is configured
+- **THEN** Ra writes them under `<RA_HOME>/memories/<cwd-hash>/`
+
+#### Scenario: Persist and load durable memories
+
+- **WHEN** Ra persists a generated durable memory artifact
+- **THEN** a later runtime can load the same artifact and preserve its metadata,
+  redaction flag, and content fields
+
+#### Scenario: Generated state guidance
+
+- **WHEN** Ra exposes or renders memory guidance
+- **THEN** it states that memory files are generated local state and that
+  required team guidance belongs in `AGENTS.md` or checked-in documentation
+
+### Requirement: Thread Memory Controls
+
+Ra SHALL allow each session/thread to control whether it can use existing
+memories and whether it can contribute future memories without changing global
+settings.
+
+#### Scenario: Thread disables memory use
+
+- **WHEN** global memories are enabled and a session disables memory use
+- **THEN** Ra suppresses prompt memory context for that session
+
+#### Scenario: Thread disables memory generation
+
+- **WHEN** global memories are enabled and a session disables memory generation
+- **THEN** Ra skips future memory generation for that session but can still use
+  existing memories if use is enabled
+
+### Requirement: Prompt Memory Context
+
+Ra SHALL load eligible durable memories into prompt context when memory use is
+enabled and suppression settings allow it.
+
+#### Scenario: Durable memories render into system prompt
+
+- **WHEN** memory use is globally enabled, thread use is enabled, and durable
+  memories exist for the current working directory
+- **THEN** Ra appends a bounded local memory context section to the system prompt
+
+#### Scenario: External context suppresses prompt memories
+
+- **WHEN** external context is present and external-context suppression is
+  enabled
+- **THEN** Ra omits memory context from the system prompt
+
+### Requirement: Memory Generation Pipeline
+
+Ra SHALL evaluate completed prior sessions for generated memories using the
+existing memoryEntry lifecycle policy before writing artifacts.
+
+#### Scenario: Eligible completed session generates memory
+
+- **WHEN** a completed session is long enough, idle long enough, inactive, above
+  the rate-limit threshold, and allowed by global and thread policy
+- **THEN** Ra writes a generated durable memory artifact
+
+#### Scenario: Active, short-lived, idle-pending, and rate-limited sessions skip
+
+- **WHEN** a session is active, too short-lived, still within the idle delay, or
+  below the configured rate-limit percentage
+- **THEN** Ra does not write a memory artifact and reports the lifecycle
+  decision reason
+
+#### Scenario: Secrets are redacted before storage
+
+- **WHEN** generated memory fields contain likely secrets
+- **THEN** Ra redacts those fields before writing and records that redaction was
+  applied
+