#147: bind memory namespace into the signed cap service (approach B)#150
Merged
Conversation
6 tasks
…h B) memory.put/get now mint the cap with service="memory:<namespace>" instead of a static "memory". Because the broker signs `service` and the worker already derives the S3 key, AAD, and on-chain scope check from cap.payload.service, this makes the namespace: - tamper-proof (signed into the cap), - authorized via the existing isServiceInScope gate, - storage-segregated (bots/<actor>/memory/memory:<ns>.enc), - AAD-bound, with NO CapPayload change, NO broker change, and no byte-exact broker<->worker signature risk. Also fixes a latent bug where every namespace collided at the single memory.enc key. No worker behavior change (it already keys/scopes/AADs off the signed service); added a test proving namespace-folded services segregate storage. Verified: cargo test -p agentkeys-mcp-server (35) + -p agentkeys-worker-memory green.
hanwencheng
force-pushed
the
claude/memory-signed-namespace
branch
from
733b929 to
8b57201
Compare
Answers 'Hermes lists many memory providers — which to pick, how to stay
compatible' and folds the strategy into the source of truth.
plan/agentkeys-memory-design.md:
- New §6a 'Engine integration — Hermes providers + the adapter seam':
- Reframe: the ~9 Hermes providers bundle engine+store+delivery; they
slot into AgentKeys' pluggable ENGINE axis, not as peers.
- Delivery stays at the pre_llm_call hook (#141), NOT the runtime
memory.provider interface (its lifecycle step 6 hands the LLM
memory-enumeration tools — breaks the no-whole-context invariant).
- Canonical engine = OpenViking (self-hosted, deterministic, zero
third-party egress); Holographic second; cloud providers = tier 3.
- Adapter seam = one MemoryEngine trait (extract/rank/synthesize);
compatibility = one conformance test with the engine swapped and
store+gate+delivery held constant.
- Two compatibility tiers, one gate: local=own-store+gate-read,
cloud=gate-egress+audit.
- New engine stage E0 (the recommended start): MemoryEngine trait +
OpenViking reference adapter + swap-the-engine conformance test.
arch.md:
- §22 pluggable surfaces: add 'Memory engine' axis row (fix stale
'six'->'eight' axes count).
- §15.2 memory-service: document namespace = signed service
'memory:<namespace>' (#147) and the pluggable-engine posture; add the
previously-missing outward links to the plan + research docs.
…es (#147) Starts implementing plan §6a / arch.md §22 'Memory engine' axis: the caller-side, deterministic engine seam that ranks/selects gate-authorized memory lines before injection (never in the worker, never an LLM in the gate). crates/agentkeys-core/src/memory_engine.rs (new): - MemoryEngine trait: select(query, lines, budget) -> lines. - PassthroughEngine: identity when unbounded (today's full-blob inject); recency-trim when a budget is set. - LexicalEngine: deterministic term-overlap ranking with a query (stopword filtered), recency fallback without one. Real reference engine, no LLM, no external service. - select_blob(): the blob->blob seam contract — swapping the engine never changes the signature, only the selected subset (plan §6a.5 conformance). - SelectionBudget + env config (AGENTKEYS_MEMORY_ENGINE / _MAX_LINES / _MAX_BYTES). 8 unit + conformance tests. crates/agentkeys-cli/src/hook.rs: - memory-inject (pre_llm_call) now runs the configured engine over each namespace blob. Default passthrough + unbounded budget = byte-identical to prior behavior (the Chengdu single-line fixture is unchanged). OpenViking/Holographic adapters implement the SAME trait and are the next adapters — deferred pending the API spike (plan §6a.3); not fabricated here. Tests: cargo test -p agentkeys-core -p agentkeys-cli green (core 38 incl. 8 new engine tests; cli unchanged-green).
…orker (#147) Task 2 — make phase1-wire-demo prove memory against the REAL provider, now through the pluggable engine (plan §6a). The engine runs caller-side in the wired pre_llm_call hook, so it must be baked at wire time (Hermes invokes the hook in the Phase-4 chat, where the demo's env can't reach). agentkeys wire (main.rs + wire.rs): - New --memory-engine (passthrough|lexical, default passthrough) and --memory-max-lines flags. They bake AGENTKEYS_MEMORY_ENGINE / AGENTKEYS_MEMORY_MAX_LINES into the generated memory-inject script. - Default passthrough + no budget emits NO engine env → the generated script is byte-identical to before (idempotency + existing tests hold). - 2 new tests: omitted-by-default, and baked-when-set (env precedes exec). harness/phase1-wire-demo.sh: - MEMORY_ENGINE / MEMORY_MAX_LINES config (default passthrough = unchanged). - Passes them to , so in --real the wired hook runs the engine over the REAL memory worker's lines. - 3.1 now surfaces engine + source ('engine=lexical via REAL worker → …'). - bash -n clean. docs/operator-runbook-wire.md: document the MEMORY_ENGINE / MEMORY_MAX_LINES knobs in the env-override list (runbook-fix-fold-back). Note: the full --real run needs the operator's live stack (broker + Heima + WebAuthn + sandbox), not runnable from this worktree; verified via cargo test (core+cli green) + bash -n. To SEE selection, run --real with a multi-line SEED_MEMORY_CONTENT and MEMORY_ENGINE=lexical MEMORY_MAX_LINES=N.
harness/storage-test.sh — proves the memory STORAGE solution from a fresh
checkout with NO external infra (no AWS, no chain, no broker, no network):
step 0 env+cache resolves CARGO_TARGET_DIR/CARGO_HOME (build cache → fast
re-runs); unsets AGENTKEYS_BROKER_URL/_DATA_ROLE_ARN so
tests can never reach a live broker (the env leak that
makes cli provision tests hit prod).
step 1 prereqs cargo / jq / curl, fail-loud with install hints.
step 2 build cargo build CLI + MCP server; reports cache-hit vs compiled.
step 3 suites runs the REAL storage code paths — envelope AES-256-GCM
(encrypt-at-rest), per-actor S3 key derivation, namespace
isolation (#147), pluggable engine. cli uses --lib to skip
the env-dependent provision integration tests.
step 4 roundtrip starts an in-process in-memory MCP server and drives
put → get → inject end-to-end, plus an engine-selection
check (lexical + max_lines budget → 1 of N lines).
Idempotent: cargo no-op when unchanged; the MCP server is killed + restarted
fresh each run (ephemeral state). Verified: two consecutive runs both ALL GREEN,
2nd reports build cache-hit.
NOT a real-S3 proof (in-memory backend = plumbing only). For the authoritative
live-worker proof, harness/phase1-wire-demo.sh --real remains the path; the
header documents this.
…#147) CI (harness-ci.yml + mcp-server.yml) failed on the new memory_engine.rs: - cargo fmt --all -- --check: long test-call lines + env_usize chain needed rustfmt wrapping. - cargo clippy --workspace -- -D warnings: clippy::unnecessary_sort_by on the PassthroughEngine recency sort → sort_by(|a,b| b.seq.cmp(&a.seq)) → sort_by_key(|l| std::cmp::Reverse(l.seq)). Verified locally against the exact CI commands: - cargo fmt --all -- --check (exit 0) - cargo clippy --workspace --all-targets -- -D warnings (exit 0) - cargo test --workspace -- --test-threads=1 (735 passed) - cargo test -p agentkeys-mcp-server --all-features (35 passed)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First real increment of #147 (the memory system), using approach B (chosen with the maintainer): fold the memory namespace into the signed
servicefield of the cap-token (service = "memory:<namespace>") instead of adding a new signed cap claim.Why approach B
The cap-token already signs
service, and both the broker (isServiceInScope) and the memory worker (key derivation + AAD + scope re-check) already operate entirely offcap.payload.service. So making the memory service carry the namespace gives, for free:memory:travelcap can't be edited intomemory:personal.isServiceInScope(operator, actor, keccak("memory:travel"))gate now authorizes the namespace. No new mechanism.bots/<actor>/memory/memory:travel.enc, physically distinct frommemory:personal.No
CapPayloadchange, no broker change, no byte-exact broker↔worker signature risk, no breaking cap-format. It also fixes a latent bug: today every namespace collides at the singlememory.enckey (the worker ignored the bodynamespace).Changes
crates/agentkeys-mcp-server/src/tools/memory.rs—memory.put/memory.getnow mint the cap withservice = "memory:<namespace>"(was a static"memory"+ a redundant body field the worker ignored).crates/agentkeys-worker-memory/src/handlers.rs— added a test proving namespace-folded services segregate storage (memory:travel≠memory:personalkey). No behavior change in the worker — it already keys/scopes/AADs off the signed service.Verification
cargo test -p agentkeys-mcp-server— green (35 tests).cargo test -p agentkeys-worker-memory— green (incl. the new segregation test).Remaining for #147 (follow-up commits on this PR)
harness/phase1-wire-demo.sh+ theheima-scope-setstep grantmemory:travel(and seed in thetravelnamespace) instead ofmemory, so the--realflow authorizes the namespaced service. (--lightis unaffected — the in-memory backend keys off the body namespace.)memory:<namespace>(canonical-names + bucket-layout note).memory:prefix in the worker S3 key (bots/<actor>/memory/travel.enc) and a cross-namespace-denial integration test in the stage-3/wire harness.Notes
/create-prpolicy: committed from the worktree (git), pushed via jj; noCo-Authored-By.