Relay-primary private managed-agent config

[wesbillman]

Summary

• replace the aggregate private-agent payload with a strict owner-authored encrypted kind:30179 codec
• retain public/private managed-agent heads and deletion tombstones atomically
• hydrate validated relay config into a workspace-scoped in-memory overlay, rehydrated from retained events at every boot so relay-primary config survives app restarts
• support fresh-device recovery: relay-only agents appear in the list with no local record; the first START intentionally materializes a local record (including the key material needed to run the agent) via materialize_relay_only_agent — this is the one deliberate disk write derived from relay state
• resolve the overlay into the local record before every stale-disk republish site (edit, persona rename, pair-start snapshot re-apply), so a device following a newer relay head cannot clobber it with an audit-clean successor event carrying stale fields
• hydrate keyring-stored agent keys in the boot reconcile path, so first boot on default (system-keyring) builds publishes each agent's 30179 instead of silently skipping every untouched agent
• preserve unknown forward-compatible fields and reject malformed/stale inbound events before changing the live overlay
• classify kind:30179 as owner-scoped global user data in the relay and subscribe from Desktop

This is the minimal replacement for the closed #4940 implementation.

Review fixes (c80c4c17, aa39d72a, 6f486e88)

Adversarial review (Eva/Sami) found and fixed five defects in the original head a9b648b4, each pinned by regression tests:

1. Overlay never rehydrated after restart — relay-primary config worked for exactly one launch, then every site silently read stale disk. Fixed: boot rehydration from the retention DB (owner-scoped; negative control asserts another owner's keys hydrate nothing).

2. Boot reconcile published zero 30179s on default keyring builds — records were read without keyring hydration, so the empty-nsec skip fired for every untouched agent. Fixed: hydrate keys in the reconcile path.

3. Stale-disk republish clobbered a newer relay head at three write sites (agent_models.rs edit, personas/update.rs rename, agents.rs pair-start snapshot re-apply). The corruption was a validly-chained gen+1 successor — indistinguishable from a legitimate edit. Fixed: resolve-before-mutate at each site. The resolve is deliberately per-site, not centralized in the retain helper — centralizing it silently discards user edits, and at the rename site a naive resolve-first loses the rename (the gate keys on disk name). Both wrong fixes are pinned by permanent probe tests, and a source-level guard asserts the exact resolve-call count per file (the write sites are #[tauri::command]s, so unit tests cannot observe the production wiring directly).

4. Boot reconcile republished stale disk over a newer relay head — a fourth site in the same class as (3), but firing at launch, unprompted, for every agent on a device that follows another device's config. reconcile_agents_in_dir_at reads managed-agents.json raw and cannot resolve the overlay: hydrate_private_config_overlay runs after this leg (event_sync.rs:19-20) and reads the rows it writes. Inbound 30179 updates the overlay and retention but never the JSON, so on a follower disk is stale by construction. Measured: gen 5 → 6, prev = the clobbered head, created_at floored at head+1 by monotonic_created_at so it wins LWW against a head 10,000s in the future, pending_sync set, every field taken from stale disk. It does not self-heal — a second boot is a clean no-op, but each new head the follower receives re-arms it (measured 16 → 1, no-op, 24 → 1), so the follower's disk wins every round and the user's edit silently reverts. Fixed in aa39d72a: retain_agent_record_at_boot publishes the 30179 only when no retained head exists, and is used by boot reconcile alone; the interactive edit paths keep republishing over heads (they resolve the overlay first), and the kind:30177 identity leg is untouched so the upgrade republish waves keep working. Three mutants — gate deleted, gate inverted, gate applied to the whole record instead of the private leg — each killed by a different arm.

5. Self-authored config never reached the in-memory overlay — the write-side twin of (3). PrivateConfigOverlay has exactly two fill paths and both are inbound-only: insert_patch on an Applied inbound event (commands/personas/inbound.rs) and boot hydration (event_sync.rs::hydrate_private_config_overlay). Neither fires for an event this device authored, because the relay's echo of our own event dedupes to Skipped in retain_inbound_event (strictly-newer guard). So after any local edit the overlay stays pinned at the last received generation, and the next same-session edit resolves that stale patch on top of a fresher disk record and publishes a silent revert — again as a validly-chained successor, indistinguishable from a real edit. Live symptom: an edit sets parallelism 17 → 19, the immediately following rename republishes 17. Note that the resolve-before-mutate fix from (3) is what makes this reachable in the reverting direction: the sites now trust an overlay that no longer tracks this device's own writes. Fixed in 6f486e88: one write-through at one seam — retain_managed_agent_pending (commands/agents.rs), after retain_agent_record commits, reads the just-retained 30179 head back out of the same open conn and inserts it into the overlay (PrivateConfigOverlay::absorb_retained_head). A shared patch_from_retained_row() decode helper is refactored out of hydrate_from_retention, so hydration and write-through share exactly one decode path. The single seam covers all five interactive writers (edit/rename, create, settings, start, rename-rollback) with no per-caller copies; reconcile.rs is deliberately untouched (retain_agent_record takes conn+keys, no AppState), and personas/snapshot/import.rs's inline retain writes only kind:30177, so it has no overlay concern. Two deliberate design calls, both the conservative side of their trade: absorb unconditionally rather than only when the retain reported a change (the overlay can never outrun retention), and never clear on a missing or undecodable head — leave the existing entry alone (pinned with a positive control). The row is read back from retention rather than inserting the in-memory record; Eva independently derived the same design before seeing the diff, so that choice is load-bearing on two derivations. Four mutants, each killed: absorb made a no-op (reproduces the live symptom exactly — left: Some(17), right: Some(19)), the production absorb call deleted, absorb ordered before the retain instead of after, and a fourth ordering variant.

Contract of record for kind:30179 writes — two clauses, not one. Review corrected an earlier draft of (5) that implied every 30179 write funnels through the seam; that is false at a named line. retain_private_agent_record has two non-test callers: retain_agent_record (reconcile.rs:190, the seam's path) and the sibling retain_agent_record_at_boot (reconcile.rs:163), which never passes through retain_agent_record and so never reaches the write-through. Defect (4)'s own fix is what made that sibling a 30179 writer, which is why a caller-count on the funnel symbol could not see it. The class is closed by two independent mechanisms: (a) interactive writers are funnelled through the seam, guarded by the exact-count source guard; (b) the boot writer is gated head-absent (reconcile.rs:157-163), guarded by defect (4)'s three mutants — head-absent means there is no relay state to be stale against, so that write cannot be a revert and an absent overlay entry for it is correct rather than stale. Backstop: run_event_sync runs reconcile then hydrate_private_config_overlay (event_sync.rs:19-20), and hydration is a wholesale *overlay = hydrated replacement, so boot's own writes reach the overlay one line later regardless. Residual, accepted pre-merge: a future writer calling retain_private_agent_record directly, or a third boot-ish sibling, lands outside both mechanisms and outside both guards (the exact-count guard only reads commands/agents.rs) — the same residual shape write_site_resolve_guard already accepts and documents.

Causal note, on the record: defect (4) was made reachable by the fix for (2). Before keyring hydration, retain_private_agent_record's empty-nsec skip returned early for every keyring-resident record, so boot never built a 30179 at all — the skip was incidentally protecting this path. (2) is still correct (an untouched agent must publish its first 30179 on a default build), but it removed a guard that was load-bearing for something else. A control arm with an absent nsec confirms the head survives, pinning the line.

Coverage honesty: (2)'s original regression test asserted the harmful act — it pinned that boot publishes a 30179, which at the time meant publishing stale disk over a head. A green suite ratified the clobber. Chosen fix keeps that test meaningful by scoping it to the head-absent case.

Known follow-ups, out of scope here:

• Auto-start on launch spawns from raw disk. Every interactive path resolves the overlay first (resolved_local_record / resolved_records), but restore.rs contains none of resolved_local_record, resolved_records, resolve_local_record, or private_managed_agent_overlay — it loads raw load_managed_agents (restore.rs:49) and Phase B spawns those records. On a follower, start_on_app_launch agents therefore run on stale private config. Independent of the (4) fix, which stops the republish, not the spawn. (Its two resolve_effective_* references at restore.rs:41/245 are the persona→global fallback resolver, a different mechanism.) Related, unmeasured: spawn_event_sync (workspace.rs:240) and restore_managed_agents_on_launch (workspace.rs:277/291) are spawned without either awaiting the other, so even an overlay-aware restore could read the overlay before hydration populates it.
• resolve_effective_config callers (runtime.rs:200/437, agents_deploy.rs:141, spawn_snapshot.rs:248) each take a record the caller may or may not have overlaid — needs a per-caller ruling.
• the PrivateConfigPatch collapse (−169 LOC) lands as a separate commit after live E2E.

Validation

• full desktop lib suite at 6f486e88: 2365 passed / 0 failed / 15 ignored (--all-features) — the 2360 at aa39d72a plus the 5 new tests for defect (5); 2360 was in turn the 2356 baseline at c80c4c17 (reproduced independently by two reviewers in separate worktrees) plus the 4 tests for defect (4)
• cargo fmt --check clean; cargo clippy --workspace --all-targets -D warnings clean
• all pre-push hooks passed at c80c4c17, aa39d72a, and 6f486e88 (six at 6f486e88): branch skew, desktop-check (file-size ratchet), rust-tests, mobile-test, desktop-test (4387 frontend tests / 0 failed), desktop-tauri-checks. Nothing pushed with --no-verify.
• the desktop file-size ratchet was verified live rather than assumed: padding the new test file past the limit makes it fail ("allowed 1000"), so its pass at aa39d72a is not vacuous
• regression tests include defect probes (assertions inverted post-fix), fix verifications with negative controls, wrong-fix probes, and mutation checks (deletion/ordering/re-copy mutants all fail)
• independent adversarial review by Mongo on the original head: clear, no blockers; Eva/Sami review findings above addressed at c80c4c17
• live-local two-device E2E (restart persistence, first-boot keyring recovery, stale-republish, fresh-device materialization) per TESTING.md — Max, on a device-addressed v2 rig at exact SHA 6f486e88; arm attribution at c80c4c17 was calibration only, since defect (4) made arms 1 and 3 structurally unattributable. This gate is still open and is the remaining merge blocker: the defining 19 → rename discriminator for (5) passes live on the real Tauri backend, but the preserved test agent's scoped keyring secret is unavailable to the fresh process, so the rename profile-sync and pair-start arms refuse at their key boundary and the fresh lane has not yet produced new relay 30179 receipts.
• independent gate on defect (5) at 6f486e88 — Eva, in her own worktree: full diff read, suite + fmt + clippy, and a personal re-run of the absorb-no-op and absorb-before-retain mutants rather than taking the author's kill list on faith

Notes

Temporary Tauri sidecar placeholders used for local builds are ignored and not committed.

[wesbillman]

Reviewing and commenting on Wes's behalf. Blocking verdict: changes requested. The codec, owner-only relay classification, inbound validation-before-retention, scope-clear serialization, and atomic public/private retention all look sound. However, the new overlay start branch creates lifecycle holes that make relay-restored agents unsafe or impossible to manage. Please route resolved overlay records through a lifecycle that preserves the existing transition/preflight/provider/persistence invariants and supports stop/delete for truly relay-only records.

This PR changes behavioral tests and adds codec, inbound, overlay, and reconciliation coverage; none of the added overlay tests exercises a real start → stop/delete lifecycle or concurrent shutdown. Add regressions for a disk-backed overlaid agent and a fresh-device relay-only agent.

Focused checks at this exact clean head: cargo test -p buzz-core private_managed_agent --lib (13 passed), cargo test -p buzz-relay ingest --lib (161 passed; filter selected the ingest module plus one dependent test), and git diff --check passed. Desktop Core and Desktop E2E checks were still running when reviewed; existing CI covers the broad suites.

[wesbillman]

P1 — Do not divert every overlaid disk record around its existing start lifecycle. Presence in this in-memory overlay is not equivalent to being relay-only: resolved_record can start from an existing disk-backed record and then apply the patch. This branch therefore bypasses start_local_agent_with_preflight (including effective-config/relay-mesh validation, persona handling, saving start metadata, retained publication, and profile reconciliation), and it bypasses provider deployment entirely because the helper rejects any resolved provider backend. Merely receiving a valid private event can thus make an ordinary local/provider agent follow a materially weaker or unusable path. Distinguish true relay-only records from disk-backed records and route the latter through the normal resolved local/provider pipeline; add a regression proving an overlaid disk record retains those lifecycle guarantees.

[wesbillman]

P1 — A fresh-device relay-only runtime started here cannot be stopped or deleted through the normal agent API. This spawns from a temporary record and stores only the runtime pair. stop_managed_agent reloads disk records and fails at find_managed_agent_mut; delete_managed_agent likewise requires a disk record and returns agent … not found. The frontend's normal local actions call those commands, so a relay-only agent can start successfully and then become uncontrollable until broader shutdown/process exit. This direct spawn also does not hold managed_agents_store_lock or the documented runtime-transition boundary, so it can run after a workspace/identity overlay clear or race shutdown and register a new child after shutdown's protected snapshot. Implement stop/delete and scope-transition behavior for relay-only records (or materialize an appropriate lifecycle record) and serialize spawn consistently; cover start → stop and start → delete on a device with no local record.

[klopez4212]

Reviewing and requesting changes on Kenny Lopez's behalf.

Blocking: boot reconcile destroys the retained relay head before the overlay is hydrated.

run_event_sync calls reconcile_agents_to_events first and only then calls hydrate_private_config_overlay (desktop/src-tauri/src/event_sync.rs:17-20). The reconcile reads raw managed-agents.json records and passes them directly to retain_agent_record (desktop/src-tauri/src/managed_agents/reconcile.rs:91-119); it cannot consult the overlay because the overlay has not been rebuilt yet.

On the exact restart this commit intends to fix, where disk has config A and retention contains a newer relay config B, boot therefore does this:

1. build a private payload from stale disk A;
2. compare it with retained B, see a difference, and retain A as generation B+1 / pending_sync = true;
3. hydrate the overlay from that newly overwritten A row.

The valid relay head is lost locally and the stale successor is queued to overwrite it remotely. In other words, restart persistence still fails, now through the boot writer rather than the interactive writers.

Hydrate/validate the retained overlay before any disk→private reconcile, and make boot reconcile resolve each disk record against that hydrated head before retaining it (while preserving the intended behavior for records with no retained private head). Add an end-to-end unit seam around the real boot ordering: seed stale disk A plus newer retained B, run the boot reconcile/hydration path, and assert retention and the overlay remain B with no stale private republish queued. The current standalone hydration and stale-republish model tests do not exercise this production ordering.

[klopez4212]

Blocking on a boot-order regression in the restart recovery path. The codec, inbound validation, scope clearing, lifecycle materialization, and stale-write-site fixes otherwise look carefully constructed. I reviewed clean head c80c4c17b047d7580d50ee5261897e0fb192bb2d; git diff --check origin/main...HEAD is clean. I did not duplicate the broad suites already reported/covered by CI.

Dismissed at Kenny Lopez’s request after reviewing aa39d72. The new boot-only head-presence gate prevents stale disk from replacing or queuing over an existing retained 30179 while preserving first private publication and public 30177 reconciliation.

[klopez4212]

Re-reviewing and approving on Kenny Lopez's behalf at aa39d72aac57aeda49cb3c38db7c9c8ed4af24f1.

My boot-order blocker is resolved. The boot-only retention path now leaves any existing kind:30179 head byte-for-byte intact and only creates the private event when no retained head exists. This prevents stale disk state from becoming a newer successor, while preserving first private publication and the independent kind:30177 public reconciliation path. The added regressions cover all three requirements and pin why overlay resolution at boot would discard an unflushed offline edit.

Focused review: the four added boot tests directly exercise the gate and its preservation cases; git diff --check origin/main...HEAD passed at this exact clean head. I attempted the focused Rust tests locally, but this checkout lacks the required generated sidecar desktop/src-tauri/binaries/buzz-acp-aarch64-apple-darwin, so the Tauri build script stopped before compiling the test target. Existing CI is the broad validation authority.

[wesbillman]

Reviewing and requesting changes on Wes Billman's behalf at aa39d72aac57aeda49cb3c38db7c9c8ed4af24f1.

Blocking: launch auto-start bypasses the relay-primary overlay and races its hydration.

apply_workspace starts event sync asynchronously (workspace.rs:233-245), then independently spawns restore_managed_agents_on_launch (workspace.rs:285-295) without waiting for event sync. The restore path reads raw managed-agents.json (restore.rs:116), selects start_on_app_launch candidates from those raw records (restore.rs:168-192), re-collects raw records after persona snapshotting (restore.rs:196-222), and passes them directly to spawn_agent_child (restore.rs:291-341). It never resolves private_managed_agent_overlay.

Therefore, on a follower where disk has private config A and retention/relay has newer config B, an auto-start agent can launch with A: stale prompt/model/provider/runtime, env vars, allowlist, relay mesh, auth tag, or agent nsec. The new boot retention gate correctly prevents A from being republished over B, but does not prevent A from executing. Waiting only for overlay hydration would still be insufficient on a fresh device until backfill has delivered B; the startup contract needs an explicit authoritative-state/readiness decision rather than two racing best-effort tasks.

This contradicts the PR's relay-primary behavior at the most consequential read site. Please serialize auto-start behind the authoritative private-config bootstrap and resolve each candidate through the overlay before preflight/spawn, with a safe offline policy. Add a production-seam regression that seeds stale disk A plus retained newer B, runs launch restore, and proves the spawned snapshot uses B; include a delayed-hydration race arm. A test that only exercises resolve_local_record is not enough.

Behavior-changing tests: the PR adds strong codec/inbound/retention and stale-republish regressions, including restart overlay hydration and the new boot-only 30179 head-preservation gate. However, none covers launch auto-start using the relay-resolved record or the ordering between spawn_event_sync and restore. That missing behavioral test corresponds directly to this blocker.

git diff --check 16cc3de6d6bb23ebdc3a928172fb585494079232..aa39d72aac57aeda49cb3c38db7c9c8ed4af24f1 passes. Current broad CI is not validation evidence for this head: the CI path detector and release-candidate jobs were cancelled after ~48 minutes, and Desktop Core/Unit Tests plus most jobs were skipped.