Codex hackathon: durable swarm memory coordinator - #5
Conversation
Greptile SummaryThe PR adds a durable, WAL-backed swarm coordinator, HTTP and Python interfaces, Codex lifecycle integration, recovery tests, and demonstration materials. The current worker identity and allocation boundaries remain unsafe:
Confidence Score: 0/5The PR is not safe to merge until transaction replay, slot ownership, claim authorization, and allocation-read isolation are bound to the authenticated worker and original operation. Applied UUIDs are replayed against newly supplied payloads, claimed-slot mutations verify only that some worker claimed the slot, unclaimed slots adopt caller-supplied identity under a shared worker credential, and the Read endpoint serializes every slot's memory allocation. Files Needing Attention: crates/fluctlightdb/src/swarm.rs, crates/fluctlightdb/src/serve.rs, sdks/python/fluctlightdb/swarm_mcp.py
|
| Filename | Overview |
|---|---|
| crates/fluctlightdb/src/swarm.rs | Adds the swarm state machine and durable lifecycle, but transaction replay and slot operations lack payload and authenticated-worker binding. |
| crates/fluctlightdb/src/serve.rs | Adds role-gated swarm endpoints, but claim authorization is not worker-specific and swarm/get exposes complete allocations to Read callers. |
| sdks/python/fluctlightdb/swarm_mcp.py | Adds Codex lifecycle tools using one shared worker credential while sending worker identity and slot selection as request data. |
| sdks/python/fluctlightdb/swarm_client.py | Adds the Python HTTP client used by swarm tooling; no independent blocking defect is identified in this file. |
| crates/fluctlightdb/src/wal.rs | Extends WAL handling for swarm persistence and replay; no separate blocking defect is identified here. |
Sequence Diagram
sequenceDiagram
participant W as Worker
participant API as Swarm HTTP API
participant S as SwarmState
participant A as Allocations
W->>API: Claim(other_slot, caller identity)
API->>API: Check tenant Write role
API->>S: Apply caller-supplied ClaimSlot
S->>S: Store caller agent_id/worktree
S->>A: Read bundle for other_slot
A-->>W: Other slot's MemoryBundle
W->>API: GET swarm (Read role)
API->>S: Serialize complete SwarmRun
S-->>W: All slot allocations
Reviews (2): Last reviewed commit: "Merge main into codex_hackathon" | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| if self.applied_transactions.contains(&transaction.id()) { | ||
| return self.replay_result(&transaction); |
There was a problem hiding this comment.
When a Write-role caller reuses an applied transaction UUID with a different Claim, Report, or Evidence payload, replay_result reads the newly requested swarm and slot without validating the original operation or worker identity, exposing another worker's memory bundle or attempt data.
| fn ensure_claimed(run: &SwarmRun, slot_id: &str) -> Result<(), SwarmError> { | ||
| let slot = run | ||
| .roster | ||
| .iter() | ||
| .find(|slot| slot.slot_id == slot_id) | ||
| .ok_or_else(|| SwarmError::SlotNotFound(slot_id.to_string()))?; | ||
| if slot.agent_id.is_none() || slot.worktree.is_none() { | ||
| return Err(SwarmError::SlotNotClaimed(slot_id.to_string())); | ||
| } | ||
| Ok(()) | ||
| } |
There was a problem hiding this comment.
When a tenant Write-role client cites or reports against another claimed worker's slot_id, ensure_claimed checks only that someone claimed the slot, allowing the caller to overwrite that worker's citations, result tree, summary, and status and corrupt subsequent evidence attribution.
# Conflicts: # crates/fluctlightdb/src/lib.rs # crates/fluctlightdb/src/manifest.rs # crates/fluctlightdb/src/wal.rs
| (None, None) => { | ||
| slot.agent_id = Some(request.agent_id); | ||
| slot.worktree = Some(request.worktree); |
There was a problem hiding this comment.
When a worker using the shared Write token claims another worker's unclaimed slot, this branch stores the caller-supplied agent_id and worktree as its owner and returns that slot's complete memory bundle, preventing the intended worker from claiming it with its own identity.
How this was verified: The claim endpoint has no authenticated worker-to-slot binding, while this branch records all worker identity fields directly from the request.
| server.with_brain_read(tenant_id, |brain| { | ||
| let run = brain | ||
| .swarm | ||
| .runs | ||
| .get(&swarm_id) | ||
| .ok_or(crate::swarm::SwarmError::SwarmNotFound(swarm_id))?; | ||
| Ok(serde_json::to_value(run).unwrap()) |
There was a problem hiding this comment.
When a tenant Read-role caller requests an active swarm, this endpoint serializes the complete SwarmRun, including every worker's allocated verified truth, mandatory warnings, and episodic memory contents, without requiring the caller to claim those slots.
How this was verified: The endpoint returns the full run under Read authorization, and SwarmRun stores the complete allocations map.
…in the merge Two regressions introduced by the conflict resolution in beefcae (merge of the defect-fix branch into the CLS organ branch), not present on either parent. 1. main does not compile. `lib.rs` exported `replicate::{open_replica_brain, sync_once, ReplicaStatus}` on BOTH sides of the merge; the resolution kept only `open_replica_brain`. `fluctlight-py` calls `fluctlightdb::sync_once`, so `fluctlightdb-native` fails with E0425 and the whole Python surface goes with it — this is why the `test`, `python-sdk` (x3), `python-sdk-native` and `pypi-wheel-smoke` (x7) jobs are red on PR #5, which was merged anyway. 2. `agent` and `governance` are silently dropped again. `load_v4_dir` still reads both segments, but the resolution dropped the matching `write_segment` calls from `write_checkpoint_dir` and the two entries from `BrainManifest::default().segments`. The reads therefore only ever reach `unwrap_or_default()`, which makes the loss look repaired while unflushed working memory and the entire compliance audit log reset on every restart. Verified before this commit: governance audit entries 1 -> 0, WM slots 1 -> 0 across a save/load cycle. The guard test written to prevent exactly this — asserting every segment the manifest declares is actually written — was dropped in the same resolution. It is restored here, adapted to the new generations/CURRENT publishing layout, along with the two round-trip tests. `swarm` is correctly declared and written and is covered by the restored guard. Not fixed here, reported instead: `roadmap::serve_reloads_after_external_snapshot_write` fails on main independently of these changes (verified by reverting the manifest fix alone). `store::load` now attaches an exclusive flock for the lifetime of the brain handle, so a second opener blocks the full 120s timeout while a BrainServer holds the path. That is plausibly intentional for CAB single-writer safety, but it breaks the capability this test guards and the test was not updated — deciding between narrowing the lock and retiring the test is a maintainer call. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TMKmCh4BkZPk9qas181uU
…in the merge Two regressions introduced by the conflict resolution in beefcae (merge of the defect-fix branch into the CLS organ branch), not present on either parent. 1. main does not compile. `lib.rs` exported `replicate::{open_replica_brain, sync_once, ReplicaStatus}` on BOTH sides of the merge; the resolution kept only `open_replica_brain`. `fluctlight-py` calls `fluctlightdb::sync_once`, so `fluctlightdb-native` fails with E0425 and the whole Python surface goes with it — this is why the `test`, `python-sdk` (x3), `python-sdk-native` and `pypi-wheel-smoke` (x7) jobs are red on PR #5, which was merged anyway. 2. `agent` and `governance` are silently dropped again. `load_v4_dir` still reads both segments, but the resolution dropped the matching `write_segment` calls from `write_checkpoint_dir` and the two entries from `BrainManifest::default().segments`. The reads therefore only ever reach `unwrap_or_default()`, which makes the loss look repaired while unflushed working memory and the entire compliance audit log reset on every restart. Verified before this commit: governance audit entries 1 -> 0, WM slots 1 -> 0 across a save/load cycle. The guard test written to prevent exactly this — asserting every segment the manifest declares is actually written — was dropped in the same resolution. It is restored here, adapted to the new generations/CURRENT publishing layout, along with the two round-trip tests. `swarm` is correctly declared and written and is covered by the restored guard. Not fixed here, reported instead: `roadmap::serve_reloads_after_external_snapshot_write` fails on main independently of these changes (verified by reverting the manifest fix alone). `store::load` now attaches an exclusive flock for the lifetime of the brain handle, so a second opener blocks the full 120s timeout while a BrainServer holds the path. That is plausibly intentional for CAB single-writer safety, but it breaks the capability this test guards and the test was not updated — deciding between narrowing the lock and retiring the test is a maintainer call. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TMKmCh4BkZPk9qas181uU
1 participant
Adds a durable FluctlightDB-backed coordinator and Codex plugin for parallel-agent memory. Workers receive shared verified truth, mandatory failure warnings, and disjoint episodic strategies. Attempts are worktree-bound, evidence-gated, targeted, WAL-backed, and checkpointed. Includes a one-command end-to-end demo and hackathon landing page.
Verified: