feat: add sqlite run coordinator shadow - #5279
Conversation
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsFINDING -- src/kiro_crew/sandbox.py:1114 -- False positive or not applicable? A repository writer can comment: |
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS Sound staged migration, but the fail-closed sandbox costs land on users now while the ledger they protect ships unwired. Watch
Suggestions
[DESIGN-REVIEWED] 790e8b1 |
5a37ff3 to
3cee769
Compare
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of All evidence gathered. Here is the review. First-Principles-Verdict: CONCERNS Two unrelated frontend fixes and a half-done reformat ride along undeclared in a security-heavy coordinator PR whose declared items are all RFC-derived. What this change shipsIntent: give the run-coordinator contract a durable, injectable SQLite implementation plus parity evidence, without touching legacy execution authority. ADDITION (stack PR 3/7 of the recorded
Watch
Subtractions
[FIRST-PRINCIPLES-REVIEWED] 790e8b1 |
Opus 4.8 Review — ✅ no blocking findingsReviewed Review detailsNo findings. FINDING — src/kiro_crew/subagent.py:2409 — parity check [OPUS-REVIEWED] 790e8b1 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
3cee769 to
8ac7451
Compare
8ac7451 to
c1bf100
Compare
c1bf100 to
777f0a4
Compare
777f0a4 to
c1bf100
Compare
c1bf100 to
fed76a5
Compare
|
|
Evidence: coordinator storage now resolves through the trusted anchor in |
|
|
bolichen97
left a comment
There was a problem hiding this comment.
Review: SQLite run coordinator + shadow parity
Reviewed origin/feat/run-coordinator-boundaries...origin/feat/run-coordinator-shadow (22 files, +2085/−143) at b915711c1. Findings verified by executing the real SQLiteRunCoordinator, ShadowRunCoordinator and _shadow_submit_accepted_run — including cross-process runs, injected mid-transaction failures, and measured latency curves.
Two framing problems worth stating before the list. First, the shadow's job is to collect parity data without risking the primary, and it can destroy the primary's committed decision in three separate ways. Second, the whole apparatus has no production caller: grep -rn 'SQLiteRunCoordinator|ShadowRunCoordinator' src/ outside run_coordinator/ returns nothing, slack/gateway.py:7875 builds SubagentManager with 16 kwargs and no coordinator=, and the RFC's mandated three-value rollout mode (legacy/shadow/coordinator, §5.9, with "rollback… is supported") exists nowhere — grep -rni 'coordinator_mode|rollout_mode' src/kiro_crew/config/ config-baseline.json is empty. So the feature is simultaneously un-gated and unreachable: it collects zero parity data, while any future caller or merge that passes coordinator= turns on durable SQLite writes on the spawn hot path with no config gate and no kill switch.
Blocking
1. run_coordinator/shadow.py:117 — except Exception does not contain BaseException, so a cancelled shadow leg discards the primary's already-committed decision — and at the _run_impl seam that lands in except asyncio.CancelledError and respawns an accepted run whose _run_inner never started. shutdown_maintenance_executor() (atexit-registered ×10, exported, called by tests) runs _coordinator_pool.shutdown(wait=False, cancel_futures=True), so a queued _offload future makes wrap_future raise CancelledError in a task that was never cancelled. It passes through _mirror's except Exception and subagent.py:2293's, and reaches subagent_manager/run.py:330. There info.tool_count == 0 and _cancel_retry_used is False (the shadow await precedes _run_inner), so the side-effect gate passes and _schedule_cancel_recovery(info) fires. Executed: _run_inner ever awaited: 0 / _schedule_cancel_recovery called (duplicate respawn): 1, and independently caller task was NEVER cancelled, yet await raised CancelledError / primary already APPLIED the submit: True. KeyboardInterrupt, SystemExit and GeneratorExit escape identically. This directly contradicts the spec's "an unhealthy shadow cannot fail an accepted legacy run"; the PR's only containment test injects RuntimeError.
2. shadow.py:120 — fields = _mismatch_fields(primary, shadow) sits OUTSIDE the try that guards the shadow call, so any failure while comparing a non-conforming shadow value propagates and destroys the primary's committed result. Five executed crash classes, every one with primary row committed = True (the mutation happened, only the decision was lost): an object whose __eq__ raises → ValueError; a dataclass whose field read raises → OSError; a dataclass with different fields → OSError; a nested list of depth 2000 → RecursionError (collect bounds itself on field count via _MAX_MISMATCH_FIELDS=8 but never on depth); a dict with non-comparable keys → TypeError from sorted(left.keys() | right.keys()). An on_mismatch observer raising BaseException escapes the same way (:130 is also except Exception). This is precisely the threat model subagent.py:2351 claims to handle: "Keep this phase primary-preserving even when an integration violates it."
3. sqlite.py:120 — migration v2's ALTER TABLE commands ADD COLUMN payload_json is not idempotent and _schema_version() returns 0 for a DB that HAS the physical schema, so any lost or corrupt metadata value permanently bricks the store on every call — including read-only get_run — with no recovery path. Executed against a healthy v2 DB, twice each so the brick is permanent rather than transient: schema_version='1' → OperationalError: duplicate column name: payload_json; row DELETED → same; metadata table DROPPED → same; schema_version='-1' → same. Second mechanism at :227 (return int(row[0]) if row is not None else 0): 'corrupt', '', '2.9' each raise an unhandled ValueError: invalid literal for int() out of _connect forever (the except BaseException only closes and re-raises), while '02' and '2 ' are silently accepted so the downgrade guard's notion of "newer" is whatever int() parses. Violates the RFC's own exit criterion "schema creation and upgrade are crash-safe and idempotent" — and note migration atomicity does hold (an injected bad statement rolls back cleanly, verified), so it is replay that is broken.
4. subagent.py:2308 — raw_task = info._raw_task or info.task is the first site in the repo to write the deliberately UNREDACTED subagent prompt to disk, and it lands in two columns of a store nothing ever prunes. info.task is redact_credentials(redact_exfiltration_urls(task)[0])[0]; info._raw_task is documented at subagent.py:1309 as "unredacted task for kiro-cli execution prompt" and previously had exactly one reader (run.py:746, in-memory prompt build, never persisted). Executed: a task containing AKIAIOSFODNN7EXAMPLE + an AWS secret key + an exfil URL is stored verbatim in both runs.task and commands.payload_json, and all three needles are present in the raw coordinator.db bytes — while the redacted form the rest of the codebase persists is deploy using [REDACTED: credential] / [REDACTED: credential]. Every other persistence/emit site uses the redacted copy (admission.py:763, run.py:847, terminal.py:115, monitoring.py:745, all SEL metadata). grep -rn 'DELETE|prune|retention|vacuum' src/kiro_crew/run_coordinator/ finds only the three full-table wipes inside _save_memory, so unlike result.txt (which has agent.subagent_result_ttl_secs and a reaper) the plaintext accumulates for the life of the install.
5. sqlite.py:150 — _database_path checks only the leaf and its immediate parent, then resolve()s through any ancestor link, so with the spec-supported symlinked data home the physical DB sits at a path is_sensitive_path does not match. Executed: with ~/.kiro/crew → ~/elsewhere/crew, is_sensitive_path('~/.kiro/crew/run-coordinator/coordinator.db') is True but is_sensitive_path('~/elsewhere/crew/…') is False, and is_sensitive_bash_command("sqlite3 ~/elsewhere/crew/run-coordinator/coordinator.db 'select task from runs'") → blocked=False (the logical spelling is blocked). So the unredacted task payloads of finding 4 are readable through an ungated spelling. Also executed with a tmp-dir ancestor symlink: submit -> applied, DB physically created inside the attacker-controlled target, no refusal. platform_compat.first_linked_ancestor exists for exactly this ("a caller that checks only the path it was handed still resolves through a linked PARENT"; on Windows an ancestor junction to \\host\share turns the first is_dir() into an authenticated SMB call) and is already paired with is_link_or_junction at clone_setup.py:141 and themes.py:242 — it is not used here. Related: is_link_or_junction cannot see hardlinks; executed, a hardlinked coordinator.db and a hardlinked -wal are both accepted and SQLite writes plaintext task text through them ('SECOND-SECRET-TASK' readable through hardlink: True, 32,992 bytes of WAL frames in the attacker inode).
6. sqlite.py:173 — the -journal sidecar that _database_files explicitly enumerates is the one file _secure_existing_database_files can never secure, and PRAGMA journal_mode=WAL's result is discarded. Executed over 60 submits with umask 022: coordinator.db-journal ['0o644'], journal EVER 0o600? False (db, -wal and -shm are all 0o600). The rollback journal is created by the WAL header write, lives at umask perms for its whole lifetime, and is deleted before any chmod pass sees it — so 0o644 pre-images of every task payload hit disk. Separately executed: with a foreign connection holding a read transaction on a not-yet-WAL DB, PRAGMA journal_mode=WAL raised: OperationalError database is locked immediately, well inside the 5s busy timeout — the code's own comment says this pragma "does not reliably honor busy_timeout", yet _JOURNAL_MODE_LOCK is a threading.Lock that serialises only threads in one interpreter, the return value is never inspected, and nothing retries. Both contradict the spec added in this commit ("The directory and database/known sidecars are tightened owner-only"; "Concurrent initialization serializes the WAL-mode transition").
7. sqlite.py:341 — a task containing a lone surrogate is accepted by the in-memory oracle but makes every SQLite mutation raise UnicodeEncodeError, which the shadow swallows, so the run is silently absent from the durable store forever. Executed end-to-end through the real manager entry point: json.loads (of an MCP/ACP tool argument containing a \udXXX escape) gives info._raw_task = 'summarise \ud83d the thread' — a legal Python str, not UTF-8 encodable. MemoryRunCoordinator.submit → applied; _save_memory's INSERT INTO runs binding run.task → UnicodeEncodeError: 'utf-8' codec can't encode character '\ud83d' → _invoke rolls back → _mirror's except Exception swallows it → primary APPLIED returned, legacy run proceeds. Result: clean-run durable row -> PRESENT / surrogate-run durable row -> MISSING, and every exact retry fails identically. commands.payload_json is unaffected because json.dumps defaults to ensure_ascii=True, so the hash column is writable while runs.task is not. The contract suite is now parametrised over memory and sqlite yet covers no non-encodable text (NUL bytes round-trip fine, so surrogates are the specific gap).
Should fix
8. sqlite.py:413 — every operation loads all three tables into a throwaway MemoryRunCoordinator, DELETEs and re-INSERTs every row, and runs a whole-database PRAGMA quick_check — under BEGIN IMMEDIATE even for the read-only get_run — so cost is O(total lifetime runs) and nothing ever prunes. Measured single-submit latency: 2.5–5.2 ms at N=0 · 17–19 ms at N=1000 · 90–107 ms at N=5000 · 391–1500 ms at N=20000 (139 MiB rewritten under synchronous=FULL for one new row) · 1025 ms at N=50000. Write amplification at N=5000/10 KB tasks: 20,006 row mutations and 52,245,499 bytes re-serialised to persist 2 useful rows; tracemalloc peak 55.1 MB per call, ×2 workers. get_run (persist=False) still does all of it: 6.07 ms @ N=1 → 43 ms @ N=5000 → 479 ms @ N=50000, vs 0.005 ms for SELECT * FROM runs WHERE run_id=? (up to 16,114×). And with one other writer holding the lock: plain WAL read 0.4 ms OK vs coordinator get_run after 5.20 s -> OperationalError: database is locked — the exact property WAL was enabled for. Consequence: submit crosses _SHADOW_SUBMIT_TIMEOUT_SECS = 1.0 with zero contention around 15k–50k lifetime runs, after which every spawn stalls a full second at run.py:319 and the shadow silently records nothing. The five declared indexes and the version OCC column are never used — grep 'WHERE' over sqlite.py hits only the two metadata lookups.
9. subagent.py:2289 — the 1.0 s asyncio.wait_for bounds only the caller, not the offloaded SQLite work, so under contention started writes commit AFTER the caller logged "shadow submission failed" while queued writes are cancelled and never mirrored — both emitting the identical warning. Executed with an external writer holding the lock and 6 concurrent shadow submits on the 2-worker pool: all six callers released at exactly 1.001 s with TimeoutError, pool queue still 4 deep, and after the lock cleared rows actually persisted: ['warm'] — 6 of 6 mirrors lost. A second run split the other way: committed anyway AFTER the caller reported failure: ['run-1','run-2'] / silently DROPPED, never mirrored: ['run-3','run-4','run-5'], the late commits landing at t=10.3 s. Measured capacity: 18.3 submits/s at N=5000/64 B (first timeout at concurrent spawn #19; burst 16 → 3 timeouts); at 10 KB tasks 2.4/s (burst 4 → 2 timeouts, burst 16 → 14). So the parity dataset this PR exists to collect is non-deterministically partial in both directions and the log cannot distinguish them. executors.py already has the two-phase queued-vs-claimed discipline (run_in_cron_pool) that would bound this; the coordinator path does not use it. The PR's own test patches the timeout to 0, so it proves nothing about the real budget.
10. sqlite.py:333 — runs and outbox are written with column-less positional INSERTs (18 and 14 placeholders) while commands names its 13 columns, so the next ADD COLUMN migration — the only shape SQLite's ALTER supports, and exactly what this PR's own v2 migration does — wedges every write while reads keep working. Executed: append (3, ("ALTER TABLE runs ADD COLUMN cancel_reason TEXT NOT NULL DEFAULT ''",)) → v2 submit: applied → migration commits → v3 submit FAILS: OperationalError table runs has 19 columns but 18 values were supplied → read still works: True → retry FAILS identically. Because _shadow_submit_accepted_run swallows everything, the only symptom is one warning per spawn while the store looks alive and accepts nothing. The inconsistency is inside one function: commands was given an explicit column list precisely because v2 appended a column to it, and the lesson was not applied to the other two tables. The same construct also means reordering RunRecord's frozen-dataclass fields silently writes values into the wrong columns — the tables are not STRICT, so a TEXT stored in lease_expires_at REAL raises nothing.
11. shadow.py:29 — the parity oracle is structurally blind in four ways at once, so a shadow with broken leases, a dropped mutation, or a wrong terminal instant compares CLEAN while the delivery path reports a permanent false mismatch — and there is no counter, so the migration gate cannot be read at all. (a) _VOLATILE_FIELDS excludes lease_expires_at/claim_expires_at/terminal_at, which are caller-supplied (memory.py:173/305/346/372), not clock-derived — executed: shadow lease 30 → 1e9 (never expires) = CLEAN, terminal_at None → 999999 = CLEAN, while version/attempt/lease_epoch/owner_id/outcome/error/result_path are caught. (b) The REJECTED early-return at :108 skips a decision class that does mutate: submit(accepted=False) writes a TERMINAL run + a REJECTED command and then returns REJECTED — executed: primary: runs=['run-A'] commands=['cmd-A'] / shadow: runs=[] commands=[] / mismatches reported = [], and a resubmit then gets primary=identity_conflict vs shadow=applied, still silent. (c) event_id masking hides a guaranteed outbox-identity divergence (two independent uuid4s) and then every claim_outbox/mark_delivered reports decision,reason,value forever, because the primary's fence id does not exist in the shadow. (d) _mirror has no timeout on any of the 10 boundaries (grep -c 'wait_for|timeout' shadow.py = 0) — a 10 s shadow returns the primary's already-computed decision after 10.008 s; only submit has a call-site bound. RFC §8 requires "shadow parity mismatch by field class" as a metric; grep -rn 'metric|counter|gauge' src/kiro_crew/run_coordinator/ finds nothing and on_mismatch has no production caller.
12. subagent.py:2311 — the "canonical execution payload" omits approval_mode, requested_model and resolved_model, so two submissions differing only in whether the run gets blanket tool auto-approval produce an identical payload_hash and are accepted as an idempotent replay instead of an idempotency conflict. approval_mode='auto' grants blanket tool auto-approval in _run_inner_impl and emits a subagent.approval_mode_auto_policy SEL record. Executed against the real _shadow_submit_accepted_run: approval_mode='' payload_hash=c7fc09689ce53e75 and approval_mode='auto' payload_hash=c7fc09689ce53e75 → identical hash for different approval policy. memory.submit's only conflict test is payload_hash != request.payload_hash, so a resubmit under the same key that flips the approval policy returns unchanged/idempotent_replay and the durable command keeps the first request's semantics. This also breaks the purpose the spec states for storing the payload ("a claimed command can be reconstructed after restart"): a restart replay drops the approval policy and re-resolves the model unpinned. Nothing anywhere validates sha256(payload_json) == payload_hash, and SubmitRun.payload_json defaults to "" while payload_hash is mandatory, so a hash-with-no-payload row is representable — the PR's own test_run_coordinator_shadow.py::_request constructs exactly that.
13. memory.py:333 — renew() never checks observed_state, so the fencing lease on an already-TERMINAL run can be extended to any future timestamp, and SQLite persists it — permanently disabling claim_commands' expired-lease reclaim arm for that run. After complete() the run is TERMINAL but owner_id, lease_epoch and lease_expires_at are left intact. The old owner's heartbeat — which has no reason to know it lost, since finding 1 of the #5277 review shows its completion was reported as a clean replay — calls renew('run-1', RunFence('run-1','gw',1), until=9999). All four guards pass. Executed on both adapters: renew(TERMINAL run) -> True lease now: 9999.0 observed: terminal, durable under SQLite. claim_commands' only recovery arm for a wedged command is status is CLAIMED and run.lease_expires_at <= now, which can now never fire, so a fenced-out worker holds a live renewable fence on a finished run indefinitely. Every other transition gates on _STARTABLE_STATES/_COMPLETABLE_STATES; renew is the exception.
14. memory.py:271 — complete()'s outbox-replay short-circuit returns BEFORE _validate_transition, so it is the only boundary that never checks the fence, lease epoch, lease expiry or expected_version — and it hands the caller the terminal OutboxEvent including its payload_json. Executed on both adapters: complete(expired lease + wrong version) -> unchanged completion_replay | leaked payload: {"summary":"SECRET RESULT"}. Every other transition validates the fence first. SQLite now makes the terminal row survive restarts, so the window is unbounded rather than process-local, and once PRs 5–7 wire terminal delivery two owners both get a success-shaped answer plus the delivery event. (Same root cause as #5277's finding 1; flagged here because this PR is what makes it durable.)
15. sqlite.py:198 — _prepare_path re-mkdirs and re-chmods the ledger directory on every transaction, and _secure_existing_database_files runs twice per call. Measured per coordinator transaction: 1 mkdir + 7 chmod + 10 lstat + 9 stat. A keyed spawn is 4 transactions, so ~4 mkdir + 28 chmod + ~76 stat/lstat syscalls per spawn on the data-home mount, for a path whose identity is already memoised in _resolved_path and in run_coordinator_anchor._anchor_cache. _invoke also spins up and tears down a brand-new asyncio event loop per transaction (asyncio.run(...) inside a worker thread) just to drive an uncontended in-memory lock — measured 0.174 ms/call, ~0.7 ms of pure loop churn per keyed spawn plus two fds created and destroyed. Cheaper: harden the path once per resolved path behind the same _path_resolution_lock that already caches it (exactly as prime_voice_runtime_sandbox_paths does for the voice runtime), drop the duplicate pre-commit securing, and drive the coroutine directly.
Below the cap (all verified)
_SCHEMA_V1 ships the RFC's nullable owner/lease columns as NOT NULL with ""/0.0 sentinels, so "unowned" is inexpressible in SQL · complete()'s replay-equality omits terminal_at, silently keeping the first value · run_id = uuid4().hex[:8] (32 bits, uniqueness contracted only among live agents) becomes a permanent PRIMARY KEY in a never-pruned table · claim_commands/complete raise bare KeyError on a partially-corrupt store · run_coordinator/__init__.py eagerly imports .sqlite, adding 15.4 ms and executors/platform_compat/ctypes/windows_acl to every consumer of the pure-stdlib contract · _claim_commands sorts and scans the entire command table even when a single command_id is requested (0.069 ms at 201 → 1.706 ms at 10,001, ~25× for identical work), and renew/complete materialise tuple(self._commands.items()) to touch one run's rows · three test defects: test_default_manager_does_not_retain_shadow_runs only re-asserts _coordinator is None, test_vanished_sqlite_sidecar_does_not_break_concurrent_preflight has no assert and contains latent unbounded recursion, and test_existing_database_files_are_restricted_before_sqlite_opens monkeypatches process-global sqlite3.connect with an asserting wrapper · conventions: the RFC's last-audited/audited-at/implementation-prs front matter is left stale against docs/request-for-change/README.md:160-162, and ~25 hand-applied (not black-produced) reformat hunks in security.py plus ~91 in test_session_sharing.py are bundled into a feature commit against AGENTS.md:358-361 · cleanup: 149 lines of hand-written row↔dataclass mapping reaching into five MemoryRunCoordinator privates with no snapshot/restore API; a 4th open-coded sidecar list that already disagrees with three siblings; a 10th verbatim lazy-pool copy with atexit.register per pool; a duplicate parity comparator in subagent.py; _refuse_newer_schema duplicated with the non-atomic outer copy dead; 10 cast() forwarders; -journal/attempt dead columns.
Note for CI: scripts/check_black_formatting.py fails on test/test_run_coordinator_sqlite.py, which this PR introduces and which is not in .github/black-baseline.txt. That makes the gate red for every PR from here up the stack — #5283's review hit the same failure independently.
Verified clean, so nobody needs to re-check
Migration atomicity genuinely rolls back (tables after failed migration: [], schema_version: ('2',)); the pre-commit ACL failure rolls the transition back (runs table: ['r1'] after injecting an OSError at :424); cross-process serialisation holds (3 processes × 40 submits → 120/120 rows, no clobber); journal_mode=wal, synchronous=FULL, foreign_keys=1, busy_timeout=5000 are all live inside the transaction and the DELETE/INSERT ordering never violates the FK or either UNIQUE; db/-wal/-shm are 0o600 on POSIX; the newer-schema refusal precedes sidecar creation; all 89 touched security.py regexes are byte-identical (only the additive run-coordinator alternations differ) and _data_consumer_exempt matches over 1404 enumerated inputs; the leaf does not over-block (run-coordinator-backup, run-coordinatorX, subagents/run-1/result.txt all fine); coordinator=None causes no None-deref; the new await at run.py:319 is on the correct side of every lifecycle anchor; all 63 coordinator tests pass on 3.12 and 3.10; semgrep/isort/mypy/docs-lint clean; jscpd is website-only; the inline wokeignore is the documented pattern; CHANGELOG absence is correct.
Execution-verified AI-assisted review (Claude Code) across 11 angles plus a gap sweep, run against a local checkout at b915711c1; the working tree is unmodified and nothing was posted elsewhere. Findings name the input that reproduces them — please push back where one misreads intent.
|
|
|
|
Addressed both current-head automated review findings in
Verification: the two regressions passed red-to-green; 409 coordinator tests passed with one valid platform skip; all 41 Seatbelt profile tests passed; Black, subprocess encoding, flake8, Linux mypy, docs-lint, agent-sdk boundary, harness parity, and brand gates passed. |
|
Disposition for the current-head GPT finding on 5e8863c: fixed in 4d5e174. Existing deterministic anchor records are now accepted only when they match the coordinator path currently resolved from the configured data home; pre-seeded records and linked-home replacement fail closed. Regression coverage exercises both restart retargeting and link-to-directory replacement. Focused verification: 409 coordinator tests passed with 1 valid skip; 41 Seatbelt profile tests passed; formatting, lint, Linux mypy, docs, agent-SDK boundary, harness-parity, and brand gates passed. |
|
|
|
|
Bolin review disposition for the current stack All fifteen numbered items from the September 2 review were rechecked.
The fixes include transactional rollback, parity containment, migration and enum hydration coverage, redaction and size bounds, symlink and stable-anchor protection, journaling and ACL hardening, explicit UTF-8 handling, and named inserts. Current submitted head: 790e8b1. |
Open PR relationship auditThis is a consolidated, point-in-time code-level audit note. It compares complete merge-base diffs and current/merged code; it does not treat a shared topic as duplication or partial coverage as completion. Relationship findings
No PR, Issue, label, branch, or review state was changed by the relationship-note portion of this audit. |
|
Kiro Crew [operator: chenmingwei23]: This PR has been inactive for 7+ days. I reviewed the blockers but they require your input:
Please land or rebase the earlier PRs in the stack (#5277/#5278) and resolve the conflict against the correct base, then the pipeline will re-assess. |
Problem / Motivation
The coordinator contract needs a durable implementation and production parity evidence before it can become authoritative.
Why it matters
An explicitly injectable, fail-contained shadow adapter exposes semantic mismatches without placing working subagent execution at risk. The production seam remains disabled until durable command authority and recovery coverage land in later stack layers.
What changed (motivation → approach → change)
Tests
test/test_run_coordinator_sqlite.pytest/test_run_coordinator_shadow.pytest/test_executors.pyManual verification
N/A — persistence, migration, security, timeout, and parity behavior are covered by automated tests.
Related Issues
No linked issue: this stack implements the locally reviewed durable run coordinator RFC.
Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)