Skip to content

build-gate: surface the gate verdict and return runBuild refusals as phased results (trace F1, F5) - #196

Merged
dsmcewan merged 2 commits into
mainfrom
claude/build-orchestrator-f1-f5
Sep 24, 2026
Merged

dsmcewan merged 2 commits into
mainfrom
claude/build-orchestrator-f1-f5

Conversation

@dsmcewan

@dsmcewan dsmcewan commented Sep 24, 2026 •

Copy link
Copy Markdown
Owner

What

Follow-up to the build orchestrator trace in #194, fixing findings F1 and F5 in build-gate/build-orchestrator.mjs. One concern: buildProject's build-phase result must not misreport the trust verdict or crash on a substrate refusal.

F1: ok:true did not mean certified. buildProject derived ok from the ledger alone (merge_status === "ready"). Under trust_mode:"advisory" the gate never certifies, so a keyless build was ok:true while result.council.certified was false and the gate's safe_next_action said do not merge. The build-phase result now carries a single derived verdict, merge_ready = ok && council.certified, beside ok. ok keeps its ledger meaning, so the advisory-mode tests and the committed evidence scripts under docs/runs/agentic-teams* are unchanged; an advisory build is ok:true, merge_ready:false. The gate's own verdict stays where it was, in result.council.certified. The docstring states both verdicts and now lists the situation phase it previously omitted.

F5: a runBuild refusal threw instead of returning. runBuild returns { error, detail, trace } with no report for PLAN_INVALID, PLAN_TAMPERED, PLAN_HASH_MISMATCH and MUTATE_FAILED (merkle-dag/orchestrate.mjs:204-206,266). buildProject dereferenced build.report.merge_status unconditionally, so any of them surfaced as TypeError: Cannot read properties of undefined (reading 'merge_status'). It now returns { phase:"build", ok:false, merge_ready:false, error, detail, trace, council, plan, advisories, situation, teams }, the same guard the lifecycle path has at proposal-orchestrator.mjs:316. Both build-phase returns are built from one shared base object.

Tests (build-gate/scripts/test-build-orchestrator.mjs):

  • The advisory happy path asserts merge_ready === false alongside ok:true, with council.certified === false.
  • PLAN_TAMPERED: a council seat caller corrupts the stored plan_hash in .telos/plan.json in the window between writePlan and runBuild. Asserts the refusal comes back as a phased result with ok:false, merge_ready:false, no report, an empty trace, and no ledger.jsonl. On the previous orchestrator this case throws the TypeError above (reproduced before the fix).
  • PLAN_HASH_MISMATCH: a seat caller writes a self-consistent substitute plan (same signers, changed requirements) during review, so the hash recomputes cleanly but is not the approved one. Same assertions. This is the case the authorizedPlanHash check exists for.
  • MUTATE_FAILED is not testable through buildProject in legacy mode: the adapter's respec only rewrites requirements, dependencies are never mutated, and legacy plans carry no obligations or lifecycle, so mutateNode has no error path to reach.

Not changed: the dead lifecycle branches (F4), triple test execution (F2) and capability packets (F6) from the trace are left for separate changes. A corrupt or unreadable plan.json (invalid JSON) still throws from readPlan inside runBuild; that is an exception, not a substrate refusal, and the lifecycle path behaves the same way. The evidence scripts under docs/runs/agentic-teams* record ok but not error; they already exit non-zero on ok:false, and adding fields to committed evidence needs the evidence regenerated, so that is a separate change.

Checklist

  • Affected package suites run locally (list them below with results)
  • No new dependencies or lockfiles in zero-dependency packages
  • No runtime .telos/ artifacts, secrets, .env*, or *.pem files committed
  • Hash-pinned/signed artifacts untouched (or regenerated through their own runners, with the regeneration described below)
  • If governance records or contracts changed: not applicable, none changed

Test results

Node 22.22.2, branch on top of main 39d3e9f:

  • cd build-gate && npm test -> exit 0 (all suites incl. test-build-orchestrator 16 cases, test-runtime-adaptation, test-proposal-orchestrator, breakout)
  • the PLAN_TAMPERED scenario run standalone against the previous build-orchestrator.mjs -> TypeError: Cannot read properties of undefined (reading 'merge_status')
  • same scenario against this branch -> { phase:"build", ok:false, merge_ready:false, error:"PLAN_TAMPERED" }

🤖 Generated with Claude Code

https://claude.ai/code/session_01JrjwsLBYk3FcnkneUTqqkQ

…phased results

Two fixes from docs/build-orchestrator-trace.md (PR #194), F1 and F5.

F1: buildProject's build-phase result now carries `certified` (the gate's
report.certified) beside `ok` (the ledger's merge_status === "ready").
Under trust_mode "advisory" the gate never certifies, so a keyless build
can be ok:true and certified:false at the same time; before, that fact
was only reachable through result.council. `ok` keeps its ledger meaning
so advisory tests and evidence scripts are unchanged; the docstring now
states both verdicts.

F5: runBuild returns { error, detail, trace } with no report for
PLAN_INVALID, PLAN_TAMPERED, PLAN_HASH_MISMATCH and MUTATE_FAILED.
buildProject dereferenced build.report.merge_status unconditionally and
threw a TypeError on any of them. It now returns
{ phase:"build", ok:false, certified, error, detail, trace, ... },
mirroring proposal-orchestrator.mjs.

Tests: the happy path asserts certified:false with ok:true in advisory
mode; a new case tampers .telos/plan.json from a council seat (the window
between writePlan and runBuild) and asserts the PLAN_TAMPERED refusal is
returned as a phased result with no ledger written. On the previous
orchestrator that case threw "Cannot read properties of undefined
(reading 'merge_status')".

Verified: cd build-gate && npm test -> exit 0 (Node 22.22.2).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JrjwsLBYk3FcnkneUTqqkQ
…H_MISMATCH test

Review follow-ups on the F1/F5 change:

- Replace the top-level `certified` with `merge_ready` = ok AND gate
  certified. Echoing report.certified beside ok:false on a refused build
  could read as "certified" for a plan the substrate just refused; one
  derived verdict cannot disagree with itself, and the gate's own verdict
  stays in council.certified.
- Build both build-phase results from one shared base object so the two
  returns cannot drift.
- Add "situation" to the docstring's phase list (it was already missing
  and this docstring is the one being rewritten).
- Test the second refusal shape: a self-consistent substitute plan written
  during council review recomputes fine but is not the approved hash, so
  runBuild refuses with PLAN_HASH_MISMATCH, returned as a phased result.
  The substitution happens on the first seat call only; the council calls
  the seat once per model.

cd build-gate && npm test -> exit 0 (Node 22.22.2).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JrjwsLBYk3FcnkneUTqqkQ

Copy link
Copy Markdown
Owner Author

CI on a4927ab: merkle-dag (node 22) failed; every other job, including merkle-dag (node 24) on this same commit and both build-gate jobs, passed.

The failure is merkle-dag/scripts/test-orchestrate.mjs Case 13, a wall-clock sanity floor (paraWall=555ms < serialWall=508ms) in code this PR does not touch. The test's own comment says the deterministic peak-concurrency counter is the primary proof and the wall-clock check is a floor that can vary with host scheduling. The same job passed on the previous commit of this branch (b00d101), which has identical merkle-dag/ contents, and on node 24 for this commit. Not this PR's failure; no fix for it exists in the repository, and hardening that timing check belongs in a merkle-dag change, not here.

Re-running the failed job once.


Generated by Claude Code

@dsmcewan
dsmcewan marked this pull request as ready for review September 24, 2026 14:27
@dsmcewan
dsmcewan merged commit 21bc81a into main Sep 24, 2026
61 of 63 checks passed
dsmcewan pushed a commit that referenced this pull request Sep 24, 2026
Bring the per-node verify stage up to date with main (079e77e), which now
carries #193 (repo code trace + keyless evidence fixes), #196 (gate verdict
surfacing + runBuild refusals as phased results, F1/F5), and #194 (trace docs).
build-orchestrator.mjs auto-merges cleanly: the verify-stage changes and the
F1/F5 changes touch different regions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rxfYqddLk2NykrPkjy1Pr
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants