Phase A: correctness closure from the 2026-07-07 audit - #47
Conversation
Phases A-F execute unattended from these two files; committing first so every phase PR cites a fixed SHA and fresh sessions can resume. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The A-2 anchored detector shipped only on the document pipeline; dev-review's execute/verify/plan gates kept the superseded <50-word heuristic (C-1/C-6). Routing them through the anchored scan then dropped the one case the old heuristic did catch — an output that IS a bare 'Unauthorized' banner — so output_is_auth_failure combines both: anchored head-scan OR loose match under 50 words (C-8, flagged independently by both review vendors). Also: timeout aborts write a terminal status, failed plans no longer spawn stray branches, and --run-dir is documented. Auth-gate sim 9->28 scenarios. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Empty agent output (call + retry) used to break the loop before any pass was recorded; the marker-free compose draft then counted zero markers and finalized 'converged' with passes=[], and --execute ran the un-reviewed draft (C-2). Both the zero-pass and partial-failure exits now die into the existing status=aborted terminal, which the scorer already gates and --execute already refuses. Also dies loudly on --agents with no comma or no value (silent self-pair defeated the cross-vendor premise), and the adjudication receipt gate now counts marker tokens per line, not lines. Marker-lifecycle sim 37->41; new audit-hardening sim 18 scenarios; byte-parity fixtures untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
evals/tests/bounce-scorer-verification.sh was referenced only by docs — a regression in score-bounce.sh would ship silently (C-5). It also failed 5/7 under Windows-native jq, which emits CRLF that the failing_checks_include read-loop never stripped. Wire it into run-all's SUITES with the \r strip. Worktree sim's Scenario F now anchors on the stable BRANCH_SPEC test instead of the full guard line, and Scenario G pins the new PLAN_EXIT gate on both setup legs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3821ee5963
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| output_contains_auth_banner "$file_path" && return 0 | ||
| [[ -s "$file_path" ]] || return 1 | ||
| words=$(wc -w < "$file_path" | tr -d '\r\n ') | ||
| (( words < 50 )) && file_contains_auth_failure "$file_path" |
There was a problem hiding this comment.
When a valid artifact is under 50 words and mentions an auth term mid-sentence (for example, a small plan documenting 401 Unauthorized handling), this loose fallback returns true even though the anchored banner check failed. Because validate_agent_artifact and the dev-review gates now call output_is_auth_failure, those short legitimate outputs are aborted or routed to manual auth review; the parent accepted this case. Please narrow the fallback to outputs that consist only of a bare auth banner/token rather than any short text containing the substring.
Useful? React with 👍 / 👎.
1 participant
Executes Phase A of the audit improvement plan (
.planning/notes/2026-07-07-audit-improvement-plan.md@ 4812210), driven by the execution loop in.planning/notes/2026-07-07-execution-loop.md.What this fixes
output_is_auth_failurehelper combines the anchored head-scan with the old short-output loose match, so both the long-auth-page hole and the bare-Unauthorized-banner hole are closed at every output path.status=abortedinstead of finalizingconvergedwithpasses=[].--executecan no longer receive an un-reviewed draft.evals/tests/bounce-scorer-verification.shnow runs in run-all/CI; it had a Windows jq-CRLF bug (5/7) that is fixed (7/7).--agentsself-pair and unbound-$2 both die loudly;strip_human_summaryno longer truncates documents with a body-level HUMAN SUMMARY heading; adjudication receipts count marker tokens, not lines; timeout aborts write a terminal status; failed plans don't create stray branches/worktrees;--run-dirdocumented.Verification
🤖 Generated with Claude Code