Repository navigation
Re-ask signed-source role mix-ups and resume classification-only rounds (#2035) - #2044
Conversation
…ds (#2035) Role/field consistency violations in requirement_evidence associations are reviewer judgements: parse_associations now raises MissingJudgementFieldError naming the entry index, requirement_id and violated rule, so the classification continuation gets the bounded judgement re-ask and, when it is exhausted, a deterministic reviewer-judgement failure instead of a schema-validation stop. Shape errors stay plain AgentLoopError. A published review whose only gap is the signed-source classification step is admitted on rerun as classification-pending (publication record, sources not covered) or classification-complete (validated associations cover every source) in an unreconciled round with an unchanged signed requirement set. Only the classification continuation reruns, pinned to the published verdict and findings; a change stops for a human decision before coder dispatch. Other rejections keep the partial-round refusal, which now names the failing check. Fixes #2035 Agent-Issue-Provenance: v1 repo=wwind123/coding-review-agent-loop issue=2035 flow=approved plan=8523c814220e10db Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue implementationImplemented the approved plan for #2035 and opened draft PR #2044. (1) parse_associations raises MissingJudgementFieldError naming the entry index, requirement_id and rule for role/field consistency violations; shape errors stay plain AgentLoopError. The classification invoke (shared CLASSIFICATION_INVOKE_OPTIONS) re-asks once, and on exhaustion takes the typed judgement path, now labelled reviewer-judgement instead of schema-validation. It uses a reviewer_judgement_turn flag instead of repair_expected_kind so Antigravity classification turns do not switch to structured output. (2) A published review whose only gap is classification is admitted on rerun as classification-pending or classification-complete in an unreconciled round with an unchanged signed requirement set. Only the pinned classification continuation reruns, and a changed verdict or finding stops for a human decision before coder dispatch. Other rejections keep the partial-round refusal with a named cause. The plan asked for the CI suite, which justified the one full-suite run (tests/ -q): 4 failures out of 25857. Two were layering/patch-inventory guards and two pinned the old behaviour, where an unacknowledged but fully classified single-reviewer record was re-run; it now resumes as classification-complete. All four were fixed, and the affected files plus the focused set (requirement_evidence, validated_agent_recovery, pr_loop_resume, partial_round_recovery, judgement re-ask, failure classification, shape audit, file size, docs and architecture guards, review_parallel/rounds, closed round, human requirements, response_validation, checkpoint resume) pass at the final tree. Docs updated: round-state-and-resume.md, its decision log and the partial-round recovery section of local_agent_loop.md. ResultPull request reported: #2044. Tests run
Test observation receiptsRisk-based mode and transition test matrix evidence
Full matrix evidence (11 rows)
Risk-matrix coverage mapOrchestrator completeness check (existence and level only; whether each cited test really exercises its row is for the reviewer to judge).
-- Anthropic Claude: claude-opus-5-5 (medium) |
|
Agent-loop managed-CI resume provenance audit record (machine-readable). Records resume provenance for managed CI on pull request #2044 at head 2f088b3. Not an agent response; keep this comment. AGENT_MANAGED_CI_UNPROTECTED_OVERRIDE_V1 nonce=n9JOLlLrZ92J1PefLpu0zRsn_2_3zLvs repo=wwind123/coding-review-agent-loop base=main head=2f088b38917f3efd9fe73e0c8fabb31e677eaf80 protection=voluntary active_label_event_id=36779423960 resume_from=7042389412 provenance_head=2f088b38917f3efd9fe73e0c8fabb31e677eaf80 generation=b7ffZ-6XXDoxwFKB Resume provenance only: the prior issue-created audit is not an authorization token. |
|
PR review scheduling audit: selected Codex; paused Claude, Antigravity; reason: strict pre-panel fallback: architecture identity changed; scheduler metadata recovery: latest scheduler metadata is missing or invalid; scheduler metadata recovery: current-head records lack scheduler metadata; scheduler metadata is missing or invalid; primary re-invoked with full context; phase: primary; head: 2f088b3; primary: Codex; active owners: (none); force-full: False (source: none); scheduler-policy calls avoided cumulatively: 2. |
|
Review verdict: Blocking Reviewed the complete local base-to-head diff at 2f088b3. The judgement re-ask change is sound, but classification recovery loses acquisition identity and can fail to persist reconciled associations. Required workflow coverage also remains incomplete. GitHub checks are passing; no tests were executed during this read-only review. Blocking issues
-- OpenAI Codex: gpt-6.1-sol (medium) |
|
PR review round 1 reconciliation: settled reviewers: Codex. Finalization continues after reconciliation. Historical approvals remain exact-head-bound; scheduler-policy calls avoided cumulatively: 2. Phase: primary; force-full: False (source: none). |
|
PR #2044 coder dispatch record: round 1, attempt 1, head |
…ion settlements (#2035) A resumed review's settlement now carries the original review's acquisition identity instead of none, so scheduler/seat acquisition checks still admit it on a later rerun. An admitted authoritative settlement whose pinned conflict continuation reconciles its role posts the new classification with its existing finding IDs, so a later rerun restores the reconciled role. Tests cover the round-2 #2310 shape in sequential and parallel reruns (with and without a scheduler policy), pin the repeated-failure rerun's final settlement, reconciliation and acknowledgement boundary, and the persisted reconciled role across a further rerun. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Coder follow-upFixed all three round-1 items. Resumed settlements now carry the original review's acquisition identity, and an admitted authoritative settlement whose role is reconciled posts its new classification with its existing finding IDs. The workflow tests now cover the round-2 #2310 shape in sequential and parallel reruns, with and without a scheduler, and the repeated-failure rerun's final outcome is pinned explicitly. Generalization: this generalizes the fix for [item-1] and [item-2]: every settlement posted for a review restored through classification-only resume carries the restored record's acquisition identity and existing finding IDs, whatever its phase. Test rationale: test_pr_loop_resume/test_partial_round_recovery/test_requirement_evidence/test_validated_agent_recovery cover the changed admission and settlement paths. pr_loop_agent_turns, human_requirements, review_parallel, closed_round, review_scheduling, checkpoint_round_resume, reviewer_boards, board_change, evidence_conversion and carried_review_context cover pr_loop's resumed-settlement posting. The size, module split and docs guards cover the edited pr_loop and docs. Addressed items
Remaining items
Sub-item progress
Tests run
Test observation receiptsRisk-based mode and transition test matrix evidence
11 changed rows; 0 unchanged since round 1
0 rows unchanged since round 1; full matrix in round 1. Risk-matrix coverage mapOrchestrator completeness check (existence and level only; whether each cited test really exercises its row is for the reviewer to judge).
-- Anthropic Claude: claude-opus-5-5 (medium) |
|
PR review scheduling audit: selected Codex; paused Claude, Antigravity; reason: strict pre-panel fallback: diff path 'docs/architecture/round-state-and-resume.md' is outside obligation scopes; primary re-invoked with full context; phase: primary; head: 505628d; primary: Codex; active owners: Codex; force-full: False (source: none); scheduler-policy calls avoided cumulatively: 4. |
|
Review verdict: Blocking Reviewed the complete base-to-head diff at 505628d. The acquisition-identity and classification-persistence defects are fixed. The risk-matrix coverage map still overstates managed-CI checkpoint coverage for resume-classification-only; the remaining carried test obligation needs completion. Other matrix rows have suitable source and test coverage. GitHub checks are passing; no tests were run during this read-only review. Prior unresolved item dispositions
-- OpenAI Codex: gpt-6.1-sol (medium) |
|
PR review round 2 reconciliation: settled reviewers: Codex. Finalization continues after reconciliation. Historical approvals remain exact-head-bound; scheduler-policy calls avoided cumulatively: 4. Phase: primary; force-full: False (source: none). |
|
PR #2044 coder dispatch record: round 2, attempt 1, head |
…on checkpoint (#2035) A qualifying managed-CI checkpoint whose requirement identity a new signed source invalidates forces the round's reviews; Codex's classification then fails. The rerun redoes only that classification, keeps the original checkpoint verbatim, reattaches the same qualification attempt without a fresh dispatch or managed-CI authorization, and records exactly the checkpoint sequence an uninterrupted control round records. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Coder follow-upCompleted item-3.s1's qualification-checkpoint obligation with a test against a real managed-CI qualification checkpoint, which is now in tests/test_pr_loop_resume.py. Production code is unchanged this round. Test rationale: test_pr_loop_resume holds the new and existing resume tests; test_partial_round_recovery, test_requirement_evidence and test_validated_agent_recovery cover the other matrix rows cited earlier; test_pr_loop_managed_ci covers the managed-CI paths the new test drives; test_module_split_imports covers the patch inventory guard. Addressed items
Remaining items
Sub-item progress
Tests run
Test observation receiptsRisk-based mode and transition test matrix evidence
11 changed rows; 0 unchanged since round 2
0 rows unchanged since round 2; full matrix in round 1. Risk-matrix coverage mapOrchestrator completeness check (existence and level only; whether each cited test really exercises its row is for the reviewer to judge).
-- Anthropic Claude: claude-opus-5-5 (medium) |
|
PR review scheduling audit: selected Codex; paused Claude, Antigravity; reason: primary phase: primary rechecks its findings before the secondary panel; phase: primary; head: 7c48c83; primary: Codex; active owners: Codex; force-full: False (source: none); scheduler-policy calls avoided cumulatively: 6. |
|
Review verdict: Approved Reviewed the complete base-to-head diff at 7c48c83 and mapped all eleven risk-matrix rows to the source and tests. The new managed-CI recovery test closes the remaining checkpoint coverage gap; sequential blocking-review recovery and repeated-failure settlement assertions are also present. No merge-blocking defects found. GitHub checks are passing; no tests were run during this read-only review. Prior unresolved item dispositions
-- OpenAI Codex: gpt-6.1-sol (medium) |
|
PR review round 3 reconciliation: settled reviewers: Codex. Finalization continues after reconciliation. Historical approvals remain exact-head-bound; scheduler-policy calls avoided cumulatively: 6. Phase: primary; force-full: False (source: none). |
Sub-item progress — review round 3
-- Orchestrator |
|
PR review scheduling audit: selected Claude, Antigravity; paused Codex; reason: independent secondary audit after exact-head primary approval; phase: secondary-audit; head: 7c48c83; primary: Codex; active owners: (none); force-full: False (source: none); scheduler-policy calls avoided cumulatively: 6. |
|
Review verdict: Approved Independent secondary audit of the full base-to-head diff at 7c48c83. Item 1: parse_associations now raises MissingJudgementFieldError(field_path=requirement_evidence) for the three role/field consistency checks, in the same order as before. Each message names the index, requirement_id and rule. The empty requirement_id check and the other shape checks stay plain AgentLoopError. The classification invoke uses CLASSIFICATION_INVOKE_OPTIONS with an explicit reviewer_judgement_turn flag instead of repair_expected_kind. The plan allows this alternative, and it keeps format repair off. An exhausted re-ask takes the deterministic judgement branch with the new reviewer-judgement classification text, not schema-validation. should_retry stays false for that public validation failure, so there is no third invocation. Item 2: admit_classification_resumes runs right after the ordinary resumed_by_name filter. It admits 'pending' (publication phase, incomplete associations) and 'complete' (validated associations cover every source, any phase) only in an unreconciled round with exact digest-identity equality, matching seat, plan and acquisition, and the same round and head. classify_review pins state and normalized blocking_items on every continuation for admitted reviewers, and forces the continuation only for pending records. The settlement paths now pass resumed_record.metadata as identity, so reposted settlements keep acquisition identity. An admitted authoritative settlement whose role was reconciled is reposted with its existing new_items. The blocking/approved branches post only for publication-phase records, so nothing is posted twice. Refusal causes are threaded through all three partial-round refusal seams. Tests exist at the reviewed head for every matrix row: re-ask and exhaustion through the production kwargs, typed consistency rules, shape errors staying fatal, the Run B rerun, repeated failure, the pending and complete pins, changed requirement sets, other mismatches, the closed round, and the managed-CI checkpoint survival. The latest coder coverage map lists only one row, but the earlier coder comment on this head lineage mapped all eleven rows to these same tests. They sit in files covered by the current-head passing receipt. GitHub checks are passing. Docs and the decision log are updated. Future follow-ups
-- Anthropic Claude: claude-opus-5-5 (medium) |
|
Review verdict: Approved Approve pull request #2044 on independent secondary audit. Signed-source role and field consistency checks in parse_associations now correctly raise MissingJudgementFieldError to trigger the bounded reviewer-judgement re-ask, with shape errors remaining plain AgentLoopError. Reruns of unreconciled rounds with unchanged signed requirements admit published reviews with missing or unacknowledged classifications as classification-pending or classification-complete without requiring review deletion, pinning published verdicts and findings during classification continuations. The implementation, architecture docs, and risk matrix test coverage are verified clean. -- Google Antigravity: Gemini 3.8 Flash (High) |
|
PR review round 4 reconciliation: settled reviewers: Antigravity, Claude. Finalization continues after reconciliation. Historical approvals remain exact-head-bound; scheduler-policy calls avoided cumulatively: 6. Phase: secondary-audit; force-full: False (source: none). |
|
Created approved-review future follow-up issues for PR #2044:
Reconciliation: 1 filed, 0 deduplicated, 0 skipped by cap. These were mentioned as future work and did not block merge readiness. -- coding-review-agent-loop |
|
Managed CI authorization for exact head 7c48c83. |
Summary
Fixes #2035
Two defects in signed-source classification, implemented per the approved plan.
1. A role mix-up gets re-asked instead of stopping the run
requirement_evidence.parse_associationsnow raisesMissingJudgementFieldError(field_path="requirement_evidence")for role/field consistency violations, following the issue's split between judgement and shape errors:"";[];requirement_evidence[i], therequirement_idand the violated rule, for example "mixed requires at least one answered_item_id; if no prior evidence item of yours is answered, use code or human-only."AgentLoopError: not an array, wrong keys, unknown role, wrong types, empty or duplicate IDs, and a repeated source.CLASSIFICATION_INVOKE_OPTIONS, so tests drive the exact production settings. It opts into the typed judgement-exhaustion branch through a newreviewer_judgement_turnflag.repair_expected_kind="pr_review"on that invoke, because doing so would also have switched Antigravity classification turns to structured output. The plan allows this fallback.reviewer-judgementcategory instead ofschema-validation, and the same label applies to the existing PR/plan judgement-exhaustion path.2. Rerunning a round where only classification failed
response_validation.classification_resume_kindadmits a same-round, same-head complete code review that the ordinary resume predicate rejects. Admission requires an open round, an unchanged signed requirement set (exact digest-identity equality) and matching plan and acquisition identities:admit_classification_resumesmerges admitted records intoresumed_by_name. Reconciled rounds and seat-binding mismatches are never admitted.classify_reviewgains two arguments:force_continuation(pending records only) runs the classification continuation even when no existing trigger fires;pinned_review(both kinds) applies to every continuation that runs. A changed state or blocking findings raisesHumanDecisionRequiredErrorbefore coder dispatch. Otherwise onlyrequirement_evidenceand the evidence requests are taken.Behaviour change to note
A single-reviewer round whose unacknowledged posted review already classifies every source now resumes as classification-complete instead of re-running the reviewer.
tests/test_pr_loop_agent_turns.pywas updated to pin this; the resumed carrier still keeps its metadata assessment and degradation record.Docs
docs/architecture/round-state-and-resume.md, plus a decision-log entry.docs/local_agent_loop.md.tests/fixtures/plan_execution_patch_inventory.jsongains two rows for the new tests' repair patches.tests/.test_durationsis untouched.Tests
tests/test_requirement_evidence.py,tests/test_validated_agent_recovery.py,tests/test_pr_loop_resume.pyandtests/test_partial_round_recovery.py.🤖 Generated with Claude Code
AGENT_MANAGED_CI_UNPROTECTED_OVERRIDE_V1 nonce=HQTdXMYIrSdzpN3Ssa8WNmAW