Skip to content

fix(quorum): add collect-evidence timeout controls#8493

Closed
scarmani wants to merge 17 commits into
mainfrom
codex/collect-evidence-timeouts-20260617
Closed

fix(quorum): add collect-evidence timeout controls#8493
scarmani wants to merge 17 commits into
mainfrom
codex/collect-evidence-timeouts-20260617

Conversation

@scarmani

Copy link
Copy Markdown
Collaborator

Summary

  • add explicit --reviewer-timeout and --overall-timeout controls to review-queue collect-evidence and scripts/collect_quorum_evidence.py
  • fail closed with JSON diagnostics when reviewer orchestration hits the overall deadline
  • keep partial timed-out runs prepare-only: no partial quorum posting, no fabricated evidence, no Tier 3/4 auto-posting

Validation

  • PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 timeout 240 /Users/armand/.pyenv/versions/3.11.11/bin/python -m pytest tests/swarm/test_quorum_evidence.py tests/cli/commands/test_review_queue.py -q
  • timeout 240 pre-commit run --files scripts/collect_quorum_evidence.py aragora/swarm/quorum_evidence.py aragora/cli/commands/review_queue.py aragora/cli/parser.py tests/swarm/test_quorum_evidence.py tests/cli/commands/test_review_queue.py
  • push hook: mypy on changed Aragora files passed

Notes

@scarmani

Copy link
Copy Markdown
Collaborator Author

Grok independent model review

Reviewer: grok (xai) — independent adversarial model review via the Aragora Grok reviewer, grounded on the exact PR head.
Head: 262d9b1 (262d9b1), committed 2026-06-17T17:06:56Z.
PR: #8493.
Model family: grok

Verdict: PASS

  • No blocking correctness, security or regression issues identified in the diff.
  • New overall-timeout path uses isolated processes + careful pgid/terminate/kill sequencing and always fails closed (intentional); per-reviewer override scoping via ContextVar is correct and does not leak into env or other calls.
  • Return code, JSON shape, apply gating and existing retry/evidence logic are preserved except for the documented timeout-fail case.
  • Added validation, tests and harness updates cover the new surfaces.

dogfood: yes

@scarmani

Copy link
Copy Markdown
Collaborator Author

OpenAI independent model review

Reviewer: openai (openai) — independent adversarial model review via Codex CLI OpenAI harness, grounded on the exact PR head.
Head: 262d9b1 (262d9b1), committed 2026-06-17T17:06:56Z.
PR: #8493.
Model family: openai

Verdict: PASS

No blocking issues found in the reviewed diff.

  • [P3] aragora/swarm/quorum_evidence.py:1674 overall timeout can be exceeded by up to _REVIEWER_CLEANUP_TIMEOUT per completed-but-lingering reviewer process during cleanup. This weakens the wall-clock guarantee, but fail-closed behavior and posting prevention are preserved.

dogfood: yes

@github-actions

github-actions Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Aragora Code Review

Advisory-only review. No issues found.

scarmani and others added 4 commits June 21, 2026 22:14
Co-authored-by: codex[bot] <codex[bot]@users.noreply.github.com>
Co-authored-by: codex[bot] <codex[bot]@users.noreply.github.com>
Co-authored-by: codex[bot] <codex[bot]@users.noreply.github.com>
scarmani added 3 commits June 22, 2026 23:54
…20260623T0453

# Conflicts:
#	aragora/swarm/quorum_evidence.py
# Conflicts:
#	tests/persistence/test_persistence_comprehensive.py
#	tests/ranking/test_leaderboard_engine_root.py
@scarmani

Copy link
Copy Markdown
Collaborator Author

Closing this PR as a frozen queue-drain churner rather than continuing repair loops.

Reason: this branch has repeatedly cycled through the same collect-evidence timeout/quorum surface across many repair lanes and heads, and it still has no merge-quorum authorization at head f5d4f03ac74b9f7c6e1700c55d0cf2d7af1dd60b. Per the queue-drain anti-treadmill rule, another patch on this substrate abstraction is not progress. The branch is preserved; any remaining timeout work should restart from current main as a smaller, fresh design rather than extending this PR.

@scarmani scarmani closed this Jun 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant