Reviewer consistency evidence and model-free code-change acceptance

[TheCryptoDonkey]

Summary

• Reviewer consistency from recorded reviews, no model calls (docs/experiments/reviewer-evidence-20260923/). Of the 5 verdict flips between v1 and v2 on the unchanged plain and Graphify arms, at least 3 come from the reviewer judging equivalent answers oppositely and 2 from real answer differences. Nine functionally identical code-change diffs received 8 accepts and 1 reject.
• Code-change acceptance on checker plus a deterministic scope check (docs/experiments/code-acceptance-20260923/). scope.mjs checks that changed paths stay inside the selection policy, that build and test configuration is untouched, and that no test is deleted, removed or disabled. The harness uses it only when a protocol sets "codeAcceptance": "checker-and-scope"; locked protocols replay unchanged. Re-scoring the 20 recorded code cells changes one verdict (v1 Graphify, the identical-code rejection).
• Paid re-review probe, designed and not run (docs/experiments/reviewer-agreement-20260923/): about $32 to $65 client-reported. The model-free evidence establishes the main point without it.
• REPEATED-RUNS.md adopts the new code-change acceptance rule.

Validation

• scope.mjs exercised on synthetic repositories: no change, a fix, a fix with a new test, a change outside the policy, a config change, a removed test, an added skip, a deleted test file and a renamed test file.
• rescore.mjs and analyse.mjs run against the recorded v1, v2, S5 and v3 evidence.
• summarise.mjs still renders the recorded v2 receipts unchanged.
• agree.mjs exercised end to end against a stub reviewer (no model calls).

🤖 Generated with Claude Code

https://claude.ai/code/session_01KzXbuBSq88i3Re82qt6c4L