diff --git a/evals/agent-blind-spots/runs/2026-05-29/spike-driver/summary.md b/evals/agent-blind-spots/runs/2026-05-29/spike-driver/summary.md new file mode 100644 index 0000000..9c2bc9e --- /dev/null +++ b/evals/agent-blind-spots/runs/2026-05-29/spike-driver/summary.md @@ -0,0 +1,24 @@ +# Spike driver run — summary + +- Run date: 2026-05-29 +- Cells attempted: 4 +- Cells with transcript: 4 +- Cells judged: 4 +- Cells errored: 0 + +## Cell matrix + +| Fixture | Agent | Tier | Catch | Evidence | Delta | Status | +|---|---|---|---|---|---|---| +| pr1-fix-clv | claude | 0 | catch | 0 | same | ok | +| pr1-fix-clv | claude | 1 | catch | 1c | improvement | ok | +| pr1-fix-clv | codex | 0 | miss | 0 | same | ok | +| pr1-fix-clv | codex | 1 | miss | 0 | same | ok | + +## Judge self-consistency (two passes on the same transcript) + +- **catch**: 100% (4/4 double-judged cells) +- **tier**: 100% (4/4 double-judged cells) +- **delta**: 75% (3/4 double-judged cells) + +**Stability bar:** ≥80% per axis. Below that, the judge can't replace human grading at N=6. diff --git a/evals/agent-blind-spots/runs/2026-05-29/spike-driver/verdicts.csv b/evals/agent-blind-spots/runs/2026-05-29/spike-driver/verdicts.csv new file mode 100644 index 0000000..d1187b1 --- /dev/null +++ b/evals/agent-blind-spots/runs/2026-05-29/spike-driver/verdicts.csv @@ -0,0 +1,5 @@ +fixture,agent,tier,catch,evidence_tier,delta,catch_2,tier_2,delta_2,returncode,error +pr1-fix-clv,claude,0,catch,0,same,catch,0,improvement,0, +pr1-fix-clv,claude,1,catch,1c,improvement,catch,1c,improvement,0, +pr1-fix-clv,codex,0,miss,0,same,miss,0,same,2, +pr1-fix-clv,codex,1,miss,0,same,miss,0,same,2,