Skip to content

Commit e09e020

Browse files
committed
docs: clarify tracked benchmark evidence
Signed-off-by: WilliamK112 <164879897+WilliamK112@users.noreply.github.com>
1 parent 17d3ab3 commit e09e020

1 file changed

Lines changed: 8 additions & 2 deletions

File tree

‎docs/community/comparison-refresh-checklist.md‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -81,8 +81,14 @@ Use this verdict rubric consistently:
8181

8282
- [ ] Put reusable competitor adapters in `benchmarks/baselines/`, not only in
8383
a pull-request comment or an uncommitted notebook.
84-
- [ ] Put machine-readable benchmark output in the existing benchmark results
85-
directory and provide a readable summary.
84+
- [ ] Commit evidence to the tracked destination for its harness:
85+
`benchmarks/cleanbench/results/` for CleanBench, the relevant accepted
86+
`baseline.json` for Gauntlet or TruthBench, or compact
87+
`benchmarks/results/performance/*-summary.json` plus `*-report.md` for the
88+
performance harness. Generic `benchmarks/results/<run>/`, Gauntlet run
89+
output, TruthBench `latest.*`, and raw performance cases are intentionally
90+
ignored; keep them as CI artifacts and link the run. Do not force-add an
91+
ignored raw result as permanent evidence.
8692
- [ ] Use committed fixtures or document how generated input is seeded.
8793
- [ ] Confirm that the comparison text matches the stored results; do not copy
8894
rounded numbers from an older run.

0 commit comments

Comments
 (0)