Skip to content

benchmarks: halve the io figures of pre-fix mini rows in the frozen copy (#555) - #571

Merged
tae898 merged 2 commits into
mainfrom
fix/io-stat-freeze-correction
Nov 6, 2026
Merged

tae898 merged 2 commits into
mainfrom
fix/io-stat-freeze-correction

Conversation

@tae898

@tae898 tae898 commented Nov 6, 2026 •

Copy link
Copy Markdown

Refs #555. Companion to #570 (the runner fix, draft, merges SECOND, held until the qC22 and qC23 re-runs finish). This PR merges FIRST and changes nothing a running stage reads.

The recorded io_* figures of mini rows written before the harness fix are about twice their value (stacked 259:0 NVMe and 252:0 dm counted twice). This PR corrects them where the rows are frozen, never in the raw rows.

What it does: make_paper_tables.freeze_paper_rows passes the rows through io_stat_correction.correct_rows before writing runs_paper.csv. For a row whose bench_host is in io_stat_fix.json (mini) and that does not carry io_stat_dedup == 1, every io.stat-derived field is halved in the frozen copy: io_read_mib, io_write_mib, the _sum pair, {client,server,server_container,sidecar_*}_io_{read,write}_mib and the memory probe's io_<phase>_{read,write}_mib. Nothing else in a row changes except the frozen stamp, which reads halved on a corrected row. A stamped row (io_stat_dedup == 1, written by the fixed runner in #570) is never halved, a row of another host is left alone, and a stamp that is present but not 1 raises. runs.jsonl and the in-memory rows the tables are built from keep the raw values, and generated/io_stat_correction.json lists every corrected row (run_id, lane, backend, scale, rep) with its raw values.

Why a stamp and not a cutoff: rows record no harness commit (only engine_commit, the engine's), and a time cutoff would need someone to set it after the fix lands. The fixed runner says it itself, so a row without the stamp predates the fix and no value has to be set later. The earlier manifest cutoff is gone.

The new frozen column: io_stat_dedup is added to page_check.NOT_PRINTED with a reason (the check that every measured field is a column or declared) and to the ignored provenance prefixes of provenance_check's schema-homogeneity check, so a cell whose reps straddle the fix passes. I ran the fixture pipeline (freeze, export, page_check, provenance_check, fairness_check, degenerate_check, resource_check) in scratch on this branch and on the fix branch and every gate returned 0 on both; on this branch the frozen CSV carries the column.

Not touched: the stage and runner path. The stage scripts run git pull --ff-only, then cell_cost_check.py, version_pin_check.py, runner.py and the lane scripts. I re-walked the imports (AST, including function-level) from those entry points: the 44 modules they reach include none of the files this PR changes (make_paper_tables, page_check, provenance_check) or io_stat_correction. Only the freeze imports the new module.

Tests: test_io_stat_correction.py (field rule, unstamped mini row halved, stamped row untouched, malformed stamp raises, other hosts left alone, the stamp declared to both gates, and the freeze writing the halved CSV plus the raw sidecar while leaving the input rows untouched). I called the test functions directly and all nine pass; CI runs them here. compile(), pyflakes and the repo's pre-commit hooks on the changed files are clean. PUBLISHING.md names the sidecar and the rule.

🤖 Generated with Claude Code

…opy (#555)

The io.stat double count (stacked 259:0 NVMe + 252:0 dm on the mini) left every io_* figure
at 2x in rows written before the harness fix. The freeze now halves them in runs_paper.csv
for rows of a stacked-device host whose runner started before io_stat_fix.json's cutoff,
leaves runs.jsonl and the in-memory rows raw, and writes the raw values of every corrected
row to generated/io_stat_correction.json. Rows carry no harness commit, so the cutoff is the
runner's manifest stamp (null until the fixed harness has run on the host). Only the freeze
imports the new module; no stage or runner path does.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…me cutoff (#555)

A row of a stacked-device host without io_stat_dedup == 1 predates the fix and is halved in the
frozen copy (its frozen stamp reads 'halved'); a stamped row is never halved; any other stamp
raises. The manifest cutoff is gone: the stamp is written by the commit that carries the fix, so
no value has to be set later. page_check.NOT_PRINTED declares the column and provenance_check's
schema-homogeneity check ignores it as provenance, so a cell whose reps straddle the fix passes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@tae898
tae898 marked this pull request as ready for review November 6, 2026 03:28
@tae898
tae898 merged commit 6454fce into main Nov 6, 2026
4 checks passed
@tae898
tae898 deleted the fix/io-stat-freeze-correction branch November 6, 2026 03:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant