Skip to content

benchmarks: tooling for the post1 rerun of the paper's embedded cells (build-only mode, wheel label, one wheel per cell, embedded-only stages) - #584

Draft
tae898 wants to merge 1 commit into
mainfrom
bench/583-post1-rerun-tooling
Draft

tae898 wants to merge 1 commit into
mainfrom
bench/583-post1-rerun-tooling

Conversation

@tae898

@tae898 tae898 commented Nov 9, 2026

Copy link
Copy Markdown

Tooling for the post1 rerun of the paper's embedded cells (CAMPAIGN section 7 row 93 is the plan, #581 and PR #582 the docs row; I do not repeat it here). Draft: I am not merging anything, and nothing was run on the mini or in docker.

Closes #583

What it adds. A build-only mode (runner.py --build-only, one guarded hook after build() in each of l1_tpc.py, l2_graph.py, e2_hybrid.py, l3d_dense.py; the row carries row_kind = "build_only"), a wheel label on the rows of a stage that opts in (wheel_label, wheel_name, wheel_sha256), a per-cell choice of one wheel at landing (land_stage.py --post1-reps N), an embedded-only stage generator with a dry run, an estimate and a selftest (make_post1_rerun_stages.py), the procedure (POST1-RERUN.md) and a rollback by wheel label (post1_rows.py strip). A build-only row may replace only build_s, ingest_s, index_s, index_before_load, setup_s, gav_build_s and the load-transport facts; every timed figure of the cell stays at its 26.11.1 value, and the build window's memory figures sit beside the full row's as build_window_*.

Why the changed files are inert for the live pass. Comparator stages pull main at every start, so each edit has to be a no-op unless asked:

file change with nothing set
runner.py import post1_rows, --build-only (default off), BENCH_BUILD_ONLY forwarded only when set, post1_rows.stamp_row(row) before the row is written the import has no side effects, the flag does nothing, stamp_row returns the row untouched (no BENCH_WHEEL_LABEL, no BENCH_BUILD_ONLY)
l1_tpc.py, l2_graph.py, e2_hybrid.py, l3d_dense.py if os.environ.get("BENCH_BUILD_ONLY") == "1": and a lazy import inside it never true
land_stage.py --post1-reps (default 0), and a refusal of rows that carry row_kind == "build_only" no existing row has row_kind
engine_pin.py jar_pin(), used in the two engine-jar comparisons identity for every pin without .postN
CAMPAIGN.md one paragraph before the paper-profile paragraph (no row, so it does not collide with row 93 in #582) docs only

make_2611_stages.py is not changed (its --check still exits 0), so stage_drafts_2611.sha256 holds.

One finding. engine_pin.verify demands that the engine jar version equals the pin, and the pin of a post1 wheel is 26.11.1.post1 while its jar is 26.11.1: without jar_pin the runner refuses every post1 cell.

Dependency on #554. The native chain needs the four-arm generator of #554 (CHAIN_BUILD, the -native image tag); the generator refuses a native chain without it. I applied this branch onto the #554 head (git apply --3way): seven files apply cleanly, engine_pin.py conflicts in the two jar comparisons (same lines), and the same two-line change resolves it there. With that resolved, test_post1_rerun.py passes and a native and a JVM chain generate (44 scripts, all bash -n clean) on the #554 head too.

Checked. compile() of every changed file, pyflakes (no new warning), uv run --no-project python test_post1_rerun.py (resolve rules, the stamp, rollback, the generator: no served arm in any BACKENDS, --build-only and the wheel exports in every stage, the post1 rows file and rep state, no overlay in a build-only stage, a split cell list covers exactly the wanted cells, a drifted anchor is refused), bash -n on every generated script and on the driver, the runner's refusal of --build-only with the pass's own rows file (run as a plain argument error).

Not checked. No lane ran with the hook, no docker cell, no mini. The freeze's handling of the added columns and of two engine_version strings in one freeze (the one-version gate of the four-arm freeze) is for the landing's dry run. verify_pair_c25.sh may compare the wheel version with the jar version and was not read for .postN. The build_window_* copy matches field names by prefix (peak_, mem_, rss, cpu_, disk_, io_, heap, jvm_, gc_) and was not compared with real rows. The hours behind the dry-run estimate come from the scope note and the pass-1 build_s medians and are estimates, not measurements.

🤖 Generated with Claude Code

… (build-only mode, wheel label, one wheel per cell, embedded-only stages) (#583)

The post1 rows override the 26.11.1 rows of the cells that are rerun, all repetitions of a cell from one wheel (DECISIONS #297, #298). The harness had no
way to stop a cell after build(), rows did not say which wheel file they ran on, the generated stages mix embedded and served arms, and nothing refused a
cell with repetitions from two wheels. All of it is additive and off by default, because comparator stages pull main at every start and a mid-run change
splits rows: no variable set means no behaviour change.

- post1_rows.py (new): build-only helpers for the lanes, the opt-in wheel stamp, resolve() (one wheel per cell, problems instead of a silent mix), strip (rollback).
- runner.py: --build-only (refused unless the results file ends _post<N>.jsonl), BENCH_BUILD_ONLY forwarded when set, stamp_row() before the row is written.
- l1_tpc.py, l2_graph.py, e2_hybrid.py, l3d_dense.py: one guarded hook after build() and before any timed work.
- land_stage.py: --post1-reps N (default 0) pulls the post1 rows file and lands the resolved rows; build-only rows are refused as cells.
- engine_pin.py: an X.Y.Z.postN pin accepts the X.Y.Z engine jar (the check refused every post1 cell).
- make_post1_rerun_stages.py (new): embedded-only stages on top of make_2611_stages.stage_script, cell list, dry run with an estimate, selftest, driver.
- POST1-RERUN.md, CAMPAIGN.md paragraph, test_post1_rerun.py.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

benchmarks: tooling for the post1 rerun of the paper's embedded cells (build-only mode, wheel label, one wheel per cell, embedded-only stages)

1 participant