Skip to content

Freeze the Nauvis render parity to fixtures (#227) - #350

Merged
wormeyman merged 1 commit into
mainfrom
feat/227-freeze-nauvis-render-parity
Aug 30, 2026
Merged

Freeze the Nauvis render parity to fixtures (#227)#350
wormeyman merged 1 commit into
mainfrom
feat/227-freeze-nauvis-render-parity

Conversation

@wormeyman

Copy link
Copy Markdown
Collaborator

Freezes the first of the three render parity specs, which is a prerequisite for
the #227 deletion rather than part of it.

Why this has to land before the deletion

The render parity specs get their TypeScript arm by calling the same function
with the engine argument left off:

const wasm = new Uint8ClampedArray(runRenderRequest(req, e).buffer);
const ts   = new Uint8ClampedArray(runRenderRequest(req).buffer);

#227 deletes the branches that second line reaches. After that it is not a
weaker arm, it is the same arm - runRenderRequest would have nothing else
to be - so every byte-identity assertion in the file would pass while grading
nothing.

What this does

test/tier3Frozen.ts folds each rendered RGBA buffer to a u64 and checks it
against test/fixtures/tier3-render-checksums.json. nauvis:render holds 73
rows
covering every window, lever setting and routed view the spec already
had. Each was recorded only after the two arms were compared and agreed, so the
table cannot be wrong while both renderers exist.

The existing expect(wasm).toEqual(ts) assertions stay, so a failure still
gives a pixel-level diff rather than two hex numbers. The deletion PR drops
those lines and the ts locals; the frozen rows are what survive.

The fold runs in JavaScript, not Rust. Both arms already hand back RGBA
bytes, so folding there keeps them symmetric and adds no export - which means
this rebuilds no engine.wasm and cannot go stale against the committed binary.
The fold takes the byte length first, so a truncated buffer cannot collide with
a shorter render that shares a prefix.

Shared plumbing, separate tables

test/frozenTable.ts now holds the table machinery, and both tier2Frozen.ts
and tier3Frozen.ts are thin wrappers over makeFrozenTable. The guards there
are the load-bearing part and each was added after a specific way of writing a
wrong table, so one implementation beats two. tier2Frozen.ts keeps its exports
and its prose; all nine tier-2 consumers stay green.

Tier 3 keeps its own file because a row means a different thing - one
rendered image, not a field folded over a grid - and because
tier2Coverage.spec.ts anchors tier 2's rows to the module's own checksum_*
exports, which render rows do not have.

Planted, not predicted

plant result
corrupt one frozen row cliffs richness 0: cliffs: wasm 11549297961623709281 != frozen 16045690984833335023
delete one frozen row cliffs richness 0: cliffs: no frozen checksum - a failure, not a skip

The table was restored byte-identically afterwards.

The row-count guard fired for real. The first record run declared 73 and
recorded 60, so flushRecording dropped the section and wrote nothing rather
than committing a short table. The 13 missing rows were the four overlay lever
loops, which share an identical body and which I had skipped. That is exactly
the failure the guard exists to catch, caught.

One test deliberately NOT frozen

refuses the engine for a spawn list longer than the ABI cap. Both its arms are
the TypeScript renderer - that is its whole claim - so a frozen row would
capture an image the engine can never reproduce, and would fail the moment the
carve-out goes. It belongs to the > 8 spawn carve-out, and the deletion
removes both together. Commented in place.

Drive-by, and why it is here

Two counts had drifted and nothing asserted them: CLAUDE.md and
PROVENANCE.json both said the tier-2 table holds 942 folds. It holds 1,168

  • nauvis grew to 84x6 and the five primitives: sections were never counted.
    Corrected rather than left, because this PR adds a sibling entry directly beside
    them.

Verification

  • pnpm run verify green in 15m13s, including the committed-wasm check.
  • wasmNauvisRenderParity.spec.ts runs 37 tests green against the frozen table
    in a normal (non-recording) run.

Refs #227.

🤖 Generated with Claude Code

https://claude.ai/code/session_015W4pLtEAk7Evo14mBkQ1HN

Comment on lines +62 to +64
*/
expectRecordedRows(SECTION, 73);
afterAll(flushRecording);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

expectRecordedRows(SECTION, 73) only guards record mode — it populates a map that flushRecording reads, and flushRecording returns immediately unless FMW_FREEZE_TIER3=1 (see test/frozenTable.ts). On a normal run, nothing asserts that all 73 frozen rows are actually consulted. tier3Frozen.ts exports frozenCount for exactly this, but it's never imported here.

All three tier-2 planet specs carry this guard in assert mode, e.g. test/wasmNauvisParity.spec.ts#L922-L928, whose own comment states the purpose: "the table must cover the whole grid, so a case or field dropped from the sweep cannot shrink the frozen surface unseen." The same reasoning applies here — once #227 deletes the TypeScript arm, this frozen table is the only thing grading these renders. If a freeze(...) call site is later deleted or skipped, its row stays in tier3-render-checksums.json forever un-consumed, and the suite still goes green with no signal that coverage shrank.

Consider adding something like if (!RECORDING) expect(frozenCount(SECTION)).toBe(73); (importing frozenCount and RECORDING from ./tier3Frozen), mirroring the tier-2 pattern.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in f8341ac - thank you, this was a real gap.

I checked the precedent before acting on it: all three tier-2 planet specs do
carry the guard in assert mode, and wasmNauvisParity.spec.ts states the reason
in its own comment. This file was missing it.

I went further than the suggested one-liner, because it does not close the hole
you describe. expect(frozenCount(SECTION)).toBe(73) compares the committed
file against a literal, and when a freeze(...) call site is deleted both
sides stay 73
- the row simply goes un-consulted, which is exactly the case
your second paragraph names. Tier 2's version works because its right-hand side
is FIELD_NAMES.length * CASES.length, derived from the spec's own arrays; mine
is a flat literal spanning seven describe blocks and four inline lever tables,
so it has nothing to shrink.

So frozenTable.ts now tracks the distinct rows each run actually looks up, and
the spec asserts both numbers. They fail on opposite mistakes: the table count
catches a re-record that wrote a different surface, the consulted count catches
a call site that stopped asking.

Planted rather than reasoned. Deleting the freeze("spawn two points", ...)
call site:

× consults every frozen row exactly once
AssertionError: distinct rows this run looked up: expected 72 to be 73
Tests  1 failed | 37 passed (38)

All 37 other tests stayed green, which is the finding rather than the fix -
without this guard that deletion was invisible.

The guard is declared last so it sees every call the run made, and under -t it
is filtered out like any other test rather than failing on a partial run.

@wormeyman
wormeyman force-pushed the feat/227-freeze-nauvis-render-parity branch from d95d865 to e8ac8dd Compare August 30, 2026 00:20
The three render parity specs get their TypeScript arm by calling
`runRenderRequest` with the engine argument left off. #227 deletes those
branches, after which both arms are the SAME code - a comparison that passes
while grading nothing. This freezes the first of the three, so the deletion is a
subtraction rather than a hole.

`test/tier3Frozen.ts` folds each rendered RGBA buffer to a u64 and checks it
against `test/fixtures/tier3-render-checksums.json`, which holds 73 rows under
`nauvis:render` - every window, lever setting and routed view the spec already
covered. Each was recorded only after the two arms were compared and agreed, so
the table cannot be wrong while both renderers exist. The fold runs in
JavaScript over bytes both arms already return, so this adds no Rust export and
rebuilds no engine.wasm.

The table plumbing moves to `test/frozenTable.ts`, shared with tier 2 so the
guards live in one place rather than two; `tier2Frozen.ts` becomes a thin
wrapper over it with its exports unchanged. All nine tier-2 consumers stay green.

The spec also asserts its own COVERAGE, which the three tier-2 planet specs
already do and this file was missing. `expectRecordedRows` guards only a record
run - it feeds `flushRecording`, which returns immediately unless
FMW_FREEZE_TIER3=1 - so nothing checked that the 73 rows are consulted on a
normal run. `frozenTable.ts` now tracks the distinct rows each run looks up, and
a final test asserts both the table count and the consulted count. The two fail
on opposite mistakes: the table count catches a re-record that wrote a different
surface, the consulted count catches a call site that stopped asking. Comparing
the file against a literal alone would move with neither.

Planted rather than predicted, all four RUN:

  - a corrupted row reddens the wasm arm by name
  - a deleted row fails "no frozen checksum" rather than skipping quietly
  - a deleted `freeze` call site leaves all 37 other tests GREEN and is caught
    only by the coverage guard, at "expected 72 to be 73"
  - the row-count guard fired for real: a first record run declared 73 and
    recorded 60, and flushRecording DROPPED the section rather than committing a
    short table. The 13 missing rows were the four overlay lever loops, which
    share an identical body.

The ABI-cap test is deliberately NOT frozen. Both its arms are the TypeScript
renderer, which is its whole claim, so a frozen row would capture an image the
engine can never reproduce. It belongs to the `> 8` spawn carve-out, and the
deletion removes both together.

Also corrects two stale counts that nothing asserted: the tier-2 table holds
1,168 folds, not 942 - nauvis grew to 84x6 and the five primitives sections
were never counted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015W4pLtEAk7Evo14mBkQ1HN
@wormeyman
wormeyman force-pushed the feat/227-freeze-nauvis-render-parity branch from e8ac8dd to f8341ac Compare August 30, 2026 00:21
@wormeyman
wormeyman merged commit 993f64b into main Aug 30, 2026
11 checks passed
@wormeyman
wormeyman deleted the feat/227-freeze-nauvis-render-parity branch August 30, 2026 00:32
wormeyman added a commit that referenced this pull request Aug 30, 2026
The second of the three render parity specs, using the machinery #350 added, so
this is the conversion only.

`vulcanus:render` holds 26 rows: 4 terrain windows, the 5 routed views, 4 rocks,
5 resources, 4 composite and 4 cliffs. Each was recorded only after the Rust and
TypeScript arms were compared and agreed. The Nauvis section came through the
re-record byte-identical, which is the merge behaviour `flushRecording` is meant
to have and is worth having seen rather than assumed.

The three `renderTiled` helpers are deliberately NOT frozen, checked rather than
assumed: every one of their `runRenderRequest` calls passes the engine, so they
compare tiled against whole rather than Rust against TypeScript. The deletion
leaves them grading exactly what they grade now.

The coverage guard is here from the start rather than added after review.
Planted: deleting the terrain `freeze` call site leaves **all 16 other tests
green** and is caught only by the guard, at "expected 22 to be 26". That is the
second spec where a deleted call site was invisible to every pre-existing
assertion.


Claude-Session: https://claude.ai/code/session_015W4pLtEAk7Evo14mBkQ1HN

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant