diff --git a/CLAUDE.md b/CLAUDE.md index f2624fa..a68af1d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1409,7 +1409,7 @@ measured rather than assumed: exactly 14,406, because the shape of the sweep does not depend on the host. **Tier 2's shelf life is now a FREEZE rather than a deadline.** It compared Rust -against TypeScript, and #227 deletes the TypeScript, so all 942 folds are +against TypeScript, and #227 deletes the TypeScript, so all 1,168 folds are committed to `test/fixtures/tier2-checksums.json` and each spec asserts BOTH arms against the frozen value instead of against each other. When the TypeScript arm goes, the wasm arm keeps running against a number captured while the two @@ -1432,6 +1432,60 @@ the table is for. **Read a moved number, do not adjust it.** Record with `FMW_FREEZE_TIER2=1`, then run the specs normally - a record run compares nothing and so proves nothing. +**Tier 3 now carries the same freeze, for the same reason (#227).** The render +parity specs got their TypeScript arm by calling `runRenderRequest` with the +engine argument left off, so after the deletion both arms would be the SAME +code - a comparison that passes while grading nothing. `test/tier3Frozen.ts` +freezes each render to a checksum in +`test/fixtures/tier3-render-checksums.json`; `nauvis:render` holds 73 rows. +Record with `FMW_FREEZE_TIER3=1`. + +**The plumbing is shared and the tables are not.** `test/frozenTable.ts` holds +the machinery, and both `tier2Frozen.ts` and `tier3Frozen.ts` are thin wrappers +over `makeFrozenTable`, so the guards live in one place rather than two. Tier 3 +keeps its own FILE because a row means a different thing - one rendered image, +not a field folded over a grid - and because `tier2Coverage.spec.ts` anchors +tier 2's rows to the module's own `checksum_*` exports, which render rows do +not have. + +**The tier-3 fold runs in JavaScript**, not Rust. Both arms already hand back +RGBA bytes, so folding there keeps them symmetric and adds no export - which +means freezing tier 3 rebuilds no `engine.wasm` and cannot go stale against the +committed binary. The fold takes the byte LENGTH first, so a truncated buffer +cannot collide with a shorter render that shares a prefix. + +**Each tier-3 spec asserts its own COVERAGE**, the way the three tier-2 planet +specs do. `expectRecordedRows` guards only a RECORD run - it feeds +`flushRecording`, which returns immediately unless the environment variable is +set - so without a second guard nothing checks that the rows are consulted on a +normal run, and a deleted `freeze` call site would leave its row in the table +un-consulted while every gate stayed green. `frozenTable.ts` tracks the distinct +rows each run looks up, and the spec asserts BOTH that count and the table's. +The two fail on opposite mistakes: the table count catches a re-record that +wrote a different surface, the consulted count catches a call site that stopped +asking. A literal compared only against the file would move with neither. + +Three planted breaks were RUN rather than predicted: + +- a corrupted row reddens the wasm arm by name + (`wasm 11549297961623709281 != frozen 16045690984833335023`) +- a deleted row fails "no frozen checksum" rather than skipping quietly +- **a deleted `freeze` call site leaves all 37 other tests GREEN**, and is + caught only by the coverage guard, at "expected 72 to be 73" + +The row-count guard also fired for real - a first record run declared 73 and +recorded 60, and `flushRecording` DROPPED the section rather than committing a +short table. The 13 missing rows were the four overlay lever loops, which share +an identical body. + +**One test is deliberately NOT frozen**: `refuses the engine for a spawn list +longer than the ABI cap`. Both its arms are the TypeScript renderer, which is +its whole claim, so a frozen row would capture a picture the engine can never +reproduce. It belongs to the `> 8` spawn carve-out, and the #227 deletion +removes both together. The spawn census on #227 is why that is safe: the most +starting points any exchange string in the repo carries is two, against a cap +of eight. + **Parity sweeps must use NON-binary origins and steps**, or they agree by construction. `test/wasmNauvisParity.spec.ts` freezes 2,365 of 2,420 positions off the f32 grid, with two tier-3-shaped windows asserted at 0 as the control. diff --git a/test/fixtures/PROVENANCE.json b/test/fixtures/PROVENANCE.json index b9623c3..76b2ce6 100644 --- a/test/fixtures/PROVENANCE.json +++ b/test/fixtures/PROVENANCE.json @@ -501,6 +501,7 @@ "starting-area-diff.txt": "a captured map exchange string that no test reads. The README says it pins the mid-block starting_area f32 offset, but the same 2026-08-18 grep found only README.md and three design docs, no spec. Its own payload stamps 2.1.9.3, so it can be given a real entry the moment a test starts reading it. Issue #240 flags it and seed-123456789.txt as either unasserted ground truth or dead.", "wasm-request.v2.json": "not Factorio ground truth, so it has no game version to record - it pins OUR OWN WASM request encoding at ABI v2 (#225, replacing the v1 entry from #223), whose layout tables live in crates/fmw-wasm/src/abi.rs. It is read by a spec, unlike most entries here: test/wasmFulgoraRenderParity.spec.ts asserts the TypeScript writer produces exactly these bytes for both a Fulgora and a Vulcanus request, so neither side of the boundary can move a field without the other going red. The bytes were checked by test/fixtures/verify-wasm-request.py, a third implementation written from the layout table, rather than by re-running the writer - which would only have proved the writer agrees with itself.", "verify-wasm-request.py": "a tool, not ground truth. It is the independent decoder that verified wasm-request.v2.json, committed beside it so a future ABI version can be re-verified the same way rather than by regenerating the bytes from the encoder under test. It deliberately does not reproduce the trig VALUES - those are V8's Math.sin after an f32 narrowing, and a second libm is exactly the disagreement #270 measured - so it checks the trig block structurally: unit norm per pair, one legitimate duplicate, and each bearing's angle recovered with atan2 and checked against the offset the game's Lua gives it. Seven planted breaks were confirmed caught, including two bearing swaps that the unit-norm check alone was measured MISSING. The cliff cell query box (#225, ABI v2's Vulcanus block growing 248 -> 280 bytes with no version bump) added FIVE more, all confirmed caught by its per-edge value check: a transposed x0/x1, four identical edges, a block shifted by one f64, a mismatched y0, and a declared length still saying 248. Its two structural checks - four distinct edges, not inverted on either axis - constrain the FIXTURE rather than catching a break, and the file says so rather than claiming credit for the five.", - "tier2-checksums.json": "our own tier-2 folds, not Factorio ground truth. 942 order-sensitive u64 checksums (fulgora 76x2, vulcanus 74x5, nauvis 84x5) recorded 2026-08-27 by the three wasm parity specs under FMW_FREEZE_TIER2=1, each captured only after the Rust and TypeScript arms were compared and agreed. It exists because #227 deletes the TypeScript, and tier 2 is the only grading that runs the port inside wasm32-unknown-unknown - cargo test links the host libm, so the #270 class is invisible to tier 1. Regenerate with the command in test/tier2Frozen.ts; a moved number is a finding, not a value to adjust." + "tier2-checksums.json": "our own tier-2 folds, not Factorio ground truth. 1,168 order-sensitive u64 checksums (fulgora 76x2, vulcanus 74x5, nauvis 84x6, plus 142 across the five primitives: sections) recorded by the wasm parity specs under FMW_FREEZE_TIER2=1, each captured only after the Rust and TypeScript arms were compared and agreed. It exists because #227 deletes the TypeScript, and tier 2 is the only grading that runs the port inside wasm32-unknown-unknown - cargo test links the host libm, so the #270 class is invisible to tier 1. Regenerate with the command in test/tier2Frozen.ts; a moved number is a finding, not a value to adjust.", + "tier3-render-checksums.json": "our own tier-3 render folds, not Factorio ground truth. 73 u64 FNV-1a checksums over rendered RGBA buffers, all under nauvis:render, recorded 2026-08-29 by test/wasmNauvisRenderParity.spec.ts under FMW_FREEZE_TIER3=1, each captured only after the Rust and TypeScript arms were compared and agreed. It exists because #227 deletes the TypeScript, and the render specs got their reference arm by calling runRenderRequest with the engine argument left off - after the deletion both arms would be the same code, so the comparison would pass while grading nothing. The fold runs in JavaScript over bytes both arms already return, so freezing this rebuilds no engine.wasm. Regenerate with the command in test/tier3Frozen.ts; a moved number is a finding, not a value to adjust." } } diff --git a/test/fixtures/tier3-render-checksums.json b/test/fixtures/tier3-render-checksums.json new file mode 100644 index 0000000..2b58e5c --- /dev/null +++ b/test/fixtures/tier3-render-checksums.json @@ -0,0 +1,77 @@ +{ + "nauvis:render": { + "all six, far | all": "0xa20d3bd22615320e", + "all six, far | resources": "0x233dc5950c87e4ee", + "cliffControls.continuity | lever": "0xcb688576d19eaec2", + "cliffControls.frequency | lever": "0xc3f92237f5cf22b5", + "cliffSettings.cliffElevation0 | lever": "0x2a53c2d5d33e12d6", + "cliffSettings.cliffElevationInterval | lever": "0xd7b1232e87011a86", + "cliffs base | cliffs": "0x58bf7c68cd5d62bc", + "cliffs richness 0 | cliffs": "0xa047594e27b96661", + "climate auxBias | terrain": "0xc520433eb15510d3", + "climate auxFrequency | terrain": "0xdac04895d7dd5410", + "climate base | terrain": "0xe88da401014f0bc4", + "climate moistureBias | terrain": "0x6cd287a690f83fbd", + "climate moistureFrequency | terrain": "0xb2efe3f5a88dbc26", + "climate segmentationMultiplier | terrain": "0x0a83c4096198974f", + "climate startingAreaMoistureFrequency alone | terrain": "0xe88da401014f0bc4", + "climate startingAreaMoistureSize | terrain": "0x36e0ab32ce5b4829", + "climate startingAreaMoistureSize+startingAreaMoistureFrequency | terrain": "0x98639039fd819c80", + "coarse at spawn | rocks": "0xdc2a9d583465bcaf", + "coarse at spawn | terrain": "0x1200f3eb03dc5884", + "coarse at spawn | trees": "0x4ff16ceefbe15e25", + "coarse, far field | rocks": "0x267a993c4b702bb1", + "coarse, far field | terrain": "0x36c766c310b3ab64", + "coarse, far field | trees": "0xf380ac5a6ed25a00", + "enemies base | enemies": "0xb4af851f064f3d6f", + "enemyControls.frequency | lever": "0xac009b4b6fd28f7e", + "enemyControls.size | lever": "0xf90752231a916b14", + "far north-east | enemies": "0x65bc252d2a60a1dc", + "far | cliffs": "0x157a5f5be6bd3e9d", + "fine at spawn | resources": "0xf3471077eb75d374", + "fine | cliffs": "0x67ba794e6b0f3b49", + "fine, far | enemies": "0x05963e1ebb000313", + "north-east | all": "0xf59cc0824bc17196", + "north-east | resources": "0x2b12c67dc87337ce", + "resources base | resources": "0x7f29bce32feb86b5", + "resources copper-ore levers | resources": "0x0d25dae652e32b7a", + "resources iron-ore levers | resources": "0xc6e34461ffe626d3", + "rockControls.frequency | lever": "0x1a36a2fb24fe4467", + "rockControls.size | lever": "0xccef29139bf6d146", + "rocks base | rocks": "0x2cb1d817b2b604d4", + "routes very far | all": "0xd35416a48907be1c", + "routes very far | cliffs": "0x58bf7c68cd5d62bc", + "routes very far | enemies": "0xb4af851f064f3d6f", + "routes very far | resources": "0xaa97cb23d994d34f", + "routes very far | rocks": "0x3bcbbf3bb4c2151b", + "routes very far | terrain": "0xa047594e27b96661", + "routes very far | trees": "0xba8131f1518f67a1", + "spawn moved | terrain": "0x5d9c3c6ed25d45ec", + "spawn two points | terrain": "0x44f3674eb7ff3c3e", + "tall | cliffs": "0xda88c9ed971b42fd", + "tall | enemies": "0x23006e905d6076da", + "tall, coarse | rocks": "0x4bfd520666b659b7", + "tall, coarse | terrain": "0x4052b3042748e53b", + "tall, coarse | trees": "0x60b323fbe78920cf", + "tall, far | resources": "0x0e29aab0138a270d", + "temperatureBias | lever": "0x0c97ad1f0e3d3dd9", + "temperatureFrequency | lever": "0x77208f99cd333ade", + "treeControls.frequency | lever": "0xbd486343234dafdf", + "treeControls.size | lever": "0xc4b2f6ca1b7e780c", + "trees base | trees": "0x69d33b5c55d6d7a5", + "very far | all": "0xd35416a48907be1c", + "very far | cliffs": "0x58bf7c68cd5d62bc", + "very far, coarse | enemies": "0xb4af851f064f3d6f", + "waterLevel | lever": "0xb0027d246c2c82ba", + "wide at spawn | all": "0xec30157cedf818ca", + "wide at spawn | resources": "0x7f29bce32feb86b5", + "wide at spawn | rocks": "0x2cb1d817b2b604d4", + "wide at spawn | trees": "0x69d33b5c55d6d7a5", + "wide | cliffs": "0x58b621d56b68617f", + "wide | enemies": "0xbf9a192bda5499f3", + "wide, far | resources": "0x003c244bc7e2efae", + "wide, offset | rocks": "0x9d67c26e606466f2", + "wide, offset | terrain": "0x4babfc28a8b1e8c2", + "wide, offset | trees": "0x98d9f0592e35f2af" + } +} diff --git a/test/frozenTable.ts b/test/frozenTable.ts new file mode 100644 index 0000000..60b1cd3 --- /dev/null +++ b/test/frozenTable.ts @@ -0,0 +1,158 @@ +import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs"; +import { dirname } from "node:path"; + +/** + * The table machinery behind the frozen checksum tiers, in one place. + * + * Extracted when tier 3 needed the same guards tier 2 already had. Both tiers + * freeze a `(section, label, field) -> u64` table, both record through the code + * that asserts it, and both need the partial-run protection that + * `flushRecording` provides. Two copies of that would have been two places for + * a guard to rot, and the guards here are the load-bearing part - each one was + * added after a specific way of writing a wrong table. + * + * What differs between tiers is only the file, the environment variable and + * what a row means. Those are the arguments; everything else is shared. + * + * `section` is the grouping key - a planet for tier 2, a render surface for + * tier 3. It is deliberately not called "planet" here, because tier 3 groups by + * something else and a name that lies is worse than a general one. + */ +export interface FrozenTable { + /** True when the tier's environment variable asks for a record run. */ + readonly RECORDING: boolean; + frozen(this: void, section: string, label: string, field: string): bigint | undefined; + record(this: void, section: string, label: string, field: string, value: bigint): void; + frozenNames(this: void): [string, string[]][]; + frozenCount(this: void, section: string): number; + /** + * Distinct rows this run has actually looked up, per section. + * + * `frozenCount` reads the committed FILE and `expectRecordedRows` only guards + * a record run, so neither notices a `freeze` call site that was deleted: its + * row sits in the table un-consulted and every gate stays green. That gap + * matters most after #227, when the table is the only thing grading. + */ + consultedCount(this: void, section: string): number; + expectFrozen( + this: void, + section: string, + label: string, + name: string, + wasm: bigint, + reference?: bigint, + ): void; + expectRecordedRows(this: void, section: string, rows: number): void; + flushRecording(this: void): void; +} + +type Table = Record>; + +/** + * u64 as a zero-padded hex string. JSON has no integer wide enough to hold one + * of these exactly - `2n ** 64n` is far past `Number.MAX_SAFE_INTEGER`, so a + * JSON number would silently round and the table would freeze the wrong value. + */ +function toHex(v: bigint): string { + return `0x${v.toString(16).padStart(16, "0")}`; +} + +export function makeFrozenTable(opts: { tablePath: string; envVar: string }): FrozenTable { + const TABLE_PATH = opts.tablePath; + const RECORDING = process.env[opts.envVar] === "1"; + + function load(): Table { + if (!existsSync(TABLE_PATH)) return {}; + return JSON.parse(readFileSync(TABLE_PATH, "utf8")) as Table; + } + + const table: Table = RECORDING ? {} : load(); + const recorded: Table = {}; + const expected = new Map(); + const consulted = new Map>(); + let dirty = false; + + function frozen(section: string, label: string, field: string): bigint | undefined { + const hex = table[section]?.[`${label} | ${field}`]; + return hex === undefined ? undefined : BigInt(hex); + } + + function record(section: string, label: string, field: string, value: bigint): void { + (recorded[section] ??= {})[`${label} | ${field}`] = toHex(value); + dirty = true; + } + + return { + RECORDING, + frozen, + record, + + frozenNames(): [string, string[]][] { + return Object.entries(table).map(([section, rows]) => [section, Object.keys(rows)]); + }, + + frozenCount(section: string): number { + return Object.keys(table[section] ?? {}).length; + }, + + consultedCount(section: string): number { + return consulted.get(section)?.size ?? 0; + }, + + expectFrozen(section, label, name, wasm, reference): void { + const where = `${label}: ${name}`; + if (RECORDING) { + if (reference !== undefined && wasm !== reference) { + throw new Error(`${where}: refusing to record, arms disagree - ${wasm} vs ${reference}`); + } + record(section, label, name, wasm); + return; + } + // Counted BEFORE the lookup, so a row that is consulted and disagrees + // still counts as covered - this tracks reach, not success. + let seen = consulted.get(section); + if (seen === undefined) { + seen = new Set(); + consulted.set(section, seen); + } + seen.add(`${label} | ${name}`); + const want = frozen(section, label, name); + if (want === undefined) throw new Error(`${where}: no frozen checksum`); + if (wasm !== want) throw new Error(`${where}: wasm ${wasm} != frozen ${want}`); + if (reference !== undefined && reference !== want) { + throw new Error(`${where}: reference ${reference} != frozen ${want}`); + } + }, + + expectRecordedRows(section: string, rows: number): void { + const already = expected.get(section); + if (already !== undefined) { + throw new Error( + `${section}: row total declared twice (${already} then ${rows}) - declare it once, ` + + `at module scope, as the whole section's total`, + ); + } + expected.set(section, rows); + }, + + flushRecording(): void { + if (!RECORDING || !dirty) return; + for (const [section, rows] of Object.entries(recorded)) { + const want = expected.get(section); + if (want === undefined || Object.keys(rows).length !== want) delete recorded[section]; + } + if (Object.keys(recorded).length === 0) return; + const merged: Table = load(); + for (const [section, rows] of Object.entries(recorded)) merged[section] = rows; + const ordered: Table = {}; + for (const section of Object.keys(merged).sort()) { + const rows = merged[section] as Record; + const sorted: Record = {}; + for (const k of Object.keys(rows).sort()) sorted[k] = rows[k] as string; + ordered[section] = sorted; + } + mkdirSync(dirname(TABLE_PATH), { recursive: true }); + writeFileSync(TABLE_PATH, `${JSON.stringify(ordered, null, 2)}\n`); + }, + }; +} diff --git a/test/tier2Frozen.ts b/test/tier2Frozen.ts index a491f90..af54942 100644 --- a/test/tier2Frozen.ts +++ b/test/tier2Frozen.ts @@ -1,5 +1,6 @@ -import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs"; -import { dirname, join } from "node:path"; +import { join } from "node:path"; + +import { makeFrozenTable } from "./frozenTable"; /** * Frozen tier-2 checksums, so the cutover in #227 does not take the wasm libm @@ -33,6 +34,11 @@ import { dirname, join } from "node:path"; * in this repo carries. A legitimate change (fixing #261, say) re-freezes * deliberately and says so; anything else is a regression. * + * **The table plumbing lives in `frozenTable.ts`**, shared with the tier-3 + * render freeze, which needs the same guards for the same reasons. What is + * tier-2 specific is the file, the environment variable, and the reasoning + * above. + * * ## Recording * * The table is generated by the code that asserts it, so a hand-edit cannot @@ -47,50 +53,22 @@ import { dirname, join } from "node:path"; * the file is rewritten on exit. Run the specs normally afterwards - a record * run proves nothing, because in it nothing is compared. */ - -const TABLE_PATH = join(import.meta.dirname, "fixtures", "tier2-checksums.json"); +const TABLE = makeFrozenTable({ + tablePath: join(import.meta.dirname, "fixtures", "tier2-checksums.json"), + envVar: "FMW_FREEZE_TIER2", +}); /** Set to record rather than assert. See the module comment. */ -export const RECORDING = process.env.FMW_FREEZE_TIER2 === "1"; - -type Table = Record>; - -function load(): Table { - if (!existsSync(TABLE_PATH)) return {}; - return JSON.parse(readFileSync(TABLE_PATH, "utf8")) as Table; -} - -const table: Table = RECORDING ? {} : load(); -const recorded: Table = {}; - -/** - * u64 as a zero-padded hex string. JSON has no integer wide enough to hold one - * of these exactly - `2n ** 64n` is far past `Number.MAX_SAFE_INTEGER`, so a - * JSON number would silently round and the table would freeze the wrong value. - */ -function toHex(v: bigint): string { - return `0x${v.toString(16).padStart(16, "0")}`; -} - -let dirty = false; +export const RECORDING = TABLE.RECORDING; /** * The frozen value for one (planet, case, field), or `undefined` when the table * has no entry - which a caller must treat as a failure, never as a pass. */ -export function frozen(planet: string, label: string, field: string): bigint | undefined { - const hex = table[planet]?.[`${label} | ${field}`]; - return hex === undefined ? undefined : BigInt(hex); -} +export const frozen = TABLE.frozen; -/** - * Record one checksum under record mode. Returns true when recording, so a - * caller can skip its assertions. - */ -export function record(planet: string, label: string, field: string, value: bigint): void { - (recorded[planet] ??= {})[`${label} | ${field}`] = toHex(value); - dirty = true; -} +/** Record one checksum under record mode. */ +export const record = TABLE.record; /** * Every frozen row name, grouped by planet. @@ -100,14 +78,10 @@ export function record(planet: string, label: string, field: string, value: bigi * case list is the point: a check sourced from the same array it validates can * only catch a deletion, never an omission. */ -export function frozenNames(): [string, string[]][] { - return Object.entries(table).map(([planet, rows]) => [planet, Object.keys(rows)]); -} +export const frozenNames = TABLE.frozenNames; /** Number of entries the table holds for one planet. */ -export function frozenCount(planet: string): number { - return Object.keys(table[planet] ?? {}).length; -} +export const frozenCount = TABLE.frozenCount; /** * Assert one fold against its frozen value, and against the reference arm while @@ -129,28 +103,7 @@ export function frozenCount(planet: string): number { * `eval_math` range entirely below its clamp, a seed sweep that never wrapped. * Those specs' constants ARE the coverage. Carry them; do not reinvent them. */ -export function expectFrozen( - planet: string, - label: string, - name: string, - wasm: bigint, - reference?: bigint, -): void { - const where = `${label}: ${name}`; - if (RECORDING) { - if (reference !== undefined && wasm !== reference) { - throw new Error(`${where}: refusing to record, arms disagree - ${wasm} vs ${reference}`); - } - record(planet, label, name, wasm); - return; - } - const want = frozen(planet, label, name); - if (want === undefined) throw new Error(`${where}: no frozen checksum`); - if (wasm !== want) throw new Error(`${where}: wasm ${wasm} != frozen ${want}`); - if (reference !== undefined && reference !== want) { - throw new Error(`${where}: reference ${reference} != frozen ${want}`); - } -} +export const expectFrozen = TABLE.expectFrozen; /** * Declare how many rows this planet's sweep must record before the table is @@ -188,18 +141,7 @@ export function expectFrozen( * `FIELD_NAMES` is already checked against the module's own * `_field_count()` export, so the derived form inherits that check. */ -export function expectRecordedRows(planet: string, rows: number): void { - const already = expected.get(planet); - if (already !== undefined) { - throw new Error( - `${planet}: row total declared twice (${already} then ${rows}) - declare it once, ` + - `at module scope, as the whole planet's total`, - ); - } - expected.set(planet, rows); -} - -const expected = new Map(); +export const expectRecordedRows = TABLE.expectRecordedRows; /** * Merge this run's recordings into the committed table and write it out. Called @@ -209,22 +151,4 @@ const expected = new Map(); * A planet that recorded fewer rows than it declared is DROPPED rather than * written, so a run that threw part-way leaves the committed table untouched. */ -export function flushRecording(): void { - if (!RECORDING || !dirty) return; - for (const [planet, rows] of Object.entries(recorded)) { - const want = expected.get(planet); - if (want === undefined || Object.keys(rows).length !== want) delete recorded[planet]; - } - if (Object.keys(recorded).length === 0) return; - const merged: Table = load(); - for (const [planet, rows] of Object.entries(recorded)) merged[planet] = rows; - const ordered: Table = {}; - for (const planet of Object.keys(merged).sort()) { - const rows = merged[planet] as Record; - const sorted: Record = {}; - for (const k of Object.keys(rows).sort()) sorted[k] = rows[k] as string; - ordered[planet] = sorted; - } - mkdirSync(dirname(TABLE_PATH), { recursive: true }); - writeFileSync(TABLE_PATH, `${JSON.stringify(ordered, null, 2)}\n`); -} +export const flushRecording = TABLE.flushRecording; diff --git a/test/tier3Frozen.ts b/test/tier3Frozen.ts new file mode 100644 index 0000000..e610fcf --- /dev/null +++ b/test/tier3Frozen.ts @@ -0,0 +1,148 @@ +import { join } from "node:path"; + +import { makeFrozenTable } from "./frozenTable"; + +/** + * Frozen tier-3 render checksums, so the cutover in #227 does not take the + * render parity specs out of the gate with it. + * + * Tier 3 grades a rendered image from the Rust engine against the same image + * from TypeScript, and it gets the TypeScript arm by calling the same function + * with the engine argument left off: + * + * ```ts + * const wasm = new Uint8ClampedArray(runRenderRequest(req, e).buffer); + * const ts = new Uint8ClampedArray(runRenderRequest(req).buffer); + * ``` + * + * #227 deletes the TypeScript branches that second line reaches. After that it + * is not a weaker arm, it is the SAME arm - `runRenderRequest` would have + * nothing else to be - so the comparison would pass while grading nothing. That + * is the failure mode this repo cares most about, and the reason the freeze has + * to land BEFORE the deletion rather than with it. + * + * The conversion is the one `tier2Frozen.ts` describes, applied to pixels: + * `expect(wasm).toEqual(ts)` becomes + * `expectFrozen(section, label, name, foldPixels(wasm), foldPixels(ts))`. + * While both renderers exist all three agree, so the table cannot be wrong. + * When the TypeScript arm goes the call drops its last argument and the wasm + * arm keeps running against a value captured while the two demonstrably agreed. + * + * **Read a moved number, do not adjust it.** A moved render checksum is a + * finding, exactly like a moved count. Re-freeze deliberately and say why. + * + * ## Three reasons this is its own table rather than rows in tier 2's + * + * 1. **A row means something different.** Tier 2 folds a field over a grid of + * coordinates; tier 3 folds one rendered image. Mixing them puts two units + * in one file under one name. + * 2. **`tier2Coverage.spec.ts` anchors tier 2's rows to the module's own + * `checksum_*` exports**, and asserts every such export has a row. Render + * rows have no export - see below - so they would have to be special-cased + * into a guard whose entire value is that it enumerates rather than lists. + * 3. **They re-record separately.** Recording tier 2 is cheap; recording tier 3 + * renders images, and the Nauvis sweep alone runs for minutes. + * + * ## The fold runs in JavaScript, and that is deliberate + * + * Tier 2's checksums come from `checksum_*` exports, because the Rust side is + * the only place that can walk a field. Tier 3 has no such need: both arms hand + * back RGBA bytes to JavaScript already, so folding here keeps the two arms + * symmetric - the same function over the same bytes - and adds no Rust export. + * + * That also means **freezing tier 3 rebuilds no `engine.wasm`**, so this cannot + * go stale against the committed binary the way a Rust change does. + * + * ## Recording + * + * ```bash + * FMW_FREEZE_TIER3=1 pnpm vp test test/wasmNauvisRenderParity.spec.ts + * ``` + * + * In record mode every `expectFrozen` call records instead of asserting, and + * the file is rewritten on exit. Run the spec normally afterwards - a record + * run proves nothing, because in it nothing is compared. + */ +const TABLE = makeFrozenTable({ + tablePath: join(import.meta.dirname, "fixtures", "tier3-render-checksums.json"), + envVar: "FMW_FREEZE_TIER3", +}); + +/** Set to record rather than assert. See the module comment. */ +export const RECORDING = TABLE.RECORDING; + +/** The frozen value for one (section, case, name), or `undefined` when absent. */ +export const frozen = TABLE.frozen; + +/** Number of entries the table holds for one section. */ +export const frozenCount = TABLE.frozenCount; + +/** + * Distinct rows this run actually looked up, for the coverage guard each + * tier-3 spec carries. See `frozenTable.ts` for why `frozenCount` alone is not + * enough. + */ +export const consultedCount = TABLE.consultedCount; + +/** Every frozen row name, grouped by section. */ +export const frozenNames = TABLE.frozenNames; + +/** + * Assert one rendered image against its frozen checksum, and against the + * TypeScript arm while that arm still exists. See the module comment. + */ +export const expectFrozen = TABLE.expectFrozen; + +/** + * Declare how many rows this section must record before the table is rewritten. + * Call it ONCE per section, at module scope. + * + * Carries the same guard tier 2 documents at length: without it, a record run + * that throws part-way rewrites the table with only the rows that ran, and the + * next normal run then fails on a missing entry that reads like a second, + * unrelated break. A section that records fewer rows than it declared is + * dropped rather than written. + */ +export const expectRecordedRows = TABLE.expectRecordedRows; + +/** Merge this run's recordings into the committed table and write it out. */ +export const flushRecording = TABLE.flushRecording; + +const FNV_OFFSET_BASIS = 0xcbf29ce484222325n; +const FNV_PRIME = 0x100000001b3n; +const MASK64 = (1n << 64n) - 1n; + +/** + * FNV-1a over a rendered image's raw RGBA bytes. + * + * **The byte length is folded in first**, so a truncated buffer cannot collide + * with a shorter render that happens to share a prefix. The specs assert the + * expected length separately, and this is belt as well as braces: a fold whose + * only input is content would let a render that returned half an image pass on + * a day someone dropped the length assertion. + * + * Deliberately NOT `fold_f64`'s sibling in the Rust checksum module. These are + * bytes rather than doubles, and nothing on the Rust side ever computes this - + * see the module comment on why the fold runs here. + */ +export function foldPixels(pixels: ArrayLike): bigint { + let hash = FNV_OFFSET_BASIS; + const fold = (byte: number): void => { + hash ^= BigInt(byte); + hash = (hash * FNV_PRIME) & MASK64; + }; + // Length first, little-endian over four bytes. A render is far below 2^32 + // pixels, so four bytes cannot truncate a real one. + const n = pixels.length; + fold(n & 0xff); + fold((n >>> 8) & 0xff); + fold((n >>> 16) & 0xff); + fold((n >>> 24) & 0xff); + for (let i = 0; i < n; i++) fold(pixels[i] as number); + return hash; +} + +/** `foldPixels` over the buffer an `ElevationRenderResult` hands back. */ +export function foldBuffer(buffer: ArrayBufferLike): bigint { + return foldPixels(new Uint8ClampedArray(buffer)); +} diff --git a/test/wasmNauvisRenderParity.spec.ts b/test/wasmNauvisRenderParity.spec.ts index 2710fce..d192d03 100644 --- a/test/wasmNauvisRenderParity.spec.ts +++ b/test/wasmNauvisRenderParity.spec.ts @@ -1,9 +1,18 @@ import { readFileSync } from "node:fs"; import { join } from "node:path"; import { inflateSync } from "node:zlib"; -import { describe, expect, it } from "vite-plus/test"; +import { afterAll, describe, expect, it } from "vite-plus/test"; import { withDiffArtifacts } from "./diffArtifacts"; +import { + RECORDING, + consultedCount, + expectFrozen, + expectRecordedRows, + flushRecording, + foldPixels, + frozenCount, +} from "./tier3Frozen"; import { decodePng } from "./oracle/decodePng"; import { compileEngine, instantiateEngine, renderThroughWasm } from "../src/noise/wasm/engine"; import { @@ -37,6 +46,44 @@ const FIXTURES = join(import.meta.dirname, "fixtures"); const SIZE = 1024; const SEED = 123456; +/** + * The tier-3 freeze section for this spec. See `tier3Frozen.ts`. + * + * Every Rust-against-TypeScript render below is ALSO checked against a frozen + * checksum, so the assertion survives #227 deleting the TypeScript arm. While + * both renderers exist all three agree; afterwards the wasm arm keeps running + * against a value captured while the two demonstrably agreed. + */ +const SECTION = "nauvis:render"; + +/** + * Rows this spec must record before the table is rewritten, as a literal. + * + * A literal rather than a derived total because the rows span seven describe + * blocks and several come from inline lever tables, so there is no single array + * to derive it from. That buys the guard tier 2 documents: adding a case makes + * a record run DROP the section until this number is updated, rather than + * quietly writing a shorter table. + * + * 15 terrain (4 windows, 9 climate, 2 spawn) + 10 trees + 8 rocks + 8 enemies + * + 12 cliffs + 9 resources + 11 composite. + */ +const ROWS = 73; + +expectRecordedRows(SECTION, ROWS); +afterAll(flushRecording); + +/** + * Freeze one render, and compare the two arms while both exist. + * + * The label/name pair is the row key, so it has to be unique across the whole + * section - which is why the composite block prefixes its per-view rows with + * `routes` rather than reusing a bare window label. + */ +function freeze(label: string, name: string, wasm: ArrayLike, ts: ArrayLike): void { + expectFrozen(SECTION, label, name, foldPixels(wasm), foldPixels(ts)); +} + /** * The game draws enemy bases in its capture and our terrain view does not. * @@ -208,6 +255,7 @@ describe("the WASM engine renders Nauvis terrain exactly as the TypeScript does" const ts = new Uint8ClampedArray(runRenderRequest(req).buffer); expect(wasm.length, `${w.label}: length`).toBe(w.width * w.height * 4); expect(Array.from(wasm), `${w.label}: pixels`).toEqual(Array.from(ts)); + freeze(w.label, "terrain", wasm, ts); } }, 300000); @@ -253,7 +301,9 @@ describe("the WASM engine renders Nauvis terrain exactly as the TypeScript does" const flat = (r: ElevationRenderRequest, eng?: typeof e) => Array.from(new Uint8ClampedArray(runRenderRequest(r, eng).buffer)); const baseWasm = flat(base, e); - expect(flat(base)).toEqual(baseWasm); + const baseTs = flat(base); + expect(baseTs).toEqual(baseWasm); + freeze("climate base", "terrain", baseWasm, baseTs); const moved: Partial[] = [ { segmentationMultiplier: 2 }, @@ -270,8 +320,14 @@ describe("the WASM engine renders Nauvis terrain exactly as the TypeScript does" const w = flat(req, e); const t = flat(req); const name = Object.keys(patch)[0]; + // JOINED rather than `[0]`, because the last two patches share a first + // key - `startingAreaMoistureSize` alone, then the same key with the + // frequency moved with it. A row name taken from `[0]` would collide, and + // the section would silently record six rows where seven ran. + const rowName = Object.keys(patch).join("+"); expect(w, `${name}: wasm vs ts`).toEqual(t); expect(w, `${name}: must actually move the render`).not.toEqual(baseWasm); + freeze(`climate ${rowName}`, "terrain", w, t); } // **`startingAreaMoistureFrequency` alone is INERT, and that is a property @@ -285,10 +341,11 @@ describe("the WASM engine renders Nauvis terrain exactly as the TypeScript does" // live on its own fails here instead of silently widening what the sweep // covers. const freqOnly = { ...base, startingAreaMoistureFrequency: 3 } as ElevationRenderRequest; - expect(flat(freqOnly, e), "frequency alone must stay inert at the default size").toEqual( - baseWasm, - ); - expect(flat(freqOnly)).toEqual(baseWasm); + const freqOnlyWasm = flat(freqOnly, e); + expect(freqOnlyWasm, "frequency alone must stay inert at the default size").toEqual(baseWasm); + const freqOnlyTs = flat(freqOnly); + expect(freqOnlyTs).toEqual(baseWasm); + freeze("climate startingAreaMoistureFrequency alone", "terrain", freqOnlyWasm, freqOnlyTs); }, 300000); it("a moved spawn renders THROUGH the engine, byte-identical to the TypeScript", async () => { @@ -311,6 +368,7 @@ describe("the WASM engine renders Nauvis terrain exactly as the TypeScript does" const withEngine = Array.from(new Uint8ClampedArray(runRenderRequest(moved, e).buffer)); const withoutEngine = Array.from(new Uint8ClampedArray(runRenderRequest(moved).buffer)); expect(withEngine).toEqual(withoutEngine); + freeze("spawn moved", "terrain", withEngine, withoutEngine); // Anti-vacuity, and it is what makes the equality above mean something: the // spawn has to actually move the render. If it did not, a module that @@ -330,12 +388,11 @@ describe("the WASM engine renders Nauvis terrain exactly as the TypeScript does" { x: -300.5, y: 96.25 }, ], }; - expect(Array.from(new Uint8ClampedArray(runRenderRequest(two, e).buffer))).toEqual( - Array.from(new Uint8ClampedArray(runRenderRequest(two).buffer)), - ); - expect(Array.from(new Uint8ClampedArray(runRenderRequest(two, e).buffer))).not.toEqual( - withEngine, - ); + const twoWasm = Array.from(new Uint8ClampedArray(runRenderRequest(two, e).buffer)); + const twoTs = Array.from(new Uint8ClampedArray(runRenderRequest(two).buffer)); + expect(twoWasm).toEqual(twoTs); + expect(twoWasm).not.toEqual(withEngine); + freeze("spawn two points", "terrain", twoWasm, twoTs); }, 300000); it("refuses the engine for a spawn list longer than the ABI cap", async () => { @@ -343,6 +400,13 @@ describe("the WASM engine renders Nauvis terrain exactly as the TypeScript does" // instead of silently dropping points, so `runRenderRequest` has to keep // such a request on the TypeScript path. Nine points, one past the eight // the block holds. + // + // **Deliberately NOT frozen.** Both arms here are the TypeScript renderer - + // that is the whole claim - so a frozen row would capture a picture the + // engine can never reproduce, and would fail the moment the carve-out goes. + // This test belongs to the carve-out, and the #227 deletion removes both + // together. The spawn census on that issue is why removing it is safe: the + // most starting points any exchange string in the repo carries is two. const e = await engine(); const many = { ...request(WINDOWS[0]), @@ -420,6 +484,7 @@ describe("the WASM engine renders the Nauvis tree overlay exactly as the TypeScr const ts = new Uint8ClampedArray(runRenderRequest(treeRequest(w)).buffer); expect(wasm.length, `${w.label}: length`).toBe(w.width * w.height * 4); expect(Array.from(wasm), `${w.label}: pixels`).toEqual(Array.from(ts)); + freeze(w.label, "trees", wasm, ts); } }, 300000); @@ -470,7 +535,9 @@ describe("the WASM engine renders the Nauvis tree overlay exactly as the TypeScr const flat = (req: ElevationRenderRequest, eng?: typeof e): number[] => Array.from(new Uint8ClampedArray(runRenderRequest(req, eng).buffer)); const baseWasm = flat(base, e); - expect(flat(base)).toEqual(baseWasm); + const baseTs = flat(base); + expect(baseTs).toEqual(baseWasm); + freeze("trees base", "trees", baseWasm, baseTs); const patches: readonly (readonly [string, Partial])[] = [ ["treeControls.frequency", { treeControls: { frequency: 3, size: 1 } }], @@ -481,8 +548,13 @@ describe("the WASM engine renders the Nauvis tree overlay exactly as the TypeScr for (const [label, patch] of patches) { const req = { ...base, ...patch } as ElevationRenderRequest; const moved = flat(req, e); - expect(moved, `${label}: the two paths must agree`).toEqual(flat(req)); + const movedTs = flat(req); + expect(moved, `${label}: the two paths must agree`).toEqual(movedTs); expect(moved, `${label}: must actually move the render`).not.toEqual(baseWasm); + // `label` alone is the row key across all four overlay blocks, because + // every lever name is already qualified - treeControls, rockControls, + // enemyControls, cliffControls, cliffSettings, waterLevel. + freeze(label, "lever", moved, movedTs); } }, 300000); @@ -562,6 +634,7 @@ describe("the WASM engine renders the Nauvis rock overlay exactly as the TypeScr const ts = new Uint8ClampedArray(runRenderRequest(rockRequest(w)).buffer); expect(wasm.length, `${w.label}: length`).toBe(w.width * w.height * 4); expect(Array.from(wasm), `${w.label}: pixels`).toEqual(Array.from(ts)); + freeze(w.label, "rocks", wasm, ts); } }, 300000); @@ -607,15 +680,22 @@ describe("the WASM engine renders the Nauvis rock overlay exactly as the TypeScr const flat = (req: ElevationRenderRequest, eng?: typeof e): number[] => Array.from(new Uint8ClampedArray(runRenderRequest(req, eng).buffer)); const baseWasm = flat(base, e); - expect(flat(base)).toEqual(baseWasm); + const baseTs = flat(base); + expect(baseTs).toEqual(baseWasm); + freeze("rocks base", "rocks", baseWasm, baseTs); for (const [label, patch] of [ ["rockControls.frequency", { rockControls: { frequency: 3, size: 1 } }], ["rockControls.size", { rockControls: { frequency: 1, size: 3 } }], ] as const) { const req = { ...base, ...patch } as ElevationRenderRequest; const moved = flat(req, e); - expect(moved, `${label}: the two paths must agree`).toEqual(flat(req)); + const movedTs = flat(req); + expect(moved, `${label}: the two paths must agree`).toEqual(movedTs); expect(moved, `${label}: must actually move the render`).not.toEqual(baseWasm); + // `label` alone is the row key across all four overlay blocks, because + // every lever name is already qualified - treeControls, rockControls, + // enemyControls, cliffControls, cliffSettings, waterLevel. + freeze(label, "lever", moved, movedTs); } }, 300000); @@ -746,6 +826,7 @@ describe("the WASM engine renders the Nauvis enemy overlay exactly as the TypeSc const ts = new Uint8ClampedArray(runRenderRequest(enemyRequest(w)).buffer); expect(wasm.length, `${w.label}: length`).toBe(w.width * w.height * 4); expect(Array.from(wasm), `${w.label}: pixels`).toEqual(Array.from(ts)); + freeze(w.label, "enemies", wasm, ts); } }, 300000); @@ -790,15 +871,22 @@ describe("the WASM engine renders the Nauvis enemy overlay exactly as the TypeSc const flat = (req: ElevationRenderRequest, eng?: typeof e): number[] => Array.from(new Uint8ClampedArray(runRenderRequest(req, eng).buffer)); const baseWasm = flat(base, e); - expect(flat(base)).toEqual(baseWasm); + const baseTs = flat(base); + expect(baseTs).toEqual(baseWasm); + freeze("enemies base", "enemies", baseWasm, baseTs); for (const [label, patch] of [ ["enemyControls.frequency", { enemyControls: { frequency: 3, size: 1 } }], ["enemyControls.size", { enemyControls: { frequency: 1, size: 3 } }], ] as const) { const req = { ...base, ...patch } as ElevationRenderRequest; const moved = flat(req, e); - expect(moved, `${label}: the two paths must agree`).toEqual(flat(req)); + const movedTs = flat(req); + expect(moved, `${label}: the two paths must agree`).toEqual(movedTs); expect(moved, `${label}: must actually move the render`).not.toEqual(baseWasm); + // `label` alone is the row key across all four overlay blocks, because + // every lever name is already qualified - treeControls, rockControls, + // enemyControls, cliffControls, cliffSettings, waterLevel. + freeze(label, "lever", moved, movedTs); } }, 300000); @@ -903,6 +991,7 @@ describe("the WASM engine renders the Nauvis cliff overlay exactly as the TypeSc const ts = new Uint8ClampedArray(runRenderRequest(cliffRequest(w)).buffer); expect(wasm.length, `${w.label}: length`).toBe(w.width * w.height * 4); expect(Array.from(wasm), `${w.label}: pixels`).toEqual(Array.from(ts)); + freeze(w.label, "cliffs", wasm, ts); } }, 300000); @@ -949,7 +1038,9 @@ describe("the WASM engine renders the Nauvis cliff overlay exactly as the TypeSc const flat = (req: ElevationRenderRequest, eng?: typeof e): number[] => Array.from(new Uint8ClampedArray(runRenderRequest(req, eng).buffer)); const baseWasm = flat(base, e); - expect(flat(base)).toEqual(baseWasm); + const baseTs = flat(base); + expect(baseTs).toEqual(baseWasm); + freeze("cliffs base", "cliffs", baseWasm, baseTs); for (const [label, patch] of [ // Frequency has to reach the slider's MINIMUM to grade much - see the // cliff-lever note in CLAUDE.md, measured over 1600 positions. @@ -967,8 +1058,13 @@ describe("the WASM engine renders the Nauvis cliff overlay exactly as the TypeSc ] as const) { const req = { ...base, ...patch } as ElevationRenderRequest; const moved = flat(req, e); - expect(moved, `${label}: the two paths must agree`).toEqual(flat(req)); + const movedTs = flat(req); + expect(moved, `${label}: the two paths must agree`).toEqual(movedTs); expect(moved, `${label}: must actually move the render`).not.toEqual(baseWasm); + // `label` alone is the row key across all four overlay blocks, because + // every lever name is already qualified - treeControls, rockControls, + // enemyControls, cliffControls, cliffSettings, waterLevel. + freeze(label, "lever", moved, movedTs); } }, 300000); @@ -982,8 +1078,11 @@ describe("the WASM engine renders the Nauvis cliff overlay exactly as the TypeSc cliffSettings: { cliffElevation0: 10, cliffElevationInterval: 40, richness: 0 }, } as ElevationRenderRequest; const terrain = Array.from(new Uint8ClampedArray(runRenderRequest(request(w), e).buffer)); - expect(Array.from(new Uint8ClampedArray(runRenderRequest(off, e).buffer))).toEqual(terrain); - expect(Array.from(new Uint8ClampedArray(runRenderRequest(off).buffer))).toEqual(terrain); + const offWasm = Array.from(new Uint8ClampedArray(runRenderRequest(off, e).buffer)); + const offTs = Array.from(new Uint8ClampedArray(runRenderRequest(off).buffer)); + expect(offWasm).toEqual(terrain); + expect(offTs).toEqual(terrain); + freeze("cliffs richness 0", "cliffs", offWasm, offTs); }, 300000); it("tiles to the same bytes as one whole render, and the halo is what makes it so", async () => { @@ -1147,6 +1246,7 @@ describe("the WASM engine renders the Nauvis resource overlay exactly as the Typ const ts = new Uint8ClampedArray(runRenderRequest(resourceRequest(w)).buffer); expect(wasm.length, `${w.label}: length`).toBe(w.width * w.height * 4); expect(Array.from(wasm), `${w.label}: pixels`).toEqual(Array.from(ts)); + freeze(w.label, "resources", wasm, ts); } }, 300000); @@ -1185,7 +1285,9 @@ describe("the WASM engine renders the Nauvis resource overlay exactly as the Typ const flat = (req: ElevationRenderRequest, eng?: typeof e): number[] => Array.from(new Uint8ClampedArray(runRenderRequest(req, eng).buffer)); const baseWasm = flat(base, e); - expect(flat(base)).toEqual(baseWasm); + const baseTs = flat(base); + expect(baseTs).toEqual(baseWasm); + freeze("resources base", "resources", baseWasm, baseTs); const withLevers = (name: string): ElevationRenderRequest => ({ ...base, @@ -1193,8 +1295,12 @@ describe("the WASM engine renders the Nauvis resource overlay exactly as the Typ }); const iron = flat(withLevers("iron-ore"), e); const copper = flat(withLevers("copper-ore"), e); - expect(iron, "iron: the two paths must agree").toEqual(flat(withLevers("iron-ore"))); - expect(copper, "copper: the two paths must agree").toEqual(flat(withLevers("copper-ore"))); + const ironTs = flat(withLevers("iron-ore")); + const copperTs = flat(withLevers("copper-ore")); + expect(iron, "iron: the two paths must agree").toEqual(ironTs); + expect(copper, "copper: the two paths must agree").toEqual(copperTs); + freeze("resources iron-ore levers", "resources", iron, ironTs); + freeze("resources copper-ore levers", "resources", copper, copperTs); expect(iron, "iron levers must move the render").not.toEqual(baseWasm); expect(copper, "copper levers must move the render").not.toEqual(baseWasm); expect(iron, "iron and copper must not be the same edit").not.toEqual(copper); @@ -1302,6 +1408,7 @@ describe("the WASM engine renders the Nauvis `all` composite exactly as the Type const ts = new Uint8ClampedArray(runRenderRequest(allRequest(w)).buffer); expect(wasm.length, `${w.label}: length`).toBe(w.width * w.height * 4); expect(Array.from(wasm), `${w.label}: pixels`).toEqual(Array.from(ts)); + freeze(w.label, "all", wasm, ts); } }, 300000); @@ -1392,10 +1499,12 @@ describe("the WASM engine renders the Nauvis `all` composite exactly as the Type "all", ] as const) { const req = { ...request(w), view } as ElevationRenderRequest; - expect( - Array.from(new Uint8ClampedArray(runRenderRequest(req, e).buffer)), - `${view}: engine and TypeScript must agree`, - ).toEqual(Array.from(new Uint8ClampedArray(runRenderRequest(req).buffer))); + const wasm = Array.from(new Uint8ClampedArray(runRenderRequest(req, e).buffer)); + const ts = Array.from(new Uint8ClampedArray(runRenderRequest(req).buffer)); + expect(wasm, `${view}: engine and TypeScript must agree`).toEqual(ts); + // Prefixed, because this block sweeps ALL_WINDOWS[2] through every view + // and a bare window label would collide with the per-view blocks above. + freeze(`routes ${w.label}`, view, wasm, ts); } const all = Array.from(new Uint8ClampedArray(runRenderRequest(allRequest(w), e).buffer)); const terrain = Array.from(new Uint8ClampedArray(runRenderRequest(request(w), e).buffer)); @@ -1473,3 +1582,29 @@ describe("the WASM engine's Nauvis terrain against the game's own preview", () = ); }, 300000); }); + +describe("the tier-3 freeze covers this spec rather than merely existing", () => { + // **Declared at the end on purpose**: tests run in declaration order within a + // file, so this sees every `freeze` call the run made. + // + // `expectRecordedRows` guards only a RECORD run - it feeds `flushRecording`, + // which returns immediately unless FMW_FREEZE_TIER3=1. So without this, a + // deleted `freeze` call site would leave its row in the table un-consulted + // and every gate would stay green while coverage shrank. That is the + // direction the three tier-2 planet specs guard with `frozenCount`, and it + // matters more here: after #227 deletes the TypeScript arm, this table is the + // only thing grading these renders. + // + // BOTH numbers are asserted because they fail on opposite mistakes. The table + // count catches a re-record that wrote a different surface; the consulted + // count catches a call site that stopped asking. A literal compared only + // against the file would move with neither. + // + // Under `-t` this test is filtered out like any other, so a partial run does + // not fail it - it simply does not run, the same way a partial RECORD run + // falls short of its declared total instead of writing a short table. + it.skipIf(RECORDING)("consults every frozen row exactly once", () => { + expect(frozenCount(SECTION), "rows in the committed table").toBe(ROWS); + expect(consultedCount(SECTION), "distinct rows this run looked up").toBe(ROWS); + }); +});