tables: the map codecs — the C++ reference, the corpus and the gate (#380)

[gafferongames]

The C++ reference's half of map[K]V (docs/SPEC-TABLES.md §2.8): the storage,
the builder, both wires, the node extent, the cook and the text form, held by a
gate with ten negative controls demonstrated red.

The language half is #487 and the page is #479. The eight ports carry none of
it — a map is a variable-class construct and the variable class is the
reference's alone — so every port refuses a map-bearing unit by name, and
schema#502 is that gap's home behind PORTING.md's M19 cells.

What a map is here

A LOOKUP over an array of one generated { key, value } ENTRY table held in
ascending key order. No new wire kind, no new framing, no new skip rule. The
holder's record carries SIXTEEN BYTES — a self-relative reference to the entry
array and an int32 count — and the entries are by-value records inside the
HOLDER'S NODE EXTENT, laid after the record's own storage, PRE-ORDER.

1. Storage. TableMap<Entry>: the reference, the live count, padding to
eight — one member, so a port that walked two pieces would account for twelve
bytes where sixteen are written.

2. Region load. LoadMeasure's term is N × sizeof( Entry ) at
alignof( Entry ) AT EVERY DEPTH, summed from the FRAMING alone — N is
framing and not a value, so it reads no field and the caller can refuse a number
it did not expect. Load carves the arrays pre-order out of the node's own
extent. Before an entry's body decodes the reader SCANS its field headers for
the key's id: ascending against the key of the last entry that LANDED, a
duplicate resets the slot it took so last wins WHOLE, a descending key stops the
map and the PARENT reads on, a key past the bound is skipped by its L and
counted clamped, and a key kind the reader does not declare empties the map
for ONE kind_mismatch. LoadBuilder produces exactly Load's report, on
every row — the gate checks the five counters and the flag against each other
for every hostile body it makes.

3. The builder. Insert returns NULL for a key past the bound and for an
arena that cannot carve another segment alike; a duplicate REPLACES, the value
reset and the entry's address unchanged. Find is the linear scan on the
builder and a binary search over the sorted array in the region. Erase marks
one bit. Each is insertion order, live entries only. The sort runs in the four
writing walks, over an array of entry pointers, because sorting the segments
would move entries whose addresses a caller holds. Nothing ever moves: the
gate holds a value handed back by the first of 200 inserts and reads it after
the last.

4. The writer. Ascending, the key first in every entry body, the two
constant ids, an empty map elided by §3's by-value rule.

5. Text form. A plain JSON object keyed by the KEY, integer keys quoted and
read by §16.2's integer rule and nothing else, ascending on the way out. A map
is the other construct the ONE walk cannot walk alone, on the pointer's own
terms (§16.7): three adapters, a stub half in a map-free unit and the real half
in a map-bearing one.

6. Cook and block. The entry array is placed by the layout rule; the slot is
the extent writer's, so a node the walk never reaches keeps an empty one. An
UNREACHED non-empty map slot — a counted array's slot past its live count — is
refused by Cook and by Lock, the same refusal §7.6 gives a pointer there;
the wire is not refused, because a counted array rides its live slots and the
unreached one simply does not ride. A
block form never holds a map — §2.7's absence, already in ir/blocklayout.go.

7. The index. IndexMeasure / Index / IndexFind, caller-owned, built at
load, never stored. Its hash and its load factor are NOT a cross-port contract
and no golden names either; what it is held to is the CONTRACT of the lookup,
and the gate checks it answers exactly what Find answers, present and absent.

11. The projections. The build version and the baseline do not move: no
golden changed, internal/baseline.Version stays 6, and tables/maps joins the
baseline corpus so the ENTRY'S ANONYMOUS KEY is pinned on real schemas —
0a53e00afba279af.d91b.9194 is a nested map's entry, keyed by the holder's wire
id and each map field's on the way in, never by the generated name.

A map-free unit is untouched, to the byte

Every emitter above is gated on the unit declaring a map. go test ./... is
green with no golden re-pin, ./build/schema_test_tables passes, the generic
JSON walk is byte-identical in all 31 generated .cpp files, and
tables-zero-cost still says "value-only tables carry no pointer or map
machinery".

The gates

make test gains tables-maps (plain and under ASan/UBSan),
tables-json-map-walk, and tables-maps-negative-controls; tables-big-endian
gains the gate's s390x twin, which reads goldens a little-endian build wrote —
§3's "little-endian, byte-oriented throughout" is a property of the WIRE and
not of the machine, and a map's framing is where that costs something. Each control
patches the GENERATOR through I2's overlay sabotage, regenerates, rebuilds and
requires the gate to go RED on a CHECK — and a sabotage that patches nothing is
itself a failure, so a control cannot rot into a no-op when the emitter it names
moves.

the sabotage	the row that meets it	red
the writer emits insertion order	an instance built OUT OF KEY ORDER	FAIL maps_main.cpp:183: memcmp( wire_full, wire_reversed, ... ) == 0
Save emits a DEAD entry	an instance that ERASES	FAIL maps_main.cpp:201: n = 378, want 424
the reader's ascending check is dropped	a SHUFFLED map	FAIL maps_main.cpp:501: v.malformed
the duplicate rule is dropped	a repeat that ELIDES a field the first set	FAIL maps_main.cpp:534: v.decoded[1] = 11, want 0
the key-kind rule decodes anyway	a row under a CHANGED KEY KIND	FAIL maps_main.cpp:626: mixed_row->entries.size() = 3, want 0
the reader CLAMPS a key	a key longer than the reader's bound	FAIL maps_main.cpp:556: v.decoded[1] = 11, want 12
the N-against-L fit check is dropped	N = 0xFFFFFFFF under a short L	FAIL maps_main.cpp:576: RowLoadMeasure( ... ) = 85899345944, want -1
LoadMeasure sums at ONE DEPTH only	an instance whose value is itself a map	FAIL maps_main.cpp:325: f->tiers.size() = 0, want 2
ToJson writes any order but ascending	the round trip's byte compare	FAIL maps_main.cpp:709: bomber != NULL && ... && bomber < fighter
Lock writes an UNREACHED non-empty map	a counted array's slot past its live count	FAIL maps_main.cpp:876: !past.Lock()

The clamp row is the one worth reading twice: the clamped count alone cannot
separate a merged entry from a dropped one, so the DECODED VALUE is the half
that says it — 11 where the rule reads 12.

The corpus

tables/maps is a map of values, a map of pointers whose two keys name ONE
node, a map of maps at three depths, and a SIGNED key where an unsigned compare
would order -3 after 2. The walk-order pin: by_id is a map[K]*T
DECLARED BEFORE the flagship pointer field, so the map reaches the shared node
first and numbers it first; a walk that grouped maps after the pointers would
number them the other way round and map_full.bin says so.
tables/maps/Rows.schema carries the report rows' two roots, which differ only
in the KEY'S KIND, and tables/maps/Depth.schema reaches the other three
by-value edges a map's extent rides under — an array of tables, an enum-keyed
array of them and a UNION ARM (§2.6) — each a different framing the load's
extent scan has to walk.

What this PR does NOT carry

The Go tool half (brief item 8) is not here, and neither are the conformance
manifest rows that depend on it: internal/tablewire, internal/tabletext and
internal/tablecook do not yet decode a map, so there is no second
implementation to hold the reference against and §4.2's differential fuzzer
gains no map rows. The reference's counters are pinned per row by the gate
above, which is what stands in until the oracle lands. schema cook-check's
entry-array containment, alignment, fit and no-overlap checks (brief item 6's
second half) are the tool's too and go with it.

That is the whole of the gap, and it is one PR's worth of work on its own. So
that it is not a silent gap, every table surface the tool has — pack,
unpack, cook, cook-check, uncook — refuses a map-bearing unit BY
NAME, names the carrier and names the follow-on. Without that refusal the
engine met a kind 14 field over element kind 13 and answered "the framing is
damaged past the point the walk could continue", which sends its reader
looking for a corrupt file when the file is fine and the reader is the one that
is short.

Two readings the page leaves open, and what this took

• Iteration on a map[K]*T. §2.8 says iteration yields "the KEY beside a
  reference to the VALUE" and, separately, that the const Find on a
  map[K]*T answers the RESOLVED pointer. Iteration here matches Find — a
  reference to the pointer SLOT would hand a caller a TableRef it has no way
  to resolve from the proxy.
• A wrong key kind that also desynchronizes the scan. The page orders the
  reader's rules but does not say which wins when a key read under another kind
  leaves the rest of the entry's headers unreadable. This counts the KIND: the
  framing damage follows from the kind, and calling it framing damage would
  report the consequence instead of the cause.
• An arm is established at selection (§2.6), so a caller that sets a union's
  tag and fills a map inside the arm without resetting it hands the builder a
  map slot out of the overlay's old bytes. That is §2.6's contract meeting
  §2.8's storage; the gate spells the reset at the call site.
• An entry whose alignment exceeds the arena's — a 128-bit value — is
  refused at compile time by a static_assert naming it, which extends the
  arena's existing node rule (alignof( T ) <= kTableAlign) rather than
  inventing one. The page does not reach this case.

🤖 Generated with Claude Code