tables: one traversal for union arms across Number, Pack, Lock, Cook and LoadMeasure (#565)

[gafferongames]

Closes #565.

What this is

One traversal for union arms. The numbering, the pack measure, the pack, the extent measure, the extent pack, the cook's extent writer and the framing scan reach a union's set arm through one arm selection, wherever the union value sits: a scalar field, an element of a bounded array, an element of a list. The static reachable set and the blob set are one IR walk, shared by the C++ emitters and the Go tool.

The corpus, red first

tables/arms (91fd8a4, rebased here as the corpus commit) declares the five shapes the issue names, and this PR adds the cross of two of them, Chain { links []Carry }, a list of unions whose arm is a table holding a list, and the cold read's two shapes, Rack { items []Slots } and Tray { entries [..2]Slots }, where Slots' arm is [..2]*Node. test/tables/arms_main.cpp crosses each with Measure and Save, LoadMeasure and Load into an exact region, the tool's path, Lock and a dereference after it, a memcpy relocation, and a cook from the arena and from the region, opened and walked. Every reference a const form holds is shown to resolve inside its block before it is followed.

Against the emitters at the corpus commit the gate does not compile under -Werror: HandCookExtent and TwigCookExtent are emitted with nothing in them and GateCookLayout never reads its context. Compiled with -Wno-unused-parameter and run against the goldens this PR pins, the red lines per shape (54 in all):

• C1, Ring (a list of unions whose arm is a pointer): measured > 0 fails and saved = 26, want -1, so Measure refuses while Save writes 26 bytes against 106 pinned. The loaded region reads r->items.size() = 0, want 4, and after Lock both ref_inside( base, bytes, first.node, ... ) and second.node fail in the locked region and in the relocated copy: the arena offsets survived the Lock. Both cooks refuse.
• C2, Holder (a table arm holding a list beside a tail): the pinned cook compare, table wire golden arms_holder_cook: 128 bytes written, 128 pinned, and opened != NULL. Same size, different bytes, and no refusal: the cook lost [7] and reported success. Every other crossing is green, which is the issue's point that two equally incomplete walks agree.
• C8, Nest (an arm that is another union holding a leaf with a list): the loaded region's report is malformed 1, leaf.items.size() = 0, want 2, ref_inside( ..., leaf.items.elements, ... ) fails in the loaded region, the locked region and the relocated copy, and the re-save reads 60 bytes against 81.
• C9, Hand (a bounded array of unions with a populated list in a selected element): the loaded region's report is malformed 1, leaf.items.size() = 0, want 3, ref_inside fails, and the re-save reads 64 bytes against 89: LoadMeasure framed the array as one arm header and Load's carve failed on valid wire.
• C10, Gate (a node and a string blob reachable only through an arm): the loaded region's report is unknown 1 for the writer's own record, ref_inside( base, bytes, g->reach.only, ... ) fails, the re-save reads 46 bytes against 83, and both arena_cook_bytes > 0 and cook_bytes > 0 fail because the layout cannot name the type. arms_gate_text (the *string arm) reads the same way, 46 against 71.

After the repair the gate is green over nine values, in the plain build and under -fsanitize=address,undefined, and the Go tool reads every pinned wire back and writes it again byte for byte, so the two walks number the arms' nodes alike.

The cold read's two additions

The cold read of this branch found two things the corpus did not hold, both listed under "Silences found, not decided" below and both closed here. Carrying the first one into the corpus turned up a third, which is the gcc leg's reading of the memset that establishes an arm.

• The arm's slot loop gets its own index. emitPointerSlotsOf spelled the slot index i, the same name the element loop uses, so under a list or an array of unions the inner i shadowed the element index and slot k of element i read element k's node. Rack and Tray hold the shape and are red before the repair: [arms_rack] measured > 0 fails, Save writes 72 bytes against 139 pinned, and the second node of each element is never numbered. The index is now k. The only other unit the rename reaches is stream, over Chunk's and Feed's [..2]*Chunk slots.
• An arm's establishment casts to void *. Rack and Tray are the first shapes where the memset that establishes a selected arm runs over storage holding a TableRef, and gcc's -Wclass-memaccess refused the header on the big-endian leg: TableRef has a default member initializer, so it is non-trivial to DEFAULT-CONSTRUCT while staying trivially copyable, and the warning reads only the first of those. The memset stays, because the PADDING inside an arm's storage rides into a cook image and a block row and = {} would leave it alone, and the address is cast. Twelve golden lines move, ten in messages and two in arms.
• An array of tables is refused as an arm, by name. union U { few [..2]Leaf } over a table Leaf was admitted by the checker and descended by every walk as ONE body, so elements past the first were unreachable. The checker now refuses it at the arm, points at the shape that does work, a table holding the array made the arm, and names schema#579 for whether to admit it. TestArrayOfTablesArmIsRefused in the break-the-language suite carries both spellings, [..N]T and [N]T, and keeps a table arm, a scalar-array arm and a [..N]*T arm green beside them.

The repair, per file

• ir/table.go: reachableEdges walks a root's closure once in the numbering walk's order, calling back at every field line that names a node, and descends every by-value edge there is, a nested table, an array element, a map entry, a list element and a union's arms with nested unions included, and a pointee. PointerReachable and PointerReachableBlobs are read off it. PointerReachable answers the tables in first-visit order and takes only the root. A pointer arm names its pointee, which is C10.
• internal/codegen/cpptable/pointers.go: the backend's pointerReachable and reachableBlobs closures are deleted in favor of the IR's. edgeExpr carries the storage under every subject a walk runs on, the value's path, the pack's twin and the cook's byte address as a base and a constant offset, with member and index for an arm and an element. emitUnionArmWalk is the one-union-value arm selection factored out of emitVariableUnionWalk, and a scalar field, a bounded-array element and a list element all take it. edgeOf makes a list of unions an edge (C1).
• internal/codegen/cpptable/lists.go: listElementUnion, and emitListEdge visits a union element through emitUnionArmWalk.
• internal/codegen/cpptable/extent.go: unionHasExtent asks an arm of an arm (C8). emitExtentWalk takes the visitor's subjects and hands the three extent emitters an edgeExpr, and emitListElementExtents descends a table element or a union element after the whole array. emitCookExtent is that walk with the record's address beside the value, its second classification loop deleted (C2). emitCookNode holds the extent written to <T>Extent before the header. emitWireExtentCases dispatches from the field's actual shape (C9): one arm header for a union field, a kind 14 body of arm headers for an array, and for a list of unions the elements' arms after the array's own term. emitUnionWireExtent emits <U>WireArmExtent and <U>WireArmsExtent per union with a container below an arm, declared before the members' walks and defined after them.
• internal/codegen/cpptable/cookwrite.go: the layout uses the IR's reachable set and names its context unused where neither the root nor any node it can name has an extent.
• internal/tablewire/decodenodes.go, test/conformance/harness/wirefuzz.go: the new PointerReachable signature, and the two wire-fuzz oracle controls in the Makefile re-anchored on it. The placeable set is named on its own line and says in a comment why: that line is the seam tables-wire-fuzz-node-type-negative-control replaces with the whole unit closure.
• tables/arms/Carry.schema, tables/arms/tables.baseline, test/tables/arms_main.cpp, Makefile: Chain and its wire root in the tool round trip. The baseline moved once, recorded in its history. The gate prints its check count.
• tools/sabotage/main.go, Makefile: six negative controls, below.
• docs/SPEC-TABLES.md: the §2.6 bullet, below.

Goldens that move

• testdata/golden/tables/arms/*: new.

• maps/DepthTable.h (83 lines): the arm framing scan moved out of DepthWireExtent into ForceWireArmExtent and ForceWireArmsExtent. DepthCookExtent now takes the shared walk: the nested field's descent is spelled inside the walk's brace, the counted array's live count as i < value.many_count && i < 3, the slots past the count are refused as the pack refuses them, and the union arm's Squad is cooked, which it was not. SquadCookNode and DepthCookNode gain the extent check.

• maps/FleetTable.h (20), maps/RowsTable.h (6), lists/ReportTable.h (9), lists/MigrateTable.h (3), lists/SaveTable.h (13), lists/SharedTable.h (10), lists/HoldersTable.h (21): the extent check in every CookNode of a record with an extent, the zero <T>Extent for a variable member without one (the corpus commit's emitter change, pinned here), and in Holders the counted array's cook spelled as the walk spells it.

• messages/MessagesTable.h (10 lines): the void * cast on the arm establishment's memset and nothing else.

• stream/StreamTable.h (27 lines): the slot-loop index rename and nothing else, in ChunkNumber, ChunkPackMeasure, ChunkPack and the same three on Feed, over their [..2]*Chunk slots.

No other list-free, map-free unit moves: examples, pointers, block, blockhome, blobs and scalars are byte-identical to their pins, and no tracked generated/ tree carries an arm establishment at all.

Negative controls

make tables-arms-negative-controls, in the tools/sabotage and Go overlay pattern, each red on a CHECK the clean tree passes and each held to the line that names its shape:

• arms-list-union-edge (C1): a list of unions is no edge of the walk. 38 red, first [arms_ring] measured > 0.
• arms-cook-skips-arm (C2): the cook alone skips a table arm's containers. 20 red, [arms_holder] S::Cook( b, from_arena, ... ): the extent check refuses the cook before a header is written, and so on Nest, Hand and Chain.
• arms-cook-check-dropped: the same skip with the extent check dropped. 8 red, table wire golden arms_holder_cook: 128 bytes written, 128 pinned, and the control requires that no S::Cook line is red: the cook loses the list and reports success, which is what the check exists to refuse.
• arms-nested-union-extent (C8): an arm of an arm is not asked. 11 red, [arms_nest] ref_inside( base, bytes, leaf.items.elements, ... ) and malformed 1.
• arms-array-of-unions-framing (C9): every union field framed as one arm header. 6 red, [arms_hand] a loaded region: the report is not silent (... malformed 1).
• arms-reachable-arm (C10): a pointer arm names nothing in the reachable walk. 47 red, [arms_gate] ref_inside( base, bytes, g->reach.only, ... ) and unknown 1.
• arms-slot-index-shadows: the arm's slot loop spells its index i again, so it shadows the element index. 34 red, first [arms_rack] measured > 0, then saved = 72, want -1, table wire golden arms_rack: 72 bytes written, 139 pinned, r->items.size() = 0, want 4 and ref_inside( base, bytes, element.many.value[k], ... ) on both live elements. The gate's own -Wshadow -Werror refuses the sabotaged header outright, so this one control compiles with -Wno-shadow to reach its CHECK, which is why arms_negative_control grew a flags parameter.

The checker refusal's control is the break-the-language suite itself: with the case f.Array != ir.ArrayNone && !f.Type.Pointer && armNamesTable(f) arm taken back out through go test -overlay, TestArrayOfTablesArmIsRefused goes red on both spellings, a bounded array of tables: the refusal went missing: [] and a fixed array of tables: the refusal went missing: [].

The page

docs/SPEC-TABLES.md §2.6, the bullet refusing a map or a list directly in an arm: the rationale said an arm's array would make the extent depend on a discriminant, which no cook could be byte-stable under, and that a table wrapper removes that dependence. A table arm's arrays are placed by the selected tag in the holder's extent too, so the wrapper removes nothing of the kind. The bullet now keeps the restriction as a surface rule with what it actually keeps, one grammar for where a container lives, a field of a record. Whether to admit the constructs in an arm under the deterministic arm rule is not decided here.

§2.6 gains a second bullet, refusing an array of tables as an arm and naming schema#579, and §15 gains the follow-on entry that bullet and the diagnostic both point at, beside the three arm refusals that already have one.

Silences found, not decided

• An arm that is an array of variable tables, union Many { few [..2]Leaf } with a list inside Leaf. CLOSED here: the checker refuses the shape by name and §2.6 and §15 say so, with schema#579 carrying whether to admit it.
• A list or an array of unions whose arm is an array of pointers: the arm's slot loop and the element loop both spell i. CLOSED here: the slot index is k, Rack and Tray hold the shape, and arms-slot-index-shadows is the control.
• cook-check over a list-bearing unit (schema#380) still does not run, so the arms cooks are pinned and walked and not crossed with the tool.

Certification

• make test locally on this MacBook Air, after the rebase onto main at 4963be9, runs green through every gate and 52 negative controls red as required, and stops at the conformance harness's Dart leg, which cannot build: this machine's shared dist has become a symlink to itself, so no pinned SDK under it resolves from any clone. That is a workstation fault, not a tree fault, and the leg it blocks is green on CI. The earlier whole run at c3ab394, before the rebase, read 13:09 to 13:20 with 108 controls.
• All 20 CI checks pass at 550b476, the nine conformance legs and the big-endian and msvc legs among them.
• The arms gate: 651 checks over nine values, green in the plain build and under -fsanitize=address,undefined, and every pinned wire round-trips through the Go tool byte for byte. The gate also runs on the big-endian leg, which is where the arm establishment's memset was caught: clang does not have -Wclass-memaccess, so the gcc legs are the only place that reading shows.
• The zero-cost sitting (law: a diagnostic costs nothing on the read or write path, measured #546): the tables bench's C++ leg on the tolerant wire, bench/tables/run.sh --only cpp --rounds 3, from this MacBook Air (arm64) with other builders in sibling clones, each run started under a one-minute load below 2.0. Before, the committed generated/bench/tables/cpp from main's compiler, load 1.88: write 0.746 to 0.760 M msg/s (1528 to 1556 MB/s), round trip 0.253 to 0.258 M msg/s (518 to 529 MB/s). After, regenerated with this branch's compiler, load 1.83: write 0.742 to 0.762 M msg/s (1519 to 1559 MB/s), round trip 0.254 M msg/s (519 to 521 MB/s). Unchanged within the run-to-run spread, as it must be: git diff generated/bench/tables/cpp after the regeneration is empty, because nothing here reaches a fixed table's codec.

🤖 Generated with Claude Code