Skip to content

The projection sees every reorder, and the codec law: enum, flags and union arm names in declaration order, ProjectionVersion 2 and a wire-law line (#462, #491, #463) - #488

Merged
gafferongames merged 1 commit into
mainfrom
projection-variant-order
Sep 4, 2026
Merged

Conversation

@gafferongames

@gafferongames gafferongames commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Closes #462. Closes #491. Closes the projection half of #463; its differential gate stays owed and stays on the page's owed list.

Three defects, each a spurious MATCH on the packet wire — the direction the projection exists to refuse — and one PR, because each moves every protocol id and the ids should move once. Rebased onto f762d6c (the maps release, the lock carve and the enum Count export), with every derived artifact REGENERATED rather than hand-merged.

The three defects

1. The projection could not see a variant reorder. An enum rendered as enum Grade max=3 storage=8 variants=3 and a flags declaration as flags Perks wirebits=3, carrying nothing per variant. Reordering either left the id byte-identical while every ordinal and every bit position changed meaning: a designer alphabetizing an enum shipped a build that shook hands with the old one and misread every value of it.

2. The projection carried no codec law. A compiler change that moves bytes under an unchanged shape left every id where it was — the 2026-08-15 fixed-point rounding amendment (ties toward +infinity to ties away from zero) did exactly that, and two builds either side of it hold the same id and disagree about bytes on exact negative ties.

3. A union with two arms of the same payload type reordered invisibly (#491). Arm order rode in the ordered payload types, which carry it only while the arms DIFFER in type. union Held { left Arm right Arm } reordered projects identically, so tag 1 means left on one build and right on the other under a single id — the enum defect exactly, in the one shape the payload types cannot describe.

The same probes under origin/main's compiler and this branch's, measured today:

                 main (f762d6c)         this branch
base             0xd3a2d101b6e89eb0     0x36077383323c947e
enum_reorder     0xd3a2d101b6e89eb0     0x159cd1fa8b041bf9   <- was identical to base
enum_rename      0xd3a2d101b6e89eb0     0xc46f5d7fc26bda56   <- was identical to base
flags_reorder    0xd3a2d101b6e89eb0     0x0a8c8d7fbae0225b   <- was identical to base
union_rename     0xd3a2d101b6e89eb0     0x536765317906637f   <- was identical to base
union_reorder    0x3a117df1cf2b6a87     0x3e92c23eb717ad85   (main saw this one)

held             0x67d428c345f37c29     0xa325e1302cdb83a7
held_reorder     0x67d428c345f37c29     0x584fc78306f6d7da   <- was identical, two Arm arms swapped

The rule

Order is the wire, and the names carry it. Every enum's and every flags declaration's variant names, and every union's arm names, now enter the projection in declaration order:

enum Grade max=3 storage=8 variants=3
  variant 1 name=Bronze
  variant 2 name=Silver
  variant 3 name=Gold
flags Perks wirebits=3
  bit 0 name=Fast
  bit 1 name=Quiet
  bit 2 name=Tough
union Shape max=2
  variant 1 name=ring payload=Ring
  variant 2 name=slab payload=Slab

That is the only way a reorder can be seen, so the consequence is accepted deliberately and stated on both pages: a variant or arm rename moves the id too, and the ship-together rule makes it free.

WireLaw rides the projection's second line, at 1. It bumps on any compiler change that can alter, for the same schema and the same values, the encoded bytes, the accepted inputs, the rejected reads, the materialized defaults or a numeric conversion, and every protocol id in existence moves with it. The invariant it holds: no generated byte and no read decision may change for the same schema and input without the protocol id changing.

ProjectionVersion goes to 2. The rendering describes what version 1 could not — the ordered names of all three vocabularies — and the number says so. The ids move once for the whole PR either way.

The id moves — one deliberate move, and nothing else

unit protocol id build version
examples 0x91a8e85156dfe2b10x682e2a15a56b78bf 0x69d6a810cfa227170x68ee62213126f184
examples128 0x42050541a90eea8a0x3a9a972a02c9e7ca 0xedde8274fbab7f850x44a8123c94d09353
tables/examples 0x9924bf6d375ec24d0x91df5fddd8edc184 0x3cf5cc8b611ce26a0x0d48f2a53a826273
tables/pointers 0x79242c6a555d3a600x2ce00b66ca6ca57c 0x8b3fa61285c779900x94b679c0a547181b
tables/block 0x7a706bf2ea47d94b0xccb9f0129fbc0ebe 0xe9c3dec8287989af0x0adbc6e7f9c4605d
tables/blockhome 0xa6612e5b3f07080a0x0fc1a1667e54b144 0xf0348a7d883219550x9ebe680f43132468
tables/messages 0x8248fac3393e55750x4a7939657edfa4ab 0x5d6070501121763a0x2e7e8c6b14f7f107
tables/stream 0xbf90766c51a5100c0x0da6b010d5e2ade7 0x093e4bddde5575de0x4556b3579a442505
tables/blobs 0xec78e4ce7f5dd9620x1cc1b12ef296a1f6 0xc20a8501aaedb1520xe7a627afef0d063f
tables/scalars 0x01ba84a4cbd2143a0xaa2ae3fa9f005efc 0x60b4cb8aaf8eaec40xa8b581f9ab3e8580
bench/corpus/Bench.schema 0xae3b1e28b96e45860x5b8227d21cba8abf 0x86eac54d593e599f0xac9196746c84b699
bench/corpus/RealWorld.schema 0x8f7228a19854fbb20x0eefe6bcd2596139 0xa82854f1174c0b7a0x2c22ca15bc43579e
bench/corpus/BenchTable.schema 0x1a38ac487752b0ae0xd093e62e6f907c36 0x1864d106cc5f2c110x5f2254539e3ad47d

Every build version moves because the cook projection carries the protocol id in whole (SPEC-TABLES §20.1 group 1).

Where those numbers are pinned, and nowhere else:

  • testdata/golden/, 123 files: id.txt, ludicrous/id.txt, the ten build-version/*.txt, and every language's generated source. Re-pinned with go test ./internal/goldens -update -run TestGolden; every changed line is an id or build-version line.
  • testdata/golden/tables/, 31 files: the frozen Table pins the block zero-cost gate compares against. Edited by hand to the new ids ONLY — see the note on stale goldens below.
  • generated/, 166 files: regenerated from the stamps CI derives. Every changed line is a ProtocolId/PROTOCOL_ID banner or constant, or a BuildVersion constant.
  • testdata/conformance/tables/cook-write/, 84 cooked artifacts: make conformance-generate. Verified byte by byte — every one of the 84 differs ONLY in the 8-byte build_version word of its header, offsets 8..15, and in no other byte.
  • docs/USAGE.md: the worked schema build-version tables/block/ answer, 0xe9c3dec8287989af0x0adbc6e7f9c4605d (held by TestUsagePageBuildVersion).
  • docs/SPEC-TABLES.md §7.3: the cook worked to the byte, 0x4efe97313c704bb50x355ef4922f004a7f, its header hexdump line with it. That number was ALREADY stale on main: the cook form's bump to 2 in tables: maps in the language — the spelling, the key rules and the generated entry (#380) #487 moved it and left the page behind. Reproduced from the page's own schema, so the page computes again.
  • internal/check/projection_test.go: the protocol-free neutrality probe, 0xad54eaab53f241b40x0d68b71928bcbcf0 (unchanged by the rebase — the neutrality probe declares no table), with its full projection text pinned beside it.
  • testdata/wire/tables/block_render.bin and block_padded.bin: the two committed block images the conformance manifest names. A block's prologue is magic, build version, byte order (§19.1) and Open is a header match, so both images take blockdemo's new number: byte offsets 8..15 in each, 0xe9c3dec8287989af0x0adbc6e7f9c4605d, and no other byte in either file. Missing these on the first push is what turned all nine conformance legs red — every leg refused a block whose prologue named a build that no longer exists, which is the header match doing its job.

bench/LOCK is untouched: the old ids inside it are the narrative of PR #348's carve, not pins.

Gates, and the controls that prove them

New in internal/check/projection_test.go:

  • TestIdMovesUnderVariantOrder — an enum reordered, an enum variant renamed, a flags declaration reordered, a flags variant renamed: each moves the id.
  • TestUnionIdMovesUnderSameTypedArmReorder — two arms of one payload type swapped, every projected type unmoved: the id moves. This is A union with two arms of the same payload type reorders invisibly: arm names must enter the projection in order, as variant names now do (#462's sibling) #491's case, and the arm names are the whole of the difference.
  • TestUnionIdBehavior gains "an arm renamed" among the moves; its old stable row is gone, because arm names are wire facts now.
  • TestWireLawLineMovesTheId — the projection opens with the rendering version and then the law line; the id is exactly the digest over that text; the same text under the next law number digests differently.
  • TestIdIsStableUnderNonWireEdits loses its "an enum variant renamed" row, which is now a wire edit.

New in internal/goldens: TestWireLawBumpMovesEveryId — for all ten pinned units, the id is the digest over the projection and the same text under the next law number is a different id. No unit can sit out a bump.

TestExportedSurfaceMovesNeitherWire, the gate #489 landed with the enum Count export, takes new literals: examples 0x682e2a15a56b78bf / 0x68ee62213126f184 and examples128 0x3a9a972a02c9e7ca / 0x44a8123c94d09353. Its own comment names this as the legitimate case — "what legitimately moves them is a change to a PROJECTION itself … which moves the corpus goldens in the same commit" — and the goldens move in this commit. Measured before and after the rebase, Count moves neither wire: every id here is what it was on the previous base.

New in ir/buildversion_test.go: TestBuildVersionSeesATableArmedUnionArmRenamed. §20.8's battery used a type-armed union's arm rename to isolate the cook projection's own union=/payload= tokens; that row now rides the protocol id like every other vocabulary row, so it can no longer isolate anything. A TABLE-ARMED union can — it has no packet wire (SPEC-TABLES §2.6), so the protocol id cannot see it — and the new test holds both halves: the arm rename moves no protocol id and must move the build version.

Three negative controls, each run red then green, in make test and on the pull-request gate: make projection-variant-order-negative-control, make projection-wire-law-negative-control, make projection-union-arm-order-negative-control. Each sabotages a COPY of the rendering through go test -overlay, so no tracked file is written, and each checks that the OTHER gates stay green — a control whose sabotage reddens everything says "something broke", not "this broke".

Without the enum and flags variant lists:

--- FAIL: TestIdMovesUnderVariantOrder (0.00s)
    an enum reordered did NOT move the protocol id (0x384a810c56bc30f5) — every ordinal changed meaning and two incompatible builds would claim compatibility
    an enum variant renamed (declaration order is spelled in the names) did NOT move the protocol id (0x384a810c56bc30f5) — ...
    a flags declaration reordered did NOT move the protocol id (0x384a810c56bc30f5) — ...
    a flags variant renamed did NOT move the protocol id (0x384a810c56bc30f5) — ...

and the union gates stay green under it.

Without the union arm names:

--- FAIL: TestUnionIdBehavior (0.00s)
    an arm renamed (arm order is spelled in names) did NOT move the protocol id (0x85b615b53eda8dea) — two incompatible builds would claim compatibility
--- FAIL: TestUnionIdMovesUnderSameTypedArmReorder (0.00s)
    two arms of one payload type reordered and the id did not move (0x1949ac0df334068b) — tag 1 means a different arm on each build and both claim compatibility

and the enum and flags gate stays green under it.

Without the codec law line:

--- FAIL: TestWireLawLineMovesTheId (0.00s)
    the projection must open with its rendering version and then its codec law; it opens:
    schema-wire-projection 2
    package probe
    ...
--- FAIL: TestWireLawBumpMovesEveryId/examples
    examples does not carry the codec law line — a rounding-rule change could not reach its id
    (and the same for all ten units)

and the variant-order gate stays green under it.

ir/buildversion_test.go also moves a flags variant reordered and one renamed from the build version's exclusions to its meaning group. They move now, through the protocol id, which the cook projection carries in whole — the direction #435 owes the cook projection its own bit positions for.

The pages

What must ride with the release

A minor with a wire bump: every protocol id in existence moves once, and deployed peers refuse newly built ones rather than misread them. The release notes state the bump FIRST, in this order: every id moves; rebuild both ends and redeploy together; a variant or union arm rename now moves the id, so a spelling fix is a redeploy; nothing else in your schemas changed.

Two things for the owner

  1. cpp-lock is carved (bench/LOCK: one-shot carve for #488 and #489 #495, bench/LOCK's prefixes suspended), so the generated C and C++ id lines pass. The emitters are byte-identical all the same — git diff origin/main...HEAD -- internal/codegen/c internal/codegen/cpp is empty.
  2. testdata/golden/tables/ is still stale on main, and not by this PR. Building origin/main's own compiler and regenerating the pins shows StreamTable.h and others differ in the tolerant read's unknown counting — PR Table reads are untrusted: the wire fuzzer with an independent oracle, C++ reference first (#391) #429 moved the C++ table emitter without re-pinning, so tables-block-zero-cost in make test is red on main today. This PR deliberately edits only the id lines in those files, so the staleness stays visible and separately fixable rather than being swept into a wire-bump diff.

Verification

make check, go test ./..., all three negative controls, and the stamp-derived regeneration leaving a clean tree — all green on this branch, rebased on f762d6c. No C++ was built here (other builders share this machine); the C++ chain is certify's.

CI state is reported in the thread below.

@gafferongames gafferongames changed the title The projection sees a variant reorder, and the codec law: enum and flags variant names in declaration order, a wire-law line beside ProjectionVersion (#462, #463) The projection sees every reorder, and the codec law: enum, flags and union arm names in declaration order, ProjectionVersion 2 and a wire-law line (#462, #491, #463) Sep 4, 2026
gafferongames added a commit that referenced this pull request Sep 4, 2026
Co-authored-by: Rowan Claude <rowan@mas-bandwidth.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
@gafferongames
gafferongames force-pushed the projection-variant-order branch from 98d7470 to d165a21 Compare September 4, 2026 05:30
Three defects, each a spurious MATCH on the packet wire — the direction the
projection exists to refuse — and one change, because each moves every
protocol id and the ids move once.

1. VARIANT ORDER (#462). An enum rendered as `enum Grade max=3 storage=8
   variants=3` and a flags declaration as `flags Perks wirebits=3`, carrying
   nothing per variant, so reordering either left the id unchanged while
   every ordinal and every bit position changed meaning. Each declaration's
   variant names now enter the projection in declaration order.

2. UNION ARM NAMES (#491). Arm order rode in the arm types, which carry it
   only while the arms differ in type: two arms of one payload type reordered
   with every projected type unmoved, so tag 1 meant `left` on one build and
   `right` on the other under a single id. Arm names now project beside the
   arm's own facts.

3. THE CODEC LAW (#463's projection half). Nothing in the projection stood
   for the compiler's own encode and decode rules, so a change that moves
   bytes under an unchanged shape — the 2026-08-15 fixed-point rounding
   amendment did exactly that — left every id where it was. `WireLaw` rides
   the projection's second line at 1, and bumps on any compiler change that
   can alter, for the same schema and values, the encoded bytes, the accepted
   inputs, the rejected reads, the materialized defaults or a numeric
   conversion.

The consequence of 1 and 2, stated on both pages: a variant or arm RENAME
moves the id, which the deploy-together rule makes free. `ProjectionVersion`
goes to 2 — the rendering describes what version 1 could not.

Gates: an enum reorder, an enum rename, a flags reorder, a flags rename, a
union reorder, a same-typed union arm reorder and an arm rename each move the
id; the id is exactly the digest over the projection text and the same text
under the next law number is a different id, for every corpus unit. Three
negative controls, each surgical: without the variant lists the enum and
flags gate goes red and the union gates stay green; without the arm names the
union gates go red and the variant gate stays green; without the law line
both law gates go red and the variant gate stays green.

§20.8's battery loses its isolating control for the cook projection's union
vocabulary — an arm rename now rides group 1 like every other vocabulary row
— so a TABLE-ARMED union's arm rename takes its place: no packet wire (§2.6),
no protocol id move, and the build version must move through group 3 alone.

Re-pinned once, deliberately, on top of the maps release's own re-pin: every
golden, the generated trees, the cooked conformance artifacts, the two block
images' build_version word, and the build versions docs/USAGE.md and
docs/SPEC-TABLES.md §7.3 print. §7.3's number was already stale on main — the
cook form's bump to 2 moved it and left the page behind — and it takes the
current value here.
@gafferongames
gafferongames force-pushed the projection-variant-order branch from d165a21 to d11254e Compare September 4, 2026 05:39
@gafferongames
gafferongames merged commit c1f9c74 into main Sep 4, 2026
20 checks passed
@gafferongames
gafferongames deleted the projection-variant-order branch September 4, 2026 05:44
gafferongames added a commit that referenced this pull request Sep 4, 2026
* bench/LOCK: the carve is spent, the prefixes are back

#488 and #489 merged under the suspended prefixes. The lock re-freezes at the new reference; this diff is the file alone.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* bench/LOCK: keep the narrative line commented

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Rowan Claude <rowan@mas-bandwidth.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant