Skip to content

docs/SPEC-TABLES.md §3: the id-table wire (#435, #439, #434) - #501

Merged
gafferongames merged 2 commits into
mainfrom
wire-id-table-spec
Sep 4, 2026
Merged

docs/SPEC-TABLES.md §3: the id-table wire (#435, #439, #434)#501
gafferongames merged 2 commits into
mainfrom
wire-id-table-spec

Conversation

@gafferongames

@gafferongames gafferongames commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Docs only. The page states the table wire's id-table form as present fact, before any implementation PR. No compiler, runtime or generated file moves here.

The wire, as §3 now states it

A saved table is three parts: the form byte, the root body, the id table.

  • The form byte is 1 and it is the whole header. It is read first, before the trailer and before any body, so a file that is both a newer form and damaged is a refusal by name and never damage. On a form the reader knows, a trailer it cannot read is malformed. Nothing else rides in front: no magic, no identity, no build version.
  • The root body ends at its own zero reference. The id table is located from the end of the wire, and any byte between the two is malformed.
  • A field is id reference, kind (u8), payload. A reference is 1-based canonical unsigned LEB128 into the id table; reference 0 names no id and is the terminator, the enum's None and the union's empty arm. Every length, count, node index and node count is the same varint, canonical, 64 bits in capability. The one fixed-width number on the wire is the id table's entry count.
  • The id table is the distinct fnv1a64 ids the body used, in first-use order across the whole wire, each a little-endian u64, then the count as a fixed u64 last. A reader resolves it once at open and dispatches every field through an array index.
  • Three kind numbers are added: 30 enum, carrying the variant id reference (Whole-standard read at max effort: 15 contradictions, 6 narrowings, 6 pattern breaks, 13 prose-without-code, 11 versioning seams — and the one-design verdict #439); 31 escape, L then L bytes, which no declaration maps to (Reserve one escape kind on the table wire: length-prefixed opaque, skip me #434); 32 payload-free, which only a union arm carries. Where the reader meets 31 or 32 decides the event: unknown at a position it cannot name, kind_mismatch at one it can. An element kind of 31 or 32 is the ordinary element-kind mismatch.
  • An arm header is a field header, so an arm carries its own kind byte and a retyped arm is reported exactly as a retyped field is.
  • Five malformed rules for the reference class: a table that cannot be read whole (stray bytes before it included), a reference past the entry count, a non-canonical varint, a reference of 0 where an id is required, and a table that carries one id twice.
  • The worked example prints all 120 bytes in hex, with each record's field references beside it, so a port can check an encoder against the page without redoing the arithmetic.

What follows elsewhere on the page

  • §4.1's silent class has four members, not five. The arm member closes by construction and the enum respelling closes under kind 30. §18's opening, §18.2's arm refusal (now the field rule) and §18.3's referent example follow.
  • §3.1: the node table rides in one field, since a 64-bit length has no ceiling to chunk around. The chunking rule, the record-never-straddles rule, the per-transport-field unknown count and the 4 GiB save-time refusals go. The reserved node-table field id is 0xFFFFFFFFFFFFFFFF, and the reserved id inside a nested body is malformed.
  • §5: one hash, fnv1a64, no fold and no rebound. A hash of 0 is an ordinary id. One id is held back, the node table's, and a declared name that produces it is refused by the checker. K1 and K2 stay.
  • §6.5 / §4.2: the smallest legal record is three wire bytes, and the measure's bound is one unit of the largest storage per two wire bytes. §4.2's mutators gain the reference class, the id table, the form byte and the arm's kind byte, with 64-bit extremes throughout.
  • §8: a descriptor's id column is uint64_t, and "no table-wire identity" is spelled with the reserved id, one value everywhere, since 0 is now a hash a real name can produce.
  • §6.3 / §7.2: a blob node's length is a u64. The wire puts no ceiling on a blob; the storage does, and a blob past §11's derived-size cap is refused at load as malformed.
  • §20: each flags variant's bit position is a group-3 meaning fact (Whole-standard read at max effort: 15 contradictions, 6 narrowings, 6 pattern breaks, 13 prose-without-code, 11 versioning seams — and the one-design verdict #439's V1). Its block is the enum block with the keyword swapped, same membership and same spelling. §20.1's table, §20.2's grammar and §20.4's lists follow.
  • The text form (§16), the cook (§7), the region's layout (§6.3) and the block form (§19) do not move, and §3 says so once. What moves is the build version, whose projection prints the wire ids and kinds, and the tables baseline, whose file records them.
  • docs/VERSIONING.md speaks of the layout in the present tense. The #435 row stays under Owed before 3.0.0, because the repository is not behind the page yet.

Numbers, recomputed and verified by first reproducing both of main's current digests byte for byte before changing a token: §20.2's worked projection is schema-build-version 3 with sixteen-hex-digit ids and grade under kind 30, giving 0xa9df771f6051391f, and the table-free unit 0xc4adaacfe6cb2809. §18.1's example is schema-tables-baseline 7. The corpus ratios, the dispatch timings and the ladder's framing figure were measured on the previous form and are re-pinned by the implementation, which the page states.

Held by test

Every rule names its pin and the one reason it goes red. The list carries the refusal verdict the report format gains for a refused form (forms 0, 2 and 0xFF), the canonical-LEB pass, the reference bound in both directions with the last slot resolving, the table-trailer pass including the repeated entry, kinds 31 and 32 at named, unnamed, arm and element positions, the enum kind against the raw integer, an enum reference no reader can name, the four extra arm pins (enum against integer, pointer against integer, bytes against string, and an L disagreeing with the LEB length under kinds 17 and 30), the body terminator, byte identity over every pinned instance, and the two reserved-id controls, which plant the collision below the hash through a compiler test hook.

Checks

go test ./compiler/... passes and make check exits 0, on origin/main at 0b8501c. No C++ or other language build was run. The docs example gate reads only fenced declaration blocks by anchor line and no declaration moved, so no exclusion was needed.

🤖 Generated with Claude Code

@gafferongames

Copy link
Copy Markdown
Contributor Author

Rulings on the six open points:

  1. The payload-free arm's kind byte is 32, as the page takes it.
  2. The reserved node-table id is 0xFFFFFFFFFFFFFFFF, and K1 and K2 stay. Add the one sentence the reservation needs: the checker refuses a declared name whose fnv1a64 is the reserved id.
  3. A repeated id-table entry is malformed. This writer emits each id once, in first-use order, and a reader that tolerated a repeat would buy nothing on a wire this schema wrote.
  4. The enum kind is 30 and the escape kind is 31, as the page takes them.
  5. The flags block renders in the cook projection with the enum block's token spelling, flags= in place of enum=, and nothing else differs.
  6. The ladder's measured figure stays as measured and is re-pinned by the implementation PR, which the page says.

The page goes to a cold read now.

@gafferongames

Copy link
Copy Markdown
Contributor Author

Rulings on the cold read (its numbering):

U1. The root body ends at its zero reference. The id table is located from the end of the wire, and any byte between the root's terminator and the table's first entry is malformed. Check order: the form byte first (an unknown form is refused as a newer form, nothing counted, before any trailer is read); on a known form, an unreadable trailer is malformed.
U2. At a position the reader names, kind 31 or kind 32 arriving where the declaration says otherwise is kind_mismatch: a field reads its default, a union reads None. Kind 31 at a position the reader does not name is unknown, as the page says. A payload-free arm declared as such and arriving under another kind is kind_mismatch, None.
U3. An element kind of 31 or 32 in an array header is the ordinary element-kind mismatch: the array reads empty, one kind_mismatch, skipped whole by L. Elements of an array of unions are arm headers and each carries its own kind, so the arm rule applies per element.
U4. The reserved node-table id inside a nested body is malformed.
U5. A repeated id-table entry is malformed. State it beside the writer rule and give the fuzzer mutant its verdict.
U6. The kind-30 paragraph states what VERSIONING.md states: a reference the reader's enum cannot name reads as the field's declared default, counted as VERSIONING.md counts it.
C1. The two "nothing reserved" sentences go. The reserved id sentence reads: no name is expected to produce it, and a declared name that does is refused by the checker.
C2. Descriptors carry a uint64_t id. "No table-wire identity" is spelled by the reserved id in descriptors too, one reserved value everywhere; a hash of 0 is an ordinary id.
C3. The two-byte body.
C4. Uniform 64: region and cook directory lengths are u64, no blob ceiling; a blob past the storage cap of §11 is refused at load as malformed.
C5. Fix the reason: the arm carries kind 32 on the wire; none is the projection's spelling of that kind.
C6. The flags block's membership and spelling are the enum block's with the keyword swapped: only flags a field or key names project. The "every flags declaration" sentence goes.
C7. The page says the ladder figure was measured on the previous form and is re-pinned by the implementation PR.
C9, C10, C11. As the read says: 64-bit LEB extremes in the mutants, one of triple or pair, eight bytes for a pointer field.
Worked example: print the 120 bytes in hex and the record field references.
Tests: the report format gains a refusal verdict distinct from a clean read, which the implementation PR adds to FORMAT.md, and the page says so; form 0 and 0xFF join the list; the last-slot reference resolves and the page says it; add the arm pins the read lists (enum arm versus integer arm, pointer versus integer, bytes versus string, an L disagreeing with the LEB length for kinds 17 and 30); the K1 and K2 controls plant the collision below the hash through a test hook, and the page says so.
Prose: 0x80 0x00 is the non-minimal zero; the entry count is the one fixed u64, say so where LEB is stated; define the three depths where used; the history and the working notes go; each repeated argument stated once; no em dashes in the added lines. British spellings elsewhere are #402's.

gafferongames and others added 2 commits September 4, 2026 16:10
§3 is rewritten to the id-table form: a one-byte form version, a body whose
ids are 1-based canonical LEB128 references into a trailing id table of
fnv1a64 ids in first-use order, LEB128 for every length, count and index, and
the entry count as a fixed u64 at the very end.

The enum takes its own kind 30 (#439), the escape kind is 31 (#434), and a
payload-free union arm takes kind 32, because an arm header is now a field
header and carries the arm's kind. That closes the silent arm-retype class,
so §4.1's silent class has four members and not five, and §18.2's arm refusal
reduces to the field rule.

§3.1, §3.2, §2.2, §2.4, §2.6, §2.8, §4, §4.1, §4.2, §5, §6.3, §6.5, §8.1,
§11, §14, §18.1, §18.2, §18.3, §19.2, §20.1, §20.2 and §20.4 are rewritten
where they named the old widths, and docs/VERSIONING.md speaks of the layout
in the present tense.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every rule the read found underdetermined is stated once as present fact: the
root body ends at its own zero reference and stray bytes before the trailer
are malformed; the form byte is read first, so a newer form is a refusal and
never damage; kinds 31 and 32 at a position the reader names are a kind
mismatch and at one it cannot name are unknown; an element kind of 31 or 32
is the ordinary element-kind mismatch; the reserved id inside a nested body is
malformed; a repeated id-table entry is malformed; and an enum reference no
reader can name is §4's unknown.

The contradictions go with them: the two "nothing reserved" openers, the
16-bit descriptor id, the five-byte empty array body, the blob's u32 region
and cook lengths, the kind=none reason, the flags block's membership, the
32-bit fuzzer extremes, triple against pair, and the four-byte pointer field.

The worked example prints all 120 bytes in hex with the record field
references. The held-by-test list gains the refusal verdict, forms 0 and 0xFF,
the last-slot reference, four arm pins and the two reserved-id controls.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant