Skip to content

spec: the message form, an id table scoped to the connection (#523) - #530

Merged
gafferongames merged 3 commits into
mainfrom
spec-connection-id-table
Sep 4, 2026
Merged

spec: the message form, an id table scoped to the connection (#523)#530
gafferongames merged 3 commits into
mainfrom
spec-connection-id-table

Conversation

@gafferongames

@gafferongames gafferongames commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Ruling 10 on #523, plus the owner rulings and named changes from two cold-read passes. Specification only, nothing is implemented.

A second wire form under the one-byte form version moves the id table off every message and onto the connection. The file form does not change. No kind is spent, no payload row moves, no skip rule changes. The form is framing and nothing else.

The form

  • Form byte 2. Two parts: the form byte and the root body, which ends at its own zero reference. No trailer.
  • The vocabulary is the UNIT's whole closure, in the cook projection's order: every field name, every enum variant and union arm name, then a fixed tail — the reserved node-table id, the three blob type ids (bytes, string, wstring), and the name id of every table in the closure in the projection's sorted record order.
  • The tail is unconditional, and the page states the choice. A unit with no pointer announces it anyway, because scoping it to pointer targets would move every slot after an inserted type id, and scoping the node-table and blob ids to units with a variable class would move the whole tail on a unit's first pointer. A slot number is a compile-time constant a generated field header carries, so it must not drift under an edit that has nothing to do with it.
  • Three properties follow. The table is a pure function of the build version, so "keyed by the build version" is literal. The whole announcement is a compile-time constant of the unit, so a backend may emit Announce and AnnounceMeasure as a constant byte array and its length rather than as a walk, and the C++ reference states which it does. The writer's slot numbers are compile-time constants in generated field headers, so there is no runtime slot lookup on the send path. The receiver resolves once.
  • The ID TABLE MESSAGE announces it, and it is an ordinary form-1 file. Its body is one required field, the build version, under a reserved id 0xFFFFFFFFFFFFFFFE, and its trailer is the connection's table. It reads tolerantly with exactly one strict check (that field present once, kind 9, eight bytes wide), so the announcement can gain a field in a later minor without a lockstep redeploy.
  • Once a connection per direction, and never again. No re-announcement and no state machine. A second announcement is refused by name and replaces nothing. A refused announcement sets no table, so every message on that connection is refused for want of one. A build change is a new process and a new connection.
  • The build version keys the table and never gates the connection. §20.5 stands.
  • A peer with no table refuses the message by name. Nothing decoded, no counter moves, malformed does not fire. The recovery is the sender's: a new connection announced first, or the file form. A receiver never guesses a table and never falls back on its own.
  • A reference past the table is §3's framing damage at the level it occurs. In the root body the body stops, malformed counts once, and the fields decoded before it stand. No second pass.
  • A connection is a transport connection: one TCP or WebSocket connection, or one reliable ordered stream or channel of QUIC or a reliable-UDP transport, per channel. A restart is a new connection with empty tables, a receiver caches nothing across connections, and stateless request-response is out of scope by name (the file form rides there).
  • What this form does not carry: which root a message is, is the application's; where a message ends is the transport's. §1's no-envelope promise at the message level.
  • SaveRetain under form 2 refuses by name and returns -1, on §6.6's misuse precedent, never a silent drop. Loading is unchanged. The two answers are the file form and a verbatim relay.
  • The packet wire is out of scope, and the page says why: it is same-or-refuse on the protocol id, so both peers must ship together, and a deployed game client and a backend do not.

The invariant across the forms

The bodies are not byte-identical: a file's slots are its own first-use order and a connection's are the unit's projection order, which sorts records and vocabularies by name. What is invariant is the resolved form, §6.6's own normal form, every reference replaced by the 64-bit id it names and every length recomputed. That is the claim and the pin. Each vector's byte length is a pinned golden of its own.

What it measures

The three backend messages of #523, verified against this repository's own compiler at bafdb69:

message file form message form
LoginRequest, non-default 106 58
MatchResult, non-default 273 225
StorePurchase, non-default 104 48
LoginRequest, defaults 10 2
MatchResult, defaults 43 27
StorePurchase, defaults 10 2

45%, 18% and 54% off the three non-default messages, which turns a loss of 2.2x, 1.4x and 2.6x against proto3 into one of about 1.2x, and MatchResult at its declared defaults into an outright win at 27 against 40.

The backenddemo unit names twenty ids in its records (twelve distinct field names, eight enum variant names, counted from schema build-version --facts) and eight more in its tail, so with the reserved build-version id its announcement is twenty-nine entries and 252 bytes. One round of the three saves 152 bytes, so it is paid back partway into the second round. Every slot is still one byte, so no message size moves. A unit of 500 ids announces about 4 KB once. The bound stays 4096 entries.

The surface, owed to §11

Named but not claimed in this change, on §6.6's precedent: TableVocabulary; the unit-scope Announce, AnnounceMeasure, AnnounceRead; the suffixes LoadMessage, MeasureMessage, SaveMessage; the refusal reasons no_vocabulary, second_announcement, vocabulary_too_large, message_form_as_file; and Dart's loadMessage / measureMessage / saveMessage plus the library-scope four.

Goldens

Twelve vectors, the three messages in each form, under a new backenddemo unit at tables/backend and a backend_conn connection, with two manifest line kinds (connection, message). Four more carry rules a value alone cannot reach: a wide vocabulary, a pointered root, a two-peer pair for per-direction independence, and a connection carrying a second announcement. The wide-vocabulary unit is generated and committedtest/vocabgen writes tables/vocab/Vocab.schema, unit key vocabdemo, ten tables of thirteen fields, on test/cookgen's precedent — so its message names one id in a one-byte slot and one in a two-byte slot.

Files

  • docs/SPEC-TABLES.md §3.3, new, the whole form.
  • docs/SPEC-TABLES.md §3, §3.1, §5, §11: the form-byte bullet names both forms, the form pin re-points to forms 0, 3 and 0xFF, the tiny-message paragraph names the message form beside the type stream, §3.1 states the reserved-id rule for both ids, and two ids are held back now rather than one.
  • docs/VERSIONING.md: a section of its own between the wire form and the text form, promise 7 amended, and Design lock-in before the sweep: the cold read's shortest list #523 on the Owed-before-3.0.0 list.

Docs-only, and it merges clean against main, #531, #532, #520 and both remaining spec branches.

Ruling 10 on #523. A second wire form under the form byte moves the id
table off every message and onto the connection, and the file form does
not change.

SPEC-TABLES.md 3.3, new
- Form byte 2 is the message form: the form byte and the root body, no
  trailer. The body is byte-identical to the file form's, so no kind is
  spent, no payload row moves and no skip rule changes.
- References resolve against the connection's table, per direction. The
  resolution rule itself is unchanged.
- The id table message announces it, and it is an ordinary form-1 file:
  one field, the build version, under the reserved id
  0xFFFFFFFFFFFFFFFE, and a trailer that is the connection's table. The
  most recent announcement from a peer governs that peer's messages.
- The build version is a label and never a gate. 20.5 stands.
- A peer with no table refuses the message by name, no counter moves.
  The two recoveries are the sender's: re-announce, or write the file
  form. A receiver never guesses a table and never falls back on its own.
- A reference past the table is malformed, and in the root body, which
  has no enclosing length, the whole message is.
- Elision, defaults and the read report are unchanged. Retention loads
  unchanged and drops on a form-2 save, with the reason and the two
  answers, a file-form save and a verbatim relay.
- Security: a lying build version buys nothing, a connection table is
  bounded by the receiver at 4096 entries, announcements are capped, a
  reference storm is linear and allocates nothing.
- The packet wire is out of scope, and the page says why: it is
  same-or-refuse on the protocol id, and these peers do not ship
  together.
- The measured effect as the page's own claim, 58 / 225 / 48 against
  106 / 273 / 104, and goldens: the three messages as six named vectors
  in each form, with the unit, the connection and the manifest lines.

SPEC-TABLES.md elsewhere
- 3: the form byte bullet names both forms; the tiny-message paragraph
  names the message form beside the type stream.
- 3, 5, 11: two ids are held back now, not one.

VERSIONING.md
- A section of its own between the wire form and the text form: what the
  form is, what the build version keys and does not gate, the refusal,
  what does not move, retention across the forms, promise 7's owed
  wording, and the two sharp edges.
- Owed before 3.0.0 gains #523.

Nothing is implemented: the subsection states its backend status, and the
form is built in the C++ reference next.
@gafferongames

Copy link
Copy Markdown
Contributor Author

Rulings from the cold read, under the owner's word of 2026-09-04:

  1. The announcement is the unit's whole vocabulary, in a compiler-settled order, once per connection per direction: the table is a pure function of the build version, slot numbers are compile-time constants in the generated field headers, no runtime slot lookup on the send path, the receiver resolves once. No re-announcement inside a connection; a second announcement is refused by name. The 4096 bound stays.
  2. SaveRetain under form 2 refuses by name and returns -1, never a silent drop.
  3. The announcement reads tolerantly with one strict check (the reserved build-version field present once, kind 9, width 8), so it can gain fields in a later minor.
  4. A connection is the transport connection (TCP, WebSocket, or one reliable ordered channel or stream), per channel; stateless request-response transports are out of scope for the form; a restart is a new connection; nothing is cached across connections.

The named page changes from the read land on this branch before it leaves draft.

…es (#530)

The two owner rulings from the cold read of #530, and the named changes.

Ruling 1: a peer announces its unit's WHOLE closure vocabulary, once per
connection per direction, in the cook projection's order.
- The table is a pure function of the build version, so "keyed by the build
  version" is literal. The writer's slot numbers are compile-time constants
  and there is no runtime slot lookup on the send path. The receiver
  resolves once.
- The re-announcement state machine is gone: no "most recent governs", no
  amendment on a vocabulary change. A second announcement on a connection
  is refused by name, replaces nothing, and the receiver closes. A build
  change is a new process and a new connection.
- The bound stays 4096 entries. A 500-id unit announces 4 KB once, which is
  the cost of ruling out re-announcement and is stated, not buried.
- The heading says what the table now is.

Ruling 2: SaveRetain under form 2 REFUSES BY NAME and returns -1, on 6.6's
misuse precedent, never a silent drop.

Named changes:
- (a) The body-identity claim was false: a file's first-use slots are not the
  connection's slots. Replaced with identity under RESOLUTION, 6.6's own
  normal form, every reference replaced by the id it names and every length
  recomputed, plus a pinned byte length per vector.
- (b) A refused announcement sets no table, and there is no earlier one to
  keep. The contradiction is gone.
- (c) A reference past the table in the root body follows 3: the body stops,
  malformed counts once, and the fields decoded before it stand. No second
  pass, no message thrown away whole.
- (d) 3.1 now states the rule for BOTH reserved ids, so 3.3 cites a rule that
  exists.
- (e) The surface is named and owed to 11's claimed set on 6.6's precedent:
  TableVocabulary, Announce / AnnounceMeasure / AnnounceRead, LoadMessage /
  MeasureMessage / SaveMessage, four refusal reason values, and Dart's
  spellings. Root dispatch is the application's and message framing is the
  transport's, both stated. The node-table id and every pointer target's type
  id are entries of the announcement, stated.
- (f) A connection is a transport connection, per reliable ordered channel. A
  restart is a new connection with empty tables, a receiver caches nothing
  across connections, and stateless request-response is out of scope by name.
- (g) 3's form-byte pin now uses forms 0, 3 and 0xFF, since 2 is a known form
  with rows of its own. Promise 7 amended in this PR. One word, "key", on
  both pages.
- (h) Four more vectors: a slot at or past 128, per-direction independence, a
  pointered message, and a refused second announcement. The writer's new-id
  question is gone by ruling 1, since an id outside the vocabulary cannot
  occur.

The announcement for the backenddemo unit is 188 bytes, twenty-one entries
(twelve distinct field names, eight variant names, the reserved id), counted
against the compiler's own projection. Message sizes are unchanged at
58 / 225 / 48 / 2 / 27 / 2, and the payback moves into the second round.
… constant (#530)

The cold reader's two remaining edits on the second pass, plus the two
statements it asked for.

- The tail, in fixed order after the closure's field, variant and arm names:
  the reserved node-table id, the three blob type ids as bytes, string and
  wstring (§3.1, and #532 for the third), then the name id of EVERY table in
  the closure, not only pointer targets, in the projection's sorted record
  order. The tail is UNCONDITIONAL and the choice is stated: if only pointer
  targets carried a type id, adding a *T to an existing table would move every
  slot after it, and if the node-table and blob ids rode only for a unit with
  a variable class, a unit's first pointer would move the whole tail. A slot
  number is a compile-time constant a generated field header carries, so it
  must not drift under an edit that has nothing to do with it.
- The resolved-form paragraph said "declaration order" where the connection's
  slots are the PROJECTION's order, which sorts records and vocabularies by
  name. Fixed.
- The announcement is a compile-time constant of the unit, so a backend may
  emit Announce and AnnounceMeasure as a constant byte array and its length
  rather than as a walk, and the C++ reference states which it does.
- The past-127-ids vector has a home: test/vocabgen writes tables/vocab, unit
  key vocabdemo, ten tables of thirteen fields, generated on test/cookgen's
  precedent and committed like any other corpus data, because a golden a
  generator has to re-derive is not a golden.

The backenddemo announcement moves with the tail, from twenty-one entries and
188 bytes to twenty-nine and 252: twenty ids in its records, eight in its tail,
and the reserved build-version id at slot 1. Message sizes do not move, every
slot is still one byte, and the payback stays in the second round.
@gafferongames
gafferongames marked this pull request as ready for review September 4, 2026 15:51
@gafferongames
gafferongames merged commit 8fd34f1 into main Sep 4, 2026
20 checks passed
@gafferongames
gafferongames deleted the spec-connection-id-table branch September 4, 2026 15:54
gafferongames added a commit that referenced this pull request Sep 4, 2026
…omments (#523) (#532)

* spec: the wstring table kind, widening, float16, Open reasons, text comments (#523)

Six rulings from #523, specification only.

1. wstring on the table wire: kind 33, `*wstring` under the reserved blob
   id fnv1a64("wstring"), the cooked storage row, the text row, the
   LoadMeasure term, the map-key refusal by name, and the #510 closure
   refusal deleted.
2. Widening on read: an integer kind into a wider one of the same
   signedness, and f32 into f64, decodes exactly under a new `widened`
   counter. Every other pair stays kind_mismatch. The silent class does
   not move.
3. Flags storage stays uint64 in every target, with the reason.
4. float16 declined, kind 34 reserved by name.
5. Open and BlockOpen fill a TableOpenReason beside the null.
6. The text form accepts // and /* */ comments on read and never writes
   them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* spec: keep the wstring refusal clause off the byte-buffers bullet's opening lines

* spec: ill-formed text is framing-class damage on both text kinds, and the cold read's named changes (#523)

* spec: reconcile with the message form (#530), and fix its form-byte fuzzer pin

* spec: keep the three-spelling list edits clear of the unbounded-array lines (#531)

* spec: the wide-text writer rows are storage built in code, not loaded

* spec: name TableOpenReason from the measure's reason table, and widen its stated scope (#531)

* spec: the refusal enum is TableRefuseReason, since LoadMeasure's -1 carries it too (#531)

* spec: the arm companion list carries wstring, three reserved-id refusals, one header on the refusal table

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 4, 2026
…530)

The fuzzer's reference pass over a form-2 wire, which §3.3 asks for by name:
every reference set to the entry count plus one and to the extremes the
encoding can spell, and to the entry count itself, which is the last legal
slot and must resolve.

The ORACLE reads and writes a message root through DecodeMessage and
EncodeMessage against the connection's table, and a variable root's region is
sized from NodeRecordTypesMessage, which is the same record scan over a body
that has no trailer behind it.

The ROSTER carries the wire's own form byte per entry, so one root is two
entries when the corpus pins it in both forms. A leg answers 1 for a form it
has a codec for and 0 for one it does not, exactly as it already answers for a
root it cannot name. No announcement rides the roster: the vocabulary is a
pure function of the build version, so the leg announces to itself from the
constant its backend emits and holds the table the oracle holds.

The C++ leg gains the six message roots the corpus pins and the five file-form
roots the two new units add, which takes the run to 112 live seeds and 103,286
enumerated mutants, no divergence, plain and sanitized.

The fuzzer earned its place on the way in: the first run went red on
login_full_message truncated to three bytes, because this leg freed the
announcement's buffer and a TableVocabulary BORROWS those bytes.

Beside it, the documentation the form is owed: the `message` surface and the
roster's form byte in the conformance contract, the two manifest line kinds on
the corpus page, the message form in PORTING.md's M20 as the ports' follow-on,
§11's suffix count moved from 38 to 41 and 54 to 57 where the earlier sentence
had been missed, the tool's two flags in both synopses, and a USAGE.md section
that teaches the form by example.
rowan-claude pushed a commit that referenced this pull request Sep 4, 2026
§3's held-by-test names three report rows for the forms no reader knows, form
0, form 3 and form 0xFF, and says form 2 is the MESSAGE FORM with rows of its
own (§3.3). The corpus still carried form 2 as the middle unknown, which is a
row that says the opposite of what the page now says: a build that carries the
message form refuses a message read as a FILE under message_form_as_file, not
under newer_form.

So form_two.bin retires and form_three.bin takes its place, the three rows
assert the reason as well as the verdict, and §3.3's own row lands beside them:
message_as_file, the whole of one real message handed to a reader that expected
a file. Every leg can answer it, because the form byte is read first and the
root is any root.

The second row §3.3 asks for, a form-2 message with NO announcement, has no
home on this surface: a report row hands a driver a wire and a root and never a
connection, and neither manifest line kind the page states can spell "no
table". It is held in test/tables/message_form.h and in the harness's own
engine test instead.
rowan-claude pushed a commit that referenced this pull request Sep 4, 2026
A gate nothing can turn red is a gate nobody is watching, and this form's
rules are the kind that leave no trace in a green run: a slot that moved still
resolves, a bound that is gone still reads a table, and a second announcement
that was accepted still decodes. So each control removes ONE rule and names
the gate that must go red.

Six ride on `go test -overlay`, so nothing tracked is written and an interrupt
cannot leave a sabotaged tree:

  the tail          the node-table id out of the vocabulary   TestTheTailIsUnconditional
  the order         the projection's record order reversed    TestTheAnnouncementIsTheUnitsOwn
  the substitution  a reference left in place of its id       TestTheTwoFormsResolveAlike
  the bound         the entry count compared against nothing  TestTheMessageFormRefusesByName
  the second        a second announcement accepted            TestTheMessageFormRefusesByName
  the writer        every slot off by one                     TestTheMessageFormRoundTripsThroughTheEngine

The seventh is the C++ EMITTER's own slot, which is the one the reference
reads and writes: moved by one through a build overlay, the unit regenerated,
and test/tables/message_negative_main.cpp round trips the corpus's pinned
login_full message against it. The golden still LOADS under a moved slot,
because the reader resolves whatever the writer named, so what goes red is the
round trip against the corpus's own bytes, which is the point of pinning them.

Three failure messages gained a first-difference offset while I was reading
them red: two byte strings of the same length that differ told a reader only
their length.
rowan-claude pushed a commit that referenced this pull request Sep 4, 2026
…#530)

The fuzzer found it on its first long pass over the new corpus: mutant 113542
of seed 24845619678, over graph_tree_message. The leg reported 2 unknown and
sized a 472-byte region; the engine reported 4 unknown and one kind_mismatch
and sized 488.

THE MESSAGE FORM IS WHAT MADE IT REACHABLE. A file's trailer carries only the
ids its own body used, so a node record can never claim a type the writer did
not write. A connection's table announces every table's name id whether or not
a pointer names it (§3.3's unconditional tail), so a mutated node record can
claim `Layer` — a table graphdemo declares, Scene nests BY VALUE, and no
pointer anywhere targets.

The reference is right and the engine was wrong. A node record is a POINTER's
pointee, so a table no pointer below this root targets is "a type id this
reader cannot name" in §3.1's own words: no region storage, the body skipped
by its length, one unknown counted. <Root>NodeStorage has always answered -1
for one. The engine named the whole unit closure, placed the node, decoded its
body and counted what was inside.

ir.PointerReachable is that set, ported from the C++ emitter's own walk, and it
lands in the engine's node table and in the fuzzer's region arithmetic. The
emitter keeps its own copy this pass; folding the two into one walk is a
follow-on and not a rule change.

The vector is pinned as message_node_type_unpointed, which takes the index's
line kind up by an optional `message` marker, because a vector's FORM is part
of what the index has to say about it. The negative control is the revert:
tables-wire-fuzz-node-type-negative-control, red on that vector by name.

Beside it, three things the red taught:

  --replay now takes --message, and a failure on a message seed PRINTS it. The
  replay command the harness printed read a form-2 mutant as a file, which is
  another wire entirely, and reproduced nothing.

  message_form.h pinned message_reserved_id_nested at 20 bytes from a 19-byte
  array. ASan caught it in the sanitized tables build; the committed golden had
  one byte of stack in it. Every pinned length there is sizeof now.

  build/schema_test_tables_asan and _be did not depend on message_form.h, so
  the sanitized and big-endian builds ran a stale binary.

And generated/bench/tables/cpp/BenchTableTable.h, regenerated: the tables bench
corpus carries the message surface like every other generated unit.
rowan-claude pushed a commit that referenced this pull request Sep 4, 2026
The 82 Table sources under testdata/golden/tables/ are re-pinned: the Table
emitter legitimately changed, which is what `make update-goldens` is for and
what the gate's own comment says of it.

THE NODE TABLE'S SLOT is emitted into a unit that HAS a node table and into no
other. The reserved node-table ID rides in every unit, because every reader
owes §3.1's refusal of it inside a nested body whoever wrote the body it is
handed; the SLOT is the writer's half, only a pointered message ever names it,
and a value-only unit carrying one is exactly what the zero-cost gate refuses.
It caught this.

And the lint half: errors.AsType where the refusal test asserted on a bare
error type, a switch where the vector index's line check had a De Morgan knot
in it, and vocabgen's two counted loops as range-over-int.

docs/VERSIONING.md said the two tool verbs are file-form tools and stay that
way, which is no longer the present state: they write and read the FILE form
and reach the message form when asked, `pack --message [--announce]` and
`unpack --announce`.
rowan-claude pushed a commit that referenced this pull request Sep 4, 2026
Every other rule in the subsection is carried by the C++ reference, the engine
and the tool as of this branch. Retention is not: §6.6 is NOT BUILT in any
language, on its own status line, so the SaveRetain refusal is a rule with no
surface to refuse in. The page says so where a reader meets it, rather than
leaving a reader to find §6.6's line for themselves.

And internal/tablewire.Trailer is a name and a doc comment rather than a
TrailerForTest hook: splitting a FILE wire into its body and its id table is
the file form's half of what Resolve needs, and the message form's half is the
connection's table, which is the whole reason the two forms can be compared
under resolution at all.
rowan-claude added a commit that referenced this pull request Sep 4, 2026
The form itself (docs/SPEC-TABLES.md §3.3). ir derives the unit's VOCABULARY in
the cook projection's order with §3.3's unconditional tail, and the ANNOUNCEMENT
that carries it: an ordinary form 1 file whose one field is the build version
under the reserved id 0xFFFFFFFFFFFFFFFE and whose trailer IS the connection's
table.

internal/tablewire gains the message path: a Vocabulary that one announcement
sets and a second cannot, the bound read from the entry count before an entry
is touched, the tolerant announcement read with its one strict check, the
encode and decode of a form 2 wire against that table, and the RESOLVED form
the two wire forms are compared under.

The C++ emitter emits the announcement as a compile-time constant byte array
and its length, the connection's table, the three unit-scope entry points and
the three suffixes. A generated field header carries its SLOT as a literal
beside its id, so a form 2 save does no lookup at all, and the variable-class
load path is one emitter over both forms.

And the corpora the ruling measured: backenddemo at tables/backend, and
vocabdemo at tables/vocab, generated by test/vocabgen so its vocabulary passes
127 ids and its message names slots on both sides of the boundary.
rowan-claude added a commit that referenced this pull request Sep 4, 2026
…#530)

The corpus gains the twelve vectors the page describes and the four that carry
rules a value alone cannot reach: a wide vocabulary, a pointered root, a
two-peer pair for per-direction independence, and a connection carrying a
second announcement. Two manifest line kinds carry them, `connection` and
`message`.

test/tables/message_form.h is the C++ reference's own half, pinning every wire
and running the refusals, the announcement's one strict check with its
tolerance, the reserved id in a body and in a nested body, the reference bound
and the last legal slot.

message_test.go is the engine's, and it is a THIRD reading of the same bytes:
the committed announcement against the one the unit derives, the tail, the two
forms under resolution, the round trip, and every refusal by name.
rowan-claude added a commit that referenced this pull request Sep 4, 2026
…#530)

§11's claimed set gains what §3.3 says it is owed: TableVocabulary, the
unit-scope Announce, AnnounceMeasure and AnnounceRead, the four refusal reasons
beside the form byte's own newer_form, and the three suffixes MeasureMessage,
SaveMessage and LoadMessage in tableGeneratedVerbs.

The checker holds back TWO ids now rather than one: the node table's
0xFFFFFFFFFFFFFFFF and the announcement's build version 0xFFFFFFFFFFFFFFFE,
refused for a field name, a `was` alias and a table name alike, each under the
planted collision that is the only input able to reach the refusal at all.

And the tool speaks both forms: `pack --message` writes one, `--announce`
writes the unit's announcement beside it, and `unpack --announce` reads a
message back against that announcement, because the table is the other half of
the wire.
rowan-claude added a commit that referenced this pull request Sep 4, 2026
#530)

A `message` surface, one case per message the manifest pins: the driver reads
the connection's announcement into one direction's table, loads the message
against it, and saves it back. The expectation is the message golden itself,
exactly as the wire surface's is the wire golden.

The C++ leg answers it for a fixed root and for a pointered one, and the eight
ports print ABSENT, which is the wire form's own absence one grain up.

The file-form vectors ride every surface an instance rides, the text form
included, so the corpus gains their JSON. The message form's vectors need none
of their own: their text is the file-form vector's byte for byte, because the
value is the same.
rowan-claude added a commit that referenced this pull request Sep 4, 2026
The fuzzer's frame locates every number in a saved table. A message has no
trailer to locate and none to mutate, so the whole of its attack surface is the
body, and the entries it resolves against are the connection's rather than its
own.

Every message the manifest pins becomes a seed, which is what §3.3 asks for by
name: the reference pass run with the table ANNOUNCED, every reference set to
the entry count plus one and to the extremes the encoding can spell, and to the
entry count itself, which is the last legal slot and must resolve.
rowan-claude pushed a commit that referenced this pull request Sep 5, 2026
The bit stream first: a TableBitWriter and a TableBitReader in the packet
wire's own layout, bit i in byte i/8 low bit first, with a bit LEB128 beside
them for the numbers no declaration bounds.

Then the per-entry RECORD the announcement now carries, borrowed from its bytes
like the entries: a fixed-stride array a reader indexes rather than searches,
and one generic TableMessageSkip over it, because a record says everything a
skipper needs and one function then serves every table.

Then the codec: <T>MeasureMessageBody, <T>SaveMessageBody and
<T>LoadMessageBody per table, and the batch's own surface per root —
MeasureMessages, SaveMessages and LoadMessages beside the batch-of-one
spellings. Every reference is a compile-time SLOT and every width a literal on
the write side, so a save does no lookup at all.

AnnounceRead gains the second strict check: the records present, exactly once,
at kind 12, one record an entry. A table with no records is a table nothing can
be read against.

The C++ reference and the compiler's engine agree BYTE FOR BYTE, and measure
equals save, on all eight value-class message vectors of the corpus:

  login_full     106 file  ->  51    match_full  273 file -> 143
  login_default   10 file  ->   3    match_default 43 file ->  10
  store_full     104 file  ->  41    vocab_low    24 file ->   8
  store_default   10 file  ->   3    vocab_wide  192 file ->  68

WHAT IT CARRIES TODAY is the value class with no map. A pointered message and a
map's cursor both take a resolution context this codec does not thread, and a
root whose closure it cannot carry gets no message entry points at all rather
than half of one.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 5, 2026
…count, the align (#530)

The page (#523) settles the wire, and this is the engine reworked onto it,
where it is more specific than the brief was.

THE VOCABULARY IS A FIELD of the announcement's body, not its trailer, and an
entry is a TRIPLE: an id, a kind, and a SHAPE. That buys three things the
brief's fixed-stride records could not. §3's writer rule that an id no body
references is never written is restored unbroken. An entry carries the widths
and the RANGE a reader needs to decode a field whose declaration has MOVED, so
every evolution row of §4 stands under a bitpacked body. And ONE NAME MAY TAKE
TWO SLOTS, at two kinds or two shapes, so a unit declaring `count uint8` in one
table and `count uint32` in another needs no canonical widening and no
ambiguity rule at all.

The rest of the page's wire, exactly: the count is a ranged integer over
[1, 256] carrying M - 1, a batch of zero is not spellable, a `string(N)` and a
`bytes(N)` ALIGN before their bytes, a fixed array spends NO count because its
min equals its max, a compressed float rides QUANTIZED as the packet wire
writes it, a pointer index is bits_required(0, node count) and the node table
is the root body's FIRST field, the trailing pad is verified zero, and damage
is TERMINAL for the batch.

The engine now reproduces the page's hand-worked arithmetic exactly:

  login_full     106 file -> 51    match_full   273 file -> 142
  login_default   10 file ->  3    match_default 43 file ->  10
  store_full     104 file -> 41    store_default 10 file ->   3
  the three as one batch -> 230    graph_tree   232 file ->  66
  vocab_low       24 file ->  8    vocab_wide   192 file ->  68
  the announcement: 28 entries, 273 bytes of them, 316 bytes whole

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 5, 2026
…he bitpacked body (#530)

The primitive is a NUMBER OF MESSAGES. EncodeMessages writes one buffer: the
form byte, the count, and the bodies as one continuous bit stream with no
alignment between them, padded to a byte at the end and nowhere else. A single
message is the batch of one, which is the only sense in which this wire carries
one.

And the body is BITPACKED. References ride in bits_required(entries), there is
no kind byte at all, values ride at their declared widths as the packet wire
writes them, lengths ride in bits_required(N), and elision is unchanged.

What a reader needs to skip an id it cannot name is therefore not on the body:
it is the ANNOUNCEMENT's per-entry RECORD, a fixed-stride array under a third
reserved id, carrying the kind, the two widths and the range base each slot
spells. The record is the WIRE CONTRACT for its id and both halves write to it,
so one field name declared at two bounds in two records rides at the widest of
them rather than at either.

The measurements, against the byte body #549 landed:

  login_full      106 file    58 byte body    51 bitpacked   49 proto3
  match_full      273 file   225 byte body   143 bitpacked  189 proto3
  store_full      104 file    48 byte body    41 bitpacked   40 proto3
  login_default    10 file     2 byte body     3 bitpacked
  match_default    43 file    27 byte body    10 bitpacked   40 proto3
  store_default    10 file     2 byte body     3 bitpacked
  the three as ONE BATCH                     230 bitpacked

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 5, 2026
`pack --message` writes a batch and `unpack --announce` reads one, because the
primitive is a number of messages and a single message is the batch of one.
`--batch <Table>=<tree-dir>` names each message after the first, in the order it
rides: which root a message is, is the application's, so the tool asks rather
than guessing.

The two verbs' single-message spellings are unchanged and are now the batch of
one, and `unpack` refuses a batch whose count is not the number of roots named
rather than writing a tree it cannot fill.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 5, 2026
The bit stream first: a TableBitWriter and a TableBitReader in the packet
wire's own layout, bit i in byte i/8 low bit first, with a bit LEB128 beside
them for the numbers no declaration bounds.

Then the per-entry RECORD the announcement now carries, borrowed from its bytes
like the entries: a fixed-stride array a reader indexes rather than searches,
and one generic TableMessageSkip over it, because a record says everything a
skipper needs and one function then serves every table.

Then the codec: <T>MeasureMessageBody, <T>SaveMessageBody and
<T>LoadMessageBody per table, and the batch's own surface per root —
MeasureMessages, SaveMessages and LoadMessages beside the batch-of-one
spellings. Every reference is a compile-time SLOT and every width a literal on
the write side, so a save does no lookup at all.

AnnounceRead gains the second strict check: the records present, exactly once,
at kind 12, one record an entry. A table with no records is a table nothing can
be read against.

The C++ reference and the compiler's engine agree BYTE FOR BYTE, and measure
equals save, on all eight value-class message vectors of the corpus:

  login_full     106 file  ->  51    match_full  273 file -> 143
  login_default   10 file  ->   3    match_default 43 file ->  10
  store_full     104 file  ->  41    vocab_low    24 file ->   8
  store_default   10 file  ->   3    vocab_wide  192 file ->  68

WHAT IT CARRIES TODAY is the value class with no map. A pointered message and a
map's cursor both take a resolution context this codec does not thread, and a
root whose closure it cannot carry gets no message entry points at all rather
than half of one.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 5, 2026
…count, the align (#530)

The page (#523) settles the wire, and this is the engine reworked onto it,
where it is more specific than the brief was.

THE VOCABULARY IS A FIELD of the announcement's body, not its trailer, and an
entry is a TRIPLE: an id, a kind, and a SHAPE. That buys three things the
brief's fixed-stride records could not. §3's writer rule that an id no body
references is never written is restored unbroken. An entry carries the widths
and the RANGE a reader needs to decode a field whose declaration has MOVED, so
every evolution row of §4 stands under a bitpacked body. And ONE NAME MAY TAKE
TWO SLOTS, at two kinds or two shapes, so a unit declaring `count uint8` in one
table and `count uint32` in another needs no canonical widening and no
ambiguity rule at all.

The rest of the page's wire, exactly: the count is a ranged integer over
[1, 256] carrying M - 1, a batch of zero is not spellable, a `string(N)` and a
`bytes(N)` ALIGN before their bytes, a fixed array spends NO count because its
min equals its max, a compressed float rides QUANTIZED as the packet wire
writes it, a pointer index is bits_required(0, node count) and the node table
is the root body's FIRST field, the trailing pad is verified zero, and damage
is TERMINAL for the batch.

The engine now reproduces the page's hand-worked arithmetic exactly:

  login_full     106 file -> 51    match_full   273 file -> 142
  login_default   10 file ->  3    match_default 43 file ->  10
  store_full     104 file -> 41    store_default 10 file ->   3
  the three as one batch -> 230    graph_tree   232 file ->  66
  vocab_low       24 file ->  8    vocab_wide   192 file ->  68
  the announcement: 28 entries, 273 bytes of them, 316 bytes whole

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 5, 2026
#530)

The uncommitted tree the previous builder left at 70d44af, measured, built
and committed as it stood: the Go engine builds, the conformance harness is
green on every message row, and the C++ tables test builds and passes.

What it carries: the variable class on the message wire in C++ (the node
table as the root body's first field, thirty-two bit counts, blob records
with a length and an align, index widths settled per body, maps carved from
the framing), the batch's three verbs over a region with one region per
batch, wstring at sixteen bits a unit and the escape kind, the batch's five
answers with batch_too_large on both sides, the wrong-sort rule for variant
and arm references with the reserved-id rule outranking it, the three
reserved ids refused in the vocabulary, the vocab9demo unit for a nine-bit
reference, the fuzzer's bit-level reference pass, schema unpack printing an
announcement, and the sabotage tool's message overlays for every harness row.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 5, 2026
…e batch, and schema#571 (#530)

Merges origin/main (#562, #564, #573) and lands the page's second round
(#575) on both engines: a ranged base encoded by its kind's signedness, the
quantized f32 row as min, max and res in float32 with the step count and the
width derived by SPEC.md §4.3's rule, quantization in float32 with two
roundings on each side, a refused first announcement terminal, and the
Envelope over a union of the three messages as the batch vector (244 bytes,
the singles at 52, 148 and 43, the announcement at 361).

Unbounded arrays ride the C++ message wire with the count the data decides
and the elements carved from the node's extent.

schema#571's six findings are fixed with a vector and a control each: surplus
elements decode into scratch (M1), a ranged 128-bit value is one arithmetic
for measure, write and read (M2), a width above the kind's domain is refused
in both shape parsers and the bit reader (M3), the Go engine writes the count
as its offset from the minimum (M4), one quantization rule (M5), and the C++
reader clamps while wide and narrows after (M6). The bases unit
(test/tables/Bases.schema) holds the vectors in both engines.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 5, 2026
…s header (#530)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 5, 2026
…nd the fuzz legs at the wire's M (#530)

Resumed from the previous builder's working tree, unchanged in intent.

A FIXED root numbers no node, so the index width it hands its body is zero
rather than a constant the body invents: LoadMessageBody takes index_bits on
both the fixed and the variable path, and a kind-17 entry inside a fixed body
is damage because there is no width to step it over. The Go reader matches by
setting the width to zero at a fixed root and refusing an index read at width
zero.

A ROOT body whose own framing gives out is damage inside that body, not a
whole-batch refusal: the scan answers the bodies up to and including it, and
MeasureMessages sizes through it and no further, which is the region the load
needs to deliver the bodies before the damage.

The fuzz legs hold room for the wire's M so a capacity refusal is never the
answer under fuzz, and the region check rejects a negative byte count before
memset.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 5, 2026
…he bitpacked body (#530)

The primitive is a NUMBER OF MESSAGES. EncodeMessages writes one buffer: the
form byte, the count, and the bodies as one continuous bit stream with no
alignment between them, padded to a byte at the end and nowhere else. A single
message is the batch of one, which is the only sense in which this wire carries
one.

And the body is BITPACKED. References ride in bits_required(entries), there is
no kind byte at all, values ride at their declared widths as the packet wire
writes them, lengths ride in bits_required(N), and elision is unchanged.

What a reader needs to skip an id it cannot name is therefore not on the body:
it is the ANNOUNCEMENT's per-entry RECORD, a fixed-stride array under a third
reserved id, carrying the kind, the two widths and the range base each slot
spells. The record is the WIRE CONTRACT for its id and both halves write to it,
so one field name declared at two bounds in two records rides at the widest of
them rather than at either.

The measurements, against the byte body #549 landed:

  login_full      106 file    58 byte body    51 bitpacked   49 proto3
  match_full      273 file   225 byte body   143 bitpacked  189 proto3
  store_full      104 file    48 byte body    41 bitpacked   40 proto3
  login_default    10 file     2 byte body     3 bitpacked
  match_default    43 file    27 byte body    10 bitpacked   40 proto3
  store_default    10 file     2 byte body     3 bitpacked
  the three as ONE BATCH                     230 bitpacked

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 5, 2026
`pack --message` writes a batch and `unpack --announce` reads one, because the
primitive is a number of messages and a single message is the batch of one.
`--batch <Table>=<tree-dir>` names each message after the first, in the order it
rides: which root a message is, is the application's, so the tool asks rather
than guessing.

The two verbs' single-message spellings are unchanged and are now the batch of
one, and `unpack` refuses a batch whose count is not the number of roots named
rather than writing a tree it cannot fill.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 5, 2026
The bit stream first: a TableBitWriter and a TableBitReader in the packet
wire's own layout, bit i in byte i/8 low bit first, with a bit LEB128 beside
them for the numbers no declaration bounds.

Then the per-entry RECORD the announcement now carries, borrowed from its bytes
like the entries: a fixed-stride array a reader indexes rather than searches,
and one generic TableMessageSkip over it, because a record says everything a
skipper needs and one function then serves every table.

Then the codec: <T>MeasureMessageBody, <T>SaveMessageBody and
<T>LoadMessageBody per table, and the batch's own surface per root —
MeasureMessages, SaveMessages and LoadMessages beside the batch-of-one
spellings. Every reference is a compile-time SLOT and every width a literal on
the write side, so a save does no lookup at all.

AnnounceRead gains the second strict check: the records present, exactly once,
at kind 12, one record an entry. A table with no records is a table nothing can
be read against.

The C++ reference and the compiler's engine agree BYTE FOR BYTE, and measure
equals save, on all eight value-class message vectors of the corpus:

  login_full     106 file  ->  51    match_full  273 file -> 143
  login_default   10 file  ->   3    match_default 43 file ->  10
  store_full     104 file  ->  41    vocab_low    24 file ->   8
  store_default   10 file  ->   3    vocab_wide  192 file ->  68

WHAT IT CARRIES TODAY is the value class with no map. A pointered message and a
map's cursor both take a resolution context this codec does not thread, and a
root whose closure it cannot carry gets no message entry points at all rather
than half of one.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 5, 2026
…count, the align (#530)

The page (#523) settles the wire, and this is the engine reworked onto it,
where it is more specific than the brief was.

THE VOCABULARY IS A FIELD of the announcement's body, not its trailer, and an
entry is a TRIPLE: an id, a kind, and a SHAPE. That buys three things the
brief's fixed-stride records could not. §3's writer rule that an id no body
references is never written is restored unbroken. An entry carries the widths
and the RANGE a reader needs to decode a field whose declaration has MOVED, so
every evolution row of §4 stands under a bitpacked body. And ONE NAME MAY TAKE
TWO SLOTS, at two kinds or two shapes, so a unit declaring `count uint8` in one
table and `count uint32` in another needs no canonical widening and no
ambiguity rule at all.

The rest of the page's wire, exactly: the count is a ranged integer over
[1, 256] carrying M - 1, a batch of zero is not spellable, a `string(N)` and a
`bytes(N)` ALIGN before their bytes, a fixed array spends NO count because its
min equals its max, a compressed float rides QUANTIZED as the packet wire
writes it, a pointer index is bits_required(0, node count) and the node table
is the root body's FIRST field, the trailing pad is verified zero, and damage
is TERMINAL for the batch.

The engine now reproduces the page's hand-worked arithmetic exactly:

  login_full     106 file -> 51    match_full   273 file -> 142
  login_default   10 file ->  3    match_default 43 file ->  10
  store_full     104 file -> 41    store_default 10 file ->   3
  the three as one batch -> 230    graph_tree   232 file ->  66
  vocab_low       24 file ->  8    vocab_wide   192 file ->  68
  the announcement: 28 entries, 273 bytes of them, 316 bytes whole

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 5, 2026
#530)

The uncommitted tree the previous builder left at 70d44af, measured, built
and committed as it stood: the Go engine builds, the conformance harness is
green on every message row, and the C++ tables test builds and passes.

What it carries: the variable class on the message wire in C++ (the node
table as the root body's first field, thirty-two bit counts, blob records
with a length and an align, index widths settled per body, maps carved from
the framing), the batch's three verbs over a region with one region per
batch, wstring at sixteen bits a unit and the escape kind, the batch's five
answers with batch_too_large on both sides, the wrong-sort rule for variant
and arm references with the reserved-id rule outranking it, the three
reserved ids refused in the vocabulary, the vocab9demo unit for a nine-bit
reference, the fuzzer's bit-level reference pass, schema unpack printing an
announcement, and the sabotage tool's message overlays for every harness row.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 5, 2026
…s header (#530)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 5, 2026
…nd the fuzz legs at the wire's M (#530)

Resumed from the previous builder's working tree, unchanged in intent.

A FIXED root numbers no node, so the index width it hands its body is zero
rather than a constant the body invents: LoadMessageBody takes index_bits on
both the fixed and the variable path, and a kind-17 entry inside a fixed body
is damage because there is no width to step it over. The Go reader matches by
setting the width to zero at a fixed root and refusing an index read at width
zero.

A ROOT body whose own framing gives out is damage inside that body, not a
whole-batch refusal: the scan answers the bodies up to and including it, and
MeasureMessages sizes through it and no further, which is the region the load
needs to deliver the bodies before the damage.

The fuzz legs hold room for the wire's M so a capacity refusal is never the
answer under fuzz, and the region check rejects a negative byte count before
memset.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 5, 2026
#530)

The rebase left the second round's controls behind a conflict marker. The
list carries every row the page holds by test, the six findings of schema#571
included.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 6, 2026
…s it (#530)

A record's extent cursor is a local of NodeMessageBody, and the node map it
was published through is the caller's, so a return with the map still naming
it leaves an address the caller could read. The dispatch answers into a bool
and the cursor is restored on the way out.

gcc 13 on the big-endian leg names this exactly, -Werror=dangling-pointer at
the message node body, and the fix is the escape rather than the flag.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 6, 2026
The committed tree is what this branch's compiler emits: the message codec's
node carve restored on the way out, and main's extent names.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 6, 2026
#530)

WHAT THE GATE ASSERTS is the BLOCK form's zero cost, in three parts: no Table
source carries one block symbol, the C# Table sources carry no build version,
and the frozen Table pins are byte-identical. The first two are the property
and both stand untouched. The third is a golden, and the gate's own text says
a golden is re-pinned when a TABLE emitter legitimately changes.

The message codec is such a change and it reaches EVERY unit, because the
three message verbs are in tableGeneratedVerbs and every table carries them
(§3.3). Every unit's header moves for the same two reasons: `batch_too_large`
joins the refusal reasons, and TableIds loses the `vocabulary` flag and its
two-argument `ref` because a bitpacked body names no id at all — its
references are compile-time slots of the announced vocabulary, so the
byte-framed form's shared id table has nothing left to do. That is the byte
body being replaced in place, which §3.3 states.

The C# and C Table goldens do NOT move, which is the check that the change
reached only the C++ reference: the eight ports carry the file form alone.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 6, 2026
…et a harness (#530)

The clamp of the rounded index to `count` is the packet wire's own last step
and is the same answer written shorter, which is what the lint asks for. The
float clamp above it keeps its negated comparisons, because those are what
send a NaN to zero rather than through.

`make tables-message-cost` measures what the page holds the form to: the
bytes each form spends over the three backend messages and the batch, and the
read and write factor the bitpacked body takes against the byte-framed body
over the same values. It is a measurement and not a gate, so it is not in
`test` — a ratio moves with the machine.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 6, 2026
THE ESTATE HAS ONE BENCHMARK and the shape gate holds that mechanically: no
timing primitive outside the sanctioned runner and tool directories. A
hand-written harness under test/ is exactly the divergence class the rule
names, and the gate refused it by name.

The number the cost rule owes is taken off-tree for now and quoted in the PR
body with its sitting. Its reproducible home is the sanctioned bench, a
message-form arm beside the tolerant-wire arm in bench/tables, which needs a
pinned message variant corpus and a row in the per-language board and is
therefore its own change.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 6, 2026
…emcpy (#530)

MEASURED FIRST. On the three backend messages and the batch, the bitpacked
body was writing at 7.9x to 11.6x the byte-framed body's time and reading at
4.2x to 6.2x, against the factor of TWO the page holds the form to. The cause
was in the primitive rather than in the codec: put and get walked ONE BIT at a
time with a division and a modulo per bit, and putbytes spent eight such
iterations per byte on the largest payload the wire carries.

§3.3 already says what the align is for: a string(N), a bytes(N) and a blob
record align before their bytes so the payload moves as one memcpy. It now
does. put and get take the bits that fall in a byte together, so a 64-bit
field costs nine touches rather than sixty-four and no division at all, and
align spends its pad in one call.

THE BITS ON THE WIRE DO NOT MOVE — bit i in byte i/8 at position i%8, low bit
first — and every pinned golden is the proof: the corpus, the round trips and
the batch vectors all reproduce byte for byte.

After: write 1.27x to 2.11x, read 2.05x to 3.54x. The write path is inside
the rule and the read path is not yet; the measurement and the cause are in
the PR body.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 6, 2026
The committed tree is what this branch's compiler emits.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 6, 2026
…bit stream moves a word at a time (#530)

docs/SPEC-TABLES.md §3.3 says THE RECEIVER RESOLVES ONCE: it reads the
announcement, resolves every entry against its own descriptors, and every body
after it dispatches through one array index. The reader did not. It kept the
OFFSET of each entry inside the announcement's bytes and re-parsed the entry, a
LEB128 shape decode, on every field of every body. That is the code failing the
page, and it is why the read factor sat above the page's bound.

THE STORAGE SHAPE IS WHAT THE SENTENCE NEEDED, and the page states it as a rule
now. The announcement is delivered once and holds for the connection's life, so
the resolved vocabulary lives with the CONNECTION and not with a read call:
the caller declares an array of resolved entries wherever it wants it, hands it
to AnnounceRead with its CAPACITY, and passes it by pointer to every read after
that. Nothing in the codec allocates. THE ENTRY BOUND IS THAT CAPACITY, so an
announcement above it is refused by name as vocabulary_too_large before an
entry is touched, and kTableMessageEntriesHere gives the unit's own entry count
for a receiver that talks only to peers of its own schema. 4096 stays the
conforming default where storage is grown rather than declared.

A RESOLVED ENTRY IS 96 BYTES: the id, the kind, the packing, the payload's
width ALREADY RESOLVED out of the kind and the announced bits, the range base,
the array bounds, and the quantized facts SPEC.md §4.3's rule leaves behind
rather than the two it consumes. TableVocabulary went from 16 KB of offsets by
value to 56 bytes, and the backenddemo unit's 33 entries take 3,168 bytes.
Per field on the read path that is one array index, no parse and no branch.

AND THE BIT STREAM MOVES A WORD AT A TIME, which is what remained once the
vocabulary was resolved: the stream walked one BYTE at a time, so a bare uint64
cost nine masked read-modify-writes to write and nine dependent loads to read,
and MatchResult carries eleven of them. A value of any width now moves in one
unaligned load or store, assembled little-end-first whatever order the host is
in, with the byte walk kept for a value inside nine bytes of the buffer's end.
put still writes EXACTLY the bytes the value occupies, so a caller's buffer
beyond the batch is still its own.

THE BITS ON THE WIRE DID NOT MOVE. Every pinned form-2 golden holds, and the
31 C++ Table pins that re-pin here move for the emitter's own text.

The cost rule is met on both paths, and the sitting is on the page beside the
factor it is held to: write 0.71x to 0.90x and read 1.56x to 1.77x against the
byte body, where head measured 1.54x to 1.95x and 2.44x to 3.52x on the same
sitting.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 6, 2026
The message form's node table read reserved its record slice from the
count it had just read off the wire. That count is a thirty-two bit
number off hostile bytes, so a mutated batch commanded gigabytes before
a single record had been read, which is the opposite of what §3.1 means
by load is a scan. The file form already had the discipline: decode.go
checks its count against the bytes it holds before it allocates.

The reservation is now the smaller of the claim and what the stream can
carry, a record costing at least its type reference. bitReader.left
reports the bits still unread, which is the bound.

The wire fuzzer is what this cost. At the N make test runs, the harness
peaked at 10.7 GB and was killed for memory before it finished. It now
peaks at 158 MB and runs at 19,000 mutants a second where it managed
5,000, so the pass is both survivable and three times deeper.

The placeable set is named on its own line here too, as the file
reader names its own, because it is the seam the node-type negative
control replaces.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 6, 2026
…530)

The target could not pass as written. The generator echoes its own --out
into the file's second line, so generating into build/vocabgen and
comparing against tables/vocab differed on that line by construction,
whatever else matched.

The generator now runs from build/vocabgen under the committed file's
own relative path, so the --out it sees is the --out the committed file
carries. cmp then compares the whole file, and the regenerate line in
the committed header is checked by the same comparison as the tables
below it. Rewriting that line in the committed file turns the target
red, which it could not do before.

This is the smaller of the two fixes. The other was to make the header
path independent, which would have meant teaching the generator a
canonical destination it does not otherwise need.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude pushed a commit that referenced this pull request Sep 6, 2026
…rule (#530)

Two negative controls could not go red. Both sabotaged
internal/tablewire/decodenodes.go, which is the FILE reader, while the
vectors they assert on are message vectors. The message form's second
round built the placeable set a second time, in messagedecode.go, and
that is the copy a message vector decodes through, so the sabotage
patched a file the vectors never touch: 111,269 mutants, 0 divergences,
and the vector replayed alone gave 0 as well.

The node-type control now sabotages messagedecode.go. It also asserts by
replaying the pinned vector rather than by running the corpus, because
the pinned vectors ride last: an enumerated mutant of an ordinary message
seed reaches the same check, so a corpus pass goes red before the vector
is ever fed and names the mutant instead of the property. With the
sabotage the replay reports

  the report differs: the leg says 1,0,0,0,false,read,
  the oracle says 3,1,0,0,false,read

and without it the same replay is 1 mutant, 0 divergences.

The vector itself was re-derived. The second round moved the message
bodies to a packed bit stream, so every message wire in testdata was
regenerated and the old 104 byte vector stopped being a wire at all. The
new one is 66 bytes, a mutant of graph_tree_message that the reader with
ir.PointerReachable and the reader without answer differently.

THE BLOB NODE CONTROL IS GONE, and its vector with it, removed by
accident in the previous commit and explained here. Under the bitpacked
body a blob record carries a thirty-two bit length that must fit the
batch's remaining bits. A mutated record does name a reserved blob id,
reached at random mutant 11993 of the default seed, and the reader still
never places a blob: over 2,111,269 mutants with both reserved ids forced
nameable, a placement never happened once. The rule stays gated by
ir.PointerReachableBlobs in the reference emitter's NodeStorage and in
the oracle, and the node-type control above covers the same rule at the
ids that are not reserved, which the corpus does reach.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
gafferongames added a commit that referenced this pull request Sep 6, 2026
#530) (#557)

* wire: the message form's second round in the engine — the batch and the bitpacked body (#530)

The primitive is a NUMBER OF MESSAGES. EncodeMessages writes one buffer: the
form byte, the count, and the bodies as one continuous bit stream with no
alignment between them, padded to a byte at the end and nowhere else. A single
message is the batch of one, which is the only sense in which this wire carries
one.

And the body is BITPACKED. References ride in bits_required(entries), there is
no kind byte at all, values ride at their declared widths as the packet wire
writes them, lengths ride in bits_required(N), and elision is unchanged.

What a reader needs to skip an id it cannot name is therefore not on the body:
it is the ANNOUNCEMENT's per-entry RECORD, a fixed-stride array under a third
reserved id, carrying the kind, the two widths and the range base each slot
spells. The record is the WIRE CONTRACT for its id and both halves write to it,
so one field name declared at two bounds in two records rides at the widest of
them rather than at either.

The measurements, against the byte body #549 landed:

  login_full      106 file    58 byte body    51 bitpacked   49 proto3
  match_full      273 file   225 byte body   143 bitpacked  189 proto3
  store_full      104 file    48 byte body    41 bitpacked   40 proto3
  login_default    10 file     2 byte body     3 bitpacked
  match_default    43 file    27 byte body    10 bitpacked   40 proto3
  store_default    10 file     2 byte body     3 bitpacked
  the three as ONE BATCH                     230 bitpacked

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* tool: pack and unpack over the message form's BATCH (#530)

`pack --message` writes a batch and `unpack --announce` reads one, because the
primitive is a number of messages and a single message is the batch of one.
`--batch <Table>=<tree-dir>` names each message after the first, in the order it
rides: which root a message is, is the application's, so the tool asks rather
than guessing.

The two verbs' single-message spellings are unchanged and are now the batch of
one, and `unpack` refuses a batch whose count is not the number of roots named
rather than writing a tree it cannot fill.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* wire: the C++ reference's bitpacked message codec and its batch (#530)

The bit stream first: a TableBitWriter and a TableBitReader in the packet
wire's own layout, bit i in byte i/8 low bit first, with a bit LEB128 beside
them for the numbers no declaration bounds.

Then the per-entry RECORD the announcement now carries, borrowed from its bytes
like the entries: a fixed-stride array a reader indexes rather than searches,
and one generic TableMessageSkip over it, because a record says everything a
skipper needs and one function then serves every table.

Then the codec: <T>MeasureMessageBody, <T>SaveMessageBody and
<T>LoadMessageBody per table, and the batch's own surface per root —
MeasureMessages, SaveMessages and LoadMessages beside the batch-of-one
spellings. Every reference is a compile-time SLOT and every width a literal on
the write side, so a save does no lookup at all.

AnnounceRead gains the second strict check: the records present, exactly once,
at kind 12, one record an entry. A table with no records is a table nothing can
be read against.

The C++ reference and the compiler's engine agree BYTE FOR BYTE, and measure
equals save, on all eight value-class message vectors of the corpus:

  login_full     106 file  ->  51    match_full  273 file -> 143
  login_default   10 file  ->   3    match_default 43 file ->  10
  store_full     104 file  ->  41    vocab_low    24 file ->   8
  store_default   10 file  ->   3    vocab_wide  192 file ->  68

WHAT IT CARRIES TODAY is the value class with no map. A pointered message and a
map's cursor both take a resolution context this codec does not thread, and a
root whose closure it cannot carry gets no message entry points at all rather
than half of one.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* wire: the engine follows the page — entries with shapes, the batch's count, the align (#530)

The page (#523) settles the wire, and this is the engine reworked onto it,
where it is more specific than the brief was.

THE VOCABULARY IS A FIELD of the announcement's body, not its trailer, and an
entry is a TRIPLE: an id, a kind, and a SHAPE. That buys three things the
brief's fixed-stride records could not. §3's writer rule that an id no body
references is never written is restored unbroken. An entry carries the widths
and the RANGE a reader needs to decode a field whose declaration has MOVED, so
every evolution row of §4 stands under a bitpacked body. And ONE NAME MAY TAKE
TWO SLOTS, at two kinds or two shapes, so a unit declaring `count uint8` in one
table and `count uint32` in another needs no canonical widening and no
ambiguity rule at all.

The rest of the page's wire, exactly: the count is a ranged integer over
[1, 256] carrying M - 1, a batch of zero is not spellable, a `string(N)` and a
`bytes(N)` ALIGN before their bytes, a fixed array spends NO count because its
min equals its max, a compressed float rides QUANTIZED as the packet wire
writes it, a pointer index is bits_required(0, node count) and the node table
is the root body's FIRST field, the trailing pad is verified zero, and damage
is TERMINAL for the batch.

The engine now reproduces the page's hand-worked arithmetic exactly:

  login_full     106 file -> 51    match_full   273 file -> 142
  login_default   10 file ->  3    match_default 43 file ->  10
  store_full     104 file -> 41    store_default 10 file ->   3
  the three as one batch -> 230    graph_tree   232 file ->  66
  vocab_low       24 file ->  8    vocab_wide   192 file ->  68
  the announcement: 28 entries, 273 bytes of them, 316 bytes whole

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* wire: the message form's second round, resumed from the builder's tree (#530)

The uncommitted tree the previous builder left at 70d44af, measured, built
and committed as it stood: the Go engine builds, the conformance harness is
green on every message row, and the C++ tables test builds and passes.

What it carries: the variable class on the message wire in C++ (the node
table as the root body's first field, thirty-two bit counts, blob records
with a length and an align, index widths settled per body, maps carved from
the framing), the batch's three verbs over a region with one region per
batch, wstring at sixteen bits a unit and the escape kind, the batch's five
answers with batch_too_large on both sides, the wrong-sort rule for variant
and arm references with the reserved-id rule outranking it, the three
reserved ids refused in the vocabulary, the vocab9demo unit for a nine-bit
reference, the fuzzer's bit-level reference pass, schema unpack printing an
announcement, and the sabotage tool's message overlays for every harness row.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* generated: the bench tables unit regenerated, the message codec on its header (#530)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* wire: the fixed body's index width, the sized-through-damage batch, and the fuzz legs at the wire's M (#530)

Resumed from the previous builder's working tree, unchanged in intent.

A FIXED root numbers no node, so the index width it hands its body is zero
rather than a constant the body invents: LoadMessageBody takes index_bits on
both the fixed and the variable path, and a kind-17 entry inside a fixed body
is damage because there is no width to step it over. The Go reader matches by
setting the width to zero at a fixed root and refusing an index read at width
zero.

A ROOT body whose own framing gives out is damage inside that body, not a
whole-batch refusal: the scan answers the bodies up to and including it, and
MeasureMessages sizes through it and no further, which is the region the load
needs to deliver the bodies before the damage.

The fuzz legs hold room for the wire's M so a capacity refusal is never the
answer under fuzz, and the region check rejects a negative byte count before
memset.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* wire: the message form's node carve never outlives the frame that owns it (#530)

A record's extent cursor is a local of NodeMessageBody, and the node map it
was published through is the caller's, so a return with the map still naming
it leaves an address the caller could read. The dispatch answers into a bool
and the cursor is restored on the way out.

gcc 13 on the big-endian leg names this exactly, -Werror=dangling-pointer at
the message node body, and the fix is the escape rather than the flag.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* generated: the bench tables unit on the merged compiler (#530)

The committed tree is what this branch's compiler emits: the message codec's
node carve restored on the way out, and main's extent names.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* goldens: the 27 C++ Table pins re-taken on the bitpacked message codec (#530)

WHAT THE GATE ASSERTS is the BLOCK form's zero cost, in three parts: no Table
source carries one block symbol, the C# Table sources carry no build version,
and the frozen Table pins are byte-identical. The first two are the property
and both stand untouched. The third is a golden, and the gate's own text says
a golden is re-pinned when a TABLE emitter legitimately changes.

The message codec is such a change and it reaches EVERY unit, because the
three message verbs are in tableGeneratedVerbs and every table carries them
(§3.3). Every unit's header moves for the same two reasons: `batch_too_large`
joins the refusal reasons, and TableIds loses the `vocabulary` flag and its
two-argument `ref` because a bitpacked body names no id at all — its
references are compile-time slots of the announced vocabulary, so the
byte-framed form's shared id table has nothing left to do. That is the byte
body being replaced in place, which §3.3 states.

The C# and C Table goldens do NOT move, which is the check that the change
reached only the C++ reference: the eight ports carry the file form alone.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* wire: the quantizer's integer clamp reads as one, and the cost rows get a harness (#530)

The clamp of the rounded index to `count` is the packet wire's own last step
and is the same answer written shorter, which is what the lint asks for. The
float clamp above it keeps its negated comparisons, because those are what
send a NaN to zero rather than through.

`make tables-message-cost` measures what the page holds the form to: the
bytes each form spends over the three backend messages and the batch, and the
read and write factor the bitpacked body takes against the byte-framed body
over the same values. It is a measurement and not a gate, so it is not in
`test` — a ratio moves with the machine.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* bench: the cost harness comes back out of the tree (#530)

THE ESTATE HAS ONE BENCHMARK and the shape gate holds that mechanically: no
timing primitive outside the sanctioned runner and tool directories. A
hand-written harness under test/ is exactly the divergence class the rule
names, and the gate refused it by name.

The number the cost rule owes is taken off-tree for now and quoted in the PR
body with its sitting. Its reproducible home is the sanctioned bench, a
message-form arm beside the tolerant-wire arm in bench/tables, which needs a
pinned message variant corpus and a row in the per-language board and is
therefore its own change.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* wire: the bit stream moves a byte at a time, and the align buys its memcpy (#530)

MEASURED FIRST. On the three backend messages and the batch, the bitpacked
body was writing at 7.9x to 11.6x the byte-framed body's time and reading at
4.2x to 6.2x, against the factor of TWO the page holds the form to. The cause
was in the primitive rather than in the codec: put and get walked ONE BIT at a
time with a division and a modulo per bit, and putbytes spent eight such
iterations per byte on the largest payload the wire carries.

§3.3 already says what the align is for: a string(N), a bytes(N) and a blob
record align before their bytes so the payload moves as one memcpy. It now
does. put and get take the bits that fall in a byte together, so a 64-bit
field costs nine touches rather than sixty-four and no division at all, and
align spends its pad in one call.

THE BITS ON THE WIRE DO NOT MOVE — bit i in byte i/8 at position i%8, low bit
first — and every pinned golden is the proof: the corpus, the round trips and
the batch vectors all reproduce byte for byte.

After: write 1.27x to 2.11x, read 2.05x to 3.54x. The write path is inside
the rule and the read path is not yet; the measurement and the cause are in
the PR body.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* generated: the bench tables unit on the byte-at-a-time bit stream (#530)

The committed tree is what this branch's compiler emits.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* wire: the receiver resolves once, into the caller's storage, and the bit stream moves a word at a time (#530)

docs/SPEC-TABLES.md §3.3 says THE RECEIVER RESOLVES ONCE: it reads the
announcement, resolves every entry against its own descriptors, and every body
after it dispatches through one array index. The reader did not. It kept the
OFFSET of each entry inside the announcement's bytes and re-parsed the entry, a
LEB128 shape decode, on every field of every body. That is the code failing the
page, and it is why the read factor sat above the page's bound.

THE STORAGE SHAPE IS WHAT THE SENTENCE NEEDED, and the page states it as a rule
now. The announcement is delivered once and holds for the connection's life, so
the resolved vocabulary lives with the CONNECTION and not with a read call:
the caller declares an array of resolved entries wherever it wants it, hands it
to AnnounceRead with its CAPACITY, and passes it by pointer to every read after
that. Nothing in the codec allocates. THE ENTRY BOUND IS THAT CAPACITY, so an
announcement above it is refused by name as vocabulary_too_large before an
entry is touched, and kTableMessageEntriesHere gives the unit's own entry count
for a receiver that talks only to peers of its own schema. 4096 stays the
conforming default where storage is grown rather than declared.

A RESOLVED ENTRY IS 96 BYTES: the id, the kind, the packing, the payload's
width ALREADY RESOLVED out of the kind and the announced bits, the range base,
the array bounds, and the quantized facts SPEC.md §4.3's rule leaves behind
rather than the two it consumes. TableVocabulary went from 16 KB of offsets by
value to 56 bytes, and the backenddemo unit's 33 entries take 3,168 bytes.
Per field on the read path that is one array index, no parse and no branch.

AND THE BIT STREAM MOVES A WORD AT A TIME, which is what remained once the
vocabulary was resolved: the stream walked one BYTE at a time, so a bare uint64
cost nine masked read-modify-writes to write and nine dependent loads to read,
and MatchResult carries eleven of them. A value of any width now moves in one
unaligned load or store, assembled little-end-first whatever order the host is
in, with the byte walk kept for a value inside nine bytes of the buffer's end.
put still writes EXACTLY the bytes the value occupies, so a caller's buffer
beyond the batch is still its own.

THE BITS ON THE WIRE DID NOT MOVE. Every pinned form-2 golden holds, and the
31 C++ Table pins that re-pin here move for the emitter's own text.

The cost rule is met on both paths, and the sitting is on the page beside the
factor it is held to: write 0.71x to 0.90x and read 1.56x to 1.77x against the
byte body, where head measured 1.54x to 1.95x and 2.44x to 3.52x on the same
sitting.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* wire: a batch's node count sizes nothing until the bits back it (#530)

The message form's node table read reserved its record slice from the
count it had just read off the wire. That count is a thirty-two bit
number off hostile bytes, so a mutated batch commanded gigabytes before
a single record had been read, which is the opposite of what §3.1 means
by load is a scan. The file form already had the discipline: decode.go
checks its count against the bytes it holds before it allocates.

The reservation is now the smaller of the claim and what the stream can
carry, a record costing at least its type reference. bitReader.left
reports the bits still unread, which is the bound.

The wire fuzzer is what this cost. At the N make test runs, the harness
peaked at 10.7 GB and was killed for memory before it finished. It now
peaks at 158 MB and runs at 19,000 mutants a second where it managed
5,000, so the pass is both survivable and three times deeper.

The placeable set is named on its own line here too, as the file
reader names its own, because it is the seam the node-type negative
control replaces.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* tables: tables-vocab-schema compares the whole file, header included (#530)

The target could not pass as written. The generator echoes its own --out
into the file's second line, so generating into build/vocabgen and
comparing against tables/vocab differed on that line by construction,
whatever else matched.

The generator now runs from build/vocabgen under the committed file's
own relative path, so the --out it sees is the --out the committed file
carries. cmp then compares the whole file, and the regenerate line in
the committed header is checked by the same comparison as the tables
below it. Rewriting that line in the committed file turns the target
red, which it could not do before.

This is the smaller of the two fixes. The other was to make the header
path independent, which would have meant teaching the generator a
canonical destination it does not otherwise need.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* certify: the node-type control points at the reader that carries the rule (#530)

Two negative controls could not go red. Both sabotaged
internal/tablewire/decodenodes.go, which is the FILE reader, while the
vectors they assert on are message vectors. The message form's second
round built the placeable set a second time, in messagedecode.go, and
that is the copy a message vector decodes through, so the sabotage
patched a file the vectors never touch: 111,269 mutants, 0 divergences,
and the vector replayed alone gave 0 as well.

The node-type control now sabotages messagedecode.go. It also asserts by
replaying the pinned vector rather than by running the corpus, because
the pinned vectors ride last: an enumerated mutant of an ordinary message
seed reaches the same check, so a corpus pass goes red before the vector
is ever fed and names the mutant instead of the property. With the
sabotage the replay reports

  the report differs: the leg says 1,0,0,0,false,read,
  the oracle says 3,1,0,0,false,read

and without it the same replay is 1 mutant, 0 divergences.

The vector itself was re-derived. The second round moved the message
bodies to a packed bit stream, so every message wire in testdata was
regenerated and the old 104 byte vector stopped being a wire at all. The
new one is 66 bytes, a mutant of graph_tree_message that the reader with
ir.PointerReachable and the reader without answer differently.

THE BLOB NODE CONTROL IS GONE, and its vector with it, removed by
accident in the previous commit and explained here. Under the bitpacked
body a blob record carries a thirty-two bit length that must fit the
batch's remaining bits. A mutated record does name a reserved blob id,
reached at random mutant 11993 of the default seed, and the reader still
never places a blob: over 2,111,269 mutants with both reserved ids forced
nameable, a placement never happened once. The rule stays gated by
ir.PointerReachableBlobs in the reference emitter's NodeStorage and in
the oracle, and the node-type control above covers the same rule at the
ids that are not reserved, which the corpus does reach.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* wire: the four rules the cold read found unheld (#557)

A reader of docs/SPEC-TABLES.md §3.3 against the engine found four rules the
page states and neither engine enforced. Each lands with the vector that
shows it and a control that removes the rule.

A QUANTIZED INDEX ABOVE `count` IS REJECTED. The page says so three times and
names the vector, ten bits spelling 1023 over a count of 1000, and both
engines clamped it instead, which is the ranged offset's reconstruct-and-clamp
rule reaching a kind it does not govern. The float's rule is the packet wire's
bit for bit, and the packet wire rejects there.

AN ANNOUNCED MAX IS BOUNDED BY WHAT ITS KIND CAN HOLD. The ceiling was
INT32_MAX in the checker and nothing on the wire, so a 63-bit length width
reached signed overflow in the skip's `n * 8`. A string and a wide string are
bounded by the int32 storage cap the checker applies to every N, and an array
and a keyed entry by the 32-bit count an unbounded array announces, which is
the widest count this form spells.

A COUNT IS BOUNDED WHILE IT IS STILL WIDE, which is M6's discipline over a
count rather than a value. The C++ array and text reads narrowed to int32
first, so a count at or above 2^31 was negative, passed the signed test
against the bound untouched, decoded every element into scratch and landed a
negative count in the caller's storage.

A FIXED-WIDTH ELEMENT'S RUN IS ARITHMETIC. The page already says the surplus
of a fixed-width element is the surplus count times the element's width, and
that nothing in this form is superlinear in a batch's length. Both engines
looped, and a ranged element whose min equals its max rides no bits at all, so
six bytes of wire bought two billion iterations. The skip, the decode, the
extent walk and the Go array now step a fixed-width run in one guarded
multiplication and walk only an element that RESOLVES something, because a
resolve that contradicts its position is damage the reader must still find.
An unbounded array's count is bounded by the bits at one bit an element, the
node table's own discipline.

Four controls: message-max-above-int32, message-quantized-index-above-count
and message-surplus-walked in MESSAGE_FORM_CONTROLS, and the count's own,
message-emitter-narrow-count-before-clamp, as an emitter control over a new
instrument, because no pinned vector reaches a count at 2^31 and nothing about
that body is ill-formed for a refusal to fire on.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* wire: a batch reader's report sink is its own, and a measure refusal is compared as one (#557)

TWO THINGS THE COLD READ FOUND OUTSIDE THE FOUR RULES.

TableMessageBatchOpen wrote a caller that passed no report into a `static
TableReport`, which is shared mutable state: two threads opening two batches
without reports would write one object. It is a member of the batch reader
now, which is what LoadMessages already does with its own local sink.

AND THE WIRE FUZZER'S VERDICT ON A MEASURE REFUSAL. tables-cpp-release went
red at N=500000 on a mutant claiming 129 bodies in a 66-byte wire: the C++
LoadMeasure refused with no counter moved, and the Go oracle decoded body one
and counted one malformed. Both are the page: a LoadMeasure refusal moves no
counter (§6.5), and the fields decoded before the damage stand with the
counters they earned (§3.3). The two sentences govern two operations and both
stand, so the defect is the harness comparing a leg's MEASURE refusal against
the oracle's DECODE by counter tuple. Its own comment already said the oracle
need only agree there was nothing to read on that branch, and the code now
does what the comment says: the refusal verdict is compared, the tuples are
not. The page is unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* wire: five rules the page was silent on, decided in both engines (#557)

THE COLD READ ASKED FIVE QUESTIONS THE PAGE DID NOT ANSWER. Each is ruled
here, held by both engines, and written on the page.

A FAILED STRICT CHECK IS DAMAGE, NOT A REFUSAL. The announcement's two checks
are the two facts its body must carry, so a build version that is absent,
doubled, under another kind or not eight bytes wide, and a vocabulary that is
absent, doubled or not a run of bytes, each say the bytes are not an
announcement rather than that this peer declined to announce. `no_vocabulary`
is left to what it names, a BODY from a peer that never announced.

THE BUILD VERSION IS KEPT THE MOMENT IT IS READ, refusal or not, so a refusal
on that connection names it, which is what the page already promised and only
the Go refusal struct carried. It is not the vocabulary, and a refused
announcement still sets none.

TWO ENTRIES THAT AGREE ON ALL THREE PARTS ARE MALFORMED. The page said so and
neither engine enforced it. Both do now, at AnnounceRead: the scan is
quadratic in an entry count bounded at 4096, on a path that runs once a
connection.

AN ELEMENT KIND OF 12 OR 33 IN AN ANNOUNCED ARRAY OR KEYED SHAPE IS MALFORMED
AT THE ANNOUNCEMENT. No declaration this language accepts is an array of
`string(N)` or of `wstring(N)`, so the shape is refused once where it is read
rather than accepted there and refused again by the skip that would meet it.

A VARIANT REFERENCE IS RESOLVED ON THE SKIP PATH TOO. Every reference above E
is damage and one naming an entry that carries a payload contradicts the
position it was used in, whether or not this reader was going to keep the
value: it is the next bit's meaning that is in doubt and not the value's. The
three skip paths resolve it, and the fixed-width run that skips a fixed
element by arithmetic excludes every element that RESOLVES something.

AND THE MEASUREMENT ARM IS NAMED FOR WHAT IT IS. The cost rule measured
against the byte body, which this section replaced and which is no longer in
the tree; the arm actually measured is the file form's Save and Load, whose
save interns ids at run time and whose load parses a trailer. The rule and the
measurement table now say the file form on both paths, and the sentence
attributing the sub-1 write factor to bit framing against byte framing is
gone: a write factor below one is a fact about those two programs.

Four controls: message-strict-check-refuses, message-duplicate-entry-accepted,
message-array-of-text-accepted and message-skipped-variant-unresolved.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs: present-state comments, dead parameters gone, and no em dashes in this section's prose (#557)

THE COMMENTS THAT DESCRIBED A TREE THAT MOVED. internal/tablewire/message.go
called a form-2 wire TWO PARTS when it is three, said the receiver closes the
connection on a second announcement when the library returns a refusal and the
application decides, described the vocabulary bound as a trailer count when
the vocabulary is a field and the bound is two numbers, and numbered the
announcement's two reserved ids as slots 1 and 2 when they are its transport
and take no slot at all. Each now says what the tree does.

AND TWO ON THE PAGE. The full byte table called the byte-body column the wire
the tree carries today; it is the size the wire that held form byte 2 before
this section would have written, and the codec change this section lands
removed it. The form-byte paragraph said the goldens are re-pinned by a change
that lands; they are re-pinned, by a change that landed.

DEAD PARAMETERS AND A PERMANENT SKIP. bitDecoder.body took a `nested` flag no
branch read, tableMessageForm took a forceInline it never used, and the strict
checks test bound a `version` it never compared, each kept alive with `_ =`.
All three are gone. TestRetentionAcrossTheForms was a permanent t.Skip for a
row that lands with retention (§6.6, schema#525), which is not built in any
language: the test is deleted and the page's row says the suite carries no
test for it yet and no skipped one either.

AND THE EM DASHES. Seventy-six of them in the Go comments, the emitted C++
comments and the page prose this branch adds, rewritten as plain sentences.
The goldens move with the emitted comments, which is what re-pins the hundred
Table sources here.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* wire: a node record's framing is its type id's, and its placement is the root's (#557)

tables-cpp-release went red at mutant 280820 of seed 24845619678 over
graph_tree_message, once the harness verdict fix let the pass get past the
earlier red. The mutant's third node record claims the reserved `*string` id
under graphdemo's Scene, which reaches no blob edge at all.

THE TWO ENGINES FRAMED IT DIFFERENTLY. The C++ reference reads the record's
thirty-two bit length by the type id alone, finds 3,739,746,528 bytes in a 66
byte batch, and refuses to measure. The oracle built its blob set from
ir.PointerReachableBlobs and so had no blob ids at all for this root, framed
the record as a TABLE BODY, read its bytes as fields and reported a clean read
with one unknown.

The reference is right and the page now says why. A record's FRAMING is its
type id's: the reserved blob ids say a length, an align and the bytes wherever
they appear, and every reader knows that by the id alone because §3.3's tail
announces all three whether or not a root names them. Its PLACEMENT is the
root's, asked second of the blob edges its pointers reach (§2.5), and a root
that reaches none commands no storage, counts one unknown at the record, and
reads null through every reference to it. Gating the framing on the second
question makes a reader read a blob record's bytes as fields, because a bit
stream carries no length for it to step the record over by instead.

message_blob_node_unpointed is pinned again, at the 66 bytes the pass found,
with tables-wire-fuzz-blob-node-negative-control beside it. #552 pinned this
class and its vector stopped reproducing when the second round moved the
bodies to bits; 4822165 removed both and reasoned that the reader never
places a blob, which is true and was not the divergence. The sabotage is the
one line that gates the framing on reachability again, and the run goes red on
the vector.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* wire: the element cost floor is one max, not three lines (#557)

modernize flags the if statement, and it is right: the floor an unbounded
array's count is bounded at is max( run, 1 ).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* wire: the surplus control points at the element that is still walked (#557)

make test caught a control the arithmetic surplus retired.
message-surplus-lands-on-zero asserted M1, that a discarded surplus element
never acquires a live destination, on the six findings' vector, whose `few` is
a raw uint32 array. A fixed-width surplus is not decoded at all any more, so
the sabotage that redirects the sink had nothing to redirect and the row
stayed green.

The rule still has a place to bite, and it is the element that is still
WALKED: TestAnOverLongArrayOfNonFixedElements sends nine `string(32)` rows to
a reader that keeps four, and the five walked past it decode into the sink.
The test now asserts element ZERO as well as element three, and the control
points at it:

  negative control (message-surplus-lands-on-zero):
  TestAnOverLongArrayOfNonFixedElements goes red on
  element zero reads "iii": a walked surplus element landed on it

The page says both halves of the rule in one sentence beside the walk it
belongs to, and the emitter's comment says which half its scratch is for.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant