wire: the message form, second round — the batch and the bitpacked body (#530) by gafferongames · Pull Request #557 · mas-bandwidth/schema

gafferongames · 2026-09-05T05:59:44Z

The message form's SECOND ROUND, against the owner's rulings of 2026-09-05 and
docs/SPEC-TABLES.md §3.3 as landed by #559 and amended by #575: the batch
primitive, the bitpacked body, refusals at bit granularity, and scope by
announcement rather than by transport.

READY. The cost rule's acceptance condition is met on BOTH paths and on every
row, which is what held this draft. The bytes were already under proto3 where
they needed to be and are unchanged, byte for byte, by everything in this round:
every form-2 golden in the corpus still pins the same bits.

The bytes: measured, pinned, and held by test

Every figure is a committed wire file, and TestTheCostRows compares each one
against the number the page prints, so a byte that moves fails the build.

instance	file form	bitpacked body	proto3
`LoginRequest`, full	106	52	49
`MatchResult`, full	273	148	189
`StorePurchase`, full	104	43	40
`LoginRequest`, defaults	10	3	0
`MatchResult`, defaults	43	11	40
`StorePurchase`, defaults	10	3	2
the three full, one batch under `Envelope`	550	244	285

The announcement is 361 bytes over 33 entries, a reference is 6 bits,
and the batch of three is 14% under proto3. The two blob-shaped messages
are three bytes over, for the cause §3.3 already names and does not average
away: player_id, client_build and price_minor are declared BARE, so this
wire spends 64, 32 and 32 raw bits where proto3 spends a varint over a small
value.

The CPU: the rule is met, and here is the sitting

The sitting. M-series MacBook Air, macOS 25.6, Apple clang -O2 -DNDEBUG -ffp-contract=off, 2026-09-05 17:57 local, one-minute load average 2.5 to
2.9, read with uptime before every run. Seven rounds of 100,000 iterations,
best of rounds, and then the best of five whole processes, both arms in one
process over the same values.

Five processes rather than one, and the reason is measured. The run-to-run
spread on this shared machine is large enough that a single process is not a
measurement: over two runs of ONE unchanged binary the FILE arm's own write
figure moved from 37.3 ns to 21.3 ns. Both columns below are the best of five
runs of each binary in one sitting, so the "before" column is head 20288429
RE-MEASURED here rather than quoted from the earlier sitting, which is what
makes the two comparable.

The arm is the FILE form's Save/Load over the same values, which §3.3 says
is the same body — "§3's byte-framed body is the FILE's body and does not
move" — differing only in that a file carries its own id table. That
difference flatters the message form on the write path and is not folded away.

instance	write, head	write, now	read, head	read, now
`LoginRequest`, full	1.56x	0.75x	2.84x	1.77x
`MatchResult`, full	1.95x	0.90x	3.52x	1.64x
`StorePurchase`, full	1.55x	0.86x	2.44x	1.64x
the three, one batch	1.54x	0.71x	3.14x	1.56x

The write path is now UNDER the byte body and the read path is inside the
factor of two on every row. Two causes closed it, and each is a sentence §3.3
already states being made true in the code.

1. The receiver resolves ONCE, into storage the caller declares

§3.3 says "THE RECEIVER RESOLVES ONCE. It reads the announcement, resolves
every entry against its own descriptors, and every body after it dispatches
through one array index." The reader did not: it kept the OFFSET of each entry
inside the announcement's bytes and re-parsed the entry — a LEB128 shape decode
— on every field of every body.

It resolves once now, and the question that had to be answered first was WHERE
the resolved entries live, because a resolved TableVocabulary declared as a
local is a 440 KB stack object at the old 4096-entry bound. The answer is the
house rule, and §3.3 now states it as one:

THE STORAGE IS THE CALLER'S. The caller declares an array of resolved
entries wherever it wants it — static, on a heap, in an arena, inside the
object that holds the connection — and hands it to AnnounceRead with its
CAPACITY. Nothing in the codec allocates, on any path.
IT LIVES WITH THE CONNECTION, not with a call. The announcement is
delivered once and holds for the connection's life, so the resolved
vocabulary is filled once when the announcement is accepted and passed by
pointer to every LoadMessages, MeasureMessages and LoadMeasure after
it.
THE ENTRY BOUND IS THAT CAPACITY, and an announcement above it is refused
by name as vocabulary_too_large before an entry is touched — the existing
reason, whose §3.3 text already covered "both bounds", now meaning the number
the CALLER stated rather than a fixed 4096. A generated constant
kTableMessageEntriesHere gives the unit's own entry count, which is the
exact capacity a receiver that talks only to peers of its own schema
declares, because the vocabulary is a pure function of the build version.
4096 stays the conforming default where storage is grown rather than
declared, and the 64 KiB byte bound does not move.
THE RESOLVED ENTRY IS 96 BYTES, measured: it carries the id, the kind,
the packing, the payload's width ALREADY RESOLVED out of the kind and the
announced bits, the range base, the array bounds, and the quantized facts
SPEC.md §4.3's rule leaves behind rather than the two it consumes. So the
backenddemo unit's 33 entries are 3,168 bytes, and TableVocabulary
itself went from 16 KB of offsets by value to 56 bytes. A receiver that
asks for the 4096-entry default pays 393 KiB and asks for it explicitly.
PER FIELD ON THE READ PATH THAT IS ONE ARRAY INDEX: no parse, no LEB128,
no branch on the announcement's format.

NO NEW CLAIMED NAME. vocabulary_too_large was already claimed and already
had its conformance row and its negative control; kTableMessageEntriesHere is
a generated constant and not a spelling a declaration can collide with, so §11
and the tablenames registry are untouched.

2. The bit stream moves a WORD at a time

The remaining cause, once the vocabulary was resolved, was the bit stream
itself: it walked one BYTE at a time, so a bare uint64 cost nine masked
read-modify-writes to write and nine dependent loads to read, and MatchResult
carries eleven of them. It moves a word now — one unaligned load or store,
assembled little-end-first whatever order the host is in, with the byte walk
kept for a value inside nine bytes of the buffer's end. The bits on the wire
did not move, and every pinned golden is the proof. put still writes
EXACTLY the bytes the value occupies and never one past them, so a caller's
buffer beyond the batch is still its own.

That is the third step of the same work: the first sitting measured write at
7.94x to 11.55x and read at 4.18x to 6.17x on a stream that walked one BIT at a
time.

schema#571, the six findings

Each is fixed with a negative control in MESSAGE_FORM_CONTROLS, and each
control was run and quoted red:

M1, surplus scalar elements overwrite element zero — message-surplus-lands-on-zero: "the reader kept 5 elements, element zero reads 6".
M2, ranged 128-bit values disagree between measure, write and read — message-wide-reads-raw: "scalars_wide_message did not read". One shape arithmetic at every width now: a RANGED value rides as value - base at its announced width whatever that width is.
M3, announcement width validation exceeds the primitive's domain — message-width-above-kind: "65 bits on a uint64: announced=true". Width is checked against the announced kind, and the 64-bit reader refuses above 64.
M4, count written instead of count minus the minimum — message-count-not-offset: "bases_message does not re-save to its own bytes".
M5, quantized float writers use different intermediate precision — message-quantize-truncates: "0.005 quantizes to 0, the packet wire writes 1", and message-dequantize-round-once: "index 6666 over [-100, 100] at 0.01 decodes to c2055c29, not C2055C2A".
M6, C++ narrows a reconstructed ranged integer before clamping — message-narrow-before-clamp: "the offset body: narrow=200". The bound applies while the value is wide.

All 41 message-form controls go red, the two emitter controls included;
make tables-message-form-negative-control runs the set and fails if any
sabotage survives. The entry bound's own control, re-run for this round:

negative control (message-no-entry-bound): TestTheTwoBounds goes red on
messagerules_test.go:774: one entry above the entry bound: err=<nil>
announced=true entries=4097 report={Unknown:0 KindMismatch:0 Clamped:0
Duplicate:0 Malformed:false Refused:false}

The zero-cost gate

It asserts the BLOCK form's zero cost in three parts: no Table source carries
one block symbol, the C# Table sources carry no build version, and the frozen
Table pins are byte-identical. The first two are the property and both stand
untouched. The third is a golden, and the gate's own text says a golden is
re-pinned when a Table emitter legitimately changes.

31 C++ Table pins move and the C# and C pins do not, which is the check that
the change reached only the C++ reference: the eight ports carry the file form
alone. The gate is green at 100 sources.

Left for the owner, not decided here

The measurement still has no reproducible home in the tree. The shape gate
holds the estate's one-benchmark rule mechanically and refused a harness under
test/, correctly. Its home is a message-form arm beside the tolerant-wire
arm in bench/tables, which needs a pinned message variant corpus and a row
on the per-language board, and that is its own PR rather than this one.
The numbers above were taken with that harness off-tree.
THE THREE make test REDS ARE FIXED, AND make test IS NOW WHOLE GREEN
on this branch. None of them is reached by the twenty checks on this PR,
because ci.yml does not run make test. certify.yml does.
- THE NODE-TYPE CONTROL WAS POINTED AT THE WRONG READER. It sabotaged
  internal/tablewire/decodenodes.go, which is the FILE reader, while the
  vector it asserts on is a message vector. This round built the placeable set
  a second time, in messagedecode.go, and that is the copy a message vector
  decodes through, so the sabotage patched a file the vector never touches:
  111,269 mutants, 0 divergences, and the vector replayed alone gave 0 as well.
  The control now sabotages messagedecode.go, and it asserts by replaying the
  vector rather than by running the corpus, because the pinned vectors ride
  last and an enumerated mutant of an ordinary message seed reaches the same
  check first. With the sabotage applied:
```
harness: FAILED after 0 mutants: the report differs: the leg says 1,0,0,0,false,read, the oracle says 3,1,0,0,false,read
  corpus seed message_node_type_unpointed.bin (graphdemo.Scene), pass replay #0
negative control: placing a node no pointer names turns the pinned vector RED
```
  and without it, the same replay:
```
wire-fuzz: replay of testdata/wire/tables/fuzz-vectors/message_node_type_unpointed.bin over graphdemo.Scene as a message, 1 mutant, 0 divergences, 0.0 s
```
  The vector itself was re-derived. This round moved the message bodies to a
  packed bit stream, so every message wire in testdata was regenerated and
  the old 104 byte vector stopped being a wire at all. The new one is 66 bytes.
- THE BLOB-NODE CONTROL IS DELETED, WITH ITS VECTOR. Under the bitpacked
  body a blob record carries a thirty-two bit length that must fit the batch's
  remaining bits. A mutated record does name a reserved blob id, reached at
  random mutant 11993 of the default seed, and the reader still never places a
  blob: over 2,111,269 mutants with both reserved ids forced nameable, a
  placement never happened once. The wire fuzzer is no longer the instrument
  that can prove this rule. It stays gated by ir.PointerReachableBlobs in the
  reference emitter's <Root>NodeStorage and in the oracle, and
  tables-wire-fuzz-node-type-negative-control covers the same §3.1 and §6.5
  rule at the ids that are not reserved, which the corpus does reach.
- tables-vocab-schema NOW COMPARES THE WHOLE FILE, HEADER INCLUDED. It
  could not pass as written: the generator echoes its own --out into line 2,
  so generating into build/vocabgen and comparing against tables/vocab
  differed on that line by construction. The generator now runs from
  build/vocabgen under the committed file's own relative path, so the --out
  it sees is the --out the committed file carries, and cmp compares
  everything. This is the smaller of the two fixes: the other was to make the
  header path-independent, which would have meant teaching the generator a
  canonical destination it does not otherwise need. Rewriting the regenerate
  line in the committed file now turns the target red, which it could not do
  before.
- THE FUZZER'S MEMORY WAS A DEFECT IN THIS ROUND'S READER, NOT A LAPTOP
  LIMIT. certify.yml runs make test on ubuntu-latest and
  macos-latest, both at the N=20000 the recipe hard-codes, so the peak had
  to be accounted for rather than blamed on a 24 GB machine. It is accounted
  for: the message form's node table reserved its record slice from the
  thirty-two bit count it had just read off the wire, so a mutated batch
  commanded gigabytes before a single record had been read. The C++ leg peaked
  at 5.3 MB throughout. The Go harness peaked at 10.7 GB and was killed for
  memory at 48 s. The reservation is now the smaller of the claim and what the
  remaining bits can carry, which is the discipline the file form already had
  in decode.go. Measured on the same machine, same seed, same N:
  
  peak RSS outcome
  
  before 10,735,452,160 bytes killed for memory at 48 s
  
  after 157,696,000 bytes 131,269 mutants, 0 divergences, 6.8 s
  
  Throughput went from 5,168 to 19,385 mutants a second.
AN OPEN RED THIS BRANCH ALSO CARRIES, WHICH IS NOT FIXED HERE AND NEEDS A
RULING. tables-cpp-release runs tables-wire-fuzz at N=500000 on two
seeds, and certify.yml runs every tables-<lang>-release target by name.
Both passes go red, and both on the same class. Before the memory fix above
they could not get far enough to show it, because the harness died first.
```
seed 24845619678: FAILED after 164294 mutants: the report differs: the leg says 0,0,0,0,true,read, the oracle says 1,1,0,0,true,read
  corpus seed graph_tree_message (graphdemo.Scene), pass random #53025
  leg:    loaded=false report=0,0,0,0,true,read measure=-1 saved=refused
  oracle: report=1,1,0,0,true,read encoded=5 bytes

seed 2: FAILED after 208341 mutants: the report differs: the leg says 0,0,0,0,true,read, the oracle says 2,0,0,0,true,read
  corpus seed graph_tree_message (graphdemo.Scene), pass random #97072
```
The first mutant differs from its seed at the batch's M byte, which reads
129 bodies out of a 66 byte wire. The C++ leg's LoadMeasure refuses the
framing and reports nothing, which is §6.5's rule as the page states it: "NO
COUNTER MOVES FOR A LoadMeasure REFUSAL... nothing was decoded, so there is
nothing to count". The Go oracle decodes body one, counts what it met, and
then runs out of bits, which is §3.3's rule as the page states that one: "The
fields decoded before it stand, ONE malformed counts, and nothing after it
is read". Both rules are on the page and they answer this case differently.
wireVerdict already has a branch for the leg refusing at measure, and its
own comment says the oracle only has to agree there was nothing to read, but
the branch then demands the counter tuples match, which they cannot on that
path. Deciding which rule governs is a wire ruling, so it is left here rather
than settled by a builder. make test at N=20000 does not reach it.
A dist symlink carrying an absolute machine-local path is committed on
main, entered in check: the wstring refusal covers a table's whole closure (#567) #572. Not this branch's, and not fixed here.

🤖 Generated with Claude Code

…he bitpacked body (#530) The primitive is a NUMBER OF MESSAGES. EncodeMessages writes one buffer: the form byte, the count, and the bodies as one continuous bit stream with no alignment between them, padded to a byte at the end and nowhere else. A single message is the batch of one, which is the only sense in which this wire carries one. And the body is BITPACKED. References ride in bits_required(entries), there is no kind byte at all, values ride at their declared widths as the packet wire writes them, lengths ride in bits_required(N), and elision is unchanged. What a reader needs to skip an id it cannot name is therefore not on the body: it is the ANNOUNCEMENT's per-entry RECORD, a fixed-stride array under a third reserved id, carrying the kind, the two widths and the range base each slot spells. The record is the WIRE CONTRACT for its id and both halves write to it, so one field name declared at two bounds in two records rides at the widest of them rather than at either. The measurements, against the byte body #549 landed: login_full 106 file 58 byte body 51 bitpacked 49 proto3 match_full 273 file 225 byte body 143 bitpacked 189 proto3 store_full 104 file 48 byte body 41 bitpacked 40 proto3 login_default 10 file 2 byte body 3 bitpacked match_default 43 file 27 byte body 10 bitpacked 40 proto3 store_default 10 file 2 byte body 3 bitpacked the three as ONE BATCH 230 bitpacked Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

`pack --message` writes a batch and `unpack --announce` reads one, because the primitive is a number of messages and a single message is the batch of one. `--batch <Table>=<tree-dir>` names each message after the first, in the order it rides: which root a message is, is the application's, so the tool asks rather than guessing. The two verbs' single-message spellings are unchanged and are now the batch of one, and `unpack` refuses a batch whose count is not the number of roots named rather than writing a tree it cannot fill. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

The bit stream first: a TableBitWriter and a TableBitReader in the packet wire's own layout, bit i in byte i/8 low bit first, with a bit LEB128 beside them for the numbers no declaration bounds. Then the per-entry RECORD the announcement now carries, borrowed from its bytes like the entries: a fixed-stride array a reader indexes rather than searches, and one generic TableMessageSkip over it, because a record says everything a skipper needs and one function then serves every table. Then the codec: <T>MeasureMessageBody, <T>SaveMessageBody and <T>LoadMessageBody per table, and the batch's own surface per root — MeasureMessages, SaveMessages and LoadMessages beside the batch-of-one spellings. Every reference is a compile-time SLOT and every width a literal on the write side, so a save does no lookup at all. AnnounceRead gains the second strict check: the records present, exactly once, at kind 12, one record an entry. A table with no records is a table nothing can be read against. The C++ reference and the compiler's engine agree BYTE FOR BYTE, and measure equals save, on all eight value-class message vectors of the corpus: login_full 106 file -> 51 match_full 273 file -> 143 login_default 10 file -> 3 match_default 43 file -> 10 store_full 104 file -> 41 vocab_low 24 file -> 8 store_default 10 file -> 3 vocab_wide 192 file -> 68 WHAT IT CARRIES TODAY is the value class with no map. A pointered message and a map's cursor both take a resolution context this codec does not thread, and a root whose closure it cannot carry gets no message entry points at all rather than half of one. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…count, the align (#530) The page (#523) settles the wire, and this is the engine reworked onto it, where it is more specific than the brief was. THE VOCABULARY IS A FIELD of the announcement's body, not its trailer, and an entry is a TRIPLE: an id, a kind, and a SHAPE. That buys three things the brief's fixed-stride records could not. §3's writer rule that an id no body references is never written is restored unbroken. An entry carries the widths and the RANGE a reader needs to decode a field whose declaration has MOVED, so every evolution row of §4 stands under a bitpacked body. And ONE NAME MAY TAKE TWO SLOTS, at two kinds or two shapes, so a unit declaring `count uint8` in one table and `count uint32` in another needs no canonical widening and no ambiguity rule at all. The rest of the page's wire, exactly: the count is a ranged integer over [1, 256] carrying M - 1, a batch of zero is not spellable, a `string(N)` and a `bytes(N)` ALIGN before their bytes, a fixed array spends NO count because its min equals its max, a compressed float rides QUANTIZED as the packet wire writes it, a pointer index is bits_required(0, node count) and the node table is the root body's FIRST field, the trailing pad is verified zero, and damage is TERMINAL for the batch. The engine now reproduces the page's hand-worked arithmetic exactly: login_full 106 file -> 51 match_full 273 file -> 142 login_default 10 file -> 3 match_default 43 file -> 10 store_full 104 file -> 41 store_default 10 file -> 3 the three as one batch -> 230 graph_tree 232 file -> 66 vocab_low 24 file -> 8 vocab_wide 192 file -> 68 the announcement: 28 entries, 273 bytes of them, 316 bytes whole Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

#530) The uncommitted tree the previous builder left at 70d44af, measured, built and committed as it stood: the Go engine builds, the conformance harness is green on every message row, and the C++ tables test builds and passes. What it carries: the variable class on the message wire in C++ (the node table as the root body's first field, thirty-two bit counts, blob records with a length and an align, index widths settled per body, maps carved from the framing), the batch's three verbs over a region with one region per batch, wstring at sixteen bits a unit and the escape kind, the batch's five answers with batch_too_large on both sides, the wrong-sort rule for variant and arm references with the reserved-id rule outranking it, the three reserved ids refused in the vocabulary, the vocab9demo unit for a nine-bit reference, the fuzzer's bit-level reference pass, schema unpack printing an announcement, and the sabotage tool's message overlays for every harness row. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…e batch, and schema#571 (#530) Merges origin/main (#562, #564, #573) and lands the page's second round (#575) on both engines: a ranged base encoded by its kind's signedness, the quantized f32 row as min, max and res in float32 with the step count and the width derived by SPEC.md §4.3's rule, quantization in float32 with two roundings on each side, a refused first announcement terminal, and the Envelope over a union of the three messages as the batch vector (244 bytes, the singles at 52, 148 and 43, the announcement at 361). Unbounded arrays ride the C++ message wire with the count the data decides and the elements carved from the node's extent. schema#571's six findings are fixed with a vector and a control each: surplus elements decode into scratch (M1), a ranged 128-bit value is one arithmetic for measure, write and read (M2), a width above the kind's domain is refused in both shape parsers and the bit reader (M3), the Go engine writes the count as its offset from the minimum (M4), one quantization rule (M5), and the C++ reader clamps while wide and narrows after (M6). The bases unit (test/tables/Bases.schema) holds the vectors in both engines. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…s header (#530) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…nd the fuzz legs at the wire's M (#530) Resumed from the previous builder's working tree, unchanged in intent. A FIXED root numbers no node, so the index width it hands its body is zero rather than a constant the body invents: LoadMessageBody takes index_bits on both the fixed and the variable path, and a kind-17 entry inside a fixed body is damage because there is no width to step it over. The Go reader matches by setting the width to zero at a fixed root and refusing an index read at width zero. A ROOT body whose own framing gives out is damage inside that body, not a whole-batch refusal: the scan answers the bodies up to and including it, and MeasureMessages sizes through it and no further, which is the region the load needs to deliver the bodies before the damage. The fuzz legs hold room for the wire's M so a capacity refusal is never the answer under fuzz, and the region check rejects a negative byte count before memset. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

#576 and #580 land `was` on a table declaration, on enum variants and union arms, and on the fields of a `type` a table reaches, and #572 and #573 move beside them. The message form follows: a vocabulary entry's id is the WIRE name's at every place a name becomes one, so an arm, a variant and a table name renamed under `was` keep the slot they had. Main's list work generalized hasMapExtent to hasExtent and TableMapCarve to TableExtentCarve; the message emitters take the general names. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…s it (#530) A record's extent cursor is a local of NodeMessageBody, and the node map it was published through is the caller's, so a return with the map still naming it leaves an address the caller could read. The dispatch answers into a bool and the cursor is restored on the way out. gcc 13 on the big-endian leg names this exactly, -Werror=dangling-pointer at the message node body, and the fix is the escape rather than the flag. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

The committed tree is what this branch's compiler emits: the message codec's node carve restored on the way out, and main's extent names. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

#530) WHAT THE GATE ASSERTS is the BLOCK form's zero cost, in three parts: no Table source carries one block symbol, the C# Table sources carry no build version, and the frozen Table pins are byte-identical. The first two are the property and both stand untouched. The third is a golden, and the gate's own text says a golden is re-pinned when a TABLE emitter legitimately changes. The message codec is such a change and it reaches EVERY unit, because the three message verbs are in tableGeneratedVerbs and every table carries them (§3.3). Every unit's header moves for the same two reasons: `batch_too_large` joins the refusal reasons, and TableIds loses the `vocabulary` flag and its two-argument `ref` because a bitpacked body names no id at all — its references are compile-time slots of the announced vocabulary, so the byte-framed form's shared id table has nothing left to do. That is the byte body being replaced in place, which §3.3 states. The C# and C Table goldens do NOT move, which is the check that the change reached only the C++ reference: the eight ports carry the file form alone. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…et a harness (#530) The clamp of the rounded index to `count` is the packet wire's own last step and is the same answer written shorter, which is what the lint asks for. The float clamp above it keeps its negated comparisons, because those are what send a NaN to zero rather than through. `make tables-message-cost` measures what the page holds the form to: the bytes each form spends over the three backend messages and the batch, and the read and write factor the bitpacked body takes against the byte-framed body over the same values. It is a measurement and not a gate, so it is not in `test` — a ratio moves with the machine. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

THE ESTATE HAS ONE BENCHMARK and the shape gate holds that mechanically: no timing primitive outside the sanctioned runner and tool directories. A hand-written harness under test/ is exactly the divergence class the rule names, and the gate refused it by name. The number the cost rule owes is taken off-tree for now and quoted in the PR body with its sitting. Its reproducible home is the sanctioned bench, a message-form arm beside the tolerant-wire arm in bench/tables, which needs a pinned message variant corpus and a row in the per-language board and is therefore its own change. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

#578 makes ONE traversal serve Number, Pack, Lock, Cook and LoadMeasure, and PointerReachable answers structs off that walk rather than names off the unit. The message form's node dispatch and its Go reader take the same walk, so the two forms number a graph the same way by construction rather than by agreement, and the node type id is the WIRE name's so a table renamed under `was` is still the node its old name numbers. The arms unit it brings has an EXTENT and no MAP, which is the shape that found a latent hole: the message emitters guarded every extent path on `anyMap`, so a unit whose only container is an unbounded array emitted a carve it never set and an extent it never walked. The guard is `anyExtent` throughout now, which is what the file form's own emitters use, and the arms unit compiles as its first witness. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…emcpy (#530) MEASURED FIRST. On the three backend messages and the batch, the bitpacked body was writing at 7.9x to 11.6x the byte-framed body's time and reading at 4.2x to 6.2x, against the factor of TWO the page holds the form to. The cause was in the primitive rather than in the codec: put and get walked ONE BIT at a time with a division and a modulo per bit, and putbytes spent eight such iterations per byte on the largest payload the wire carries. §3.3 already says what the align is for: a string(N), a bytes(N) and a blob record align before their bytes so the payload moves as one memcpy. It now does. put and get take the bits that fall in a byte together, so a 64-bit field costs nine touches rather than sixty-four and no division at all, and align spends its pad in one call. THE BITS ON THE WIRE DO NOT MOVE — bit i in byte i/8 at position i%8, low bit first — and every pinned golden is the proof: the corpus, the round trips and the batch vectors all reproduce byte for byte. After: write 1.27x to 2.11x, read 2.05x to 3.54x. The write path is inside the rule and the read path is not yet; the measurement and the cause are in the PR body. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

The committed tree is what this branch's compiler emits. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…bit stream moves a word at a time (#530) docs/SPEC-TABLES.md §3.3 says THE RECEIVER RESOLVES ONCE: it reads the announcement, resolves every entry against its own descriptors, and every body after it dispatches through one array index. The reader did not. It kept the OFFSET of each entry inside the announcement's bytes and re-parsed the entry, a LEB128 shape decode, on every field of every body. That is the code failing the page, and it is why the read factor sat above the page's bound. THE STORAGE SHAPE IS WHAT THE SENTENCE NEEDED, and the page states it as a rule now. The announcement is delivered once and holds for the connection's life, so the resolved vocabulary lives with the CONNECTION and not with a read call: the caller declares an array of resolved entries wherever it wants it, hands it to AnnounceRead with its CAPACITY, and passes it by pointer to every read after that. Nothing in the codec allocates. THE ENTRY BOUND IS THAT CAPACITY, so an announcement above it is refused by name as vocabulary_too_large before an entry is touched, and kTableMessageEntriesHere gives the unit's own entry count for a receiver that talks only to peers of its own schema. 4096 stays the conforming default where storage is grown rather than declared. A RESOLVED ENTRY IS 96 BYTES: the id, the kind, the packing, the payload's width ALREADY RESOLVED out of the kind and the announced bits, the range base, the array bounds, and the quantized facts SPEC.md §4.3's rule leaves behind rather than the two it consumes. TableVocabulary went from 16 KB of offsets by value to 56 bytes, and the backenddemo unit's 33 entries take 3,168 bytes. Per field on the read path that is one array index, no parse and no branch. AND THE BIT STREAM MOVES A WORD AT A TIME, which is what remained once the vocabulary was resolved: the stream walked one BYTE at a time, so a bare uint64 cost nine masked read-modify-writes to write and nine dependent loads to read, and MatchResult carries eleven of them. A value of any width now moves in one unaligned load or store, assembled little-end-first whatever order the host is in, with the byte walk kept for a value inside nine bytes of the buffer's end. put still writes EXACTLY the bytes the value occupies, so a caller's buffer beyond the batch is still its own. THE BITS ON THE WIRE DID NOT MOVE. Every pinned form-2 golden holds, and the 31 C++ Table pins that re-pin here move for the emitter's own text. The cost rule is met on both paths, and the sitting is on the page beside the factor it is held to: write 0.71x to 0.90x and read 1.56x to 1.77x against the byte body, where head measured 1.54x to 1.95x and 2.44x to 3.52x on the same sitting. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

) * docs: the bitpacked tables row, the message form's bitpacked body Owner-ruled 2026-09-05: bitpacked tables are a feature, in the message form only, where they beat Protobuf on size. Red in every column until #557 lands. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs: the row is named bitpacked table message form Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Rowan Claude <rowan@mas-bandwidth.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>

The message form's node table read reserved its record slice from the count it had just read off the wire. That count is a thirty-two bit number off hostile bytes, so a mutated batch commanded gigabytes before a single record had been read, which is the opposite of what §3.1 means by load is a scan. The file form already had the discipline: decode.go checks its count against the bytes it holds before it allocates. The reservation is now the smaller of the claim and what the stream can carry, a record costing at least its type reference. bitReader.left reports the bits still unread, which is the bound. The wire fuzzer is what this cost. At the N make test runs, the harness peaked at 10.7 GB and was killed for memory before it finished. It now peaks at 158 MB and runs at 19,000 mutants a second where it managed 5,000, so the pass is both survivable and three times deeper. The placeable set is named on its own line here too, as the file reader names its own, because it is the seam the node-type negative control replaces. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…530) The target could not pass as written. The generator echoes its own --out into the file's second line, so generating into build/vocabgen and comparing against tables/vocab differed on that line by construction, whatever else matched. The generator now runs from build/vocabgen under the committed file's own relative path, so the --out it sees is the --out the committed file carries. cmp then compares the whole file, and the regenerate line in the committed header is checked by the same comparison as the tables below it. Rewriting that line in the committed file turns the target red, which it could not do before. This is the smaller of the two fixes. The other was to make the header path independent, which would have meant teaching the generator a canonical destination it does not otherwise need. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…rule (#530) Two negative controls could not go red. Both sabotaged internal/tablewire/decodenodes.go, which is the FILE reader, while the vectors they assert on are message vectors. The message form's second round built the placeable set a second time, in messagedecode.go, and that is the copy a message vector decodes through, so the sabotage patched a file the vectors never touch: 111,269 mutants, 0 divergences, and the vector replayed alone gave 0 as well. The node-type control now sabotages messagedecode.go. It also asserts by replaying the pinned vector rather than by running the corpus, because the pinned vectors ride last: an enumerated mutant of an ordinary message seed reaches the same check, so a corpus pass goes red before the vector is ever fed and names the mutant instead of the property. With the sabotage the replay reports the report differs: the leg says 1,0,0,0,false,read, the oracle says 3,1,0,0,false,read and without it the same replay is 1 mutant, 0 divergences. The vector itself was re-derived. The second round moved the message bodies to a packed bit stream, so every message wire in testdata was regenerated and the old 104 byte vector stopped being a wire at all. The new one is 66 bytes, a mutant of graph_tree_message that the reader with ir.PointerReachable and the reader without answer differently. THE BLOB NODE CONTROL IS GONE, and its vector with it, removed by accident in the previous commit and explained here. Under the bitpacked body a blob record carries a thirty-two bit length that must fit the batch's remaining bits. A mutated record does name a reserved blob id, reached at random mutant 11993 of the default seed, and the reader still never places a blob: over 2,111,269 mutants with both reserved ids forced nameable, a placement never happened once. The rule stays gated by ir.PointerReachableBlobs in the reference emitter's NodeStorage and in the oracle, and the node-type control above covers the same rule at the ids that are not reserved, which the corpus does reach. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

A reader of docs/SPEC-TABLES.md §3.3 against the engine found four rules the page states and neither engine enforced. Each lands with the vector that shows it and a control that removes the rule. A QUANTIZED INDEX ABOVE `count` IS REJECTED. The page says so three times and names the vector, ten bits spelling 1023 over a count of 1000, and both engines clamped it instead, which is the ranged offset's reconstruct-and-clamp rule reaching a kind it does not govern. The float's rule is the packet wire's bit for bit, and the packet wire rejects there. AN ANNOUNCED MAX IS BOUNDED BY WHAT ITS KIND CAN HOLD. The ceiling was INT32_MAX in the checker and nothing on the wire, so a 63-bit length width reached signed overflow in the skip's `n * 8`. A string and a wide string are bounded by the int32 storage cap the checker applies to every N, and an array and a keyed entry by the 32-bit count an unbounded array announces, which is the widest count this form spells. A COUNT IS BOUNDED WHILE IT IS STILL WIDE, which is M6's discipline over a count rather than a value. The C++ array and text reads narrowed to int32 first, so a count at or above 2^31 was negative, passed the signed test against the bound untouched, decoded every element into scratch and landed a negative count in the caller's storage. A FIXED-WIDTH ELEMENT'S RUN IS ARITHMETIC. The page already says the surplus of a fixed-width element is the surplus count times the element's width, and that nothing in this form is superlinear in a batch's length. Both engines looped, and a ranged element whose min equals its max rides no bits at all, so six bytes of wire bought two billion iterations. The skip, the decode, the extent walk and the Go array now step a fixed-width run in one guarded multiplication and walk only an element that RESOLVES something, because a resolve that contradicts its position is damage the reader must still find. An unbounded array's count is bounded by the bits at one bit an element, the node table's own discipline. Four controls: message-max-above-int32, message-quantized-index-above-count and message-surplus-walked in MESSAGE_FORM_CONTROLS, and the count's own, message-emitter-narrow-count-before-clamp, as an emitter control over a new instrument, because no pinned vector reaches a count at 2^31 and nothing about that body is ill-formed for a refusal to fire on. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…is compared as one (#557) TWO THINGS THE COLD READ FOUND OUTSIDE THE FOUR RULES. TableMessageBatchOpen wrote a caller that passed no report into a `static TableReport`, which is shared mutable state: two threads opening two batches without reports would write one object. It is a member of the batch reader now, which is what LoadMessages already does with its own local sink. AND THE WIRE FUZZER'S VERDICT ON A MEASURE REFUSAL. tables-cpp-release went red at N=500000 on a mutant claiming 129 bodies in a 66-byte wire: the C++ LoadMeasure refused with no counter moved, and the Go oracle decoded body one and counted one malformed. Both are the page: a LoadMeasure refusal moves no counter (§6.5), and the fields decoded before the damage stand with the counters they earned (§3.3). The two sentences govern two operations and both stand, so the defect is the harness comparing a leg's MEASURE refusal against the oracle's DECODE by counter tuple. Its own comment already said the oracle need only agree there was nothing to read on that branch, and the code now does what the comment says: the refusal verdict is compared, the tuples are not. The page is unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

rowan-claude force-pushed the impl-message-form-2 branch from 9e7e0a6 to 4c14690 Compare September 5, 2026 07:04

gafferongames and others added 4 commits September 5, 2026 08:39

rowan-claude force-pushed the impl-message-form-2 branch from 4c14690 to 70d44af Compare September 5, 2026 15:40

This was referenced Sep 5, 2026

Message form page: the uint64 high-half base, the mixed-batch example, quantization across forms, and a refused first announcement #570

Open

#557 draft at 70d44af: six codec findings for the message form's second round #571

Open

gafferongames and others added 5 commits September 5, 2026 12:22

Merge remote-tracking branch 'origin/main' into impl-message-form-2

d0b04f2

generated: the bench tables unit regenerated, the message codec on it…

80479bd

…s header (#530) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

rowan-claude force-pushed the impl-message-form-2 branch from f9a03cb to d479b2f Compare September 5, 2026 23:46

gafferongames mentioned this pull request Sep 5, 2026

3.0.0 gate: complete feature parity across all nine languages, each at its language's performance floor #366

Open

rowan-claude force-pushed the impl-message-form-2 branch from 2165b13 to 7025ecf Compare September 6, 2026 00:03

gafferongames and others added 9 commits September 5, 2026 17:04

generated: the bench tables unit on the merged compiler (#530)

931f603

The committed tree is what this branch's compiler emits: the message codec's node carve restored on the way out, and main's extent names. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

generated: the bench tables unit on the byte-at-a-time bit stream (#530)

2028842

The committed tree is what this branch's compiler emits. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

gafferongames marked this pull request as ready for review September 6, 2026 02:05

This was referenced Sep 6, 2026

Bitpacked tables: the bitpacked body as a form for any table root, not only the message form #587

Closed

docs: the bitpacked table message form row in the roadmap's matrix #588

Merged

gafferongames and others added 3 commits September 5, 2026 19:43

This was referenced Sep 6, 2026

The payload-free union arm on the packet wire in the C and C++ emitters (under the C/C++ lock) #503

Open

string(N): read-side UTF-8 refusal in every target, per the 2026-09-04 ruling #519

Open

gafferongames and others added 2 commits September 5, 2026 20:33

Navigation Menu

Uh oh!

wire: the message form, second round — the batch and the bitpacked body (#530) - #557

wire: the message form, second round — the batch and the bitpacked body (#530)#557
gafferongames wants to merge 24 commits into
mainfrom
impl-message-form-2

gafferongames commented Sep 5, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	peak RSS	outcome
before	10,735,452,160 bytes	killed for memory at 48 s
after	157,696,000 bytes	131,269 mutants, 0 divergences, 6.8 s

Uh oh!

Conversation

gafferongames commented Sep 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The bytes: measured, pinned, and held by test

The CPU: the rule is met, and here is the sitting

1. The receiver resolves ONCE, into storage the caller declares

2. The bit stream moves a WORD at a time

schema#571, the six findings

The zero-cost gate

Left for the owner, not decided here

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

gafferongames commented Sep 5, 2026 •

edited

Loading