spec: the message form's base encoding, envelope batch, quantization rule and terminal refusal; retain-unknown's recursive exclusion and record lifetime (#569, #570)

[gafferongames]

Lands the rulings on schema#569 and schema#570 on docs/SPEC-TABLES.md, with the figures that depend on them moved in docs/USAGE.md and docs/VERSIONING.md. Prose only.

#570, the message form (§3.3)

D1, the ranged base. The shape row for kinds 2 to 9 now encodes the base by the kind's signedness: zigzag LEB128 for the signed kinds 2 to 5, unsigned LEB128 for 6 to 9, both canonical on §3's rule, sixteen bytes for 18 and 19 as before. A normative paragraph after the shape table states it with the 2^63 case. Test row "The base's two encodings" pins four announcements as bytes with the recovered values: [2^63, 2^63 + 1] (base 80 x9 01), [2^64 - 2, 2^64 - 1] (base FE FF x8 01), int32 [-5, 10] (bits 04, base zigzag 09), and uint8 [7, 7] (bits 00, base 07).

D2, the batch. The envelope, not the fallback. The batch is three bodies of one root Envelope { payload Payload } over union Payload { login LoginRequest, match MatchResult, purchase StorePurchase }. A root is a table on this wire (§3), so the ruling's union rides as the one field of a table root, on §2.6's own ToolMessage shape. The envelope adds five entries to backenddemo (the field at kind 15, three arms at kind 13, the table's name id in the tail), so E goes 28 to 33 and every reference in the unit goes from 5 bits to 6. Every figure that depends on it moved, with the arithmetic shown bit by bit on the page:

• singles: LoginRequest 416 bits = 52 (was 51), MatchResult 1178 bits = 148 (was 142), StorePurchase 341 bits = 43 (was 41). Defaults 3 / 11 / 3 (was 3 / 10 / 3).
• the batch: 16 framing + envelope 1 (12 + 396 + 6 = 414, login's align 2 bits) + envelope 2 (12 + 1162 + 6 = 1180) + envelope 3 (12 + 319 + 6 = 337, purchase's align 0 bits) = 1947 bits, pad 5, 1952 bits = 244 bytes. The three envelopes as three batches of one: 430, 1196, 355 bits = 54 + 150 + 45 = 249. The three bare messages alone: 243.
• proto3 for the same envelope, a oneof arm being a one-byte tag and a length varint before the embedded message: 51 + 192 + 42 = 285, summed because proto3 has no batch.
• the other batch columns, so the row compares one thing: packet wire 196 (the 2-bit tag absorbed by sku's align), file form 128 + 296 + 126 = 550, byte body 64 + 232 + 54 = 350.
• readings: against the byte body 10% / 34% / 10% and 59% on MatchResult defaults (was 12 / 37 / 15 / 63). Against proto3 the batch is 14% under (was 17), MatchResult 22% under (was 25), LoginRequest and StorePurchase three bytes over each (was two and one). Against the packet wire 6 / 33 / 7 (was 5 / 27 / 5), with 30 of MatchResult's 33 the row references. The range remedies now land one byte over proto3 rather than level and under: client_build max = 65535 gives 50, price_minor max = 9999 gives 41.
• the announcement: 33 entries, 318 entry bytes (138 + 99 + 81), 361 bytes (was 316), pays for itself in the ninth round at 41 bytes saved a round (was seventh at 48).
• the four-variant enum residual is stated at three bits under the six-bit reference (was two under five).

USAGE.md and VERSIONING.md carry the new table and the over-by lines. The goldens section declares Payload and Envelope in the corpus block and pins backend_round at 244 with the three envelopes alone at 54 / 150 / 45.

Quantization. The QUANTIZED shape is now min, max, res as float32, which is what SPEC.md §4.3's rule takes, and the page states the rule in full: delta, count = ceil(delta / res) clamped to [1, 4294967040], bits_required(0, count), the writer's clamp, product rounded to float32, + 0.5, floor, integer clamp, and the reader's index / count, product rounded to float32, + min. The old min + index * step with an announced rounded step is gone. The round-trip paragraph names three cross-form value changes (mask width, names, quantized index) with the mask and the float as the two round-trip exceptions, and the §4 "whole list" sentence now says it lists §4's rows. Test row "The quantized index across the forms": [0, 10] @ 0.01 carrying 0.005 (tie, index 1), 0.123 (off grid, 12), 11.0 (clamp, 1000), file codec against serialize_compressed_float, and the 6666 / 20000 decode at 0xC2055C2A.

The refused first announcement. Refusal is terminal: a new bullet in the scope section, a sentence in the storm bullet, and test row "A refused first announcement" (refused as vocabulary_too_large, then a valid one refusing as second_announcement, then a body refusing as no_vocabulary).

#569, retain-unknown (§6.6)

V2. The kind-17 exclusion row now covers a node index met anywhere in a recursively walked payload, rejecting the whole record for one retain_lost and stating the sibling trade. The sentence claiming kind 17 never reaches the walk is replaced by the rule that the walk looks for it at every depth. Test row: an unknown outer table holding a nested pointer three bodies down beside an unknown scalar, retained nothing, retain_lost one.

V3. New paragraph after the collision argument: a retained record belongs to the body occurrence that carried it and is discarded when a known ancestor is reset or replaced by a later legal occurrence (repeated table field, union arm written again, duplicate map key, keyed slot written again). Test row: child { future = 7 } then child { known = 2 }, the save carrying no future.

Applied, not decided

• The quantized shape drops bits: the width derives from the triple by the rule, on the array row's precedent (min, max derive the count width). A triple whose delta or delta / res is not finite, whose min is not below max, or whose res is not above zero is refused as a hostile shape, which is SPEC.md's non-conforming declaration meeting the page's existing "every width is checked before it is used". The hostile-shape test row gains those two cases.

Silences, left as found

• V3 and the counters. Records discarded with a reset occurrence: the ruling does not say whether they count retain_lost, and retained was already incremented when they were kept. Not decided.
• A quantized index above count on the message wire. The page's existing ranged-offset rule (reconstruct and clamp, not damage) reads on it. The packet wire's read rejects. Not restated either way.
• D2's corpus. tables/backend/Backend.schema does not yet declare Payload and Envelope, and the build version, backend_conn.bin and the batch vector's manifest line are the codec PR's to re-pin, as the page already says of every form-2 golden. The manifest line kind for a batch vector is unchanged.
• The three envelopes as batches of one. Pinned on the page at 54 / 150 / 45 under the existing test row. No vector names given.

🤖 Generated with Claude Code