tables: wstring(N) on the id-table wire, kind 33 (#522)

[gafferongames]

Closes #522's implementation half. wstring(N) rides kind 33 on the
id-table wire in the C++ reference, the Go oracle and the tool, and
SPEC-TABLES.md §11's closure refusal — the land-and-expand half #510 wrote and
#572 widened — is deleted with its tests inverted to acceptance.

The page half landed as #532/#592 and is law here: nothing below decides
anything the page states, and every silence I had to settle is listed at the
bottom.

What kind 33 is, in the reference

L, a BYTE length, then L / 2 UTF-16 code units two bytes each
little-endian (§3). An odd L is framing damage on the body that carries it.
An unpaired surrogate or a zero code unit is DAMAGE and not data, checked on
the payload AS IT ARRIVES and before the reader's bound: the field reads its
declared default, one malformed counts, and the parent reads on past L. A
payload longer than the bound clamps at a code unit boundary and never
splits a pair — a high surrogate whose low half did not fit is dropped with
it — so no clamp can invent the one thing no wire may put into storage.

Storage is §7.2's row: char16_t[N + 1] and an int32 used length in CODE
UNITS, with the terminating zero unit at index length. The cook writes the
units as two-byte scalars in the cook's byte order, because a record is
written piece by piece and a swap has to know where every scalar begins. The
text form is §16.2's row: the same text transcoded at the boundary, through
ONE escape grammar shared with kind 12 — an unpaired surrogate built in code
writes U+FFFD, a zero unit writes the \u0000 escape (§16.3).

Every site, and its control

wstring(N) has four kind-33 sites and they take one rule, because §3 states
it once for the kind: a table's own field, a type a table reaches, a
union arm, and an element (a type inside an array). Each is declared
once in examples-wide/Caption.schema and read once in
test/wide/table_main.cpp, whose rows are named from the serialize corpus
with -table on them so the two wires' rows line up by name.

wstring(N) as a MAP KEY is refused by name (§2.8) and has no site;
*wstring is specified ahead of its implementation (§2.5) and no backend
emits the blob record, so it has none either. Both refusals are held by their
own test.

Six controls, each quoted red in the report:

control	what it removes	the row that goes red
wide-table-surrogate-negative-control	the pairing rule from the emitted runtime	wstring-table-refuse-first-high-surrogate-alone
wide-table-zero-unit-negative-control	the zero-unit rule	wstring-table-refuse-nul-as-the-only-group
wide-table-clamp-negative-control	the high-surrogate drop in the clamp	wstring-table-clamp-never-splits-a-pair
wide-table-odd-length-negative-control	the odd-L check	the odd-L row's report.malformed
wide-table-byte-length-negative-control	L's doubling on write	the default-and-count row's buffer[3] == 6
tables-wire-fuzz-wide-text-negative-control	the pairing rule, under the FUZZER	a report divergence at mutant 78954

The cook's byte order takes a seventh, run by hand rather than pinned as a
target: with table_cook_units replaced by a memcpy of the whole run, the
gate goes red on wstring-table-cook-unit-byte-order and on nothing else. The
older difference check, that a little and a big cook are not byte-identical,
stayed GREEN under that sabotage — it is satisfied by the swapped
label_length and seq alone — which is why cook_layout now reads all
three units of the char16_t[N + 1] byte for byte in both orders.

The corpus, the conformance rows and the fuzzer

examples-wide/ gains Caption.schema beside the renamed WideText.schema
(the rename removes a WideTable.h collision with tables/examples). Five
instances are pinned under testdata/wire/tables/wide_*.bin and named by the
conformance manifest as their own widedemo unit — a leg without the
construct answers ABSENT, exactly as it does for blobdemo. wide_sites
carries the kind at all four sites at once, which is what gives the wire
fuzzer's new ill-formed wide text and odd L passes four places to
plant at (§4.2).

No wire golden moved. Adding a kind changes no existing byte, which is the
closed set doing its job; what moved is the emitted runtime and the JSON
walker, re-pinned.

What the merge with main moved

Three PRs landed while this branch sat, and all three reach package wide.

The protocol id moved once, 0xcd3b46b8e6e1a6d7 to
0x4b9b89c718af5d5e. That is #616's arm-defaults rule applying to this unit
like every other, and it applies to Caption.schema too because
Caption.schema is now part of the unit — the file this PR adds joined the
move rather than causing one. testdata/golden/wide/id.txt and the announce
bytes carry it.

#621 takes the text runtime's lengths as the wire's own 64-bit numbers, which
reaches kind 12's TableUtf8Valid and TableUtf8Clamp in this unit's
generated runtime. Kind 33's twin is unchanged and does not need the same
hardening: every kind-33 length passes room() before it is used, so it is
already bounded by the buffer and non-negative, and *wstring has no blob
path to reach the helper by (§2.5).

#612's unit registry emits WideView.h and WideView.cpp for this unit.

All of it is regeneration from the merged compiler, proved by go-test's
golden comparison rather than asserted.

The measurement

None claimed. The kind is a wire feature and the readers' kind switch gained a
case, so a sitting is owed, but no receipt for one is committed here, and this
bench could not produce a valid window: BENCH-STANDARD §2.5's load preamble
records a one-minute average between 4 and 6 throughout, with three other
lanes compiling beside this one. The numbers an earlier revision of this
description carried have no CSV under bench/results/ behind them, so they
are withdrawn rather than repeated. The sitting is owed before the tables
layer's next release gate, not before this merge.

What the page left silent

1. The cook's byte order for a wstring buffer. §7.2 gave the storage row
   and never said whether the units are swapped in a foreign-order cook. The
   general rule settles it — a record is written piece by piece and a swap has
   to know where every scalar begins — and a char16_t is a two-byte scalar.
   Settled on the page here: §7.2's wstring(N) row now states that each
   unit is written at its own two-byte width in the cook's order, never as a
   run of bytes, and that the terminator is a unit like any other. The narrow
   twin's char[N + 1] is order-free and gave no precedent.
2. The write side. SPEC.md §4.12 said a zero code unit is refused "on
   every write in every target", while SPEC-TABLES.md §3 leaves the table
   wire's write side unrescoped and kind 12's table Save carries no twin of
   §4.7's write-side check. Settled on the page here: §4.12's sentence is
   scoped to the PACKET wire, and the table wire's Save is stated to check
   the used length and nothing about the content, which is the shape kind 12
   already has there. Ill-formed text is a read-side verdict on that wire.
3. The message form's content rule and clamp for KIND 12. §3.3 states both
   and neither the C++ message loader nor the Go message decoder carries them
   for kind 12 today, a tables: widening on read, the refusal reasons, ill-formed text and comments (#532) #592 gap that predates this PR. Kind 33's half is
   implemented here (terminal refusal, a clamp that never splits a pair);
   kind 12's twin is owed and untouched. Filed as The message form's content rule and clamp for string(N) are stated in §3.3 and carried by neither engine #620.
4. A text map VALUE does not compile, on main. map[K]string(N) emits
   const /* ? */ * from TableEntryFound. Verified against origin/main;
   wstring(N) as a map value inherits it exactly, and no page sentence
   refuses the declaration. Not fixed here: it is kind 12's problem first and
   a design question about the entry's value/value_length pair.
   Filed as A text map value does not compile: map[K]string(N) emits const /* ? */ * in the C++ reference #619.
5. SPEC.md §4.12 contradicted itself — "an odd L as framing damage and
   nothing else refused" sat two sentences before "The validation above holds
   on both". The first clause is deleted as a prose bug; the goldens carry the
   rule.

The cell that moves on #366

wstring(N) on the table wire — cpp: ❌ to ✅.

🤖 Generated with Claude Code