Skip to content

tables: maps in the language — the spelling, the key rules and the generated entry (#380) - #487

Merged
gafferongames merged 8 commits into
mainfrom
maps-cpp
Sep 4, 2026
Merged

tables: maps in the language — the spelling, the key rules and the generated entry (#380)#487
gafferongames merged 8 commits into
mainfrom
maps-cpp

Conversation

@gafferongames

@gafferongames gafferongames commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

The LANGUAGE half of maps (docs/SPEC-TABLES.md §2.8): the map[K]V spelling, every key rule and its refusal, the variable-class derivation, the generated <Table><Field>Entry table, the names the construct claims, the two projections that record it, and every target refusing a map-bearing unit by name. The codecs are not here and the "What is NOT here" section below says exactly what that leaves.

Based on #479's page, which is merged; this PR implements it.

The spelling and the key

ships map[string(32)]ShipConfig. The value is a whole field spelling, so map[uint32]*ShipConfig, map[uint32]?[..4]int32 and map[string(16)]map[uint8]Item are one production and a map of maps is nothing special.

Keys are bounded strings and integers and nothing else. Each refusal names its reason: an enum key is refused naming [E]T with the enum the author wrote substituted in; bool, floats, flags, bits(N), bytes(N), a 128-bit integer, a fixed-point key, a type, a table, a pointer, an optional and a union each get their own sentence. A map in a type body, [..N]map, [N]map, ?map, a default on a map and an attribute on a map are refused by name. So is a qualification or a default on the KEY: the grammar takes map[uint32 | max = 4]int32 so the diagnostic can say what is wrong, which is that a key is an identity and clamping an identity merges two entries. was and json stay legal, because both are about the field rather than the construct.

The enum-key refusal is the only one that names a replacement, and it never names one this checker refuses: map[E][..4]int32 and map[E]*V get the sentence that says what [E]T takes instead, with the value spelled as written, the array bound kept.

The generated entry

Every map generates <Table><Field>Entry with the field's name in §11's PascalCase, so Fleet.by_id generates FleetByIdEntry; a map whose value is a map names the inner entry <OuterEntry>ValueEntry. The entry is a real table of the closure, declared before its holder. It rides as kind 14 over element kind 13 with the two constant field ids 0xA079 and 0x9194, which is what makes a user's own [..N]Pair the same bytes.

It is not a root. §2.8 makes the entry the one exception to §7's "a root is any table", so the C++ reference emits it no Open, no Cook, no Save and no Load; its walk, its layout and its cook body are the whole of what it carries.

A map is a variable edge, whatever its key and value are, so its holder rides with the pointers and has no block form. Neither does the entry: §2.8 states that a map's entries are never rows, and the rule is categorical rather than derived, because an entry of a string(32) key and a by-value value looks perfectly fixed and would otherwise take a block form, its counts, its storage and its projection asserts. Where a table has no block form, the reason now names the edge the walk found, a pointer or a map or the member that reaches one.

The names it claims

Eight against the field, on the rule §11 already states for the block form's row accessors: <Table> plus the field's PascalCase, then Entry, Insert, Find, Erase, Each, IndexMeasure, Index and IndexFind. The runtime names TableMap and TableMapIndex are claimed the way TableKeyed is: whenever a unit declares a table, with the construct rather than with the codec.

The two projections

A generated entry is ANONYMOUS in both. Its record line carries the holder's wire id and the map field's wire id joined by a dot, in place of a name, and both projections sort over that text, so a was rename of the map field moves no line, no record order and no build version. The map field's own line is an array line: kind=14, array=map, elem= the entry's storage size, and no bound=, because a map declares no extent.

The baseline records the map shape and the key's kind and capacity, with two policy rows: a key kind is fixed, because a key under a kind the reader does not declare empties the map; a key bound is an extent, because an entry whose key does not fit is skipped whole and counted. [..N]Pair to map[K]V is one WARN saying the bytes are the same where the element's fields are exactly a key and a value, and that the map direction gains the order check.

Two version bumps ride here, and they move numbers. The baseline rendering version goes to 5 (two new judged tokens) and the two committed baselines are regenerated with their history intact. The cook projection's FORM version goes to 2, because this rendering emits tokens the last one did not, so every unit's build version moves, map-free ones included. §20.2's worked number, the compiler's pin, docs/USAGE.md's printed number, the build-version goldens, the generated headers' BuildVersion constants and all 84 cooked conformance artifacts are re-pinned in the same commit.

schemafmt

needSpace had no case for map, so every command rewrote map[string(32)]ShipConfig to map [string(32)]ShipConfig in place, silently and idempotently. Fixed, with the pre-existing [..4]*Node to [..4] * Node mangling one line away, which is why tables/stream/Stream.schema moves. The formatter's structural safety check rendered no map at all, so it could not tell two different maps apart; it renders the whole field type now. The gate reads §2.8's example off the page and compares bytes.

Docs

docs/SPEC.md gains map in the reserved words and a Map production in the Type grammar with the same TABLE BODIES ONLY note ? and * carry; #479 is SPEC-TABLES.md only, so nothing covered it. In docs/SPEC-TABLES.md: §11's Maps list gains the 128-bit and fixed-point keys the checker already refuses, §20.1 and §20.2 say that elem= on a map line is the entry's storage size like every other array line, and §20.2's two worked numbers are re-pinned.

What is NOT here

No codec, in any language. Every target, the C++ reference included, refuses a unit whose table closure declares a map, by name, naming the fields and saying the codecs are this issue's next PR. Nothing half-emitted is reachable.

What the C++ emitter does produce for a map-bearing unit, behind that refusal, is the three entry structs with their records, their Reset, their descriptors and their cook bodies. The map field's own storage is not emitted yet: Fleet comes out with /* ? */ ships = 0; for each map slot, which does not compile. That is the codec PR's first job, and it is why the refusal is unconditional rather than a warning.

Also not here: the sixteen-byte slot's writer and reader, the sort, the ascending check, the duplicate merge, Find, the builder's insert and erase, the text form, cook-check's entry-array check, the conformance corpus and the goldens.

Gates

go build, go test ./..., make check, make tables-zero-cost, make conformance-generate with a clean tree after, gofmt -l, go vet, and golangci-lint v2.12.2 at 0 issues. The zero-cost gate's symbol list gains the map machinery, so a map symbol leaking into a map-free unit's header is a build failure.

🤖 Generated with Claude Code

gafferongames and others added 7 commits September 4, 2026 13:48
… the generated entry (#380)

The scanner takes `map` as a keyword; the AST carries the value as a whole
FIELD spelling, so a map of arrays, of optionals and of maps are one
production; the parser reads `map[K]V` wherever a field type stands.

The IR gains ir.TMap and Field.MapEntry — the generated `{ key, value }`
table, a real table of the closure with its own record, Reset and descriptor.
ir/tablemap.go carries the derived facts: the entry's name, its two constant
ids as ordinary hashes of two ordinary names, the census a refusal names.
VariableTables gains §2.8's clause (a map is a variable edge whatever its key
and value are) and TableClosure reaches through the entry.

The checker resolves the value through the ordinary field path and holds every
rule §2.8 states, each refused by name with the page's own reason: a map in a
`type` body, an enum key (naming [E]T), bool, float, flags, bits(N),
bytes(N), fixed, a type, a table, a pointer, an optional and a union keys, a
bound on the map, ?map, a default, an attribute, the by-value cycle through
the entry, and a declared table under the claimed <Table><Field>Entry name.

Every other backend refuses a map-bearing unit by name, through
compiler/tablesmaps.go per the registry split.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
#380)

The IR facts a map's codec will read, and the gate that keeps a half-codec
from being emitted before one exists.

- ir/tablekind.go: a map rides as kind 14 over element kind 13 — an array of
  the generated entry, no new kind and no new framing (§2.8, §3) — and
  TableTypeSpelling renders `map[K]V` so a diagnostic and a descriptor name
  the declaration the author wrote.
- ir/blocklayout.go: a map field is ONE sixteen-byte piece at eight (§2.8,
  §7.2) — an int64 self-relative reference and a uint32 count, then padding —
  one piece and not two for the reason the block form's out-of-line triple is
  one: both backends spell it as one member of a TableMap type, and a port
  that walked two would account for twelve bytes where sixteen are written.
- Every backend refuses a map-bearing unit by name, the C++ reference
  included, until its codec lands (compiler/tablesmaps.go). The refusal names
  the fields and schema#380, and target_cpp drops its call and registers
  through registerMapCarrier when its half lands.
- The zero-cost gate's header scan gains the map symbols (§2.2, §2.8): every
  unit it scans is map-free by construction, and the symbols say so
  mechanically rather than by inspection.
- The parser takes `?` on a key so the checker's OPTIONAL-key diagnostic lands
  where the spelling is, rather than a parse error naming a bracket.

Held by: compiler/tablesmaps_test.go (the derived facts at three depths, the
two constant ids, every target's refusal, and a map-free unit untouched) and
24 new rows in internal/check's refusal suite.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…380)

refuseMaps has two shapes — no carrier yet, and a carrier named — and only the
first had a call site. The test registers cpp through registerMapCarrier and
holds the second, so the day target_cpp.go registers for real the message it
produces is already pinned. It also keeps the registry function from being
dead code between the two PRs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
#482 excluded the block because maps did not parse. They do now, so the block
is anchored like every other declaration example and the header comment says
what the gate actually is: the PARSER's, so a block enters the moment its
construct has a grammar, whether or not a backend carries the codec.

Negative control: respelling the example to the REJECTED map<K, V> form turns
the gate red, naming the page, the anchor and the line.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ir.TableTypeSpelling rendered map[uint32]*ShipConfig as map[uint32]ShipConfig,
so a diagnostic and a descriptor would have named a declaration nobody wrote —
and map[K]T against map[K]*T is exactly the edit §3.1 spends a kind to keep
from being silent. Pinned by TestMapTypeSpelling over the three shapes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…to its type (#380)

Every command formats a source in place before it processes it, so a spelling
the formatter does not know is a spelling it silently rewrites. `needSpace` had
no case for the `map` keyword and none for the `]` that closes a map key, so
§2.8's own example came back as `map [string(32)]ShipConfig`,
`map [uint32] * ShipConfig` and `map [uint32] ?[..4]int32`: idempotent, and
away from the page for good.

The `] * ` half is not new. `[..2]*Chunk` in the stream corpus has been stored
as `[..2] * Chunk` since pointers landed, because type position was recognized
only at index 1. It is the same defect one line away, so it is fixed here.
Type position is index 1, the `]` that closes an array bound or a map key, and
the optional `?`. No expression in this grammar ends in either, so the rule
takes no case away from multiplication.

The formatter's SAFETY CHECK was blind to the construct too. Format re-parses
its own output and compares the two ASTs structurally, and the fingerprint
rendered no map at all, so two different maps and a map against a bare field
all fingerprinted alike. It renders the whole field type now: the key, the
bound, the star and a nested map included.

The gate reads §2.8's example OFF THE PAGE and compares the formatted bytes,
with the mangled spellings as its controls. Each is an input schemafmt itself
used to produce, and each must come back as the page states it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…nd SPEC.md carries the grammar (#380)

The FRONT END's half of §2.8 and §11, and the SPEC.md lines the construct was
missing.

- A QUALIFICATION OR A DEFAULT ON THE KEY is a refusal with the page's own
  sentence, not a parse error about a bracket. The grammar takes
  `map[uint32 | max = 4]int32` and `map[uint32 = 5]int32` on the terms it
  already takes `map[?uint32]`, so the diagnostic names the real problem: a key
  is an identity, and clamping an identity merges two entries.
- THE ENUM-KEY SUGGESTION never names a spelling the next compile refuses, and
  it keeps the value's array bound. `map[E][..4]int32` suggested `[E]int32`, a
  different declaration; `map[E]*V` suggested `[E]*V`, which the checker
  refuses as a named follow-on. The replacement clause now asks whether `[E]T`
  takes the value at all, and says what to do instead when it does not.
- THE MAP CLAIMS EIGHT NAMES against its field, on the rule §11 already states
  for the block form's row accessors: `<Table>` plus the field's PascalCase,
  then Entry, Insert, Find, Erase, Each, IndexMeasure, Index and IndexFind.
  The runtime names TableMap and TableMapIndex join internal/tablenames on
  TableKeyed's terms, claimed with the construct rather than with the codec,
  because a name freed now is a collision the day the codec lands.
- THE ENUM-KEY TEST fixture is renamed. It was an enum literally named `E`
  against a diagnostic that prints `[E]T`, so it could not tell a substitution
  from a hardcoded string. It is `ShipType` now.
- ir.MapEntryStructs, ir.MapKeyIsString and ir.HasMap go: scaffolding with no
  caller outside the tests. ir.MapFields lists the fields an AUTHOR WROTE, so
  a target's refusal no longer names `FleetLoadoutsEntry.value`, a generated
  name in no source file.

SPEC.md gains `map` in the reserved words and a `Map` production in the `Type`
grammar, with the same TABLE BODIES ONLY note the `?` and `*` productions
carry. #479 is SPEC-TABLES.md only, so nothing else covered it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…he entry's name (#380)

The DERIVED FACTS half, and the cook projection's form version with it.

THE BLOCK FORM. §2.8 states that a map's entries are never rows, and it was
not so. `blockFormable` asked only whether a table is variable-length, and
`FleetShipsEntry`, a `string(32)` key and a by-value `ShipConfig`, is fixed,
so it took a block form, its counts, its storage and its projection asserts.
The rule is categorical now, because the entry can look perfectly fixed. And
the reason a table has no block form NAMES THE EDGE THE WALK FOUND, a pointer,
a map, or the member that reaches one, instead of the hardcoded sentence about
a pointer, which was emitted verbatim over a table whose only variable edge was
a map.

THE TWO PROJECTIONS. A generated entry is ANONYMOUS in both (§18.1, §20.2):
its record line carries the holder's wire id and the map field's wire id
joined by a dot, in place of a name, and both projections SORT over that text.
The entry's generated name is derived from the field's source spelling, so a
`was` rename moved a line, moved the record order and invalidated every cooked
file, under an edit that moves no byte. A test pins that the rename now moves
neither the projection nor the build version.

The map field's line is an array line: `kind=14`, `array=map`, `elem=` the
generated entry's own storage size, which is the pitch its entries lie at and
what `elem=` means on every other array line this projection writes, and no
`bound=`, because a map declares no extent. The KEY's kind and capacity ride on
the entry's own `key` line, which is where a key edit moves the id.

THE BASELINE gains the map shape token and the key's two facts beside it, and
the two policy rows that judge them. A key kind is fixed, because a key
arriving under a kind the reader does not declare resets the whole map to
empty; a key bound is an extent, because an entry whose key does not fit is
skipped whole and counted. The `[..N]Pair` to `map[K]V` edit is one WARN with
the page's reason, and it is one finding rather than four: the tokens the
construct owns describe the same edit, so judging them beside the shape would
say it three times and say it harsher. Two new judged tokens bump the baseline
rendering version to 5, and the two committed baselines are regenerated with
their history intact.

THE COOK PROJECTION'S FORM VERSION moves to 2, because this rendering emits
tokens the last one did not. Every unit's build version moves with it, map-free
ones included, which is what a form version is for. §20.2's worked number,
compiler's pin, docs/USAGE.md's printed number, the build-version goldens, the
generated headers' BuildVersion constants and every cooked conformance
artifact are re-pinned in this commit.

THE ENTRY IS NOT A ROOT. §2.8 makes it the one exception to §7's "a root is
any table": it is reached only through the map that generates it, so the C++
reference emits it no Open, no Cook, no Save and no Load, and its walk, its
layout and its cook body are the whole of what it carries.

§11's Maps list gains the two keys the checker already refuses and the page
did not name: a 128-bit integer and a fixed-point key. The map slot's count is
an int32 like every companion, and the IR says so.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@gafferongames
gafferongames merged commit 183b6b6 into main Sep 4, 2026
20 checks passed
@gafferongames
gafferongames deleted the maps-cpp branch September 4, 2026 04:07
gafferongames added a commit that referenced this pull request Sep 4, 2026
#487 bumped the cook projection's form version, which moves every build
version; the two protocol ids are unmoved. Both numbers are taken from main's
own compiler at 6859a75, so the gate still says what it is for: this branch
reproduces the base's wire identities exactly, and `Count` reaches neither.

  examples     protocol 0x91a8e85156dfe2b1  build version 0x69d6a810cfa22717
  examples128  protocol 0x42050541a90eea8a  build version 0xedde8274fbab7f85

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
gafferongames added a commit that referenced this pull request Sep 4, 2026
* enums export Count beside Max, in all nine targets (#456)

The owner's ruling: `Count` is the number of DECLARED variants, excluding the
implicit `None`. `Max` stays the extent — equal to `Count` without headroom,
the declared `| max = K` with it. Flags keep `Count` and no `Max`.

Every emitter gains it in its own idiom, the spelling each already uses for a
flags declaration's `Count`: `Count = N,` in C++, C# and JS objects,
`<Enum>Count` in Go, `pub const COUNT` in Rust, `#define <ENUM>_COUNT` in C,
`static const int count` in Dart, `public static final <typ> count` in Java,
and `def count, do: N` in Elixir. The generated `<Union>Type` tag enums carry
`Max` alone: a tag set takes no headroom, so its count and its extent are one
number.

`E.Count` folds in schema expressions the way `E.Max` does, so `[..E.Count]T`
is a counted array over the declared variants beside `[E.Max]T`'s keyed one,
and unlike `.Max` it needs no bound — an enum whose `| max = ...` failed to
resolve still answers it exactly. `Count` is a claimed name and a refused
variant name, on the same terms as `Max`.

NEITHER WIRE MOVES. Count is generated-code surface: it enters no wire-shape
projection and no cook projection, and a new gate pins both corpora's protocol
id and build version as literals rather than reading the goldens a wrong
re-pin would carry along.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* ir, ast: the .Count reference in the two doc comments that name the node

`ExprHasEnumMax` and `ast.MaxExpr` both described `.Count` as a flags-only
spelling. It reaches enums now; the behavior was already right.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* goldens: the two build-version literals track main's cook form version 2

#487 bumped the cook projection's form version, which moves every build
version; the two protocol ids are unmoved. Both numbers are taken from main's
own compiler at 6859a75, so the gate still says what it is for: this branch
reproduces the base's wire identities exactly, and `Count` reaches neither.

  examples     protocol 0x91a8e85156dfe2b1  build version 0x69d6a810cfa22717
  examples128  protocol 0x42050541a90eea8a  build version 0xedde8274fbab7f85

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant