tables: a blob string on a message body carries the content rule (#632) - #671
Merged
Conversation
… first (#632) docs/SPEC-TABLES.md §3.1 states kinds 12 and 33's content rule MET AT A NODE: "A TEXT blob's CONTENT is refused on the same terms", so a *string blob whose bytes are not well-formed UTF-8, or which carries a zero byte, is damage. §3.3 says a form-2 body's content rules are §3's, unchanged in what they reject, and that what differs is only the recovery, which a bit stream does not have. Neither engine carries it at the message site, where both carry it at the file site (decodenodes.go and pointers.go). The two gates are the page's rows over blobdemo's Catalog, whose numbering reaches a *string blob through note and a *bytes blob through thumb. test/tables/message_blob_main.cpp is the C++ reference's and TestAStringBlobRecordOnAMessageBodyCarriesTheContentRule is the oracle's. Both are RED at this commit: the truncated sequence, the zero byte, the overlong encoding and the lead byte 0xFF all load with a silent report. The last row of each is a *bytes blob carrying the same bytes, which must stay silent: a *bytes blob is bytes and never text, so the rule is the string id's alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ule (#632) Both engines read a form-2 blob record's bytes through the SAME function the FILE form reads one with, so there is one rule and no second copy. The C++ emitter: messagevariable.go's PASS TWO, where the record's bytes are already in hand. The align a blob record spends before its bytes leaves the span on a byte boundary, so the same pointer goes to the runtime's TableUtf8Valid and to the memcpy below it, and the line is emitted under rootReachesStringBlob, the same guard pointers.go emits the file form's under. The check does NOT ride in MessageRecordScan, which is the framing walk LoadMeasure shares: ill-formed content is sizeable, so a measure still answers the region the framing commands and only the decode refuses. The Go oracle: messagedecode.go's placement loop calls textValid on the record's bytes under the TString arm alone, which is decodenodes.go's line for the file form. What differs from the file form is only the recovery, which a bit stream does not have: where a file counts the record malformed and reads on with every slot naming it null, a batch ends there. One malformed counts, the bodies before it stand, and nothing after is read (§3.3). A *bytes blob keeps no rule, because it is bytes and never text (§3.1), and both gates carry the row that says so. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… main left stale (#632) FOUR LINES ARE THIS BRANCH'S, one per emitted message reader whose root's numbering reaches a *string blob: blobs/AssetsTable.h twice, for Asset and for Catalog; maps/DocsTable.h once, for a map whose VALUE is a text buffer; and arms/GateTable.h once, for a blob reached only through a union arm. Each is the same line, the record's bytes handed to TableUtf8Valid. NO WIRE GOLDEN MOVED: the write side is untouched, and a read that refuses damage changes no byte a writer produces. THE REST WAS ALREADY STALE ON MAIN and `make tables-block-zero-cost` was red before this branch: e77093c (#658) added the retain walk's `case 15: case 30:` and e123f1b (#662) moved the map entry readers' text path, and neither re-pinned tables/maps. maps/ChunksTable.h, RunsTable.h, SlotsTable.h and SpansTable.h carry nothing of this branch's at all, and CellsTable.h carries none of its four lines either. The gate now compares 117 Table sources byte-identical to their pins. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…#632) ONE BLADE AN ENGINE, each removing exactly one line and each naming what it turns red. message-blob-accepts-ill-formed drops textValid from the oracle's message placement loop; message-emitter-blob-accepts-ill-formed drops the emitted TableUtf8Valid call from messagevariable.go's PASS TWO. Each blade is drawn twice. Against the PAGE: the oracle's rides in MESSAGE_FORM_CONTROLS against TestAStringBlobRecordOnAMessageBodyCarriesTheContentRule, and the emitter's in the new tables-message-form-blob-negative-control, which builds test/tables/message_blob_main.cpp against a regenerated tables/blobs. The true run of that program is the target's own first step, so the gate is green before the blade is drawn, and the sabotage reddens four rows and leaves the two silent-report rows standing. Against EACH OTHER: message_blob_ill_formed_text is blob_str8 re-encoded as a batch of one with the fourth byte of note's eight byte payload replaced by 0xFF, and message_blob_zero_byte is the same wire with 0x00 there instead, so the two separate the two halves of one check. Nothing else moves, so what the two readers answer differently is the content rule and nothing else. The *bytes record beside it carries bytes no UTF-8 rule would accept and is never checked, which is the vector's own control. tables-wire-fuzz-message-blob-oracle-negative-control and its leg twin replay both vectors alone and require red on each, with the reports mirrored: the leg says 0,0,0,0,0,true,read, the oracle says 0,0,0,0,0,false,read the leg says 0,0,0,0,0,false,read, the oracle says 0,0,0,0,0,true,read AND THE LEG GREW THE ROOT THE VECTORS NEED. blobdemo's Catalog is the only root on the fuzzer's roster whose numbering can PLACE a text blob record, so it is the only one whose wire can carry §3.1's rule at a node. Its AssetsTable.cpp was already in CONFORMANCE_SOURCES, so the leg costs one include and one MESSAGE_VARIABLE row, and the corpus pass now runs 45 roots rather than 44. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
#632) No sentence of §3 or §3.1 moves and none is added: §3.1 already says a text blob's CONTENT is refused on the same terms as a kind 12 payload, and §3.3 already says a form-2 body's content rules are §3's. What was stale is HELD BY TEST, which carries one bullet per rule with its red clause and carried none for this one. The bullet names the six rows the two engines' gates run, the *bytes row among them, and the three ways a leg goes red. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #632
docs/SPEC-TABLES.md§3.1 states kinds12and33's content rule MET AT A NODE, "A TEXT blob's CONTENT is refused on the same terms", and §3.3 says a form-2body's content rules are §3's, unchanged in what they reject. A*stringblob record carried by a message body was read by BOTH engines without that check, where both apply it at the file site. The two engines were symmetric, so no vector separated them and the corpus could not see it.Red first
Two gates, one an engine, over
blobdemo'sCatalog, whose numbering reaches a*stringblob throughnoteand a*bytesblob throughthumb.test/tables/message_blob_main.cppis the C++ reference's andTestAStringBlobRecordOnAMessageBodyCarriesTheContentRuleis the oracle's. At the first commit both are RED on four rows each: the truncated sequence, the interior zero byte, the overlong encoding and the lead byte0xFFall load with a silent report.The two silent-report rows stand at that commit and never move: a well-formed blob loads clean, and so do the same ill-formed bytes under the reserved
bytesid, because a*bytesblob is bytes and never text.The fix, one function at the message site
messagevariable.go's PASS TWO, where the record's bytes are already in hand. A blob record aligns before its bytes, so the same pointer goes to the runtime'sTableUtf8Validand to thememcpyunder it, and the line is emitted underrootReachesStringBlob, the guardpointers.goemits the file form's under. The check does NOT ride inMessageRecordScan, which is the framing walkLoadMeasureshares: ill-formed content is sizeable, so a measure still answers the region the framing commands and only the decode refuses.messagedecode.go's placement loop, callingtextValidunder theTStringarm alone, which isdecodenodes.go's line for the file form.What differs from the file form is only the recovery: where a file counts the record malformed and reads on with every slot naming it null, a batch ends there. One
malformed, the bodies before it stand, nothing after is read (§3.3).The controls
One blade an engine, each matching exactly one line, each drawn twice.
Against the PAGE:
message-blob-accepts-ill-formedinMESSAGE_FORM_CONTROLS, andmessage-emitter-blob-accepts-ill-formedin the newtables-message-form-blob-negative-control, whose first step is the gate's own true run.Against EACH OTHER, on two pinned vectors:
message_blob_ill_formed_textisblob_str8re-encoded as a batch of one with the fourth byte ofnote's payload replaced by0xFF, andmessage_blob_zero_byteis that wire with0x00there instead.tables-wire-fuzz-message-blob-oracle-negative-controland its leg twin replay both alone and require red on each, with the reports mirrored:The fuzzer's leg grew one root for them:
blobdemo'sCatalogis the only root on the roster that can PLACE a text blob record.AssetsTable.cppwas already inCONFORMANCE_SOURCES, so it costs one include and oneMESSAGE_VARIABLErow, and the corpus pass runs 45 roots rather than 44.The page
No new rule and no sentence moved: §3.1 and §3.3 already state it. One HELD BY TEST bullet in §3.3 names the six rows and the three ways a leg goes red.
Results
go test ./...greenmake tables-wire-fuzz N=20000: 137 seeds over 45 roots, 123311 enumerated + 20000 random = 143311 mutants, 0 divergences, both the plain and the ASan legmake tables-message-form-negative-controlgreen, 49 blades including the new onemake tables-vocab-schemagreenmake tables-block-zero-cost: 117 Table sources byte-identical to their pinsmake teststops attables-json-list-walk, which is RED ON MAIN before this branch and unrelated to it: all tentables/mapsunits now emit the JSON list half and the gate still calls them list-free. Proven by regeneratingtables/mapswith main's own emitter through ago build -overlay.Goldens
Four lines are this branch's, one per emitted message reader whose root reaches a
*stringblob:blobs/AssetsTable.htwice,maps/DocsTable.honce,arms/GateTable.honce. NO WIRE GOLDEN MOVED. The other re-pinned files were already stale on main from #658 and #662, which moved the emitter without re-pinningtables/maps; the re-pin commit names them.🤖 Generated with Claude Code