Repository navigation
Decode JSON values inside Dynamic shared cells - #29
Conversation
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit ec675b3. Configure here.
AI review gate — AI review passedClean review from: Cursor Bugbot. |
polyglotAI-bot
left a comment
There was a problem hiding this comment.
Approved. I found no correctness, safety or wire-contract defect. The inline comments are not blocking.
What I checked at ec675b3:
- Gate in a clean Rust 1.96 container:
cargo fmt --all -- --check,cargo clippy --workspace --all-targets -- -D warnings,cargo test --workspaceandcargo +1.81 check --workspace --all-targetsall pass. - Regression proof: with
src/native/binary_value.rsandsrc/native/type_binary.rsreverted to the merge base, the 7 new fixture tests fail. With the PR code they pass. - Server source at v26.8.1.2041-lts:
SerializationObject::serializeBinaryanddeserializeBinary,SerializationDynamic::serializeBinary,ColumnDynamic::serializeValueIntoSharedVariant,SerializationNothinganddecodeDataTypeImplagree with the decoder and with the new CODEC_CONTRACT text. Typed paths have no descriptor, other paths have one, Dynamic NULL is the single 0x00 byte, and the cell layout is the same in V1 and V2. When the server reads a value, it sorts shared paths by raw bytes and rejects duplicates by default, so the sort and the duplicate errors here match it. The Variant skip indexes the global discriminator order, whichnormalize_variant_alternativesgives. - Live server 26.8.1.2041: I decoded and re-encoded all 74 shared cells of
json_shared_rev0,json_shared_rev54485andjson_multipart_rev0, and encoded the bulktypedandnestedcolumns withencode_binary_value. I inserted them asDynamic(max_types=0)shared cells and compareddynamicType(d), toString(d)with the original server cells. The result is the same in a Memory table and after a MergeTreeOPTIMIZE FINAL. 18 cells have different bytes, and the server reads them the same. A JSON cell inencoded_shared_cells_insert_through_serverwould keep this covered in CI.
Merge: CHANGELOG.md conflicts with main after #28. The other files merge without conflict.
Nits: the too_deep() message change is not necessary for JSON support. The module doc still says "confirmed at v26.3". NullDefaultBudget now also meters missing typed paths and zero-width defaults, so a more general name would help. encode_json_value repeats the lookup that DynamicColumn::value_position does.
decode_binary_value and encode_binary_value now handle JSON, including JSON nested in Array, Tuple, Map and Nullable. Before, they returned Unsupported and bindings fell back to raw bytes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stop over-charging null JSON defaults, check dynamic child types on encode, and list SkipPlan variants explicitly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
a246301
ec675b3 to
a246301
Compare
There was a problem hiding this comment.
Auto-approved by the AI review gate on a246301. Details in the approver's comment on this pull request.
AI review gate — auto-approved
History (latest 5 heads, newest first)
|

JSON values stored in a Dynamic shared variant came back as raw bytes.
decode_binary_valuereturnedUnsupportedfor JSON, and bindings fall back to bytes when that happens.decode_binary_valueandencode_binary_valuenow handle JSON, including inside Array, Tuple, Map and Nullable.Note
Medium Risk
Large new codec surface in
binary_value.rsfor JSON and nested containers, with new bounds and error semantics that bindings may rely on.Overview
On-demand
decode_binary_value/encode_binary_valuenow materialize structured JSON (typed path columns plus sorted shared path cells), including JSON nested in Array, Tuple, Map, and Nullable. Bulk decode still leaves Dynamic shared blobs opaque; bindings use the helpers per cell.Encoding enforces the same rules as decode: typed paths need supported leaf codecs; untyped paths can keep Dynamic/Variant tagged as opaque bytes. Non-null
NothingisInvalid, notUnsupported. Binary descriptor complexity rises from 1000 to 2001 nodes (enough for JSON with 1000 scalar typed paths); depth errors readnesting exceeds 100.SkipPlancompiles once per descriptor so repeated array/map walks don’t re-clone type trees. Real-server fixtures and integration tests cover JSON in shared cells at revisions 0, 54460, and 54485.Reviewed by Cursor Bugbot for commit a246301. Bugbot is set up for automated code reviews on this repo. Configure here.