Repository navigation
Encode and decode SimpleAggregateFunction alias chains below Nullable - #32
Conversation
column_variant_matches stripped Nullable only at the top level and container arms called inner() before resolving aliases, so SimpleAggregateFunction(f, Nullable(T)) inside Array, Tuple, Map, Nested and Variant was rejected as a mismatched column. The encoder, decoder, skip path and resumable stream check also resolved only one alias level below Nullable.
There was a problem hiding this comment.
Auto-approved by the AI review gate on 61101e0. Details in the approver's comment on this pull request.
AI review gate — auto-approved
History (latest 5 heads, newest first)
|
polyglotAI-bot
left a comment
There was a problem hiding this comment.
Approving. I checked the fix against the code and against a live server.
- The root cause matches the description. The container arms in
column_variant_matchespeeled.inner()before alias resolution, and the four wire paths expanded only one alias level belowNullable. All four now resolve the full chain, the same asempty_columnalready did. - No
Nullable,Array, orLowCardinalitycan appear after the resolve.can_be_inside_nullablein the parser also resolves the full chain, and encode re-parses the rendered header. - State prefix and suffix,
is_encodable, and the Arrow export already recurse through aliases at each level, so no other site needs the change. - No hot-path cost. The resolve runs once per column, and for a non-alias inner it is the same single match as before.
Checks on 61101e0:
cargo fmt --check,cargo clippy --workspace --all-targets -- -D warnings,cargo test --workspace, andcargo +1.81 check --workspace --all-targetspass.- With the three production files reverted to
main, 5 of the new or changed tests fail. With the PR code they pass. - Live, server 26.10.4.207: I used the 14 type shapes from the new tests (Array, Tuple, named Tuple, Map, Nested, Variant, chained SAF, SAF over Point) with null maps
[0, 1, 0]and[1, 1]. The server accepts each type in CREATE TABLE, the crate'sINSERT ... FORMAT Nativesucceeds, and the server'sSELECT ... FORMAT Nativebytes are byte-identical to the bytes the crate sent. The decoded values match, and the Arrow export of each decoded batch succeeds.
One non-blocking note inline, about an error-class change for a caller-built Nullable(Nullable(T)).
Optional follow-up: the live suite has no column of this shape yet. A column such as Array(SimpleAggregateFunction(max, Nullable(UInt64))) in geo_saf_nested_roundtrip_through_server would pin the reported INSERT case against a real server.
8f8bfa4
AI review gate — Human approval in place of AI reviewApproved on the head commit by chernser (write access, not the author), standing in for the AI review:
What to do:
Evaluated at |
There was a problem hiding this comment.
Auto-approved by the AI review gate on 8f8bfa4. Details in the approver's comment on this pull request.
Inserting
SimpleAggregateFunction(f, Nullable(T))inside Array, Tuple, Map, Nested or Variant failed with a mismatched column error. Top-level columns worked.column_variant_matchesonly stripped Nullable at the top level. The container arms called.inner()before resolving the alias, so the child reached the matcher as a Nullable with no matching arm. The matcher now strips Nullable at every level.The encoder, decoder, skip path and resumable stream check also resolved only one alias level below Nullable. So
Nullable(SAF(f, SAF(g, T)))orNullable(SAF(f, Point))could not be written or read. All four now useresolved_physical_delegate_ref.Type names in the header and the nullability checks are unchanged.
Note
Medium Risk
Touches core native encode/decode/skip paths and batch validation for nullable and alias types; behavior change is localized but affects all nested SAF+Nullable wire handling.
Overview
Fixes
encode_blockrejecting columns typed asSimpleAggregateFunction(..., Nullable(T))(including chained SAF aliases) when nested in Array, Tuple, Map, Nested, or Variant, which previously surfaced as a mismatched-column /InconsistentBatcherror while top-level columns worked.column_variant_matchesnow unwrapsNullableat every recursion level and matches container children against the full type (no extra.inner()stripping before alias resolution), so physical column buffers align with nested nullable SAF shapes.Encode, decode, skip, and resumable stream paths switch from single-level
physical_delegate_reftoresolved_physical_delegate_refbelowNullable, so chains likeNullable(SAF(..., SAF(...)))orNullable(SAF(..., Point))round-trip. Type headers and nullability rules are unchanged.Adds broad roundtrip, validation, rev0 byte-framing, and
StreamDecodercoverage for these types, and updates a stream scanner test that now treats the framed block as complete instead ofUnsupportedType.Reviewed by Cursor Bugbot for commit 8f8bfa4. Bugbot is set up for automated code reviews on this repo. Configure here.