Skip to content

fix(import): preserve root names for opaque extension carriers - #5

Merged
aunjgr merged 2 commits into
matrixorigin:feature/26155-substrait-anti-joinsfrom
aunjgr:fix/28968-opaque-root-names
Oct 8, 2026
Merged

aunjgr merged 2 commits into
matrixorigin:feature/26155-substrait-anti-joinsfrom
aunjgr:fix/28968-opaque-root-names

Conversation

@aunjgr

@aunjgr aunjgr commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

MO Q1 preparation crashes when Decimal256 is followed by another result column: its private four-field DuckDB carrier is counted as nested SQL fields, advancing past RelRoot.names. This importer change preserves the extension handler's opaque scalar boundary and rejects insufficient root names before indexing.

  • Add converter-scoped IsOpaqueType(LogicalType const&), defaulting to false. An opting-in carrier consumes one SQL name; ordinary STRUCT flattening remains unchanged, including opaque children inside a real STRUCT.
  • Check both STRUCT-name inspection and final root-name iteration. An aliased carrier without the explicit policy fails safely when its nested names are absent.
  • Extend the existing importer fixture with following-column values/types, typed NULLs, sort/fetch wrappers, nested opaque values, ordinary STRUCT controls and short-name rejection. Disable unrelated extension autoload in this fixture; retain explicit core-function setup.

Validation: frozen Pixi mo, Linux, Sirius's pinned DuckDB 069cc9f9b5be802405797faecc284961b07c70ef. The canonical importer test source passed 157 assertions / 3 cases in the existing native binding executable; the complete binding group with the prepared Sirius callback passed 1009 / 84. Changed ranges were formatted with clang-format 20.1.4 and the DuckDB style; git diff --check passed. The repository's separate macOS C-test CI is not claimed as a local pass.

This targets the compatible importer branch already pinned by Sirius. The additive C++ hook requires rebuilding consumers; the C ABI and numeric arithmetic are unchanged. Sirius must opt its canonical exact carriers into the hook when it pins this merged change. MO C will then pin that merged Sirius revision and rerun complete native/public validation; no unmerged dependency pin or rollout acceptance is claimed.

Refs matrixorigin/matrixone#28968 and matrixorigin/matrixone#28966.

CI follow-up: retained the legacy final-STRUCT format with only its root heading and added an independent value/name regression. Bounds are checked before actual array reads, rather than requiring an unused trailing cursor to remain within the heading list. The five later distribution failures reproduce at clean importer 95d9ce8d78490db3991ab6145653716aa3ec42c9 with its bundled DuckDB d8cdaa33fda8df955cc76ef58a280f68f4cd43fa: issue_01:26 (DISTINCT_FROM), TPCH:62 and TPCDS:113 (CTE), literals:197 (existing expected-error mismatch), and types:85 (empty-plan root). All five were selected explicitly and reproduced (80 assertions, 5 failures) with no source changes. The clean-base Linux job also fails STRUCT field selection at line 46 and then crashes in nested types at line 66 before reaching those later cases. This PR retains safe rejection of those unsupported/mismatched plans; no test is skipped or relaxed and full distribution CI is not claimed green.

@aunjgr
aunjgr marked this pull request as ready for review October 8, 2026 17:26
@aunjgr
aunjgr marked this pull request as draft October 8, 2026 17:38
@aunjgr
aunjgr marked this pull request as ready for review October 8, 2026 18:06
@aunjgr
aunjgr merged commit 99c7ca3 into matrixorigin:feature/26155-substrait-anti-joins Oct 8, 2026
5 of 6 checks passed
zuyu pushed a commit that referenced this pull request Oct 14, 2026
* fix(import): preserve root names for opaque extension carriers

* fix(import): retain final STRUCT root heading compatibility
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant