Skip to content

fix(cast): [branch-1.2] fall back for duplicate struct field names (#6671) - #7653

Merged
andygrove merged 2 commits into
apache:branch-1.2from
andygrove:backport-6671-branch-1.2
Nov 19, 2026
Merged

andygrove merged 2 commits into
apache:branch-1.2from
andygrove:backport-6671-branch-1.2

Conversation

@andygrove

Copy link
Copy Markdown
Member

Which issue does this PR close?

Backport of #6671 to branch-1.2. Closes #6591 on this branch.

Rationale for this change

Spark accepts a cast to a struct type with repeated field names, but Arrow Java collapses the fields by name when the native result is imported, so the query fails with ArrowArray struct has 2 children (expected 1) (#6591, priority:high). branch-1.2 has the same cast serde and the same import path. The fix routes that shape through fallback before it crosses Arrow FFI. See #6671 for the full analysis.

What changes are included in this PR?

Cherry-pick of bb76f12 (git cherry-pick -x). CometCast.isSupported rejects a target that contains a struct with duplicate field names, at any depth through arrays, maps and nested structs, and a null literal of such a type is declined too. It comes with the source PR's SQL file test and its tests in CometNativeCastSuite and CometLiteralSuite.

One adaptation: the import block of CometCast.scala conflicted because main and branch-1.2 list different org.apache.comet.serde names. The pick keeps branch-1.2's serde import line and takes only the changed DataTypeSupport import line, which is the only line the source commit touches there. The diff against the source commit is otherwise identical apart from line numbers and blob hashes.

Other release branches: branch-1.2 is the newest release branch, so there is no newer one to port to. Whether branch-1.1 needs a backport is not part of this PR.

How are these changes tested?

On this branch, with JDK 17 and the default Spark profile: ./mvnw test-compile succeeds. The new test in CometNativeCastSuite, CometLiteralSuite in full and the expressions/cast/cast_duplicate_struct_fields.sql file test pass (7 tests, 0 failures) against a native library built from this branch. No native source changed. With spark/src/main reverted to branch-1.2's version (keeping only the reason constant so the tests compile), cast to a struct with duplicate field names is unsupported, a null literal with duplicate struct field names is unsupported and the SQL file test fail, so the branch has the bug and the tests catch it.

* fix(cast): fall back for duplicate struct fields

Signed-off-by: Samarth Mukhija <139998526+Samearth17@users.noreply.github.com>

* test(cast): cover duplicate fields in SQL fixture

* fix(cast): reject duplicate targets through null paths

---------

Signed-off-by: Samarth Mukhija <139998526+Samearth17@users.noreply.github.com>
(cherry picked from commit bb76f12)
@andygrove andygrove changed the title fix: [branch-1.2] fall back for duplicate struct field names in cast (#6671) fix(cast): [branch-1.2] fall back for duplicate struct field names (#6671) Nov 16, 2026

@comphead comphead left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, great PR

@andygrove
andygrove merged commit a0273d5 into apache:branch-1.2 Nov 19, 2026
138 checks passed
@andygrove
andygrove deleted the backport-6671-branch-1.2 branch November 19, 2026 12:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants