Skip to content

fix: [branch-1.2] respect SQL execution config when captured keys are unset (#7348) - #7643

Merged
andygrove merged 2 commits into
apache:branch-1.2from
andygrove:backport-7348-branch-1.2
Nov 19, 2026
Merged

andygrove merged 2 commits into
apache:branch-1.2from
andygrove:backport-7348-branch-1.2

Conversation

@andygrove

Copy link
Copy Markdown
Member

Which issue does this PR close?

Backport of #7348 to branch-1.2. Closes #7312 on this branch.

Rationale for this change

A captured Comet setting may be unset on the session after a plan is cached. Inside a later SQL execution Spark propagates the current SQL configuration to the task, but if none of the captured keys remains set, CapturedSQLConf.withConf mistakes the absence of those keys for an unpropagated configuration and reinstalls the stale capture, so the task runs with settings the user has since unset. branch-1.2 has the same withConf check (it came in with #7302, backported in #7337). See #7348 and #7312 for the full analysis.

What changes are included in this PR?

Cherry-pick of 8386a34 (git cherry-pick -x) with no conflicts and no adaptations. withConf now skips the captured configuration when the task has a SQL execution id. The source PR's regression test in CometSessionExtensionsSuite is included.

Other release branches: branch-1.2 is the newest release branch, so there is no newer one to port to. Whether branch-1.1 needs a backport is not part of this PR.

How are these changes tested?

On this branch, with JDK 17 and the default Spark profile (4.1): ./mvnw test-compile passes, and CometSessionExtensionsSuite passes (3 tests, 0 failures) against a native library built from branch-1.2. With CapturedSQLConf.scala reverted to branch-1.2's version, the new test captured configs do not override a SQL execution after a setting is unset fails, so the branch has the bug and the test catches it. I ran the default Spark profile only.

@comphead comphead left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks

@andygrove
andygrove merged commit b204c51 into apache:branch-1.2 Nov 19, 2026
274 of 276 checks passed
@andygrove
andygrove deleted the backport-7348-branch-1.2 branch November 19, 2026 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants