[SPARK-46625][SQL][FOLLOWUP] Resolve identifier expression in InsertIntoStatement/V2WriteCommand table slot#56024
Open
haoyangeng-db wants to merge 1 commit into
Conversation
…ntoStatement/V2WriteCommand table slot
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This is a follow-up to SPARK-46625 (PR #55949 - "Place IDENTIFIER placeholder in command name slot").
SPARK-46625 moved
PlanWithUnresolvedIdentifierfrom wrapping the whole command into the command's identifier slot at parse time. ForInsertIntoStatementandV2WriteCommandthe placeholder now lives in.table, which is a non-childLogicalPlanslot (override def child = query). That PR correctly added explicit recursion for that slot inBindParameters(parameter binding) andResolveIdentifierClause(materializing the placeholder onceidentifierExpris resolved), but the same recursion was missing fromResolveReferences, which owns column / variable resolution.This PR adds two cases at the top of
ResolveReferences.doApplythat mirror the existing pattern: whenInsertIntoStatement.tableorV2WriteCommand.tableis an unresolvedPlanWithUnresolvedIdentifier, resolveidentifierExprviaresolveExpressionByPlanChildren(..., includeLastResort = true)(which runs the resolveColsLastResortpath:resolveVariables compose resolveOuterRef). The!identifierExpr.resolved` guard keeps the cases idempotent under bottom-up traversal.Why are the changes needed?
Without this,
INSERT INTO IDENTIFIER(<sql-variable>) ...fails analysis: theUnresolvedAttributefor the variable name sitting insidePlanWithUnresolvedIdentifier.identifierExpris never rewritten to aVariableReference. SinceResolveIdentifierClauseonly fires whenidentifierExpr.resolved && childrenResolved, the placeholder never materializes; the plan reachesPreprocessTableInsertionwith an unresolved attribute and errors out (e.g.UNSUPPORTED_INSERT.RDD_BASED).Repro on master before this fix:
The same shape applies to
OverwriteByExpression.table(e.g.REPLACE WHERE,REPLACE ON,REPLACE USINGvariants of INSERT) - fixed by the sameV2WriteCommandcase.Does this PR introduce any user-facing change?
No. Bug-fix only.
How was this patch tested?
New test added.
Was this patch authored or co-authored using generative AI tooling?
Co-authored with Claude Code.