Skip to content

[SPARK-46625][SQL][FOLLOWUP] Resolve identifier expression in InsertIntoStatement/V2WriteCommand table slot#56024

Open
haoyangeng-db wants to merge 1 commit into
apache:masterfrom
haoyangeng-db:spark-46625-followup-resolve-identifier
Open

[SPARK-46625][SQL][FOLLOWUP] Resolve identifier expression in InsertIntoStatement/V2WriteCommand table slot#56024
haoyangeng-db wants to merge 1 commit into
apache:masterfrom
haoyangeng-db:spark-46625-followup-resolve-identifier

Conversation

@haoyangeng-db
Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

This is a follow-up to SPARK-46625 (PR #55949 - "Place IDENTIFIER placeholder in command name slot").

SPARK-46625 moved PlanWithUnresolvedIdentifier from wrapping the whole command into the command's identifier slot at parse time. For InsertIntoStatement and V2WriteCommand the placeholder now lives in .table, which is a non-child LogicalPlan slot (override def child = query). That PR correctly added explicit recursion for that slot in BindParameters (parameter binding) and ResolveIdentifierClause (materializing the placeholder once identifierExpr is resolved), but the same recursion was missing from ResolveReferences, which owns column / variable resolution.

This PR adds two cases at the top of ResolveReferences.doApply that mirror the existing pattern: when InsertIntoStatement.table or V2WriteCommand.table is an unresolved PlanWithUnresolvedIdentifier, resolve identifierExpr via resolveExpressionByPlanChildren(..., includeLastResort = true) (which runs the resolveColsLastResortpath:resolveVariables compose resolveOuterRef). The !identifierExpr.resolved` guard keeps the cases idempotent under bottom-up traversal.

Why are the changes needed?

Without this, INSERT INTO IDENTIFIER(<sql-variable>) ... fails analysis: the UnresolvedAttribute for the variable name sitting inside PlanWithUnresolvedIdentifier.identifierExpr is never rewritten to a VariableReference. Since ResolveIdentifierClause only fires when identifierExpr.resolved && childrenResolved, the placeholder never materializes; the plan reaches PreprocessTableInsertion with an unresolved attribute and errors out (e.g. UNSUPPORTED_INSERT.RDD_BASED).

Repro on master before this fix:

CREATE TABLE t (a INT) USING PARQUET;
DECLARE OR REPLACE VARIABLE target_table STRING;
SET VAR target_table = 't';
INSERT INTO IDENTIFIER(target_table) SELECT 42 AS a;

The same shape applies to OverwriteByExpression.table (e.g. REPLACE WHERE, REPLACE ON, REPLACE USING variants of INSERT) - fixed by the same V2WriteCommand case.

Does this PR introduce any user-facing change?

No. Bug-fix only.

How was this patch tested?

New test added.

Was this patch authored or co-authored using generative AI tooling?

Co-authored with Claude Code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant