[SPARK-56977][SQL] RewriteNearestByJoin should respect joinType in the synthetic join

[zhidongqu-db]

What changes were proposed in this pull request?

This PR changes RewriteNearestByJoin to construct its synthetic cross-join with the user's joinType (Inner or LeftOuter) instead of always using LeftOuter. The Generate operator's outer flag continues to be derived from joinType == LeftOuter, so the externally observable semantics are unchanged.

Why are the changes needed?

The original implementation hardcoded the synthetic join to LeftOuter and justified it on the grounds that LEFT OUTER and INNER are equivalent for an unconditioned join when the right side is non-empty, and Generate(outer = false) would drop unwanted rows for INNER when right is empty.

That reasoning holds for correctness but has a major performance cost:

• INNER NEAREST BY cannot be planned as a Cartesian product. Spark's strategy picks CartesianProductExec only for Inner joins with no condition; an unconditioned LeftOuter join falls back to BroadcastNestedLoopJoin, which tries to broadcast the right side. When the right relation is large, the broadcast either OOMs or exceeds spark.sql.autoBroadcastJoinThreshold and the planner is left with no good option. CartesianProductExec partitions both sides and streams pairs, so it scales naturally with right-side size. Respecting the user's INNER join type re-enables this strategy for the common INNER NEAREST BY case.
• It also makes the EXPLAIN output misleading (shows LeftOuter even though the user wrote INNER).
• For INNER with an empty right side, the old plan generates one row per left input and then filters them away via Generate(outer = false) and the size(matches) > 0 filter -- extra work that respecting joinType avoids at the source.

Does this PR introduce any user-facing change?

No change in query results. EXPLAIN output for INNER NEAREST BY queries now shows Inner rather than LeftOuter for the synthetic join node, and the physical plan for such queries can now use CartesianProductExec instead of BroadcastNestedLoopJoin when the right relation is too large to broadcast.

How was this patch tested?

• RewriteNearestByJoinSuite: expectedRewrite now takes a joinType: JoinType and the existing tests (similarity/distance x inner/leftouter, EXACT, boundary k, self-join, nondeterministic ranking) assert the synthetic join matches the user's join type.
• Golden file sql-tests/results/join-nearest-by.sql.out

Was this patch authored or co-authored using generative AI tooling?

Coauthored-by: Claude Code (Opus 4.7), human-reviewed and tested

[andylam-db]

Since this wasn't caught in the previous PR, could we add tests to make sure the logic is correct?