[SPARK-58043][SQL] Use explicit .apply instead of a case class companion as a function for Scala 3 compatibility#57129
Open
fangchenli wants to merge 4 commits into
Open
Conversation
…anion as a function for Scala 3 compatibility ### What changes were proposed in this pull request? Replace bare case class companions used as function values with an explicit `.apply` method reference across `catalyst`, `sql-core`, `hive`, and `sql-api` (190 call sites), e.g.: - `xs.reduceLeft(And)` -> `xs.reduceLeft(And.apply)` - `convertUnaryExpr(expr, IsNull)` -> `convertUnaryExpr(expr, IsNull.apply)` - `.map(sources.Not)` -> `.map(sources.Not.apply)` Two lines are reflowed to stay within the 100-character limit. ### Why are the changes needed? In Scala 2 a case class's companion object automatically extends `FunctionN`, so the bare companion can be passed wherever a function is expected. Scala 3 removed this (case companions no longer extend `FunctionN`), so these call sites fail to compile. Referencing `Foo.apply` -- the method the synthetic `FunctionN.apply` forwarded to -- restores the explicit form. This is part of the incremental Scala 3 readiness tracked in SPARK-54150. ### Does this PR introduce _any_ user-facing change? No. `Foo.apply` is the exact method the synthetic `FunctionN.apply` delegated to, so the change is behavior- and byte-identical on Scala 2.13. ### How was this patch tested? Existing tests. The `catalyst`, `sql-core`, `hive`, and `sql-api` modules compile under both the default build and `-Xsource:3`, `scalastyle` passes, and no `case-companion-function` migration warnings remain in these modules. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 4.8) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nion-function # Conflicts: # sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/V2ExpressionUtils.scala
… count for eta-expansion The case-companion -> .apply change in QueryStartedEvent's jobTags serialization eta-expands to a lambda, which the Scala 2.13 compiler lifts to a synthetic public $anonfun method on the class. Reflection-based test_number_of_public_methods sees it via getMethods(), so bump the expected count from 16 to 17. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
In Scala 2 a case class's companion object automatically extends FunctionN, so the bare companion can be passed wherever a function is expected. Scala 3 removed this (case companions no longer extend FunctionN), so these call sites fail to compile. Referencing Foo.apply, the method the synthetic FunctionN.apply forwarded to, restores the explicit form.
Why are the changes needed?
It's required for Scala 3 compatibility.
Does this PR introduce any user-facing change?
No
How was this patch tested?
Compiled with -Xsource:3
Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 4.8)