Skip to content

perf: divide once per row in decimal / and div - #7699

Draft
mbutrovich wants to merge 2 commits into
apache:mainfrom
mbutrovich:perf-decimal-div-once
Draft

mbutrovich wants to merge 2 commits into
apache:mainfrom
mbutrovich:perf-decimal-div-once

Conversation

@mbutrovich

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #7694.

Rationale for this change

See #7694.

What changes are included in this PR?

  • spark_decimal_div_internal rescales the operands to the result scale instead of one digit past it, so the i128 and BigInt branches each divide once per row. / rounds half away from zero from the remainder, and div truncates.
  • With one digit fewer in the rescale exponents, some inputs take the other branch, as perf: divide once per row in decimal / and div #7694 describes. Both branches compute the same quotient.
  • benches/decimal_div.rs gets cases with Spark's result types: / on Decimal(10, 4) by Decimal(10, 3), div at scale 0, the BigInt branch, and integer div on Decimal(19, 0) with small and with full-range values. Arrow's decimal div is the control, since this PR doesn't touch it.

How are these changes tested?

The tests are in native/spark-expr/src/math_funcs/div.rs. test_decimal_div_matches_exact_quotient compares every row of / and div with an exact BigInt quotient across scale combinations on both branches. It covers both signs, exact halves, and values at the precision bound. It also passes against main, so it checks that the rewrite changes no result. test_integral_divide_overflow_through_kernel checks the ANSI long-overflow error on both branches.

Benchmarks

TODO

AI usage disclosure

I wrote this PR with help from Claude but understand and support the core approach, implementation, and test coverage.

@github-actions github-actions Bot added enhancement New feature or request performance area:expressions Expression evaluation labels Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:expressions Expression evaluation enhancement New feature or request performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf: divide once per row in decimal / and div

1 participant