Skip to content

perf: compute try-mode integer +, -, and * in one pass - #7698

Draft
mbutrovich wants to merge 2 commits into
apache:mainfrom
mbutrovich:perf-try-arith-one-pass
Draft

mbutrovich wants to merge 2 commits into
apache:mainfrom
mbutrovich:perf-try-arith-one-pass

Conversation

@mbutrovich

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #7695.

Rationale for this change

See #7695.

What changes are included in this PR?

  • Try-mode +, -, and * on Int8 through Int64 use a new kernel. One pass computes the wrapping values and whether any row overflowed. A second pass, which builds the validity of the rows that didn't overflow, runs only when one did.
  • A scalar operand is no longer expanded to an array. Two scalar operands return a scalar, as the ANSI path already does.
  • Values under null slots are left as computed instead of zeroed. The Arrow format doesn't require a null slot to hold a particular value (Columnar.rst).
  • Try-mode division keeps the existing kernel.
  • benches/checked_arithmetic.rs gets try-mode cases for Int32 and Int64 add and mul. The inputs have no overflow, one overflow per batch, random overflows in about 1% of rows, and overflows only under nulls. There are also a scalar case and a chained try_add(try_add(a, b), c). The ANSI cases are the control, since this PR doesn't touch them.

How are these changes tested?

The tests are in native/spark-expr/src/math_funcs/checked_arithmetic.rs. test_try_integer_arithmetic_matches_i128 compares every row with exact i128 arithmetic for each type and op. It sweeps pairs of edge values in the array, scalar, and null-scalar layouts. It also moves one overflowing row through sliced batches of 300 rows, with the row valid and with it null, and with overflowing values under the other nulls. It also passes against main, so it checks that the new kernel changes no result.

Benchmarks

TODO

AI usage disclosure

I wrote this PR with help from Claude but understand and support the core approach, implementation, and test coverage.

@github-actions github-actions Bot added enhancement New feature or request performance area:expressions Expression evaluation labels Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:expressions Expression evaluation enhancement New feature or request performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf: compute try-mode integer +, -, and * without a per-row overflow branch

1 participant