perf: reuse input null buffer and add no-nulls fast path in Spark hex

[jackylee-ch]

Which issue does this PR close?

• Closes Further optimize Spark hex byte encoding: reuse input NullBuffer and special-case the no-nulls path #23674.

Rationale for this change

Follow-on from review feedback on #23473. hex_encode_bytes in datafusion/spark/src/function/math/hex.rs rebuilt the output NullBuffer row by row via NullBufferBuilder, even though hex encoding maps rows 1:1 (a null in maps to a null out), so the output nulls are always identical to the input's.

This PR applies both optimizations suggested in the issue:

1. Reuse the input NullBuffer. The input array's null buffer is now cloned (a cheap Arc-based clone) and attached to the output array directly, removing the per-row append_null/append_non_null bookkeeping.
2. Special-case the no-nulls path. When the input has no null buffer, a tight loop encodes every value without the per-row Option check.

What changes are included in this PR?

• hex_encode_bytes now takes the input array's Option<NullBuffer> and reuses it for the output array instead of rebuilding it with NullBufferBuilder; all call sites (top-level arrays and dictionary values) pass array.nulls().cloned().
• The encoding loop is split into a no-nulls fast path and a has-nulls path.

Benchmark results from the existing datafusion/spark/benches/hex.rs (null_density = 0.1) against main on an M-series MacBook:

hex_utf8/size=1024      time:   [17.466 µs 17.507 µs 17.552 µs]   change: −4.98%
hex_utf8/size=4096      time:   [73.478 µs 73.592 µs 73.727 µs]   change: −7.16%
hex_utf8/size=8192      time:   [147.63 µs 148.19 µs 148.82 µs]   change: −6.97%
hex_binary/size=1024    time:   [17.583 µs 17.613 µs 17.650 µs]   change: −5.48%
hex_binary/size=4096    time:   [74.234 µs 74.326 µs 74.435 µs]   change: −7.38%
hex_binary/size=8192    time:   [148.91 µs 149.04 µs 149.19 µs]   change: −7.19%

Are these changes tested?

Yes. Existing unit tests cover the string/binary/dictionary paths, and a new test test_spark_hex_utf8_with_nulls exercises the has-nulls path with interleaved nulls and empty strings to verify the reused null buffer stays aligned with the encoded values.

Are there any user-facing changes?

No.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 50.64935% with 38 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.69%. Comparing base (3b1ce16) to head (4467fd1).

Files with missing lines	Patch %	Lines
datafusion/spark/src/function/math/hex.rs	50.64%	34 Missing and 4 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #23721      +/-   ##
==========================================
- Coverage   80.69%   80.69%   -0.01%     
==========================================
  Files        1088     1088              
  Lines      367766   367826      +60     
  Branches   367766   367826      +60     
==========================================
+ Hits       296781   296803      +22     
- Misses      53276    53311      +35     
- Partials    17709    17712       +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[kumarUjjawal]

Hi @jackylee-ch we already have an open pr for this #23688

[jackylee-ch]

Hi @jackylee-ch we already have an open pr for this #23688

Thanks, sorry for miss that.