fix: generate_series overflow panics at i64 boundary and out-of-range dates#23723
Open
u70b3 wants to merge 1 commit into
Open
fix: generate_series overflow panics at i64 boundary and out-of-range dates#23723u70b3 wants to merge 1 commit into
u70b3 wants to merge 1 commit into
Conversation
… dates Two overflow fixes in the generate_series/range table functions: - Integer series: advancing past i64::MAX (or i64::MIN for negative steps) panicked in debug builds and silently wrapped in release. advance() now uses checked_add and clamps the series end so iteration stops after the last reachable value, matching PostgreSQL/DuckDB (e.g. generate_series(9223372036854775806, 9223372036854775807, 2) returns one row 9223372036854775806). - Date series: converting Date32 days to timestamp nanoseconds used an unchecked multiplication, panicking at planning time for dates outside the nanosecond timestamp range. It now returns a planning error. Supersedes apache#22250 (closed unmerged); the integer-side approach follows the one approved there. Closes apache#22208, closes apache#22193.
u70b3
force-pushed
the
fix/generate-series-overflow
branch
from
July 21, 2026 01:09
00ccd1e to
bb38d8e
Compare
|
Thank you for opening this pull request! Reviewer note: cargo-semver-checks reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch). Details |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #23723 +/- ##
==========================================
- Coverage 80.71% 80.71% -0.01%
==========================================
Files 1089 1089
Lines 368275 368291 +16
Branches 368275 368291 +16
==========================================
+ Hits 297250 297255 +5
- Misses 53309 53318 +9
- Partials 17716 17718 +2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Supersedes #22250 (closed unmerged by its author on 2026-06-04 after the approach was approved). Note: both issues are currently assigned to @xiedeyantu (idle since 2026-05; #22193 was explicitly blocked on #22250). I've left a note on both issues; happy to coordinate if the assignee has work in progress.
Rationale for this change
Two overflow bugs in the
generate_series/rangetable functions, both panicking in debug builds and silently wrapping in release:SELECT * FROM generate_series(9223372036854775806, 9223372036854775807, 2)panics withattempt to add with overflowwhen advancing pasti64::MAX. PostgreSQL and DuckDB both return one row (9223372036854775806), stopping after the last reachable value.Date32days to timestamp nanoseconds uses an unchecked multiplication, sogenerate_series(DATE '0001-01-01', ...)panics at planning time. Dates outside the nanosecond timestamp range (1677-09-21 – 2262-04-11) cannot be represented and must error cleanly instead.What changes are included in this PR?
datafusion/functions-table/src/generate_series.rsSeriesValue::advancenow takesend: &mut Self; thei64implementation useschecked_addand, on overflow, clampsendso the series terminates after the last reachable value (the approach approved in fix generate_series table function overflows #22250). The per-batch loop stops right after emitting the final value.TimestampValue::advancealready errors on out-of-range interval addition and is unchanged apart from the signature.checked_muland returns a planning error naming the offending argument.datafusion/sqllogictest/test_files/table_functions.slti64::MAX,range(end-exclusive) overflow, and first/second date argument out of range.Are these changes tested?
Yes:
cargo fmt, workspace clippy (avro,integration-tests,extended_tests,-D warnings),cargo test -p datafusion-functions-table, and the extended workspace suite (ci profile withavro,json,backtrace,extended_tests,recursive_protection,parquet_encryption) — all green.Are there any user-facing changes?
Only the bug fixes: integer series near
i64::MAX/MINnow return the reachable values instead of panicking or wrapping (matching PostgreSQL/DuckDB), and out-of-range dates produce a planning error instead of a panic. No API changes.