Repository navigation
Description
Apache Iceberg Rust version
main (5994664, 2026-11-06)
Describe the bug
1. year and month of dates and timestamps beyond chrono's range.
The spec defines year and month for every date and timestamp value. Iceberg Java's DateTimeUtil computes them through java.time, whose range covers every int day and long microsecond. iceberg-rust goes through chrono, whose dates end at about ±262,000 years, so:
- The array transforms (
Year::transform,Month::transform, via Arrow'sdate_part) return NULL for non-null inputs outside that range. A writer that uses them files such rows under anullpartition. A predicate on the source column then projects to a non-null partition value and prunes the file that holds the row. transform_literalreturnsDataInvalid(Fail to convert timestamp to date in year transform) for such timestamps, and panics for such dates (Date32Type::to_naive_date_opt(*v).unwrap(),transform/temporal.rs:80and:173).
| Input | Expected year | Expected month | Array transform | transform_literal |
|---|---|---|---|---|
timestamp i64::MAX µs (294247-01-10) |
292277 | 3507324 | null |
DataInvalid |
timestamp i64::MIN µs (-290308-12-22) |
-292278 | -3507325 | null |
not tested |
date i32::MAX (5881580-07-11) |
5879610 | 70555326 | null |
panic |
date i32::MIN (-5877641-06-24) |
-5879611 | -70555327 | null |
not tested |
The expected values use floor division to days and an exact civil-from-days conversion. i64::MAX microseconds is a common "end of time" sentinel.
2. Transform::adjust_boundary overflows at type extremes.
adjust_boundary (spec/transform.rs:775 onwards) computes v - 1 for < and v + 1 for > unchecked. With overflow checks (debug builds, tests) x < i32::MIN (int or date) and x > i64::MAX panic (spec/transform.rs:778). Without them the boundary wraps to one that matches every value, which only makes the projection over-inclusive.
For context: the related day floor, nanosecond bucket, decimal truncate literal, truncate overflow and nanosecond hour issues are already fixed on main (#3323, #3243, #3573, #3026), and LargeBinary truncate has an open PR (#3120).
To Reproduce
Append to crates/iceberg/src/spec/transform.rs and run cargo test -p iceberg --lib audit_out_of_range -- --nocapture:
#[cfg(test)]
mod audit_out_of_range {
use std::panic::{AssertUnwindSafe, catch_unwind};
use std::sync::Arc;
use arrow_array::{ArrayRef, Date32Array, TimestampMicrosecondArray};
use super::*;
#[test]
fn audit_out_of_range() {
for t in [Transform::Year, Transform::Month] {
let f = create_transform_function(&t).unwrap();
let ts: ArrayRef = Arc::new(TimestampMicrosecondArray::from(vec![i64::MAX, i64::MIN, 0]));
println!("{t:?} timestamp array [MAX, MIN, 0]: {:?}", f.transform(ts));
let d: ArrayRef = Arc::new(Date32Array::from(vec![i32::MAX, i32::MIN, 0]));
println!("{t:?} date array [MAX, MIN, 0]: {:?}", f.transform(d));
println!(
"{t:?} timestamp literal MAX: {:?}",
f.transform_literal(&Datum::timestamp_micros(i64::MAX))
);
let p = catch_unwind(AssertUnwindSafe(|| f.transform_literal(&Datum::date(i32::MAX))));
println!("{t:?} date literal MAX panicked: {}", p.is_err());
}
for (op, d) in [
(PredicateOperator::LessThan, Datum::int(i32::MIN)),
(PredicateOperator::GreaterThan, Datum::long(i64::MAX)),
(PredicateOperator::LessThan, Datum::date(i32::MIN)),
] {
let label = format!("{op:?} {d:?}");
let p = catch_unwind(AssertUnwindSafe(|| Transform::adjust_boundary(&op, &d)));
println!("adjust_boundary {label} panicked: {}", p.is_err());
}
}
}Output: both array transforms print [null, null, 0] for timestamps and for dates; the timestamp literals print Err(DataInvalid ...); the date literals and all three adjust_boundary calls panic.
Expected behavior
yearandmonthreturn the spec value for every input, for example with floor division to days and an exact civil-from-days conversion instead of chrono, in both the array transforms andtransform_literal. Failing that, both paths should return an error, never NULL for a non-null input and never a panic.adjust_boundaryuses checked arithmetic.x < MINandx > MAXare always false, so the projection can return an always-false predicate (or no boundary).
Activity