Skip to content

Year and month transforms return NULL or panic beyond chrono's date range; adjust_boundary overflows at type extremes #3637

Open

Description

@sesteves

Apache Iceberg Rust version

main (5994664, 2026-11-06)

Describe the bug

1. year and month of dates and timestamps beyond chrono's range.

The spec defines year and month for every date and timestamp value. Iceberg Java's DateTimeUtil computes them through java.time, whose range covers every int day and long microsecond. iceberg-rust goes through chrono, whose dates end at about ±262,000 years, so:

  • The array transforms (Year::transform, Month::transform, via Arrow's date_part) return NULL for non-null inputs outside that range. A writer that uses them files such rows under a null partition. A predicate on the source column then projects to a non-null partition value and prunes the file that holds the row.
  • transform_literal returns DataInvalid (Fail to convert timestamp to date in year transform) for such timestamps, and panics for such dates (Date32Type::to_naive_date_opt(*v).unwrap(), transform/temporal.rs:80 and :173).
Input Expected year Expected month Array transform transform_literal
timestamp i64::MAX µs (294247-01-10) 292277 3507324 null DataInvalid
timestamp i64::MIN µs (-290308-12-22) -292278 -3507325 null not tested
date i32::MAX (5881580-07-11) 5879610 70555326 null panic
date i32::MIN (-5877641-06-24) -5879611 -70555327 null not tested

The expected values use floor division to days and an exact civil-from-days conversion. i64::MAX microseconds is a common "end of time" sentinel.

2. Transform::adjust_boundary overflows at type extremes.

adjust_boundary (spec/transform.rs:775 onwards) computes v - 1 for < and v + 1 for > unchecked. With overflow checks (debug builds, tests) x < i32::MIN (int or date) and x > i64::MAX panic (spec/transform.rs:778). Without them the boundary wraps to one that matches every value, which only makes the projection over-inclusive.

For context: the related day floor, nanosecond bucket, decimal truncate literal, truncate overflow and nanosecond hour issues are already fixed on main (#3323, #3243, #3573, #3026), and LargeBinary truncate has an open PR (#3120).

To Reproduce

Append to crates/iceberg/src/spec/transform.rs and run cargo test -p iceberg --lib audit_out_of_range -- --nocapture:

#[cfg(test)]
mod audit_out_of_range {
    use std::panic::{AssertUnwindSafe, catch_unwind};
    use std::sync::Arc;

    use arrow_array::{ArrayRef, Date32Array, TimestampMicrosecondArray};

    use super::*;

    #[test]
    fn audit_out_of_range() {
        for t in [Transform::Year, Transform::Month] {
            let f = create_transform_function(&t).unwrap();
            let ts: ArrayRef = Arc::new(TimestampMicrosecondArray::from(vec![i64::MAX, i64::MIN, 0]));
            println!("{t:?} timestamp array [MAX, MIN, 0]: {:?}", f.transform(ts));
            let d: ArrayRef = Arc::new(Date32Array::from(vec![i32::MAX, i32::MIN, 0]));
            println!("{t:?} date array [MAX, MIN, 0]: {:?}", f.transform(d));
            println!(
                "{t:?} timestamp literal MAX: {:?}",
                f.transform_literal(&Datum::timestamp_micros(i64::MAX))
            );
            let p = catch_unwind(AssertUnwindSafe(|| f.transform_literal(&Datum::date(i32::MAX))));
            println!("{t:?} date literal MAX panicked: {}", p.is_err());
        }
        for (op, d) in [
            (PredicateOperator::LessThan, Datum::int(i32::MIN)),
            (PredicateOperator::GreaterThan, Datum::long(i64::MAX)),
            (PredicateOperator::LessThan, Datum::date(i32::MIN)),
        ] {
            let label = format!("{op:?} {d:?}");
            let p = catch_unwind(AssertUnwindSafe(|| Transform::adjust_boundary(&op, &d)));
            println!("adjust_boundary {label} panicked: {}", p.is_err());
        }
    }
}

Output: both array transforms print [null, null, 0] for timestamps and for dates; the timestamp literals print Err(DataInvalid ...); the date literals and all three adjust_boundary calls panic.

Expected behavior

  1. year and month return the spec value for every input, for example with floor division to days and an exact civil-from-days conversion instead of chrono, in both the array transforms and transform_literal. Failing that, both paths should return an error, never NULL for a non-null input and never a panic.
  2. adjust_boundary uses checked arithmetic. x < MIN and x > MAX are always false, so the projection can return an always-false predicate (or no boundary).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions