fix: unwrap identity Date cast in comparison unwrapping

[adriangb]

Problem

unwrap_cast_in_comparison refuses to fold an identity CAST(col AS DATE) on a Date32 column, leaving a residual Cast(col AS Date32) in the predicate instead of comparing against the bare column. SQL cast(col AS date) plants a real Expr::Cast with no identity elision (unlike the arrow_cast UDF, whose simplify() short-circuits identity), so this cast reaches try_cast_literal_to_type and is rejected. The residual cast defeats downstream pruning/pushdown that expects a bare column comparison.

Root cause

In datafusion/expr-common/src/casts.rs, is_lossy_temporal_cast:

(is_date_type(from_type) && to_type.is_temporal())
    || (is_date_type(to_type) && from_type.is_temporal())

For an identity Date32 -> Date32 cast this is true && true, because arrow's DataType::is_temporal() is true for both Date32 and Date64. The identity cast is therefore misclassified as a lossy temporal cast, try_cast_literal_to_type returns None, and unwrap_cast_in_comparison leaves the cast in place.

Fix

Short-circuit an identity cast as non-lossy at the top of is_lossy_temporal_cast:

if from_type == to_type {
    return false; // an identity cast never changes comparison semantics
}

Caveat: Date32 <-> Date64 stays blocked

This is deliberately scoped to identical types only, not "any date-to-date". Date32 counts days while Date64 counts milliseconds, but try_cast_numeric_literal uses mul = 1 for both, so allowing a Date32 <-> Date64 unwrap would convert units wrongly. The from_type == to_type identity guard is the exact correct scope; a regression test asserts Date32 <-> Date64 still does not unwrap.

Tests

• test_try_cast_identity_date_allowed — identity Date32 -> Date32 / Date64 -> Date64 now fold (try_cast_literal_to_type returns Some), and is_lossy_temporal_cast reports them non-lossy.
• test_try_cast_date32_date64_still_blocked — Date32 <-> Date64 remains lossy/blocked.
• test_unwrap_identity_date_cast (optimizer) — end-to-end: cast(date_col AS DATE) = DATE '...' simplifies to date_col = <lit>.

All datafusion-expr-common and datafusion-optimizer tests pass; cargo fmt + cargo clippy -D warnings clean on the touched crates.

Upstream

apache/datafusion main carries the same is_lossy_temporal_cast, so this bug exists upstream too and this is a candidate to port to apache/datafusion.

🤖 Generated with Claude Code

[adriangb]

Superseded by the upstream PR apache#23727 — this is a general upstream bug, so the canonical fix belongs in apache/datafusion. It will reach pydantic-main when apache merges and the fork rebases; the platform-side workaround already covers production in the meantime. Closing this fork PR in favor of the upstream one.