fix: unwrap identity Date cast in comparison unwrapping#23727
Open
adriangb wants to merge 3 commits into
Open
Conversation
This was referenced Jul 20, 2026
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #23727 +/- ##
=======================================
Coverage 80.70% 80.70%
=======================================
Files 1089 1089
Lines 368137 368170 +33
Branches 368137 368170 +33
=======================================
+ Hits 297121 297149 +28
- Misses 53311 53313 +2
- Partials 17705 17708 +3 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Add an SLT test to `simplify_expr.slt` for the predicate `cast(d AS date) = DATE '...'` where `d` is a `Date32` column. This is an identity cast that should fold away, leaving a bare-column comparison. This commit records the *current* (buggy) behavior: the logical plan retains a residual `CAST(dates.d AS Date32)` instead of simplifying to `dates.d = Date32(...)`. The residual cast defeats downstream pruning / filter pushdown that expects a bare-column comparison. The next commit fixes this and this test's assertion is updated to the corrected plan, making the fix's effect visible in the diff. Signed-off-by: Adrian Garcia Badaracco <1755071+adriangb@users.noreply.github.com>
`unwrap_cast_in_comparison` refused to fold an identity `CAST(col AS DATE)`
on a `Date32` column, leaving a residual `Cast(col AS Date32)` in the
predicate instead of comparing against the bare column. SQL `cast(col AS
date)` plants a real `Expr::Cast` with no identity elision (unlike the
`arrow_cast` UDF, whose `simplify()` short-circuits identity), so this cast
reached `try_cast_literal_to_type` and was rejected. The residual cast
defeats downstream pruning/pushdown that expects a bare-column comparison.
The root cause is in `is_lossy_temporal_cast`:
(is_date_type(from) && to.is_temporal())
|| (is_date_type(to) && from.is_temporal())
For an identity `Date32 -> Date32` cast this is `true && true`, because
`DataType::is_temporal()` is true for both `Date32` and `Date64`. The cast
is misclassified as a lossy temporal cast, `try_cast_literal_to_type`
returns `None`, and the residual cast is left in place.
Fix: short-circuit an identity cast (`from_type == to_type`) as non-lossy at
the top of `is_lossy_temporal_cast`. An identity cast can never change
comparison semantics.
This is deliberately scoped to *identical* types only, not "any date-to-date".
`Date32` counts days while `Date64` counts milliseconds, but
`try_cast_numeric_literal` uses `mul = 1` for both, so a `Date32 <-> Date64`
cast would convert units wrongly and must stay blocked. A regression test
asserts that.
The SLT characterization test added in the previous commit now fails (its
assertion still shows the residual cast); the next commit updates it to the
corrected plan, so the diff shows exactly what the fix changed.
Signed-off-by: Adrian Garcia Badaracco <1755071+adriangb@users.noreply.github.com>
With the identity short-circuit in `is_lossy_temporal_cast`, the predicate
`cast(d AS date) = DATE '...'` on a `Date32` column now folds the identity
cast away. Update the SLT assertion from the residual
Filter: CAST(dates.d AS Date32) = Date32("2024-01-01")
to the corrected bare-column comparison
Filter: dates.d = Date32("2024-01-01")
The diff of this commit is the observable effect of the fix.
Signed-off-by: Adrian Garcia Badaracco <1755071+adriangb@users.noreply.github.com>
adriangb
force-pushed
the
fix-identity-date-cast-unwrap-upstream
branch
from
July 21, 2026 00:17
8a5e75d to
0ec7b72
Compare
Contributor
Author
|
@kosiew another hopefully small one to look at |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
No existing issue; this PR both reports and fixes the bug. Happy to file a tracking issue if preferred.
Rationale for this change
unwrap_cast_in_comparisonfails to fold an identityCAST(col AS DATE)on aDate32column. Instead of rewriting the predicate to compare against the bare column, it leaves a residualCast(col AS Date32)in place, which defeats downstream optimizations (pruning / filter pushdown) that expect a bare-column comparison.Reproduction (logical plan for
SELECT * FROM t WHERE cast(d AS date) = DATE '2024-01-01', wheredisDate32): thecast(d AS Date32)survives simplification rather than collapsing tod. Notearrow_cast(d, 'Date32')already folds, because thearrow_castUDF'ssimplify()short-circuits an identity cast; the SQLCAST ... AS dateplanner plants a realExpr::Castwith no such elision, so it reachestry_cast_literal_to_typeand is wrongly rejected.The root cause is in
is_lossy_temporal_cast(datafusion/expr-common/src/casts.rs):For an identity
Date32 -> Date32cast this evaluates totrue && true, becauseDataType::is_temporal()is true for bothDate32andDate64. The identity cast is therefore misclassified as a lossy temporal cast,try_cast_literal_to_typereturnsNone, andunwrap_cast_in_comparisonleaves the cast in the plan.What changes are included in this PR?
Short-circuit an identity cast as non-lossy at the top of
is_lossy_temporal_cast:This is deliberately limited to identical types, not "any date-to-date".
Date32counts days whileDate64counts milliseconds, buttry_cast_numeric_literalusesmul = 1for both, so allowing aDate32 <-> Date64unwrap would convert units incorrectly. Thefrom_type == to_typeidentity guard is the exact correct scope, andDate32 <-> Date64remains blocked.Are these changes tested?
Yes. The PR is structured as a test-driven, stacked sequence so the effect of the fix is visible in the diff:
test: characterize identity Date cast in comparison unwrapping— adds an SLT test tosimplify_expr.slt(explain select d from dates where cast(d as date) = DATE '2024-01-01') whose assertion records the current, buggy output: the logical plan keeps the residualFilter: CAST(dates.d AS Date32) = Date32(...). Passes onmain.fix: unwrap identity Date cast in comparison unwrapping— the production change plus twoexpr-commonunit tests:test_try_cast_identity_date_allowed— identityDate32 -> Date32/Date64 -> Date64now fold (try_cast_literal_to_typereturnsSome), andis_lossy_temporal_castreports them non-lossy.test_try_cast_date32_date64_still_blocked—Date32 <-> Date64stays lossy/blocked (guards the units caveat above, which SLT can't easily express).At this commit the SLT test is intentionally red, proving it exercises the bug.
test: update identity Date cast assertion to folded plan— updates the SLT assertion to the correctedFilter: dates.d = Date32(...). The one-line diff is the observable effect of the fix.cargo fmt --checkandcargo clippy -D warningsare clean on the changed crate;datafusion-expr-commonand thesimplify_exprsqllogictest pass.Are there any user-facing changes?
No API changes. Queries with
CAST(date_col AS DATE)predicates onDate32columns simplify to bare-column comparisons, which can enable additional pruning/pushdown. No behavioral change to query results.