Skip to content

fix(expression): handle missing lower bound in StrictMetricsEvaluator… - #1153

Open
deepaknettem wants to merge 1 commit into
apache:mainfrom
deepaknettem:fix/strict-metrics-not-in-missing-bound
Open

deepaknettem wants to merge 1 commit into
apache:mainfrom
deepaknettem:fix/strict-metrics-not-in-missing-bound

Conversation

@deepaknettem

Copy link
Copy Markdown

Problem:
In StrictMetricsVisitor::NotIn, if lower_bounds is not set for a column and upper_bounds is present, literals_view is always empty and NotIn always returns kRowsMustMatch (true).

For example, let's say there's a file with a single row id = 50, upper_bound = 50, and no lower_bound is set.
For a statement like DELETE FROM t WHERE id NOT IN (40, 50) (which should keep row 50), StrictMetricsEvaluator returns true instead of false, so ManifestFilterManager::ShouldDelete is incorrectly marking the file for deletion and drops row 50.

Fix:
Treat a missing lower_bound or upper_bound as unbounded on that side in StrictMetricsVisitor::NotIn. This seems to match Java's StrictEvalVisitor.notIn implementation.

Tests:
Added tests in strict_metrics_evaluator_test.cc for missing lower_bound, missing upper_bound, and both bounds missing. Verified the missing lower_bound test fails before the fix and passes after the fix. Also ran expression_test and pre-commit.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant