OpenSearchFilterRule.resolveViableBackends threw UnsupportedOperationException
for a deterministic predicate with no field references, asserting that
ReduceExpressionsRule must have already eliminated it. That assumption breaks
for a predicate embedding a backend-only function the plan-time RexExecutor
cannot evaluate — e.g. a PPL conversion UDF over a literal:
... | eval s = '10/18/2003 20:07:13' | convert mktime(s) | where s > 1000000000
mktime(<literal>) is constant and deterministic, but the constant folder skips
it (no plan-time implementation exists — only DataFusion evaluates it
natively), so the constant predicate survives to the filter rule and trips the
assertion with a 500.
Such a predicate carries no field, so backend selection is unconstrained by
field-type support. Hand it to any backend viable for the child — identical to
the existing non-deterministic branch (RAND() > 0) — and let the backend
evaluate the constant at runtime. The change is strictly additive: the branch
is only reachable when the previous code already threw, so no previously
succeeding filter is affected.
Adds two regression tests to ConversionFunctionsIT covering a constant
predicate that passes and one that filters every row out.
Signed-off-by: Kai Huang <ahkcs@amazon.com>
Description
OpenSearchFilterRule.resolveViableBackendsrejected any deterministic filter predicate with no field references, throwing:The rule assumed
ReduceExpressionsRulealways folds such predicates away. That assumption breaks when the predicate embeds a function the plan-timeRexExecutorcannot evaluate — for example a PPL conversion UDF applied to a literal:mktime(<literal>)is constant and deterministic, but the constant folder skips it because no plan-time implementation exists (only the DataFusion backend evaluates the conversion UDF natively). The unfolded constant predicate then reaches the filter rule and trips the assertion, returning a 500.Fix
A predicate with no field references carries no field, so backend selection is unconstrained by field-type support. Hand it to any backend viable for the child — exactly as the existing non-deterministic branch (
RAND() > 0) already does — and let the backend evaluate the constant at runtime.The change is strictly additive: the branch is only reachable when the previous code already threw
UnsupportedOperationException, so no previously succeeding filter is affected.Verification
Reproduced and verified against a
:runcluster with the analytics-engine plugin set, routing every test index through the analytics engine.PPL
convertcommand, run through the analytics-engine path:CalciteConvertCommandIT(opensearch-project/sql, analytics route)ConversionFunctionsIT(QA,:sandbox:qa:analytics-engine-rest)The single prior
CalciteConvertCommandITfailure (testConvertTimeformatWithWhere) now passes. Two regression tests are added toConversionFunctionsIT— one where the constant predicate passes a row through, one where it filters every row out.Check List
--signoff.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.