perf(fts): short-circuit conjunction confirmations - #8354
Open
BubbleCal wants to merge 1 commit into
Open
Conversation
Contributor
There was a problem hiding this comment.
✅ Gate recommendation: approve.
The change addresses eager MUST-clause confirmation at the right layer: a stable, cost-ordered index controls only two-phase confirmation, while scoring and bounds remain in query order. The deterministic canary demonstrates the claimed 200-to-120 reduction, and multi-fragment phrase plus nested-Boolean coverage preserves exact results.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Xuanwo
approved these changes
Aug 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the performance issue?
RequiredConjunctionScorerconfirmed every two-phase child for each aligned candidate, even after one required child had already rejected the document. In compound FTS queries, expensive phrase confirmations could therefore run for candidates that a cheaper required clause would discard.How does this PR improve performance?
match_costhint, independently of query-order scoring and score-bound arithmetic.Benchmark
matches()calls for 100 aligned candidates with two required children (lower is better)Methodology: deterministic counting-scorer canary with the same 100 aligned candidates and top-k collection path. The query-order first child has cost
10, accepts all candidates, and represents the expensive confirmation; the second child has cost1, accepts 20% of candidates, and represents the cheap filter. The baseline (origin/mainat4d63a12ef) eagerly confirms both children for all 100 candidates. This PR confirms the cheap child 100 times and the expensive child only for the 20 survivors. This measures confirmation work, not wall-clock latency.Validation
cargo test -p lance-index --lib scalar::inverted::compound::tests(12 passed)cargo test -p lance --lib dataset::tests::dataset_index::test_compound_phrase_confirmation_short_circuit_is_exact -- --exact(1 passed)cargo test -p lance --lib dataset::tests::dataset_index::test_same_column_compound_scorer_is_exact_and_bounded -- --exact(1 passed)cargo fmt --all -- --checkcargo clippy --all --tests --benches -- -D warningsgit diff origin/main...HEAD --checkNo public API or index-format changes.
Closes OSS-1707.