Repository navigation
Filtered search: opt-in scan-first plan for a small allowed share - #57
Merged
Merged
Conversation
A filtered walk spends its whole visit budget before it learns that too few ids qualify, and the exact scan then runs on top of it. With filterScanBelow(fraction) above 0, GraphIndex, SealedGraph and TieredIndex test the predicate on 256 live slots (FilterPlan) and scan exactly when at most that share passes. Default 0 keeps every result as before. Tests: predicate-call counts (one estimate plus one scan), exactness against brute force, unchanged hits above the threshold, validation, tier segments. Not compiled or run here (no JDK on this machine); CI compiles and runs the tests. Refs #12 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #12, closes #56.
I added an opt-in
filterScanBelow(fraction)toGraphIndex,SealedGraphandTieredIndex(default 0, off). With a filter and a fraction above 0, the search tests the predicate on 256 live slots spread over the index (FilterPlan), and if at most that share passes it scans the allowed ids exactly instead of walking first. Above the threshold nothing changes, and with the option off nothing changes at all, so this does not alter any current result. The tier passes its setting to every segment; the tails are already exact.Why it is off: a query answered by the scan can get a different (better) answer than the walk gave, and the right fraction needs the filter study of #12 re-run with the option. The motivation and the measurements are in #56.
Tests added (same style as the existing ones):
GraphTest.testFilterScanBelowcounts predicate calls (one estimate plus one scan, no walk before it), checks exactness against brute force on the mutable and the sealed graph at shares 0.01 and 0.001, checks that a share of 0.5 walks and returns the same hits with the option on and off, and checks validation and the default.TierTests.testFilterScanBelowchecks the same on a tier with several segments and a tail.docs/DESIGN.mddescribes the option.Nothing ran locally (no JDK on my machine), so CI is the first compile and test run of this change.
🤖 Generated with Claude Code