perf(vector/search): parallelize SegmentFanoutSearcher's per-segment fan-out (#926) - #929
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Runs
SegmentFanoutSearcher::search's per-segment probes concurrently on native targets (spun off from the #683 triage; rebased on #927/#928, whose shared-basis rescore it parallelizes along with each probe).min_similarityre-filter — is extracted into a self-containedprobe_segmenthelper. Masking reads only immutable newer readers and thePreparedQueryis a shared read-only borrow, so one probe per worker thread is safe.into_par_iter().map(probe_segment).collect::<Result<_>>(), gated toreaders.len() > 1(single-segment indexes keep the zero-overhead serial call); wasm32 keeps the serial loop per the crate convention. The indexed collect preserves newest-first segment order, so the merged input to the bug(vector/search): cross-segment scores are not comparable — out-of-range queries clamp to similarity 1.0 per segment #927 distance sort is byte-identical to the serial path — results are bit-for-bit the same by construction.score_basis = "f32-rerank", bug(vector/search): cross-segment scores are not comparable — out-of-range queries clamp to similarity 1.0 per segment #927) skip the rescore inside their own probe task, exactly as in the serial code. Flat/IVF segmented indexes share the fan-out and get the parallelism for free.Measured win
#683 probe fixture (10 segments, 20k vectors, dim 64, 100 queries, release, best of 3):
Recall identical within build-RNG noise. This more than pays back #927's rescore overhead — 25.4ms is ~37% below even the pre-#927 serial baseline (40.1ms). The gap to core count comes from tiered-merge segment-size skew (the largest segment dominates the critical path).
Tests
segment_fanout_parallel_test.rs: repeated identical searches over a multi-segment fixture (stale duplicates + a deletion) return bit-identical results — parallel probing must not leak scheduling nondeterminism into the merged order.segment_score_comparability_test.rs(its brute-force fixture now exercises the concurrent fan-out on native) and the Stage-2 interaction byvector_rerank_engine_test.rs— both green unchanged.Verification
cargo fmt --checkandcargo clippy --all-targets -- -D warningsclean on stable and 1.97.0cargo test -p laurus --lib: 1258 passed;--tests: all 64 binaries okcargo check -p laurus-wasm --target wasm32-unknown-unknown: clean (serial fallback)Closes #926. Spun off from the #683 triage; depends on #927 (merged as #928).