perf(vector/search): two-stage rerank exists only for HNSW; pipeline is hard-coded

[mosuka]

Round-3 perf push sub-issue (tracked under umbrella #536).

[L] Two-stage rerank exists only for HNSW; pipeline is hard-coded

• Where: laurus/src/vector/index/hnsw/searcher.rs:493-522 implements Stage-2 inline.
  laurus/src/vector/index/flat/searcher.rs:31-37 and
  laurus/src/vector/index/ivf/searcher.rs:121-127 return NotImplemented when
  rerank_factor is set. No shared ReRankScorer trait.

• Current behavior: Each searcher must reimplement rerank. Flat and IVF can't use the
  LRS1 sidecar even though the storage pool exists. 3+ stage chains (PQ → SQ → f32) are
  not expressible.

• Why it might be a bottleneck / risk: Composable rerank is essential for FastScan-style
  3-stage pipelines (PQ initial → SQ verify → f32 final). The current code is locked to a
  single int8 → f32 path.

• Reference precedent: Qdrant ReScorer trait pluggable per index; LanceDB IVF_PQ
  reranks against original f32 stored in the rowgroup; FAISS IndexRefine wraps any base
  index; Lucene 9.10 ScalarQuantizedVectorScorer with score() separate from rescore().

• Data structure (proposed):

  trait RerankScorer: Send + Sync {
      fn prepare(&self, query: &[f32]) -> Self::Prepared;
      fn rescore(&self, prepared: &Self::Prepared, doc_id: u64) -> Option<f32>;
  }
  struct RerankPipeline {
      stages: Vec<Box<dyn RerankScorer>>,
      factors: Vec<usize>,           // top_k * f[0], then top_k * f[1], ...
  }
  // Each stage narrows the candidate set by factor[i]. Composable per index type.
  // Candidate buffer is SoA: (Vec<u64>, Vec<f32>) — not a tuple Vec.
  

• Suggested direction: Generic RerankScorer trait + RerankPipeline. Implement
  Pq8Scorer, Pq4FastScanScorer, Sq8Scorer, F32Scorer. HNSW/Flat/IVF emit a
  candidate set and let the pipeline run.

• Risk / scope: Large architectural. Removes duplication, unlocks 3+ stage rerank,
  normalises behaviour across index types.

---

ID: VS-07 — see ~/.claude/tasks/laurus/20260523_perf_round3_audit/task_list.md for the full Round-3 issue list.

---

Task list

This is now the umbrella for a 3-PR campaign (see the investigation/plan comment, 2026-08-02):

• refactor(vector/search): RerankStage/RerankPipeline abstraction, port HNSW Stage-2 (#650 PR-1) #931 — PR-1: RerankStage/RerankPipeline abstraction, HNSW ported (zero behavior change) — implementation report: refactor(vector/search): RerankStage/RerankPipeline abstraction, port HNSW Stage-2 (#650 PR-1) #931 (comment)
• feat(vector/search): Stage-2 rerank for Flat/IVF via the shared pipeline (#650 PR-2) #932 — PR-2: Flat/IVF sidecar emit + load + consume (closes the NotImplemented gap) — implementation report: feat(vector/search): Stage-2 rerank for Flat/IVF via the shared pipeline (#650 PR-2) #932 (comment)
• perf(vector/search): PQ rerank pipeline not exposed: PQ → SQ → f32 not possible #673 — PR-3: SQ sidecar for PQ segments + the PQ → SQ → f32 three-stage chain

[mosuka]

Investigation + campaign plan (2026-08-02)

Investigation findings

• The HNSW Stage-2 implementation is a single inline match arm in hnsw/searcher.rs ((Some(factor), Some(pool))), since extended by bug(vector/search): cross-segment scores are not comparable — out-of-range queries clamp to similarity 1.0 per segment #927 with the score_basis metadata stamp the multi-segment fan-out relies on.
• Flat / IVF are missing all three layers: writers never emit the LRS1 sidecar, readers never load it, searchers return NotImplemented — even though the schema-level rerank_storage field round-trips everywhere since bug(server): proto HnswOption lacks rerank_storage/quantizer — gRPC-created HNSW fields can't emit the Stage-2 sidecar or use PQ #793 and the rerank_sidecar module (header + write_sidecar/read_sidecar) is already index-type-agnostic. The only HNSW-specific part is the call sites.
• The perf(vector/search): PQ rerank pipeline not exposed: PQ → SQ → f32 not possible #673 three-stage chain (PQ → SQ → f32) additionally requires an int8 representation for PQ segments, which does not exist on disk — a new sidecar kind, i.e. format work independent of the trait refactor.

Campaign structure (this issue becomes the umbrella; #673 becomes PR-3)

1. PR-1 [M] — RerankStage / RerankPipeline abstraction, HNSW ported, zero behavior change. New vector/search/rerank.rs with an object-safe stage trait (prepare once per query / rescore a candidate set, SoA candidate buffer) and a pipeline that narrows by per-stage factors. The inline HNSW arm becomes the f32-sidecar stage; the score_basis stamp moves into the pipeline so the fan-out contract (bug(vector/search): cross-segment scores are not comparable — out-of-range queries clamp to similarity 1.0 per segment #927) is unchanged. Gate: existing rerank tests bit-identical, rerank bench flat.
2. PR-2 [M] — Flat / IVF emit + load + consume. Writers call the shared write_sidecar off config.rerank_storage; readers load leniently-if-absent like HNSW; searchers replace NotImplemented with pipeline consumption. E2e mirrors of vector_rerank_engine_test for both types. Segmented paths come free through the shared fan-out.
3. PR-3 [M] = perf(vector/search): PQ rerank pipeline not exposed: PQ → SQ → f32 not possible #673 — SQ sidecar for PQ segments + the 3-stage chain. RerankStorageKind::Int8 (serde/proto/bindings exposure ladder like perf(vector/index): train-once / reuse PQ codebook across segments instead of per-segment training #631 PR-2/PR-3), then [Pq → Sq → F32] composed through the pipeline, with two-layer recall gates (kernel + e2e, per the perf(vector): Graph Search large-win roadmap — Scalar / Product Quantization + two-stage rerank #481 lessons).

Child issues for PR-1/PR-2 will be filed and embedded in this issue's task list; #673 is annotated as PR-3 rather than duplicated.

Risks

• PR-1's zero-behavior-change claim interacts with bug(vector/search): cross-segment scores are not comparable — out-of-range queries clamp to similarity 1.0 per segment #927's shared-basis rescore and distance sort — pinned by a rerank-on/off × single/multi-segment test matrix before the port.
• PR-3's format addition fans out to bindings/server/docs; it follows the established exposure ladder.