Use bulk scoring more places for HNSW graphs

[benwtrent]

Description

While we have added bulk scoring, which can provide a substantial performance boost, we still don't use it everywhere. I happened to notice its missing from filter search & higher level searches (fixed for 10.4), but there are some others.

Looking at the code, there are various places where I still see RandomVectorScorer.score that might benefit from the bulk scorer API.

• HnswGraphBuilder#diversityCheck (I am looking at this one now)
• NeighborArray#isWorstNonDiverse

Now, there are also places that use VectorScorer.score() that could benefit from bulk scoring

• DiversifyingChildrenVectorScorer#nextParent (bulk score the children?)
• VectorSimilarityScorerSupplier should maybe satisfy the BulkScorer interface and delegate correctly?
• FullPrecisionFloatVectorSimilarityValuesSource satisfies the DoubleValues interface, which...doesn't have any bulk interfaces :(. But might benefit from them.
• Same for VectorSimilarityValuesSource

Places that use VectorUtil directly, that might be harder to refactor but could benefit from bulk scoring:

• KMeans
• BpVectorReorderer

[ML-dev-crypto]

Hi maintainers

PLEASE ASSIGN THUS ISSUE TO ME

I’d like to take this issue and work on expanding bulk scoring usage in the remaining places where we still rely on single-vector scoring.

From reviewing the code and the discussion here, I see several clear candidates where RandomVectorScorer.score() / VectorScorer.score() are still used and could benefit from the bulk scorer API:

Areas I plan to focus on

HNSW graph construction

HnswGraphBuilder#diversityCheck
This looks like a strong candidate for bulk scoring during diversity checks, and I’m currently reviewing this path in detail.

NeighborArray#isWorstNonDiverse
Could potentially batch vector comparisons instead of scoring one-by-one.

Higher-level vector scorers

DiversifyingChildrenVectorScorer#nextParent
Investigate bulk-scoring children vectors rather than individual scoring calls.

VectorSimilarityScorerSupplier
Explore whether it can implement or delegate to BulkScorer when available.

Value sources

FullPrecisionFloatVectorSimilarityValuesSource

VectorSimilarityValuesSource
These currently expose DoubleValues, which lacks bulk interfaces, but there may be opportunities to extend or adapt these paths to support bulk scoring where possible.

Direct VectorUtil usage

KMeans

BpVectorReorderer
These may be harder to refactor, but I’ll evaluate whether limited bulk scoring can still be applied safely.

Goal

The goal would be to:

Reduce remaining single-vector scoring hot paths

Improve consistency of bulk scoring usage across ANN and higher-level search flows

Deliver measurable performance improvements without changing scoring semantics

Thanks!

[benwtrent]

I would focus on one issue at a time @ML-dev-crypto and please, for any AI generated code, be the human in the middle and review and verify before opening any proposed changes.

[ML-dev-crypto]

Understood, thank you for the guidance.
To keep the scope small and focused, I will start by looking into NeighborArray#isWorstNonDiverse only. I'll open a PR for that specific change once I have verified the implementation and tests locally.

[leng25]

Hi, I noticed #15667 has been stale since Feb 21 with no activity.
I have opened a new PR for this #15958 .
Benchmarks are included in the PR description.