Skip to content

fix: preserve integer indices in empty SparseEmbedding.from_dict results - #762

Closed
RAMZI0TO99 wants to merge 1 commit into
qdrant:mainfrom
RAMZI0TO99:fix/empty-sparse-indices
Closed

RAMZI0TO99 wants to merge 1 commit into
qdrant:mainfrom
RAMZI0TO99:fix/empty-sparse-indices

Conversation

@RAMZI0TO99

Copy link
Copy Markdown
Contributor

SparseEmbedding.from_dict({}) currently returns float64 indices, despite the
class declaring integer indices. Using the empty embedding to index a NumPy
array raises IndexError. Concatenating it with nonempty integer indices also
promotes the combined indices to float64.

Specify dtype=np.int64 in the empty-input branch. Add regression tests for
empty indexing, nonempty pairing and dictionary roundtrips, concatenation in
both orders, and real BM25 document embedding after all tokens are filtered.

Reproduction:

import numpy as np
from fastembed.sparse.sparse_embedding_base import SparseEmbedding

embedding = SparseEmbedding.from_dict({})
print("Index dtype:", embedding.indices.dtype)
dense = np.zeros(5)
dense[embedding.indices] = embedding.values
print("Result:", dense)

Before: float64 indices, followed by
IndexError: arrays used as indices must be of integer (or boolean) type.
After: int64 indices and Result: [0. 0. 0. 0. 0.].

Environment: Windows 11 build 10.0.26200, Python 3.13.5, NumPy 2.3.5,
FastEmbed 0.8.1 installed from upstream commit
7d36728.

Validation:

  • New regression cases before the fix: 6 failed, 1 passed.
  • After the fix: 16 passed across the new module, common tests, and two existing
    BM25 argument-validation tests on Python 3.10.11 / NumPy 2.2.6.
  • Four direct regression cases and the reproduction pass on Python 3.13.5 /
    NumPy 2.3.5. A local sandbox temporary-directory restriction prevented the
    three BM25 cases from running in that environment; all three pass on 3.10.
  • Ruff 0.3.4 lint/format and git diff --cached --check pass.
  • The full model-download suite and CI matrix were not run locally.

The BM25 tests use local temporary paths and inline stopwords and require no
downloads. Related historical PR #285 added the empty-input branch but did
not preserve its index dtype: #285

All Submissions

  • Followed the guidelines in the Contributing document.
  • Checked for other open pull requests for the same change. No duplicate
    found in ten all-state GitHub searches and the newest ten open PR diffs;
    API rate limits prevented inspecting the remaining 21 open PR diffs.

@RAMZI0TO99
RAMZI0TO99 requested a review from joein as a code owner October 3, 2026 13:23
@coderabbitai

coderabbitai Bot commented Oct 3, 2026

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 8c00e1cc-ee03-4c5b-a853-1d025be8f2b1
📥 Commits

Reviewing files that changed from the base of the PR and between 7d36728 and 67dce4e.

📒 Files selected for processing (2)
  • fastembed/sparse/sparse_embedding_base.py
  • tests/test_sparse_embedding_base.py

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.


📝 Walkthrough

Walkthrough

SparseEmbedding.from_dict now creates empty indices arrays with np.int64 dtype. New tests cover empty and nonempty dictionary conversions, concatenation with empty embeddings, and BM25 outputs for empty, stopword-only, and punctuation-only documents.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 67dce

Empty sparse embeddings now keep integer indices, so they can index NumPy arrays and concatenate without promoting indices to float. The change is small and well covered by the added tests, so merge risk is minimal.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 2 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main change: preserving integer indices in empty SparseEmbedding.from_dict results.
Description check ✅ Passed The description explains the bug, the fix, the regression tests, and the reported validation. It is directly related to the changeset.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant