fix(index): clear the legacy embeddings table on reset and deindex#543
fix(index): clear the legacy embeddings table on reset and deindex#543michiot05 wants to merge 1 commit into
Conversation
index_db.reset() promises to drop every derived table — "leaving stale rows here means semantic search can return orphaned hits after a reindex" — but the delete list skipped the legacy embeddings table, the one search_embeddings actually scans and _rebuild_embeddings repopulates on every vouch index. deindex() had the same gap: it cleared embedding_index but left the legacy row behind. the consequence is exactly the orphaned-hits case the docstring warns about, made permanent: delete an artifact, reindex, and its vector still comes back as a semantic hit, while fsck's orphan_embedding warning can never be cleared because the natural remedy — reindexing — never touched the table. add the legacy table to reset()'s delete list and give deindex() the matching per-row delete, mirroring how both already handle embedding_index.
|
Warning Review limit reached
Next review available in: 38 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
what changed
index_db.reset()now deletes from the legacyembeddingsvector table along with every other derived table, andindex_db.deindex()removes the deleted artifact's row from it — mirroring how both functions already handleembedding_index.why
reset()'s own docstring states the contract: it "clears every derived table — FTS indexes, embedding vectors, …" because "leaving stale rows here means semantic search can return orphaned hits after a reindex." the delete list coveredembedding_indexbut skipped the legacyembeddingstable — which is the tablesearch_embeddingsactually scans at query time and_rebuild_embeddingsrepopulates viaINSERT OR REPLACEon everyvouch index.two user-visible consequences:
vouch index— the "drop and rebuild… idempotent" pass — and the deleted artifact's vector survives, so semantic search keeps returning it (snippet degraded to the bare id). the same applies to a single delete:deindex()'s docstring promises the embedding row is removed "for any kind", but it only clearedembedding_index._check_orphan_embeddingsdeliberately scans both tables ("either one drifting silently breaks semantic retrieval") and flags the stale row asorphan_embedding— but the natural remedy it points the operator at, reindexing, never touched the table, so the warning was permanent.what would break for an existing
.vouch/nothing —
state.dbis a derived cache and both changed functions are already destructive-by-contract on derived rows only. kbs carrying stale legacy vectors get them swept away on their nextvouch indexor gated delete, which is the documented behavior.test plan
three regression tests, all failing on the previous code:
tests/test_index.py::test_reset_clears_legacy_embeddings_table—reset()leavesembeddingsemptytests/test_index.py::test_deindex_removes_legacy_embedding_row—deindex()removes the artifact's legacy rowtests/test_health.py::test_fsck_orphan_embedding_cleared_by_reindex— the operator-visible symptom: anorphan_embeddingfinding is actually cleared byhealth.rebuild_index, instead of surviving it forevervalidation run locally: