Skip to content

Delete / tombstone support via a sidecar skip-bitmap #88

@Fieldnote-Echo

Description

@Fieldnote-Echo

Summary

All four index types are append-only. Production corpora churn — documents are deleted, embeddings are recomputed as models update. A minimal tombstone facility would cover the common case without a full rebuild.

Ask

  • delete(doc_id: u64) sets a bit in a sidecar tombstone bitmap.
  • The scan path masks tombstoned doc IDs before they enter TopK.
  • The sidecar is separate from the existing on-disk storage, so no .tvr* format change.

⚠️ Direction note (maintainer call)

ROADMAP.md lists "persistence and transaction machinery" as an explicit non-goal ("ordvec will not grow into a vector database"). A read-time skip-bitmap is arguably within scope — it's not compaction, MVCC, or transactions — but the framing needs a maintainer decision:

  • Do tombstones persist alongside the index, or stay caller-managed (passed into search)?
  • Explicitly defer compaction / space reclamation (that's the database territory the ROADMAP rules out).

If the answer is "caller-managed skip set passed to search," that may be a smaller, more on-philosophy first step than an owned mutable tombstone bitmap.


Filed from an external technical review (May 2026); verified append-only against v0.2.0.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions