feat(mem_wal): validate maintained indexes against the writer's rules

[hamersaw]

Problem

is_maintainable_index_type inspects only an index's protobuf type url, which cannot decide whether the MemWAL can maintain that index. Every vector index sub-type carries VectorIndexDetails and so maps to MemIndexKind::Hnsw, but the memtable's HNSW requires a FixedSizeList<Float32> column.

A caller that filters a maintained set on the type url alone can therefore commit an index that every shard-writer open then rejects, leaving the table unwritable. Concretely: an IVF-PQ index over FixedSizeList<Float64> is admitted by is_maintainable_index_type, gets committed into maintained_indexes, and then the first merge_insert fails opening the shard writer with HNSW index requires a FixedSizeList<Float32> column.

The rules that actually decide this already exist in validate_index_configs — they just only run at ShardWriter::open, far from the call that chose the set. is_maintainable_index_type's own doc comment made this reachable by advising callers to "filter on this before committing a maintained set".

Change

• Extract the index-config building loop out of mem_wal_writer into build_index_configs.
• Add validate_maintained_indexes(dataset, index_names), which builds the same configs and runs validate_index_configs against the same shard schema (base + _tombstone) with the same derived PK columns.
• Both mem_wal_writer and the new entry point go through that one path, so a set that validates is a set the writer can open, and the two cannot drift as more index kinds become maintainable.
• Correct the is_maintainable_index_type docs to say it is necessary but not sufficient, and point at the new function.

The check is all-or-nothing by design: it reports the first index it cannot maintain rather than returning a usable subset. A caller inferring a set surfaces that error rather than silently dropping an index the caller believes is maintained and the writer never touches.

Motivation

lancedb is adding an inferred default for maintained_indexes (lancedb/lancedb#3748). Inference needs to ask "can the MemWAL maintain this index?" and there is currently no API that answers it correctly. Keeping the rule here, next to the code that enforces it, means every future widening of what the MemWAL can maintain reaches callers for free instead of being re-derived downstream.

Testing

Four new tests in mem_wal::api:

• test_validate_maintained_indexes_rejects_non_f32_vector_column — asserts both halves of the bug: is_maintainable_index_type admits an IVF index over FixedSizeList<Float64>, and validate_maintained_indexes rejects it.
• test_validate_maintained_indexes_accepts_f32_vector_column
• test_validate_maintained_indexes_accepts_btree — guards the shard-schema plumbing (field ids resolve against base + _tombstone).
• test_validate_maintained_indexes_rejects_unknown_name

Full dataset::mem_wal suite: 582 passed, 1 ignored, 0 failed.

🤖 Generated with Claude Code

[lance-gatekeeper]

❌ Gate recommendation: request changes.

The maintained-index check needs to be enforced at the transaction that persists the set, not left as an optional preflight. A viable revision should run the shared validation from InitializeMemWalBuilder::execute, ensure a commit rebase cannot invalidate the selected index/schema, and keep writer-open revalidation for later changes.

Please mark this PR with the breaking-change label.

[lance-gatekeeper]

❌ Gate recommendation: request changes.

The mandatory validation fixes the ordinary initialization path, but validity is still not atomic with commit rebasing. A viable revision should either revalidate the maintained set against the rebased manifest before _mem_wal lands, or make changes to every referenced index/schema dependency a retryable conflict; writer-open validation should remain as the later safety net.

[lance-gatekeeper]

This validates only the current Dataset snapshot. CommitBuilder can subsequently rebase this CreateIndex transaction over a concurrent drop_index: conflict resolution checks concurrently created names but ignores removed_indices, and no validation reruns against the rebased manifest. The result can still persist _mem_wal with a missing maintained index, after which every writer open fails. Concurrent replacement or projection can invalidate the same dependency.

Revalidate _mem_wal details against the latest manifest during finish_create_index, or make removal/replacement/projection of a referenced index or field conflict and retry from a refreshed snapshot.

Reproducer run on this head

#[tokio::test]
async fn gate_reproducer_initialization_rebases_over_index_drop() {
    let tmp = tempfile::tempdir().unwrap();
    let uri = format!("{}/base", tmp.path().to_str().unwrap());
    let schema = id_v_schema();
    let reader = RecordBatchIterator::new(
        [Ok(id_v_batch(&schema, &[1, 2, 3]))], schema.clone());
    let mut stale = Dataset::write(reader, &uri, Some(WriteParams::default()))
        .await.unwrap();
    stale.create_index(
        &["id"], IndexType::BTree, Some("id_idx".to_string()),
        &ScalarIndexParams::default(), true).await.unwrap();

    let maintained_indexes = vec!["id_idx".to_string()];
    validate_maintained_indexes(&stale, &maintained_indexes).await.unwrap();
    let details = MemWalIndexDetails {
        num_shards: 1,
        sharding_specs: vec![unsharded_sharding_spec()],
        maintained_indexes,
        ..Default::default()
    };
    let index_meta = new_mem_wal_index_meta(stale.manifest.version, details).unwrap();
    let transaction = Transaction::new(
        stale.manifest.version,
        Operation::CreateIndex { new_indices: vec![index_meta], removed_indices: vec![] },
        None);

    let mut concurrent = stale.clone();
    concurrent.drop_index("id_idx").await.unwrap();

    let rebased = CommitBuilder::new(Arc::new(stale))
        .execute(transaction).await
        .expect("the stale MemWAL transaction currently rebases over the drop");
    assert!(rebased.load_indices_by_name("id_idx").await.unwrap().is_empty());
    assert!(rebased.mem_wal_index_details().await.unwrap().is_some());

    let error = match rebased
        .mem_wal_writer(Uuid::new_v4(), ShardWriterConfig::default()).await
    {
        Ok(_) => panic!("the rebased set unexpectedly opened a writer"),
        Err(error) => error,
    };
    assert!(error.to_string().contains("id_idx")
        && error.to_string().contains("not found"));
}

cargo test -p lance dataset::mem_wal::api::tests::gate_reproducer_initialization_rebases_over_index_drop -- --nocapture passed, reproducing the accepted stale commit and deterministic writer-open failure. A fixed regression should instead assert that the stale commit is rejected and no MemWAL index is persisted.

[hamersaw]

WAL does not currently guard against changes made the to the schema after it has been enabled. So things like index changes (dropping + changing types) or column updates (same) are not tracked. We will address this work as a follow up and have added a "limitation" comment in the code regarding it.

[lance-gatekeeper]

❌ Gate recommendation: request changes.

The new documentation accurately describes the symptom, but the initialization race is still an atomicity failure rather than a post-initialization limitation: the invalidating transaction can commit first and _mem_wal can land afterward already unusable.

A viable revision should revalidate the writer-open invariant on the rebased manifest, or make referenced index/schema changes retryable conflicts. The limitation can remain for genuinely later mutations.

[lance-gatekeeper]

Documenting this race does not make the initialization result safe. In the reproduced ordering, drop_index commits at N+1 while execute is in progress; the stale _mem_wal transaction then rebases and commits at N+2. The dependency change therefore precedes initialization in committed history, yet execute returns success with metadata that no writer can open.

Keep this limitation for genuinely later mutations, but make initialization revalidate maintained_indexes against the rebased manifest in finish_create_index, or conflict and retry whenever a referenced index or schema field changes.

Reproducer run on this head

#[tokio::test]
async fn gate_reproducer_initialization_rebases_over_index_drop() {
    let tmp = tempfile::tempdir().unwrap();
    let uri = format!("{}/base", tmp.path().to_str().unwrap());
    let schema = id_v_schema();
    let reader = RecordBatchIterator::new(
        [Ok(id_v_batch(&schema, &[1, 2, 3]))], schema.clone());
    let mut stale = Dataset::write(reader, &uri, Some(WriteParams::default()))
        .await.unwrap();
    stale.create_index(
        &["id"], IndexType::BTree, Some("id_idx".to_string()),
        &ScalarIndexParams::default(), true).await.unwrap();

    let maintained_indexes = vec!["id_idx".to_string()];
    validate_maintained_indexes(&stale, &maintained_indexes).await.unwrap();
    let details = MemWalIndexDetails {
        num_shards: 1,
        sharding_specs: vec![unsharded_sharding_spec()],
        maintained_indexes,
        ..Default::default()
    };
    let index_meta = new_mem_wal_index_meta(stale.manifest.version, details).unwrap();
    let transaction = Transaction::new(
        stale.manifest.version,
        Operation::CreateIndex { new_indices: vec![index_meta], removed_indices: vec![] },
        None);

    let mut concurrent = stale.clone();
    concurrent.drop_index("id_idx").await.unwrap();

    let rebased = CommitBuilder::new(Arc::new(stale))
        .execute(transaction).await
        .expect("the stale MemWAL transaction currently rebases over the drop");
    assert!(rebased.load_indices_by_name("id_idx").await.unwrap().is_empty());
    assert!(rebased.mem_wal_index_details().await.unwrap().is_some());

    let error = match rebased
        .mem_wal_writer(Uuid::new_v4(), ShardWriterConfig::default()).await
    {
        Ok(_) => panic!("the rebased set unexpectedly opened a writer"),
        Err(error) => error,
    };
    assert!(error.to_string().contains("id_idx")
        && error.to_string().contains("not found"));
}

cargo test -p lance dataset::mem_wal::api::tests::gate_reproducer_initialization_rebases_over_index_drop -- --nocapture passed on 05a3463f573623a68a9b419080d2dc837565c61c, reproducing the accepted stale commit and deterministic writer-open failure. A fixed regression should instead assert that the stale commit is rejected and no MemWAL index is persisted.

[hamersaw]

WAL does not currently guard against changes made the to the schema after it has been enabled. So things like index changes (dropping + changing types) or column updates (same) are not tracked. We will address this work as a follow up and have added a "limitation" comment in the code regarding it.

[Xuanwo]

Thank you!