docs(python): show how to specify row ids in update transactions

[wjones127]

Aims to make it clear how to specify updates in Lance when using the low-level transaction API.

_{Stack created with GitHub Stacks CLI • Give Feedback 💬}

[lance-gatekeeper]

⚠️ Gate recommendation: approve with a non-blocking risk.

The documented transaction mechanism matches the implementation and the mixed rewrite/insert example executes with the shown row IDs and lineage. Reconcile the remaining public stable-ID option descriptions with this lifetime guarantee so users see one unambiguous update contract.

[lance-gatekeeper]

This lifetime guarantee matches the implementation, but it leaves the public API contract contradictory: python/python/lance/dataset.py:7592 and python/python/lance/fragment.py:1189 still tell users stable row IDs are “not [stable] after updates.” Users entering through write_dataset or write_fragments cannot tell whether this manual update is supported. Please reconcile those descriptions with this contract (or clearly scope any update path that remains unstable). On this head, rg -n 'not after updates' python/python/lance/{dataset,fragment}.py returns both stale statements.

[lance-gatekeeper]

❌ Gate recommendation: request changes.

The documentation direction is valid, but the live revision now also reintroduces stale RowIdSequence implementation changes that regress performance and weaken the global-identity safety contract already present on the current base.

Please restack this PR onto the current base, retain its linear iterator and complete uniqueness/allocator warning, and carry forward the operation-specific documentation additions.

[lance-gatekeeper]

Repeated absolute slices here make iteration over gapped encodings quadratic. RowIdSeqSlice::iter applies the offset with segment.iter().skip(...), so each refill rescans a larger prefix. This regresses the current base's one-pass iterator. Retain the base's linear materialization, or first add a stateful forward cursor in lance-table.

Reproducer run on this head

import time
from lance.fragment import RowIdSequence

for size in (320_000, 640_000, 1_280_000):
    sequence = RowIdSequence(range(0, size * 2, 2))
    started = time.perf_counter()
    assert sum(1 for _ in sequence) == size
    print(size, time.perf_counter() - started)

Observed 0.114550s, 0.437517s, and 1.710564s: doubling the input takes about 4× as long instead of the roughly 2× expected from linear iteration.

[lance-gatekeeper]

“Duplicates are rejected” is false at dataset scope. The constructor checks only this sequence; commit accepts an already-live ID and creates duplicate logical identities. Retain the current base's warning that commit does not validate dataset-wide uniqueness: supplied IDs must be existing IDs relocated by the same transaction, their old occurrences must be removed, and arbitrary IDs must not be invented because a complete sequence does not advance the allocator.

Reproducer run on this head

import tempfile
import lance
import pyarrow as pa
from lance.fragment import RowIdSequence, write_fragments

with tempfile.TemporaryDirectory(dir="/home/agent/tmp") as tmp:
    schema = pa.schema([("id", pa.int64())])
    ds = lance.write_dataset(
        pa.table({"id": [1, 2]}, schema=schema),
        tmp,
        enable_stable_row_ids=True,
    )
    (frag,) = write_fragments(pa.table({"id": [99]}, schema=schema), tmp, schema=schema)
    frag.row_id_meta = RowIdSequence([0]).to_inline_metadata()
    ds = lance.LanceDataset.commit(
        tmp,
        lance.LanceOperation.Update(new_fragments=[frag]),
        read_version=ds.version,
    )
    assert ds.to_table(with_row_id=True)["_rowid"].to_pylist() == [0, 1, 0]

The documented contract implies rejection; the commit succeeds and returns two live rows with ID 0.

[lance-gatekeeper]

⚠️ Gate recommendation: approve with a non-blocking risk.

The manual transaction guidance now matches the implementation and retains the allocator and dataset-wide uniqueness warnings. One non-blocking documentation inconsistency remains: write_dataset and write_fragments still describe stable row IDs as unstable after updates. Please align those parameter docstrings with the supported update contract so users see one unambiguous guarantee.

[westonpace]

Do we (could we? should we?) reject row ids that are beyond the high water mark of the read version?

[westonpace]

Suggested change

	order, and the commit mints ids for the remaining rows.
	order, and the commit generates new ids for the remaining rows.

Just avoiding the phrase commit mints.

[westonpace]

Suggested change

	Do not mint ids for the new rows yourself. Row ids are handed out from a counter
	Do not generate ids for the new rows yourself. Row ids are handed out from a counter

[westonpace]

Suggested change

	every earlier occurrence of them. Do not mint ids for new rows yourself --
	every earlier occurrence of them. Do not generate ids for new rows yourself --