Repository navigation
Description
Warehouse::append_if_head (crates/storage/src/lib.rs:171-221 at a0b5e6c) commits new rows only if the table's head snapshot is still the one the caller read, as ADR 0006 requires (docs/adr/0006-ordered-job-commits.md:52-54). It loads the table and checks the head (crates/storage/src/lib.rs:198-202), writes the Parquet files (:203), and commits through commit_if_head (:219, :523-549). commit_if_head wraps the SQL catalog in HeadGuard (:404-449), a catalog whose load_table sets a tripped flag and fails whenever the table's current snapshot id is not the expected head (:431-445). The runner in #13 will reuse the same guard so that a stale run fails instead of committing.
The guard is needed because of how iceberg-rust, at the revision pinned in Cargo.toml:29, commits a transaction. Transaction::commit loads the table through the catalog it is given, rebases the transaction onto that table if the table changed, runs the transaction's actions (an append writes its manifest and manifest list here), and calls the catalog's update_table. SqlCatalog::update_table then loads the table a second time, checks the commit's requirements against it, writes the new metadata.json, and switches the table to it with one SQL UPDATE that matches only the old metadata location, which makes it a compare-and-swap. A failed requirement and a failed compare-and-swap both raise a retryable ErrorKind::CatalogCommitConflicts. Transaction::commit retries such errors up to commit.retry.num-retries times, a count it reads from the table the transaction started from, and every retry begins with a new load through the guard.
A foreign commit can therefore land in one of three windows. In the first, before the guard's load, the head check stops the writer if the foreign commit landed before append_if_head loaded the table, and the guard stops it otherwise. In the second, between the guard's load and the SQL catalog's own load in update_table, the requirement that fast_append adds on the main branch (TableRequirement::RefSnapshotIdMatch) fails. In the third, after that load and before the UPDATE, the compare-and-swap fails. In the second and third windows the retry reloads the table through the guard, the guard trips, and commit_if_head reports Error::Conflict (crates/storage/src/lib.rs:539-546).
This ticket closes four gaps. First, when there is no retry, because commit.retry.num-retries is 0 or the retries are used up, the conflict error reaches commit_if_head with tripped still false and becomes Error::Storage (crates/storage/src/lib.rs:547), so the caller cannot tell a lost race from a broken catalog. Nothing pins the retry count either, because create_table writes only the Parquet codec into a new table's properties (:122-126). Second, the guard compares only the snapshot id (:433-434), so a schema change that lands after the writer's load goes unnoticed: iceberg-rust rebases the commit onto the changed table, and Parquet written with the old schema (:342-343) can end up in a snapshot that records the new schema id. Third, the guard's doc comment (:408-410) credits the compare-and-swap for every commit that lands between the guard's load and the catalog update, although in the second window the branch requirement is what stops it. Fourth, the guard test (:845-859) commits the foreign snapshot before commit_if_head runs, so no test commits inside the second or third window.
The code and its unit tests are all in crates/storage/src/lib.rs, with the tests in mod tests (:654-655). Run them with cargo test -p infostash-storage, and run the checks in CONTRIBUTING.md:29-34 before opening a pull request.
Crate: crates/storage (src/lib.rs only)
Done when
- The
HeadGuarddoc comment (crates/storage/src/lib.rs:404-410) names the three windows and what stops a foreign commit in each: the head check at:198-202or the guard itself before the guard's load, themainbranch requirement between the guard's load and the SQL catalog's own load inupdate_table, and the compare-and-swap on the metadata location after that load. It also says that the last two raise a retryableErrorKind::CatalogCommitConflicts, and that the retry reloads the table through the guard, which trips, socommit_if_headreportsError::Conflict. - The same doc comment says that a transaction which only sets properties carries no branch requirement and is last-writer-wins, so the
infostash.*properties (docs/adr/0006-ordered-job-commits.md:49-51) are protected only becauseappend_if_headcommits them in one transaction with afast_append(crates/storage/src/lib.rs:205-218). -
commit_if_head(crates/storage/src/lib.rs:537-548) also returnsError::Conflictwhentrippedis false but the error's kind isErrorKind::CatalogCommitConflicts, reloading the head for the message as the tripped branch does (:539-546). Every other error still becomesError::Storage(:547), and the doc comment oncommit_if_head(:523-524) describes both cases. -
create_table(crates/storage/src/lib.rs:122-131) also writescommit.retry.num-retries(TableProperties::PROPERTY_COMMIT_NUM_RETRIES) next to the Parquet codec, set to iceberg-rust's default of 4 (TableProperties::PROPERTY_COMMIT_NUM_RETRIES_DEFAULT). A test besidenew_tables_write_zstd(:764-774) checks it on a new table the way:768-770checks the codec. storage: set a snapshot expiry floor at create_table #91 adds two expiry properties to the same map (:123-126), so whichever change lands second keeps the other's properties. -
HeadGuardalso holds the schema id of the table the writer loaded atcrates/storage/src/lib.rs:198, whose schemawrite_filesuses (:342-343). It setstrippedand refuses any load whose current schema id differs, as it does for a moved head (:434-443), and its doc comment says so.commit_if_headand the code at:219,:855and:865-870are updated to supply the schema id. When the head did not move, theError::Conflictmessage says that the schema changed, instead of the two equal head ids thatmoved(:326-333) would print. A concurrent schema change thus becomesError::Conflict, which is right under one writer per table (docs/adr/0006-ordered-job-commits.md:73-74). - Test:
HeadGuardandcommit_if_headtake the inner catalog as&dyn Cataloginstead of&SqlCatalog(crates/storage/src/lib.rs:412,:526), so that a test can put its own catalog under the guard. That test-only catalog, defined inmod tests, delegates every method to the SQL catalog the way the guard does (:447-520), except that itsupdate_tablefirst commits another snapshot directly through the SQL catalog, for example withWarehouse::appendas at:850, and then delegates. The test asserts thatcommit_if_headreturnsError::Conflictand that the foreign snapshot is the table's only one. It also counts the files under the table's directory by kind against a count taken right after the table is created, and asserts two new Parquet files, two new manifests, two new manifest lists (the.avrofiles whose names start withsnap-) and one newmetadata.json. That is one of each from the foreign commit, plus the losing writer's Parquet file, manifest and manifest list, which are the orphans storage: remove orphan data files #40 will remove. - Test: the same race on a table whose
commit.retry.num-retrieswas set to 0 before the writer loaded it, set the waycrates/storage/src/lib.rs:780-790sets the codec (iceberg-rust reads the retry count from the table a transaction starts from), returnsError::Conflictwhere the code today returnsError::Storage. - Test: modeled on
crates/storage/src/lib.rs:845-859, a column added after the writer loaded the table, through a transaction such asupdate_schema().add_column(AddColumn::optional(..))committed with&wh.catalog, makescommit_if_headreturnError::Conflictand leaves the table with no snapshot, where today the commit succeeds.
Points: 2 (M4 HeadGuard conflict mapping and doc fix 2)
Depends on: nothing
Blocks: nothing
Source: design review of 2026-11-01, finding C20.
Activity