You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Core: RewriteManifests can commit stale sequence numbers and row IDs after a transaction retry #19026
When RewriteManifests explicitly replaces a manifest staged earlier in the same transaction, a concurrent commit can cause the transaction to retry and publish the replacement with metadata from the original attempt.
Stage an append of file A with Transaction.newFastAppend().
Read the staged manifest and write a replacement that copies its entries as EXISTING. In the same transaction, use RewriteManifests to delete the staged manifest and add the replacement.
Append file B outside the transaction, then call commitTransaction().
On retry, the staged manifest keeps its path but can inherit a new sequence number and first row ID from the new manifest list. validateDeletedManifests checks presence by path, so the caller-supplied replacement can still be committed with the original values.
The regression test reports a stale data sequence number for file A in format v2 and overlapping row-ID ranges for files A and B in formats v3 and v4. A stale data sequence number can also affect which equality deletes apply.
The commit should preserve the sequence numbers and row IDs assigned on retry, or reject the replacement if its source manifest's inherited metadata changed.
@manuzhang I opened #19027 for this. Following apache/iceberg-cpp#1104, it rejects the commit when an explicitly replaced manifest's sequence numbers or first row ID changed on retry, and it includes your regression test from manuzhang#214, changed to expect that failure. Happy to switch to the preserve approach if you'd prefer.
Activity
shivadarshan-devadiga commentedon Nov 18, 2026
@manuzhang I opened #19027 for this. Following apache/iceberg-cpp#1104, it rejects the commit when an explicitly replaced manifest's sequence numbers or first row ID changed on retry, and it includes your regression test from manuzhang#214, changed to expect that failure. Happy to switch to the preserve approach if you'd prefer.