Skip to content

fix(table): merge v2 position deletes into first v3 DV - #2624

Open
iceberg-kitten wants to merge 24 commits into
apache:mainfrom
iceberg-kitten:fix/mor-v3-legacy-pos-deletes-2604
Open

iceberg-kitten wants to merge 24 commits into
apache:mainfrom
iceberg-kitten:fix/mor-v3-legacy-pos-deletes-2604

Conversation

@iceberg-kitten

@iceberg-kitten iceberg-kitten commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

What

Carry older position deletes into the first DV for a data file.

Why

Iceberg readers may skip position-delete files when a DV applies. If the older positions are not copied, rows can reappear.

Implementation

Merge old deletes per data file. Write a DV only when a new position is found. Keep a shared delete file live while another data file still needs it. Cover multiple old delete files and historical partition specs.

Fixes #2604.

Minh Vu and others added 4 commits November 17, 2026 01:29
Signed-off-by: Minh Vu <vuhoangminh97@gmail.com>
Signed-off-by: Minh Vu <vuhoangminh97@gmail.com>
Signed-off-by: Minh Vu <vuhoangminh97@gmail.com>

@badalprasadsingh badalprasadsingh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

first pass @iceberg-kitten

Comment thread table/transaction_delete.go Outdated
Comment on lines +1039 to +1045
for dataFilePath, deleteFiles := range oldPositionDeletes.fileScoped {
if _, replaced := seen[dataFilePath]; !replaced {
continue
}
for _, deleteFile := range deleteFiles {
updater.removeDeleteFile(deleteFile)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removing these files makes the commit "non-replayable", so if another writer commits first -> this delete now fails with "ErrCommitFailed" instead of retrying

the delete docs (transaction_delete.go ;ine275, table.go line422) name only dv removals, could you update both?

Comment thread table/transaction_delete.go Outdated
Comment on lines 1047 to 1051
for dataFilePath, dvFile := range existingDVs {
if _, replaced := seen[dataFilePath]; replaced {
updater.removeDeletionVector(dvFile)
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no test covers a v3 delete that finds no new rows in a file with a dv -> It relies on the seen check here and the "formatVersion < 3" guard at line1027. without either, the dv is dropped and deleted rows come back.

please add a test

Comment thread table/arrow_utils.go Outdated
Comment on lines +2484 to +2491
// Seed a file's existing deletes only after a new position is found for
// it. A filter can select files from their stats even when no live row
// matches; seeding those files eagerly would turn an empty delete into a
// DV rewrite. The scan does not apply existing DVs, so skip positions
// already present in the seed before deciding that a replacement is
// needed.
hasEntries := false
for filePath, bitmap := range args.existingDVs {
pCtx, ok := partitionContextByFilePath[filePath]
if !ok {
yield(nil, fmt.Errorf("unexpected missing partition context for existing deletion vector path %s", filePath))

return
}
writer.Load(filePath, bitmap, pCtx.specID, pCtx.partitionData)
hasEntries = true
}
seededPaths := make(map[string]struct{}, len(args.existingDVs))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this now seeds older pos-deletes too, and only once a new position is found, but the doc of "recordWritingArgs.existingDVs" still describes only dvs, and both it and" existingDVBitmaps" have dv-only names.

better: can you rename them (say "existingDeletes") and update that doc?

@nssalian nssalian left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @iceberg-kitten for the PR.
I reverted arrow_utils.go, transaction_delete.go and table.go to the merge base and re-ran the new tests. On the merge base the upgrade cases bring deleted rows back. For example, TestV3MergeOnReadUpgradePreservesPositionDeletes/single_file returns ids [1 3] instead of [3], and TestV3MergeOnReadUpgradeMergesMultiplePositionDeleteFiles returns [1 2 4 5] instead of [4 5]. All of them pass on this head.

The handling of old deletes matches Java's DV writer. It merges every earlier position for a data file into the new DV, but it only removes delete files that are scoped to that one data file, and shared position-delete files stay live (https://github.com/apache/iceberg/blob/a6bc1f95e3fe388419fd62186667a3bd08ccd74a/core/src/main/java/org/apache/iceberg/deletes/BaseDVFileWriter.java#L125-L131). Go's scan also skips position-delete files once a DV applies to a data file (scanner.go:1626). So the tests that check scan results are, in effect, checking that the DV carries the old positions.

Worth fixing the CI.

Comment thread table/mor_delete_pruning_test.go Outdated
func TestV3MergeOnReadMixedExistingDVsKeepsUnchangedFile(t *testing.T) {
ctx := t.Context()
tbl := newPartitionedMergeOnReadTestTable(t)
arrowSchema := arrow.NewSchema([]arrow.Field{

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test builds its six rows by a bit by hand: an Arrow schema for id and category, then array.TableFromJSON, then tbl.Append. TestV3MergeOnReadUpgradePreservesPartitionedPositionDeletes (line 712) repeats the same ~12 lines, and so does the existing TestV3MergeOnReadDoubleDeletePartitionedMergesDeletionVectors (line 232). The appendRows(t, tbl, json) helper in cow_rewrite_deletes_test.go:408 gets the Arrow schema from the table and does the append, so each block can be a single tbl = appendRows(t, tbl, ...) call.

Comment thread website/src/api.md Outdated
With `write.delete.mode=merge-on-read`, format v2 writes position-delete
files and format v3 writes deletion vectors (DVs). After a table is upgraded
from v2 to v3, its older position-delete files remain valid. The **first**
DV written for a data file includes all positions previously deleted from

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: "includes all positions previously deleted from that file" could be read as including rows removed by equality deletes. Only position deletes are folded into the DV, and equality deletes keep applying on their own. The release note's wording, "applicable legacy position-delete files", is more precise and could be used here too.

Minh Vu added 2 commits November 18, 2026 13:18
Signed-off-by: Minh Vu <vuhoangminh97@gmail.com>
Signed-off-by: Minh Vu <vuhoangminh97@gmail.com>

@badalprasadsingh badalprasadsingh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approving @iceberg-kitten :)

@AynSiddiqui

Copy link
Copy Markdown
Contributor

I re-ran my local tests at 6655a71. Two cases beyond the upgrade ones already covered here: if a peer commit lands while a delete that removes a legacy position-delete file is staged, the commit fails with ErrCommitFailed and retrying the delete succeeds; and an equality delete stays live next to the new DV.

I also applied #2639 (the v2 position-delete spec fix for #2606) on top. There are no conflicts and go test ./table/... passes. With a v2 delete after a spec change, then the upgrade and a v3 delete, the DV keeps the data file's spec and partition and the scans return [2 3] and then [3].

Minh Vu added 2 commits November 18, 2026 19:42

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Transaction: merge-on-read Delete on a table upgraded to v3 brings back rows deleted before the upgrade

4 participants