Skip to content

fix(transaction): drop already-Deleted entries and preserve manifest schema/spec on overwrite - #3

Closed
u70b3 wants to merge 1 commit into
glitchy:feat/overwrite-actionfrom
u70b3:fix/pr2185-p0-overwrite-issues
Closed

u70b3 wants to merge 1 commit into
glitchy:feat/overwrite-actionfrom
u70b3:fix/pr2185-p0-overwrite-issues

Conversation

@u70b3

@u70b3 u70b3 commented Jul 5, 2026 •

Copy link
Copy Markdown

This PR fixes two overwrite correctness bugs in the OverwriteAction CoW delete support from apache#2185:

  1. Drop already-Deleted entries on rewrite — OverwriteOperation::rewrite_manifest previously sent only entry.is_alive() && deleted_file_paths.contains(...) through add_deleted_entry; entries already marked ManifestStatus::Deleted fell through to add_existing_entry, flipping them back to Existing. Now if !entry.is_alive() { continue; } drops already-Deleted entries so their deletion record stays intact in the prior snapshot.

    Regression test: test_overwrite_preserves_previously_deleted_entries (asserts the previously-Deleted entry does not reappear in the rewritten manifest; the newly deleted entry is Deleted; unrelated alive entries stay alive).

  2. Use the original manifest's schema / partition spec on rewrite — rewrite_manifest previously built the ManifestWriterBuilder with table.metadata().current_schema() and default_partition_spec(), which after schema/partition evolution writes a manifest whose schema-id / partition-spec-id don't match the original entries. Now uses manifest.metadata().schema and manifest.metadata().partition_spec.

    Regression test: test_overwrite_rewritten_manifest_preserves_original_partition_spec.

Tests

cargo test -p iceberg transaction::overwrite --lib → 7 passed. transaction::append --lib → 7 passed.

cc @glitchy

@u70b3
u70b3 force-pushed the fix/pr2185-p0-overwrite-issues branch 2 times, most recently from 1a37caf to 7d61df8 Compare July 5, 2026 09:02
@u70b3 u70b3 changed the title fix(transaction): preserve delete-only manifests and existing Deleted entries in overwrite fix(transaction): drop already-Deleted entries and preserve manifest schema/spec on overwrite Jul 5, 2026
@u70b3
u70b3 force-pushed the fix/pr2185-p0-overwrite-issues branch from 7d61df8 to 603d071 Compare July 5, 2026 09:21
@u70b3 u70b3 closed this Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant