Skip to content

Add ExpireSnapshots.retain_last - #4437

Open
rambleraptor wants to merge 1 commit into
apache:mainfrom
rambleraptor:expire-snapshots-retain-last
Open

rambleraptor wants to merge 1 commit into
apache:mainfrom
rambleraptor:expire-snapshots-retain-last

Conversation

@rambleraptor

@rambleraptor rambleraptor commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

Rationale for this change

This helper method lets you retain the N most recent ancestors of a branch. I've included some doc examples of how you can use this for chaining multiple commands together.

Are these changes tested?

Unit tests included

Are there any user-facing changes?

  • Added retain_last method to ExpireSnapshots.

Port Java's ExpireSnapshots.retainLast: snapshots that older_than would
expire are kept if they are among the N most recent ancestors of a
branch. Snapshots expired by ID are unaffected.
@rambleraptor
rambleraptor requested a review from Fokko November 19, 2026 23:25
@rambleraptor
rambleraptor force-pushed the expire-snapshots-retain-last branch from 142a52d to 0b81a7f Compare November 19, 2026 23:25
self._snapshot_ids_older_than.add(snapshot.snapshot_id)
return self

def retain_last(self, num_snapshots: int) -> ExpireSnapshots:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Java, retainLast(n) alone still expires snapshots older than history.expire.max-snapshot-age-ms, keeping n per branch (RemoveSnapshots, test: apache/iceberg#19049). Here it does nothing without older_than.

Could we note that in the docstring? Honoring history.expire.max-snapshot-age-ms and history.expire.min-snapshots-to-keep (and the ref-level overrides) looks like a separate follow-up.

assert {snapshot.snapshot_id for snapshot in txn.table_metadata.snapshots} == {2, 3, 4, 5}


def test_expire_snapshots_retain_last_does_not_retain_by_id(example_table_metadata_v2: dict[str, Any]) -> None:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add a test that retain_last alone expires nothing? It pins the current behavior, so it's a visible change if the history.expire.* defaults get wired in later.

def test_expire_snapshots_retain_last_without_older_than(example_table_metadata_v2: dict[str, Any]) -> None:
    table = _table_with_branches(example_table_metadata_v2)

    txn = table.transaction()
    ExpireSnapshots(txn).retain_last(1).commit()

    assert {snapshot.snapshot_id for snapshot in txn.table_metadata.snapshots} == {1, 2, 3, 4, 5}

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants