Repository navigation
fix(maintenance): require full snapshot history for orphan cleanup - #2599
iceberg-kitten wants to merge 8 commits into
Conversation
badalprasadsingh
left a comment
There was a problem hiding this comment.
few blocking comments @iceberg-kitten
|
|
||
| func validateCompleteSnapshotHistory(tbl *table.Table) error { | ||
| if tbl.SnapshotHistoryLimitedToRefs() { | ||
| return fmt.Errorf("table snapshot history is incomplete under REST snapshot-loading-mode=refs; reload with snapshot-loading-mode=all") |
There was a problem hiding this comment.
ran the lint locally, and its failing here: fmt.Errorf with no arguments, also, no blank lines before many "return err"
better: wrap the sentinel, like the refs-only cherry-pick check, so callers can use "errors.Is", and assert it with "require.ErrorIs"
return fmt.Errorf("%w: table snapshot history is incomplete under REST snapshot-loading-mode=refs; reload with snapshot-loading-mode=all", table.ErrInvalidOperation)| // concurrently and is bounded to defaultPurgeMaxConcurrency operations because | ||
| // it is typically I/O-bound. | ||
| func PurgeFiles(ctx context.Context, tbl *table.Table) error { | ||
| if err := validateCompleteSnapshotHistory(tbl); err != nil { |
There was a problem hiding this comment.
"PurgeFiles" only deletes. with the default gc.enabled=true it walks the table location, so refs-only history can only miss files outside it -> catalogs call it after the drop, when the table can't be reloaded as the hint says.
better:drop this check, its getReferencedFiles copy and the release-note mention.
zeroshade
left a comment
There was a problem hiding this comment.
Thanks, the guard itself looks right: it runs before tbl.FS() in all three orphan-cleanup entry points, and both new rejection tests fail if the check is disabled. CI is green. The changes requested are mostly docs and description:
website/src/api.mdalready has a### Orphan file cleanupsection; this adds a second one (see inline).- The
api.mdremediation text is misleading for library users (see inline). - The PR description says "Block orphan cleanup and PurgeFiles", but the code and docs deliberately allow
PurgeFiles. Please update it, since it becomes the squash commit message. - Commit ab02732 copies four unrelated release-note entries from main into
releases.md(RowDelta singleCommit, SQL nested namespaces, Directory IO context, RowDelta deletion-vector conflict). They merge cleanly, but please rebase onto main so the diff only contains this PR's entry.
Non-blocking:
TestPlanOrphanFilesRejectsRefsOnlyWithLiveHistoricalDatabuilds an httptest REST server and IO registry inside the maintenance package. The REST-specific part (thatLoadTablesetsSnapshotHistoryLimitedToRefs()forrefsvsall) would fit better incatalog/resttests, leaving the maintenance test to usetable.WithSnapshotHistoryLimitedToRefsdirectly.- Java's REST catalog lazily loads the full snapshot list in refs mode, so orphan cleanup works there. Rejecting here is the conservative choice; worth a sentence in the description.
- Possible follow-up:
ExpireSnapshotswith post-commit deletion has no refs-only guard either. Expiring a tag head on a refs-only table may delete files still referenced by an omitted but valid snapshot.
|
|
||
| Pass `table.WithPostCommit(true)` to delete the unreferenced data and metadata files after the commit lands. The `iceberg expire-snapshots` CLI command wraps the same operation - see [CLI](./cli.md). | ||
|
|
||
| ### Orphan cleanup |
There was a problem hiding this comment.
This page already has ### Orphan file cleanup under ## Maintenance (around line 773). Could the plan/execute description and the safety paragraph move into that section instead of adding a second orphan-cleanup heading?
| `snapshot-loading-mode=refs`, including dry runs. Refs-only metadata may | ||
| omit snapshots whose files are still live, so cleanup fails with | ||
| `table.ErrInvalidOperation` (check with `errors.Is`) before touching | ||
| the filesystem. Configure the REST server to load |
There was a problem hiding this comment.
For library users snapshot-loading-mode is a client catalog property (e.g. rest.WithAdditionalProps), not only a server setting. The server can force it via /v1/config overrides, which win over client props. Suggest something like: "Create the REST catalog with snapshot-loading-mode=all (or omit it); if the server forces refs through config overrides, orphan cleanup is not available through that catalog."
| if cfg.invalidPlanOption != nil { | ||
| return OrphanCleanupResult{}, cfg.invalidPlanOption | ||
| } | ||
| if err := validateCompleteSnapshotHistory(tbl); err != nil { |
There was a problem hiding this comment.
nit: here the refs-only check runs before validateOrphanCleanupGC, while DeleteOrphanFiles/PlanOrphanFiles check GC first (refs-only is checked inside planOrphanFiles). A table with both conditions returns a different error depending on the entry point. Consider using the same order everywhere.
nssalian
left a comment
There was a problem hiding this comment.
I ran TestPlanOrphanFilesRejectsRefsOnlySnapshotHistory with orphan_cleanup.go swapped back to the merge-base: plan, delete and execute all fail, and they pass on this head. git merge-tree against main and against #2601 (the ExpireSnapshots follow-up mentioned above) is clean, so the two v0.8.0 blockers can land in either order. Nothing blocking from me beyond the doc and description items already raised.
| - **`RowDelta.Commit` can be called once per builder.** A second `Commit` on the same `RowDelta` now returns an error wrapping `table.ErrInvalidOperation` and stages nothing, even if the first call failed, as `RewriteFiles.Commit` does. Previously, calling `Commit` again after a failed first call could succeed, and calling it again after a successful one staged the builder's files a second time. Create a new builder with `Transaction.NewRowDelta` for each commit. | ||
| - **Data-file replacements now reject concurrent deletes.** Commits staged through `ReplaceDataFiles`, `ReplaceDataFilesByPath`, `ReplaceFiles`, or rewrite operations can return `table.ErrConflictingDeleteFiles` when a concurrent position delete targets a replaced file or any concurrent equality delete is added. Compaction on tables with streaming equality deletes may see additional conflicts. | ||
| - **Orphan cleanup now rejects `gc.enabled=false`.** `DeleteOrphanFiles`, `PlanOrphanFiles`, `ExecuteOrphanCleanup`, and dry runs now return an error before filesystem access. The `iceberg clean-orphan-files` command is also affected. | ||
| - **Orphan cleanup requires complete snapshot history.** `DeleteOrphanFiles`, `PlanOrphanFiles`, and `ExecuteOrphanCleanup` now return an error when the table was loaded with REST `snapshot-loading-mode=refs`. Reload with `snapshot-loading-mode=all` before orphan cleanup. `maintenance.PurgeFiles` is unaffected. |
There was a problem hiding this comment.
the gc.enabled entry right above names dry runs and iceberg clean-orphan-files. Both hit this rejection too (the CLI calls PlanOrphanFiles before its dry-run branch in cmd/iceberg/clean_orphan_files.go:50), so worth naming them here as well.
| } | ||
|
|
||
| meta, err := table.NewMetadata(iceberg.NewSchema(0), nil, | ||
| table.UnsortedSortOrder, location, iceberg.Properties{"gc.enabled": tc.gcEnabled}) |
There was a problem hiding this comment.
table.GCEnabledKey instead of the "gc.enabled" literal, like the other tests in this package.
8f92ec3 to
1681697
Compare
Signed-off-by: Minh Vu <vuhoangminh97@gmail.com>
Signed-off-by: Minh Vu <vuhoangminh97@gmail.com>
Signed-off-by: Minh Vu <vuhoangminh97@gmail.com>
badalprasadsingh
left a comment
There was a problem hiding this comment.
the failing ci job is a lint timeout issue so a re-run should fix ci
otherwise this is approved from my side :)
…s-only-history-20261115 Signed-off-by: Minh Vu <vuhoangminh97@gmail.com>
…s-only-history-20261115
Signed-off-by: Minh Vu <vuhoangminh97@gmail.com>
Signed-off-by: Minh Vu <vuhoangminh97@gmail.com>
What
Require full snapshot history for orphan cleanup. This covers planning, dry runs, execution of saved plans, and the refreshes around one-shot cleanup.
Why
REST refs mode can omit snapshots that still reference live files. Cleanup needs the complete history to determine which files are safe to remove.
Implementation
table.ErrInvalidOperationwhen a table was loaded with refs-only history or a refresh returns refs-only history.snapshot-loading-mode=alland how server overrides affect catalog configuration.PurgeFilesavailable for table drops.