Repository navigation
fix(storage): refuse a replace on a partitioned table - #114
Conversation
replace_if_head documents that it works on unpartitioned tables only, but nothing checked it. On a partitioned table the new files carry no partition values, and writing the manifest against the table's partition spec panics in iceberg-rust's manifest writer, whose zip_eq over the partition values and the spec's fields finds lengths that differ. By then the data files are already on disk. prepare_replace now refuses a table whose default partition spec has any field, before the head check and before any file is written. Signed-off-by: Mayur Kale <hello@mayurkale.com>
|
No actionable comments were generated in the recent review. Summary
Merge Risk:
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #114 +/- ##
==========================================
+ Coverage 99.03% 99.04% +0.01%
==========================================
Files 16 16
Lines 2477 2521 +44
==========================================
+ Hits 2453 2497 +44
Misses 24 24 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
What
replace_if_headrefuses a table that has a partition spec, before anyfile is written. Follows up #106.
How
prepare_replacechecks the table's default partition spec right afterthe format version check and refuses it if it has any field. The check
uses the field list rather than
is_unpartitioned, because a void fieldstill widens the partition tuple the manifest writer expects.
Things to know
(
write_filesbuilds with none) andwrite_snapshotwrites themagainst the table's spec. iceberg-rust's manifest writer then panics in
a
zip_eqover the two, after the data files are on disk.code. Both touch
replace.rsin different places; whichever landssecond gets main merged in.
Tests
only_unpartitioned_tables_are_replacedcreates a table partitionedon
id, asks for a replace and checks it is refused with no Parquetfile written. Without the check it fails with a moved head error
instead.