From bb93becd45603f1a20f0931a981e21c3c9ccfabb Mon Sep 17 00:00:00 2001 From: Manu Zhang Date: Thu, 19 Nov 2026 22:53:55 +0800 Subject: [PATCH] Spec: Clarify per-file name mapping fallback The column projection rules can be read as permitting name mapping for individual fields in an ID-bearing file. Clarify file-level resolution and how missing nested IDs are handled. Generated-by: Codex --- format/spec.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/format/spec.md b/format/spec.md index 58a21d4e5c3d..9a40ae5137fb 100644 --- a/format/spec.md +++ b/format/spec.md @@ -453,10 +453,12 @@ Struct evolution requires the following rules for default values: Columns in Iceberg data files are selected by field id. The table schema's column names and order may change after a data file is written, and projection must be done using field ids. -Values for field ids which are not present in a data file must be resolved according the following rules: +Before projection, readers must check the full file schema for field IDs at any depth. If any are present, readers must use stored IDs and treat fields without IDs, lists missing an element ID, and maps missing a key or value ID as absent. + +Values for absent fields must be resolved according to the following rules: * Return the value from partition metadata if an [Identity Transform](#partition-transforms) exists for the field and the partition value is present in the `partition` struct on `data_file` object in the manifest. This allows for metadata only migrations of Hive tables. -* Use `schema.name-mapping.default` metadata to map field id to columns without field id as described below and use the column if it is present. +* If the data file contains no field IDs and the table defines `schema.name-mapping.default`, use the name mapping described below to resolve the field ID and return the value if the field is present. * Return the default value if it has a defined `initial-default` (See [Default values](#default-values) section for more details). * Return `null` in all other cases.