You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Parquet reader: fall back to positional field IDs when a file has no field IDs and no name mapping #1046
#1044 adds name mapping support to the Parquet reader. When a Parquet file has no field IDs and no name mapping is provided, the reader returns InvalidSchema, which matches the current C++ Avro reader (avro_reader.cc).
The Java Parquet reader handles this case differently. As pointed out in #1044, ReadConf falls back to addFallbackIds and assigns positional IDs to top-level columns, so such files can still be read.
Proposal
When the Parquet file schema has no field IDs and no name mapping is provided, assign fallback IDs by top-level column position, as Java does, instead of returning an error.
Open questions:
Should the Avro reader get the same fallback for consistency?
@wgtmac I saw your comments and seems we should keep the current behaviour, I did see that iceberg-rust has this fallback but it seems they have a valid use-case for implementing this, apart from that every other version does not seem to have this fallback.
Activity
dino-saurabh commentedon Nov 13, 2026
@wgtmac I saw your comments and seems we should keep the current behaviour, I did see that iceberg-rust has this fallback but it seems they have a valid use-case for implementing this, apart from that every other version does not seem to have this fallback.