Skip to content

Vortex: Store a name mapping instead of the Iceberg schema - #41

Merged
robert3005 merged 1 commit into
mainfrom
rk/simplermapping
Sep 27, 2026
Merged

robert3005 merged 1 commit into
mainfrom
rk/simplermapping

Conversation

@robert3005

@robert3005 robert3005 commented Sep 27, 2026 •

Copy link
Copy Markdown
Member

Vortex files persisted the full Iceberg schema only so readers could
tag Arrow fields with field IDs. Readers never used the stored types,
so a name mapping carries all the needed information in less space.

The mapping omits unknown struct fields because the file does not
store them.

Vortex files persisted the full Iceberg schema only so readers could
tag Arrow fields with field IDs. Readers never used the stored types,
so a name mapping carries all the needed information in less space.

The mapping omits unknown struct fields because the file does not
store them. Readers match list elements by position because Vortex
reads them back as "item", not "element".

Generated-by: Claude Code
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@robert3005 robert3005 changed the title Store a name mapping instead of the Iceberg schema Vortex: Store a name mapping instead of the Iceberg schema Sep 27, 2026
@robert3005
robert3005 merged commit 662d8fb into main Sep 27, 2026
40 of 49 checks passed
@robert3005
robert3005 deleted the rk/simplermapping branch September 27, 2026 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant