Skip to content

feat: Decode dictionary batches - #858

Merged
paleolimbot merged 17 commits into
apache:mainfrom
paleolimbot:decode-dictionaries-2
Apr 7, 2026
Merged

paleolimbot merged 17 commits into
apache:mainfrom
paleolimbot:decode-dictionaries-2

Conversation

@paleolimbot

@paleolimbot paleolimbot commented Mar 25, 2026 •

Copy link
Copy Markdown
Member

After #856 we can know which nodes in an ArrowSchema are dictionary encoded and what their IDs are. This PR implements the ArrowIpcDictionaries tracker and the actual decode step.

Doesn't quite solve #845 because we need one more step (thread the ArrowIpcDictionaries through the decode, which involves solving some things about cloning arrays and array views/validation.

@paleolimbot
paleolimbot marked this pull request as ready for review April 4, 2026 03:50
@paleolimbot

Copy link
Copy Markdown
Member Author

If there are no objections here, I'll merge this evening to move on to the harder pieces.

@paleolimbot
paleolimbot merged commit 86a0b07 into apache:main Apr 7, 2026
42 checks passed
@paleolimbot
paleolimbot deleted the decode-dictionaries-2 branch April 7, 2026 01:15
@paleolimbot paleolimbot added this to the nanoarrow 0.9.0 milestone Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant