Repository navigation
fix: validate unaligned offset buffers safely - #946
Merged
paleolimbot merged 2 commits intoSep 25, 2026
Merged
paleolimbot merged 2 commits into
paleolimbot merged 2 commits into
Conversation
Signed-off-by: Efe Gökdemir <efe@rexcode.co.uk>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #946 +/- ##
=======================================
Coverage 78.19% 78.19%
=======================================
Files 106 106
Lines 16843 16845 +2
Branches 1985 1985
=======================================
+ Hits 13170 13172 +2
Misses 2439 2439
Partials 1234 1234 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Signed-off-by: Efe Gökdemir <efe@rexcode.co.uk>
paleolimbot
approved these changes
Sep 25, 2026
paleolimbot
left a comment
Member
There was a problem hiding this comment.
Thank you!
I'll open a follow up to try to integrate these more...there are definitely a few more places we're reading unaligned buffers.
pedrobtz
added a commit
to pedrobtz/arrow-nanoarrow
that referenced
this pull request
Oct 22, 2026
…naligned buffers safely Default and full validation read these through aligned pointers, which is undefined behaviour for the unaligned buffers the C Data interface allows. Follows the fix for offsets buffers in apache#946. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
paleolimbot
added a commit
that referenced
this pull request
Nov 5, 2026
…naligned buffers safely (#990) #946 fixed `ArrowArrayViewValidate()` reading offsets buffers through aligned pointers (#945). Validation still does the same in three other places: - run ends, through `ArrowArrayViewGetIntUnsafe()`, in default validation (first and last run end) and full validation (the increasing check); - list view and large list view offsets and sizes, through `data.as_int32[i]` and `data.as_int64[i]`, in full validation; - dictionary indices, through `ArrowArrayViewGetIntUnsafe()`, in full validation's range check. Each is a misaligned load when the buffer isn't aligned to the element width. That is undefined behaviour, and UBSan reports it, for input the C Data interface allows. This adds a static `ArrowArrayViewGetIntUnaligned()` to `array.c`. It reads the 16-, 32- and 64-bit integer types with `memcpy` and falls back to `ArrowArrayViewGetIntUnsafe()` for the others. The run-end and dictionary-index sites use it. The list view sites use `ArrowBufferViewGetInt32Unsafe()` and `ArrowBufferViewGetInt64Unsafe()` from #946. The new `ArrayViewTestValidateFullUnalignedIntegers` test validates, at the full level, a dictionary-encoded int32 view with indices at an odd address. It does the same for a run-end-encoded view with odd-addressed run ends, and for list view and large list view with odd-addressed offsets and sizes. Without the fix, UBSan reports the misaligned load (`inline_array.h:1160`). With it, `ctest` is clean under `UBSAN_OPTIONS=halt_on_error=1`. Found by fuzzing the IPC reader in the [zuarrow](https://github.com/pedrobtz/zuarrow) R package, which vendors nanoarrow. A pre-0.15 message whose body buffer starts at an odd offset reached the dictionary index check through `ArrowIpcArrayStreamReader`. `ArrowArrayAppendStorageFromArrayViewElement()` and `ArrowArrayAppendStorageFromArrayViewRange()` read integers, list view sizes and run ends the same way. They are not validation, so they are left for a separate change. `ctest` passes in a plain Debug build and with ASan and UBSan with `halt_on_error=1` (410 tests each). The changed files are formatted with the pinned clang-format 16.0.6. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Dewey Dunnington <dewey@dunnington.ca>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #945 by making validation of Arrow offset buffers safe for unaligned C Data Interface buffers.
Changes
memcpyhelpers for reading 32-bit and 64-bit buffer values.Testing
ArrowArrayViewValidate: 0with no UBSan diagnostics.cc -std=c99 -Wall -Wextra -Werror -I/tmp/nanoarrow945-dist-fixed/include -Isrc -fsyntax-only src/nanoarrow/common/array.c: passed.git diff --check: passed.Notes
The change keeps the existing validation results and avoids typed dereferences of externally supplied offset buffers. AI assistance was used during investigation and implementation; the submitter reviewed the changed lines and validation results.