Repository navigation
refactor: guard oversized bucket hash input in BucketIndex - #1023
Merged
Merged
Conversation
BucketUtils::HashBytes still rejects inputs larger than INT32_MAX with an IcebergError, but BucketIndex now checks the byte length of string, binary, and fixed literals before hashing and returns InvalidArgument instead, so the Result-returning entry point no longer throws. This replaces the internal HashBytes(const uint8_t*, size_t) overload and the bucket_util_internal.h header, which existed only so a test could reach the guard without materializing an oversized value. That overload, the header, and the HashBytesRejectsOversizedInput test are all removed: the guard cannot be exercised from a test without either a 2 GiB allocation or an invalid std::span.
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
3 open findings
HashedByteSize()duplicates knowledge about whichTypeIds are variable-length and hashed as raw… · NewICEBERG_CHECK_OR_DIEcommonly implies process termination rather than an exception; however, the… · New The guard boundsbytes.size()byINT32_MAX(kMaxHashBytes), but the call narrows toint… · New
What changed in this PR
Refactors oversized bucket-hash input handling by removing the internal pointer/length hashing entry point and instead validating literal byte lengths at the BucketIndex Result<> boundary (while keeping BucketUtils::HashBytes as a defensive check).
Changes:
- Removed
bucket_util_internal.hand the internalHashBytes(const uint8_t*, size_t)overload (and its test). - Added
BucketIndexpre-validation for string/binary/fixed literal byte lengths to returnInvalidArgumentinstead of throwing. - Kept
BucketUtils::HashBytesoversized-length rejection, updating implementation to use a sharedkMaxHashBytes.
| File | Description |
|---|---|
| src/iceberg/util/bucket_util_internal.h | Removes internal hashing entry point scaffolding introduced for testing oversized inputs. |
| src/iceberg/util/bucket_util.h | Updates BucketIndex documentation to no longer claim it may throw for oversized inputs. |
| src/iceberg/util/bucket_util.cc | Adds literal byte-size precheck in BucketIndex; refactors HashBytes guard to use kMaxHashBytes. |
| src/iceberg/test/bucket_util_test.cc | Removes oversized-input test and associated internal-header dependencies. |
🧠 Review effort: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Copilot stopped reviewing on behalf of
wgtmac due to an error
October 15, 2026 09:12
blackmwk
approved these changes
Oct 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Follow-up to #961. That PR rejected oversized inputs inside
BucketUtils::HashBytes, but to cover the guard with a test it also added an internalHashBytes(const uint8_t*, size_t)overload plusbucket_util_internal.h, and the regression test needed a pointer/length pair because astd::spancovering more elements than the object provides is undefined behavior ([span.cons]).This PR removes that scaffolding and enforces the limit at the
Resultboundary instead:BucketIndexchecks the byte length of string, binary, and fixed literals before hashing and returnsInvalidArgument, so theResult-returning entry point never throws. Its documentation no longer claims otherwise.BucketUtils::HashByteskeeps rejecting inputs aboveINT32_MAXwithIcebergErrorand keeps documenting that throw; it remains the defensive barrier for direct callers.bucket_util_internal.h, the internal overload, andHashBytesRejectsOversizedInputare removed.The bound is unchanged:
INT32_MAXis what iceberg-java can express, since Java hashes int-indexedbyte[]values, so no legal bucket input is rejected.On test coverage: reaching the guard requires either a 2 GiB allocation or an invalid
std::span, so the behavior is covered by review and CI rather than by a fabricated input. Deleted tests: 1.Test plan
util_test(379 tests, 44 suites) passes on this head.bucket_util.ccandbucket_util_test.cccompile with warnings as errors.