Skip to content

refactor: guard oversized bucket hash input in BucketIndex - #1023

Merged
wgtmac merged 1 commit into
mainfrom
codex/bucket-index-oversized-guard
Oct 15, 2026
Merged

wgtmac merged 1 commit into
mainfrom
codex/bucket-index-oversized-guard

Conversation

@wgtmac

@wgtmac wgtmac commented Oct 15, 2026

Copy link
Copy Markdown
Member

Summary

Follow-up to #961. That PR rejected oversized inputs inside BucketUtils::HashBytes, but to cover the guard with a test it also added an internal HashBytes(const uint8_t*, size_t) overload plus bucket_util_internal.h, and the regression test needed a pointer/length pair because a std::span covering more elements than the object provides is undefined behavior ([span.cons]).

This PR removes that scaffolding and enforces the limit at the Result boundary instead:

  • BucketIndex checks the byte length of string, binary, and fixed literals before hashing and returns InvalidArgument, so the Result-returning entry point never throws. Its documentation no longer claims otherwise.
  • BucketUtils::HashBytes keeps rejecting inputs above INT32_MAX with IcebergError and keeps documenting that throw; it remains the defensive barrier for direct callers.
  • bucket_util_internal.h, the internal overload, and HashBytesRejectsOversizedInput are removed.

The bound is unchanged: INT32_MAX is what iceberg-java can express, since Java hashes int-indexed byte[] values, so no legal bucket input is rejected.

On test coverage: reaching the guard requires either a 2 GiB allocation or an invalid std::span, so the behavior is covered by review and CI rather than by a fabricated input. Deleted tests: 1.

Test plan

  • util_test (379 tests, 44 suites) passes on this head.
  • bucket_util.cc and bucket_util_test.cc compile with warnings as errors.

BucketUtils::HashBytes still rejects inputs larger than INT32_MAX with an
IcebergError, but BucketIndex now checks the byte length of string, binary, and
fixed literals before hashing and returns InvalidArgument instead, so the
Result-returning entry point no longer throws.

This replaces the internal HashBytes(const uint8_t*, size_t) overload and the
bucket_util_internal.h header, which existed only so a test could reach the
guard without materializing an oversized value. That overload, the header, and
the HashBytesRejectsOversizedInput test are all removed: the guard cannot be
exercised from a test without either a 2 GiB allocation or an invalid
std::span.
Copilot AI balanced review requested due to automatic review settings October 15, 2026 08:53

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

3 open findings
What changed in this PR

Refactors oversized bucket-hash input handling by removing the internal pointer/length hashing entry point and instead validating literal byte lengths at the BucketIndex Result<> boundary (while keeping BucketUtils::HashBytes as a defensive check).

Changes:

  • Removed bucket_util_internal.h and the internal HashBytes(const uint8_t*, size_t) overload (and its test).
  • Added BucketIndex pre-validation for string/binary/fixed literal byte lengths to return InvalidArgument instead of throwing.
  • Kept BucketUtils::HashBytes oversized-length rejection, updating implementation to use a shared kMaxHashBytes.
File Description
src/​iceberg/​util/​bucket_util_internal.h Removes internal hashing entry point scaffolding introduced for testing oversized inputs.
src/​iceberg/​util/​bucket_util.h Updates BucketIndex documentation to no longer claim it may throw for oversized inputs.
src/​iceberg/​util/​bucket_util.cc Adds literal byte-size precheck in BucketIndex; refactors HashBytes guard to use kMaxHashBytes.
src/​iceberg/​test/​bucket_util_test.cc Removes oversized-input test and associated internal-header dependencies.

🧠 Review effort: Lite


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/iceberg/util/bucket_util.cc Resolved
Comment thread src/iceberg/util/bucket_util.cc Resolved
Comment thread src/iceberg/util/bucket_util.cc Resolved
@wgtmac
wgtmac merged commit 60e67fa into main Oct 15, 2026
16 of 17 checks passed
@wgtmac
wgtmac deleted the codex/bucket-index-oversized-guard branch October 15, 2026 09:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants