Skip to content

Commit c44fc29

Browse files
committed
fix(deletion-vector): reject a bitmap count larger than the payload
_deserialize_bitmap read an 8-byte count and used it directly as a loop bound, so a small blob could declare a large number of bitmaps. Reading past the payload surfaced as an IndexError from the native bitmap decoder. Validate the count against the remaining bytes, since every bitmap contributes at least a 4-byte key.
1 parent 1312c55 commit c44fc29

2 files changed

Lines changed: 14 additions & 0 deletions

File tree

‎pyiceberg/table/deletion_vector.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,11 @@ def _deserialize_bitmap(pl: bytes) -> list[BitMap]:
4242
number_of_bitmaps = int.from_bytes(pl[0:8], byteorder="little")
4343
pl = pl[8:]
4444

45+
# Every bitmap contributes at least a 4-byte key, so a count that cannot fit
46+
# in the remaining payload is invalid and must not be used as a loop bound.
47+
if number_of_bitmaps * 4 > len(pl):
48+
raise ValueError(f"Payload declares {number_of_bitmaps} bitmaps, but only holds {len(pl)} bytes")
49+
4550
bitmaps = []
4651
last_key = -1
4752
for _ in range(number_of_bitmaps):

‎tests/table/test_deletion_vector.py‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,15 @@ def test_map_spread_vals() -> None:
6666
assert expected == actual
6767

6868

69+
def test_map_declared_count_exceeds_payload() -> None:
70+
# A truncated payload that claims a large number of bitmaps must be rejected,
71+
# rather than driving the deserialization loop on data that is not there.
72+
puffin = (2**32).to_bytes(8, byteorder="little") + b"\x00\x00\x00\x00"
73+
74+
with pytest.raises(ValueError, match="Payload declares 4294967296 bitmaps, but only holds 4 bytes"):
75+
_ = DeletionVector._deserialize_bitmap(puffin)
76+
77+
6978
def test_map_high_vals() -> None:
7079
puffin = _open_file("64maphighvals.bin")
7180

0 commit comments

Comments
 (0)