Description
Describe the bug, including details regarding any error messages, version, and platform.
A decimal type's scale can be any int32_t value (DecimalType::Make only validates the precision), and the scale of a decimal string comes from its exponent. Several places turn such scales into a scale change and use it to index the fixed-size tables of powers of ten behind IncreaseScaleBy, ReduceScaleBy, GetWholeAndFraction and the floating-point PowerOfTen. These only accept [0, kMaxScale] ([-76, 76] for the floating-point tables) and only check it with a DCHECK, so release builds read out of bounds:
- truncating casts (
allow_decimal_truncate) from decimal to integer, from decimal to decimal and from string to decimal, e.g. casting"1E-39"todecimal128(38, 0),decimal128(38, 0)todecimal128(38, 39)ordecimal128(38, 39)toint64. Castingdecimal128(38, 20)todecimal32(9, 0)reads past theDecimal32table as well, because the value is narrowed before it is rescaled (GH-53288: [C++] Rescale and range-check before narrowing decimals to Decimal32 #53327); productandhash_producton a decimal type with a negative scale or a scale abovekMaxScale(for instanceproductofdecimal128(5, -2)values returns a wrong result);Decimal32/64/128/256::ToReal, i.e. casting decimal to float or double,variance,stddev,skew,kurtosis,tdigest,quantileand their hash versions, when the scale is abovekMaxScaleand the value is too large to be converted exactly;Decimal32/64/128/256::FromReal, i.e. casting float or double to decimal, when the scale is far outside[-kMaxScale, kMaxScale].
With pyarrow 26.0.0, pc.cast(pa.array(["1E-100000000"]), pa.decimal128(38, 0), safe=False) segfaults. In an ASan/UBSan build each case above reports a global-buffer-overflow on kDecimal128PowersOfTen (or the matching table) or index ... out of bounds for type 'const BasicDecimal128[39]'.
Expected: the result where it is well defined (a truncating downscale by more than kMaxScale gives zero), an error otherwise, and no out-of-bounds read.
Decimal::Rescale has the same problem for a scale change above kMaxScale; that one is #50964.
Seen on main (b8079cd).
Component(s)
C++
Activity
apacheGH-55340: [C++] Check out-of-range decimal scales before indexi…