Repository navigation
Round float and double literals to decimal on the shortest repr - #4442
Conversation
Fokko
left a comment
There was a problem hiding this comment.
Thanks @Ankitraj-builds! Verified locally: literal(2.675).to(DecimalType(10, 2)) gives 2.67 on main and 2.68 with this change, and the new parametrized tests (10 cases) pass. Using Decimal(str(value)) is indeed what BigDecimal.valueOf(double) does in Java, so this makes the two behave the same.
One thing to think about: Decimal(str(value)) and Java's BigDecimal.valueOf both rely on the shortest repr produced by the language's float-to-string conversion, so they agree in practice. That is good enough.
LGTM!
|
Thanks @Fokko for verifying it locally and for the review! If it looks good, could you approve it so it can be merged? |
Fokko
left a comment
There was a problem hiding this comment.
@Ankitraj-builds Yes, let's push this forward.
Closes #4441
Rationale for this change
FloatLiteral.to(DecimalType)andDoubleLiteral.to(DecimalType)usedDecimal(self.value), which takes the exact binary value of the float. For values like 2.675 that value is 2.67499999..., soROUND_HALF_UProunded down:literal(2.675).to(DecimalType(10, 2))gave 2.67 instead of 2.68. This also changed the bound literal of filters on decimal columns, e.g.EqualTo("price", 2.675)bound to 2.67.This now uses
Decimal(str(self.value)), the shortest decimal that round-trips to the float. That matches Java, which usesBigDecimal.valueOf(value).setScale(scale, RoundingMode.HALF_UP).Are these changes tested?
Yes.
test_float_and_double_to_decimal_round_half_up_on_shortest_reprintests/expressions/test_literals.pycovers 2.675, 1.005 and -2.675 (which round the wrong way on main), plus 0.125 and 2.674 as controls, for bothFloatLiteralandDoubleLiteral. The three wrong cases fail on main and pass with this change. All tests intests/expressionsandtests/test_conversions.pypass.Are there any user-facing changes?
Yes. Float and double values converted to a decimal type (for example in filters on decimal columns) now round like Java, Spark and Postgres: 2.675 becomes 2.68.