Skip to content

Float/Double literal to Decimal rounds on the exact binary value instead of the shortest decimal repr #4441

Closed

Description

@Shahzeb-DataEngineer

What happens?

FloatLiteral.to(DecimalType) and DoubleLiteral.to(DecimalType) convert the float with Decimal(self.value), which uses the exact binary expansion of the double. Values whose decimal spelling ends in 5 then round the wrong way under ROUND_HALF_UP:

from pyiceberg.expressions.literals import literal
from pyiceberg.types import DecimalType

literal(2.675).to(DecimalType(10, 2))   # 2.67, expected 2.68
literal(1.005).to(DecimalType(10, 2))   # 1.00, expected 1.01
literal(-2.675).to(DecimalType(10, 2))  # -2.67, expected -2.68

The wrong value leaks into predicate binding, so a filter on a decimal column uses the wrong literal:

from pyiceberg.expressions import EqualTo
from pyiceberg.schema import Schema
from pyiceberg.types import NestedField, DecimalType

schema = Schema(NestedField(1, "price", DecimalType(10, 2), required=True))
EqualTo("price", 2.675).bind(schema, case_sensitive=True).literal  # 2.67, expected 2.68

Expected behavior

Match Apache Iceberg Java, which converts with BigDecimal.valueOf(value).setScale(scale, RoundingMode.HALF_UP) in Literals.java (all four numeric literal types). BigDecimal.valueOf(double) uses Double.toString, i.e. the shortest decimal that round-trips to the double, so 2.675 quantizes to 2.68. Spark and Postgres agree (2.675 casts to 2.68).

Root cause

pyiceberg/expressions/literals.py, FloatLiteral.to and DoubleLiteral.to for DecimalType:

Decimal(self.value).quantize(Decimal((0, (1,), -type_var.scale)), rounding=ROUND_HALF_UP)

Decimal(2.675) is 2.67499999999999982..., which quantizes down to 2.67. Using Decimal(str(self.value)) instead gives Decimal("2.675"), which quantizes to 2.68, matching BigDecimal.valueOf.

Environment

pyiceberg 0.12.0, Python 3.12, Linux.

Happy to open the fix PR.

Activity

  1. Ankitraj-builds commented on Nov 20, 2026

    @Ankitraj-builds
    Contributor

    Hi @Shahzeb-DataEngineer, I have a fix ready for this (using Decimal(str(value)) in both FloatLiteral and DoubleLiteral, with regression tests). I’ll open a PR, unless you’ve already started one

  2. Ankitraj-builds commented on Nov 20, 2026

    @Ankitraj-builds
    Contributor

    Opened #4442 for this. @Fokko could you take a look when you have time?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions