Skip to content

Render decimal human strings in plain notation - #4430

Open
AP24110011695 wants to merge 2 commits into
apache:mainfrom
AP24110011695:fix/decimal-human-plain-notation
Open

AP24110011695 wants to merge 2 commits into
apache:mainfrom
AP24110011695:fix/decimal-human-plain-notation

Conversation

@AP24110011695

Copy link
Copy Markdown
Contributor

What's changed

to_human_string for decimals falls through to plain str(), which renders small magnitudes in scientific notation — a decimal partition value of 0.00000012 surfaces as 1.2E-7 in the partitions inspect table's lower/upper bound columns (_partition_summaries_to_rows renders bounds via field.transform.to_human_string), while #4420 established plain notation for decimal JSON values with the exact same str() hazard.

This registers a Decimal override on the human-string helper using the same rule as the JSON path: plain f-notation when the exponent is zero or negative, str() otherwise (which keeps the negative-scale scientific convention the conversions docstring documents). The human-string helper is what the identity transform and the inspect path route through.

Test

Four new parametrize cases in test_identity_human_string (small magnitude, zero, negative small magnitude, and a 38,18 case) — verified failing-first on main (1.2E-7/0E-8 renderings) and passing with the override. The 17 pyarrow-parametrized failures in this file are the pristine-main environment baseline on this machine (verified by stash A/B: identical set, unchanged by this PR).

AI disclosure: drafted with AI assistance, verified locally by running the failing-first cycle and an end-to-end write + inspect.partitions() probe.

to_human_string for decimals falls through to str(), which renders
small magnitudes in scientific notation — a decimal partition value of
0.00000012 shows up as 1.2E-7 in the partitions inspect table's lower
and upper bound columns, while apache#4420 established plain notation for
decimal JSON values.

Register a Decimal override on the human-string helper using the same
rule as the JSON path (plain f-notation when the exponent is zero or
negative, str otherwise, which keeps the negative-scale scientific
convention). The four new parametrize cases fail on main and pass with
the override; the 17 pyarrow-parametrized failures in this file are the
pristine-main environment baseline, unchanged.

Refs apache#4420.

Signed-off-by: Ayush Kumar Saha <225407166+AP24110011695@users.noreply.github.com>
Decimal.as_tuple's exponent element is the string 'n'/'N'/'F' for NaN
and Infinity specials, so mypy rejects comparing it against int
directly. Restore the isinstance narrowing the conversions.py
counterpart uses (the one-line compression dropped it).

Signed-off-by: Ayush Kumar Saha <225407166+AP24110011695@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant