You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
get_athena_type maps Arrow decimal256 to string #888
types.Decimal256Type is the type class, not the type id types.Type_DECIMAL256, so the comparison never matches a decimal256 field. The field falls through to the final return "string", 2147483647, 0.
to_column_info uses this mapping to build the cursor description from a Parquet schema, as with the unload option of the Arrow and pandas cursors.
Impact is limited in practice. Athena's DECIMAL precision is at most 38, which fits decimal128, so Athena's own UNLOAD output should not contain decimal256. Other Parquet schemas passed to to_column_info can.
PyAthena master (a86a180); the same code is in v3.36.0.
pyarrow 25.0.1, Python 3.13.
Proposed fix (optional)
Compare with types.Type_DECIMAL256, and add a unit test for decimal256 next to the existing get_athena_type / to_column_info tests. No AWS resources are needed.
PR #900 fixes this. It also maps Arrow decimal32 and decimal64 (pyarrow >= 19.0.0, covered by the current pyarrow>=22.0.0 floor), which fell through to string in the same way. Parquet written by pyarrow keeps those widths when it is read back.
Activity
laughingman7743 commentedon Sep 29, 2026
PR #900 fixes this. It also maps Arrow
decimal32anddecimal64(pyarrow >= 19.0.0, covered by the currentpyarrow>=22.0.0floor), which fell through tostringin the same way. Parquet written by pyarrow keeps those widths when it is read back.