You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Python] Bug in pyarrow.from_pandas() when input has MultiIndex index columns having non-string names #38983
Describe the bug, including details regarding any error messages, version, and platform.
When the input dataframe has MultiIndex index columns having non-string names, the output is not as expected:
import pyarrow as pa
print(f"version: {pa.__version__}")
import pandas as pd
index = pd.MultiIndex.from_tuples([(10,20), (30,40), (50,60), (70,80)], names = [1,2])
df = pd.DataFrame({"a": [1,2,3,4], "b": [5,6,7,8]}, index=index)
print(df)
print(pa.Table.from_pandas(df))
Outputs:
version: 14.0.1
sys:1: UserWarning: The DataFrame has non-str index name `[1, 1]` which will be converted to string and not roundtrip correctly.
pyarrow.Table
a: int64
b: int64
1: int64
1: int64
----
a: [[1,2,3,4]]
b: [[5,6,7,8]]
1: [[10,30,50,70]]
1: [[10,30,50,70]]
def _get_index_level_values(index):
n = len(getattr(index, 'levels', [index]))
if isinstance(index, _pandas_api.pd.MultiIndex):
return [index.levels[i] for i in range(n)]
else:
return [index.get_level_values(i) for i in range(n)]
changed the title [-]Bug in pyarrow.from_pandas() when input has MultiIndex index columns having non-string names[/-][+][Python] Bug in pyarrow.from_pandas() when input has MultiIndex index columns having non-string names[/+]on Nov 10, 2026
Activity
qsourav commentedon Nov 29, 2023
The same seems to be fixed from the below workaround:
arrow/python/pyarrow/pandas_compat.py
Line 511 in 1d904d6
qsourav commentedon Nov 29, 2023
Some issues with the "levels" parameter when the index columns have non-string names. Even names = ["1", 2] works fine!
qsourav commentedon Nov 29, 2023
might be related to:
pandas-dev/pandas#16160
pandas-dev/pandas#2770
[-]Bug in pyarrow.from_pandas() when input has MultiIndex index columns having non-string names[/-][+][Python] Bug in pyarrow.from_pandas() when input has MultiIndex index columns having non-string names[/+]GH-38983: [Python] Bug in pyarrow.from_pandas() when input has MultiI…
AlenkaF commentedon Nov 12, 2026
Issue resolved by pull request 55485
#55485