You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[BUG]: cudf.read_parquet with columns=[] gives a shape-(0, 0) dataframe #24523
Environment overview (please complete the following information)
Environment details
In [13]: cudf.__version__
Out[13]: '26.12.00a342.post261009155743'
Additional context
The behavior of pandas depends on the runtime version of pyarrow. prior to apache/arrow#51151 (released in pyarrow 26) pandas gives (0, 0), and we match that. With pyarrow>=26, pandas gives (3, 0).
I think the pandas behavior was a bug (apache/arrow#51151 (comment)) and the new result of (3, 0) is correct, so that's what we should try to meet.
#24520 pinned pyarrow<26 for now. We can remove / bump that pin once this fixed.
Thanks @akhil9tiet . One thing that needs to be determined is how to handle pandas compatibility: do we try to match the version of pandas/pyarrow installed at runtime? Right now, we're leaning towards trying to match pandas/pyarrow.
So the expected result will depend on what version of arrow you have installed and it'll always match what pandas happens to give you.
I'm hitting this bug right now in the 26.10 hotfix. I don't want to add an upper bound pin on pyarrow<26.0.0 for a 26.10.1 hotfix release when no upper bound pin existed in the 26.10.0 release.
I'm going to file a proper fix for this targeting main. I will match the behavior of the installed PyArrow version and will backport it for the hotfix PR #24652.
Activity
akhil9tiet commentedon Oct 9, 2026
I'd like to work on this — please assign it to me. I'll open a PR shortly.
TomAugspurger commentedon Oct 9, 2026
Thanks @akhil9tiet . One thing that needs to be determined is how to handle pandas compatibility: do we try to match the version of pandas/pyarrow installed at runtime? Right now, we're leaning towards trying to match pandas/pyarrow.
So the expected result will depend on what version of arrow you have installed and it'll always match what pandas happens to give you.
vyasr commentedon Oct 15, 2026
Yes, let's stick with that plan.
bdice commentedon Oct 17, 2026
I'm hitting this bug right now in the 26.10 hotfix. I don't want to add an upper bound pin on
pyarrow<26.0.0for a 26.10.1 hotfix release when no upper bound pin existed in the 26.10.0 release.I'm going to file a proper fix for this targeting
main. I will match the behavior of the installed PyArrow version and will backport it for the hotfix PR #24652.Match PyArrow's row count for read_parquet with columns=[] (#24663)
Match PyArrow's row count for read_parquet with columns=[] (NVIDIA#24663
Match PyArrow's row count for read_parquet with columns=[] (#24663)