Skip to content

fix(python): Handle empty list chunks during iteration - #1021

Merged
paleolimbot merged 1 commit into
apache:mainfrom
peterolkhov:codex/fix-python-empty-list-chunks
Nov 16, 2026
Merged

paleolimbot merged 1 commit into
apache:mainfrom
peterolkhov:codex/fix-python-empty-list-chunks

Conversation

@peterolkhov

Copy link
Copy Markdown
Contributor

AI assistance: Codex generated the guard and regression tests and ran the local checks below.

iter_py() raises IndexError on an empty LIST or LARGE_LIST chunk because _list_iter() indexes empty start/end slices. A zero-row record batch with a list column also aborts direct stream iteration before later rows are returned.

Return immediately for a zero-length list slice, matching _map_iter(). The regression tests cover both list offset widths, empty arrays and slices, and a real RecordBatchReader with empty batches before, between and after populated batches.

Validation on macOS arm64 / Python 3.13.14 using a rebuilt and reinstalled Meson/Cython wheel:

  • New regression cases fail on the unchanged implementation and pass with the guard.
  • Python suite: 288 passed, 4 skipped.
  • Package doctests: 118 passed; compiled-module stubtests: all 8 passed.
  • Black 22.3.0, Flake8 7.1.1 and isort 5.12.0 pass for both changed files.

@paleolimbot paleolimbot left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@paleolimbot
paleolimbot merged commit 057e253 into apache:main Nov 16, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants