Skip to content

GH-40576: [Python][Docs] Clarify Dataset scan batch sizes - #55581

Open
kalyanamdewri wants to merge 1 commit into
apache:mainfrom
kalyanamdewri:gh-40576-dataset-batch-size-docs
Open

kalyanamdewri wants to merge 1 commit into
apache:mainfrom
kalyanamdewri:gh-40576-dataset-batch-size-docs

Conversation

@kalyanamdewri

@kalyanamdewri kalyanamdewri commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary:

  • Correct the default Dataset scan batch size to 131,072 rows and distinguish it from the Parquet writer's default row group size.
  • Add a runnable example showing that a larger scan batch size does not combine Parquet row groups.

Validation:

  • python -m pytest --doctest-glob='*.rst' docs/source/python/dataset.rst -q — 1 passed with PyArrow 26.0.1
  • git diff --check — passed

Was AI used for this PR?
AI assisted with the documentation patch and draft description. I reviewed the diff and ran the checks locally.

GitHub Issue: #40576

@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #40576 has been automatically assigned in GitHub to PR creator.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants