Skip to content

GH-49194: [Python][Annotations] Compute module - #49201

Draft
rok wants to merge 134 commits into
apache:mainfrom
rok:pyarrow-stubs-pr4-compute
Draft

rok wants to merge 134 commits into
apache:mainfrom
rok:pyarrow-stubs-pr4-compute

Conversation

@rok

@rok rok commented Feb 9, 2026 •

Copy link
Copy Markdown
Member

Rationale for this change

Closes: #49194

What changes are included in this PR?

Adds type annotation stubs for compute, _compute, _compute_docstring, and _acero modules. Includes source fixes in compute.py and acero.py, and type-ignore annotations in related tests.

Are these changes tested?

Yes, by CI.

Are there any user-facing changes?

Compute module functions will be type annotated.

@github-actions

github-actions Bot commented Feb 9, 2026

Copy link
Copy Markdown

⚠️ GitHub issue #49194 has been automatically assigned in GitHub to PR creator.



filter = array_filter
take = array_take

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: redefine take rather than alias to exclude Expression

rok and others added 24 commits November 10, 2026 00:32
Co-authored-by: Dan Redding <125183946+dangotbanned@users.noreply.github.com>
Co-authored-by: Dan Redding <125183946+dangotbanned@users.noreply.github.com>
Co-authored-by: Dan Redding <125183946+dangotbanned@users.noreply.github.com>
Co-authored-by: Dan Redding <125183946+dangotbanned@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Add type annotation stubs for array, table, tensor, builder, memory,
device, config, and types modules. Includes type-ignore annotations in related tests.
Drop the TypeVar/Generic parameters from the DataType subclasses,
Field, Array, ChunkedArray and the tabular classes. Where a property
can only return one of a few types (dictionary index type, run end
type, time units) it now returns a union or Literal instead.

Most of these parameters weren't buying much yet: scalar.pyi and
lib.pyi are still placeholders, so e.g. Array[Scalar[...]] collapsed
to Any anyway, and several annotations were being flagged by mypy and
pyright (TypeVars used only once, type arguments on subclasses that
weren't actually generic). Plain classes are a lot easier to read and
review, and we can bring parameters back later where they clearly
pay off.

This also removes the stub-only _BasicDataType and _HasPrecisionScale
helpers, which have no runtime counterpart, plus the IntoArray and
SingleOrList aliases.
@rok

rok commented Nov 17, 2026

Copy link
Copy Markdown
Member Author

Given there were changes in the data types PR you might want to rebase or merge them here @AlenkaF.

tadeja and others added 4 commits November 17, 2026 22:39
Annotate pyarrow.compute, pyarrow._compute, pyarrow._acero and
pyarrow._compute_docstrings, following the approach of apacheGH-55264: plain
classes and unions instead of generics, in the flat pyarrow-stubs/
layout.

- Array arguments accept any Array or ChunkedArray, since pa.array()
  and most of the API return plain Array instances. The return types
  name the kind of array where the function determines it. The scalar
  classes are placeholders (Any) until scalar.pyi is annotated.
- The signatures match the runtime, as checked by stubtest.
- pyarrow.compute and pyarrow._compute are type checked instead of
  being treated as Any.
- Add type annotation tests for the compute and Acero stubs.
@AlenkaF
AlenkaF force-pushed the pyarrow-stubs-pr4-compute branch from e8779e3 to 300602b Compare November 18, 2026 05:21
@AlenkaF

AlenkaF commented Nov 18, 2026

Copy link
Copy Markdown
Member

Oh, should have squashed, sorry.

@@ -0,0 +1,18 @@
# Licensed to the Apache Software Foundation (ASF) under one

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this file needed or we can remove it?

from pyarrow import lib

ArrayLike: TypeAlias = Any
ScalarLike: TypeAlias = Any

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to myself: when #55592 is merged, update the ScalarLike alias and make sure to narrow return types that use this alias.

@@ -25,6 +25,7 @@ from pyarrow.lib import Array, ChunkedArray
from pyarrow import lib

ArrayLike: TypeAlias = Any

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to myself: define a separate private _ArrayLike type alias for compute stubs only (after comment #55264 (comment) is resolved).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Python][Annotations] Compute module

4 participants