add support for abi3t by Icxolu · Pull Request #556 · PyO3/rust-numpy

Icxolu · 2026-06-15T21:49:58Z

This adds abi3t support by

switching to opaque structs
exposing data access helpers from slots 369-378
redirect all field accesses through the access helpers

Todo:

changelog entry
CI testing for 3.15/3.15t

See numpy/numpy#31091
Closes #555

ngoldbaum · 2026-06-16T21:15:12Z

Ping @kumaraditya303 - we were just talking about this today.

neutrinoceros · 2026-06-17T05:24:40Z

I confirmed that this resolved the runtime error I reported in #555
example logs (remaining failures are unrelated, they're just due to numpy not having cp315 nightlies for windows yet)

kumaraditya303 · 2026-06-18T09:15:31Z

I think we should also add a check that when abi3t is used, the minimum numpy version should be 2.5.

Icxolu · 2026-06-20T09:51:07Z

Looks like 32bit windows still has some issues... not sure why though
There was also https://github.com/PyO3/rust-numpy/actions/runs/27866655836/job/82471861978 earlier

neutrinoceros · 2026-06-20T10:18:07Z

My guess is that'd be because there are no wheels for cp315*-*win32
xref numpy/numpy-release#53

Icxolu · 2026-06-20T10:32:53Z

To me it looks like there are cp315-win32 wheels here and they installed fine. The version specific run had no issues as well. It's just the abi3t run that failed, so I think this is a genuine failure, but I'm not sure which fault it could be: ours, pyo3, numpy or python

kumaraditya303 · 2026-06-20T10:54:38Z

I think it could alignment issue on 32 bit MSVC, I am not very familiar with rust but it seems flags is defined as npy_uint64 which rust gives 8 bytes alignment whereas in C I think it is 4 bytes which could be an issue.

kumaraditya303 · 2026-06-20T11:11:12Z

@Icxolu Can you try forcing a 4 byte alignment layout using #[cfg_attr(all(Py_LIMITED_API, Py_GIL_DISABLED), repr(C, packed(4)))]?

ngoldbaum · 2026-06-20T11:29:26Z

If you all don't sort this out in the meantime I'll try to look on Monday. While win32 isn't so important per se, I do find it useful to detect and sort out CPU architecture-sensitive issues. It's possible this issue is in any one of CPython, NumPy, or rust-numpy.

Glad to see everything mostly works otherwise!

Icxolu · 2026-06-20T14:12:04Z

I did a bit of digging with compiler explorer: https://godbolt.org/z/qYe3Ghz7a
Both rustc and msvc agree on both architectures on the structure size and that flags is at offset 16 (and thus 8 byte aligned as it should), so I think there is no issue with our definitions here. I did however also find this: rust-lang/rust#112480, which does not really paint a great picture about win32... perhaps we ran into that...

ngoldbaum · 2026-06-25T19:30:13Z

I think I know what the issue is.

PyArray_Descr_fields has a field that causes alignment issues on 32bit builds:

https://github.com/numpy/numpy/blob/1d656e454dea882b2f2a20ccdd93b02f3c7880c1/numpy/_core/include/numpy/ndarraytypes.h#L650

It's explicitly npy_uint64 - 8 bytes. In this case, it happens to lead to a field access with incorrect alignment and you get a segfault.

I think the fix needs to live in NumPy's headers. There probably needs to be a mirrored change here to ensure correct alignment for the flags field in PyArray_Descr_fields and _PyArray_LegacyDescr_fields. I'll have a PR open on NumPy shortly.

ngoldbaum · 2026-06-25T20:45:59Z

I opened numpy/numpy#31759

ngoldbaum · 2026-06-25T22:04:22Z

It turns out my first attempt at a fix had a flaw and I think the only real fix requires NumPy to add new a C API function to fix this issue: numpy/numpy#31762

Icxolu · 2026-06-26T11:50:16Z

Thank you both for continuing to look into this, much appreciated ❤️. That patch seems to make it run indeed. What I don't really understand is, why this should work. If I got it correctly, then from numpy's definition there are padding bytes inserted to align the flags field on 32bit. If we now tell Rust to use a packed layout, wouldn't we read the padding bytes instead of the field? I'd also assume that this than effects all 32bit platforms, not just windows and not just x86, we just don't have any CI configured for other cases.

ngoldbaum · 2026-06-26T11:52:25Z

I think we need a similar fix on the NumPy side too.

This does affect all 32 bit platforms, I think.

neutrinoceros · 2026-08-26T12:56:59Z

If I may ask, what's the expected timeline for this to land into a release ?

Icxolu · 2026-08-26T13:28:51Z

Whenever the next pyo3 (0.30) releases, I'll prepare a new release which includes this.

ngoldbaum · 2026-08-26T16:39:23Z

Whenever the next pyo3 (0.30) releases, I'll prepare a new release which includes this.

I'm in the monthly PyO3 community call with David and Lucas Colley and the topic of shipping 0.30 came up. He's fine with it but is curious why you can't do a rust-numpy 0.29.1 release.

Icxolu · 2026-08-26T17:20:49Z

The restructuring of the public npyffi to allow the abi3t compatible opaque layout is a breaking api change.

davidhewitt · 2026-08-26T17:32:39Z

In which case let's move forward with PyO3 0.30 👍

rust-numpy 0.29.0 reads dtype fields through struct layouts that assume a visible PyObject header. The abi3t build makes PyObject opaque, so datetime64 arrays crash with a null reference on 3.15. rust-numpy main fixes this (PyO3/rust-numpy#556) but the fix is not released yet. Until it is, build a cp315t wheel; 3.15 GIL builds use the cp312-abi3 wheel. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… 3.15 (#209) * Build abi3/abi3t wheels and add free-threaded and Python 3.15 support Bump pyo3 to 0.29 and rust-numpy to 0.29 for Python 3.15 support, and enable the abi3-py312 and abi3t-py315 features. Each platform now ships one cp312-abi3 wheel for GIL builds 3.12+, one cp314t wheel, one cp315-abi3t wheel for 3.15+ GIL and free-threaded builds, and PyPy. The test job runs on 3.12 through 3.15t and checks the GIL stays off. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Build a version-specific cp315t wheel instead of abi3t rust-numpy 0.29.0 reads dtype fields through struct layouts that assume a visible PyObject header. The abi3t build makes PyObject opaque, so datetime64 arrays crash with a null reference on 3.15. rust-numpy main fixes this (PyO3/rust-numpy#556) but the fix is not released yet. Until it is, build a cp315t wheel; 3.15 GIL builds use the cp312-abi3 wheel. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

The free-threaded CI checks, pixi envs and Nix shells now use 3.15t. The release workflow builds a 3.15t wheel and keeps the 3.14t wheel. The 3.15t wheel is abi3t (PEP 803). rust-numpy 0.29.0 reads array fields at fixed offsets, and these offsets are wrong under abi3t. The numpy crate is pinned to the git commit that has PyO3/rust-numpy#556 until a crates.io release has it. pyo3 gives simple enums a default `__repr__`. A custom `__repr__` adds a second Py_tp_repr slot, and 3.15 rejects the type (PyO3/pyo3#6494). This change removes the custom `__repr__` from SpecVersion and ConflictType. netcdf4 has no 3.15 build, so it moves to a feature that only the default env uses.

Add `-i 3.15t` to the version-specific wheel builds on every native platform, so we publish `cp315-cp315t` wheels alongside the existing `cp314-cp314t` ones. Bump the native jobs' uv pin to 0.13.x so that `uv python install 3.15t` resolves the final 3.15 release. We are not building `abi3t` wheels yet: rust-numpy 0.29 has the wrong struct layout under abi3t and segfaults in numpy interop (PyO3/rust-numpy#555). The fix (PyO3/rust-numpy#556) will only ship alongside PyO3 0.30. Co-authored-by: Shaan Majid <70789625+shaanmajid@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

) * build(mise): bump uv to 0.13.1 for CPython 3.15.0 uv 0.12.23 knows CPython 3.15 only up to 3.15.0rc3; 0.13.0 adds 3.15.0 final (and makes it the default stable version). * feat(python)!: abi3t wheels for CPython 3.15, require CPython 3.12 Three wheels per platform, each built with its family's oldest CPython: cp312-abi3 (GIL-enabled 3.12 to 3.14), cp315-abi3.abi3t (PEP 803: both builds of 3.15 and newer, which uv and pip prefer there), and cp314-cp314t (free-threaded 3.14 has no stable ABI). PyO3's abi3-py312 and abi3t-py315 features let the build interpreter pick the extension. - rust-numpy comes from a pinned git revision: its abi3t support (PyO3/rust-numpy#556) is not in a release yet. The abi3t extension reads arrays through NumPy 2.5's C API, so CPython 3.15 and newer require numpy>=2.5. - requires-python >=3.12: numpy 2.5 and SciPy 1.18 already dropped 3.11, and 3.12 is the most-used CPython. ruff and ty target 3.12, and the type aliases become PEP 695 `type` statements. - CI builds the abi3 and abi3t wheels on every platform (cp314t on x86_64 Linux) and tests them on 3.12 and 3.14, 3.15 and 3.15t (3.14t); publish.yml builds and tests all three, and the CUDA runtime's, on every platform. The tests name each version, since uv's "3" also matches an installed free-threaded interpreter. BREAKING CHANGE: CPython 3.11 is no longer supported; installers there keep resolving hessboost 0.3.1. * ci: test the GIL-enabled CPython an unsuffixed version names For CPython 3.14 and newer uv takes an installed free-threaded interpreter for a plain version request, and keeps a free-threaded project environment for one. publish.yml therefore tested the abi3 wheels against 3.14t, which refuses them. ci.yml tested its "3.15" runs on the 3.15t build interpreter. test-wheel.sh now installs the requested version (uv downloads the GIL-enabled build for a plain version) and asks for it as "+gil".

numpy 0.30 ships the abi3t fix from PyO3/rust-numpy#556, which we need before we can publish abi3t wheels (#532). pyo3-arrow builds against both without source changes. arro3 itself can't move to pyo3 0.30 yet, because arro3-io needs pyo3 0.30 releases of pyo3-async-runtimes, pyo3-bytes, pyo3-object_store, and pyo3-file. As described in DEVELOP.md, pin the arro3 crates to pyo3-arrow at the current main commit until then. The root Cargo.lock also picks up thiserror 2 for pyo3-arrow, which #539 changed without updating that lockfile. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

numpy 0.30.0 has the abi3t support from PyO3/rust-numpy#556, so the git pin is replaced with the crates.io release. numpy 0.30 needs pyo3 0.30, so pyo3, pyo3-async-runtimes and pyo3-bytes move to their 0.30 releases. pyo3 0.30 makes `skip_from_py_object` the default and deprecates the option, so this change removes it from all `#[pyclass]` attributes.

The free-threaded CI checks, pixi envs and Nix shells now use 3.15t. The release workflow builds a 3.15t wheel and keeps the 3.14t wheel. The 3.15t wheel is abi3t (PEP 803). rust-numpy 0.29.0 reads array fields at fixed offsets, and these offsets are wrong under abi3t. The numpy crate is pinned to the git commit that has PyO3/rust-numpy#556 until a crates.io release has it. pyo3 gives simple enums a default `__repr__`. A custom `__repr__` adds a second Py_tp_repr slot, and 3.15 rejects the type (PyO3/pyo3#6494). This change removes the custom `__repr__` from SpecVersion and ConflictType. netcdf4 has no 3.15 build, so it moves to a feature that only the default env uses.

numpy 0.30.0 has the abi3t support from PyO3/rust-numpy#556, so the git pin is replaced with the crates.io release. numpy 0.30 needs pyo3 0.30, so pyo3, pyo3-async-runtimes and pyo3-bytes move to their 0.30 releases. pyo3 0.30 makes `skip_from_py_object` the default and deprecates the option, so this change removes it from all `#[pyclass]` attributes.

The free-threaded CI checks, pixi envs and Nix shells now use 3.15t. The release workflow builds a 3.15t wheel and keeps the 3.14t wheel. The 3.15t wheel is abi3t (PEP 803). rust-numpy 0.29.0 reads array fields at fixed offsets, and these offsets are wrong under abi3t. The numpy crate is pinned to the git commit that has PyO3/rust-numpy#556 until a crates.io release has it. pyo3 gives simple enums a default `__repr__`. A custom `__repr__` adds a second Py_tp_repr slot, and 3.15 rejects the type (PyO3/pyo3#6494). This change removes the custom `__repr__` from SpecVersion and ConflictType. netcdf4 has no 3.15 build, so it moves to a feature that only the default env uses.

numpy 0.30.0 has the abi3t support from PyO3/rust-numpy#556, so the git pin is replaced with the crates.io release. numpy 0.30 needs pyo3 0.30, so pyo3, pyo3-async-runtimes and pyo3-bytes move to their 0.30 releases. pyo3 0.30 makes `skip_from_py_object` the default and deprecates the option, so this change removes it from all `#[pyclass]` attributes.

…ls (earth-mover#2472) The free-threaded CI checks, pixi envs and Nix shells now use 3.15t. The release workflow builds a 3.15t wheel and keeps the 3.14t wheel. The 3.15t wheel is abi3t (PEP 803). rust-numpy 0.30.0 includes PyO3/rust-numpy#556 fixing behavior for 3.15t, but brings a PyO3 bump to 0.30.1 too. pyo3 gives simple enums a default `__repr__`. A custom `__repr__` adds a second Py_tp_repr slot, and 3.15 rejects the type (PyO3/pyo3#6494). This change removes the custom `__repr__` from SpecVersion and ConflictType. netcdf4 has no 3.15 build, so it moves to a feature that only the default env uses.

Icxolu force-pushed the abi3t branch from e086d8e to 97d7b8d Compare June 16, 2026 20:38

Icxolu mentioned this pull request Jun 16, 2026

BUG: function built with -F pyo3/abi3t raises TypeError at runtime (CPython 3.15.0b2-free-threaded)) #555

Closed

kumaraditya303 reviewed Jun 17, 2026

View reviewed changes

Comment thread src/npyffi/array.rs Outdated

kumaraditya303 reviewed Jun 17, 2026

View reviewed changes

Comment thread src/npyffi/array.rs Outdated

kumaraditya303 reviewed Jun 17, 2026

View reviewed changes

Comment thread src/npyffi/array.rs Outdated

Icxolu force-pushed the abi3t branch from 97d7b8d to 31e2df5 Compare June 17, 2026 19:03

kumaraditya303 reviewed Jun 18, 2026

View reviewed changes

Comment thread src/npyffi/objects.rs

Icxolu force-pushed the abi3t branch 4 times, most recently from cfef0a3 to 23f2955 Compare June 20, 2026 09:28

ngoldbaum mentioned this pull request Jun 25, 2026

BUG: aligned access for the descriptor fields struct on 32bit abi3t numpy/numpy#31759

Closed

ngoldbaum mentioned this pull request Jun 25, 2026

BUG: descriptor accessors should account for struct padding on 32bit abi3t builds numpy/numpy#31762

Closed

This comment was marked as outdated.

Sign in to view

add changelog

2b01089

Icxolu force-pushed the abi3t branch from fe4addd to 2b01089 Compare August 22, 2026 12:27

Icxolu merged commit 9df4023 into PyO3:main Aug 23, 2026
138 of 140 checks passed

Icxolu deleted the abi3t branch August 23, 2026 16:44

This was referenced Oct 3, 2026

Build abi3t wheels for Python 3.15 support huggingface/tokenizers#2488

Open

0.30 release PyO3/pyo3#6126

Open

samantha-earthmover mentioned this pull request Oct 9, 2026

build(python): abi3 wheels, plus free-threaded 3.14t/3.15t and Python 3.15 mpiannucci/gribberish#209

Merged

li-em mentioned this pull request Oct 10, 2026

Use Python 3.15t for free-threaded CI and add abi3t to release wheels earth-mover/icechunk#2472

Merged

kylebarron mentioned this pull request Oct 12, 2026

Publish abi3t wheels for free-threaded Python once PyO3 0.30 and rust-numpy support it kylebarron/arro3#532

Open

4 tasks

kylebarron mentioned this pull request Oct 12, 2026

ci: Build abi3t wheels for free-threaded Python 3.15 kylebarron/arro3#530

Closed

brndnmtthws mentioned this pull request Oct 14, 2026

feat(python)!: abi3t wheels for CPython 3.15, require CPython 3.12 brndnmtthws/hessboost#162

Merged

kylebarron mentioned this pull request Oct 15, 2026

chore(pyo3-arrow): bump pyo3 and numpy to 0.30 kylebarron/arro3#544

Merged

kylebarron mentioned this pull request Oct 15, 2026

ci: Skip 3.14t wheels on 32-bit Linux developmentseed/obstore#800

Merged

Conversation

Icxolu commented Jun 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ngoldbaum commented Jun 16, 2026

Uh oh!

neutrinoceros commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kumaraditya303 commented Jun 18, 2026

Uh oh!

Icxolu commented Jun 20, 2026

Uh oh!

neutrinoceros commented Jun 20, 2026

Uh oh!

Icxolu commented Jun 20, 2026

Uh oh!

kumaraditya303 commented Jun 20, 2026

Uh oh!

kumaraditya303 commented Jun 20, 2026

Uh oh!

ngoldbaum commented Jun 20, 2026

Uh oh!

Icxolu commented Jun 20, 2026

Uh oh!

ngoldbaum commented Jun 25, 2026

Uh oh!

ngoldbaum commented Jun 25, 2026

Uh oh!

ngoldbaum commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

Icxolu commented Jun 26, 2026

Uh oh!

ngoldbaum commented Jun 26, 2026

Uh oh!

Uh oh!

neutrinoceros commented Aug 26, 2026

Uh oh!

Icxolu commented Aug 26, 2026

Uh oh!

ngoldbaum commented Aug 26, 2026

Uh oh!

Icxolu commented Aug 26, 2026

Uh oh!

davidhewitt commented Aug 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Icxolu commented Jun 15, 2026 •

edited

Loading

neutrinoceros commented Jun 17, 2026 •

edited

Loading

ngoldbaum commented Jun 25, 2026 •

edited

Loading