Skip to content

gh-158401: Implement Py_INCREF() as function call in stable ABI - #162120

Open
vstinner wants to merge 3 commits into
python:mainfrom
vstinner:limited_incref
Open

vstinner wants to merge 3 commits into
python:mainfrom
vstinner:limited_incref

Conversation

@vstinner

@vstinner vstinner commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

Py_INCREF(), Py_XINCREF(), Py_DECREF() and Py_XDECREF() are now implemented as function calls on all limited C API versions. Previously, they were only implemented as function calls on the limited C API version 3.12 or newer.

Previously, if a C extension was built with the limited C API version 3.11 or newer, Py_DECREF() was implemented as "op->ob_refcnt--". But Python 3.12 made Py_DECREF() more complicated to implement immortal objects in an efficient way. "op->ob_refcnt--" can convert an immortal object to a mortal object which causes various kinds of nasty bugs.

Py_INCREF(), Py_XINCREF(), Py_DECREF() and Py_XDECREF() are now
implemented as function calls on all limited C API versions.
Previously, they were only implemented as function calls on the
limited C API version 3.12 or newer.

Previously, if a C extension was built with the limited C API version
3.11 or newer, Py_DECREF() was implemented as "op->ob_refcnt--". But
Python 3.12 made Py_DECREF() more complicated to implement immortal
objects in an efficient way. "op->ob_refcnt--" can convert an
immortal object to a mortal object which causes various kinds of
nasty bugs.
@vstinner

Copy link
Copy Markdown
Member Author

I checked manually the change by looking at xx_modexec() assembly code on x86-64 of the xxlimited_35 extension (built with #define Py_LIMITED_API 0x03050000: target the limited C API version 3.5). The assembly code calls Py_IncRef() as expected.

C code:

static int
xx_modexec(PyObject *m)
{
    ...
    Py_INCREF(ErrorObject);
    ...
}

gdb:

$ gdb ./python
...
(gdb) run
>>> import xxlimited_35
>>> C^

Program received signal SIGINT, Interrupt.
(gdb) disassemble xx_modexec
Dump of assembler code for function xx_modexec:
...
   0x00007ffff7aa5477 <+79>:	call   0x7ffff7aa50d0 <Py_IncRef@plt>

@vstinner

Copy link
Copy Markdown
Member Author

Ah, the change also replaces #if Py_LIMITED_API+0 >= 0x030a00A7 with #if Py_LIMITED_API+0 >= _Py_PACK_VERSION(3, 10). Usually, the limited C API version is defined as 0x03<hexversion>0000: lower bits are set to zero (0000). So when targeting the limited C API version 3.10, _Py_IncRef() wasn't used even if it was available.

See #158401 (comment) for the rationale of this change.

Comment thread Include/refcount.h Resolved

@da-woods da-woods left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My overall view:

It obviously simplifies it a bit to go through function calls for all Limited API versions, and it helps with any future changes to reference counting.

If you compile with the Python 3.14 headers (on any Limited API version) then it does handle immortal reference counting correctly (because the logic is inlined). So this doesn't really fix anything. But it might be worthwhile for the simplification.

Comment thread Include/refcount.h Resolved
vstinner and others added 2 commits November 16, 2026 22:29
Co-authored-by: Hood Chatham <roberthoodchatham@gmail.com>
@vstinner

Copy link
Copy Markdown
Member Author

@ngoldbaum suggests backporting the change to all bugfix branches and even security branches: https://discuss.python.org/t/limited-c-api-implement-py-incref-and-py-decref-as-function-calls/27592/22.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants