You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
PyGILStateEnsure() (Probably) Crashes If Called After Finalization #124619
I'm pretty sure PyGILState_Ensure() will crash if called after finalization, particularly in the case where the thread doesn't already have a thread state active. I noticed this while looking at the function for something else.
Hitting what looks like this same crash in real-world (non-synthetic) code, via PyGObject + GDBus, on Python 3.14.7 — so this is still unfixed on stable 3.14, not just the 3.14.0a7 build from #132948.
Reproducer:powerprofilesctl get (a short-lived Python script shipped with power-profiles-daemon, using PyGObject to talk to power-profiles-daemon over D-Bus). It queries the current profile, prints it, and exits. Invoked frequently (once per status-bar poll interval) it segfaults intermittently with SIGSEGV / SEGV_MAPERR.
coredumpctl info backtrace of the crashing thread (a GDBus worker thread, not the main thread):
Frame #0 is unresolved (no debug symbols on this build), but given PyEval_RestoreThread → PyGILState_Ensure directly above it, it's almost certainly the same _PyThreadState_Attach/bind_gilstate_tstate path as the original report.
At the same instant, the main thread is already inside exit():
Mechanism appears to be: the main thread finishes and calls exit(), kicking off interpreter finalization, while a GDBus-owned worker thread is mid-g_object_unref on a proxy/connection object and its GObject finalizer calls back into Python (PyGObject's _gi), which calls PyGILState_Ensure() on a thread state that finalization has already begun tearing down.
Reproduced twice on this machine so far (identical stack both times), a few days apart, under ordinary desktop use — no synthetic thread-spam required, just a program that talks to D-Bus from a background thread and exits promptly afterward. Given power-profiles-daemon, NetworkManager's Python bindings, and plenty of other GLib/GDBus-using Python tools follow this same "spin up async D-Bus, do one thing, exit" pattern, this is likely to affect more than just this one tool as more distros ship 3.14.
We hit this in a Rust/PyO3 extension (russfellows/s3dlio#302) and narrowed it to a pure C reproducer with no Rust or PyO3 involved. The results add something to this issue: 3.13.8+ survives this pattern and 3.14 does not, so it is a regression in 3.14, not only a long-standing API hazard.
Reproducer
N native (non-Python) threads loop on PyGILState_Ensure() / small Python work / PyGILState_Release(), with a short random sleep between attaches, while the main thread returns and the interpreter finalizes. Many objects with __del__ keep bytecode running during finalization.
23/30 and 25/30 in two batches: double free or corruption (out), SIGSEGV, Fatal Python error: PyInterpreterState_Delete: remaining threads
3.13.18
0/30
3.13.7, 3.12.16
0/30 (from our investigation's runs)
Analysis (3.14.10 vs 3.13.18 sources)
_Py_Finalize marks only the thread states that already exist._PyThreadState_RemoveExcept() takes the list of thread states, and only those get _PyThreadState_SetShuttingDown() (pylifecycle.c ~2092). The comment at ~2118 notes the gap: "XXX Make sure we are preventing the creating of any new thread states".
New thread states are still created. A native thread calling PyGILState_Ensure() during finalization has no thread state, so new_threadstate() creates a fresh one with no finalization check. On a default (GIL) build _PyEval_StopTheWorldAll is a no-op.
In 3.14, take_gil parks only flagged thread states. It decides via _PyThreadState_MustExit(tstate), which checks the thread state's own shutting-down flag. The fresh thread state is not flagged, so the thread proceeds and later races PyInterpreterState_Delete / zapthreads. That race produces the three failure modes above:
a SIGSEGV in _PyThreadState_Attach while zapthreads frees the thread state;
a double free when the thread's own PyGILState_Release frees it too;
"remaining threads" when the thread state is created after zapthreads ran.
The thread-state-based check in 3.14 appears to have lost the "late native thread" case that 3.13.8+ handles. Refusing (or parking) thread-state creation in PyGILState_Ensure once finalization has started would close it.
Workaround on our side: no native thread attaches during shutdown; Python objects are built only on Python threads. Happy to test a patch.
I was exploring peak RSS gaps of the free-threaded build with mimalloc v3 (#159718) with overnight stress tests and ran into the same kind of crash on the free-threaded build of main, with a small C extension whose own threads keep creating thread states with PyThreadState_New() while the process exits. I came to the same cause as the comment of 2026-10-30 above. Below: how it crashes on the free-threaded build, what I saw for PyGILState_Ensure() on main, and a sketch that worked here, also with the reproducer above on 3.14. An AI assistant helped with the analysis.
In short:
On 3.15 and main, the interpreter guard from PEP 788 seems to cover PyGILState_Ensure(): the reproducer above didn't fail in any of 100 runs there. PyThreadState_New() has no guard and still crashes.
On 3.14, a check in new_threadstate() (sketch below) made the reproducer pass: 0 of 100 runs, against 38 (default build) and 79 (free-threaded) without it.
On the free-threaded build
The extension's 8 threads loop over PyThreadState_New(), PyEval_RestoreThread(), a little Python work, PyThreadState_Clear() and PyThreadState_DeleteCurrent() (code in the fold of #162109). The process exits after 20 ms. In a loop of fresh processes on main at e8c0584 (2026-11-10), Linux arm64, release build, 337 of 12,288 exits crashed (SIGSEGV), in finalization's interpreter_clear() while it clears another thread's thread state (backtrace trimmed):
#0 llist_remove (node=0x264ea014b68) at Include/internal/pycore_llist.h:83
#1 _Py_brc_remove_thread (tstate=0x264ea010010) at Python/brc.c:220
#2 PyThreadState_Clear (tstate=0x264ea010010) at Python/pystate.c:1967
#3 interpreter_clear (interp=0xaaaabe12fa30 <_PyRuntime+160968>, tstate=0xaaaabe150708 <_PyRuntime+295328>) at Python/pystate.c:918
#6 _Py_Finalize (runtime=0xaaaabe108568 <_PyRuntime>) at Python/pylifecycle.c:2584
In the three core files I looked at, the interpreter's list of thread states holds, besides the finalizing thread's, only the newest one or two, so they were created after finalization had taken the others out. The one being cleared belongs to a thread that is attached and running; the older threads hang in _PyThreadState_HangThread(), as they should.
The change in _PyThreadState_MustExit() that the comment of 2026-10-30 describes came with #130649, the fix for #124878. If I read the documentation right, such a thread should hang: calling PyEval_RestoreThread() while the runtime is finalizing "will hang the thread until the program exits, even if the thread was not created by Python".
PyGILState_Ensure() on main
On 3.15 and main, PyGILState_Ensure() holds an interpreter guard from PEP 788 (#149116) while it creates and binds its thread state, and finalization waits until no guard is held before it takes the thread states out. As far as I can see, that covers it there: on main, release builds, the reproducer above didn't fail in any of 100 runs, on the default and on the free-threaded build, and the same extension with PyGILState_Ensure()/PyGILState_Release() instead of PyThreadState_New() didn't crash in 12,868 exits. 3.14, which the reports above are about, doesn't have the guard. PyThreadState_New() has no guard on main either.
A sketch that worked here
I'm not sure this is the right place for a fix; it is what worked in my tests. In new_threadstate(): when the interpreter is finalizing and the caller isn't the finalizing thread, the new thread state is treated like the ones finalization took out. It is marked as shutting down and not added to the interpreter's list, so its thread hangs when it tries to attach. Two things turned out to matter:
Keeping it out of the list. Marking it but leaving it in the list (what I tried first) made the exit hang: a stop-the-world (the final garbage collection has one) waits for every thread state in the list to pause, and one that is shutting down never does. That hung 5 of 2,651 exits of a release build, and a debug build aborts on the assertion in _PyThreadState_ResumeDetached().
Checking before the TLBC index and the QSBR slot are reserved. PyInterpreterState_Delete() frees both at the very end of finalization, while such threads may still be creating thread states. With the check placed after the reservations, a release build still crashed in 82 of 47,214 exits; the 4 core files I kept all crash in heap_pop() of the TLBC index pool. A thread state created this late never attaches, so it needs neither.
The sketch also makes tstate_delete_common() unlink a thread state only if it is in the list. A thread state created this late is leaked: its thread hangs, and nothing else frees it.
main, free-threaded release build
exits
crashed
as it is
12,288
337
with the sketch
12,783
0
The full test suite passes with it on Linux (free-threaded release build), and together with the sketch in #162109 also on free-threaded debug and default builds. On main's default build the extension didn't crash (0 of 4,446 exits as it is, 0 of 8,712 with both sketches). A debug build still aborts now and then, for two other reasons: the window in that issue (a thread state cleared while its thread is still binding it), and an assertion in mimalloc that shows up without finalization too.
On 3.14
Here PyGILState_Ensure() also creates its thread state with new_threadstate() and has no guard, so the same check covers it. I ported the sketch to the 3.14 branch at 8aa5d65 (2026-11-10); it differs only where 3.14's code does, in the order of the two reservations and in the stop-the-world check in tstate_delete_common(). Release builds, Linux:
on 3.14
runs as it is
failed
runs with the sketch
failed
the reproducer above, default build
100
38
100
0
the reproducer above, free-threaded build
100
79
100
0
the extension with PyThreadState_New(), default build
8,750
2
8,794
0
the extension with PyThreadState_New(), free-threaded build
8,579
259
8,584
0
the extension with PyGILState_Ensure(), default build
8,728
6
8,820
0
the extension with PyGILState_Ensure(), free-threaded build
8,544
240
8,500
1
The one failure with the sketch is an abort whose output mentions a freed thread state. I haven't looked into it; it may be the window of #162109, since PyGILState_Ensure() on 3.14 binds its new thread state itself, outside both sketches.
With the 3.14 port, the full test suite passes on 3.14 too (default and free-threaded builds). Three test files that compile a script into a zip file failed in my runs with and without the sketch, because I had PYTHONPYCACHEPREFIX set; without it they pass.
I may well be missing context here, and I'm glad to test other approaches.
--- a/Python/pystate.c+++ b/Python/pystate.c@@ -1738,6 +1738,16 @@
interp->threads.head = tstate;
}
+// Finalization has taken the other thread states out of the interpreter+// (_PyThreadState_RemoveExcept()) and counts on no new ones being added.+static int+finalization_took_threads(PyInterpreterState *interp)+{+ return (_PyInterpreterState_GetFinalizing(interp) != NULL &&+ _PyInterpreterState_GetFinalizingID(interp)+ != PyThread_get_thread_ident());+}+
static PyThreadState *
new_threadstate(PyInterpreterState *interp, int whence)
{
@@ -1755,17 +1765,24 @@
return NULL;
}
#endif
+ // A thread state created this late never attaches: it needs no TLBC+ // index nor QSBR slot (which the end of finalization frees).+ int late = finalization_took_threads(interp);
#ifdef Py_GIL_DISABLED
- int32_t tlbc_idx = _Py_ReserveTLBCIndex(interp);- if (tlbc_idx < 0) {- free_threadstate(tstate);- return NULL;- }- Py_ssize_t qsbr_idx = _Py_qsbr_reserve(interp);- if (qsbr_idx < 0) {- _Py_UnreserveTLBCIndex(interp, tlbc_idx);- free_threadstate(tstate);- return NULL;+ int32_t tlbc_idx = -1;+ Py_ssize_t qsbr_idx = -1;+ if (!late) {+ tlbc_idx = _Py_ReserveTLBCIndex(interp);+ if (tlbc_idx < 0) {+ free_threadstate(tstate);+ return NULL;+ }+ qsbr_idx = _Py_qsbr_reserve(interp);+ if (qsbr_idx < 0) {+ _Py_UnreserveTLBCIndex(interp, tlbc_idx);+ free_threadstate(tstate);+ return NULL;+ }
}
#endif
@@ -1777,16 +1794,29 @@
uint64_t id = interp->threads.next_unique_id;
init_threadstate(tstate, interp, id, whence);
- // Add the new thread state to the interpreter.- PyThreadState *old_head = interp->threads.head;- add_threadstate(interp, (PyThreadState *)tstate, old_head);+ // Checked again under the lock that _PyThreadState_RemoveExcept() takes.+ late = late || finalization_took_threads(interp);+ if (late) {+ // Treat it like the thread states that finalization took out: keep+ // it out of the interpreter's list and mark it as shutting down, so+ // that its thread hangs when it tries to attach.+ ((PyThreadState *)tstate)->state = _Py_THREAD_SHUTTING_DOWN;+ }+ else {+ // Add the new thread state to the interpreter.+ PyThreadState *old_head = interp->threads.head;+ add_threadstate(interp, (PyThreadState *)tstate, old_head);+ }
HEAD_UNLOCK(interp->runtime);
#ifdef Py_GIL_DISABLED
- // Must be called with lock unlocked to avoid lock ordering deadlocks.- _Py_qsbr_register(tstate, interp, qsbr_idx);- tstate->tlbc_index = tlbc_idx;+ if (!late) {+ // Must be called with lock unlocked to avoid lock ordering deadlocks.+ _Py_qsbr_register(tstate, interp, qsbr_idx);+ tstate->tlbc_index = tlbc_idx;+ }+ // A reservation made before finalization started is left unused.
#endif
return (PyThreadState *)tstate;
@@ -2023,17 +2053,20 @@
_PyRuntimeState *runtime = interp->runtime;
HEAD_LOCK(runtime);
+ // A thread state created during finalization isn't in the list.+ int linked = (tstate->prev != NULL || interp->threads.head == tstate);
if (tstate->prev) {
tstate->prev->next = tstate->next;
}
- else {+ else if (linked) {
interp->threads.head = tstate->next;
}
if (tstate->next) {
tstate->next->prev = tstate->prev;
}
int state = _Py_atomic_load_int_relaxed(&tstate->state);
- if (state != _Py_THREAD_SUSPENDED &&+ if (linked &&+ state != _Py_THREAD_SUSPENDED &&
state != _Py_THREAD_SUSPENDED_WAITING)
{
// Any ongoing stop-the-world request should not wait for us because
Activity
PyGILState_Ensurecalled from background thread during finalization on Python 3.14.0a7 #132948 as a duplicate of this issueSccucci84 commentedon Sep 1, 2026
Hitting what looks like this same crash in real-world (non-synthetic) code, via PyGObject + GDBus, on Python 3.14.7 — so this is still unfixed on stable 3.14, not just the 3.14.0a7 build from #132948.
Reproducer:
powerprofilesctl get(a short-lived Python script shipped withpower-profiles-daemon, using PyGObject to talk topower-profiles-daemonover D-Bus). It queries the current profile, prints it, and exits. Invoked frequently (once per status-bar poll interval) it segfaults intermittently withSIGSEGV/SEGV_MAPERR.coredumpctl infobacktrace of the crashing thread (a GDBus worker thread, not the main thread):Frame #0 is unresolved (no debug symbols on this build), but given
PyEval_RestoreThread→PyGILState_Ensuredirectly above it, it's almost certainly the same_PyThreadState_Attach/bind_gilstate_tstatepath as the original report.At the same instant, the main thread is already inside
exit():Mechanism appears to be: the main thread finishes and calls
exit(), kicking off interpreter finalization, while a GDBus-owned worker thread is mid-g_object_unrefon a proxy/connection object and its GObject finalizer calls back into Python (PyGObject's_gi), which callsPyGILState_Ensure()on a thread state that finalization has already begun tearing down.Reproduced twice on this machine so far (identical stack both times), a few days apart, under ordinary desktop use — no synthetic thread-spam required, just a program that talks to D-Bus from a background thread and exits promptly afterward. Given
power-profiles-daemon,NetworkManager's Python bindings, and plenty of other GLib/GDBus-using Python tools follow this same "spin up async D-Bus, do one thing, exit" pattern, this is likely to affect more than just this one tool as more distros ship 3.14.Poll the active power profile over D-Bus instead of powerprofilesctl
russfellows commentedon Oct 30, 2026
We hit this in a Rust/PyO3 extension (russfellows/s3dlio#302) and narrowed it to a pure C reproducer with no Rust or PyO3 involved. The results add something to this issue: 3.13.8+ survives this pattern and 3.14 does not, so it is a regression in 3.14, not only a long-standing API hazard.
Reproducer
N native (non-Python) threads loop on
PyGILState_Ensure()/ small Python work /PyGILState_Release(), with a short random sleep between attaches, while the main thread returns and the interpreter finalizes. Many objects with__del__keep bytecode running during finalization.attacher.c (build with
gcc -O2 -shared -fPIC -I$(python3 -c 'import sysconfig;print(sysconfig.get_path("include"))') attacher.c -o attacher.so):drv.py (run:
python3 drv.py ./attacher.so 64, where 64 is the number of native threads):Results (Linux x86_64 glibc, default GIL build, 64 native threads, 30 runs each)
double free or corruption (out), SIGSEGV,Fatal Python error: PyInterpreterState_Delete: remaining threadsAnalysis (3.14.10 vs 3.13.18 sources)
_Py_Finalizemarks only the thread states that already exist._PyThreadState_RemoveExcept()takes the list of thread states, and only those get_PyThreadState_SetShuttingDown()(pylifecycle.c ~2092). The comment at ~2118 notes the gap: "XXX Make sure we are preventing the creating of any new thread states".PyGILState_Ensure()during finalization has no thread state, sonew_threadstate()creates a fresh one with no finalization check. On a default (GIL) build_PyEval_StopTheWorldAllis a no-op.take_gilparks only flagged thread states. It decides via_PyThreadState_MustExit(tstate), which checks the thread state's own shutting-down flag. The fresh thread state is not flagged, so the thread proceeds and later racesPyInterpreterState_Delete/zapthreads. That race produces the three failure modes above:_PyThreadState_Attachwhile zapthreads frees the thread state;PyGILState_Releasefrees it too;_PyThreadState_MustExitchecks globally whether the runtime is finalizing, so the late thread state is parked too (pthread_exit & PyThread_exit_thread from PyEval_RestoreThread etc. are harmful #87135 backport), and there is no crash.The thread-state-based check in 3.14 appears to have lost the "late native thread" case that 3.13.8+ handles. Refusing (or parking) thread-state creation in
PyGILState_Ensureonce finalization has started would close it.Workaround on our side: no native thread attaches during shutdown; Python objects are built only on Python threads. Happy to test a patch.
maikschulze commentedon Nov 16, 2026
I was exploring peak RSS gaps of the free-threaded build with mimalloc v3 (#159718) with overnight stress tests and ran into the same kind of crash on the free-threaded build of main, with a small C extension whose own threads keep creating thread states with
PyThreadState_New()while the process exits. I came to the same cause as the comment of 2026-10-30 above. Below: how it crashes on the free-threaded build, what I saw forPyGILState_Ensure()on main, and a sketch that worked here, also with the reproducer above on 3.14. An AI assistant helped with the analysis.In short:
PyGILState_Ensure(): the reproducer above didn't fail in any of 100 runs there.PyThreadState_New()has no guard and still crashes.new_threadstate()(sketch below) made the reproducer pass: 0 of 100 runs, against 38 (default build) and 79 (free-threaded) without it.On the free-threaded build
The extension's 8 threads loop over
PyThreadState_New(),PyEval_RestoreThread(), a little Python work,PyThreadState_Clear()andPyThreadState_DeleteCurrent()(code in the fold of #162109). The process exits after 20 ms. In a loop of fresh processes on main at e8c0584 (2026-11-10), Linux arm64, release build, 337 of 12,288 exits crashed (SIGSEGV), in finalization'sinterpreter_clear()while it clears another thread's thread state (backtrace trimmed):In the three core files I looked at, the interpreter's list of thread states holds, besides the finalizing thread's, only the newest one or two, so they were created after finalization had taken the others out. The one being cleared belongs to a thread that is attached and running; the older threads hang in
_PyThreadState_HangThread(), as they should.The change in
_PyThreadState_MustExit()that the comment of 2026-10-30 describes came with #130649, the fix for #124878. If I read the documentation right, such a thread should hang: callingPyEval_RestoreThread()while the runtime is finalizing "will hang the thread until the program exits, even if the thread was not created by Python".PyGILState_Ensure()on mainOn 3.15 and main,
PyGILState_Ensure()holds an interpreter guard from PEP 788 (#149116) while it creates and binds its thread state, and finalization waits until no guard is held before it takes the thread states out. As far as I can see, that covers it there: on main, release builds, the reproducer above didn't fail in any of 100 runs, on the default and on the free-threaded build, and the same extension withPyGILState_Ensure()/PyGILState_Release()instead ofPyThreadState_New()didn't crash in 12,868 exits. 3.14, which the reports above are about, doesn't have the guard.PyThreadState_New()has no guard on main either.A sketch that worked here
I'm not sure this is the right place for a fix; it is what worked in my tests. In
new_threadstate(): when the interpreter is finalizing and the caller isn't the finalizing thread, the new thread state is treated like the ones finalization took out. It is marked as shutting down and not added to the interpreter's list, so its thread hangs when it tries to attach. Two things turned out to matter:_PyThreadState_ResumeDetached().PyInterpreterState_Delete()frees both at the very end of finalization, while such threads may still be creating thread states. With the check placed after the reservations, a release build still crashed in 82 of 47,214 exits; the 4 core files I kept all crash inheap_pop()of the TLBC index pool. A thread state created this late never attaches, so it needs neither.The sketch also makes
tstate_delete_common()unlink a thread state only if it is in the list. A thread state created this late is leaked: its thread hangs, and nothing else frees it.The full test suite passes with it on Linux (free-threaded release build), and together with the sketch in #162109 also on free-threaded debug and default builds. On main's default build the extension didn't crash (0 of 4,446 exits as it is, 0 of 8,712 with both sketches). A debug build still aborts now and then, for two other reasons: the window in that issue (a thread state cleared while its thread is still binding it), and an assertion in mimalloc that shows up without finalization too.
On 3.14
Here
PyGILState_Ensure()also creates its thread state withnew_threadstate()and has no guard, so the same check covers it. I ported the sketch to the 3.14 branch at 8aa5d65 (2026-11-10); it differs only where 3.14's code does, in the order of the two reservations and in the stop-the-world check intstate_delete_common(). Release builds, Linux:PyThreadState_New(), default buildPyThreadState_New(), free-threaded buildPyGILState_Ensure(), default buildPyGILState_Ensure(), free-threaded buildThe one failure with the sketch is an abort whose output mentions a freed thread state. I haven't looked into it; it may be the window of #162109, since
PyGILState_Ensure()on 3.14 binds its new thread state itself, outside both sketches.With the 3.14 port, the full test suite passes on 3.14 too (default and free-threaded builds). Three test files that compile a script into a zip file failed in my runs with and without the sketch, because I had
PYTHONPYCACHEPREFIXset; without it they pass.I may well be missing context here, and I'm glad to test other approaches.
The sketch (against e8c0584, 2026-11-10)