gh-153062: Fix a crash iterating itertools.tee on the free-threaded build

[tonghuaroot]

Iterating an itertools.tee iterator concurrently crashes the free-threaded build. tee iterators are documented as not thread-safe, so concurrent use returning undefined results or raising RuntimeError is by design and is preserved here; this fixes only the memory-unsafety (corrupted reference counts and a segfault), which the free-threaded build must not permit. The other itertools iterators already lock their __next__; tee was the remaining one.

The shared teedataobject cells are locked while read, extended, or cleared. Each branch's position (dataobj and index) is snapshotted under the tee object's own critical section and revalidated before it is advanced, so the tee lock and the data-object lock are never nested: a nested critical section can be suspended under contention, which would break the atomicity the position fetch depends on (this is why Py_BEGIN_CRITICAL_SECTION2 on the two objects does not work here).

Tests in Lib/test/test_free_threading/ cover both a single branch consumed by many threads and sibling branches each consumed by their own thread; run_concurrently is barrier-synchronized so the threads actually contend. The re-entrancy guard's RuntimeError is tolerated, since it is a documented outcome. Single-threaded behavior is unchanged and test_itertools still passes.

Backport labels left to a maintainer.

• Issue: itertools.tee: concurrent iteration crashes the free-threaded build #153062

[corona10]

I didn't check the PR detail, but if you write whole code with AI, please consider that the approach is proper. Adding critical section is not a free.

[nascheme]

I took a look at this. I think critical sections are a reasonable tool in this case. Using them here is similar to what other itertools iterators do. One gap, need to make tee_copy_impl safe as well:

--- a/Modules/itertoolsmodule.c
+++ b/Modules/itertoolsmodule.c
@@ -1028,8 +1028,10 @@ tee_copy_impl(teeobject *to)
     if (newto == NULL) {
         return NULL;
     }
+    Py_BEGIN_CRITICAL_SECTION(to);
     newto->dataobj = (teedataobject *)Py_NewRef(to->dataobj);
     newto->index = to->index;
+    Py_END_CRITICAL_SECTION();
     newto->weakreflist = NULL;
     newto->state = to->state;
     PyObject_GC_Track(newto);

I think after that change, this PR would be okay to merge.

[tonghuaroot]

Applied in 3dab9e0, thanks. test_itertools and the tee race test pass on a free-threaded build.

[miss-islington-app]

Thanks @tonghuaroot for the PR, and @nascheme for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

[miss-islington-app]

Thanks @tonghuaroot for the PR, and @nascheme for merging it 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

[miss-islington-app]

Sorry, @tonghuaroot and @nascheme, I could not cleanly backport this to 3.14 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 6c389c446e487bcb9d09e85de30aa42052990f1c 3.14

[bedevere-app]

GH-153475 is a backport of this pull request to the 3.15 branch.

[bedevere-app]

GH-153476 is a backport of this pull request to the 3.14 branch.