gh-153083: Defer GC tracking of an array to the end of construction.

[corona10]

• Issue: Need to defer GC tracking of array.array construction #153083

[corona10]

TSAN Script

# TSAN_OPTIONS=halt_on_error=0 ./python.exe tsan_array.py
import gc, threading
from array import array

N = 4000
box = []
done = False

def reader():
    while not done:
        if box:
            try:
                len(box[0]); repr(box[0])
            except Exception:
                pass

def gen():
    for i in range(N):
        if i == 5 and not box:  # leak the half-built array
            box.extend(o for o in gc.get_objects()
                       if type(o) is array and 0 < len(o) < N)
        yield i

t = threading.Thread(target=reader)
t.start()
for _ in range(30):
    box.clear()
    array('q', gen())
done = True
t.join()
print("observed half-built:", bool(box))

AS-IS

SUMMARY: ThreadSanitizer: data race arraymodule.c:195 in array_resize
==================
==29994==ERROR: ThreadSanitizer: SEGV on unknown address 0x000104000100 (pc 0x00010601baec bp 0x00016c046010 sp 0x00016c045fd0 T5647275)
==29994==The signal is caused by a READ memory access.
    #0 __tsan_atomic64_load <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64e+0x5faec)
    #1 _PyMem_MiFree obmalloc.c:298 (python.exe:arm64+0x100164ac8)
    #2 PyMem_Free obmalloc.c:1286 (python.exe:arm64+0x100166b48)
    #3 list_dealloc listobject.c:569 (python.exe:arm64+0x1000e8524)
    #4 _Py_Dealloc object.c:3319 (python.exe:arm64+0x10013ee18)
    #5 _Py_MergeZeroLocalRefcount object.c (python.exe:arm64+0x10013f074)
    #6 array_repr arraymodule.c:2633 (array.cpython-316t-darwin.so:arm64+0x536c)
    #7 PyObject_Repr object.c:784 (python.exe:arm64+0x100140264)
    #8 builtin_repr bltinmodule.c:2677 (python.exe:arm64+0x100286d28)
    #9 _PyEval_EvalFrameDefault generated_cases.c.h:2712 (python.exe:arm64+0x100292e58)
    #10 _PyEval_Vector ceval.c:2172 (python.exe:arm64+0x10028b578)
    #11 _PyFunction_Vectorcall call.c (python.exe:arm64+0x100090918)
    #12 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x10009205c)
    #13 method_vectorcall classobject.c:55 (python.exe:arm64+0x10009536c)
    #14 context_run context.c:731 (python.exe:arm64+0x1002d761c)
    #15 method_vectorcall_FASTCALL_KEYWORDS descrobject.c:421 (python.exe:arm64+0x1000a785c)
    #16 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x1000902f0)
    #17 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:768 (python.exe:arm64+0x10028c058)
    #18 _PyEval_EvalFrameDefault generated_cases.c.h:1906 (python.exe:arm64+0x100290f88)
    #19 _PyEval_Vector ceval.c:2172 (python.exe:arm64+0x10028b578)
    #20 _PyFunction_Vectorcall call.c (python.exe:arm64+0x100090918)
    #21 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x10009205c)
    #22 method_vectorcall classobject.c:55 (python.exe:arm64+0x10009536c)
    #23 _PyObject_Call call.c:348 (python.exe:arm64+0x100090598)
    #24 PyObject_Call call.c:373 (python.exe:arm64+0x10009060c)
    #25 thread_run _threadmodule.c:388 (python.exe:arm64+0x100444458)
    #26 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x100383b00)
    #27 __tsan_thread_start_func <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64e+0x337a0)
    #28 _pthread_start <null> (libsystem_pthread.dylib:arm64e+0x6c54)
    #29 thread_start <null> (libsystem_pthread.dylib:arm64e+0x1c18)

==29994==Register values:
 x[0] = 0x0000000105b24000   x[1] = 0x0000100208000200   x[2] = 0x00000000c0b414ff   x[3] = 0x0000000000000008
 x[4] = 0x0000000000000003   x[5] = 0x0000000000000000   x[6] = 0x0000000000000029   x[7] = 0x00000000e6477557
 x[8] = 0x0000000000000000   x[9] = 0x0000000000000014  x[10] = 0x00000000c0000000  x[11] = 0x0000000000000000
x[12] = 0x0000000105ab1ef0  x[13] = 0x0000100208000200  x[14] = 0x0000000000000003  x[15] = 0x0000000000001f8c
x[16] = 0x0000000000000000  x[17] = 0x0000000106070a40  x[18] = 0x0000000000000000  x[19] = 0x0000000104000100
x[20] = 0x0000000105b24000  x[21] = 0x0000000104fb0acc  x[22] = 0x0000000000000000  x[23] = 0x000000016c0470e0
x[24] = 0x0000000000000000  x[25] = 0x0000000000000000  x[26] = 0x00000003023644c0  x[27] = 0x0000000105f6c208
x[28] = 0x0000000105f6c1a8     fp = 0x000000016c046010     lr = 0x000000010601badc     sp = 0x000000016c045fd0
ThreadSanitizer can not provide additional info.
SUMMARY: ThreadSanitizer: SEGV obmalloc.c:298 in _PyMem_MiFree
==29994==ABORTING

TO-BE

observed half-built: False

[vstinner]

There are many methods still calling newarrayobject(). I would prefer to fix all methods at once to defer the GC tracking.

[sergey-miryanov]

Should we update tp_alloc slot?

[vstinner]

LGTM.

[miss-islington-app]

Thanks @corona10 for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14, 3.15.
🐍🍒⛏🤖

[miss-islington-app]

Sorry, @corona10, I could not cleanly backport this to 3.14 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 8615062fd580934fc431746f64b1ac7ee6ef8e88 3.14

[bedevere-app]

GH-153336 is a backport of this pull request to the 3.15 branch.

[miss-islington-app]

Sorry, @corona10, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 8615062fd580934fc431746f64b1ac7ee6ef8e88 3.13

[bedevere-app]

GH-153337 is a backport of this pull request to the 3.14 branch.

[bedevere-app]

GH-153338 is a backport of this pull request to the 3.13 branch.