M8 Concurrency, runtime and the C API

Roughly nine weeks. Lessons C01 through C08 and R01 through R09. The GIL and what replaces it, subinterpreters, startup, import, and the C API surface. After this the book covers all of CPython, every subsystem has a blueprint, and the reimplementation track is documented but not yet demonstrated. That is a sane place to stop.

Exit criterion. Every concurrency lesson runs on both a GIL build and a free threaded build with the results reported side by side from the same script. BP-CAPI-SURFACE inventories the stable ABI and the header tiers, generated from Doc/data/stable_abi.dat and Tools/c-analyzer rather than compiled by hand.

Content

C01 through C08: the GIL, PEP 703 free threading, the free threaded object header, biased reference counting, per object locks, thread state, subinterpreters and PEP 734 (all eight written and merged)
R01 through R09: startup, the runtime and interpreter state, the import system, frozen modules, PEP 810 lazy imports, the C API tiers, the stable ABI including abi3t, and shutdown (all nine written and merged)

Blueprints

BP-FT-OBJECT, BP-STATE, BP-IMPORT, BP-CAPI-SURFACE

Artifacts

The gil_lab app
Four animations

Standing decision

The default build stays primary throughout. Every affected lesson gets a clearly marked free threading section rather than a parallel lesson, and that gets revisited if the steering council moves toward making free threaded the default.

C01. One lock, one interpreter (#169)

C01, one lock, one interpreter, is merged in #169. First lesson of this milestone.

The GIL is the thing in CPython that gets explained most and measured least, so the lesson measures it first and then reads the code that produced each number. Nine cells, all of which run on whatever Python the reader already has.

The two headline measurements are the ones everybody has heard about, and it was worth running them anyway. Two threads each adding up two million numbers finish no faster than doing both jobs in a row, and usually a little slower, because handing the lock back and forth costs something. Change the work from adding to sleeping and change nothing else, and the same two threads finish twice as fast. The Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS pair is the whole explanation, and it is short enough to read in the header, so the lesson points at it rather than paraphrasing it.

The section I did not expect to be the best one is where the lock can actually be dropped. Most writing on this says the eval loop checks between every instruction. It does not. The eval breaker is read by a _CHECK_PERIODIC instruction, and the compiler puts one of those in front of backward jumps and function resumes and nowhere else. A thread inside a single call into C has neither, so it cannot be interrupted however long the call takes, and sys.setswitchinterval has nothing to say about it.

That turns into a cell that I think is the most useful thing in the lesson. A ticking thread does nothing but write down the time, so a pause becomes a gap in a list. Give the main thread two jobs of roughly equal length, a loop written in Python and one call to list.sort, and the sort shuts the other thread out for about ninety five percent of its own duration while the Python loop shuts it out for a quarter. Same machine, same interval, same two threads, and the only difference is whether there is a backward jump inside the work.

The switch interval gets a section because the usual advice about it is wrong in an interesting way. It is described as how often threads take turns, so people turn it down when threads look unfair. What take_gil actually does with it is wait that long on a condition variable before setting the drop request bit, which makes it a measure of how patient a waiting thread is. So turning it down does not create more turns, it makes waiters more impatient, and every handoff stops one thread and starts another. Measured over four settings, one microsecond costs about half the total throughput and buys nothing, since the two threads were already within a few percent of each other at the default. And at half a second, which is longer than the whole measurement, the second thread waits out an interval that never expires and finishes having done literally nothing. That last row is the one that makes the point.

Two tier1 recordings close it out, which is the side by side this milestone's exit criterion asks for. The identical two thread benchmark on the free threaded image comes out at 1.98x on two threads, from the same source that got 1.0x in the cell above. And the counting thread keeps counting the whole way through somebody else's list.sort, because there is no lock for it to be waiting on. Both are best of five after a warmup, because the image runs under emulation on a small virtual machine and a single cold timing there is worse than useless.

Six diagrams, nineteen citations against the pin, and four new glossary terms in a new Threads group: GIL, switch interval, eval breaker and periodic check. The last two were being used in M08's prose already without ever being defined, so this closes that.

One practical note. A browser cannot start a thread at all, so every timing cell checks once at the top and prints a note rather than a number. threading.Thread(...).start() raises a catchable RuntimeError on Pyodide, which is the nicest possible version of that problem, and all seven C01 cells come back green from the probe.

Next is C02 and what PEP 703 actually did. The place I want to start is the object header, because on 3.15 the default build's header is a 32 bit refcount packed next to a 16 bit flags field, and the free threaded build's is a thread id, a one byte mutex, a byte of collector bits, a local count and a shared count. Two counts on one object is already measured in M06, so C02 gets to spend its time on what had to be added back once the lock was gone: per object locks, critical sections, and the fact that the GIL can be switched back on at runtime when a module says it needs one.

C02. What the lock was actually protecting (#170)

C02 is merged in #170. It is the lesson that changes what a reader thinks their own code was getting from the GIL.

The spine is a counter. Four threads, one shared global, a hundred thousand increments each, with the switch interval turned all the way down so handoffs happen as often as they can. The answer is exactly four hundred thousand, every run. Then swap the line for counter = add_one(counter) and about half the increments disappear. Write it as a read, a loop that goes round once and does nothing, and a write back, and about two thirds disappear. Same build, same lock, same interval, same arithmetic. That is C01's periodic check result showing up in code people actually write: counter = counter + 1 has no backward jump and no function resume in it, so no other thread can get in partway through. The counter was never safe, it was unreachable, and those two look identical right up until somebody refactors the line.

The half of the sentence that is a real promise is the interpreter's own data. Four threads doing 1,600,000 appends into one list lose nothing on any build. On the free threaded build that is a critical section taking the list's own lock, and the declaration is a single @critical_section line in an Argument Clinic block with a script writing the C that takes it. B04 gets a callback there.

Then the machinery that replaced the lock. PyMutex is one byte with two bits used and an uncontended take is a single compare and exchange, which is the whole reason a lock per object is affordable. Deadlock is avoided by letting a critical section suspend itself rather than by ordering locks, with a separate macro for the cases that genuinely need two objects, which is what list_a + list_b uses. On a build with the GIL every bit of it compiles to a pair of braces.

The last section is the one people do not expect. Free threading is not a one way door. The same interpreter turns its lock back on from -X gil=1 at startup, or from importing one C extension with no Py_mod_gil slot, which enables it permanently for the whole process with a RuntimeWarning as the only sign. The cell runs on an ordinary build too, where -X gil=1 is quietly accepted and does nothing while -X gil=0 refuses to start the interpreter at all.

Two recordings on the free threaded image. Four threads appending to four lists come out at 2.44x while four threads appending to one list come out at 0.52x, slower than one thread doing all of it, which is per object locking being genuinely per object. And the same three counters run twice on the same binary, once with -X gil=0 where all three lose increments and once with -X gil=1 where the first is exact again and the other two are not.

Six diagrams, seventeen citations against the pin, three new glossary terms for race condition, per object lock and critical section. All seven runnable cells come back green from the Pyodide probe, guarded on whether the runtime can start a thread and, for the last one, a process. The exit criterion for this milestone asks that every concurrency lesson runs on both a GIL build and a free threaded build with results side by side, and C02 was checked on 3.15.0rc1 default, 3.14 and 3.15.0rc1 free threaded before it shipped.

One small correction went in with it. C01 closed by promising that C02 would be about the object header getting bigger, but O01 and M06 had already covered that between them, so C01's closing paragraph now points at what C02 actually turned out to be.

C03 goes down one more level, to what a thread actually is to the interpreter. Every one of them has a PyThreadState, they hang off the interpreter in a list, and attaching and detaching from that list is what the GIL handoff in C01 was really doing.

Tagged as v0.0.45.

C03. Every thread gets a struct (#171)

C03 is merged in #171. Every thread gets a struct.

C01 and C02 both kept saying "a thread hands the lock over" without ever saying what a thread is to the interpreter, so this one goes and looks. It is a PyThreadState, a plain struct with about forty fields in it, and five public C API calls through ctypes are enough to walk the interpreter's own linked list of them from Python. No debugger, no debug build. Three parked threads turn [1] into [5, 4, 3, 1] and back to [1] once they finish.

The result the lesson is built on is the one about identity. Three waves of three threads, joined between each wave, give state ids that climb from 6 to 14 and never repeat, while threading.get_ident() hands back the exact same three values in every wave. That is the bug behind every dict that was keyed on get_ident() and kept after the thread exited. It reproduces on 3.14, on 3.15.0rc1 and on the free threaded build.

From there the rest falls out. sys._current_frames() is that same walk with current_frame read out as well, keyed by the recycled number rather than the id, which is why its keys match threading.enumerate() exactly. The threading.local values, the scratch dict behind PyThreadState_GetDict() and the exception being handled are all fields on the state, which is why a self referential list prints as [...] on two threads at once without either one confusing the other.

The state field is where the C01 handshake actually lives. Py_BEGIN_ALLOW_THREADS expands to PyEval_SaveThread, which is one line that detaches the thread state, so "release the GIL" and "detach" are the same thing said two ways and only the second one survives on a build with no GIL. Suspended is what stopping the world does, which is where the M7 collector lessons were leaning without saying so. Shutting down is what finalization does, and a daemon thread that reads it is hung at its next periodic check with its finally blocks never run.

One Tier 1 recording on the free threaded image nails that last part down. A daemon child on a build with no lock at all still never reaches its finally, which is what you would expect if shutdown works through the thread state rather than through the GIL.

An earlier draft had a second recording measuring what sys._current_frames() costs when it has to stop eight spinning threads. The first native run showed 0.04 us alone against 23.8 us under load, which looked like a great result. It did not reproduce in the container, and a proper sweep across spinner counts with 400 samples per point showed both builds have long noisy tails, with the default build recording the single worst case of the lot. The 23.8 us was a scheduling artifact. The section was cut rather than shipped.

Six diagrams, twenty three citations against the pin, three new glossary terms. Every cell verified on 3.15.0rc1, 3.14 and the free threaded build, and green in the browser probe, where ctypes.pythonapi turns out to work fine and the thread and subprocess cells say so plainly instead of failing.

Tagged as v0.0.46.

C04 next. It stays on the object header and picks up what C02 left out, which is how free threading keeps a reference count that every thread can touch without the counter becoming the bottleneck. The answer is two counters, one the owning thread can change with no locking at all and one everybody else shares, with ob_tid naming the owner. That field holds the same ident from this lesson, sitting in every object in the heap.

Add C04, more than one interpreter in one process (#172)

C04, more than one interpreter in one process, is merged in #172 and is the fourth concurrency lesson.

The first three all assumed a process holds exactly one interpreter and never said so. This one drops that assumption and finds the same shape one level up. An interpreter is a struct on a linked list hanging off the runtime, newest first, the same way a thread state is a struct on a linked list hanging off the interpreter. The lesson walks both lists at once through ctypes, which is the C03 trick with one more loop around it. Interpreter ids count up across the process and are never reused. Thread state ids restart at one inside every new interpreter, so the same state id can name two live threads at the same instant.

The anchor result needs no benchmarking. The GIL is a field on the interpreter rather than a global, and the switch interval lives on the lock, so setting it in one interpreter does not change what the other one reads. That means the dial C01 spent a section turning was never the process's, which is a nice thing to be able to hand a reader four lessons in.

The sharing section lands on something M05 already set up. Only immutable objects may be runtime global, and the set that qualifies is exactly the immortal objects. An object whose reference count neither interpreter may touch is the only kind both can point at. Everything else crosses as a copy through a queue neither of them owns.

The last section is this milestone's exit criterion in one cell plus two recordings. Four counting jobs, four operating system threads on both sides, one interpreter against four. Keeping the thread count equal matters more than it looks, because timing threaded work against a single threaded baseline is not a fair test. On this laptop a one thread baseline can measure slower than a two thread run, purely because of how macOS places a lone thread on an efficiency core. Both sides use four threads and only the interpreter count changes.

The two Tier 1 recordings run that program in the same fixed four core container, once on the release image and once on the free threaded one. With a GIL, four interpreters finish 2.01 times faster. Without one, the same change buys 1.15 times, because the four threads were already running at once. Same script, two builds, results side by side.

Twenty three citations against v3.15.0rc1, six diagrams, six runnable cells, seven claims and two recordings. Three glossary terms added for interpreter state, subinterpreter and shareable object, taking GLOSSARY.md to 201 terms. All 16 CI checks passed on the first run.

Everything is green in a browser tab as well, though not much happens there. Pyodide will not make a second interpreter, so the first cell checks once and the rest say so instead of printing nonsense. The interpreter walk itself does work in the browser and prints the one interpreter Pyodide has, which was worth confirming.

One correction shipped with this PR. The end of C03 promised a C04 about biased reference counting and ob_tid. M06 already covers all of that in detail, so that paragraph now points at subinterpreters. The C04 preview in my previous comment on this issue is superseded for the same reason.

Tagged as v0.0.47.

C05 goes back to startup. The runtime struct that owns the interpreter list has to exist before there is an interpreter to build it from, and the order startup does things in explains a handful of rules that look arbitrary from Python, like why the encoding of sys.stdout cannot be changed after the fact.

Add C05, the bit that interrupts a running thread (#173)

C05, the bit that interrupts a running thread, is merged in #173 and is the fifth concurrency lesson.

C01 named the eval breaker in one sentence and then went back to timing the GIL. This lesson takes the word of bits seriously, because five separate features are built on it and none of them look related from the outside. The shape is always the same: somebody sets a bit and returns immediately, and the thread that owns the bit notices at its next periodic check. That is how Ctrl-C reaches a Python function, how the collector gets a turn, how one thread throws an exception into another, how a C extension asks the main thread to do something, and how a debugger in a different process makes this one run a script.

The first cell shows that the periodic check is an instruction rather than a timer. A plain while loop compiles to one JUMP_BACKWARD, which runs the check. A yield from loop compiles to two JUMP_BACKWARD_NO_INTERRUPT, which deliberately does not, because an interruption in a delegation chain should land in the innermost generator rather than in whoever is delegating to it. Counting both across every function in asyncio.base_events gives 53 of the first and 86 of the second, which is a satisfying thing to be able to measure from Python.

The middle four cells are the same idea with four different bits. A SIGINT raised from a worker fifty milliseconds in is handled at 678 ms when the main thread is inside one call to list.sort and at 67 ms when it is running an ordinary Python loop, and the Python handler runs on MainThread both times even though a worker raised it. PyThreadState_SetAsyncExc through ctypes ends a spinning thread in about 1 ms and a thread inside time.sleep(1.5) in about 1298 ms. Py_AddPendingCall scheduled from a thread named Worker runs on MainThread, because the public one always targets the main thread of the main interpreter.

The cell I like most needs no C at all. The allocator never runs a collection, it only sets a bit past the threshold, so building sixty thousand small dictionaries with json.loads runs the collector once and building the same sixty thousand with a list display runs it 89 times. Same objects, same count, same allocator, and the only difference is that one of them stops to look.

The last section is sys.remote_exec, which is the same trick across a process boundary and is how a debugger attaches to a program that was not started under one. The two Tier 1 recordings run it in a fixed container where the operating system allows it. An injected script arrives 3.2 ms after the ask against a child running bytecode and 2981 ms against a child inside one sort, 946 times later. The free threaded recording is the same program on a build with no GIL and comes out the same shape, 2920 times. That is the point of running it twice: the eval breaker is easy to file away as part of the GIL, because that is where most people meet it, and the lock is one bit out of eight.

Twenty citations against v3.15.0rc1, six diagrams, six runnable cells, seven claims and two recordings. Three glossary terms added for pending call, asynchronous exception and signal handler, taking GLOSSARY.md to 204 terms.

Two of the cells do real work in a browser tab as well. The dis scan gives the same counts on Pyodide, and so does the collector cell, right down to the same 1 against 89. The three thread cells and the subprocess cell say so and stop.

One correction shipped with this PR. The end of C04 promised a C05 about startup, and startup is R01 on this checklist, so that paragraph now points at the eval breaker. The C05 preview in my previous comment here is superseded for the same reason.

One thing CI caught that local runs did not. The remote_exec cell called sys.remote_exec straight after Popen, which races on Linux: a process that has not finished starting up has no running interpreter in it yet, and asking early raises RuntimeError rather than waiting. The child now prints a line first and the cell waits for it, which is what the Tier 1 programs were already doing. That is worth knowing for anybody writing this by hand, so it is in the lesson text now rather than only in the code.

Tagged as v0.0.48.

C06 stays on the free threaded build and looks at what replaced the lock. If two threads may run Python at the same instant, every container in the standard library needs an answer for what happens when both reach for it, and the answers are different for a list, a dict and a set. The interesting part is how much of it turns out not to be locking at all.

Add C08, sending work to another interpreter (#177)

C08 is merged in #177, which finishes the concurrency half of this milestone. All eight C lessons are written.

C08 is the practical half of C04. C04 built a second interpreter and showed that the two share almost nothing, and this one asks what follows from that: how do you give one of them a job, and what does the handover cost. Four kinds of callable handed to Interpreter.call, seven values checked against is_shareable, pickle and a real queue at the same time, an exception from another interpreter taken apart to show it is a snapshot, six standard library modules imported into a subinterpreter where two are refused by name, twenty thousand queue round trips timed for three payloads, and two jobs on a four worker InterpreterPoolExecutor that come out with opposite verdicts in the same cell.

The result worth writing down is that there is no ranking, only a question about the workload. On a build with the lock, four interpreters take the arithmetic job from 507 ms to 208 ms while four threads do nothing at all. On a build without the lock the ranking flips and threads win at 179 ms against 359 ms, because threads share their objects and interpreters have to copy theirs. The data heavy job is destroyed on both builds, 6 ms one at a time against 255 ms across four interpreters, because there the crossing is the job. Both recordings also time the channel on its own, so the ratio has a cause attached rather than being a bare number.

The lambda row in the second cell is my favourite thing in the lesson. is_shareable says no and pickle.dumps says no and the queue carries it anyway, which is a way to see the three route fallback chain in _PyObject_GetXIData without reading any C at all.

The PR also corrects C07's closing paragraph, which promised a C08 about what a second interpreter is. That was C04's material. This is the fourth time a closing paragraph written before the next lesson was researched has turned out to be wrong, so from here the closing paragraph gets written last, after the source reading for the following lesson is done.

Where this leaves the milestone. The eight C lessons are done and the nine R lessons are not started: startup, runtime and interpreter state, the import system, frozen modules, PEP 810 lazy imports, the C API tiers, the stable ABI including abi3t, and shutdown. Still outstanding as well are the blueprints BP-FT-OBJECT, BP-STATE, BP-IMPORT and BP-CAPI-SURFACE, the gil_lab app, and four animations.

Worth noting against the exit criterion. It asks that every concurrency lesson runs on both builds with the results reported side by side from the same script, and that is now true for every lesson that has anything build dependent to report. There are fifteen Tier 1 recordings across C01 to C08, fourteen of them in pairs running the same program on both builds through the same image pipeline, plus one in C03 for a daemon thread question that only has one answer.

Next up is R01, startup. Release v0.0.51 covers C08.

Add R01, before your first line (#178)

R01 is merged in #178, which starts the runtime half of this milestone.

The lesson is about everything that has already happened by the time your first statement runs. Eight cells, all of which start a child interpreter and ask it a question rather than looking at the notebook, because a notebook has been up for minutes and imported hundreds of modules and cannot be a witness about its own startup.

Four things in it are worth writing down here.

With site out of the way, none of the modules loaded at startup were read from a file. They are either C compiled into the binary or Python bytecode frozen into it. That is not an optimisation, it is the only way out of a circle: the interpreter cannot import os from a file until it knows where files are, and it cannot know where files are until it has imported os.

The command line does not beat the environment. The helper that reads a flag out of the environment is if (*flag < value) *flag = value;, and -O increments rather than assigns, so the answer is the highest number anybody asked for. PYTHONOPTIMIZE=2 in a shell profile survives passing -O today and nothing tells you. -E is the only way out, and it works by deleting the environment rather than outranking it.

sys.path is produced by a Python program. Modules/getpath.py is compiled to bytecode when CPython is built, stored in the binary as a marshalled blob, unmarshalled during main startup and evaluated with eleven C functions standing in for the os.path it cannot import. Its output always includes a pythonXY.zip that usually is not there, which the cell makes obvious by printing an exists check next to every entry.

The front of sys.path is added after startup is over. That is the complete mechanism behind a local random.py shadowing the standard library, and -P fixes it by never adding the entry rather than by removing it.

The recordings are the first release against debug pair in the book, and they turned up something better than what I went looking for. The debug build starts in 56.5 ms against 26.5 ms, and I expected to attribute that to assertions and reference count bookkeeping. The module counts say otherwise. Release reports 17 frozen and 0 from a file. Debug reports 3 frozen and 14 from a file. A debug build turns frozen modules off in the defaults, one #ifdef, so that anybody debugging CPython steps through the real Lib/os.py. The import bill goes from 11.6 ms to 30.8 ms with it, which is most of the gap.

Also in the PR: a new glossary group for startup and shutdown, with two phase initialisation, path configuration and safe path. GLOSSARY.md is 216 terms and CLAIMS.md is 564 claims across 72 lessons.

Where this leaves the milestone. All eight C lessons are done, R01 is done, and R02 through R09 are not started: the runtime and interpreter state, the import system, frozen modules, PEP 810 lazy imports, the C API tiers, the stable ABI including abi3t, and shutdown. Still outstanding as well are the blueprints BP-FT-OBJECT, BP-STATE, BP-IMPORT and BP-CAPI-SURFACE, the gil_lab app, and four animations.

Next up is R02, where all the state this book has been poking at for seventy two lessons actually lives. R01's closing paragraph only promises what the checklist above already commits R02 to, which is a weaker version of the rule I set last time but is at least not a guess. Release v0.0.52 covers R01.

Add R02, where the state lives (#180)

R02 is merged in #180. That is two of the nine runtime lessons.

The lesson is about the three levels a running Python keeps state at: a runtime that there is one of per process, interpreters inside it, and threads inside those. Everything this book has taught you belongs to exactly one of the three, and the level decides who can see a change when you make one. Rather than assert that, every section makes a second interpreter, asks both of them the same question, and looks for where the answers stop matching.

Four things in it are worth writing down here.

The shared pile is small and it is structural. None, the booleans, the small ints, the one character strings, the fixed identifier strings and the static type objects are fields of the runtime struct rather than allocations. Nothing allocated them, so nothing can free them, which is why they are immortal without needing any special handling, and because there is one runtime per process every interpreter ends up pointing at the same bytes. Everything else came out with two different addresses.

Where the small int run ends is a #define rather than a policy, and the cell finds the edge by asking both interpreters for the address of every int in a range instead of looking it up. It is minus five to 256 on 3.14 and minus five to 1024 on 3.15. That is _PY_NSMALLNEGINTS and _PY_NSMALLPOSINTS, and it means a great deal more integer arithmetic never allocates on 3.15 than on 3.14. It is declared on the cell, so the versions job checks it stays true on both.

Signals are the cleanest thing in the lesson. The operating system has never heard of interpreters, so the handler table has to be one array per process, and one table means one writer. The test is _Py_IsMainThread() && _Py_IsMainInterpreter(interp), one condition for each of the two upper levels, and signal.signal calls it before it does anything else. The error message you have probably seen says main thread of the main interpreter, which is the C read out loud. The cell tries all three positions and the two failures come back with that exact string.

The recordings price it. An operating system thread takes a few hundred microseconds. A whole interpreter takes about fifty times that, and keeps a couple of megabytes of resident memory for as long as you hold it. The free threaded build charges more for both, and that is the allocator rather than noise: a build with the lock gives each interpreter its own obmalloc pools and a free threaded build uses mimalloc heaps, so an extra interpreter is not the same price on the two of them.

One thing did not go to plan. While working out which cells were safe to run I found that sys._current_frames() called from inside a subinterpreter aborts the whole process on a build with the GIL, and it is not catchable. _PyThread_CurrentFrames walks every interpreter's thread states and materialises frame objects for frames it does not own, so an object allocated against one interpreter's pools gets freed against another's. The free threaded build survives it, which is the clue. That is #179. The lesson names it rather than quietly avoiding the function, because a function that reads the whole runtime instead of the current interpreter is exactly what the lesson is about.

Also in the PR: two glossary terms, runtime state and static object. GLOSSARY.md is 218 terms and CLAIMS.md is 571 claims across 73 lessons.

Where this leaves the milestone. All eight C lessons are done, R01 and R02 are done, and R03 through R09 are not started: the import system, frozen modules, PEP 810 lazy imports, the C API tiers, the stable ABI including abi3t, and shutdown. Still outstanding as well are the blueprints BP-FT-OBJECT, BP-STATE, BP-IMPORT and BP-CAPI-SURFACE, the gil_lab app, and four animations.

Next is R03, on the import system. R02 ended by noting that a brand new interpreter imports dozens of modules of its own before it will run a line, which is the natural way in. The import system is the largest piece of Python written in Python and it has a documented protocol you can extend, so R03 starts with the four steps between import x and the module appearing in sys.modules. Release v0.0.53 covers R02.

Add R03, what import does (#181)

R03 is merged in #181. That is three of the nine runtime lessons.

The lesson is about what one import statement actually does. The framing is that import is not a keyword doing something the language will not explain: it compiles to a call to an ordinary builtin, that builtin is written in Python in a file anybody can open, and every step it takes is reachable from inside the language. So the lesson watches rather than describes, ten code cells of it.

Five things in it are worth writing down here.

The statement compiles to two opcodes and nothing else. IMPORT_NAME looks the name __import__ up in the builtins of the current frame on every single import, which is exactly why replacing builtins.__import__ works and is not a hack. Compiling the four spellings side by side also settles the question people get wrong most often: import os.path asks for os.path and stores os, and the submodule is reached through an attribute the machinery sets on the parent. A relative import compiles to an IMPORT_NAME with the empty string as its argument and the number of leading dots as its level.

The three finders on sys.meta_path are asked in order and the first to answer wins, which means a name can have more than one answer. os is the case that matters: it exists as a file, PathFinder would happily find that file, and it never gets asked, because FrozenImporter says yes first. import os does not open os.py. That is the hand off to R04.

The module object goes into sys.modules before its body runs. This is the part that explains more real bugs than anything else in the lesson, and it is two lines of _load_unlocked, 908 and 918. Publishing early is what makes circular imports possible at all, because otherwise two modules that import each other would loop forever, and it also decides exactly how much of a half loaded module the other side can see: whatever was defined by the line that triggered the import, and nothing after it. The cell catches a module in the act. If the body raises, the entry is deleted again, so a failed import does not leave a broken half module for the next attempt to find.

Three caches sit in front of a repeat import: sys.modules keyed by name, sys.path_importer_cache keyed by sys.path entry, and a directory listing inside each FileFinder. The second of those is the one that bites. A directory that was not there when it was first looked for is remembered as None, so creating it afterwards does not help and importlib.invalidate_caches() is not optional for anything that generates code at runtime. The cell shows all three states.

The recordings settle the import lock, which almost everybody has wrong. It is one lock per module name and has been since 3.3, not one lock for the process. Four threads importing four different modules keep 1.02 cores busy on a release build and 3.63 on a free threaded one, so what serialises them is the GIL rather than the import lock. Four threads importing the same module keep almost exactly one core busy on both builds, and the counter at the end of the program says the body ran once, which is the per module lock doing precisely what it says on the tin. A hit in sys.modules costs 282 nanoseconds with the lock and 507 without, which is the same uncontended locking tax C06 and C07 found on everything else.

That experiment took two goes and the first one was wrong in a way worth recording. It compared wall clock for four sequential imports against four threaded ones and reported a ratio that would have meant bytecode running in parallel with the GIL held. Adding time.process_time() showed what was really going on: macOS puts one busy thread on an efficiency core and promotes four runnable ones, so the sequential case was being handicapped by the scheduler rather than by anything in CPython. Measuring processor time divided by wall clock takes the machine back out of the answer, and the control case reads 1.00 on both builds. Worth remembering for any future experiment on this laptop that compares one thread against several.

Also in the PR: four glossary terms, module spec, meta path finder, path entry finder and module lock. Six diagrams and seventeen citations. GLOSSARY.md is 222 terms and CLAIMS.md is 580 claims across 74 lessons.

One small first. All ten cells run end to end on Pyodide with nothing skipped, which no earlier R lesson managed, and the browser run is more interesting than the native one in two places: json comes back loaded by a zipimporter instance because the standard library is in a zip, and the directory listing has no __pycache__ because nothing is writing bytecode.

Where this leaves the milestone. All eight C lessons are done, R01 through R03 are done, and R04 through R09 are not started: frozen modules, PEP 810 lazy imports, the C API tiers, the stable ABI including abi3t, and shutdown. Still outstanding as well are the blueprints BP-FT-OBJECT, BP-STATE, BP-IMPORT and BP-CAPI-SURFACE, the gil_lab app, and four animations.

Next is R04, on frozen modules. R03 ended on import os never opening os.py, so R04 picks that up: how the bytecode for the importlib bootstrap gets compiled during CPython's own build and written into the binary as a C array, where the loop is cut so that an import system written in Python can be imported without an import system, and what all of that is worth in startup time. Release v0.0.54 covers R03.

Add R04, frozen modules (#182)

R04 is merged in #182. That is four of the nine runtime lessons.

R03 ended on a fact it deliberately did not explain: import os never opens os.py, because FrozenImporter is asked before PathFinder and answers first. R04 is about why that module was already in the binary, and about the problem freezing exists to solve.

The problem is a real chicken and egg. The import system is written in Python, in Lib/importlib/_bootstrap.py, so there is no way to import it. CPython cuts the loop by compiling that file during its own build, marshalling the code object, and writing the bytes out as a C array that the C compiler puts in the binary. The lesson opens by proving it rather than asserting it: the compiled module body of _frozen_importlib contains zero IMPORT_NAME opcodes, which is exactly what makes it loadable with nothing running, while _frozen_importlib_external, which is loaded second and by then has an import system to use, contains eight. init_importlib builds _imp by hand and hands sys and _imp in as arguments, which is why _bootstrap.py never writes import sys anywhere in it.

Five things worth writing down here.

Thirty three names are frozen in a stock 3.15 build, in three arrays that the flag treats differently. Three are the import system and no setting can remove them, and the reason is four lines of C: look_up_frozen walks the bootstrap array unconditionally and only the other two arrays sit behind a check of the flag. Nineteen are what a bare startup needs, which is an odd looking list until you remember it is not the modules people use most, it is the modules that get imported before your first line runs. Eleven are __hello__ and its variations, which exist so CPython's own test suite has something to test the machinery on.

A frozen module knows perfectly well which file it would have opened. FrozenImporter.find_spec works the source path out from sys._stdlib_dir and parks it on the spec as loader_state, which is a field ModuleSpec sets to None and never looks at again, so it belongs to whichever loader filled the spec in. The loader then copies it onto __file__. The result is that __spec__.origin says frozen, __spec__.cached says None, and __file__ says a real path on disk, and all three are true at once. That is the whole reason a traceback through frozen code is readable. When linecache sees a filename starting with <frozen , it ignores it and reads __file__ out of the module globals it was handed instead, and inspect has its own copy of the same fix up.

The cost is not where you would guess. It is tempting to think frozen wins because unmarshalling out of memory beats unmarshalling out of a file, but both paths end in the same marshal.loads over the same bytes and that call is most of the bill. What freezing removes is the finder search and the file read in front of it. The -v cell settles the other half of it: every file the flag-off startup reads that the flag-on one does not is a .pyc for a module in the frozen standard library group, and a .pyc is already compiled, so freezing is not saving compilation either. It is saving a dozen rounds of asking finders, listing directories, opening files and checking timestamps.

The two builds disagree about the default, and finding that out is what shaped the experiment. initconfig.c sets use_frozen_modules to zero under Py_DEBUG, so a debug build ships with frozen modules off unless you ask for them, which is a deliberate developer experience choice: someone editing Lib/importlib/_bootstrap.py on a debug build wants their edit to take effect. The first version of the Tier 1 program relied on the default and died on the debug build with No such frozen object named 'os'. The rewrite never relies on it. It asks for on and off explicitly on every child process and reports the default as one more measured fact. Freezing gives back 14.2 percent of a release startup and 9.9 percent of a debug one.

One cell had to be rewritten after its numbers refused to reproduce, and that is worth recording alongside the R03 note about macOS core assignment. The cell originally claimed loading frozen cost about half of reading the .pyc, and a standalone probe agreed at 274.9 against 578.9 microseconds. Under a real IPython kernel the same measurement came back 438.0 against 432.8, a ratio of 1.0. The gap was noise around the unmarshal, which is identical on both paths. The cell now times four steps separately instead of comparing two totals, and the claim says what the numbers support. Both recordings also alternate the two cases round by round rather than running one case forty times and then the other, which is the same page cache trap C07 and R03 hit.

Also in the PR: four glossary terms, frozen module, import bootstrap, loader state and module alias. Six diagrams and twenty one citations. GLOSSARY.md is 226 terms and CLAIMS.md is 589 claims across 75 lessons. Nine of the ten cells run end to end in a browser, and the tenth needs a second process so it prints a line saying so.

Where this leaves the milestone. All eight C lessons are done, R01 through R04 are done, and R05 through R09 are not started: PEP 810 lazy imports, the C API tiers, the stable ABI including abi3t, and shutdown. Still outstanding as well are the blueprints BP-FT-OBJECT, BP-STATE, BP-IMPORT and BP-CAPI-SURFACE, the gil_lab app, and four animations.

Next is R05, on PEP 810 lazy imports, which is the natural follow on. R03 showed what an import costs and R04 showed what freezing takes off that bill for the standard library. R05 is about the other answer to the same question: not doing the import until somebody actually touches the name. Release v0.0.55 covers R04.

R05, lazy imports, is merged in #183. That is the fifth of the nine runtime lessons.

PEP 810 landed in 3.15 and added one soft keyword to the import statement. The lesson weighs what an unused import costs, shows the five line LazyLoader dance you had to write before 3.15, and then disassembles four import statements to show that a lazy import compiles to the same IMPORT_NAME opcode as a plain one, with the name index shifted up by two bits and the mode carried in the two bits underneath.

The middle of it works in a temporary directory on sys.path, so every module it touches is one the cell above wrote and nothing else in the process has already imported it. That is what lets it show the placeholder honestly: five fields rather than a module, absent from sys.modules, present in sys.lazy_modules, and left alone by a dict lookup, a membership test or a repr because only two opcodes know about it. It then compiles six spellings of the keyword to see which four the symbol table refuses and with what message, turns the behaviour on without the keyword using both sys.set_lazy_imports_filter and a __lazy_modules__ list, and reads the two part traceback that comes back when a module raises during resolution.

Three recordings back the measured parts. Deferring is worth about three quarters of a run on a file that imports twelve modules and uses one, with 54 modules in sys.modules rather than 189 and 15 code objects read off disk rather than 110.

The finding I did not expect is that waking a placeholder does not take the per module name lock an ordinary import takes. _PyImport_LoadLazyImportTstate takes the interpreter wide import lock and holds it for the whole resolution, module body included. On a free threaded build, four threads importing four different modules the ordinary way keep 3.60 cores busy, and four threads waking four different deferred imports keep 0.98. Filed separately as #184.

Everything lazy is 3.15 only, CI runs notebooks on 3.14 and Pyodide is 3.14.2, so every lazy cell is guarded and prints one honest line rather than going silent.

The same PR carries a fix to R03. Re-recording the browser probe turned up an intermittent ModuleNotFoundError there, because the Emscripten filesystem keeps whole second timestamps and the directory mtime does not always move when a file is written into it, so FileFinder keeps serving a stale listing.

Also in: six diagrams, three glossary terms (lazy import, import placeholder, global import lock), rows in both READMEs, and the refreshed citation lock, claims list and probe recording.

Next on this milestone is R06, the C API tiers.

R06, the C API tiers, is written and merged in #185.

This is the first lesson in the book that looks at CPython from the C side. The C API is three directories and two macros: Include/ is open to any extension, Include/cpython/ needs Py_LIMITED_API to be undefined, and Include/internal/ starts nearly every file with three lines that stop the compiler unless you define Py_BUILD_CORE. On 3.15 that third directory is 148 files and 42552 lines, more than the other two put together.

The lesson measures the guard rather than describing it. Walking the preprocessor stack over the public headers gives 580 functions a limited build may call and 186 it may not, plus 974 in the other two directories it never opens. The point it then makes is that the missing functions are the cheap part. The struct layouts go too, so Py_TYPE stops being a field read and becomes a call, and Py_DECREF becomes a call to _Py_DecRef.

The finding worth recording is that none of the tiers survive into the binary, and that this is deliberate. Inside Include/internal/, 530 names are declared with PyAPI_FUNC and 493 of them resolve through ctypes.pythonapi on the release build, against 757 declared with a plain extern of which 3 do. Both spellings sit in the same files, often two lines apart, and 168 comments name which bundled shared extension needs each export. The free threaded recording finds six more, which are the ones behind Py_GIL_DISABLED.

_PyDict_SizeOf gets called by hand through ctypes and returns exactly what dict.__sizeof__ returns, 16 bytes short of sys.getsizeof because that adds the collector header. Going round the outside gives the same answer the front door does, which is the honest summary of what the internal tier is: not hidden, just unsupported.

Two Tier 1 recordings, six diagrams, four glossary terms, and five differs= notes checked against a real 3.14 run. Half the cells need the headers, which a browser tab does not have, so they sit behind one flag and say so; the other half only needs ctypes, so all eleven probed cells run under Pyodide.

R07 picks up the last cell. EXTENSION_SUFFIXES is three entries on 3.14 and six on 3.15, and the new ones end in abi3t.

Next up is R07, the stable ABI.

R07, the stable ABI, is merged in #187.

This one is about what happens after R06. R06 settled which C functions an extension may call. R07 asks what stands between the file you compiled and a working module on somebody else's machine, and the answer is two gates that both turn out to be drivable from Python.

The first gate is the file name, read before anything is opened. The tag has to be one of the six in the C array at Python/dynload_shlib.c:39-60, and the lesson drives it by writing empty files into temporary directories and asking the finder what it sees. The rule falls out on its own: a free threaded build takes abi3t and refuses abi3, an ordinary 3.15 build takes both, and 3.14 has never heard of abi3t. Putting them all in one directory shows the order comes from that array, and that a compiled file shadows a Python file of the same name.

The second gate is PyABIInfo, twelve bytes and new in 3.15. PyABIInfo_Check is a plain exported function and the struct is five integers, so the lesson builds one in ctypes and calls the real thing. Eight hypothetical extensions go in and four refusals come back, each for a different reason, printed in CPython's own wording. The lesson also says plainly how weak the gate is: it is opt in, and the extension's own code has already run by the time it fires.

The last two cells date the stable ABI by reading the version gates around the declarations. 173 functions added since 3.2, busiest in the last three releases, and the 528 that were there in 3.2 are not on the chart because there is nothing to gate on.

Two Tier 1 recordings run both gates on a release build and on a build made with --disable-gil. Six tags become four and four refusals become six, which is the argument for abi3t in two files.

Ten notebook cells, all ten clean under Pyodide, six of them doing real work there. Six diagrams, three glossary terms, and a new issue filed at #188 for a pyxray reader that would let the lesson use the real extensions on the reader's own disk instead of made up ones.

R08 and R09 to go on this milestone.

R08, when the interpreter stops, is merged in #189. That leaves R09 as the last lesson in this milestone.

Shutdown turned out to be a good subject for a lesson made entirely of child interpreters, since a notebook cannot watch its own ending. The cells walk the order of the end with sys.is_finalizing() printed next to each step, show atexit callbacks coming back newest first and one registered during shutdown that never runs, ask a late __del__ what it can still reach, let both a callback and a finalizer raise so you can watch the exit status stay at zero, catch the single generation two collection, and leave a subinterpreter open so its warning turns up.

The finding worth keeping is the daemon thread. Everyone knows a daemon thread is not waited for. What is less obvious is that its stack frame holds a reference to the globals of the module its function was defined in, so if you pass a function of your own the module dict is never cleared and none of your finalizers run. Pass time.sleep instead and everything is freed as normal. The debug recording puts a number on it: 12680 references still alive at the end, against zero for the same five thousand objects held any other way.

Two Tier 1 recordings, a release build and a debug build, running the same nine endings. Twelve citations. Six diagrams. Three glossary terms: interpreter finalisation, atexit callback, and finalizing flag.

One thing R08 does not do well is run in a browser. All nine of its code cells say the runtime cannot start a process, which is the honest answer under Pyodide but leaves nothing to play with. Filed as #190.

Released as v0.0.59.

R09 is merged in #191, which finishes the lessons in this milestone. C01 through C08 and R01 through R09, seventeen of them, all written and merged.

R09 is the one that crosses over and writes C. Four rules an extension owes the runtime, each broken on purpose and then fixed: hand back what you own, tell the collector what you are holding, do your cleanup in the slot meant for it, and say honestly whether you are safe without the lock. The point of the lesson is that breaking any of the four is silent. No error, no warning, no crash, just a program that works and slowly stops working.

The middle section is the one worth reading if you only read one. The same box type is compiled three ways: no GC flag, the flag with a tp_traverse that reports its type and forgets its field, and the flag with a complete one. Only the third is ever freed out of a cycle. Half is registered with the collector, gets asked what it is holding, gives an answer that is merely incomplete, and leaks exactly as thoroughly as the type that was never registered. subtract_refs cannot tell an incomplete answer from a live reference from outside, and there is nothing in the runtime that can check your answer for you.

Two Tier 1 recordings, which brings the total to 45. On the debug build, CPython's own leak hunter over four small tests that all pass an ordinary run, two of which it fails and one of which it excuses because the thing it saw was a cache filling on first use rather than a leak. On the free threaded build, one shared object from CPython's test suite loaded three ways, where the load that declares Py_MOD_GIL_USED turns the lock back on for the whole process and prints a warning naming the module, and the same load under PYTHON_GIL=0 leaves it off and says nothing.

Also three glossary terms, traverse function, clear function and module state, which takes the glossary to 242.

What is left in this milestone is not lessons any more. Four blueprints, BP-FT-OBJECT, BP-STATE, BP-IMPORT and BP-CAPI-SURFACE, the gil_lab app, and four animations. BP-CAPI-SURFACE still wants #186 first, since a generated tier inventory is the thing it would be describing.

Navigation Menu

Repository navigation

M8 Concurrency, runtime and the C API #27

Content

Blueprints

Artifacts

Standing decision

4 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

M8 Concurrency, runtime and the C API #27

Description

Content

Blueprints

Artifacts

Standing decision

Activity

tamnd commented on Sep 2, 2026

tamnd commented on Sep 2, 2026

tamnd commented on Sep 2, 2026

tamnd commented on Sep 2, 2026

tamnd commented on Sep 2, 2026

4 remaining items

tamnd commented on Sep 5, 2026

tamnd commented on Sep 5, 2026

tamnd commented on Sep 5, 2026

tamnd commented on Sep 5, 2026

tamnd commented on Sep 5, 2026

tamnd commented on Sep 6, 2026

tamnd commented on Sep 6, 2026

tamnd commented on Sep 6, 2026

tamnd commented on Sep 6, 2026

tamnd commented on Sep 6, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions