Skip to content

Process-global state that upstream keeps per thread: grad mode, device-mode stack, capture flag; RNG stream free race #76

Closed

Description

@b-re-w

Found in the free-threading audit for #51; I checked these myself against the code. They are wrong today, with the GIL on: the GIL serialises bytecode, it does not make process-global state per thread.

  1. Grad mode is process-wide; upstream keeps it per thread. It lives in torch_c/src/tensor.rs:4054 (static GRAD_ENABLED: AtomicBool, NoGradGuard swap/restore at :4125-4137) and in the bootstrap's state dict (bootstrap.py ~7183). torch.no_grad() in thread A stops grad_fn recording in thread B, and two guards that overlap across threads can restore the wrong value. Upstream: GradMode is thread-local (c10/core/GradMode.h).
  2. RNG stream use-after-free window (torch_c/src/rng.rs:263-298). stream() copies the leaked &'static Mutex<CpuGenerator> out of STREAMS, then drops the table lock before locking the generator. stream_free (from __del__) can Box::from_raw it in between. The SAFETY comment's argument is "under the GIL", and that no longer holds on 3.15t.
  3. CAPTURING is a single global flag (src/capture.rs:64, set :1741, cleared :1756), but the recorder is thread-local. One thread ending its capture drops another thread's mid-capture ops from its trace.
  4. _MODE_STACK is process-wide (bootstrap.py:4978, 4997-5018). with torch.device(...) leaks into other threads, and the pop-for-duration can raise IndexError in another thread.

Done when

  • Grad mode, the device-mode stack and the capture flag are per thread, as upstream's are.
  • RNG streams are Arc-held, so freeing one cannot invalidate a live borrow.
  • A threaded test (8 threads, Barrier) fails before each fix and passes after, on 3.13 with the GIL and on 3.15t with PYTHON_GIL=0.

The full audit (28 safe / 7 unsafe / 9 unclear, with file:line) feeds #51's thread hammer.

Activity

  1. b-re-w commented on Oct 5, 2026

    @b-re-w
    MemberAuthor

    Audit follow-up, not fixed yet: B6 _DISPATCH_REGISTRATIONS (bootstrap.py:2014-2075, torch._C._dispatch_get_registrations_for_dispatch_key) is built lazily into the shared dict.

    • The if _DISPATCH_REGISTRATIONS: return check runs before the table is complete, so a second thread can get back a partial table.
    • Two threads filling it at once append each name twice.
    • Fix: build locally, then publish under a threading.Lock.
    • Left open because I have no failing test for it yet: it needs the source read slowed down or hooked.
  2. b-re-w commented on Oct 5, 2026

    @b-re-w
    MemberAuthor

    Cross-thread backward (audit item R3) was measured as a real divergence from upstream and filed as #78.

  3. added a commit that references this issue on Oct 16, 2026
  4. b-re-w commented on Oct 16, 2026

    @b-re-w
    MemberAuthor

    Part 1 (per-thread grad mode and device-mode stack) landed in develop via #77; capture flag and RNG stream free race remain.

  5. added a commit that references this issue on Oct 17, 2026
  6. b-re-w commented on Oct 17, 2026

    @b-re-w
    MemberAuthor

    Landed in develop via #112 (B6); capture flag and RNG streams via #70, grad mode and device stack via #77

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions