Skip to content

Cross-thread backward: forward on one thread, .backward() on another fails with a misnamed 'second time' error #78

Closed

Description

@b-re-w

Found by the free-threading audit for #51 (item R3), measured on the 3.13 shim with the GIL on. It is not a free-threading-only issue.

x = torch.ones(3, requires_grad=True)
# forward in a worker thread
y = (x * 3).sum()          # computed in threading.Thread
y.backward()               # main thread
  • Upstream torch 2.13: x.grad == tensor([3., 3., 3.]).
  • Shim: RuntimeError: torch._C eager: Trying to backward through the graph a second time -- the saved intermediate values have already been freed ...

Cause. The eager tape (EAGER in torch_c/src/capture.rs:66-75) is a thread_local. The backward thread finds no tape and reports graph reuse. Upstream's graph lives on the tensors, so any thread can walk it.

This is wrong twice over:

  • the shim refuses something upstream does;
  • the refusal names the wrong cause (AGENTS.md §18).

Done when

  • A cross-thread forward/backward gives upstream's gradient. The likely fix is that the tape becomes reachable from the tensors' grad_fn rather than from the calling thread, or is process-wide under a lock.
  • Until that lands: if the tape is missing because the forward ran on another thread, the error says that, not "a second time".
  • A test runs the forward in a thread and the backward in another, compared against upstream.

Activity

  1. added a commit that references this issue on Oct 16, 2026
  2. b-re-w commented on Oct 16, 2026

    @b-re-w
    MemberAuthor

    Landed in develop via #79

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions