Skip to content

Clear annotation_locals after collection#776

Open
tobiasraabe wants to merge 3 commits intomainfrom
codex/clear-annotation-locals
Open

Clear annotation_locals after collection#776
tobiasraabe wants to merge 3 commits intomainfrom
codex/clear-annotation-locals

Conversation

@tobiasraabe
Copy link
Member

This clears pytask_meta.annotation_locals after collection so task functions stay picklable for process-based parallel backends.

When tasks must be pickled by value (e.g. loop-generated tasks), cloudpickle serializes the function dict, which currently includes the decoration-time locals snapshot. That snapshot can reference non-picklable objects (for example locks), causing TypeError: cannot pickle '_thread.RLock' object.

Changes:

  • Clear annotation_locals at the end of collection (after hooks/logging).
  • Add regression test using threading.RLock() + cloudpickle.dumps(task.function).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant