Skip to content

Fix pickling errors in parallel workers#137

Merged
tobiasraabe merged 9 commits intomainfrom
fix/issue-136-pickling
Feb 5, 2026
Merged

Fix pickling errors in parallel workers#137
tobiasraabe merged 9 commits intomainfrom
fix/issue-136-pickling

Conversation

@tobiasraabe
Copy link
Member

@tobiasraabe tobiasraabe commented Jan 31, 2026

Closes #136

Summary

  • pickle task modules by value only when not importable to avoid serializing module globals
  • ensure worker processes can import task modules by setting root on sys.path/PYTHONPATH
  • add regression tests for mark import and closed file handles

@tobiasraabe tobiasraabe added this pull request to the merge queue Feb 5, 2026
github-merge-queue bot pushed a commit that referenced this pull request Feb 5, 2026
* Fix pickling errors by importing task modules in workers

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Document fix for pickling errors

* Fix typing, lint, and docs checks

* Allow global worker root with ruff ignore

* Add regression test for mark import loop

* Strip local from task.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Release 0.5.2

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to no response for status checks Feb 5, 2026
@tobiasraabe tobiasraabe added this pull request to the merge queue Feb 5, 2026
Merged via the queue into main with commit ad193e2 Feb 5, 2026
21 checks passed
@tobiasraabe tobiasraabe deleted the fix/issue-136-pickling branch February 5, 2026 18:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PicklingError when running pytask-parallel in a module with YAML file loaded in context manager.

1 participant