Skip to content

fix: keep render buffers out of thread_local storage on Windows - #462

Merged
PeterStaar-IBM merged 3 commits into
docling-project:mainfrom
usercjr:fix/357-worker-owned-render-buffers
Nov 13, 2026
Merged

PeterStaar-IBM merged 3 commits into
docling-project:mainfrom
usercjr:fix/357-worker-owned-render-buffers

Conversation

@usercjr

@usercjr usercjr commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Resolves #357.

The Windows wheels are built with MinGW, whose emulated TLS frees a thread's storage before the C++ thread_local destructors run. The renderer kept its spare page canvas (thread_local BLImage) and its CMYK memo (two thread_local std::vectors) in such objects, so each render worker that exited destroyed them on freed memory. The result is an access violation or heap corruption at worker exit, after the page has rendered.

  • The spare canvas is now a std::shared_ptr<BLImage> owned by docling_threaded_renderer::worker_loop(), next to the FreeType and glyph-bbox caches it already owns, and passed to each page renderer through a new optional constructor argument. A renderer built without one gets its own, so the other constructors behave as before.
  • The CMYK memo is now zero-initialised thread_local arrays, which have no destructor. Slots store key + 1, so 0 marks an empty one. The key is 33 bits, so key + 1 doesn't wrap.

The remaining thread_locals in the tree are trivially destructible and unaffected.

The same fault was reported independently on 2026-11-10 in #357 by a second user, on Windows 11 with the PyPI win_amd64 wheel, with minidumps: in 12 of 14, an access violation at the same address inside pdf_parsers.cp313-win_amd64.pyd, on a thread other than the main one, and in their faulthandler reports a faulting thread that ran no Python code. That is consistent with this fault: the release it used (7.22.3) already has these thread_locals.

Evidence:

17898535961965146676-4.pdf p1 from the regression dataset, already covered by soft-masked-drop-shadow in test_regression_pypdfium_cited_pages.py. Rendered with tests.pdf_builder.render_page() and one worker thread, 20 runs per build, each in a fresh process (local build, Windows 11, MSYS2 MinGW g++ 15.2, main at 7cf06e6):

build runs that crashed
main 7 / 20
this branch 0 / 20

CI runs the tests on Ubuntu only, so this doesn't show there. A Windows regression worker was proposed in #332 and closed on 2026-11-10, so the Windows-only test below will skip on every current CI job.

This changes no pixels. Against pypdfium2, with the page selection of test_regression_pypdfium_render.py (954 pages of 457 documents), mean_abs_error and changed_pixels_ratio are identical on every page between main and this branch, as recorded to 5 and 6 decimals. Corpus mean: 2.45693 on both. Pages above the 9.0 tolerance: 13 on both. Since nothing changes visually, there are no comparison images, and the enforced test below checks for the crash rather than comparing against pypdfium2.

Tests:

  • tests/test_unit_render_worker_exit.py (Windows only) runs 120 rendered loads of a small 3-page PDF filled in DeviceCMYK, in a subprocess with faulthandler enabled, and fails on any access violation or a non-zero exit. On main it failed 5 of 5 runs with 5 to 8 access violations each (one run died outright with 0xC0000005); on this branch it passed 4 of 4 with none. It takes about 85 s. An RGB fill only holds the spare canvas and faulted in 4 of 5 runs, which is why the page is CMYK.
  • pytest tests/test_unit_*.py on Windows: main 640 passed, 61 skipped, and prints 3 access violations; this branch 641 passed, 61 skipped, and prints none. Two tests that feat: working on parallel efficiency #435 added fail on Windows, the same on main and on this branch: test_reference_preflight_handles_native_abort[-6] (signal.Signals(6) doesn't exist there) and test_mid_run_commit_requires_exact_source_equivalence[None].

Ground truth changes:

None.

Checklist:

  • One topic per pull request, rebased on main, no merge commits (of the open pull requests, only feat: improving memory footprint of docling #459 touches src/render/blend2d_renderer.h, in other functions; the branches merge cleanly. Checked per pull request by changed files on 2026-11-13)
  • Superseded pull requests closed and linked (none)
  • Regression dataset documents and pages named
  • Before/after evidence for each cited page
  • Rendering only: pypdfium2 comparison images, corpus-wide metrics table, and an enforced image regression test (crash fix with no pixel changes: corpus table and crash test above, no images)
  • Blend2D only: link to the upstream report (not applicable)
  • Changes to existing ground truth listed with the reason for each page

This PR was prepared with Claude Code (Claude Opus 5.5). The measurements and tests above were run locally on my machine.

🤖 Generated with Claude Code

The Windows wheel is built with MinGW, whose emulated TLS frees a thread's
storage before the C++ thread_local destructors run. The renderer kept its
spare page canvas (a BLImage) and the CMYK conversion memo (two
std::vectors) in thread_local objects, so every render worker that exited
destroyed them on freed memory. Windows swallows the fault in the loader's
thread-detach callback and the process continues with a corrupted heap,
or segfaults later.

Give the canvas pool to the render worker, which already owns its
FreeType and glyph-bbox caches, and pass it to each page renderer; a
renderer constructed without one gets its own. Make the CMYK memo plain
zero-initialised arrays, which have no destructor, and store key + 1 so
that 0 marks an empty slot.

Fixes docling-project#357.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: usercjr <92197541+usercjr@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

✅ DCO Check Passed

Thanks @usercjr, all your commits are properly signed off. 🎉

@mergify

mergify Bot commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

@PeterStaar-IBM
PeterStaar-IBM self-requested a review November 13, 2026 06:37
…render-buffers

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>

@PeterStaar-IBM PeterStaar-IBM left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@PeterStaar-IBM
PeterStaar-IBM merged commit 88f1c38 into docling-project:main Nov 13, 2026
43 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Windows: threaded renderer corrupts the heap when its worker threads exit (thread_local destructors run after MinGW emutls frees their storage)

2 participants