Skip to content

Windows: threaded renderer corrupts the heap when its worker threads exit (thread_local destructors run after MinGW emutls frees their storage) #357

Closed

Description

@mickey-mikey

Environment

docling-parse 7.21.0 (win_amd64 wheel), Python 3.12, Windows 11, and GitHub's windows-latest runner. Linux and macOS are unaffected. Docling 2.128 and later reach this through the default ThreadedDoclingParseDocumentBackend, which renders pages by default (render_pages=True).

Symptom

faulthandler prints Windows fatal exception: access violation with no Current thread line, and the process keeps running. Later, unrelated Python objects are corrupt, or the process segfaults. For example, we saw re.error on a valid pattern, a pydantic error about a missing attribute on a RefItem, and a DoclingDocument failing validation with "Duplicate ref".

Minimal reproduction (docling-parse only)

import faulthandler, sys, tempfile, pathlib
faulthandler.enable()
import pypdfium2 as pdfium
from docling_parse.pdf_parser import DoclingThreadedPdfParser, ThreadedPdfParserConfig, DecodeConfig, RenderConfig

render = sys.argv[1] == "render"
path = pathlib.Path(tempfile.mkdtemp()) / "blank.pdf"
doc = pdfium.PdfDocument.new()
for _ in range(3):
    doc.new_page(612, 792)
doc.save(path)
doc.close()

for i in range(1000):
    parser = DoclingThreadedPdfParser(
        ThreadedPdfParserConfig(threads=4, render_config=RenderConfig() if render else None), DecodeConfig()
    )
    key = parser.load(path)
    for result in parser.iterate_results():
        pass
    parser.unload(key)
print("done")

On an i7-7700 with Windows 11:

  • python repro.py render: 18 access violations, then a segfault (exit 139) before the loop finished.
  • python repro.py parse: no access violations, and the loop completed.

Through Docling's backend, over 300 loads of a 3-page PDF, rendering faulted 7 times with 4 threads and 4 times with 1. With render_pages=False it didn't fault at either thread count.

Likely cause

  • feat: optimization of the parse/render with up to 3.84× speedup  #333 (7.17.0) added thread_local objects with non-trivial destructors to the Blend2D renderer. These are the canvas pool (thread_local BLImage, src/render/blend2d_renderer.h around line 231) and the CMYK memo vectors (around lines 1419-1420).
  • The win_amd64 wheel is built with MinGW GCC using emulated TLS: pdf_parsers.*.pyd imports __emutls_get_address from the bundled libgcc_s_seh-1.dll. The render workers are std::threads, so they are winpthreads threads.
  • When one of those threads exits, winpthreads runs its key destructors before _endthreadex. These include libgcc's emutls_destroy, which frees the thread's TLS storage. The C++ thread_local destructors registered through the CRT's tls_atexit run later, at DLL_THREAD_DETACH, on memory that has already been freed. The fault happens inside the loader's thread-detach callback, which is why Windows swallows it instead of ending the process. This is the long-standing MinGW problem in mingw-w64 bugs 727 and 859.
  • Workers exit as soon as the task queue is empty, so this can happen on every document.

Suggested fix

Avoid thread_local objects with non-trivial destructors in the renderer on Windows. For example, make the canvas pool and the CMYK memo per-worker objects owned by the worker loop and passed into the renderer, as the FreeType font cache and glyph bbox cache already are (docling_threaded_renderer.h). A Windows CI job, as in #332, would catch a regression: the access violations print on every run.

Workaround

We turn the renderer off with ThreadedDoclingParseBackendOptions(render_pages=False) and render page images with pypdfium2 instead.

Activity

  1. SergiiShcherbak commented on Oct 20, 2026

    @SergiiShcherbak

    Still reproduces with docling-parse 7.22.3 and Docling 2.140.0 (Python 3.14, Windows 11, default ThreadedDoclingParseDocumentBackend): 14 of 27 processes exited with 0xC0000374 (heap corruption) while converting their first PDF, also with parser_threads=1. faulthandler shows the page producer in _timings_from_decoder while the preprocessing stage is in get_image.

  2. usercjr commented on Oct 31, 2026

    @usercjr
    Contributor

    Still reproduces on 7.24.1, and I have a fix along the lines @mickey-mikey suggested, tested on Windows 11 with a local MinGW build.

    Reproduction. Using the script above with render, 300 loads:

    build access violations per 300 loads
    official 7.24.1 wheel 7
    main, local MinGW build 15 / 17, then a segfault / 8 (three runs)
    with the fix, same local build 0 / 0 / 0

    With parse instead of render there are none, as reported.

    Fix (branch ready, not opened as a PR yet). The renderer had three thread_local objects with non-trivial destructors in blend2d_renderer.h:

    • The spare page canvas (thread_local BLImage) becomes a std::shared_ptr<BLImage> owned by the render worker in docling_threaded_renderer.h::worker_loop(), next to the FreeType and glyph-bbox caches it already owns. It's passed to each page renderer through a new optional constructor argument; a renderer built without one gets its own, so the other constructors behave as before.
    • The CMYK memo (thread_local std::vector ×2) becomes zero-initialised thread_local arrays, which have no destructor. Slots store key + 1, so 0 marks an empty one; the key is 33 bits, so key + 1 never wraps.

    The remaining thread_locals in the tree (int, a pointer, a POD timer struct) are trivially destructible and unaffected.

    Test. A Windows-only unit test runs 120 rendered loads of a small 3-page PDF in a subprocess with faulthandler enabled, and fails on any "access violation" or a non-zero exit. On main the subprocess dies with 0xC0000005 (2 of 2 runs); with the fix it passes in ~30 s. pytest tests/test_unit_*.py: 292 passed, 12 skipped. As a side effect, test_unit_bitmap_artifacts.py no longer prints the intermittent access violations it showed on my local builds.

    @cau-git, how would you like this submitted under #428? It changes blend2d_renderer.h, which is in the renderer topic, but none of the hunks of #386, #390, #397 or #413 touch the canvas pool or the CMYK memo. Can it go as a standalone crash fix, or should it wait for that topic? It changes no pixels, so I'd expect the pypdfium2 metrics to be identical; I can include the corpus table to confirm.

    (Prepared with Claude Code; I reviewed the change and ran the tests above myself.)

  3. usercjr commented on Nov 5, 2026

    @usercjr
    Contributor

    A document in the regression dataset reproduces this on current main (f4b12cf).

    I rendered 17898535961965146676-4.pdf p1, one of the cited pages that #439 added, with tests.pdf_builder.render_page() and one worker thread. Each of the 20 runs per build used a fresh process, on a local MinGW build (MSYS2 g++ 15.2, the toolchain the Windows wheels use):

    build runs that died with 0xC0000374 (heap corruption)
    main before #439 6 / 20
    main (f4b12cf) 9 / 20
    main + the fix described above 0 / 20

    So soft-masked-drop-shadow in test_regression_pypdfium_cited_pages.py is flaky on Windows. CI runs the tests on Ubuntu only, which is why it doesn't show there. The renderer changes in #439 only make the crash more likely; on the same page it already happened before them.

    The fix is rebased on main. It includes a Windows-only regression test that fails on main and passes with the fix.

    • pytest tests/test_unit_*.py: 396 passed, 16 skipped, no access violations. Without the fix, the same run prints several, from test_unit_bitmap_artifacts.py and test_unit_ccitt_images.py.
    • It changes no pixels. Over the 940 selected pages of the regression corpus, 938 render byte-identical to the 7.24.1 wheel. The other 2 differ by 1e-5 mean absolute error, the same wheel-vs-local-build noise those pages show without any change.

    @cau-git, the renderer topic is merged and no open PR touches blend2d_renderer.h. Unless you'd rather handle it differently, I'll open this as a standalone crash fix following the #428 / CONTRIBUTING rules.

    (Prepared with Claude Code; I ran the builds and tests above myself.)

  4. ionutvoinea191978-sys commented on Nov 10, 2026

    @ionutvoinea191978-sys

    We see what looks like this fault on docling-parse 7.22.3 with Docling 2.140.0, and can add the native crash position from minidumps, the rates we measured and two small synthetic PDFs. We have not tried 7.24.x or 7.25.0. Everything attached is synthetic (below).

    Environment

    • Windows 11 Pro, version 10.0.26200, x64; 8 cores, 32 GB.
    • CPython 3.13.16 (python.org build, tags/v3.13.16:cbc944f, MSC v.1944), started with -I -X faulthandler.
    • docling 2.140.0, docling-parse 7.22.3 (the win_amd64 wheel from PyPI, pdf_parsers.cp313-win_amd64.pyd with its libstdc++-6.dll, libgcc_s_seh-1.dll and libwinpthread-1.dll), docling-core 2.103.0, docling-ibm-models 4.0.3, torch 2.14.1 (CPU), torchvision 0.29.1, opencv-python-headless 5.0.0.93, numpy 2.5.4, pillow 12.3.0, pypdfium2 5.14.0, pydantic 2.14.1, onnxruntime 1.31.0, rapidocr 3.10.0, transformers 5.20.2. Layout model "heron" (docling-project/docling-layout-heron at 8f39ad3c0b4c58e9c2d2c84a38465abf757272d8) and TableFormer accurate (docling-project/docling-models at fc0f2d45e2218ea24bce5045f58a389aed16dc23, tag v2.3.0), the revisions Docling 2.140.0 names, from local files; no network.
    • Docling's default PDF backend, ThreadedDoclingParseDocumentBackend, with its default options (pages rendered); PdfPipelineOptions(do_ocr=False, do_table_structure=True, table_structure_options=TableStructureOptions(mode=TableFormerMode.ACCURATE, do_cell_matching=True), generate_parsed_pages=True). Our longer runs also used RapidOCR (onnxruntime) and Tesseract on scanned pages; the crash happened with and without OCR.

    Setup (a clean Windows 11 x64 machine; C:\repro is an example folder)

    1. Python 3.13.16 x64 from python.org (the installer, or the python 3.13.16 package on nuget.org, the same build).

    2. The Microsoft Visual C++ 2015–2022 Redistributable x64, 14.44 or later (https://aka.ms/vs/17/release/vc_redist.x64.exe), unless C:\Windows\System32\msvcp140.dll is already there: PyTorch and ONNX Runtime load msvcp140.dll, msvcp140_1.dll, msvcp140_atomic_wait.dll and vcruntime140_threads.dll, which neither Windows nor Python brings. (Or copy those four files into the msvc folder of step 5.)

    3. A venv with the pins (PyPI's Windows torch wheel is the CPU build, so no extra index):

      py -3.13 -m venv C:\repro\venv
      C:\repro\venv\Scripts\python.exe -m pip install docling==2.140.0 docling-parse==7.22.3 docling-core==2.103.0 docling-ibm-models==4.0.3 torch==2.14.1 torchvision==0.29.1 opencv-python-headless==5.0.0.93 numpy==2.5.4 pillow==12.3.0 pypdfium2==5.14.0 pydantic==2.14.1 onnxruntime==1.31.0 rapidocr==3.10.0 transformers==5.20.2
      

      The other dependencies resolve to their latest releases: on 8 November 2026, 19 of them were newer than in our runs (for example pandas 3.1.1 for 3.0.6, python-dateutil 4.0.0 for 2.9.0.post0); we can send our full list.

    4. The two models, at the revisions above (the hf command comes with huggingface_hub, installed in step 3):

      C:\repro\venv\Scripts\hf.exe download docling-project/docling-layout-heron --revision 8f39ad3c0b4c58e9c2d2c84a38465abf757272d8 --local-dir C:\repro\pack\models\docling-project--docling-layout-heron
      C:\repro\venv\Scripts\hf.exe download docling-project/docling-models --revision fc0f2d45e2218ea24bce5045f58a389aed16dc23 --include "model_artifacts/tableformer/accurate/*" --local-dir C:\repro\pack\models\docling-project--docling-models
      

      model.safetensors is SHA-256 00333a43451945aaf89db8ca9c0a17e75d1537c17db60fdb91aa95f4c7929e0c and tableformer_accurate.safetensors 2a7d6c924b3cd12fb99a09280ca9c33a89c5d60b93253617d2e088c1a40374d9.

    5. Create the empty folder C:\repro\pack\msvc (or put the four files of step 2 in it). C:\repro\pack is the <folder> that repro.py takes: it passes <folder>\models to Docling as artifacts_path, adds <folder>\msvc to the DLL search and appends <folder>\site-packages to sys.path, which the venv's Python does not need (the folder need not exist). Give the folders as absolute paths: os.add_dll_directory refuses a relative one (WinError 87).

    Steps

    1. One long-lived Python process converts PDFs one after another with a single DocumentConverter. Before each conversion it also reads the same PDF with DoclingPdfParser(loglevel="fatal") (DecodeConfig(enforce_same_font=True), line cells) and unloads it.
    2. repro.py (attached) does exactly this in a loop, here for 720 seconds: C:\repro\venv\Scripts\python.exe -I -X faulthandler repro.py C:\repro\pack C:\repro\work1 720 minutes-a.pdf minutes-b.pdf. Our run E (below) started four of these at once, each with its own work folder. Each conversion prints a line (1 minutes-a.pdf lines=34 items=11 11.4s). Python 3.13 also prints a SyntaxWarning for \s in the script's docstring at start-up; it is harmless.

    Expected: every conversion returns a document.

    Actual: now and then the process ends with an unhandled access violation (exit code -1073741819, 0xC0000005), or reports one and goes on. Our runs:

    Run Conversions Process crashes Where
    A: 30 synthetic PDFs (scans, native text and tables, mixed), 6 rounds, one process at a time, a debugger attached (for minidumps) 180 8 all 8 at pdf_parsers.cp313-win_amd64.pyd+0x7e7bd7
    B: the same 30 PDFs, 6 rounds, again with a debugger attached 180 6 4 at pdf_parsers…+0x7e7bd7; 2 at python313.dll+0x2b80c0 and +0x2b80dd
    C: the same PDFs without a debugger, in the program that uses Docling (9 profiling runs) 405 (270 PDFs) 0
    D: the same panel's items (PDFs, Word and e-mail) in the program, without a debugger, four later runs 180 1 (a mixed scanned/native PDF) (no dump)
    E: repro.py alone, no debugger, 4 processes at once for 12 minutes, minutes-a.pdf and minutes-b.pdf 162 0, but 2 of the 4 processes reported an access violation they survived faulthandler, below

    With a debugger attached the rate is much higher. The debugger saw docling-parse throw and catch between 24 and 1,660 GCC C++ exceptions (0x20474343) in each watched process, each a round trip to the debugger, which slows the parser's threads and probably widens a race. The two smallest PDFs that crashed are attached: minutes-a.pdf (3,103 bytes, 2 pages, a native text page with a table) crashed two processes in a row in run B, and minutes-b.pdf (3,368 bytes, 2 pages) crashed one. Both convert normally almost every time.

    The crash, from the minidumps (14 dumps; exception record and module list; the stack is a raw scan of the faulting thread's stack for addresses inside loaded modules, from the stack pointer upward, so a line may be a stale value and not a real frame):

    • In 12 of 14: ACCESS_VIOLATION reading 0xFFFFFFFFFFFFFFFF (one: 0xFFFFFFFFFFFFFFF8) at pdf_parsers.cp313-win_amd64.pyd+0x7e7bd7, on a thread that is not the main thread (not the dump's first thread). The same scan in all 12:

      pdf_parsers.cp313-win_amd64.pyd+0x7e7bd7   <- faulting instruction
      cv2.pyd+0x3a72b02
      pdf_parsers.cp313-win_amd64.pyd+0x49b08d
      cv2.pyd+0x3ad9153
      cv2.pyd+0x3a64bda
      pdf_parsers.cp313-win_amd64.pyd+0x49b020
      ntdll.dll+0xe5c7b
      cv2.pyd+0x3a64a91
      cv2.pyd+0x3a39ec8
      ntdll.dll+0xe5c60
      ntdll.dll+0x11ef6a
      org.sw.demo.uclouvain.openjpeg.openjp2-2.5.4.dll+0x46356
      ntdll.dll+0x1b2e3
      

      The ntdll addresses under the docling-parse frames may be the thread-exit path (TLS callbacks), which would fit Windows: threaded renderer corrupts the heap when its worker threads exit (thread_local destructors run after MinGW emutls frees their storage) #357's account; we have no symbols to confirm it. The cv2.pyd lines are probably stale values (OpenCV is loaded by the layout stage).

    • In 2 of 14: ACCESS_VIOLATION in python313.dll+0x2b80c0 (reading 0x1CCD9280128) and +0x2b80dd (reading 0x100000007), also on a non-main thread, with python313.dll and ntdll.dll addresses on its stack. This may be the later effect of a corrupted heap that Windows: threaded renderer corrupts the heap when its worker threads exit (thread_local destructors run after MinGW emutls frees their storage) #357 describes.

    • Python's faulthandler wrote "Windows fatal exception: access violation" and every thread's stack; we kept only its last 2 KiB, whose last line was our worker's main loop, File "extract_worker.py", line 2082 in <module>.

    In run E two processes printed "Windows fatal exception: access violation" once each during the 12 minutes and went on: every later conversion returned a document and both exited 0. Each report lists 12 Python threads and no "Current thread" line, so the faulting thread ran no Python code (as #357 says). At that moment the main thread was in DocumentConverter.convert → StandardPdfPipeline._build_document → get_batch (waiting), Docling's stage threads were in get_batch or _run_model, and one thread was in the layout model's forward pass (torch.nn.Conv2d._conv_forward in report 1, torch.nn.functional._max_pool2d under RTDetrV2 in report 2). Both reports are attached as written, the package folder shown as <site-packages>.

    Attachments

    File Bytes SHA-256 What it is
    minutes-a.pdf 3,103 6dcc2f381036bad855e8ff1680ec59f51d48d5ae870120f65b7cfdb247126885 synthetic meeting minutes, 2 pages, native text with a table
    minutes-b.pdf 3,368 50236065268641f7f669c0460645882a5fb34c469f7273235300c90f3e404d61 the same kind, another generated item
    repro.py 3,510 c9d27eae8725a9cd4c73f8d23513aa081e96f9318b9aba426018e3f51b2f4a7f the loop above
    faulthandler-run1.txt 4,748 f8f4593b21238e4c6a9fc7da54abfaea529850862a09ac90d5ddf6818373c405 run E, process 1: the survived access violation, every thread
    faulthandler-run2.txt 7,236 6026465bfda17a01cc28e9bec0e6f96605983089ebecaf29b916eeb5df7b08be run E, process 2: the same

    Every attachment is synthetic. The two PDFs were made by a test generator for an invented project ("North Quay Works, Phase 2 (synthetic)"); they describe no real person, organisation, place or document, and their metadata is empty. The script and the two faulthandler logs are ours; in the logs the package folder is shown as <site-packages> and the script by its file name. We do not attach the minidumps: they hold the paths and account details of the computer they were written on. We can read further values from them on request (registers, other threads' stacks).

    The files. They are inline below rather than uploaded (posted through the API, which takes no attachments). The hashes in the table are of the original bytes: the text files with LF line endings as shown, the PDFs after decoding the base64 (certutil -decode minutes-a.b64 minutes-a.pdf, or base64 -d).

    repro.py
    """Convert synthetic PDFs again and again with Docling (docling-parse backend), as an extraction
    worker does without OCR, until the time is up or the process crashes.
    
    Usage (Windows, CPython 3.13, packages as pinned in the report):
      python -I -X faulthandler repro.py <folder with site-packages, models and msvc> <work folder> <seconds> <pdf> [<pdf> ...]
    
    <folder>\site-packages holds the pinned packages, <folder>\models the Docling layout and TableFormer
    model files, <folder>\msvc the Visual C++ runtime DLLs. Each round reads every PDF with
    docling-parse's own reader (text lines) and converts it with Docling (layout model, TableFormer
    accurate with cell matching, no OCR). A crash ends the process with exit code -1073741819
    (0xC0000005) and Python's faulthandler writes every thread's stack to standard error.
    """
    import os
    import sys
    import time
    
    pack, work, seconds, pdfs = sys.argv[1], sys.argv[2], float(sys.argv[3]), sys.argv[4:]
    sys.path.append(os.path.join(pack, "site-packages"))
    os.add_dll_directory(os.path.join(pack, "msvc"))
    home, temp = os.path.join(work, ".home"), os.path.join(work, ".tmp")
    for folder in (home, temp):
        os.makedirs(folder, exist_ok=True)
    os.environ.update({"HF_HUB_OFFLINE": "1", "TRANSFORMERS_OFFLINE": "1", "HF_DATASETS_OFFLINE": "1", "HF_HUB_DISABLE_TELEMETRY": "1",
                       "DO_NOT_TRACK": "1", "USERPROFILE": home, "HOME": home, "HF_HOME": os.path.join(home, "hf"),
                       "TORCH_HOME": os.path.join(home, "torch"), "XDG_CACHE_HOME": os.path.join(home, "cache"), "TEMP": temp, "TMP": temp})
    
    from docling.datamodel.base_models import InputFormat  # noqa: E402
    from docling.datamodel.pipeline_options import PdfPipelineOptions, TableFormerMode, TableStructureOptions  # noqa: E402
    from docling.document_converter import DocumentConverter, PdfFormatOption  # noqa: E402
    from docling_parse.pdf_parser import DecodeConfig, DoclingPdfParser  # noqa: E402
    from docling_core.types.doc.page import TextCellUnit  # noqa: E402
    from importlib.metadata import version  # noqa: E402
    
    print("python %s; docling %s; docling-parse %s; docling-core %s; torch %s" % (
        sys.version.split()[0], version("docling"), version("docling-parse"), version("docling-core"), version("torch")), flush=True)
    options = PdfPipelineOptions(artifacts_path=os.path.join(pack, "models"), do_ocr=False, do_table_structure=True,
                                 table_structure_options=TableStructureOptions(mode=TableFormerMode.ACCURATE, do_cell_matching=True),
                                 enable_remote_services=False, generate_parsed_pages=True)
    converter = DocumentConverter(allowed_formats=[InputFormat.PDF], format_options={InputFormat.PDF: PdfFormatOption(pipeline_options=options)})
    parser, config = DoclingPdfParser(loglevel="fatal"), DecodeConfig(enforce_same_font=True)
    start, n = time.monotonic(), 0
    while time.monotonic() - start < seconds:
        for pdf in pdfs:
            doc = parser.load(path_or_stream=pdf, decode_config=config)
            lines = sum(len(list(doc.get_page(p).iterate_cells(unit_type=TextCellUnit.LINE))) for p in range(1, doc.number_of_pages() + 1))
            doc.unload()
            result = converter.convert(pdf, raises_on_error=True)
            n += 1
            print("%d %s lines=%d items=%d %.1fs" % (n, os.path.basename(pdf), lines, len(list(result.document.iterate_items())),
                                                   time.monotonic() - start), flush=True)
    print("no crash: %d conversions in %.0f s" % (n, time.monotonic() - start), flush=True)
    faulthandler-run1.txt
    Windows fatal exception: access violation
    Thread 0x000022a0 (most recent call first):
      File "threading.py", line 363 in wait
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 214 in get_batch
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 319 in _run
      File "threading.py", line 995 in run
      File "threading.py", line 1044 in _bootstrap_inner
      File "threading.py", line 1015 in _bootstrap
    Thread 0x0002cfb4 (most recent call first):
      File "threading.py", line 363 in wait
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 214 in get_batch
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 319 in _run
      File "threading.py", line 995 in run
      File "threading.py", line 1044 in _bootstrap_inner
      File "threading.py", line 1015 in _bootstrap
    Thread 0x000270a0 (most recent call first):
      File "threading.py", line 363 in wait
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 214 in get_batch
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 319 in _run
      File "threading.py", line 995 in run
      File "threading.py", line 1044 in _bootstrap_inner
      File "threading.py", line 1015 in _bootstrap
    Thread 0x0003060c (most recent call first):
      File "threading.py", line 363 in wait
      File "threading.py", line 660 in wait
      File "concurrent\futures\_base.py", line 306 in wait
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 279 in _run_model
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 387 in _process_batch
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 322 in _run
      File "threading.py", line 995 in run
      File "threading.py", line 1044 in _bootstrap_inner
      File "threading.py", line 1015 in _bootstrap
    Thread 0x00009494 (most recent call first):
      File "threading.py", line 363 in wait
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 214 in get_batch
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 319 in _run
      File "threading.py", line 995 in run
      File "threading.py", line 1044 in _bootstrap_inner
      File "threading.py", line 1015 in _bootstrap
    Thread 0x0002b1d0 (most recent call first):
      File "concurrent\futures\thread.py", line 90 in _worker
      File "threading.py", line 995 in run
      File "threading.py", line 1044 in _bootstrap_inner
      File "threading.py", line 1015 in _bootstrap
    Thread 0x0000311c (most recent call first):
      File "concurrent\futures\thread.py", line 90 in _worker
      File "threading.py", line 995 in run
      File "threading.py", line 1044 in _bootstrap_inner
      File "threading.py", line 1015 in _bootstrap
    Thread 0x0001a320 (most recent call first):
      File "concurrent\futures\thread.py", line 90 in _worker
      File "threading.py", line 995 in run
      File "threading.py", line 1044 in _bootstrap_inner
      File "threading.py", line 1015 in _bootstrap
    Thread 0x00031a80 (most recent call first):
      File "concurrent\futures\thread.py", line 90 in _worker
      File "threading.py", line 995 in run
      File "threading.py", line 1044 in _bootstrap_inner
      File "threading.py", line 1015 in _bootstrap
    Thread 0x000336ec (most recent call first):
      File "<site-packages>\torch\nn\modules\conv.py", line 560 in _conv_forward
      File "<site-packages>\torch\nn\modules\conv.py", line ??? in forward
      <invalid frame>
    Thread 0x00006478 (most recent call first):
      File "threading.py", line 363 in wait
      File "threading.py", line 660 in wait
      File "<site-packages>\tqdm\_monitor.py", line 69 in run
      File "threading.py", line 1044 in _bootstrap_inner
      File "threading.py", line 1015 in _bootstrap
    Thread 0x00033d68 (most recent call first):
      File "threading.py", line 363 in wait
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 214 in get_batch
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 983 in _build_document
      File "<site-packages>\docling\pipeline\base_pipeline.py", line 99 in execute
      File "<site-packages>\docling\document_converter.py", line 1024 in _execute_pipeline
      File "<site-packages>\docling\document_converter.py", line 988 in _process_document
      File "<site-packages>\docling\document_converter.py", line 970 in _process_indexed_document
      File "<site-packages>\docling\document_converter.py", line 910 in _convert
      File "<site-packages>\docling\document_converter.py", line 771 in convert_all
      File "<site-packages>\docling\document_converter.py", line 700 in convert
      File "<site-packages>\pydantic\_internal\_validate_call.py", line 137 in __call__
      File "<site-packages>\pydantic\_internal\_validate_call.py", line 40 in wrapper_function
      File "m3-17-repro.py", line 50 in <module>
    
    faulthandler-run2.txt
    Windows fatal exception: access violation
    Thread 0x0002f75c (most recent call first):
      File "threading.py", line 363 in wait
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 214 in get_batch
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 319 in _run
      File "threading.py", line 995 in run
      File "threading.py", line 1044 in _bootstrap_inner
      File "threading.py", line 1015 in _bootstrap
    Thread 0x00006074 (most recent call first):
      File "threading.py", line 363 in wait
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 214 in get_batch
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 319 in _run
      File "threading.py", line 995 in run
      File "threading.py", line 1044 in _bootstrap_inner
      File "threading.py", line 1015 in _bootstrap
    Thread 0x0003380c (most recent call first):
      File "threading.py", line 363 in wait
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 214 in get_batch
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 319 in _run
      File "threading.py", line 995 in run
      File "threading.py", line 1044 in _bootstrap_inner
      File "threading.py", line 1015 in _bootstrap
    Thread 0x0002ffb4 (most recent call first):
      File "threading.py", line 363 in wait
      File "threading.py", line 660 in wait
      File "concurrent\futures\_base.py", line 306 in wait
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 279 in _run_model
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 387 in _process_batch
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 322 in _run
      File "threading.py", line 995 in run
      File "threading.py", line 1044 in _bootstrap_inner
      File "threading.py", line 1015 in _bootstrap
    Thread 0x00033e28 (most recent call first):
      File "threading.py", line 363 in wait
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 214 in get_batch
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 319 in _run
      File "threading.py", line 995 in run
      File "threading.py", line 1044 in _bootstrap_inner
      File "threading.py", line 1015 in _bootstrap
    Thread 0x000317f8 (most recent call first):
      File "concurrent\futures\thread.py", line 90 in _worker
      File "threading.py", line 995 in run
      File "threading.py", line 1044 in _bootstrap_inner
      File "threading.py", line 1015 in _bootstrap
    Thread 0x00006988 (most recent call first):
      File "concurrent\futures\thread.py", line 90 in _worker
      File "threading.py", line 995 in run
      File "threading.py", line 1044 in _bootstrap_inner
      File "threading.py", line 1015 in _bootstrap
    Thread 0x00033960 (most recent call first):
      File "concurrent\futures\thread.py", line 90 in _worker
      File "threading.py", line 995 in run
      File "threading.py", line 1044 in _bootstrap_inner
      File "threading.py", line 1015 in _bootstrap
    Thread 0x00032be0 (most recent call first):
      File "concurrent\futures\thread.py", line 90 in _worker
      File "threading.py", line 995 in run
      File "threading.py", line 1044 in _bootstrap_inner
      File "threading.py", line 1015 in _bootstrap
    Thread 0x0003244c (most recent call first):
      File "<site-packages>\torch\nn\functional.py", line 837 in _max_pool2d
      File "<site-packages>\torch\_jit_internal.py", line 637 in fn
      File "<site-packages>\torch\nn\modules\pooling.py", line 224 in forward
      File "<site-packages>\torch\nn\modules\module.py", line 1794 in _call_impl
      File "<site-packages>\torch\nn\modules\module.py", line 1783 in _wrapped_call_impl
      File "<site-packages>\transformers\models\rt_detr\modeling_rt_detr_resnet.py", line 113 in forward
      File "<site-packages>\torch\nn\modules\module.py", line 1794 in _call_impl
      File "<site-packages>\torch\nn\modules\module.py", line 1783 in _wrapped_call_impl
      File "<site-packages>\transformers\models\rt_detr\modeling_rt_detr_resnet.py", line 401 in forward
      File "<site-packages>\transformers\backbone_utils.py", line 172 in wrapper
      File "<site-packages>\transformers\utils\generic.py", line 943 in wrapper
      File "<site-packages>\torch\nn\modules\module.py", line 1794 in _call_impl
      File "<site-packages>\torch\nn\modules\module.py", line 1783 in _wrapped_call_impl
      File "<site-packages>\transformers\models\rt_detr_v2\modeling_rt_detr_v2.py", line 807 in forward
      File "<site-packages>\torch\nn\modules\module.py", line 1794 in _call_impl
      File "<site-packages>\torch\nn\modules\module.py", line 1783 in _wrapped_call_impl
      File "<site-packages>\transformers\models\rt_detr_v2\modeling_rt_detr_v2.py", line 1513 in forward
      File "<site-packages>\transformers\utils\generic.py", line 943 in wrapper
      File "<site-packages>\torch\nn\modules\module.py", line 1794 in _call_impl
      File "<site-packages>\torch\nn\modules\module.py", line 1783 in _wrapped_call_impl
      File "<site-packages>\transformers\models\rt_detr_v2\modeling_rt_detr_v2.py", line 1861 in forward
      File "<site-packages>\transformers\utils\generic.py", line 943 in wrapper
      File "<site-packages>\torch\nn\modules\module.py", line 1794 in _call_impl
      File "<site-packages>\torch\nn\modules\module.py", line 1783 in _wrapped_call_impl
      File "<site-packages>\docling\models\inference_engines\object_detection\transformers_engine.py", line 231 in predict_batch
      File "<site-packages>\docling\models\stages\layout\layout_object_detection_model.py", line 115 in predict_layout
      File "<site-packages>\docling\models\base_layout_model.py", line 62 in __call__
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 277 in <lambda>
      File "concurrent\futures\thread.py", line 59 in run
      File "concurrent\futures\thread.py", line 93 in _worker
      File "threading.py", line 995 in run
      File "threading.py", line 1044 in _bootstrap_inner
      File "threading.py", line 1015 in _bootstrap
    Thread 0x000270f0 (most recent call first):
      File "threading.py", line 363 in wait
      File "threading.py", line 660 in wait
      File "<site-packages>\tqdm\_monitor.py", line 69 in run
      File "threading.py", line 1044 in _bootstrap_inner
      File "threading.py", line 1015 in _bootstrap
    Thread 0x00032fa0 (most recent call first):
      File "threading.py", line 363 in wait
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 214 in get_batch
      File "<site-packages>\docling\pipeline\standard_pdf_pipeline.py", line 983 in _build_document
      File "<site-packages>\docling\pipeline\base_pipeline.py", line 99 in execute
      File "<site-packages>\docling\document_converter.py", line 1024 in _execute_pipeline
      File "<site-packages>\docling\document_converter.py", line 988 in _process_document
      File "<site-packages>\docling\document_converter.py", line 970 in _process_indexed_document
      File "<site-packages>\docling\document_converter.py", line 910 in _convert
      File "<site-packages>\docling\document_converter.py", line 771 in convert_all
      File "<site-packages>\docling\document_converter.py", line 700 in convert
      File "<site-packages>\pydantic\_internal\_validate_call.py", line 137 in __call__
      File "<site-packages>\pydantic\_internal\_validate_call.py", line 40 in wrapper_function
      File "m3-17-repro.py", line 50 in <module>
    
    minutes-a.pdf (base64)
    JVBERi0xLjQKJeLjz9MKMSAwIG9iago8PCAvVHlwZSAvQ2F0YWxvZyAvUGFnZXMgMiAwIFIgPj4K
    ZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFs1IDAgUiA3IDAgUl0gL0NvdW50
    IDIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL1R5cGUgL0ZvbnQgL1N1YnR5cGUgL1R5cGUxIC9CYXNl
    Rm9udCAvSGVsdmV0aWNhIC9FbmNvZGluZyAvV2luQW5zaUVuY29kaW5nID4+CmVuZG9iago0IDAg
    b2JqCjw8IC9MZW5ndGggMTk2NSA+PnN0cmVhbQpCVCAvRjEgMTUgVGYgNzIgNzIwIFRkIChNSU5V
    VEVTIE5RVy1OUERGLTAwOCkgVGogRVQKQlQgL0YxIDEyIFRmIDcyIDY5NCBUZCAoUHJvamVjdDog
    Tm9ydGggUXVheSBXb3JrcywgUGhhc2UgMiBcKHN5bnRoZXRpY1wpKSBUaiBFVApCVCAvRjEgMTIg
    VGYgNzIgNjc2IFRkIChEYXRlOiA1IEFwcmlsIDIwMjUpIFRqIEVUCkJUIC9GMSAxMiBUZiA3MiA2
    NTggVGQgKFN1YmplY3Q6IHRoZSBjb25jcmV0ZSBzdXBwbHkgaW50ZXJydXB0aW9uKSBUaiBFVApC
    VCAvRjEgMTEgVGYgNzIgNjMwIFRkIChUaGUgRW5naW5lZXIgaXMgYXNrZWQgdG8gY29uZmlybSB3
    aGV0aGVyIGEgdmFyaWF0aW9uIGlzIGluc3RydWN0ZWQgZm9yIHRoaXMgd29yay4pIFRqIEVUCkJU
    IC9GMSAxMSBUZiA3MiA2MTQgVGQgKFRoZSBuZXh0IHJldmlldyBvZiB0aGUgY29uY3JldGUgc3Vw
    cGx5IGludGVycnVwdGlvbiBpcyBzZXQgZm9yIHRoZSBwcm9ncmVzcykgVGogRVQKQlQgL0YxIDEx
    IFRmIDcyIDU5OCBUZCAobWVldGluZyBhZnRlciA1IEFwcmlsIDIwMjUuKSBUaiBFVApCVCAvRjEg
    MTEgVGYgNzIgNTgyIFRkIChMYWJvdXIgYW5kIHBsYW50IHN0b29kIGlkbGUgZm9yIHBhcnQgb2Yg
    dGhlIHNoaWZ0IHdoaWxlIHRoZSBjb25jcmV0ZSBzdXBwbHkpIFRqIEVUCkJUIC9GMSAxMSBUZiA3
    MiA1NjYgVGQgKGludGVycnVwdGlvbiB3YXMgcmVzb2x2ZWQuKSBUaiBFVApCVCAvRjEgMTEgVGYg
    NzIgNTUwIFRkIChObyB3b3JrIHdhcyBjYXJyaWVkIG91dCBpbiB0aGUgcmVzdHJpY3RlZCB6b25l
    IHVudGlsIHRoZSBwZXJtaXQgd2FzIHJlbmV3ZWQuKSBUaiBFVApCVCAvRjEgMTEgVGYgNzIgNTM0
    IFRkIChUaGUgQ29udHJhY3RvciByZWZlcnMgdG8gdGhlIGNvbmNyZXRlIHN1cHBseSBpbnRlcnJ1
    cHRpb24gYW5kIHJlY29yZHMgdGhlIHBvc2l0aW9uKSBUaiBFVApCVCAvRjEgMTEgVGYgNzIgNTE4
    IFRkIChhcyBhdCA1IEFwcmlsIDIwMjUuKSBUaiBFVApCVCAvRjEgMTIgVGYgNzIgNDg4IFRkIChB
    Q1RJT05TIEFHUkVFRCkgVGogRVQKMC42IHcKNzIgNDc2IG0gNTUyIDQ3NiBsIFMKNzIgNDU0IG0g
    NTUyIDQ1NCBsIFMKNzIgNDMyIG0gNTUyIDQzMiBsIFMKNzIgNDEwIG0gNTUyIDQxMCBsIFMKNzIg
    Mzg4IG0gNTUyIDM4OCBsIFMKNzIgMzY2IG0gNTUyIDM2NiBsIFMKNzIgNDc2IG0gNzIgMzY2IGwg
    UwoyOTIgNDc2IG0gMjkyIDM2NiBsIFMKNDEyIDQ3NiBtIDQxMiAzNjYgbCBTCjU1MiA0NzYgbSA1
    NTIgMzY2IGwgUwpCVCAvRjEgMTAgVGYgNzggNDYxIFRkIChBY3Rpb24pIFRqIEVUCkJUIC9GMSAx
    MCBUZiAyOTggNDYxIFRkIChPd25lcikgVGogRVQKQlQgL0YxIDEwIFRmIDQxOCA0NjEgVGQgKER1
    ZSkgVGogRVQKQlQgL0YxIDEwIFRmIDc4IDQzOSBUZCAoUGVybWl0IHJlbmV3YWwgMSkgVGogRVQK
    QlQgL0YxIDEwIFRmIDI5OCA0MzkgVGQgKENvbnRyYWN0b3IpIFRqIEVUCkJUIC9GMSAxMCBUZiA0
    MTggNDM5IFRkICgxMiBBcHJpbCAyMDI1KSBUaiBFVApCVCAvRjEgMTAgVGYgNzggNDE3IFRkIChS
    ZXZpc2VkIGRyYXdpbmcgMikgVGogRVQKQlQgL0YxIDEwIFRmIDI5OCA0MTcgVGQgKFN1YmNvbnRy
    YWN0b3IpIFRqIEVUCkJUIC9GMSAxMCBUZiA0MTggNDE3IFRkICgxOSBBcHJpbCAyMDI1KSBUaiBF
    VApCVCAvRjEgMTAgVGYgNzggMzk1IFRkIChDb3N0IHJlY29yZCAzKSBUaiBFVApCVCAvRjEgMTAg
    VGYgMjk4IDM5NSBUZCAoRW5naW5lZXIpIFRqIEVUCkJUIC9GMSAxMCBUZiA0MTggMzk1IFRkICgy
    NiBBcHJpbCAyMDI1KSBUaiBFVApCVCAvRjEgMTAgVGYgNzggMzczIFRkIChTdXJ2ZXkgNCkgVGog
    RVQKQlQgL0YxIDEwIFRmIDI5OCAzNzMgVGQgKENvbnRyYWN0b3IpIFRqIEVUCkJUIC9GMSAxMCBU
    ZiA0MTggMzczIFRkICgzIE1heSAyMDI1KSBUaiBFVApCVCAvRjEgMTEgVGYgNzIgMzQyIFRkIChF
    bmQgb2YgdGhlIG1pbnV0ZXMuIE5leHQgbWVldGluZyBvbiAxOSBBcHJpbCAyMDI1LikgVGogRVQK
    ZW5kc3RyZWFtCmVuZG9iago1IDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL01l
    ZGlhQm94IFswIDAgNjEyIDc5Ml0gL1Jlc291cmNlcyA8PCAvRm9udCA8PCAvRjEgMyAwIFIgPj4g
    Pj4gL0NvbnRlbnRzIDQgMCBSID4+CmVuZG9iago2IDAgb2JqCjw8IC9MZW5ndGggMzM3ID4+c3Ry
    ZWFtCkJUIC9GMSAxMyBUZiA3MiA3MjAgVGQgKFNDSEVEVUxFIE9GIFJFQ09SRFMgTlFXLU5QREYt
    MDA4KSBUaiBFVApCVCAvRjEgMTEgVGYgNzIgNzAwIFRkIChSZWNvcmQgMTogcGVybWl0LCA1IEFw
    cmlsIDIwMjUuKSBUaiBFVApCVCAvRjEgMTEgVGYgNzIgNjgwIFRkIChSZWNvcmQgMjogc2l0ZSBk
    aWFyeSwgNiBBcHJpbCAyMDI1LikgVGogRVQKQlQgL0YxIDExIFRmIDcyIDY2MCBUZCAoUmVjb3Jk
    IDM6IGFsbG9jYXRpb24gc2hlZXQsIDcgQXByaWwgMjAyNS4pIFRqIEVUCkJUIC9GMSAxMSBUZiA3
    MiA2NDAgVGQgKFJlY29yZCA0OiBwaG90b2dyYXBoIGxvZywgOCBBcHJpbCAyMDI1LikgVGogRVQK
    ZW5kc3RyZWFtCmVuZG9iago3IDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL01l
    ZGlhQm94IFswIDAgNjEyIDc5Ml0gL1Jlc291cmNlcyA8PCAvRm9udCA8PCAvRjEgMyAwIFIgPj4g
    Pj4gL0NvbnRlbnRzIDYgMCBSID4+CmVuZG9iagp4cmVmCjAgOAowMDAwMDAwMDAwIDY1NTM1IGYg
    CjAwMDAwMDAwMTUgMDAwMDAgbiAKMDAwMDAwMDA2NCAwMDAwMCBuIAowMDAwMDAwMTI3IDAwMDAw
    IG4gCjAwMDAwMDAyMjQgMDAwMDAgbiAKMDAwMDAwMjI0MCAwMDAwMCBuIAowMDAwMDAyMzY2IDAw
    MDAwIG4gCjAwMDAwMDI3NTMgMDAwMDAgbiAKdHJhaWxlcgo8PCAvU2l6ZSA4IC9Sb290IDEgMCBS
    ID4+CnN0YXJ0eHJlZgoyODc5CiUlRU9GCg==
    
    minutes-b.pdf (base64)
    JVBERi0xLjQKJeLjz9MKMSAwIG9iago8PCAvVHlwZSAvQ2F0YWxvZyAvUGFnZXMgMiAwIFIgPj4K
    ZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFs1IDAgUiA3IDAgUl0gL0NvdW50
    IDIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL1R5cGUgL0ZvbnQgL1N1YnR5cGUgL1R5cGUxIC9CYXNl
    Rm9udCAvSGVsdmV0aWNhIC9FbmNvZGluZyAvV2luQW5zaUVuY29kaW5nID4+CmVuZG9iago0IDAg
    b2JqCjw8IC9MZW5ndGggMjIyNiA+PnN0cmVhbQpCVCAvRjEgMTUgVGYgNzIgNzIwIFRkIChNSU5V
    VEVTIE5RVy1OUERGLTAxMikgVGogRVQKQlQgL0YxIDEyIFRmIDcyIDY5NCBUZCAoUHJvamVjdDog
    Tm9ydGggUXVheSBXb3JrcywgUGhhc2UgMiBcKHN5bnRoZXRpY1wpKSBUaiBFVApCVCAvRjEgMTIg
    VGYgNzIgNjc2IFRkIChEYXRlOiAxNCBNYXJjaCAyMDI1KSBUaiBFVApCVCAvRjEgMTIgVGYgNzIg
    NjU4IFRkIChTdWJqZWN0OiB0aGUgcmV2aXNlZCBwaWxlIGNhcCBkcmF3aW5ncykgVGogRVQKQlQg
    L0YxIDExIFRmIDcyIDYzMCBUZCAoVGhlIEVuZ2luZWVyIGlzIGFza2VkIHRvIGNvbmZpcm0gd2hl
    dGhlciBhIHZhcmlhdGlvbiBpcyBpbnN0cnVjdGVkIGZvciB0aGlzIHdvcmsuKSBUaiBFVApCVCAv
    RjEgMTEgVGYgNzIgNjE0IFRkIChUaGUgbmV4dCByZXZpZXcgb2YgdGhlIHJldmlzZWQgcGlsZSBj
    YXAgZHJhd2luZ3MgaXMgc2V0IGZvciB0aGUgcHJvZ3Jlc3MgbWVldGluZykgVGogRVQKQlQgL0Yx
    IDExIFRmIDcyIDU5OCBUZCAoYWZ0ZXIgMTQgTWFyY2ggMjAyNS4pIFRqIEVUCkJUIC9GMSAxMSBU
    ZiA3MiA1ODIgVGQgKExhYm91ciBhbmQgcGxhbnQgc3Rvb2QgaWRsZSBmb3IgcGFydCBvZiB0aGUg
    c2hpZnQgd2hpbGUgdGhlIHJldmlzZWQgcGlsZSBjYXApIFRqIEVUCkJUIC9GMSAxMSBUZiA3MiA1
    NjYgVGQgKGRyYXdpbmdzIHdhcyByZXNvbHZlZC4pIFRqIEVUCkJUIC9GMSAxMSBUZiA3MiA1NTAg
    VGQgKE5vIHdvcmsgd2FzIGNhcnJpZWQgb3V0IGluIHRoZSByZXN0cmljdGVkIHpvbmUgdW50aWwg
    dGhlIHBlcm1pdCB3YXMgcmVuZXdlZC4pIFRqIEVUCkJUIC9GMSAxMSBUZiA3MiA1MzQgVGQgKFRo
    ZSBDb250cmFjdG9yIHJlZmVycyB0byB0aGUgcmV2aXNlZCBwaWxlIGNhcCBkcmF3aW5ncyBhbmQg
    cmVjb3JkcyB0aGUgcG9zaXRpb24gYXMpIFRqIEVUCkJUIC9GMSAxMSBUZiA3MiA1MTggVGQgKGF0
    IDE0IE1hcmNoIDIwMjUuKSBUaiBFVApCVCAvRjEgMTEgVGYgNzIgNTAyIFRkIChBIHJldmlzZWQg
    bWV0aG9kIHN0YXRlbWVudCB3aWxsIGJlIGlzc3VlZCB3aXRoaW4gc2V2ZW4gZGF5cyBvZiB0aGlz
    IHJlY29yZC4pIFRqIEVUCkJUIC9GMSAxMiBUZiA3MiA0NzIgVGQgKEFDVElPTlMgQUdSRUVEKSBU
    aiBFVAowLjYgdwo3MiA0NjAgbSA1NTIgNDYwIGwgUwo3MiA0MzggbSA1NTIgNDM4IGwgUwo3MiA0
    MTYgbSA1NTIgNDE2IGwgUwo3MiAzOTQgbSA1NTIgMzk0IGwgUwo3MiAzNzIgbSA1NTIgMzcyIGwg
    Uwo3MiAzNTAgbSA1NTIgMzUwIGwgUwo3MiAzMjggbSA1NTIgMzI4IGwgUwo3MiA0NjAgbSA3MiAz
    MjggbCBTCjI5MiA0NjAgbSAyOTIgMzI4IGwgUwo0MTIgNDYwIG0gNDEyIDMyOCBsIFMKNTUyIDQ2
    MCBtIDU1MiAzMjggbCBTCkJUIC9GMSAxMCBUZiA3OCA0NDUgVGQgKEFjdGlvbikgVGogRVQKQlQg
    L0YxIDEwIFRmIDI5OCA0NDUgVGQgKE93bmVyKSBUaiBFVApCVCAvRjEgMTAgVGYgNDE4IDQ0NSBU
    ZCAoRHVlKSBUaiBFVApCVCAvRjEgMTAgVGYgNzggNDIzIFRkIChNZXRob2Qgc3RhdGVtZW50IDEp
    IFRqIEVUCkJUIC9GMSAxMCBUZiAyOTggNDIzIFRkIChTdWJjb250cmFjdG9yKSBUaiBFVApCVCAv
    RjEgMTAgVGYgNDE4IDQyMyBUZCAoMjEgTWFyY2ggMjAyNSkgVGogRVQKQlQgL0YxIDEwIFRmIDc4
    IDQwMSBUZCAoUGVybWl0IHJlbmV3YWwgMikgVGogRVQKQlQgL0YxIDEwIFRmIDI5OCA0MDEgVGQg
    KEVuZ2luZWVyKSBUaiBFVApCVCAvRjEgMTAgVGYgNDE4IDQwMSBUZCAoMjggTWFyY2ggMjAyNSkg
    VGogRVQKQlQgL0YxIDEwIFRmIDc4IDM3OSBUZCAoUmV2aXNlZCBkcmF3aW5nIDMpIFRqIEVUCkJU
    IC9GMSAxMCBUZiAyOTggMzc5IFRkIChDb250cmFjdG9yKSBUaiBFVApCVCAvRjEgMTAgVGYgNDE4
    IDM3OSBUZCAoNCBBcHJpbCAyMDI1KSBUaiBFVApCVCAvRjEgMTAgVGYgNzggMzU3IFRkIChDb3N0
    IHJlY29yZCA0KSBUaiBFVApCVCAvRjEgMTAgVGYgMjk4IDM1NyBUZCAoU3ViY29udHJhY3Rvcikg
    VGogRVQKQlQgL0YxIDEwIFRmIDQxOCAzNTcgVGQgKDExIEFwcmlsIDIwMjUpIFRqIEVUCkJUIC9G
    MSAxMCBUZiA3OCAzMzUgVGQgKFN1cnZleSA1KSBUaiBFVApCVCAvRjEgMTAgVGYgMjk4IDMzNSBU
    ZCAoRW5naW5lZXIpIFRqIEVUCkJUIC9GMSAxMCBUZiA0MTggMzM1IFRkICgxOCBBcHJpbCAyMDI1
    KSBUaiBFVApCVCAvRjEgMTEgVGYgNzIgMzA0IFRkIChFbmQgb2YgdGhlIG1pbnV0ZXMuIE5leHQg
    bWVldGluZyBvbiAyOCBNYXJjaCAyMDI1LikgVGogRVQKZW5kc3RyZWFtCmVuZG9iago1IDAgb2Jq
    Cjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL01lZGlhQm94IFswIDAgNjEyIDc5Ml0gL1Jl
    c291cmNlcyA8PCAvRm9udCA8PCAvRjEgMyAwIFIgPj4gPj4gL0NvbnRlbnRzIDQgMCBSID4+CmVu
    ZG9iago2IDAgb2JqCjw8IC9MZW5ndGggMzQxID4+c3RyZWFtCkJUIC9GMSAxMyBUZiA3MiA3MjAg
    VGQgKFNDSEVEVUxFIE9GIFJFQ09SRFMgTlFXLU5QREYtMDEyKSBUaiBFVApCVCAvRjEgMTEgVGYg
    NzIgNzAwIFRkIChSZWNvcmQgMTogcGVybWl0LCAxNCBNYXJjaCAyMDI1LikgVGogRVQKQlQgL0Yx
    IDExIFRmIDcyIDY4MCBUZCAoUmVjb3JkIDI6IHNpdGUgZGlhcnksIDE1IE1hcmNoIDIwMjUuKSBU
    aiBFVApCVCAvRjEgMTEgVGYgNzIgNjYwIFRkIChSZWNvcmQgMzogYWxsb2NhdGlvbiBzaGVldCwg
    MTYgTWFyY2ggMjAyNS4pIFRqIEVUCkJUIC9GMSAxMSBUZiA3MiA2NDAgVGQgKFJlY29yZCA0OiBw
    aG90b2dyYXBoIGxvZywgMTcgTWFyY2ggMjAyNS4pIFRqIEVUCmVuZHN0cmVhbQplbmRvYmoKNyAw
    IG9iago8PCAvVHlwZSAvUGFnZSAvUGFyZW50IDIgMCBSIC9NZWRpYUJveCBbMCAwIDYxMiA3OTJd
    IC9SZXNvdXJjZXMgPDwgL0ZvbnQgPDwgL0YxIDMgMCBSID4+ID4+IC9Db250ZW50cyA2IDAgUiA+
    PgplbmRvYmoKeHJlZgowIDgKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE1IDAwMDAwIG4g
    CjAwMDAwMDAwNjQgMDAwMDAgbiAKMDAwMDAwMDEyNyAwMDAwMCBuIAowMDAwMDAwMjI0IDAwMDAw
    IG4gCjAwMDAwMDI1MDEgMDAwMDAgbiAKMDAwMDAwMjYyNyAwMDAwMCBuIAowMDAwMDAzMDE4IDAw
    MDAwIG4gCnRyYWlsZXIKPDwgL1NpemUgOCAvUm9vdCAxIDAgUiA+PgpzdGFydHhyZWYKMzE0NAol
    JUVPRgo=
    
  5. usercjr commented on Nov 13, 2026

    @usercjr
    Contributor

    Thank you, this is the first independent reproduction on this issue with dumps, and it lines up with the report:

    • In 12 of your 14 dumps the fault is on a thread other than the main one, and your faulthandler reports from run E show a faulting thread that ran no Python code. The render workers are such threads, and the fault happens as one exits.
    • Two of your four run-E processes survived the access violation and kept converting. That fits a destructor running on freed thread-local storage, where what gets damaged depends on what else is on the heap at the time, better than it fits a fault in parsing a particular document.
    • 7.22.3, the release you used, already has the objects involved: in src/render/blend2d_renderer.h, the spare page canvas (thread_local BLImage) and the CMYK memo (two thread_local std::vectors). Both have destructors, and MinGW's emulated TLS frees a thread's storage before those run.

    I can't map +0x7e7bd7 in the PyPI wheel to a source line, so that part stays an inference.

    The fix is in #462: the canvas is now owned by the worker loop, and the memo is a set of arrays without destructors. On my Windows build, a regression-dataset page that crashes 7 of 20 fresh-process renders on main crashes 0 of 20 with it, and no pixel changes across the corpus. CI runs the tests on Ubuntu only, so if it's easy for you, it would help to know whether the access violations also stop on your workload with that branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions