Skip to content

feat: improving memory footprint of docling - #459

Merged
PeterStaar-IBM merged 13 commits into
mainfrom
feat/reduce-memory-usage
Nov 13, 2026
Merged

PeterStaar-IBM merged 13 commits into
mainfrom
feat/reduce-memory-usage

Conversation

@PeterStaar-IBM

@PeterStaar-IBM PeterStaar-IBM commented Nov 11, 2026 •

Copy link
Copy Markdown
Member

resolving: #345 , #448

Reduce parser memory usage and automate performance benchmark publication

Reduce memory retained while interpreting PDF content streams and capturing clipping state, with benchmark tooling to measure, resume, and publish results.

Addresses #345 and the dense-page workload reported in docling-project/docling#4058.

Changes

  • Interpret page, Form, appearance, soft-mask, and Type3 content as QPDF delivers objects, avoiding complete token-list retention and unnecessary copies.
  • Replace copied clipping histories with shared immutable storage and a renderer cursor, removing quadratic retained geometry.
  • Preserve nested graphics-state restoration, damaged-stream recovery, and existing output semantics.
  • Add native memory probes and regression coverage for streaming, clipping, and nested resource lifetimes.
  • Collect independent process memory peaks and optional sampled RSS profiles.
  • Resume interrupted benchmarks by reusing validated configurations and rerunning missing or incomplete configurations with the original settings.
  • Publish benchmark documentation by dataset and machine setup, with preview mode, provenance, generated titles, and enforced file-size budgets.
  • Update Bo20 and Bo767 results and fix the Linux memory test’s sensitivity to pytest’s inherited pre-exec RSS.

Evidence

  • Controlled synthetic Type3 comparisons recorded approximately 72% lower decode/render peak memory and 81% lower shapes-disabled peaks at 250,000 lines. These are workload-specific measurements.
  • Final native smoke validation recorded 80 measurements and six byte-identical PNG equivalence checks.
  • Published Bo20 and Bo767 results include three repetitions and thread sweeps at 1, 4, 8, and 12 threads.

Validation

  • Benchmark recovery, reporting, memory, and publication tests passed.
  • After the CI test fix, all 18 memory tests passed locally on macOS.
  • Full MyPy and Ruff checks passed.
  • Documentation size, generated-file integrity, and link checks passed.
  • Linux CI verification of the latest test fix remains pending.

Ground truth changes

None. Existing regression ground truth and tolerances are unchanged.
Checklist:

  • One topic per pull request, rebased on main, no merge commits
  • Superseded pull requests closed and linked
  • Regression dataset documents and pages named, or a dataset pull request that adds a new document
  • Before/after evidence for each cited page
  • Rendering only: pypdfium2 comparison images, corpus-wide metrics table, and an enforced image regression test
  • Blend2D only: link to the upstream report
  • Changes to existing ground truth listed with the reason for each page

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
@github-actions

Copy link
Copy Markdown
Contributor

✅ DCO Check Passed

Thanks @PeterStaar-IBM, all your commits are properly signed off. 🎉

@mergify

mergify Bot commented Nov 11, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
@PeterStaar-IBM
PeterStaar-IBM merged commit 2f687d8 into main Nov 13, 2026
47 checks passed
@PeterStaar-IBM
PeterStaar-IBM deleted the feat/reduce-memory-usage branch November 13, 2026 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants