Skip to content

ev2r: read PDFs page by page with per-page OCR, retry and budgets - #89

Merged
biggestbinary merged 4 commits into
mainfrom
feat/ev2r-pdf-per-page
Nov 18, 2026
Merged

biggestbinary merged 4 commits into
mainfrom
feat/ev2r-pdf-per-page

Conversation

@biggestbinary

Copy link
Copy Markdown
Collaborator

Summary

Kata ev2r (owner: "unstaple PDFs and extract text or process with OCR per page. We must keep retry."). Builds on #86 (ye2s). Cohort PDFs are now read page by page through two new modules, tools/extraction/pdf_pages.py (coordinator, units, retry, budgets, assembly) and tools/extraction/pdf_page_worker.py (isolated pypdfium2 worker: inventory + single-page render). pypdfium2 is reused from the pinned Docling environment; no new dependency, and requirements-docling.* are unchanged. Images still go through pdf_image unchanged.

Behaviour

  • Split and route per page. An isolated inventory unit reads every page's text layer and image count. A page keeps its text layer when it has ≥200 alphanumeric chars that make up ≥50% of its visible chars. Otherwise it is OCRed: it is rendered alone (216 dpi, downscaled per page below Docling's pixel cap, so oversized sheets no longer refuse the document) and converted by the pinned Docling worker. Docling identity files are not touched.

  • Retry kept. Every unit (the inventory and each OCR page) is retried once on timeout or crash. Deterministic refusals such as pixel-limit or malformed-page are not retried. A page that still fails becomes a failed page with its code and attempt count; the rest of the document is kept.

  • One Markdown rendition (pdf-pages) with pages in order. Failed pages get a visible placeholder like [Page N not extracted: extract-<code> after 2 attempts]. payload.pages records the outcome of every page (text-layer / ocr / failed + code + attempts, plus OCR render_scale and confidence). unread_pages covers pages beyond the budget.

  • Refused only when the document cannot be opened (encrypted or malformed), every page failed (their shared code, else extract-all-pages-failed), or no page has text (extract-empty-conversion).

  • Budgets replace the >100-page refusal.

    Budget Limit When exceeded
    Pages 5,000 Later pages are recorded unread with extract-page-budget
    Text-layer output 24 MiB Later pages are recorded unread with extract-output-limit
    Line records 200,000, placeholders included Later pages fail with extract-record-limit behind one range placeholder
    OCR time 7,200 s per document Later OCR pages fail with extract-time-budget without running
    Per OCR page 300 s wall Unit is stopped
    Per unit 16 GiB address space, 600 CPU-s Unit is stopped
    Per page worker 2 GiB Worker is stopped

    Nothing is truncated without a record.

  • Confidence is the union of the page kinds: pdf-text gives converted-format, ocr gives machine-ocr. Missing or failed pages add a new reason, incomplete-rendition (low; details list pages and codes, capped at 20).

  • Concurrency. All units of all documents share one --extract-concurrency pool.

  • No orphaned work. Each unit runs as PID 1 of its own PID namespace (unshare -Ur --pid --fork --kill-child). When it ends or is killed, every page or Docling worker it started dies with it, even workers in their own sessions.

  • python -m tools.extraction.pdf_pages probe runs the measurement.

Identity and rebuild impact

Live-instance rollout (falcon)

  1. Let any in-flight cohort finish registering.
  2. Upgrade, then rebuild the index once.
  3. Run the pinned-runtime preflight as for ye2s: read PDFs and images in cohort intake (text layer first, hardened Docling OCR) #86.
  4. Run cohort intake with --extract-concurrency 4.
  5. Optionally, file correction_of requests for PDFs ye2s: read PDFs and images in cohort intake (text layer first, hardened Docling OCR) #86 refused.

Measurements

The sample was 58 falcon PDFs (seed 2107), copied read-only to a scratch directory and deleted afterwards. It totals 4,454 pages and includes both 102-page scans (refused by #86), a 210-page mixed document, the 34- and 55-page scans, a 1,265-page manual and an oversized-page document. The run used concurrency 4 on a 14-core / 47 GiB host with the real runtime.

  • Success: 58/58 documents. 3,994 text-layer pages and 460 OCR pages, 0 failed pages, 0 retries needed, 0 unread.
  • OCR page unit: p50 19.8 s, p90 31.5 s, mean 25.6 s, max 159.5 s under 4-way load. Peak RSS was p50 1.5 GiB and max 2.5 GiB. A lone one-page scan took 13.4 s at 1.46 GB.
  • Inventory: mean 0.32 s, max 3.8 s per document.
  • Wall time: 49.6 min total. Host memory used rose by about 9 GiB at peak.
  • Falcon full run: 960 PDFs, 19,847 pages, of which 17,740 are text-layer and 2,107 OCR. 2,107 × 25.6 s ÷ 4 ≈ 3.75 h, plus about 1–2 min of inventory. ye2s: read PDFs and images in cohort intake (text layer first, hardened Docling OCR) #86 estimated about 9.7 h whole-document, with 3 page-limit refusals and 2 timeouts.
  • Slowest page: 160 s, close to the initial 180 s per-page wall, so that limit was raised to 300 s.

Tests

  • New tests/test_pdf_pages.py with 18 tests: routing; assembly, placeholders, budgets, the line cap and confidence; hardening with stand-in units (retry, timeouts, crashes, no surviving processes including other sessions, the time budget and the shared concurrency bound); and real-runtime tests (mixed text, scanned and oversized PDFs, a 120-page PDF, encrypted and malformed PDFs).
  • tests/test_pdf_image.py cohort tests were updated.
  • Full suite: 1,060 tests OK (15 skipped). Schema check OK. ruff clean.

Roborev

Job 834: Fail with 3 findings, all fixed in a4c2a66 (one fix pass) and recorded with roborev job comment 834.

  • Medium: workers in other sessions could outlive a unit that was killed abruptly. Fixed with PID namespaces.
  • Medium: the inventory's total output was unbounded. Fixed with the 24 MiB budget and extract-output-limit.
  • Low: placeholders could exceed the line cap. Fixed by reserving lines and adding one range placeholder.

Not done

biggestbinary and others added 4 commits November 18, 2026 12:53
Unstaple each PDF in an isolated pypdfium2 worker; keep each page's text
layer or OCR that page alone through the pinned Docling adapter. Every
unit is retried once on timeout/crash, then the page fails alone with a
placeholder and code. Budgets (pages, lines, OCR seconds) replace the
100-page refusal; confidence is the union of page kinds plus
incomplete-rendition. pdf_image keeps its identity and reads images.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ut, placeholders within the line cap

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The falcon sample's slowest page took 160 s under 4-way load, too close to 180.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@biggestbinary
biggestbinary merged commit e9057d5 into main Nov 18, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant