Skip to content

fix: avoid numeric AFM bounds for composite fonts - #469

Open
ainuo-AI wants to merge 1 commit into
docling-project:mainfrom
ainuo-AI:fix/cff-cid-glyphs
Open

ainuo-AI wants to merge 1 commit into
docling-project:mainfrom
ainuo-AI:fix/cff-cid-glyphs

Conversation

@ainuo-AI

Copy link
Copy Markdown

Fixes #466.

Page 10 of the Georgia DDS driver's manual loses every n in its heading when rendered through PageParseResult.get_image(). The text cells still contain the letters, but the rendered image has blanks. The embedded CFF glyph itself has a normal outline.

The problem is the numeric AFM bbox fallback. In this font, character code 0x0020 maps to CID 32 and then to n; the fallback reads standard-font slot 32, which is a space with a zero-height bbox. That becomes the glyph's render quad, and the renderer skips it.

This keeps Type0 character codes out of numeric AFM lookup. They address CIDs through a CMap, so a matching standard-font name does not make the numeric slots interchangeable. Named-glyph lookup and simple-font handling retain their existing behavior. No font data or rendering-library code changes are included.

Evidence:

The existing corpus already has a second example: 10749817875312063576_009.pdf, page 5, equation (9) in Nonideal Flow Models in Homogeneous Chemical Reactors. Its embedded SymbolMT TrueType subset puts = at CID 32. The same fallback removes the equals signs, and this change restores them. No new dataset PDF is needed.

PDFium, main and fixed rendering of the existing corpus equation

I also reproduced the original report with the unmodified Georgia DDS manual, SHA256 e311de54ad0ee6f5913e18e694914a560dac3592987437290c271f53c0c7fa71. Its heading goes from 5633 to 6279 dark pixels; PDFium has 6406.

PDFium, main and fixed rendering of the original heading

The evidence snapshot includes both full-page comparisons, a portable public-API reproducer, the complete corpus table, the 52-page manual table, and test logs. It is pinned separately from the fix, so the PR diff stays at three source files.

All 457 corpus PDFs were hash-checked at dataset revision 12531342c926708dcb1387ea25213db4603a7255. The comparison covers all 954 pages in the repository's default selection, plus all 52 pages of the manual. Images and corpus metrics use pypdfium2 5.15.2 for both builds, with the repository's RGBA MAE and luminance threshold of 12:

Comparison Scale MAE before MAE after Changed ratio before Changed ratio after
Original manual, page 10 3 1.554735 1.489869 0.04279359 0.04238853
Original manual, heading crop 3 6.697171 4.027851 0.14482657 0.12677837
Existing article, page 5 3 1.244643 1.220723 0.02889124 0.02885689
Article, first = crop 3 6.474206 1.925397 0.05026455 0.05185185
Corpus, average of 954 page metrics 2 2.469684 2.469494 0.06073283 0.06073243

The equals crop still has edge-pixel differences: its changed-pixel ratio rises slightly while MAE falls. The missing glyph returns, with 72 dark pixels against PDFium's 74, up from zero. The enforced crop test checks both metrics, image size and that the reference contains ink.

The affected character-cell diff is empty. Counts and complete JSON hashes of character, word and line cells are unchanged on all 954 corpus pages and all 52 manual pages. I did not rerun Docling's OCR pipeline.

The three small MAE increases and the validation limits

Of the 954 corpus pages, 934 have identical pixels, 17 have lower MAE, and 3 have slightly higher MAE at scale 2:

Document / page MAE increase Changed native A/B pixels Maximum channel change
15643741724817381873-1.pdf / 1 0.000086678 641 / 1,479,608 1 / 255
926659986622553168-1.pdf / 2 0.000000374 1 / 2,003,960 1 / 255
indexed_iccbased.pdf / 1 0.000003782 1773 / 1,850,688 19 / 255

These are repeatable differences. Each build produced identical hashes over three fresh renders of each page, matching the initial corpus run. I also checked scales 1, 2, 3 and 4; the maximum MAE increase remains 0.000086678. The scale table, diagnostic details, and magnified comparisons retain the full results. At scale 1 the second page also has a glyph that stops hitting the old 0.5-pixel early return and moves closer to PDFium; the scale-2 channel-change bounds above do not describe every scale.

On these three pages, baseline positions, character codes, embedded font identities, instruction order and non-text instructions are unchanged. Reconstructing the effective font size from the exported quads and metrics gives a maximum difference of 1.84e-12 pixels at scale 2, consistent with rounding sensitivity. This is a local diagnostic, not an instrumented Blend2D trace or a guarantee of zero risk on other platforms.

For the original manual, 28 pages have lower MAE, 13 are unchanged, and 11 increase by at most 0.000007737. Those increases involve 1–24 changed pixels per page, with a maximum channel difference of 9/255. The full table is linked above.

The added tests distinguish the reproductions from controls:

  • Eight constructed embedded-TrueType cases isolate code 32 under Helvetica, Times-Roman, Courier and Symbol AFM matches, through the FreeType and Blend2D paths. All eight fail on main.
  • One enforced PDFium comparison uses the real article's equals sign and fails on main.
  • Three enforced whole-page PDFium controls cover the small-increase pages and pass on both versions. All their text fonts are embedded. Their MAE limits (5.0 / 2.5 / 4.0) are below the general report cutoff of 9.0. Their changed-ratio limits are 0.15 / 0.07 / 0.14: the first and third are above the report cutoff of 0.09 because main already measures 0.13577 / 0.12286 on those pages. These controls do not tighten both metrics, and no global tolerance was changed.

Local results:

  • 97 passed, 1 existing skip in the related suite, including font mapping, metrics and text-rendering cases. The skip is the existing Arabic-fallback case without a shippable fixture.
  • The project's locked pypdfium2 5.14.0 also gives 17 passed across the two modified/added modules. On main, both PDFium versions give the same nine failures and three passing controls. The full-corpus measurements above use 5.15.2.
  • Ruff format/check, offline uv lock --check and whitespace checks pass.
  • MyPy reports the same 19 baseline errors on main and the fix in this Windows environment, with no new diagnostics; both logs are included. This is not a green full pre-commit claim.

The native builds use Windows x64, Python 3.12.10 and GCC 13.1.0 in Release mode. macOS/Linux and the complete stored-groundtruth suite have not run locally. Remote CI is separate from these local checks.

Ground truth changes:

Normalized render-instruction output changes on 145 selected pages in 81 documents, including 125 pages whose pixels remain identical. Changes are limited to text quads and normalized font ascent/descent: they now use PDF font metrics instead of unrelated numeric AFM entries. Instruction counts, ordering, mappings, baseline anchors and all non-text instruction data are unchanged.

The per-page field list and hashed normalized outputs are included in the evidence. This is a fresh main/fix comparison using the repository's three-decimal normalization. Stored remote groundtruth blobs were not downloaded or rewritten, and HF_DATASET_REVISION is unchanged. The following pages need instruction-groundtruth review/update through the maintainers' process for the reason above.

All 145 pages, grouped by document
  • 0b9f86cde166b9ab_0018.pdf: 1
  • 10040284511886890357_002.pdf: 1, 2
  • 10446922681186632184_002.pdf: 8
  • 10572911635253446040-13.pdf: 6, 8, 10, 11, 16
  • 10749817875312063576_009.pdf: 5, 6, 9
  • 10869285887791127292_005.pdf: 1
  • 11033738705194660817-2.pdf: 11
  • 1111108925592240862_007.pdf: 1, 2, 4, 5, 6
  • 11794545469969901016-2.pdf: 1
  • 11831013430589524848.pdf: 1
  • 12014666386554038795_001.pdf: 1, 2
  • 12025925399039829714_014.pdf: 13, 24
  • 12076365341177918911_001.pdf: 1, 2, 3, 4
  • 1265837786161076401-1.pdf: 1
  • 13828292398120620566-1.pdf: 1
  • 14327672304519301096-2.pdf: 1, 2, 3, 5, 6
  • 14445963425523837694-2.pdf: 1, 2
  • 14453006507210496387-6.pdf: 2, 4, 6, 8, 16
  • 14770497121209673752.pdf: 1
  • 14911938403731335361-1.pdf: 1, 2
  • 15643741724817381873-1.pdf: 1
  • 16293520921922171261-1.pdf: 1, 2, 3
  • 16338905573487211761-1.pdf: 1
  • 16687286043211380927-1.pdf: 1, 4, 7, 15, 17
  • 17791c05056ff856_0022.pdf: 1
  • 19246029784032167-4.pdf: 2, 3, 4, 5, 6
  • 2203.01017v2.pdf: 1, 8, 12
  • 2213713933117287130-1.pdf: 1, 2
  • 2305.03393v1.pdf: 7, 8, 10
  • 3040977140767198439-1.pdf: 1
  • 335fea0ba454ba15_0004.pdf: 1
  • 3584964174907633090-1.pdf: 1, 2
  • 3848385316711437784-4.pdf: 1, 2, 3, 4
  • 3934995182849940852-6.pdf: 1, 2, 3, 5, 6
  • 3960996440028888999-8.pdf: 4
  • 4796728975040539044-1.pdf: 1
  • 4865216256588543301.pdf: 1, 5, 6, 7, 8
  • 4c9bb33c9466bc71_0001.pdf: 1
  • 5d17819264eac373_0016.pdf: 1
  • 5d969923927df690_0006.pdf: 1
  • 6439951649404873078-1.pdf: 1
  • 6515533261975788184-1.pdf: 1
  • 6789603da459eee1_0017.pdf: 1
  • 6909c04bc2f0fd31_0003.pdf: 1
  • 7345174473351516034-1.pdf: 1
  • 7474312693447806103-1.pdf: 1
  • 7933217462246965845-6.pdf: 4, 6, 7
  • 8313873609013042575-4.pdf: 1, 2, 3, 4, 5
  • 8780050732516639302-1.pdf: 1, 5, 6, 7, 8
  • 926659986622553168-1.pdf: 1, 2
  • 9528c2089295e005_0001.pdf: 1
  • 985a9a85d020ac41_0003.pdf: 1
  • b766d7d18edf92b1_0002.pdf: 1
  • bitmap_decoding_01.pdf: 1
  • da04f7ddb6f59004_0025.pdf: 1
  • dln_104f22abf9f481aa323444ac8b404fca3d52df55c9397c294bc43075f2c078a8.pdf: 1
  • dln_116ab5b0f99390658c24887efe0b34060dad4eb8809fa90c2794e933d6ba54a9.pdf: 1
  • dln_1f3567ab5e1bd6150bbbc9c591e05df3f4b34331ce8ca7bac69e2b038812b64d.pdf: 1
  • dln_24f6a9fb5a1343ac9cd2ccb2ba409fb06bedca068c01d0b9bb9879e29288806f.pdf: 1
  • dln_835d4dd76390a0ccb3d5b257f3ce021d3a03e2b1249d68558821930fb8bdf069.pdf: 1
  • dln_d125e23af5add755072795fca6e91d10c6120abdcb3c2c9181e7f3a3bbf402f6.pdf: 1
  • dln_dcea5d0eb77fbcac9289cc49d9388348fa023cd75cc737f3cc93c1dd60271f4d.pdf: 1
  • eac6df0b2a6f9cf0_0002.pdf: 1
  • ead2c552b489aac1_0004.pdf: 1
  • ec62f8dc0265e965_0018.pdf: 1
  • fillable_form.pdf: 1
  • font_02.pdf: 1
  • font_03.pdf: 1
  • font_08.pdf: 1
  • indexed_iccbased.pdf: 1
  • jbig2_test_01.pdf: 1
  • jpeg_2000_01.pdf: 1
  • ligatures_01.pdf: 1
  • redp5110_sampled.pdf: 14
  • right_to_left.pdf: 1
  • right_to_left_01.pdf: 1
  • right_to_left_03.pdf: 1
  • right_to_left_04.pdf: 1
  • rotated_image.pdf: 1
  • stream_parameter_misinterpretation_01.pdf: 1
  • type3_fonts.pdf: 1

Checklist:

  • One topic, based on current main (88f1c388), one signed-off commit and no merge commits
  • Superseded pull requests closed and linked: none; this is independent of fix: load bundled PDF resources from non-ASCII Windows paths #468
  • Existing regression dataset document and page named; no dataset PR needed
  • Before/after evidence for the real examples, including the unchanged text-cell diff
  • Rendering: PDFium comparison images, full corpus metrics and an enforced image regression test
  • Blend2D upstream report: not applicable, no Blend2D patch
  • Existing groundtruth changes listed with their common reason

Type0 character codes resolve to CIDs rather than standard-font AFM slots.
Keep them out of the numeric AFM bbox fallback so an unrelated space entry
cannot give a visible glyph a zero-height render quad.

Add generated CID mapping regressions and an enforced PDFium comparison
on a real corpus formula. Keep whole-page controls for the three corpus
pages with small edge-pixel differences.

Fixes docling-project#466.

Signed-off-by: ainuo-AI <2470447645@qq.com>
@github-actions

Copy link
Copy Markdown
Contributor

✅ DCO Check Passed

Thanks @ainuo-AI, all your commits are properly signed off. 🎉

@mergify

mergify Bot commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

@PeterStaar-IBM
PeterStaar-IBM self-requested a review November 16, 2026 11:54

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

get_image() leaves out some glyphs of embedded CID-keyed CFF fonts (Microsoft Print to PDF)

1 participant