Repository navigation
Conversation
Type0 character codes resolve to CIDs rather than standard-font AFM slots. Keep them out of the numeric AFM bbox fallback so an unrelated space entry cannot give a visible glyph a zero-height render quad. Add generated CID mapping regressions and an enforced PDFium comparison on a real corpus formula. Keep whole-page controls for the three corpus pages with small edge-pixel differences. Fixes docling-project#466. Signed-off-by: ainuo-AI <2470447645@qq.com>
Contributor
|
✅ DCO Check Passed Thanks @ainuo-AI, all your commits are properly signed off. 🎉 |
Contributor
Merge Protections🟢 Merge protection satisfied — ready to merge. Show 1 satisfied protection🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
PeterStaar-IBM
self-requested a review
November 16, 2026 11:54
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #466.
Page 10 of the Georgia DDS driver's manual loses every
nin its heading when rendered throughPageParseResult.get_image(). The text cells still contain the letters, but the rendered image has blanks. The embedded CFF glyph itself has a normal outline.The problem is the numeric AFM bbox fallback. In this font, character code
0x0020maps to CID 32 and then ton; the fallback reads standard-font slot 32, which is a space with a zero-height bbox. That becomes the glyph's render quad, and the renderer skips it.This keeps Type0 character codes out of numeric AFM lookup. They address CIDs through a CMap, so a matching standard-font name does not make the numeric slots interchangeable. Named-glyph lookup and simple-font handling retain their existing behavior. No font data or rendering-library code changes are included.
Evidence:
The existing corpus already has a second example:
10749817875312063576_009.pdf, page 5, equation (9) in Nonideal Flow Models in Homogeneous Chemical Reactors. Its embedded SymbolMT TrueType subset puts=at CID 32. The same fallback removes the equals signs, and this change restores them. No new dataset PDF is needed.I also reproduced the original report with the unmodified Georgia DDS manual, SHA256
e311de54ad0ee6f5913e18e694914a560dac3592987437290c271f53c0c7fa71. Its heading goes from 5633 to 6279 dark pixels; PDFium has 6406.The evidence snapshot includes both full-page comparisons, a portable public-API reproducer, the complete corpus table, the 52-page manual table, and test logs. It is pinned separately from the fix, so the PR diff stays at three source files.
All 457 corpus PDFs were hash-checked at dataset revision
12531342c926708dcb1387ea25213db4603a7255. The comparison covers all 954 pages in the repository's default selection, plus all 52 pages of the manual. Images and corpus metrics use pypdfium2 5.15.2 for both builds, with the repository's RGBA MAE and luminance threshold of 12:=cropThe equals crop still has edge-pixel differences: its changed-pixel ratio rises slightly while MAE falls. The missing glyph returns, with 72 dark pixels against PDFium's 74, up from zero. The enforced crop test checks both metrics, image size and that the reference contains ink.
The affected character-cell diff is empty. Counts and complete JSON hashes of character, word and line cells are unchanged on all 954 corpus pages and all 52 manual pages. I did not rerun Docling's OCR pipeline.
The three small MAE increases and the validation limits
Of the 954 corpus pages, 934 have identical pixels, 17 have lower MAE, and 3 have slightly higher MAE at scale 2:
15643741724817381873-1.pdf/ 1926659986622553168-1.pdf/ 2indexed_iccbased.pdf/ 1These are repeatable differences. Each build produced identical hashes over three fresh renders of each page, matching the initial corpus run. I also checked scales 1, 2, 3 and 4; the maximum MAE increase remains 0.000086678. The scale table, diagnostic details, and magnified comparisons retain the full results. At scale 1 the second page also has a glyph that stops hitting the old 0.5-pixel early return and moves closer to PDFium; the scale-2 channel-change bounds above do not describe every scale.
On these three pages, baseline positions, character codes, embedded font identities, instruction order and non-text instructions are unchanged. Reconstructing the effective font size from the exported quads and metrics gives a maximum difference of 1.84e-12 pixels at scale 2, consistent with rounding sensitivity. This is a local diagnostic, not an instrumented Blend2D trace or a guarantee of zero risk on other platforms.
For the original manual, 28 pages have lower MAE, 13 are unchanged, and 11 increase by at most 0.000007737. Those increases involve 1–24 changed pixels per page, with a maximum channel difference of 9/255. The full table is linked above.
The added tests distinguish the reproductions from controls:
Local results:
uv lock --checkand whitespace checks pass.The native builds use Windows x64, Python 3.12.10 and GCC 13.1.0 in Release mode. macOS/Linux and the complete stored-groundtruth suite have not run locally. Remote CI is separate from these local checks.
Ground truth changes:
Normalized render-instruction output changes on 145 selected pages in 81 documents, including 125 pages whose pixels remain identical. Changes are limited to text quads and normalized font ascent/descent: they now use PDF font metrics instead of unrelated numeric AFM entries. Instruction counts, ordering, mappings, baseline anchors and all non-text instruction data are unchanged.
The per-page field list and hashed normalized outputs are included in the evidence. This is a fresh main/fix comparison using the repository's three-decimal normalization. Stored remote groundtruth blobs were not downloaded or rewritten, and
HF_DATASET_REVISIONis unchanged. The following pages need instruction-groundtruth review/update through the maintainers' process for the reason above.All 145 pages, grouped by document
0b9f86cde166b9ab_0018.pdf: 110040284511886890357_002.pdf: 1, 210446922681186632184_002.pdf: 810572911635253446040-13.pdf: 6, 8, 10, 11, 1610749817875312063576_009.pdf: 5, 6, 910869285887791127292_005.pdf: 111033738705194660817-2.pdf: 111111108925592240862_007.pdf: 1, 2, 4, 5, 611794545469969901016-2.pdf: 111831013430589524848.pdf: 112014666386554038795_001.pdf: 1, 212025925399039829714_014.pdf: 13, 2412076365341177918911_001.pdf: 1, 2, 3, 41265837786161076401-1.pdf: 113828292398120620566-1.pdf: 114327672304519301096-2.pdf: 1, 2, 3, 5, 614445963425523837694-2.pdf: 1, 214453006507210496387-6.pdf: 2, 4, 6, 8, 1614770497121209673752.pdf: 114911938403731335361-1.pdf: 1, 215643741724817381873-1.pdf: 116293520921922171261-1.pdf: 1, 2, 316338905573487211761-1.pdf: 116687286043211380927-1.pdf: 1, 4, 7, 15, 1717791c05056ff856_0022.pdf: 119246029784032167-4.pdf: 2, 3, 4, 5, 62203.01017v2.pdf: 1, 8, 122213713933117287130-1.pdf: 1, 22305.03393v1.pdf: 7, 8, 103040977140767198439-1.pdf: 1335fea0ba454ba15_0004.pdf: 13584964174907633090-1.pdf: 1, 23848385316711437784-4.pdf: 1, 2, 3, 43934995182849940852-6.pdf: 1, 2, 3, 5, 63960996440028888999-8.pdf: 44796728975040539044-1.pdf: 14865216256588543301.pdf: 1, 5, 6, 7, 84c9bb33c9466bc71_0001.pdf: 15d17819264eac373_0016.pdf: 15d969923927df690_0006.pdf: 16439951649404873078-1.pdf: 16515533261975788184-1.pdf: 16789603da459eee1_0017.pdf: 16909c04bc2f0fd31_0003.pdf: 17345174473351516034-1.pdf: 17474312693447806103-1.pdf: 17933217462246965845-6.pdf: 4, 6, 78313873609013042575-4.pdf: 1, 2, 3, 4, 58780050732516639302-1.pdf: 1, 5, 6, 7, 8926659986622553168-1.pdf: 1, 29528c2089295e005_0001.pdf: 1985a9a85d020ac41_0003.pdf: 1b766d7d18edf92b1_0002.pdf: 1bitmap_decoding_01.pdf: 1da04f7ddb6f59004_0025.pdf: 1dln_104f22abf9f481aa323444ac8b404fca3d52df55c9397c294bc43075f2c078a8.pdf: 1dln_116ab5b0f99390658c24887efe0b34060dad4eb8809fa90c2794e933d6ba54a9.pdf: 1dln_1f3567ab5e1bd6150bbbc9c591e05df3f4b34331ce8ca7bac69e2b038812b64d.pdf: 1dln_24f6a9fb5a1343ac9cd2ccb2ba409fb06bedca068c01d0b9bb9879e29288806f.pdf: 1dln_835d4dd76390a0ccb3d5b257f3ce021d3a03e2b1249d68558821930fb8bdf069.pdf: 1dln_d125e23af5add755072795fca6e91d10c6120abdcb3c2c9181e7f3a3bbf402f6.pdf: 1dln_dcea5d0eb77fbcac9289cc49d9388348fa023cd75cc737f3cc93c1dd60271f4d.pdf: 1eac6df0b2a6f9cf0_0002.pdf: 1ead2c552b489aac1_0004.pdf: 1ec62f8dc0265e965_0018.pdf: 1fillable_form.pdf: 1font_02.pdf: 1font_03.pdf: 1font_08.pdf: 1indexed_iccbased.pdf: 1jbig2_test_01.pdf: 1jpeg_2000_01.pdf: 1ligatures_01.pdf: 1redp5110_sampled.pdf: 14right_to_left.pdf: 1right_to_left_01.pdf: 1right_to_left_03.pdf: 1right_to_left_04.pdf: 1rotated_image.pdf: 1stream_parameter_misinterpretation_01.pdf: 1type3_fonts.pdf: 1Checklist:
main(88f1c388), one signed-off commit and no merge commits