Repository navigation
fix: look up standard-font metrics by the declared encoding's glyph name - #461
Merged
PeterStaar-IBM merged 4 commits intoNov 13, 2026
Merged
Conversation
Without /Widths, the advance and glyph bbox of a non-embedded standard font come from its AFM, which numbers glyphs in the font's built-in (Standard) encoding. The raw character code was looked up directly, so a font declaring /WinAnsiEncoding got the metrics of whatever Standard has at that code: `©` (0xA9) the width of `quotesingle`, `é` (0xE9) that of `Oslash`, the apostrophe (0x27) that of `quoteright`. Codes with no Standard glyph, like `í` (0xED), fell back to 500. docling-project#370 already records the width of unencoded (`C -1`) glyphs by name. Record their bbox by name too, add name-based width and bbox accessors to base_font, and resolve the code through get_glyph_name() (declared /Encoding and /Differences) before the code-based lookup in get_width(), nonzero_fallback_width() and has_char_bbox()/ get_char_bbox(). Fonts without an explicit encoding are unchanged. Add 7413869934284023443-1.pdf p2 to the cited-page tests: its Helvetica-Bold maps 193 to Aacute through /Differences, and the heading's "Á" was drawn with the width of `grave`. Fixes docling-project#403. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: usercjr <92197541+usercjr@users.noreply.github.com>
Contributor
|
✅ DCO Check Passed Thanks @usercjr, all your commits are properly signed off. 🎉 |
Contributor
Merge Protections🟢 Merge protection satisfied — ready to merge. Show 1 satisfied protection🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
PeterStaar-IBM
self-requested a review
November 13, 2026 06:37
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
PeterStaar-IBM
previously approved these changes
Nov 13, 2026
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
PeterStaar-IBM
approved these changes
Nov 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #403.
Without
/Widths, the advance and glyph bbox of a non-embedded standard font come from its AFM, which numbers glyphs in the font's built-in (Standard) encoding.get_width(),nonzero_fallback_width()andhas_char_bbox()/get_char_bbox()looked the raw character code up there, ignoring the font's declared/Encodingand/Differences. Every code whose glyph differs from Standard's got the metrics of Standard's glyph at that code: WinAnsi©(0xA9) the width ofquotesingle,é(0xE9) that ofOslash, the apostrophe (0x27) that ofquoteright, a/DifferencesAacuteat 193 that ofgrave. Codes with no Standard glyph (í, 0xED) fell back to 500: in the Latin AFMs those are the unencoded (C -1) glyphs, all accented letters, which nothing looked up by name.#370 already records the width of
C -1glyphs by name and addedbase_font::has_glyph_name(), but nothing inpage_fontlooks a width up by name, so the cases above still fail onmain. This builds on it:base_font::initialise()also keeps the bbox ofC -1glyphs by name, andbase_fontgets name-basedget_glyph_width(),has_glyph_bbox()andget_glyph_bbox().get_width(),nonzero_fallback_width()andhas_char_bbox()/get_char_bbox()resolve the code throughget_glyph_name()first and fall back to the code-based lookup. Fonts without an explicit encoding or/Differencesare unchanged. The bbox change also covers thetext.hglyph-rect path @wittjeff found on Wrong advance widths for non-ASCII characters in non-embedded standard-14 fonts with /WinAnsiEncoding (© gets the width of quotesingle) #403, which calls these functions.This changes width and bbox metrics only, not glyph-to-Unicode decoding. #459 also changes
page_font.h, in other functions; the hunks don't overlap and the branches merge cleanly.Evidence:
Measured on
main(7cf06e6) and this branch, same local build setup (Windows, MSYS2 MinGW g++ 15.2), dataset at the pinned revision.Text cells,
main→ this branch:7413869934284023443-1.pdfp2. Helvetica-Bold with/Differences [193 /Aacute 211 /Oacute]and no base encoding.Ágetsgrave's 333 units instead of 722, so the heading word is split. Accented letters elsewhere on the page move the words after them. The page goes from 414 to 413 word cells; 54 cells change.Á(228.60–233.91),REA(240.12–267.71)ÁREA(228.60–267.71)COMPETICIÓN(292.80–383.03)COMPETICIÓN(292.80–386.63)repescará,(155.40–201.85),si(204.27–210.83)repescará,(155.40–197.86),si(200.28–206.84)7413869934284023443-1.pdfp1 has the same heading, soÁandREAbecomeÁREAandCOMPETICIÓNwidens the same way (442 → 441 word cells).annots_02.pdfp1. Helvetica, WinAnsi apostrophe. 4 word cells change:don't(335.38–357.06)don't(335.38–356.75)forget(359.84–385.41)forget(359.53–385.10)annots_01.pdfp1. Helvetica, WinAnsi€and others. 23 word cells change, for example€174.77–179.27 → 174.77–179.78, then95.00181.77–204.29 → 182.28–204.80.Rendering against pypdfium2, with
measure_image_pairand the page selection oftest_regression_pypdfium_render.py(954 pages of 457 documents, dataset b09d0574):7413869934284023443-1.pdfp2annots_01.pdfp17413869934284023443-1.pdfp1annots_02.pdfp19 other pages differ by at most 3e-05 in
mean_abs_error, and no page gets worse by more than 1e-05. The other 941 pages are unchanged. Corpus mean: 2.45693 → 2.45591. Pages above the 9.0 tolerance: 13 → 13.Panels are pypdfium2 |
main| this branch:Some
mandcglyphs of the body text are missing from both docling-parse panels. That is a separate problem, the same onmain, and this PR doesn't change it.These numbers come from Windows, where non-embedded Helvetica resolves to a different system font than on Linux. I also checked the cited page on Ubuntu 24.04 with the font packages that
checks.ymlinstalls:main3.992 / 0.0833, this branch 3.349 / 0.0798, so the test fails onmainand passes here on Linux too.pytest tests/test_unit_*.pythere: 695 passed, 10 skipped.Tests:
test_regression_pypdfium_cited_pages.pygainsstandard-font-differences-width(7413869934284023443-1.pdfp2, limits 3.6 / 0.081). It fails onmainand passes here.tests/test_unit_standard_font_widths.pygains cases for WinAnsi©,é,íand the apostrophe, which fail onmainand pass here.pytest tests/test_unit_*.pyon Windows: 642 passed, 61 skipped (main: 640 passed). Two tests that feat: working on parallel efficiency #435 added fail on Windows, the same onmainand on this branch:test_reference_preflight_handles_native_abort[-6](signal.Signals(6)doesn't exist there) andtest_mid_run_commit_requires_exact_source_equivalence[None].Ground truth changes:
I haven't regenerated the stored ground truth. Comparing the word cells of all 954 pages between
mainand this branch, 4 pages change, because the width or bbox of glyphs in non-embedded standard fonts changes:7413869934284023443-1.pdfp1 and p2: word, char and rendered output (ÁREAbecomes one word).annots_01.pdfp1: word, char and rendered output.annots_02.pdfp1: word, char and rendered output.Checklist:
main, no merge commitsThis PR was prepared with Claude Code (Claude Opus 5.5). The measurements and tests above were run locally on my machine.
🤖 Generated with Claude Code