Layout: two departures from Word in `restore_word_line_heights`, the line gap and pictures under proportional spacing

[hadim]

picture-line-264.pdf

Since #137 and #138, rdocx and Word give the same page count on the documents I test, and rdocx is
closer to Word than LibreOffice on body prose. This is the separate issue announced in PR #148 ("Word's
single line height ... deserves its own issue rather than riding on this one"), with the figures measured
there. What still moves page breaks comes down to one function,
restore_word_line_heights (rdocx-layout/src/convert.rs:137), which computes every auto line as
(ascent + descent) * line / 240 with line_gap = 0. Two consequences, measured separately below. I would
ask for both to be tested against Word-exported PDFs of neutral fixtures, which I can provide if useful
(I have Word on macOS), so that a fix of one does not move the other.

1. A single-spaced text line leaves out the font's line gap

restore_word_line_heights (rdocx-layout/src/convert.rs:137) sets line_gap = 0.0 and bases the line
height on ascent + descent only. For Arial (and Liberation Sans, metric-compatible, which is what renders
here) that is 2288 / 2048 = 1.117 em. Word's single line for Arial is ascent + descent + the hhea line gap
(67 units), 2355 / 2048 = 1.150 em: 13.8 pt at 12 pt, the value commonly quoted for Word, and what I measure
on PDFs exported from Word. LibreOffice matches Word here.

import os, re, shutil, subprocess, tempfile
import rdocx
from docx import Document
from docx.shared import Pt
from docx.oxml.ns import qn


def build(path, line):
    d = Document()
    st = d.styles["Normal"]; st.font.name = "Arial"; st.font.size = Pt(12)
    fonts = st.element.get_or_add_rPr().find(qn("w:rFonts"))
    for a in ("w:ascii", "w:hAnsi", "w:cs", "w:eastAsia"):
        fonts.set(qn(a), "Arial")
    for i in range(40):
        p = d.add_paragraph("Line %02d lorem ipsum dolor sit amet" % (i + 1))
        ppr = p._p.get_or_add_pPr()
        sp = ppr.makeelement(qn("w:spacing"), {})
        sp.set(qn("w:before"), "0"); sp.set(qn("w:after"), "0")
        sp.set(qn("w:line"), str(line)); sp.set(qn("w:lineRule"), "auto")
        ppr.append(sp)
    d.save(path)
    return path


def pitch(pdf):
    out = subprocess.run(["pdftotext", "-bbox", "-f", "1", "-l", "1", pdf, "-"], capture_output=True, text=True).stdout
    ys = sorted({round(float(m.group(1)), 2) for m in re.finditer(r'<word xMin="[\d.]+" yMin="([\d.]+)"[^>]*>Line</word>', out)})
    d = [b - a for a, b in zip(ys, ys[1:])]
    return sum(d) / len(d)


def lo_pdf(docx):
    t = tempfile.mkdtemp(); shutil.copy(docx, os.path.join(t, "p.docx"))
    subprocess.run(["soffice", "--headless", "--convert-to", "pdf", "p.docx", "--outdir", t], cwd=t,
                   check=True, capture_output=True, timeout=300)
    return os.path.join(t, "p.pdf")


for line in (240, 264):
    f = build("lh_%d.docx" % line, line)
    open(f + ".rd.pdf", "wb").write(rdocx.Document.open(f).to_pdf())
    print(f"w:line={line}: rdocx {pitch(f + '.rd.pdf'):.2f} pt/line | LibreOffice {pitch(lo_pdf(f)):.2f} pt/line | "
          f"Word, 1.150 em x {line / 240:.2f}: {12 * 2355 / 2048 * line / 240:.2f} pt")

w:line=240: rdocx 13.41 pt/line | LibreOffice 13.80 pt/line | Word, 1.150 em x 1.00: 13.80 pt
w:line=264: rdocx 14.75 pt/line | LibreOffice 15.15 pt/line | Word, 1.150 em x 1.10: 15.18 pt

What it does to a real document: on a long report where Word and rdocx now give the same page count, fewer
than half of the pages start on the same line in both, and on a page holding a long table
rdocx fits one more row than Word, which pushes the empty paragraph that follows the table onto a page of
its own before the next pageBreakBefore. That is the symptom of #138; with its row fix in, what remains
on that file is the line height, so I file it here rather than reopening #138. Every cached page number written from rdocx's layout (TOC,
PAGEREF, NUMPAGES) drifts accordingly. The gap is 0.39 pt per line at 12 pt, about one extra line every 35.

Arial is not the worst case. The same measurement on the other bundled families, against Word's single
line computed from the font tables (ascent + descent + line gap; Word's documented rule is the win ascent
and descent plus the external leading, which gives the same value for these fonts):

font requested      rendered with (bundled)  rdocx      Word
Calibri             Carlito                  1.000 em   1.221 em   (11.00 pt against 13.43 pt at 11 pt)
Arial               Liberation Sans          1.117 em   1.150 em
Times New Roman     Liberation Serif         1.107 em   1.150 em
Cambria             Caladea                  1.172 em   1.172 em

Calibri, the default body font of Word for years, loses its whole line gap (the bundled Carlito-Regular.ttf declares hhea ascent 1536, descent -512 and line gap 452 on 2048 units), so a
Calibri document is laid out about 22 per cent denser than Word lays it out. The metrics are the hhea
values read through ttf-parser (oxml-layout/src/font.rs:1220 to 1228); the gap is dropped in
restore_word_line_heights. Acceptance: a font matrix (at least the four bundled families and their
Microsoft names, at two sizes and at line 240 and 264) asserting constant expected pitches, rather than a
comparison with another renderer.

2. Proportional line spacing scales the height of a line holding an inline picture

In a paragraph with w:spacing w:line="264" w:lineRule="auto" (1.1 lines; the umbrella's report fixture
uses 276, 1.15 lines, with the same effect), rdocx gives a line holding a 400 pt inline picture a height of 440 pt, and the
caption below it starts 441.3 pt after the text above. Word keeps the line at the picture's height: the
attached picture-line-264.pdf is this same fixture exported from Word for Mac, where the distance is
402.7 pt (400.9 pt in LibreOffice).

The visible consequence is with tall figures: a picture and its one-line caption that share a page in Word
are split across two pages by rdocx. The umbrella's report fixture (#158) shows it: figure 1 ends page 5 and its
caption opens page 6, where LibreOffice keeps both on page 5.

The rule in restore_word_line_heights (rdocx-layout/src/convert.rs:137) multiplies ascent + descent
by line / 240 for every line, and for a line holding an inline picture the ascent is the picture's height.

import io, os, re, shutil, subprocess, tempfile
import rdocx
from docx import Document
from docx.shared import Pt
from docx.oxml.ns import qn
from PIL import Image


def build(path, line):
    d = Document()
    ppr = d.styles.element.find(qn("w:docDefaults")).find(qn("w:pPrDefault")).find(qn("w:pPr"))
    sp = ppr.find(qn("w:spacing"))
    if sp is None:
        sp = ppr.makeelement(qn("w:spacing"), {}); ppr.append(sp)
    for k, v in (("w:after", "0"), ("w:before", "0"), ("w:line", str(line)), ("w:lineRule", "auto")):
        sp.set(qn(k), v)
    d.add_paragraph("Lead paragraph above the picture.")
    png = io.BytesIO(); Image.new("RGB", (400, 400), (210, 210, 215)).save(png, "PNG"); png.seek(0)
    d.add_picture(png, height=Pt(400))
    d.add_paragraph("Caption below the picture.")
    d.save(path)
    return path


def lead_to_caption(pdf):
    out = subprocess.run(["pdftotext", "-bbox", pdf, "-"], capture_output=True, text=True).stdout
    lead = float(re.search(r'yMax="([\d.]+)">Lead</word>', out).group(1))
    cap = float(re.search(r'yMin="([\d.]+)"[^>]*>Caption</word>', out).group(1))
    return cap - lead


for line in (240, 264):
    f = build("plm_%d.docx" % line, line)
    doc = rdocx.Document.open(f)
    pic = [fr for fr in doc.layout() if fr.body_index == 1][0]
    open(f + ".rd.pdf", "wb").write(doc.to_pdf())
    t = tempfile.mkdtemp(); shutil.copy(f, os.path.join(t, "p.docx"))
    subprocess.run(["soffice", "--headless", "--convert-to", "pdf", "p.docx", "--outdir", t], cwd=t, check=True,
                   capture_output=True, timeout=300)
    print(f"w:line={line}: picture 400.0 pt | rdocx picture paragraph {pic.bounds.height:.1f} pt | "
          f"lead to caption: rdocx {lead_to_caption(f + '.rd.pdf'):.1f} pt, "
          f"LibreOffice {lead_to_caption(os.path.join(t, 'p.pdf')):.1f} pt")

w:line=240: picture 400.0 pt | rdocx picture paragraph 400.0 pt | lead to caption: rdocx 400.0 pt, LibreOffice 398.4 pt
w:line=264: picture 400.0 pt | rdocx picture paragraph 440.0 pt | lead to caption: rdocx 441.3 pt, LibreOffice 400.9 pt

Environment: main at 9a7ed714 (S75), release build, linux x86_64, LibreOffice 24.2.7.2 for the
comparison, python-docx 1.2.0 and Pillow for the fixture.