Skip to content

Align split_run, bookmarks and story comments on the direct body index - #179

Open
hadim wants to merge 4 commits into
tensorbee:mainfrom
hadim:fix/direct-body-index-coordinates
Open

hadim wants to merge 4 commits into
tensorbee:mainfrom
hadim:fix/direct-body-index-coordinates

Conversation

@hadim

@hadim hadim commented Sep 27, 2026

Copy link
Copy Markdown
Contributor

Summary

  • rdocx: Document::split_run resolves its first argument as the direct
    body child index, the index that find_content_index, RunPosition,
    add_comment, insert_paragraph and layout().body_index already use. It
    reads and writes back the paragraph at that one slot. An index that names a
    table, a block content control or preserved XML returns an error naming that
    kind (body index 1 is a table, not a paragraph) instead of splitting
    another paragraph. rdocx-py: Document.split_run also accepts a
    Paragraph handle in place of the index, which reaches paragraphs inside
    block content controls. A table cell paragraph handle is refused with a
    ValueError. The rustdoc of RunPosition::body_index and
    Document::paragraph now says which index each takes. Stub, typing smoke,
    HLD 03 and HLD 10 updated.
  • rdocx: additive BookmarkRef::direct_range(). It reports the bookmark
    with the direct body index that add_bookmark took, and is None when
    either marker sits in a table cell or a block content control. range()
    keeps its recursive paragraph ordinal. HLD 03 and HLD 10 updated.
  • rdocx: add_story_comment resolves a body paragraph item to its direct
    body slot and a cell paragraph item to the direct cell paragraph at the same
    position, so a block content control before the target no longer moves the
    comment into the control. This one is not in the issue and has the same
    cause.
  • Archive row of rdocx re-recorded.

Part of #163. This covers the index class of the issue: split_run, the
bookmark listing and the story comment path now agree with
find_content_index. What remains of #163 is left to later PRs: commenting a
paragraph inside a block content control, which needs an API shape decision,
the add_comment_on_text helper, and rdocx comment add --anchor.

Why

split_run took an argument named body_index and resolved it with
Document::paragraph, which counts paragraphs, skips body tables and enters
block content controls. Every other index-taking body API, and the spec of
F-X109 and HLD 10, use the direct body child index. Splitting a run then
anchoring a comment at the index find_content_index returned therefore split
one paragraph and commented another as soon as a table or a block content
control came first. The issue's reproduction printed ['Gamm', 'a paragraph at the end.'] and the workflow_docx.py step "comment on a piece of text after a
table" of #158 failed on fixture-report.docx with a run length error. The
same confusion made bookmarks() read back a bookmark with a different body
index than the one add_bookmark took, after a table with more than one cell
or a block content control.

add_story_comment counted the paragraph items before its target, which are
direct paragraphs only, and then resolved that count through a walker that
enters block content controls. On a body or a cell where a block content
control with paragraphs precedes the target, the comment landed silently on a
paragraph inside the control. On the report fixture that is any body
paragraph after the table of contents control.

Notes

  • Behaviour fix, for the release notes. The integer argument of
    split_run (Rust and Python) is now the direct body index, as HLD 10 and
    F-X109 already stated. This was preferred over keeping the paragraph index
    and adding a second entry point, which would have left a fourth meaning of
    body_index in the API. A caller that passed a doc.paragraphs index to
    split_run in a body where a table or a block content control precedes the
    target now addresses a different body child. Such callers pass the index
    from find_content_index or, in Python, the Paragraph handle itself. In
    Rust, paragraph_mut(i).split_run(...) still splits by paragraph index, and
    content_index_of_paragraph converts. Suggested line: "split_run now takes
    the direct body index that find_content_index returns, like RunPosition
    and add_comment, and Python also accepts a Paragraph handle. Code that
    passed a paragraph index with a table or content control before the target
    addressed another paragraph and must switch." The out-of-range message
    changes from body paragraph index N is out of range to body index N is out of range. A non-integer argument raises TypeError: body_index must be an int or a Paragraph handle, and a negative one still raises OverflowError.
  • Python handle path. The handle is checked for ownership and revision like
    find_content_index does. A handle to a direct body paragraph is converted
    with content_index_of_paragraph and takes the same native path as its
    index, so a zero or end offset leaves bytes, revision and the layout cache
    alone. A handle to a paragraph inside a block content control splits through
    Document::paragraph_mut, which clears the layout cache even on those
    offsets (bytes and revision stay unchanged), and HLD 03 now says so. The
    revision advances only when a continuation run is created, measured on the
    paragraph that was split.
  • Cell handles are refused. A cell Paragraph handle indexes
    CellRef::paragraph, which counts the paragraphs inside cell content
    controls, while Cell::paragraph_mut counts direct paragraphs only. With a
    cell [sdt{In control.}, Direct one.], splitting through
    cell.paragraphs[0] would split Direct one.. split_run never accepted
    cell paragraphs before this PR, so refusing them loses nothing. The same
    read and write mismatch already affects the other Python cell handle
    mutators (run.text, add_run, the font and paragraph format setters).
    Making Cell::paragraph_mut
    enter cell content controls would fix all of them and let split_run accept
    cell handles. That is a behaviour change of a public Rust accessor outside
    Document.split_run(body_index, ...) counts paragraphs only, so it splits the wrong paragraph once a table precedes it #163, so it is left for the maintainer to decide, as its own issue.
  • Bookmarks. direct_range() is additive and range() is unchanged,
    because REF numbering in field.rs keys on its recursive ordinal. The run
    indexes of direct_range() are the same accepted-view boundaries as
    range(). They equal the direct run index add_bookmark and add_comment
    take only in a paragraph without inline content controls or tracked
    insertions, which the rustdoc and HLD 10 now state. Reconciling the run axis
    is Comment run positions skip runs inside inline content controls, so comments anchor on the wrong text without an error #172 and is left to that PR. The Python bookmark binding of rdocx Python bindings: the complete list of what a production editing chain still needs, as one checklist #168 can
    expose direct_range() from the start.
  • Story comments. The body route takes the count of direct story items
    before the target as its direct body slot, from the scan it already has, and
    requires a typed paragraph there. The body section properties are serialized
    last, so they never precede a paragraph. This is the slot
    StoryItemRef::direct_body_index reports. The cell route takes the k-th
    direct CellContent::Paragraph. No extra scan is added.
  • API impact. No signature change in Rust. BookmarkRef gains a private
    field and a public accessor, and it has no public constructor, so nothing
    breaks. Python widens the first parameter of split_run to int | Paragraph and keeps its name body_index for keyword callers.
  • Conflicts to expect. The rdocx archive row, like every PR of this wave
    that touches rdocx. Its "Measured on" date is left at 2026-09-26, as the
    shared re-record of this wave does, so the maintainer can re-measure once
    at merge time. Document::text sits a few lines above split_run, so the
    Producer traits: a matrix over every operation, and what still fails in it and around it #160 content control readers PR may touch nearby lines.

Tests

Added to crates/rdocx/tests/regression_test.rs, as
mod direct_body_index_coordinates right after
checked_table_cell_comment_range_is_atomic_and_reopens:

  • split_run_takes_the_direct_body_index_after_a_table: the issue
    reproduction, then add_comment at the same index anchors on Beta and
    insert_paragraph at the same index lands right before it.
  • split_run_takes_the_direct_body_index_after_a_block_content_control.
  • split_run_names_the_body_child_that_is_not_a_paragraph: table, content
    control, a w:customXml body child kept as preserved XML, and out of range,
    with the bytes unchanged.
  • bookmark_direct_range_reports_the_index_add_bookmark_took: after a 2x2
    table and a block content control, direct_range() equals the range given
    to add_bookmark while range() keeps ordinal 7.
  • bookmark_direct_range_is_none_when_a_marker_is_nested: in a cell, in a
    control, and spanning from a direct paragraph into a control.
  • story_comment_after_a_block_content_control_anchors_on_its_paragraph and
    story_comment_in_a_cell_after_a_block_content_control_anchors_on_its_paragraph.

The five tests that use existing APIs failed on 9a7ed71 before the fix: the
split output was ['Gamm', 'a paragraph at the end.'] as in the issue, the
split after a control hit Control two., the table index split a control
paragraph, and both story comments landed on Control one..

Added to crates/rdocx-py/tests/test_core.py, next to the existing split
tests:

  • test_split_run_takes_the_direct_body_index_after_a_table: the issue
    reproduction, the table index error, the TypeError message and the
    OverflowError of a negative index.
  • test_split_run_accepts_a_paragraph_handle_in_a_control_or_the_body: a
    no-op split keeps run handles and a split stales them, for a control
    paragraph and for a direct paragraph. Both paragraphs of a cell that holds
    a block content control are refused with the bytes unchanged. A handle from
    another document is refused.
  • test_story_comment_after_a_block_content_control_anchors_on_its_paragraph,
    which failed on the extension built before the story comment commit.

typing_smoke.py gains split_run(paragraph, 0, 1).

Run on this branch:

  • cargo fmt --all --check: clean.
  • cargo clippy -p rdocx -p rdocx-py --all-targets --all-features -- -D warnings: clean.
  • cargo test -p rdocx --no-fail-fast: regression 568 passed and 7 ignored,
    doctests 2 passed. The lib binary has the two known local
    failures (large_word_and_presentation_pdfs_preserve_logical_reading_order,
    word_and_powerpoint_chart_pixels_are_identical, pinned pdftotext and
    rasterizer). The integration binary has 314 passed and 5 failed: the three
    known ones (odt_reader_matches_pinned_libreoffice_structure,
    public_authored_theme_and_fonts_match_pinned_word_resolution,
    sanitized_public_authoring_fixture_passes_every_conformance_stage) plus
    every_conditional_table_region_matches_word and
    section_page_semantics_match_pinned_libreoffice_render, which assert the
    pinned LibreOffice 26.2.5.2 against the local 26.8.0.3 and fail the same
    way on 9a7ed71.
  • cargo test -p rdocx-py: 2 passed with DYLD_LIBRARY_PATH set to the
    libpython the build linked, and cannot load libpython without it.
  • RUSTDOCFLAGS="-D warnings" cargo doc -p rdocx --no-deps --all-features:
    clean.
  • Python 3.12 venv, extension built with maturin develop: pytest crates/rdocx-py/tests 68 passed, 2 failed, both in
    test_rendering_threads.py on the pinned pdfinfo 26.01.0 (local 26.09.0).
    mypy --strict on typing_smoke.py and the package: clean.
    mypy.stubtest rdocx: clean.
  • python3 scripts/hash_harness.py --check: 49 entries match.
  • python3 scripts/prose_check.py: 0 violations.
  • python3 scripts/readme_doctests.py, the Docs job check that includes the
    archive rows: passes after the re-record.
  • The issue's python-docx reproduction prints ['Beta', ' paragraph after the table.'], and a cell paragraph handle now raises split_run does not accept a table cell paragraph handle. workflow_docx.py on
    fixture-report.docx, rerun on the final branch, passes "comment on a piece
    of text after a table". Its three other failures (table of contents rebuild
    on a fresh open, replacement inside content controls, redline) belong to
    other issues of Production readiness for editing real docx and pptx files: an acceptance contract, two realistic fixtures and two matrices #158.

Document::split_run resolved its first argument with
Document::paragraph, which counts paragraphs, skips tables and enters
block content controls. RunPosition, find_content_index, add_comment
and insert_paragraph use the direct body child index instead, which is
also what HLD 10 and F-X109 state for split_run. With a table or a
block content control before the target, splitting a run then
anchoring a comment at the same index split one paragraph and
commented another, or failed on a run length.

split_run now reads the paragraph at that direct slot and writes it
back to the same slot, so one resolution serves both steps. An index
that names a table, a block content control or preserved XML returns
an error naming that kind instead of splitting another paragraph. The
Python method also accepts a Paragraph handle, which reaches
paragraphs inside block content controls, and it measures the
revision bump on the paragraph it split. A handle to a direct body
paragraph takes the same native path as its index. A table cell
paragraph handle is refused, because its index counts the paragraphs
inside cell content controls and Cell::paragraph_mut does not. The
rustdoc of RunPosition and Document::paragraph and the HLD 03 and 10
statements now name the index each one takes.

GitHub issue tensorbee#163.
Document::add_bookmark takes a RunRange whose body index is the direct
body child index, but Document::bookmarks reports the paragraph ordinal
counted recursively through table cells and block content controls. A
bookmark added after a table with more than one cell, or after a block
content control, read back with a different body index than the one it
was added with, so that index could not be passed back to add_comment,
split_run or add_bookmark.

The recursive ordinal stays in range() because REF numbering in
field.rs reads it as its paragraph key. BookmarkRef gains an additive
direct_range() that carries the direct body child index, recorded while
the paragraphs are collected, and is None when either marker sits in a
table cell or a block content control. Run indexes are unchanged and
stay the accepted-view boundaries of range(). They equal the direct run
indexes add_bookmark takes only in a paragraph without inline content
controls or tracked insertions, which GitHub issue tensorbee#172 covers.

GitHub issue tensorbee#163.
Document::add_story_comment resolved a StoryRunRange paragraph by
counting the paragraph items before it, which are the direct
paragraphs of the owner only, and then took that many paragraphs
through a resolver that also enters block content controls. When a
block content control with paragraphs came before the target, in the
body or in a table cell, the comment was anchored on a paragraph
inside the control, or refused when that paragraph had too few runs.

A body paragraph item now resolves to its direct body slot, the count
of direct story items before it, which is the slot
StoryItemRef::direct_body_index reports. A cell paragraph item
resolves to the direct cell paragraph at the same position. Neither
side enters a block content control, so the count and the lookup
agree.

GitHub issue tensorbee#163.
The direct body index changes grow the rdocx package, mostly through
the new regression tests, so the crates.io archive row of the root
README and its ARCHIVE_MEASUREMENTS entry in scripts/readme_doctests.py
are re-measured with cargo package, as the Docs job requires.

GitHub issue tensorbee#163.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant