Skip to content

Anchor comments on the runs Paragraph.runs lists, inside controls and on text - #191

Open
hadim wants to merge 8 commits into
tensorbee:mainfrom
hadim:fix/comment-anchoring-run-index
Open

hadim wants to merge 8 commits into
tensorbee:mainfrom
hadim:fix/comment-anchoring-run-index

Conversation

@hadim

@hadim hadim commented Sep 27, 2026

Copy link
Copy Markdown
Contributor

Stacked on #179 (branch fix/direct-body-index-coordinates). The first
four commits of this branch are #179 and are reviewed there. Please review
the last four commits only, or the diff against
fix/direct-body-index-coordinates. I will rebase once #179 lands.

Summary

  • rdocx, rdocx-oxml: comment and bookmark run positions use the
    accepted-view run index, the runs that Paragraph.runs, rdocx text --json,
    split_run and bookmarks() already count, including the runs inside inline
    content controls and tracked insertions. This covers add_comment,
    add_story_comment, rdocx comment add and add_bookmark. The markers go
    inside w:sdtContent when a range starts or ends between two runs of a
    control, and around the whole w:sdt when the range covers it. A range that
    cannot be anchored exactly is refused with an error, never shifted.
    remove_comment now also clears markers and reference runs that sit inside
    w:sdtContent, also in a document that gives the Word namespace another
    prefix. Bookmarks placed inside a control keep a distinct id when saving
    renumbers them, and read back their own text in the same session.
    ParagraphRef::runs() lists the same accepted-view runs as
    Paragraph::runs(). CLI --help, CLI README, rustdoc, HLD 03 and HLD 10
    updated. Closes Comment run positions skip runs inside inline content controls, so comments anchor on the wrong text without an error #172.
  • rdocx, rdocx-py: a comment reaches a paragraph inside a block content
    control. A body ContentLocation may have two segments, the control's story
    item index then the paragraph's position among the control's paragraphs, and
    the new Document::paragraph_story_location(paragraph_index) returns that
    path (or the one-segment path of a direct paragraph). Python
    StoryRunPosition(paragraph=p, run_index=i) builds the position from a
    Paragraph handle. rebuild_toc documentation now says that it drops the
    markers on the entries it replaces.
  • rdocx, rdocx-py: new Document::add_comment_on_text(anchor, occurrence, author, initials, text, date) and Python Document.add_comment_on_text(anchor, *, author, text, occurrence=0, initials=None, date=None). It splits the runs
    at both ends of the match and anchors the comment on them, with no index
    bookkeeping. A match whose commented range would also show other text, such
    as a field result, is refused.
  • Archive rows of rdocx, rdocx-oxml and rdocx-cli re-recorded on top of
    the stacked tree.

Part of #163. With #179, this covers the index class of the issue, commenting
a paragraph inside a block content control, and the add_comment_on_text
helper. What remains is rdocx comment add --anchor TEXT, left to the later
CLI PR so it lands after the CLI output policy work of #174.

Why

validate_run_range and the story comment path checked run_index against
paragraph.runs.len(), the direct runs, and the markers were written at that
direct index. The read APIs list the accepted view instead. In the issue's
paragraph before | control TARGET | after, Paragraph.runs and
text --json both report three runs, but runs [1, 2) anchored on after
with exit 0, and runs [2, 3) were refused as out of range, so the last run
could not be commented at all. add_bookmark and ParagraphRef::runs() had
the same mismatch, which the #179 notes already pointed at.

Separately, remove_comment walked into content controls but ignored their
preserved children, so w:commentRangeStart and w:commentRangeEnd inside
w:sdtContent (as Word writes them) stayed behind as orphans.

For #163, a paragraph inside a block content control had no position at all:
its direct body index names the control, a StoryRunRange over the control
item is refused because it is not a paragraph, and story_items lists no item
for the paragraphs inside a control. Commenting a piece of text also took a
text search, two split_run calls and index bookkeeping, which these index
mismatches made unreliable.

Notes

How a range is placed. CT_P::anchor_accepted_range looks at the runs on
both sides of each boundary. Their recursive owners are the paragraph and the
inline controls that hold them. It picks the outermost owner that holds the
first and the last run of the range and reaches both boundaries:

  • a range that covers a whole control gets its markers around the w:sdt,
  • a range that starts or ends between two runs of a control gets them inside
    its w:sdtContent, with the reference run right after the end marker,
  • nested controls follow the same rule, so a range over an inner control is
    written around it inside the outer one.

The markers hug the text as before: a start goes just before the first run of
the range and an end just after its last run. For a paragraph without inline
controls or tracked insertions the output is byte for byte what it was, and
the hash harness is unchanged.

What is refused, each with a message naming the boundary, and the document
unchanged:

  • a range that crosses the edge of an inline content control, for example from
    a direct run to between two runs of a control,
  • a boundary between two runs of one tracked insertion or move,
  • a boundary next to a tracked change inside a hyperlink, where the end marker
    would have to go inside the hyperlink before the change,
  • a boundary inside a control when the range continues into another paragraph,
  • for add_comment_on_text, a match whose range would also show text that the
    literal text leaves out, such as a w:fldSimple or complex field result
    between two of its runs.

A whole tracked insertion can be commented. Commenting across a control edge
could be written, since the markers are cross-structure annotations, but the
issue asked for exact anchoring or an error, so I kept the refusal.

Markers inside a control and the rest of the pipeline. The markers written
inside w:sdtContent use the fixed w prefix, also in a document that gives
the Word namespace another prefix, such as the ns0 of ElementTree-based
generators. CT_Sdt::remove_comment_anchors therefore recognizes w next to
the control's own prefixes, as the bookmark projection refresh already does.
Two other paths had to learn about markers inside a control:

  • Saving renumbers the added bookmarks in document order, and
    CT_P::remap_authored_bookmark_ids rewrote only the markers at paragraph
    level. A bookmark inside a control kept its old id, which could be the new
    id of another bookmark, and the saved file then failed to reopen with a
    duplicate bookmark id. The renumbering now also reaches the markers inside
    inline controls.
  • The in-session bookmark projection re-parses the paragraph on its own, where
    the source prefix of a control's preserved runs is not declared. It now also
    takes the prefixes of the paragraph's inline controls, so a bookmark inside
    or after a control of an ns0 document reads back its own text before a
    save. The bookmark after such a control read the wrong text before this
    branch too, since the projection counted none of the control's runs.

Behaviour changes, for the release notes.

  • The run_index of RunPosition, StoryRunPosition and rdocx comment add --start-run/--end-run, and the bookmark range of add_bookmark, now count
    accepted-view runs. In a paragraph without inline content controls or
    tracked insertions nothing changes. A caller who counted direct runs in such
    a paragraph addresses different runs now, which is the fix. Suggested line:
    "Comment and bookmark run indexes count the runs that Paragraph.runs and
    rdocx text --json list, including runs inside inline content controls and
    tracked insertions. A range that cannot be anchored exactly is an error."
  • ParagraphRef::runs() (Rust) returned direct runs only while
    ParagraphRef::run_count() and run() used the accepted view. It now lists
    the accepted view too. No caller in the workspace outside tests relied on
    the old list.
  • Two pinned tests are re-pinned on purpose.
    comment_insertion_keeps_content_at_the_hyperlink_end_boundary commented
    runs [0, 2) of a hyperlink whose tracked insertion follows run 1, and the
    old code silently widened the range over the insertion. That range is now
    refused, and the test comments [0, 3) instead, which covers the insertion
    and still checks that the content at the hyperlink end survives insertion
    and removal. new_bookmark_after_an_accepted_control_has_live_accepted_coordinates
    passed [0, 1) to mean direct target after a control. It passes [1, 2)
    now, the same index bookmarks() reports back.

API additions. No signature of a published type changes.

  • rdocx: Document::paragraph_story_location,
    Document::add_comment_on_text.
  • rdocx-oxml: RangeAnchor and RangeAnchorError, and the hidden
    CT_P::anchor_accepted_range, CT_P::accepted_literal_text,
    CT_P::split_accepted_literal_span, CT_P::comment_range_text and
    CT_Sdt::remove_comment_anchors.
    CT_P::insert_bookmark_start and insert_bookmark_end stay, since TOC
    heading bookmarks still use them.
  • Python: StoryRunPosition(paragraph=...) next to item=... (exactly one of
    them), and Document.add_comment_on_text. Stub and typing smoke updated.

add_comment_on_text semantics. The occurrence is zero-based. Matches are
case-sensitive, non-overlapping, and never span two paragraphs. They are found
in document order through body paragraphs, tables and block content controls,
over the literal run text that split_run offsets count, so tabs and breaks
have no width. A field result is not in that text, so it is not searched, and
a match whose commented range would also show one, or any other text that the
literal text leaves out, is refused. The check reads back the text of the
paragraph XML between the new markers and compares it with the anchor, so the
comment never covers more than the anchor. A match inside an inline control is
commented inside it, and one that crosses a control edge is refused like any
range. A missing occurrence raises RdocxError. The issue proposed text for both the anchor
and the comment body, which Python cannot express, so the anchor is the first
positional argument anchor.

Decisions left to the maintainer, with my recommendation.

  • Tabs and breaks inside an add_comment_on_text match. The literal text
    gives them no width, so AB matches A<tab>B and the comment covers the
    tab. I kept that, because a refusal would leave no way to comment such text
    with this helper. Refusing them like a field result is a small change if you
    prefer it.
  • Reference run inside w:sdtContent when a range ends inside a control. It
    is schema-valid and LibreOffice keeps it. I recommend keeping it there
    rather than after the control, so the reference stays next to its range. I
    could not get a Word round trip (see Tests), so a check in Word would be
    welcome, in particular for a plain text control.
  • The two-segment StoryItem built from a handle carries the paragraph text
    and empty xml, because the paragraph has no item snapshot of its own. Only
    comment positions accept that path, other story APIs refuse it as an invalid
    path. I think that is enough for now. Listing nested paragraphs in
    story_items would renumber every later item, which Question: StoryItem.index_path and RunPosition.body_index count body items differently #86 ruled out.
  • A table cell Paragraph handle is refused by StoryRunPosition, as Align split_run, bookmarks and story comments on the direct body index #179
    refuses it for split_run, because the cell handle index and the cell
    writer count paragraphs in cell controls differently. Once
    Cell::paragraph_mut agrees, both can accept cell handles.
  • rebuild_toc replaces the cached entry paragraphs, which drops the comment
    and bookmark markers on them and leaves such a comment unanchored in the
    comments part. I documented it rather than change it. Keeping markers across
    a rebuild would need an entry matching rule of its own.

Conflicts to expect. comments.rs and document.rs with #179 (stacked),
crates/rdocx-cli/src/main.rs with #174 (only the --start-run and
--end-run help text), the shared test files regression_test.rs and
test_core.py (new groups placed next to the #179 groups), and the archive
rows, which the maintainer re-records at merge. The measurement dates are
left as recorded, as in the other PRs of this round.

Tests

Added to crates/rdocx/tests/regression_test.rs, as three named groups right
after mod direct_body_index_coordinates:

  • mod accepted_run_index_anchoring (Comment run positions skip runs inside inline content controls, so comments anchor on the wrong text without an error #172):
    comment_run_index_counts_the_runs_of_an_inline_control (the issue
    reproduction, markers around the control, the last run addressable, reopen),
    comment_inside_a_control_is_written_inside_its_content,
    comment_on_a_nested_control_goes_around_it_inside_the_outer_one,
    ranges_that_cannot_be_anchored_exactly_are_refused (control edge, tracked
    insertion, cross-paragraph from inside a control, out of range, for both
    comments and bookmarks, bytes unchanged, then the whole insertion and a
    cross-paragraph range that are accepted),
    removing_a_comment_clears_markers_inside_control_content (the reopened
    main part equals the original),
    removing_a_producer_comment_clears_its_markers_inside_control_content (a
    Word-style source with markers inside w:sdtContent),
    removing_a_comment_clears_fixed_prefix_markers_in_a_document_of_another_prefix
    (an ns0: document: remove in the same session, then a new comment reuses
    the id and owns the only marker pair, also through add_comment_on_text),
    bookmarks_inside_and_around_a_control_keep_their_text_and_distinct_ids
    (three bookmarks added against document order, in a w: and an ns0:
    document, read back before and after a save with ids 0, 1 and 2), and
    story_comment_and_bookmark_use_the_same_run_index.
  • mod block_control_comment_positions:
    story_comment_reaches_a_paragraph_inside_a_block_control and
    two_segment_positions_must_name_a_paragraph_of_a_block_control.
  • mod comment_on_text: the issue Document.split_run(body_index, ...) counts paragraphs only, so it splits the wrong paragraph once a table precedes it #163 fixture with a table before the target,
    occurrence counting, non-overlapping matches and a table cell, a match
    across two runs that keeps their formatting, a match inside an inline
    control and inside a block control, the refusals (missing, wrong case,
    occurrence out of range, empty, across a control edge) with bytes unchanged,
    and comment_on_text_refuses_a_range_that_would_show_a_field_result (a
    w:fldSimple and a complex field between runs, a tab inside the match).
  • Re-pinned as described above:
    comment_insertion_keeps_content_at_the_hyperlink_end_boundary and
    new_bookmark_after_an_accepted_control_has_live_accepted_coordinates.

Added to crates/rdocx-cli/tests/integration.rs, after the comment round trip
test: comment_add_counts_the_runs_that_text_json_lists builds a paragraph
with an inline control, checks the text --json runs, anchors
--start-run 1 --end-run 3 on TARGET, and checks that a range crossing the
control edge exits 1 without writing the output.

Added to crates/rdocx-py/tests/test_core.py, next to the #179 split and story
comment tests: add_comment and add_story_comment on a paragraph with an
inline w:sdt asserting the anchored text, the refusals and remove_comment
cleanup, StoryRunPosition(paragraph=...) inside a block control, on a direct
paragraph, and its errors (cell handle, stale handle, both or neither
argument), and add_comment_on_text on the issue #163 fixture. typing_smoke.py
covers the new signatures.

The nine tests of mod accepted_run_index_anchoring fail on the stacked
base without the fix, the producer removal test on the orphan markers it
leaves. The ns0: removal test and the bookmark id test also fail without
the prefix and renumbering changes described in the Notes, and the field
result test fails without the read-back check. The other new tests call the
new APIs.

Run:

  • cargo fmt --all --check, python3 scripts/prose_check.py: clean.
  • cargo clippy -p rdocx-oxml -p rdocx -p rdocx-cli -p rdocx-py --all-targets --all-features -- -D warnings: clean.
  • cargo test -p rdocx-oxml -p rdocx -p rdocx-cli: all pass except the
    environment-only failures of this machine (pinned tool versions): lib
    large_word_and_presentation_pdfs_preserve_logical_reading_order and
    word_and_powerpoint_chart_pixels_are_identical, integration
    odt_reader_matches_pinned_libreoffice_structure,
    public_authored_theme_and_fonts_match_pinned_word_resolution,
    sanitized_public_authoring_fixture_passes_every_conformance_stage,
    section_page_semantics_match_pinned_libreoffice_render and
    every_conditional_table_region_matches_word. The regression binary passes
    in full (583 tests, 7 ignored).
  • cargo check --workspace --all-targets --exclude rdocx-py --exclude rpptx-py and cargo check --target wasm32-unknown-unknown -p rdocx-wasm:
    clean.
  • python3 scripts/hash_harness.py --check: 49 entries match.
  • python3 scripts/readme_doctests.py: passes with the re-recorded rows.
  • Python gate: pytest crates/rdocx-py/tests 73 passed, 2 failed in
    test_rendering_threads.py (pinned pdfinfo, environment only).
    test_python_docx_parity.py passes here. mypy --strict on
    typing_smoke.py and the package, and mypy.stubtest rdocx: clean.
  • The issue reproduction, run as written with python-docx 1.2.0 building the
    input: Paragraph.runs and text --json list before , TARGET,
    after, Python runs [1, 2) and CLI --start-run 1 --end-run 2 anchor on
    TARGET, and runs [2, 3) anchor on after. python-docx opens all three
    outputs and reads the comment.
  • The same Python checks on an ns0: document with a two-run inline control
    (add_comment or add_comment_on_text, remove_comment, then a new
    comment, which owns the only marker pair) and on a paragraph with a
    w:fldSimple between two runs (after tail is refused with the text its
    range would show).
  • Producer check: probes for a range inside a control, around a control, a
    block control paragraph and add_comment_on_text pass rdocx validate with
    only the missing title and author warnings, and a LibreOffice 26.8 round
    trip keeps every anchor on the intended text. A Word for Mac round trip did
    not complete: after the first probe, Word stopped opening any file, a plain
    one included, and I could not see why without GUI access, so I have no Word
    result either way.

Document::split_run resolved its first argument with
Document::paragraph, which counts paragraphs, skips tables and enters
block content controls. RunPosition, find_content_index, add_comment
and insert_paragraph use the direct body child index instead, which is
also what HLD 10 and F-X109 state for split_run. With a table or a
block content control before the target, splitting a run then
anchoring a comment at the same index split one paragraph and
commented another, or failed on a run length.

split_run now reads the paragraph at that direct slot and writes it
back to the same slot, so one resolution serves both steps. An index
that names a table, a block content control or preserved XML returns
an error naming that kind instead of splitting another paragraph. The
Python method also accepts a Paragraph handle, which reaches
paragraphs inside block content controls, and it measures the
revision bump on the paragraph it split. A handle to a direct body
paragraph takes the same native path as its index. A table cell
paragraph handle is refused, because its index counts the paragraphs
inside cell content controls and Cell::paragraph_mut does not. The
rustdoc of RunPosition and Document::paragraph and the HLD 03 and 10
statements now name the index each one takes.

GitHub issue tensorbee#163.
Document::add_bookmark takes a RunRange whose body index is the direct
body child index, but Document::bookmarks reports the paragraph ordinal
counted recursively through table cells and block content controls. A
bookmark added after a table with more than one cell, or after a block
content control, read back with a different body index than the one it
was added with, so that index could not be passed back to add_comment,
split_run or add_bookmark.

The recursive ordinal stays in range() because REF numbering in
field.rs reads it as its paragraph key. BookmarkRef gains an additive
direct_range() that carries the direct body child index, recorded while
the paragraphs are collected, and is None when either marker sits in a
table cell or a block content control. Run indexes are unchanged and
stay the accepted-view boundaries of range(). They equal the direct run
indexes add_bookmark takes only in a paragraph without inline content
controls or tracked insertions, which GitHub issue tensorbee#172 covers.

GitHub issue tensorbee#163.
Document::add_story_comment resolved a StoryRunRange paragraph by
counting the paragraph items before it, which are the direct
paragraphs of the owner only, and then took that many paragraphs
through a resolver that also enters block content controls. When a
block content control with paragraphs came before the target, in the
body or in a table cell, the comment was anchored on a paragraph
inside the control, or refused when that paragraph had too few runs.

A body paragraph item now resolves to its direct body slot, the count
of direct story items before it, which is the slot
StoryItemRef::direct_body_index reports. A cell paragraph item
resolves to the direct cell paragraph at the same position. Neither
side enters a block content control, so the count and the lookup
agree.

GitHub issue tensorbee#163.
The direct body index changes grow the rdocx package, mostly through
the new regression tests, so the crates.io archive row of the root
README and its ARCHIVE_MEASUREMENTS entry in scripts/readme_doctests.py
are re-measured with cargo package, as the Docs job requires.

GitHub issue tensorbee#163.
Comment and bookmark run positions counted only the direct runs of a
paragraph, while Paragraph.runs, rdocx text --json, split_run and
bookmarks() also count the runs inside inline content controls and
tracked insertions. A run index read from either view landed one or
more runs further right, so add_comment, add_story_comment, the CLI
comment add and add_bookmark wrote their markers on the wrong text and
reported success, and the last runs of such a paragraph could not be
addressed at all.

The accepted view is now the single run index space. The new
CT_P::anchor_accepted_range resolves both boundaries to a physical
place. Markers go inside w:sdtContent when a range starts or ends
between two runs of a control, and around the whole w:sdt when the
range covers it. A range that crosses the edge of a control, has a
boundary between two runs of a tracked insertion or move, sits next to
a tracked change inside a hyperlink, or continues into another
paragraph from inside a control is refused instead of shifted.
ParagraphRef::runs() now lists the same accepted-view runs as
Paragraph::runs().

remove_comment also left markers inside w:sdtContent behind, because
it ignored the preserved children there. It now removes them and any
reference run left empty. It also recognizes the fixed w prefix of
the markers added in the session when the source document gives the
Word namespace another prefix, as ElementTree-based producers do.

Saving renumbers the added bookmarks in document order, and that pass
rewrote only the markers at paragraph level. A bookmark inside a
control kept its old id, which could then collide with another one
and make the saved file fail to reopen. The pass now reaches the
markers inside inline controls, and the in-session bookmark projection
also reads the preserved runs of a control written with another
prefix.

GitHub issue tensorbee#172.
add_comment could not reach a paragraph inside a block content control.
Its direct body index names the control, a StoryRunRange over the
control item was refused as not a paragraph, and story_items lists no
item for the paragraphs inside a control.

A body ContentLocation may now have two segments, the control's story
item index and the paragraph's position among the control's
paragraphs, and add_story_comment resolves it inside the control. The
new Document::paragraph_story_location returns that path, or the
one-segment path of a direct paragraph, for a paragraph index. In
Python, StoryRunPosition accepts a Paragraph handle in place of a
StoryItem and builds the item from it. Table cell handles are refused.

rebuild_toc replaces the cached entry paragraphs, which drops the
comment and bookmark markers placed on them, and its documentation now
says so.

GitHub issue tensorbee#163.
Anchoring a comment on part of a run took a text search, two split_run
calls and run index bookkeeping, which the coordinate mismatches of the
run and body indexes made unreliable.

Document::add_comment_on_text, and Document.add_comment_on_text in
Python, comments on the zero-based occurrence of a literal text in the
main story, through tables and block content controls. Matches are
case-sensitive, non-overlapping and within one paragraph, over the
literal run text that split_run offsets count. The runs at both ends
of the match are split with CT_P::split_accepted_literal_span and the
comment is anchored like add_comment, so a match inside an inline
content control is commented there and one that crosses a control
edge is refused. A missing occurrence is an error and the document is
unchanged. The comment and thread entry creation shared by the three
add paths moves to one helper.

The literal text leaves out field results, so a match could span one
and the comment then covered more than the anchor. The text between
the new markers is read back with CT_P::comment_range_text, and a
match whose range would show anything other than the anchor is refused.

GitHub issue tensorbee#163.
The accepted-view anchoring, the block content control comment path
and add_comment_on_text change the sources, tests and README of these
three published crates, so their crates.io archive rows in
scripts/readme_doctests.py and the crate READMEs are measured again on
top of the stacked tree. The measurement dates are left as recorded.

GitHub issues tensorbee#172 and tensorbee#163.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Comment run positions skip runs inside inline content controls, so comments anchor on the wrong text without an error

1 participant