Skip to content

Save documents and presentations atomically, keeping links and modes - #178

Open
hadim wants to merge 4 commits into
tensorbee:mainfrom
hadim:fix/atomic-library-save
Open

hadim wants to merge 4 commits into
tensorbee:mainfrom
hadim:fix/atomic-library-save

Conversation

@hadim

@hadim hadim commented Sep 27, 2026

Copy link
Copy Markdown
Contributor

Summary

  • oxml-opc: OpcPackage::save builds the ZIP in memory and publishes it
    through a new public oxml_opc::write_atomic_file. The writer stages a
    sibling file, syncs it, renames it over the destination and removes it on
    any failure. Where a plain rename would differ from the old in-place write,
    it keeps the old result. It follows a symbolic link at the path and
    replaces the file the link names. On Unix the staged file gets the mode of
    the file it replaces when it is created. It refuses a file it cannot open
    for writing, such as a read-only file or another user's file. It writes a
    device or a FIFO such as /dev/null in place. It shortens a long
    file name in the staging name, and it does not panic on wasm targets.
    rdocx::Document::save goes through OpcPackage::save, so it becomes
    atomic with this commit.
  • rdocx: the savers that already staged their output
    (save_as_package_class, save_encrypted, save_flat_opc, save_mhtml,
    save_odt, save_rtf, save_epub) drop the private copy of the writer and
    call the shared one, with the same staging names and error messages. They
    pick up all of the cases above.
  • rpptx: Presentation::save and save_as_package_class (and so
    save_as_show) switch from std::fs::write to the shared writer.
    save_encrypted and save_odp drop their two private copies for it.
  • Archive rows of oxml-opc, rdocx and rpptx re-recorded.

Closes #164

Why

The plain .docx and .pptx saves, the ones every script calls, wrote the
target in place. OpcPackage::save truncated it with File::create and then
compressed into it, and Presentation::save used std::fs::write. During the
write, or after a failure half way, a sync client, an editor or a file share
watching the file saw a truncated ZIP, and a sync client can upload it as a
new version. The issue's inode check shows it: the inode stayed the same on
both saves before this change and changes after it.

A naive rename would regress several things the in-place write got right. It
would turn a symbolic link into a regular file, reset a 0600 document to the
umask default, overwrite a read-only file, and replace /dev/null or a FIFO
with a regular file. The existing staged savers of rdocx and rpptx already
had those problems, so the fix covers them as well.

Notes

  • One writer instead of four. oxml-* cannot depend on the facades, and there
    were already three copies of the same loop (rdocx/src/document.rs,
    rpptx/src/lib.rs behind agile-encryption, rpptx/src/odp.rs). Adding the
    special cases to each would have duplicated the subtle part, so oxml-opc
    owns the one implementation and the copies are removed. Outside the tests
    the diff is about even (roughly 270 lines added, 260 removed).
  • Release ordering, please read. After the second and third commits, rdocx
    and rpptx call oxml_opc::write_atomic_file at compile time, and the
    published oxml-opc 0.12.1 does not have it. The workspace requirement
    oxml-opc = { ..., version = "0.12.1" } (Cargo.toml line 60) has to be
    raised to the oxml-opc release that ships the function, and that release
    has to be published before any rdocx or rpptx publish that carries this
    change, including the pending rdocx 0.14.0. If that ordering does not
    suit you, the second commit can be dropped on its own. rdocx then builds
    against 0.12.1 again, Document::save still gets the fix through
    OpcPackage::save, and only the staged rdocx savers keep their old writer.
  • API surface: write_atomic_file(path, bytes, staging_tag, invalid_name_message, exhausted_message) is additive, and the rdocx-opc
    shim re-exports it through its glob. The last three parameters exist so the
    existing staging names (.{name}.rdocx-{pid}-{n}.tmp, .{name}.rpptx-...,
    .{name}.rpptx-odp-...) and error texts, which rdocx tests pin, stay byte
    for byte. The rustdoc now says staging_tag must not contain a path
    separator. If you would rather not commit to this signature,
    #[doc(hidden)] with a note that it serves the facades is a one-line
    change. New staging names: OpcPackage::save, and so Document::save, use
    oxml-opc, and the plain rpptx saves use rpptx with "PowerPoint package"
    messages. write_to and to_bytes are unchanged, and the saved bytes are
    identical.
  • Behaviour kept from the in-place write, for the plain saves:
    • A file the process cannot open for writing is refused with the error of
      that open, PermissionDenied for a 0444 file, and left untouched, as
      File::create refused it. A rename needs write access to the directory
      only, so a check of the mode bits would still let a user replace another
      user's 0644 file in a shared directory, or a 0464 file whose owner has no
      write bit. The writer therefore opens the file for writing, without
      truncating it, and closes it before staging. An inotify watcher sees an
      IN_OPEN and an IN_CLOSE_WRITE on the old file. Root can open a 0444
      file, so root replaces it, as File::create as root overwrote it. For the
      staged savers this is new, since they used to replace such a file.
    • A path that is not a regular file or a directory, after the kernel
      follows every link (so /dev/null, /dev/stdout, a FIFO, a socket), is
      written in place with std::fs::write, exactly as before. Nothing is
      staged and nothing is renamed over it. For the staged savers this is
      new, since they used to stage in /dev or replace the FIFO.
    • A file name close to the 255-byte limit still saves. The staging name
      keeps only the prefix of the file name that fits, cut on a character
      boundary.
    • On wasm targets, where std panics when asked for a process id, the
      staging name uses 0 and the retry loop handles collisions. On
      wasm32-unknown-unknown a path save returns an Unsupported I/O error
      again instead of aborting the module. On WASI it should stage and
      rename, which I could not run here. The staged rdocx savers and
      save_odp had this panic on main.
  • Behaviour changes that follow from replacing by rename, all shared with the
    savers that already staged their output:
    • The inode changes by design. The issue floats an opt-out keyword for
      callers that rely on it. I left it out, since to_bytes plus their own
      write covers that case.
    • The directory holding the destination (the link target's directory for a
      link) must be writable. A writable file in a read-only directory could be
      rewritten in place before and now fails to save.
    • A single-file bind mount (docker run -v ./report.docx:/w/report.docx)
      now fails with EBUSY, since a mount point cannot be renamed over. The
      in-place write worked there.
    • Owner and group, ACLs, extended attributes and other hard links of the
      replaced file are not carried over. The writer's rustdoc says so. A
      group-writable file saved by another member of the group, or any file
      saved by root, changes owner.
  • Costs: every plain save now syncs the staged file before the rename. On
    macOS File::sync_all is F_FULLFSYNC. A release probe during review
    measured 200 writes of 30 KB at 1.04 s through the writer against 23 ms
    with std::fs::write on an Apple silicon Mac, so about 5 ms per save.
    That matters for a batch of thousands of files. The sync is what makes a
    full disk fail the save instead of the next read, and the staged savers and
    the CLI outputs in oxml-cli-support already pay it. SQLite falls back to
    fsync because
    F_FULLFSYNC reportedly fails on some network volumes. I did not verify
    that and did not add a fallback. OpcPackage::save also holds the
    compressed ZIP in memory next to the serialized parts, where it used to
    stream into the file. rpptx already built its bytes in memory.
  • Symbolic links are resolved by hand (symlink_metadata and read_link,
    relative targets against the link's directory) rather than with
    canonicalize, so a dangling link still creates the file it names, as the
    in-place write did. Up to 40 links are followed and a longer chain is
    refused as a loop. The # Errors section lists that error.
  • On Unix the staged file is created with the kept mode (OpenOptionsExt::mode,
    narrowed by the umask) and set to the exact mode right after, so it is
    never more readable than the file it replaces. Mode bits are copied only on
    Unix.
  • Windows: the path keeps the moved
    MoveFileExW(MOVEFILE_REPLACE_EXISTING | MOVEFILE_WRITE_THROUGH) code
    unchanged. It is the rdocx code moved verbatim. I could not compile it here
    (no Windows target installed), and the only Windows build is the release
    wheel job in wheels.yml, which built it inside rdocx. It brings one
    limit to the plain saves. MoveFileExW gets the path without the \\?\
    prefix that std adds, so a destination path longer than MAX_PATH fails to
    rename where File::create and std::fs::write handled it. The staged
    savers already had that limit, and it is why
    save_shortens_the_staging_name_of_a_long_file_name is Unix-only.
    std::fs::rename on Windows is MoveFileExW(MOVEFILE_REPLACE_EXISTING) on
    prefixed paths with a POSIX-semantics fallback, so using it in
    replace_file would lift the limit and drop the unsafe block, at the cost
    of MOVEFILE_WRITE_THROUGH. I left that choice to you.
  • Seen while testing, left alone: rpptx-py's Presentation.save keeps the
    GIL during the write, where rdocx-py releases it, so a Python thread
    cannot read a FIFO that rpptx is writing. That was already the case on
    main.
  • Left out: Document::save_pdf and save_pdf_with_options still use
    std::fs::write. They are not packages, and the CLI side of that is rdocx convert --to pdf|md|html, rpptx convert --to pdf and rpptx thumbnail overwrite any existing output, the input included #156.
    The CLI staged outputs in oxml-cli-support refuse an existing output and
    are untouched. No HLD statement became false, so docs/hld is untouched.

Tests

Added in oxml-opc (package.rs unit tests):

  • save_replaces_an_existing_file_by_rename_and_keeps_its_mode (Unix: new
    inode, 0600 kept, exact bytes, no staging file). Fails on origin/main.
  • failed_save_keeps_the_destination_and_leaves_no_staging_file (portable: a
    directory at the path fails after staging with nothing left behind, and
    exhausted staging names fail with the original bytes intact). The
    exhaustion part fails on origin/main.
  • save_through_a_symbolic_link_replaces_the_file_it_names (Unix: relative
    link from another directory kept, target replaced with 0640 kept, dangling
    link creates its target, a chain of 40 links is followed and 41 are
    refused). This one guards against a naive rename. It passes on
    origin/main, where File::create follows the link, except for the
    40-link chain on macOS, whose kernel follows at most 32 links (checked with
    a shell redirect through the same chain).
  • save_to_a_device_or_fifo_writes_in_place_and_keeps_the_node (Unix: a save
    to /dev/null succeeds and leaves a character device, and a save to a FIFO
    made with mkfifo delivers the bytes to a reader and leaves a FIFO).
  • save_refuses_a_read_only_file_and_leaves_it_as_it_was (Unix: a 0444 file
    and a 0464 file, PermissionDenied, bytes intact, no staging file. Skipped
    for a user who can open them for writing, such as root).
  • save_shortens_the_staging_name_of_a_long_file_name (Unix, see the Windows
    note: a 255-byte, 235-character name with the cut inside a run of
    three-byte characters, saved new and over an existing file).

The last three and the chain assertion pin the old contract. They pass on
origin/main. Earlier rounds of this branch failed them by renaming over the
FIFO, overwriting the 0444 file, failing with "File name too long",
refusing the 40-link chain and, with a check of the mode bits, replacing the
0464 file.

Added in the facades:

  • rdocx tests/regression_test.rs:
    path_saves_replace_the_file_by_rename_and_keep_links_and_permissions
    covers Document::save (inode, mode, link) and save_flat_opc (mode), plus
    a failed save over a directory and no staging file.
  • rpptx tests/integration.rs: the test of the same name covers save,
    save_as_show through a link, save_odp and a failed save over a
    directory.

A mutation run that skipped link resolution and mode copying failed the
symlink and mode tests, so they pin both points.

Run on macOS arm64 at the branch head:

  • cargo fmt --all --check, python3 scripts/prose_check.py (including each
    commit message), python3 scripts/sync_agent_skills.py --check: clean.
  • cargo clippy -p oxml-opc -p rdocx -p rpptx --all-targets --all-features -- -D warnings: clean. Each commit passes cargo check -p oxml-opc -p rdocx -p rpptx --all-targets --all-features on its own.
  • RUSTDOCFLAGS="-D warnings" cargo doc -p oxml-opc -p rdocx -p rpptx --no-deps --all-features: clean.
  • cargo check --target wasm32-unknown-unknown -p oxml-opc -p rdocx-wasm -p rpptx-wasm: clean. A scratch cdylib that calls write_atomic_file, built
    for wasm32-unknown-unknown and run under Node, returns an Unsupported
    error at the branch head and trapped with unreachable before the fix.
  • cargo test -p oxml-opc: 48 pass. With --all-features: 71 pass, 1
    ignored. cargo test -p rdocx-opc: pass.
  • cargo test -p rpptx --all-features: 84 unit and 244 integration tests
    pass.
  • cargo test -p rdocx --all-features --no-fail-fast: the regression binary
    (562) and doc tests pass. Failures are all environment-only: the two lib
    tests pinned to Poppler 26.01
    (large_word_and_presentation_pdfs_preserve_logical_reading_order,
    word_and_powerpoint_chart_pixels_are_identical), and five integration
    tests pinned to LibreOffice 26.2.5.2 or to an offline run
    (odt_reader_matches_pinned_libreoffice_structure,
    public_authored_theme_and_fonts_match_pinned_word_resolution,
    sanitized_public_authoring_fixture_passes_every_conformance_stage,
    section_page_semantics_match_pinned_libreoffice_render,
    every_conditional_table_region_matches_word). The last two assert the
    local LibreOffice 26.8.0.3 version string and fail the same way on
    origin/main.
  • Direct consumers: cargo test -p oxml-drawing -p oxml-chart -p oxml-sml -p rdocx-cli -p rpptx-cli pass except the known
    validate_rejects_corruption_and_accepts_the_pinned_corpus (gitignored
    corpus missing).
  • python3 scripts/hash_harness.py --check: 49 entries match.
  • python3 scripts/readme_doctests.py (the Docs job): 27 READMEs and 22
    package inventories validated, archive rows included.
  • Through maturin develop builds of both bindings: the issue's inode check
    prints "replaced by rename" for rdocx and rpptx, and a save through a
    link keeps the link and the 0600 mode. A save to os.devnull leaves a
    character device. A save to a FIFO delivers a ZIP to a cat reader and
    leaves the FIFO. A 0444 file and a 0464 file raise PackageError with
    "Permission denied (os error 13)", the class and the message main
    raised, and keep their bytes. No staging file is left. No binding or stub
    changed, so the pytest suites were not rerun. They passed in the first
    round apart from the known test_rendering_threads.py failures pinned to
    Poppler.

OpcPackage::save serialized the parts, truncated the destination with
File::create and then compressed the ZIP into it. rdocx Document::save
goes through it, so while the write ran, or after it failed half way,
anything watching the file (a sync client, an editor, a file share) saw
a truncated package.

The ZIP is now built in memory and published by a new public
write_atomic_file. It writes a sibling temporary file, syncs it and
renames it over the destination, and removes it on any failure.

A plain rename gets several cases wrong that the in-place write got
right, so the writer handles them. A symbolic link at the path is
followed and kept, and the file it names is replaced instead of the
link. On Unix the staged file takes the permission bits of the file it
replaces from its creation, so a 0600 document stays 0600 and is never
more open while staged. A read-only file is refused with a permission
error, as File::create refused it. A device or a FIFO such as
/dev/null has no file to replace and is written in place, where a
rename would swap it for a regular file. A file name close to the
255-byte limit is shortened in the staging name instead of failing,
and wasm targets, where std panics on a process id, use 0 in that name.

The writer takes the staging tag and the error messages as arguments,
so the private copies in rdocx and rpptx can move onto it without
changing their staging names or their errors.

GitHub issue tensorbee#164.
save_as_package_class, save_encrypted and the Flat OPC, MHTML, ODT, RTF
and EPUB savers staged their output through a private copy of the
atomic writer. Its rename replaced a symbolic link with a regular file
and reset the permission bits of the replaced file to the umask
default. They now call oxml_opc::write_atomic_file with the same
staging names and error messages, so they keep links and modes as
Document::save now does.

They also take the other cases of the shared writer. A read-only file
is refused instead of replaced, a device or a FIFO is written in place
instead of replaced by a regular file, a long file name no longer
overflows the staging name, and a wasm target no longer panics on the
process id.

The regression test covers the plain save and one staged saver. It
checks the replaced inode, the kept modes, a kept relative link from
another directory, a failed save over a directory, and that no staging
file is left behind.

GitHub issue tensorbee#164.
Presentation::save and save_as_package_class, and so save_as_show,
wrote with std::fs::write, which truncates the destination and writes
into it in place. They now publish through oxml_opc::write_atomic_file
like the encrypted save. save_encrypted and save_odp drop their private
copies of the writer for the shared one and keep their staging names
and error messages, so every rpptx path save keeps a symbolic link and
the permission bits of the file it replaces. As with std::fs::write, a
read-only file is refused and /dev/null takes the bytes in place, which
save_encrypted and save_odp now do as well.

The integration test covers save, save_as_show through a link, save_odp
and a failed save over a directory.

GitHub issue tensorbee#164.
The shared atomic writer and its tests grow the oxml-opc and rdocx
packages, and rpptx shrinks because its two private writers are gone,
so the README archive rows of the three crates and their
ARCHIVE_MEASUREMENTS entries are re-measured.

GitHub issue tensorbee#164.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Document.save() and Presentation.save() write the target in place, where the CLIs stage and publish

1 participant