Skip to content

fix(local): sync the directories a durable write actually creates; bound the unsynced set (0.12.5) - #9

Merged
reidkaufmann merged 3 commits into
base-object-store-0125from
fsync-dir-sync-fixes-0125
Nov 13, 2026
Merged

reidkaufmann merged 3 commits into
base-object-store-0125from
fsync-dir-sync-fixes-0125

Conversation

@reidkaufmann

Copy link
Copy Markdown
Collaborator

Mirror of #5 onto base-object-store-0125 (object_store 0.12.5), so the influxdb_pro 3.12 backport of https://github.com/influxdata/influxdb_pro/pull/6818 pins the same LocalFileSystem behaviour as main (base-object-store-0132 @ 41b600a).

The three commits of #5 cherry-picked with -x, no conflicts:

  • fix(local): sync the directories a durable write actually creates
  • fix(local): record only existing directories as unsynced after a failed creation
  • docs(local): stores with different fsync settings must not share a root

cargo test --all-features, clippy -D warnings and cargo fmt --check pass locally. The RAT job needs the Python fix from #8, backported to this branch separately.

Copilot AI balanced review requested due to automatic review settings November 13, 2026 21:05

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the LocalFileSystem durability backport.

No open issues 🎉

What changed in this PR

Backports LocalFileSystem durability fixes to object_store 0.12.5, aligning directory-sync behavior with the main branch.

Changes:

  • Tracks directories actually created and syncs their parent entries, including when concurrent cleanup or creation changes the directory tree.
  • Avoids retaining known-missing directories in the unsynced set after failed creation.
  • Adds regression tests and documents that stores with different fsync settings must not share a root.
File Description
src/​local.rs Updates directory creation and sync tracking, adds regression tests, and clarifies fsync documentation.

Review effort: Balanced

`create_parent_dirs` recorded the deepest existing ancestor, called
`create_dir_all`, and synced from the new parent up to that ancestor.
A concurrent delete with automatic cleanup can remove the ancestor in
between; `create_dir_all` recreated it, and its new entry in the
directory above was never synced.

Create the missing components one `create_dir` at a time, registering
each before creating it, and sync every directory from the deepest one
created up to the one holding the highest one's entry. That also covers
a directory another creation made in between, which holds an entry made
here.

(cherry picked from commit 4d3a27f)
…ed creation

A creation that failed before creating anything (EACCES, ENOSPC) still
recorded its would-be directories in the process-global unsynced set,
and nothing cleared them. Record only directories that exist, or whose
state cannot be determined.

(cherry picked from commit 1d068f4)
@reidkaufmann
reidkaufmann force-pushed the fsync-dir-sync-fixes-0125 branch from bef4fdf to 3b3cdb7 Compare November 13, 2026 21:16
@reidkaufmann
reidkaufmann merged commit 36cca19 into base-object-store-0125 Nov 13, 2026
8 checks passed
@reidkaufmann
reidkaufmann deleted the fsync-dir-sync-fixes-0125 branch November 13, 2026 21:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants