Skip to content

Consolidate the directory durability helpers #2006

Description

@gaborbernat

Problem

Two independent implementations of the same directory-durability helpers now live in the tree, and they disagree about error type and about how much they cover.

peryx-storage owns the canonical set in crates/peryx-storage/src/blob/mod.rs, added when #1391 made durability failures propagate:

fn sync_parent(path: &Path) -> std::io::Result<()>      // line 45
fn create_dir_durable(dir: &Path) -> std::io::Result<()> // line 63
fn sync_dir(path: &Path) -> std::io::Result<()>          // line 73, unix
fn sync_dir(_path: &Path) -> std::io::Result<()>         // line 80, non-unix no-op

crates/peryx/src/operator/mod.rs carries a parallel set over anyhow, added by #1394 for atomic backup publication:

fn sync_tree(path: &Path) -> anyhow::Result<()>   // line 414
fn sync_parent(path: &Path) -> anyhow::Result<()> // line 428
fn sync_dir(path: &Path) -> anyhow::Result<()>    // line 436, unix
fn sync_dir(_path: &Path) -> anyhow::Result<()>   // line 445, non-unix no-op

peryx already depends on peryx-storage (crates/peryx/Cargo.toml:56), so the duplication is not forced by the crate graph. Neither copy is a superset: peryx-storage has create_dir_durable, which flushes each level a fan-out creates; the operator copy has sync_tree, a leaves-to-root walk. A fix or platform correction applied to one silently leaves the other wrong, and the non-unix no-op arm is now written twice.

Example

A future change to how a directory flush handles an interrupted fsync, or to the non-unix behavior, has to be found and applied in two crates. A contributor reading operator/mod.rs has no signal that a more complete implementation exists one crate away.

Required change

Keep one implementation. peryx-storage is the natural owner: it is the storage crate, peryx already depends on it, and its set is the one with create_dir_durable. Move sync_tree to join it so the operator's callers lose nothing, and have crates/peryx/src/operator/ call through instead of keeping a second copy.

Settle the error type deliberately rather than by accident. The storage helpers are io::Result, the operator's are anyhow::Result, and the operator callers want context attached; converting at the call site keeps peryx-storage free of anyhow while letting the operator add its own context.

One constraint to respect rather than trip over: the storage helpers are currently private. Exporting them widens peryx-storage's public surface, which the repo's semver shards check. Prefer the narrowest visibility that lets peryx reach them, and do not make an item pub when a tighter option works.

Acceptance criteria

  • One implementation of each helper in the workspace; crates/peryx/src/operator/mod.rs holds none of its own.
  • Backup publication and restore keep their current behavior, including the leaves-to-root ordering and the non-unix no-op.
  • Blob publication, reconcile and upload staging keep theirs.
  • No item is made more visible than the consolidation requires.
  • Existing tests for both call sites pass unchanged.

Architecture and test boundary

Shared code belongs in the shared crate and must stay ecosystem-neutral, so the helper set moves to peryx-storage with its behavioral tests, and the operator keeps only tests for the behavior it adds on top. This is a refactor with no intended behavior change, so it should not need a new test to prove anything beyond the existing ones still passing.

Coordination

Flagged independently by the lanes that landed #1391 (PR #1999) and #1501 (PR #2005); both deliberately left it alone as out of scope, and #1501's lane noted the operator copy is now the only remaining duplicate. Depends on nothing beyond #1394, #1391 and #1501, all merged. Purely internal, so it does not block or unblock any other issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:backupBackup and restorearea:storageBlob stores, object storage, and storage backend workpriority:P3Convenience or long-term featuretype:featureNew user-facing or protocol capability

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions