Skip to content

fix(storage): name truncated history and read rewritten manifests - #115

Open
codecapo wants to merge 1 commit into
mainfrom
fix/storage-delta-reads
Open

codecapo wants to merge 1 commit into
mainfrom
fix/storage-delta-reads

Conversation

@codecapo

Copy link
Copy Markdown
Owner

What

The storage half of #89: an incremental read now tells expired history
apart from a snapshot that was never an ancestor, and finds rows that a
manifest rewrite carried forward.

Part of #89

How

  • infostash_model::Error gains TruncatedHistory, the variant ADR 0009
    says engine: fall back to a full refresh at plan time and fix files_added_by #89 adds. It is never retried; the reader must read the table in
    full.
  • The walk from to back to from is a private iterator,
    ancestors_after, which appends_between uses and the plan-time check
    will share. It looks to up first, as before, stops at the child of
    from without looking from up, and turns a missing ancestor into
    TruncatedHistory. A parent of None is still the non-ancestor
    NotFound, and the non-append refusal is unchanged.
  • files_added_by reads every manifest that to lists and keeps the
    live entries (Added or Existing) whose snapshot is in the range. It
    used to skip manifests written outside the range and keep only Added
    entries, which drops a file a later replace carries forward as
    Existing.

Things to know

Tests

  • files_added_in_the_range_are_found_in_a_rewritten_manifest: two files
    added in one manifest, then a replace removes one, rewriting the
    manifest. The range holding the first replace returns exactly the kept
    file: not the removed one, not the files it deleted, and nothing from
    before the range.
  • a_range_whose_history_expired_is_truncated_not_missing: four appends
    with expiry simulated by dropping snapshots from the metadata. Dropping
    older snapshots or the checkpoint alone still reads the range; dropping
    the checkpoint and its child gives TruncatedHistory; a made-up from
    and a missing to stay NotFound.
  • Both tests fail against the old code. Workspace line coverage 98.1%.

An incremental read walks back from `to` to its checkpoint `from`. It
looked `from` up too, so a checkpoint that expiry removed failed the
read even when its child was kept, and a gap in the history failed with
the same NotFound as a snapshot that was never an ancestor. The walk
now stops at the child of `from`, and a missing snapshot other than
`to` is the new Error::TruncatedHistory that ADR 0009 calls for. The
walk is its own iterator so the plan-time range check can share it.

files_added_by read only the manifests written inside the range and
kept only Added entries. A replace that rewrites a manifest carries a
file forward as Existing under the id of the snapshot that added it, so
those rows were dropped. It now reads every manifest `to` lists and
keeps the live entries added in the range.

Part of #89

Signed-off-by: Mayur Kale <hello@mayurkale.com>
@coderabbitai

coderabbitai Bot commented Nov 12, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 33 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 2492912 and bfec6e0.

Files selected for processing: 2

  • crates/model/src/lib.rs
  • crates/storage/src/read.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@codecov

codecov Bot commented Nov 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.08%. Comparing base (2492912) to head (bfec6e0).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #115      +/-   ##
==========================================
+ Coverage   99.04%   99.08%   +0.03%     
==========================================
  Files          16       16              
  Lines        2521     2626     +105     
==========================================
+ Hits         2497     2602     +105     
  Misses         24       24              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant