Skip to content

feat(repo): presence-based partial-clone engine (P37b-i) - #178

Merged
tonibergholm-codento merged 2 commits into
mainfrom
p37b-i
Oct 17, 2026
Merged

tonibergholm-codento merged 2 commits into
mainfrom
p37b-i

Conversation

@tonibergholm

@tonibergholm tonibergholm commented Oct 17, 2026 •

Copy link
Copy Markdown
Member

What

Phase 37b-i of ADR-0048 (merged as #177): the client-side walks on a partial clone ask the store what is present instead of asking the fetch filter.

The bug on main this fixes

A partial clone's filter records what it asked for, not what is present. An ordinary sc fetch subtracts the full closure of every have, so a new in-filter file whose bytes equal an object already reachable at an out-of-filter path is left out of the reply. Repro: the origin has docs/readme.txt; clone with --filter src/; the origin adds an identical src/readme.txt; the clone fetches. Then sc gc and a fast-forward sc merge fail with object … not found, and sc verify exits 0 reporting only the expected docs/ gap.

Changes

  • reachable_objects_partial walks what the store holds and records absent children as gaps with their referrer. An absent root tree is a gap. Snapshots, parents, secrets and manifests stay strict. reachable_objects_filtered is unchanged (the origin uses it to cut transfers), and a test pins that.
  • gc, push, verify use it. gc's hand-written walk-in of present gap ids is gone; decided roots use the same gap policy on a partial clone.
  • Push checks every gap id against the actual destination before any put_pack and refuses with PushGap, naming the id. A push to a remote that is not a mirror of the origin now fails safe.
  • GapInView preflight runs at the top of worktree::materialize, so every call site fails before its first file write, with the sc backfill <prefix> repair named. sc undo (against will_be_tip) and sparse set (against the proposed spec) also check ahead of their ref and spec mutations. No network.
  • Commit graft refuses a new top-level name only when it is outside both the filter and the sparse view.
  • Refusals: sc fetch from a non-origin remote, sc push to a git remote, an empty sc sparse set, sc rewrap, sc grant/revoke, and switching away from a private branch now say so instead of failing with a raw NotFound.

Deviation from the ADR text, recorded in it: the preflight is centralized in materialize rather than called at each site, and "materialize takes the old spec" moves to 37b-ii (until a widen exists the sparse view is inside the filter, so the filter bound is a safe superset). The partial: N object(s) outside filter line of sc verify is unchanged because the demos match on it.

Verification

  • cargo fmt --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test --workspace --all-targets: green locally. 599 scl-repo lib tests, 8 partial_clone_safety CLI tests.
  • demo/run_partial_clone_demo.sh and demo/run_lazy_fetch_demo.sh pass unchanged.
  • Mutation check: removing the materialize preflight fails the switch and fast-forward tests.
  • New tests: the fetch-then-gc/fast-forward repro (repo and CLI level), switch and undo ordering, sparse set leaving the old spec, push refusal for a lost local object and for a non-mirror remote, origin-only fetch, whole-tree command refusals, the graft union, gc keeping filled objects, and the presence-walk unit tests.

Not in this PR

37b-ii (fill_closure, ensure_view fill, sparse widen, reply-cap alignment) and the merge/replay design records.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes

    • gc and verify now handle partial clones more reliably, including after ordinary fetches.
    • Operations that require unavailable files now fail before changing files or repository state and provide a suggested backfill.
    • Pushes from partial clones check that required objects are available at the destination before sending changes.
  • Partial Clone Limitations

    • Partial clones can fetch only from their origin. Pushes to Git remotes and certain other operations are refused until required objects are backfilled.
    • Empty sparse views are refused for partial clones; widening a sparse view can make additional paths available.

A partial clone's fetch filter records what it asked for, not what is
present: an ordinary `sc fetch` subtracts the closure of every have, so a
new in-filter object equal to one reachable at an out-of-filter path is left
out of the reply. gc and fast-forward then failed with `object ... not
found` while `sc verify` reported only the expected gap.

- reachable: `reachable_objects_partial` walks what the store holds and
  records absent children as gaps with their referrer (an absent root tree is
  a gap too). `reachable_objects_filtered` keeps its meaning; the origin uses
  it to cut transfers.
- gc, push and verify use the presence walk. gc's hand-written walk-in of
  present gap ids is gone.
- push checks every gap id against the destination before any put_pack and
  refuses with `PushGap`; a partial clone fetches from its promisor origin
  only; `sc push` to a git remote is refused up front.
- materialize runs a no-network view preflight before its first write, so
  switch, fast-forward and the rest fail with `GapInView` (naming the
  `sc backfill` prefix) instead of a raw NotFound after a partial write. undo
  and `sparse set` also preflight ahead of their ref and spec mutations.
- commit graft refuses a new top-level name only when it is outside both the
  filter and the sparse view.
- explicit refusals for an empty `sparse set`, rewrap, grant, revoke and
  switching away from a private branch.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-17T07:03:34.182726Z 0452597 New commits
🔒 Security Review ✅ Completed 2026-10-17T07:06:23.574442Z 0452597 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 05aa7dda-b16c-4b79-b75b-2eef7d8ca64a
📥 Commits

Reviewing files that changed from the base of the PR and between 3574e52 and 0452597.

📒 Files selected for processing: 6
  • CLAUDE.md
  • crates/repo/src/error.rs
  • crates/repo/src/partial_walks.rs
  • crates/repo/src/reachable.rs
  • crates/repo/src/worktree.rs
  • docs/adr/0048-presence-based-partial-clones.md
🚧 Files skipped from review as they are similar to previous changes: 3
  • crates/repo/src/error.rs
  • docs/adr/0048-presence-based-partial-clones.md
  • crates/repo/src/reachable.rs

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.

📝 Walkthrough

The change adds presence-based reachability for partial clones and checks required objects before view changes. Partial-clone fetches are limited to the promisor origin, pushes check destination gaps before sending objects, and selected operations are refused.

Changes: Partial-clone presence and safety

Layer / File(s) Summary
Presence-based reachability and GC
crates/repo/src/reachable.rs, crates/repo/src/gc.rs, crates/repo/src/promisor.rs, crates/repo/src/partial_walks.rs, crates/repo/src/lib.rs, CLAUDE.md, ROADMAP.md, docs/adr/*
Reachability records locally present objects and missing references. GC and partial-clone gap counts use the presence-based walk. Tests and project documents cover the behavior and phase status.
View preflight and state-preserving operations
crates/repo/src/error.rs, crates/repo/src/worktree.rs, crates/repo/src/sparse.rs, crates/repo/src/oplog.rs, crates/repo/src/repo.rs, crates/cli/tests/partial_clone_safety.rs, crates/repo/src/partial_walks.rs, docs/adr/0048-presence-based-partial-clones.md
Materialization, sparse updates, undo, and switching check required objects before changing files, refs, or sparse specifications. Partial-clone old-root walks skip missing subtrees.
Sync and partial-clone operation boundaries
crates/repo/src/sync.rs, crates/repo/src/error.rs, crates/repo/src/promisor.rs, crates/repo/src/protect_ops.rs, crates/repo/src/rewrap.rs, crates/repo/src/repo.rs, crates/cli/src/main.rs, crates/cli/tests/partial_clone_safety.rs, crates/repo/src/partial_walks.rs, CLAUDE.md
Partial clones fetch only from their promisor origin. Push checks destination gaps before sending objects. The CLI and repository refuse selected operations on partial clones.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Bug fix · Severity of issue fixed: Medium

Sequence Diagram(s)

sequenceDiagram
  participant RepoPush as Repo::push
  participant Reachability as reachable_objects_partial
  participant Destination as Destination remote
  RepoPush->>Reachability: Walk locally present reachable objects and report gaps
  Reachability-->>RepoPush: Present objects and missing-object gaps
  RepoPush->>Destination: Check whether each gap exists
  Destination-->>RepoPush: Return gap presence
  RepoPush->>Destination: Send present objects when all gaps exist
Loading
Merge Risk: ⚪ Minimal · up to 04525

This change makes partial-clone operations check which objects are actually present and refuse unsafe operations before modifying state. No concrete merge-blocking problem was identified in the supplied evidence.

Security architecture risk: 🔵 Low · up to 04525

The inspected changes strengthen missing-object handling without establishing a new access-control bypass. They prevent the targeted failures before affected state changes, but do not make filesystem operations crash-atomic. Incomplete coverage limits confidence in the remaining paths.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The inspected consequential operations affect the local repository object store and working tree, plus objects and branch refs at the configured push destination. The changed paths do not establish additional filesystem authority beyond the existing materializer or broader service-level privileges.

Trust Boundaries and Controls

  • observed — Tree-controlled write and removal paths continue through the unchanged lexical safe_join check, and protected-content writes retain their decryption gate. These controls were not replaced by object-presence checks.
  • observed — The destination gap check establishes frontier-object presence only. LocalTransport implements that probe with contains, while its separate ref update performs locked compare-and-swap without recursive closure validation. The PR strengthens the previous assumption that omitted out-of-filter objects already existed remotely; it does not establish an atomic destination-closure transaction.

Resilience and Maintainability Implications

  • observed — GC retains its branch, remote-tracking, undo-history, workspace, and in-progress-operation roots. On partial clones it now protects connected present content using the shared traversal, including decided trees, and writes the replacement pack before deleting reachable loose objects or superseded packs.
  • observed — Push retains transfer-before-ref ordering. LocalTransport documents that failed application of an already-verified pack may leave content-addressed residue but must not advance refs, and that retrying object insertion is idempotent. Its locked ref comparison prevents a competing push from silently overwriting a changed branch tip.
Pre-merge checks: Passed 5
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely identifies the main change: a presence-based partial-clone engine for Phase 37b-i.
Docstring Coverage Passed Docstring coverage is 90.91% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 55 functions across 15 files. (2 skipped: 2…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 90.91% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 55 functions across 15 files. (2 skipped: 2 unsupported.)

✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

…i review)

The old-root removal walk is bounded by the promisor filter, which is only a
superset of the sparse view. A fetch can leave a tree inside the filter but
outside the view absent, so a later switch failed with a raw NotFound and undo
failed after moving the ref. The old root was laid out under a preflighted
view, so an absent subtree has nothing on disk and is skipped. Also: PushGap
hint for a lost local object, PartialReach doc, ADR/CLAUDE.md wording.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

@tonibergholm-codento tonibergholm-codento left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed; review findings addressed, checks green on 0452597.

@tonibergholm-codento
tonibergholm-codento merged commit 8dc2193 into main Oct 17, 2026
8 checks passed
@tonibergholm
tonibergholm deleted the p37b-i branch October 17, 2026 07:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants