From 124f6095a2a12e6352cc8e2bc5d00f539393f6ed Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Sun, 18 Oct 2026 15:54:14 +0300 Subject: [PATCH 1/4] feat(repo): conflicts on a partial clone (P37c-iii) A merge with a conflict inside the sparse view now works on a partial clone. Markers land, `sc conflicts`/`sc resolve` find the path by descending its components, `merge --abort` restores ours, and `sc commit` completes the merge. The decided root carries untouched subtrees by id, and completion grafts the out-of-view part from it, so a never-fetched subtree comes through unread. A conflict outside the view, or a result that must be sealed outside it, is refused before any write. Abort now checks the view before it removes anything, and materialize_conflict_state fills the view closure first. ADR-0049 decision 8 is amended with the contract. Co-Authored-By: Claude Sonnet 5.5 --- CLAUDE.md | 8 +- ROADMAP.md | 4 +- crates/cli/tests/partial_clone_safety.rs | 82 +++++-- crates/repo/src/conflicts.rs | 30 ++- crates/repo/src/partial_merge.rs | 274 ++++++++++++++++++++++- crates/repo/src/replay.rs | 2 + crates/repo/src/repo.rs | 133 ++++++++--- crates/repo/src/worktree.rs | 36 +++ demo/run_lazy_fetch_demo.sh | 45 +++- docs/adr/0049-merge-on-partial-clones.md | 27 ++- docs/adr/README.md | 2 +- 11 files changed, 555 insertions(+), 88 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index b79da4a6..d9c1242d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -106,7 +106,7 @@ the project tree if desired. ## Capability map (what's built, by phase) -All 36 phases, P37a, P37b-i, P37b-ii and P37c-ii are built and tested. One line of current fact per phase; the +All 36 phases, P37a, P37b-i, P37b-ii, P37c-ii and P37c-iii are built and tested. One line of current fact per phase; the authoritative rationale and full semantics live in the linked ADR, the design in `ARCHITECTURE.md`. The old per-phase narrative log this table replaced is archived verbatim at `docs/archive/claude-md-phase-log-2026-07.md` — do not @@ -153,7 +153,7 @@ the code, those win. | P36 | Bucket WAL remotes (sc+wal://, sc+s3://): immutable packs + CAS'd manifest, checkpoints + log-tail cold start, bucket-backed `sc serve --store` with disposable instances | [0046](docs/adr/0046-wal-bucket-remotes.md) | | P37a | Want-by-id fetch: batched `GetObjects` verb (wire v5), opt-in `.sc/promisor-lazy`, `sc cat :` fills only the absent trees and blob on its path; `Store::get` stays network-free; sealed objects refused by kind; sparse widen and the lifted P27 refusals are P37b | [0047](docs/adr/0047-want-by-id-fetch-and-gap-fill.md) | | P37b-i | Presence-based partial clones: push, verify and gc walk what the store holds (`reachable_objects_partial`), not the fetch filter; push checks every gap id against the destination before sending; a view gap fails with `GapInView` before any write (preflight at the top of `materialize`, plus `undo` and `sparse set`); a partial clone fetches from its origin only. 37b-ii: on a lazy, online clone `ensure_view` fills the view closure by id before switch, fast-forward, undo and `sparse set`, `sparse set` may widen past the filter, a by-id reply is capped at `min(--max-pack-size, 512 MiB)` and a refused batch is split. | [0048](docs/adr/0048-presence-based-partial-clones.md) | -| P37c-ii | Clean merge on a partial clone: a tree-recursive engine (`merge_trees`) carries a subtree by id when the sides agree and no protection rule applies, reads only where they differ or a rule intersects, and on a lazy, online clone fills what it asks for by id outside the store lock (a non-lazy clone gets the gap error with `sc backfill `); a carried protected subtree keeps the complete input wrap union; a conflict is refused before any ref, merge-state or working-tree change (fetched objects stay). A subtree identical on all three sides is carried even under a protection rule. Mode-only edits now merge and a file/directory collision fails before any write, on full clones too. Cherry-pick/rebase and conflicts are next | [0049](docs/adr/0049-merge-on-partial-clones.md) | +| P37c-ii, P37c-iii | Merge on a partial clone: a tree-recursive engine (`merge_trees`) carries a subtree by id when the sides agree and no protection rule applies, reads only where they differ or a rule intersects, and on a lazy, online clone fills what it asks for by id outside the store lock (a non-lazy clone gets the gap error with `sc backfill `); a carried protected subtree keeps the complete input wrap union; a conflict inside the view is marked and resolved like on a full clone (`sc conflicts`/`sc resolve`/`merge --abort`/`sc commit`; the out-of-view part is grafted from the decided root), while a conflict outside the view, or a result that must be sealed outside it, is refused before any ref, merge-state or working-tree change (fetched objects stay). A subtree identical on all three sides is carried even under a protection rule. Mode-only edits now merge and a file/directory collision fails before any write, on full clones too. Cherry-pick/rebase are next | [0049](docs/adr/0049-merge-on-partial-clones.md) | ## Standing boundaries & gotchas @@ -171,8 +171,8 @@ transport-adjacent. The rest, imperatively: identity material. - **Partial clones refuse cherry-pick/rebase, `sc ws fork`/`harvest`, `sc work`, `sc export`, `sparse disable`, an empty `sparse set` (and, on a non-lazy clone, one outside the filter), `sc rewrap`, - `sc grant`/`revoke`, creating, switching to or leaving a private branch, a merge that has conflicts (no ref, state or working-tree change), `sc push` to a git remote, and - `sc fetch` from any native remote but the promisor origin.** A clean `sc merge` works (P37c-ii). `sc backfill --all` + `sc grant`/`revoke`, creating, switching to or leaving a private branch, a merge whose conflict lies outside the view or needs sealing outside it (no ref, state or working-tree change), `sc push` to a git remote, and + `sc fetch` from any native remote but the promisor origin.** `sc merge` works, including conflicts inside the view (P37c). `sc backfill --all` converts to a genuine full clone and re-enables them. The filter records what a clone *asked for*, not what is present: after a fetch an in-filter object can be absent. On a lazy, online clone a switch, fast-forward, rebase, undo diff --git a/ROADMAP.md b/ROADMAP.md index 3b526251..320ff2f7 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -558,11 +558,11 @@ clone: the view closure is filled by id before switch, fast-forward, undo and `sparse disable` stay refused. Design and open questions in [ADR-0048](docs/adr/0048-presence-based-partial-clones.md). -**P37c — merge on partial clones (proposed; 37c-i engine and 37c-ii clean merge built).** A tree-recursive three-way merge +**P37c — merge on partial clones (proposed; 37c-i engine, 37c-ii clean merge and 37c-iii conflicts built).** A tree-recursive three-way merge that carries a subtree by id where the three sides agree, checks protection rules before carrying, never drops wraps for subtrees it does not enumerate, and fills (lazy) or refuses (otherwise) the region where the sides differ, -the region that intersects a protection rule, and the sparse view's closure. Clean merges are built (`sc merge` on a partial clone; a conflict is refused before any write). Conflicts are next, then cherry-pick/rebase; sessions +the region that intersects a protection rule, and the sparse view's closure. Merges are built (`sc merge` on a partial clone): a conflict inside the view is marked, resolved with `sc conflicts`/`sc resolve`, aborted or committed like on a full clone; a conflict outside the view, or one that needs sealing outside it, is refused before any write. Cherry-pick/rebase are next; sessions are undecided. Design in [ADR-0049](docs/adr/0049-merge-on-partial-clones.md). diff --git a/crates/cli/tests/partial_clone_safety.rs b/crates/cli/tests/partial_clone_safety.rs index b473b820..b55fd1b2 100644 --- a/crates/cli/tests/partial_clone_safety.rs +++ b/crates/cli/tests/partial_clone_safety.rs @@ -160,14 +160,15 @@ fn status_diff_and_gc_succeed_on_fresh_partial_clone() { /// P27 final review I2: `sc backfill --all` is the real escape hatch — /// fetches every remaining object and removes `.sc/promisor`, after which -/// a merge that was refused as unsupported on a partial clone works. +/// a cherry-pick that was refused as unsupported on a partial clone works. #[test] fn backfill_all_lifts_partial_clone_refusals() { let (src_root, dst_root) = make_partial_clone("backfill-all"); - // A merge with conflicts is refused while partial (a clean one works, P37c-ii). - // Create a divergent, non-conflicting branch (different files) for the clean - // merge after backfill, and a clashing one (same file) for the refusal. + // Replay is still refused while partial (a merge, clean or conflicting + // inside the view, works since P37c). Create a clashing branch (same + // file) for the refusal and a non-conflicting one for the merge after + // backfill. assert!(sc(&dst_root, &["branch", "clash"]).status.success()); assert!(sc(&dst_root, &["switch", "clash"]).status.success()); std::fs::write(dst_root.join("src/a.txt"), "clash-edit\n").unwrap(); @@ -181,19 +182,19 @@ fn backfill_all_lifts_partial_clone_refusals() { std::fs::write(dst_root.join("src/a.txt"), "main-edit\n").unwrap(); assert!(sc(&dst_root, &["commit", "-m", "main"]).status.success()); - let merge_before = sc(&dst_root, &["merge", "clash"]); + let pick_before = sc(&dst_root, &["cherry-pick", "clash"]); assert!( - !merge_before.status.success(), - "a conflicting merge must still be refused before backfill --all" + !pick_before.status.success(), + "a cherry-pick must still be refused before backfill --all" ); assert!( - stderr(&merge_before).contains("backfill --all"), + stderr(&pick_before).contains("backfill --all"), "refusal must point at `sc backfill --all`: {}", - stderr(&merge_before) + stderr(&pick_before) ); assert!( !dst_root.join("src/a.txt.theirs").exists(), - "a refused merge writes nothing" + "a refused cherry-pick writes nothing" ); assert_eq!( std::fs::read_to_string(dst_root.join("src/a.txt")).unwrap(), @@ -220,12 +221,12 @@ fn backfill_all_lifts_partial_clone_refusals() { stderr(&sparse_disable) ); - // The conflicting merge is no longer refused: it reaches the ordinary + // The cherry-pick is no longer refused: it reaches the ordinary // conflict state (markers in the file), which `--abort` undoes. - let conflict_after = sc(&dst_root, &["merge", "clash"]); + let conflict_after = sc(&dst_root, &["cherry-pick", "clash"]); assert!( !stderr(&conflict_after).contains("partial clone"), - "a conflicting merge must no longer be refused after backfill --all: {}", + "a cherry-pick must no longer be refused after backfill --all: {}", stderr(&conflict_after) ); assert!( @@ -234,7 +235,7 @@ fn backfill_all_lifts_partial_clone_refusals() { .contains("<<<<<<<"), "the ordinary conflict markers are written" ); - assert!(sc(&dst_root, &["merge", "--abort"]).status.success()); + assert!(sc(&dst_root, &["cherry-pick", "--abort"]).status.success()); let merge_after = sc(&dst_root, &["merge", "feature"]); assert!( @@ -247,6 +248,59 @@ fn backfill_all_lifts_partial_clone_refusals() { std::fs::remove_dir_all(&dst_root).unwrap(); } +/// P37c-iii: a merge with a conflict inside the view works on a partial clone. +/// Markers land, `sc conflicts` and `sc resolve` find the path without +/// flattening a tree, and `sc commit` completes the merge. The untouched +/// `docs/` subtree is never fetched or written. +#[test] +fn conflicting_merge_on_a_partial_clone_resolves_and_commits() { + let (src_root, dst_root) = make_partial_clone("conflict-merge"); + assert!(sc(&dst_root, &["branch", "clash"]).status.success()); + assert!(sc(&dst_root, &["switch", "clash"]).status.success()); + std::fs::write(dst_root.join("src/a.txt"), "clash-edit\n").unwrap(); + assert!(sc(&dst_root, &["commit", "-m", "clash"]).status.success()); + assert!(sc(&dst_root, &["switch", "main"]).status.success()); + std::fs::write(dst_root.join("src/a.txt"), "main-edit\n").unwrap(); + assert!(sc(&dst_root, &["commit", "-m", "main"]).status.success()); + + let merge = sc(&dst_root, &["merge", "clash"]); + assert!(!merge.status.success(), "a conflicting merge exits nonzero"); + assert!( + !stderr(&merge).contains("partial clone"), + "not refused as unsupported: {}", + stderr(&merge) + ); + assert!(std::fs::read_to_string(dst_root.join("src/a.txt")) + .unwrap() + .contains("<<<<<<<")); + assert!( + !dst_root.join("docs").exists(), + "docs/ stays out of the view" + ); + + let listed = sc(&dst_root, &["conflicts"]); + assert!(listed.status.success(), "{}", stderr(&listed)); + assert!(stdout(&listed).contains("src/a.txt")); + let shown = sc(&dst_root, &["conflicts", "src/a.txt"]); + assert!(shown.status.success(), "{}", stderr(&shown)); + assert!(stdout(&shown).contains("main-edit") && stdout(&shown).contains("clash-edit")); + + let resolved = sc(&dst_root, &["resolve", "--theirs", "src/a.txt"]); + assert!(resolved.status.success(), "{}", stderr(&resolved)); + let commit = sc(&dst_root, &["commit", "-m", "merge clash"]); + assert!(commit.status.success(), "{}", stderr(&commit)); + assert_eq!( + std::fs::read_to_string(dst_root.join("src/a.txt")).unwrap(), + "clash-edit\n" + ); + assert!(!dst_root.join("docs").exists()); + let verify = sc(&dst_root, &["verify"]); + assert!(verify.status.success(), "{}", stderr(&verify)); + + std::fs::remove_dir_all(&src_root).unwrap(); + std::fs::remove_dir_all(&dst_root).unwrap(); +} + #[test] fn sparse_widen_beyond_partial_filter_is_refused() { let (src_root, dst_root) = make_partial_clone("sparse-widen"); diff --git a/crates/repo/src/conflicts.rs b/crates/repo/src/conflicts.rs index 6a603e39..c62fae7f 100644 --- a/crates/repo/src/conflicts.rs +++ b/crates/repo/src/conflicts.rs @@ -8,7 +8,9 @@ use scl_core::{Object, ObjectId, Store, PROTECTED}; use crate::error::{Error, Result}; use crate::repo::Repo; -use crate::worktree::{safe_join, tree_file_entries_with_perms}; +#[cfg(test)] +use crate::worktree::tree_file_entries_with_perms; +use crate::worktree::{safe_join, tree_entry_at}; /// Which in-progress operation owns the current conflicts. #[derive(Debug, Clone, Copy, PartialEq, Eq)] @@ -140,20 +142,20 @@ impl Repo { /// else non-UTF8 ours/theirs content → Binary; else Text). pub fn conflict_kind(&self, path: &str) -> Result { let triple = self.op_triple()?; + let partial = self.promisor()?.is_some(); let store_arc = self.vfs.store(); let mut store = store_arc.lock().unwrap(); for id in [triple.ours, triple.theirs] { let root = store.get_snapshot(&id)?.root; - let entries = tree_file_entries_with_perms(&mut store, root)?; - if let Some((_, _, perms)) = entries.get(path) { + if let Some((_, _, perms)) = tree_entry_at(&mut store, root, path, partial)? { if perms & PROTECTED != 0 { return Ok(ConflictKind::Protected); } } } // Neither tip is protected: a None identity is fine to inspect content. - let ours = side_for(&mut store, triple.ours, None, path)?; - let theirs = side_for(&mut store, triple.theirs, None, path)?; + let ours = side_for(&mut store, triple.ours, None, path, partial)?; + let theirs = side_for(&mut store, triple.theirs, None, path, partial)?; let is_binary = |s: &Side| matches!(s, Side::Present(b) if std::str::from_utf8(b).is_err()); if is_binary(&ours) || is_binary(&theirs) { Ok(ConflictKind::Binary) @@ -170,14 +172,15 @@ impl Repo { identity: Option<&scl_crypto::SecretKey>, ) -> Result { let triple = self.op_triple()?; + let partial = self.promisor()?.is_some(); let store_arc = self.vfs.store(); let mut store = store_arc.lock().unwrap(); let base = match triple.base { - Some(id) => side_for(&mut store, id, identity, path)?, + Some(id) => side_for(&mut store, id, identity, path, partial)?, None => Side::Absent, }; - let ours = side_for(&mut store, triple.ours, identity, path)?; - let theirs = side_for(&mut store, triple.theirs, identity, path)?; + let ours = side_for(&mut store, triple.ours, identity, path, partial)?; + let theirs = side_for(&mut store, triple.theirs, identity, path, partial)?; Ok(ConflictVersions { base, ours, theirs }) } @@ -256,12 +259,12 @@ impl Repo { ResolveSide::Ours => triple.ours, ResolveSide::Theirs => triple.theirs, }; + let partial = self.promisor()?.is_some(); let store_arc = self.vfs.store(); let blob_id = { let mut store = store_arc.lock().unwrap(); let snap = store.get_snapshot(&chosen_snapshot)?; - let entries = tree_file_entries_with_perms(&mut store, snap.root)?; - entries.get(path).map(|(id, _, _)| *id) + tree_entry_at(&mut store, snap.root, path, partial)?.map(|(id, _, _)| id) }; if let Some(blob_id) = blob_id { let mut cache = self.open_protected_cache()?; @@ -326,12 +329,15 @@ fn side_for( snapshot_id: ObjectId, identity: Option<&scl_crypto::SecretKey>, path: &str, + partial: bool, ) -> Result { let snap = store.get_snapshot(&snapshot_id)?; - let entries = tree_file_entries_with_perms(store, snap.root)?; - let Some((blob_id, _mode, perms)) = entries.get(path).copied() else { + let Some((blob_id, _mode, perms)) = tree_entry_at(store, snap.root, path, partial)? else { return Ok(Side::Absent); }; + if partial && !store.contains(&blob_id) { + return Err(crate::promisor::partial_gap_hint(path)); + } let bytes = match store.get(&blob_id)? { Object::Blob(b) => b, _ => return Err(Error::BadRef(format!("{path}: tree entry is not a blob"))), diff --git a/crates/repo/src/partial_merge.rs b/crates/repo/src/partial_merge.rs index f5e5d8c8..e0095fef 100644 --- a/crates/repo/src/partial_merge.rs +++ b/crates/repo/src/partial_merge.rs @@ -210,19 +210,179 @@ fn differing_region_outside_the_view_fills_when_lazy_and_is_refused_otherwise() fx.cleanup(); } -/// A conflict is refused before any merge output, state or working-tree write. +/// The same edits as the clone's conflicting merge, done on a full repo: the +/// origin commits `ours` on a side branch and merges `theirs` into it. The +/// result is what a partial clone's completed merge must equal, by tree id. +fn full_merge_root( + fx: &Fx, + theirs: &str, + ours_edit: &[(&str, &[u8])], + resolve: &[&str], +) -> ObjectId { + fx.origin.branch("expect").unwrap(); + fx.origin.switch("expect").unwrap(); + for (p, bytes) in ours_edit { + std::fs::write(fx.origin_root.join(p), bytes).unwrap(); + } + fx.origin.commit("t", "ours").unwrap(); + assert!(matches!( + fx.origin.merge(theirs, "t").unwrap_err(), + Error::MergeConflicts(_) + )); + for p in resolve { + fx.origin + .resolve_path(p, crate::conflicts::ResolveSide::Ours, None) + .unwrap(); + } + let id = fx.origin.commit("t", "merge").unwrap(); + let root = fx.origin.snapshot(&id).unwrap().root; + fx.origin.switch("main").unwrap(); + root +} + +/// A conflict inside the view is marked on disk; the subtree only theirs +/// changed is carried by id and never fetched, on a clone that cannot fetch. +/// Resolving and committing completes the merge, and the result is the tree a +/// full clone produces from the same edits (ADR-0049 decision 8). #[test] -fn conflict_on_a_partial_clone_is_refused_before_any_write() { +fn a_conflict_in_the_view_completes_with_the_carried_subtree_grafted_from_the_decided_root() { let fx = Fx::origin("conflict", |_| {}); - fx.origin_branch("b1", &[("src/a.txt", b"ONE\ntwo\nthree\n")]); - fx.origin_branch("b2", &[("src/a.txt", b"uno\ntwo\nthree\n")]); + let b1 = fx.origin_branch( + "b1", + &[ + ("src/a.txt", b"ONE\ntwo\nthree\n"), + ("docs/api/guide.md", b"g changed\n"), + ], + ); + let theirs_docs = dir_id(&fx.origin, &b1, "docs"); + let expect = full_merge_root( + &fx, + "b1", + &[("src/a.txt", b"uno\ntwo\nthree\n")], + &["src/a.txt"], + ); + let clone = fx.clone(false); + clone_edit_src(&fx, &clone, b"uno\ntwo\nthree\n"); + let ours = refs::read_branch_tip(clone.layout(), "main").unwrap(); + assert!(!has(&clone, &theirs_docs), "premise: theirs' docs is a gap"); + + let err = clone.merge("origin/b1", "t").unwrap_err(); + assert!(matches!(err, Error::MergeConflicts(1)), "got {err:?}"); + assert!(clone.merge_in_progress()); + assert_eq!(clone.active_conflicts().unwrap(), vec!["src/a.txt"]); + let marked = std::fs::read_to_string(fx.clone_root.join("src/a.txt")).unwrap(); + assert!(marked.contains("<<<<<<<"), "markers on disk: {marked}"); + assert!(!fx.clone_root.join("docs").exists()); + assert!(!has(&clone, &theirs_docs), "carried, never fetched"); + assert_eq!( + refs::read_branch_tip(clone.layout(), "main").unwrap(), + ours, + "no ref moved" + ); + + // `sc conflicts` / `sc resolve` look a path up by descending it. + let v = clone.conflict_versions("src/a.txt", None).unwrap(); + assert!(matches!(&v.ours, crate::conflicts::Side::Present(b) if b == b"uno\ntwo\nthree\n")); + assert!(matches!(&v.theirs, crate::conflicts::Side::Present(b) if b == b"ONE\ntwo\nthree\n")); + assert!(matches!(&v.base, crate::conflicts::Side::Present(b) if b == SRC_A)); + // A path under a gap is a gap error, never an absent side. + let gap = clone + .conflict_versions("docs/api/guide.md", None) + .unwrap_err(); + assert!(matches!(gap, Error::GapOutsideFilter(_)), "got {gap:?}"); + + clone + .resolve_path("src/a.txt", crate::conflicts::ResolveSide::Ours, None) + .unwrap(); + let merged = clone.commit("t", "merge").unwrap(); + assert!(!clone.merge_in_progress()); + let snap = clone.snapshot(&merged).unwrap(); + assert_eq!(snap.parents.len(), 2); + assert_eq!(Some(snap.parents[0]), ours); + assert_eq!(snap.root, expect, "same tree as the full merge"); + assert_eq!(dir_id(&clone, &merged, "docs"), theirs_docs); + assert!(!has(&clone, &theirs_docs), "and still never fetched"); + clone.gc(std::time::Duration::from_secs(0)).unwrap(); + assert!(has(&clone, &merged)); + drop(clone); + fx.cleanup(); +} + +/// Aborting restores ours and removes the markers; the carried subtree was +/// never on disk and the ref never moved. +#[test] +fn aborting_a_partial_conflict_restores_ours() { + let fx = Fx::origin("abort", |_| {}); + fx.origin_branch( + "b1", + &[ + ("src/a.txt", b"ONE\ntwo\nthree\n"), + ("docs/api/guide.md", b"g changed\n"), + ], + ); + let clone = fx.clone(false); + clone_edit_src(&fx, &clone, b"uno\ntwo\nthree\n"); + let ours = refs::read_branch_tip(clone.layout(), "main").unwrap(); + assert!(matches!( + clone.merge("origin/b1", "t").unwrap_err(), + Error::MergeConflicts(1) + )); + clone.merge_abort().unwrap(); + assert!(!clone.merge_in_progress()); + assert_eq!( + std::fs::read(fx.clone_root.join("src/a.txt")).unwrap(), + b"uno\ntwo\nthree\n" + ); + assert!(!fx.clone_root.join("docs").exists()); + assert_eq!(refs::read_branch_tip(clone.layout(), "main").unwrap(), ours); + assert!(clone.status().unwrap().modified.is_empty()); + drop(clone); + fx.cleanup(); +} + +/// Abort checks the view before it removes anything: when ours' view is +/// missing an object, it fails with `GapInView` and the conflicted state, the +/// markers and the sidecar are all still there to retry. +#[test] +fn an_abort_that_hits_a_view_gap_changes_nothing() { + let fx = Fx::origin("abort-gap", |_| {}); + fx.origin_branch("b1", &[("src/a.txt", b"\xff\xfeTHEIRS\n")]); + let clone = fx.clone(false); + clone_edit_src(&fx, &clone, b"\xff\xfeOURS\n"); + let ours = refs::read_branch_tip(clone.layout(), "main") + .unwrap() + .unwrap(); + assert!(matches!( + clone.merge("origin/b1", "t").unwrap_err(), + Error::MergeConflicts(1) + )); + let sidecar = fx.clone_root.join("src/a.txt.theirs"); + assert!(sidecar.exists(), "a binary conflict writes a sidecar"); + + let src = dir_id(&clone, &ours, "src"); + clone.vfs().store().lock().unwrap().delete(&src).unwrap(); + let err = clone.merge_abort().unwrap_err(); + assert!(matches!(err, Error::GapInView { .. }), "got {err:?}"); + assert!(clone.merge_in_progress(), "the merge is still open"); + assert!(sidecar.exists(), "the sidecar is still there"); + drop(clone); + fx.cleanup(); +} + +/// A conflict outside the view is refused before any write, with the widen +/// hint; nothing is marked and no state is recorded. +#[test] +fn a_conflict_outside_the_view_is_refused_before_any_write() { + let fx = Fx::origin("conflict-out", |_| {}); + fx.origin_branch("b1", &[("docs/a.md", b"a1\n")]); + fx.origin_branch("b2", &[("docs/a.md", b"a2\n")]); let clone = fx.clone(true); - clone.merge("origin/b1", "t").unwrap(); + clone.merge("origin/b1", "t").unwrap(); // fast-forward let before = refs::read_branch_tip(clone.layout(), "main").unwrap(); let err = clone.merge("origin/b2", "t").unwrap_err(); assert!( - matches!(&err, Error::PartialCloneUnsupported(op) if op.contains("conflicts")), + matches!(&err, Error::InvalidArgument(m) if m.contains("outside your sparse checkout")), "got {err:?}" ); assert_eq!( @@ -230,12 +390,7 @@ fn conflict_on_a_partial_clone_is_refused_before_any_write() { before ); assert!(!clone.merge_in_progress()); - assert_eq!( - std::fs::read(fx.clone_root.join("src/a.txt")).unwrap(), - b"ONE\ntwo\nthree\n", - "no markers" - ); - assert!(!fx.clone_root.join("src/a.txt.theirs").exists()); + assert!(!fx.clone_root.join("docs").exists()); drop(clone); fx.cleanup(); } @@ -445,3 +600,98 @@ fn an_interrupted_fill_changes_nothing_and_the_retry_succeeds() { drop(clone); fx.cleanup(); } + +/// A conflict in the view while a protected subtree is carried: the decided +/// root holds the subtree by id, and the completion keeps its wraps, so the +/// recipient can still decrypt it afterwards. +#[test] +fn a_carried_protected_subtree_survives_a_conflicted_merge_and_its_completion() { + let (sk, pk) = scl_crypto::generate_keypair(); + let fx = protected_origin("conflict-wraps", &pk); + let base = fx.origin.head_tip().unwrap().unwrap(); + let base_snap = fx.origin.snapshot(&base).unwrap(); + let vault = dir_id(&fx.origin, &base, "vault"); + fx.origin_branch("b1", &[("src/a.txt", b"ONE\ntwo\nthree\n")]); + let clone = fx.clone(false); + clone_edit_src(&fx, &clone, b"uno\ntwo\nthree\n"); + assert!(matches!( + clone.merge("origin/b1", "t").unwrap_err(), + Error::MergeConflicts(1) + )); + clone + .resolve_path("src/a.txt", crate::conflicts::ResolveSide::Theirs, None) + .unwrap(); + let merged = clone.commit("t", "merge").unwrap(); + let snap = clone.snapshot(&merged).unwrap(); + assert_eq!(dir_id(&clone, &merged, "vault"), vault, "carried by id"); + assert!(!has(&clone, &vault), "never fetched"); + for (id, wraps) in &base_snap.protection.wrapped { + assert_eq!( + snap.protection.wrapped.get(id), + Some(wraps), + "wraps of {id}" + ); + } + let (cipher_id, _) = base_snap.protection.wrapped.iter().next().unwrap(); + let cipher = { + let arc = fx.origin.vfs().store(); + let mut s = arc.lock().unwrap(); + match s.get(cipher_id).unwrap() { + Object::Blob(b) => b, + other => panic!("expected a blob, got {}", other.kind_name()), + } + }; + let plain = + crate::protect::decrypt_with(&cipher, cipher_id, &[&snap.protection], &sk, "vault/s.txt") + .unwrap(); + assert_eq!(&plain[..], b"l1\nl2\nl3\n"); + drop(clone); + fx.cleanup(); +} + +/// A conflict in the view together with a clean protected merge outside it: +/// completion could not seal the outside plaintext, so the merge is refused +/// before any write and names the path. +#[test] +fn a_conflict_with_sealing_outside_the_view_is_refused_before_any_write() { + let (sk, pk) = scl_crypto::generate_keypair(); + let fx = protected_origin("conflict-seal", &pk); + fx.origin_branch( + "b1", + &[ + ("src/a.txt", b"ONE\ntwo\nthree\n"), + ("vault/s.txt", b"L1\nl2\nl3\n"), + ], + ); + fx.origin_branch( + "b2", + &[ + ("src/a.txt", b"uno\ntwo\nthree\n"), + ("vault/s.txt", b"l1\nl2\nL3\n"), + ], + ); + let clone = fx.clone(true); + clone.merge("origin/b1", "t").unwrap(); // fast-forward + let before = refs::read_branch_tip(clone.layout(), "main").unwrap(); + let on_disk = std::fs::read(fx.clone_root.join("src/a.txt")).unwrap(); + + let err = clone + .merge_with_identity("origin/b2", "t", Some(&sk)) + .unwrap_err(); + assert!( + matches!(&err, Error::InvalidArgument(m) if m.contains("vault/s.txt") && m.contains("sealing")), + "got {err:?}" + ); + assert_eq!( + refs::read_branch_tip(clone.layout(), "main").unwrap(), + before + ); + assert!(!clone.merge_in_progress()); + assert_eq!( + std::fs::read(fx.clone_root.join("src/a.txt")).unwrap(), + on_disk + ); + assert!(!fx.clone_root.join("vault").exists()); + drop(clone); + fx.cleanup(); +} diff --git a/crates/repo/src/replay.rs b/crates/repo/src/replay.rs index 5ab34aae..67dc7c09 100644 --- a/crates/repo/src/replay.rs +++ b/crates/repo/src/replay.rs @@ -546,6 +546,7 @@ impl Repo { }; let conflict_root = self.materialize_conflict_state( &carried, + &[], &to_encrypt, &sidecars, &conflict_prot, @@ -967,6 +968,7 @@ impl Repo { }; let conflict_root = self.materialize_conflict_state( &carried, + &[], &to_encrypt, &sidecars, &conflict_prot, diff --git a/crates/repo/src/repo.rs b/crates/repo/src/repo.rs index 71e26654..b60acadd 100644 --- a/crates/repo/src/repo.rs +++ b/crates/repo/src/repo.rs @@ -407,7 +407,14 @@ impl Repo { // Refuse loudly instead of ever letting that happen; a clean // (non-conflicted) pick/rebase fold still has `decided_root: None` // and keeps using the single-tip graft path, unaffected. - if partial && (decided_root.is_some() || merge_head.is_some()) { + // + // A conflicted MERGE completion is the exception (ADR-0049 decision + // 8): the conflict recorded a decided root holding the merge's whole + // outcome, carried subtrees included, so the out-of-view part is + // grafted from it below. A merge with no decided root, and every + // pick or rebase completion, still refuse. + let merge_completion = merge_head.is_some() && decided_root.is_some(); + if partial && !merge_completion && (decided_root.is_some() || merge_head.is_some()) { return Err(crate::promisor::partial_clone_unsupported( "merge/pick completion", )); @@ -664,11 +671,22 @@ impl Repo { // for the id-only walk). Scoped to a plain single-tip commit — // `decided_root`/`merge_head` both absent — matching the carry // block's own documented merge/pick boundary above. - if partial && decided_root.is_none() && merge_head.is_none() { - if let Some(t) = tip { + // A merge completion grafts from the DECIDED root instead of the tip: + // it already holds the merge's outcome for everything out of view. + let graft_from: Option = if merge_completion { + decided_root + } else if decided_root.is_none() && merge_head.is_none() { + match tip { + Some(t) => Some(self.snapshot(&t)?.root), + None => None, + } + } else { + None + }; + if partial { + if let Some(parent_root) = graft_from { let store_arc = self.vfs.store(); let mut store = store_arc.lock().unwrap(); - let parent_root = store.get_snapshot(&t)?.root; let p = promisor .as_ref() .expect("partial implies promisor().is_some()"); @@ -1376,14 +1394,6 @@ impl Repo { (theirs_snap.root, &theirs_snap.protection), identity, )?; - // Conflicts need a state contract this phase does not build - // (ADR-0049 decision 8): refuse before any merge output, merge - // state or working-tree write. - if !done.merge.conflicts.is_empty() { - return Err(crate::promisor::partial_clone_unsupported( - "a merge with conflicts", - )); - } let wrapped_carry = done.merged_wraps( &ours_snap.protection, &theirs_snap.protection, @@ -1485,8 +1495,20 @@ impl Repo { prefixes: union_prefixes, wrapped: merge_result.wrapped_carry.clone(), }; + // A partial clone's carried subtrees ride into the decided root + // by id (ADR-0049 decision 8). A carried root cannot coexist with + // a conflict: it means the three roots agreed. + let carried_subtrees: &[(String, scl_core::TreeEntry)] = match &tree_carried { + Some((crate::merge_tree::Carried::Entries(entries), _)) => entries, + Some((crate::merge_tree::Carried::Root(_), _)) => { + debug_assert!(false, "a carried root leaves no conflict"); + &[] + } + None => &[], + }; let conflict_root = self.materialize_conflict_state( &carried, + carried_subtrees, &to_encrypt, &merge_result.sidecars, &conflict_prot, @@ -1651,6 +1673,7 @@ impl Repo { pub(crate) fn materialize_conflict_state( &self, carried: &[(String, Vec, scl_core::FileMode, u8)], + carried_subtrees: &[(String, scl_core::TreeEntry)], to_encrypt: &[(String, Vec, scl_core::FileMode, Vec<[u8; 32]>)], sidecars: &[(String, Vec)], conflict_prot: &scl_core::Protection, @@ -1673,7 +1696,28 @@ impl Repo { "conflict in {names} is outside your sparse checkout; run `sc sparse set` to include it, then retry" ))); } - let conflict_root = self.vfs.write_tree_with_perms(carried)?; + // A partial clone completes from its view alone, so plaintext that + // must be sealed at completion has to sit inside the view (ADR-0049 + // decision 8). A full clone writes it wherever it lies and reads it + // back; here it would be left behind and dropped. + if self.promisor()?.is_some() { + if let Some((path, ..)) = to_encrypt.iter().find(|(p, ..)| !sparse.matches(p)) { + return Err(Error::InvalidArgument(format!( + "merge has conflicts, and {path} (outside your sparse checkout) needs sealing at completion; \ + run `sc sparse set` to include it, or `sc backfill --all`, then retry" + ))); + } + } + let conflict_root = if carried_subtrees.is_empty() { + self.vfs.write_tree_with_perms(carried)? + } else { + self.vfs + .write_tree_with_carried(carried, carried_subtrees)? + }; + // A carried subtree can sit inside the view: fill its closure before + // the first working-tree write (lazy partial clones only), outside + // the store lock. + self.ensure_view(conflict_root, &sparse)?; { let store_arc = self.vfs.store(); let mut store = store_arc.lock().unwrap(); @@ -1683,7 +1727,6 @@ impl Repo { // skipped content, so nothing is lost by not surfacing the list // here (the `Err`/`Stopped` return can't carry it). let mut cache = self.open_protected_cache()?; - // ensure-view-exempt: conflict materialization; a partial clone refuses conflicts before any write let _skipped = worktree::materialize( &self.layout, &mut store, @@ -1728,9 +1771,13 @@ impl Repo { } let theirs_id = crate::merge_state::read_merge_head(&self.layout)? .expect("in_progress is true but MERGE_HEAD is absent"); - // Remove any .theirs sidecars recorded as conflicts. - for path in crate::merge_state::read_conflicts(&self.layout)? { - let _ = std::fs::remove_file(self.layout.root.join(format!("{path}.theirs"))); + // Fill the view's closure first (lazy partial clones only), outside + // the store lock and before anything is removed: a view gap must + // leave the conflicted state intact and retryable (ADR-0049 + // decision 8). The sidecars go last for the same reason. + if let Some(ours_id) = self.head_tip()? { + let ours_root = self.snapshot(&ours_id)?.root; + self.ensure_view(ours_root, &self.sparse_spec()?)?; } // Restore working tree to ours tip. Pass theirs' root as `old_root` so // the deletion pass drops files the conflicted merge pulled in from @@ -1748,7 +1795,7 @@ impl Repo { // (not decrypted). With nothing decrypted, `materialize` records // nothing into a cache, so pass `None` — no cache to open or save // (P33 Task 7 item 7). - // ensure-view-exempt: merge abort; no merge can be in progress on a partial clone + // ensure_view ran above, before the lock and before any removal. skipped = worktree::materialize( &self.layout, &mut store, @@ -1760,6 +1807,10 @@ impl Repo { None, )?; } + // Remove any .theirs sidecars recorded as conflicts. + for path in crate::merge_state::read_conflicts(&self.layout)? { + let _ = std::fs::remove_file(self.layout.root.join(format!("{path}.theirs"))); + } crate::merge_state::clear(&self.layout)?; Ok(skipped) } @@ -6020,20 +6071,25 @@ mod tests { } #[test] - fn merge_refuses_on_partial_clone_instead_of_silently_dropping_gaps() { - // T5-I4 (mandatory reassignment): a two-parent merge completion on a - // partial clone would silently DROP out-of-filter subtrees (the - // Task 4 graft is single-tip-scoped). Refuse loudly instead of - // relying on an incidental NotFound from deep inside `three_way`. + fn conflicting_merge_on_partial_clone_completes_without_dropping_the_gap() { + // T5-I4 history: a two-parent merge completion on a partial clone + // used to silently DROP out-of-filter subtrees, so it was refused. + // Since P37c-iii a conflict inside the view is supported and the + // completion grafts the out-of-view part from the decided root: the + // never-fetched docs/ subtree must come through by id. let (src, src_root, dst, dst_root) = tmp_repo_with_src_and_docs_partial("merge-guard-partial"); + let docs_before = { + let tip = dst.head_tip().unwrap().unwrap(); + let arc = dst.vfs().store(); + let mut st = arc.lock().unwrap(); + let root = st.get_snapshot(&tip).unwrap().root; + st.get_tree(&root).unwrap().get("docs").unwrap().id + }; // Fork a second branch off the initial commit, then diverge both // sides so `merge` needs a genuine (non-FF) three-way, not just an - // adopt/fast-forward (both of which stay gap-tolerant via - // `materialize` and don't need this guard). `switch` between them - // (exercising its own P27 Task 5 fix to `materialize`'s old-root - // walk, below). + // adopt/fast-forward. dst.branch("feature").unwrap(); dst.switch("feature").unwrap(); std::fs::write(dst_root.join("src/a.txt"), b"feature-side-edit").unwrap(); @@ -6044,16 +6100,19 @@ mod tests { dst.commit("t", "main edit").unwrap(); let err = dst.merge_with_identity("feature", "t", None).unwrap_err(); - assert!( - matches!(err, Error::PartialCloneUnsupported(_)), - "expected the explicit partial-clone-unsupported refusal, got {err:?}" - ); - assert!(err.to_string().contains("backfill")); - assert!(err.to_string().contains("not supported on a partial clone")); - assert!( - !dst.merge_in_progress(), - "the refusal must be a preflight, not a conflict state" - ); + assert!(matches!(err, Error::MergeConflicts(1)), "got {err:?}"); + assert!(dst.merge_in_progress()); + dst.resolve_path("src/a.txt", crate::conflicts::ResolveSide::Ours, None) + .unwrap(); + let merged = dst.commit("t", "merge feature").unwrap(); + let docs_after = { + let arc = dst.vfs().store(); + let mut st = arc.lock().unwrap(); + let root = st.get_snapshot(&merged).unwrap().root; + st.get_tree(&root).unwrap().get("docs").unwrap().id + }; + assert_eq!(docs_after, docs_before, "the gap subtree is kept by id"); + assert!(!dst.merge_in_progress()); drop(src); drop(dst); diff --git a/crates/repo/src/worktree.rs b/crates/repo/src/worktree.rs index fd9b2fcf..4b05bfc5 100644 --- a/crates/repo/src/worktree.rs +++ b/crates/repo/src/worktree.rs @@ -355,6 +355,42 @@ fn walk_entries_with_perms( Ok(()) } +/// The file entry at `path` under `root`, found by descending its components +/// instead of flattening the tree (ADR-0049 decision 8). `Ok(None)` means the +/// path is not in the tree. A tree on the way that this store lacks is never +/// read as absence: on a partial clone it is the gap error naming `path`, on a +/// full clone the plain not-found, so a gap cannot pass for a deletion. +/// A path that names a directory is also `None`: callers ask for files. +pub(crate) fn tree_entry_at( + store: &mut Store, + root: ObjectId, + path: &str, + partial: bool, +) -> Result> { + let comps: Vec<&str> = path.split('/').collect(); + let mut cur = root; + for (i, comp) in comps.iter().enumerate() { + if !store.contains(&cur) { + return Err(if partial { + crate::promisor::partial_gap_hint(path) + } else { + scl_core::Error::NotFound(cur).into() + }); + } + let tree: Tree = store.get_tree(&cur)?; + let Some(e) = tree.get(comp) else { + return Ok(None); + }; + let last = i + 1 == comps.len(); + match (e.kind, last) { + (EntryKind::Blob, true) => return Ok(Some((e.id, e.mode, e.perms))), + (EntryKind::Tree, false) => cur = e.id, + _ => return Ok(None), + } + } + Ok(None) +} + /// Flatten a snapshot's root tree to `path -> blob id`. pub fn tree_file_ids(store: &mut Store, root: ObjectId) -> Result> { let mut out = BTreeMap::new(); diff --git a/demo/run_lazy_fetch_demo.sh b/demo/run_lazy_fetch_demo.sh index a0c54fb1..32d19dc8 100755 --- a/demo/run_lazy_fetch_demo.sh +++ b/demo/run_lazy_fetch_demo.sh @@ -196,4 +196,47 @@ cmp "$MFULL/docs/api/big.bin" "$W/feature-big.bin" || fail "the merge must carry echo "push + full re-clone: merged result has both sides, docs byte-identical ✔" echo -echo "RESULT: lazy by-id fetch verified (path-only fetch, offline/non-lazy gap errors, http v5, widen/commit/push/undo, merge)" +echo "=== 10: a conflict inside the view resolves and completes without the other side's region (ADR-0049) ===" +CORIGIN="$W/corigin" +mkdir -p "$CORIGIN" +( + cd "$CORIGIN" + "$SC" init >/dev/null + mkdir -p src docs/api + printf 'one\ntwo\nthree\n' > src/a.txt + printf 'guide\n' > docs/api/guide.md + head -c 65536 /dev/urandom | od -An -tx1 | tr -d ' \n' > docs/api/big.bin + "$SC" commit -m "base" --author demo >/dev/null + "$SC" branch feature + "$SC" switch feature >/dev/null + printf 'ONE\ntwo\nthree\n' > src/a.txt + head -c 65536 /dev/urandom | od -An -tx1 | tr -d ' \n' > docs/api/big.bin + cp docs/api/big.bin "$W/conflict-feature-big.bin" + "$SC" commit -m "feature edits src and docs" --author demo >/dev/null + "$SC" switch main >/dev/null +) +CONFLICT="$W/conflict" +"$SC" clone --filter src/ --lazy "$CORIGIN" "$CONFLICT" >/dev/null +printf 'uno\ntwo\nthree\n' > "$CONFLICT/src/a.txt" +(cd "$CONFLICT" && "$SC" commit -m "local src edit" --author demo >/dev/null && "$SC" fetch origin >/dev/null) +CBEFORE=$(objects "$CONFLICT") +if (cd "$CONFLICT" && "$SC" merge origin/feature --author demo >/dev/null 2>&1); then + fail "a conflicting merge must stop for resolution" +fi +grep -q '<<<<<<<' "$CONFLICT/src/a.txt" || fail "markers must be written in the view" +[ ! -e "$CONFLICT/docs" ] || fail "docs/ is outside the view and must not be written" +(cd "$CONFLICT" && "$SC" conflicts | grep -q 'src/a.txt') || fail "sc conflicts must list the path" +(cd "$CONFLICT" && "$SC" resolve --ours src/a.txt >/dev/null) || fail "sc resolve must work on a partial clone" +(cd "$CONFLICT" && "$SC" commit -m "merge feature" --author demo >/dev/null) || fail "completing the merge must work" +CAFTER=$(objects "$CONFLICT") +[ "$(($CAFTER - $CBEFORE))" -le 6 ] || fail "a conflict in the view must not fetch the other side's region ($CBEFORE -> $CAFTER)" +echo "conflict in src/ resolved and committed, docs/ unwritten and unfetched ($CBEFORE -> $CAFTER objects) ✔" +(cd "$CONFLICT" && "$SC" push origin >/dev/null) || fail "pushing the merge must work" +CFULL="$W/conflict-full" +"$SC" clone "$CORIGIN" "$CFULL" >/dev/null +[ "$(cat "$CFULL/src/a.txt")" = "$(printf 'uno\ntwo\nthree')" ] || fail "the resolution must be the merged result" +cmp "$CFULL/docs/api/big.bin" "$W/conflict-feature-big.bin" || fail "feature's docs blob must be carried byte-identical" +echo "push + full re-clone: resolution and carried docs both present ✔" + +echo +echo "RESULT: lazy by-id fetch verified (path-only fetch, offline/non-lazy gap errors, http v5, widen/commit/push/undo, merge, conflict)" diff --git a/docs/adr/0049-merge-on-partial-clones.md b/docs/adr/0049-merge-on-partial-clones.md index e85a293a..e56a95d7 100644 --- a/docs/adr/0049-merge-on-partial-clones.md +++ b/docs/adr/0049-merge-on-partial-clones.md @@ -1,6 +1,6 @@ # ADR-0049: Merge on partial clones (P37c) -- **Status:** Proposed (37c-i engine and 37c-ii clean merge built; conflicts, replay and sessions open) +- **Status:** Proposed (37c-i engine, 37c-ii clean merge and 37c-iii conflicts built; replay and sessions open) - **Date:** 2026-10-17 - **Phase:** 37c - **Builds on:** ADR-0048 (presence-based partial clones), ADR-0047 (want-by-id fetch), ADR-0012 (three-way merge), ADR-0025 (protected merge and replay), ADR-0043 (randomized sealing) @@ -86,9 +86,9 @@ Failure contract: fetched objects persist (fill rounds write immutable CAS addit Flatten drops empty directories and directory metadata and rebuilds directories with mode `0755`/perms `0` (`worktree.rs:348`, `vfs/lib.rs:175`); a carried subtree keeps whatever it has, including an empty directory entry that `graft_out_of_sparse` can produce (`worktree.rs:313`). Equivalence is defined on **file sets, modes and perms**, not on tree ids. A merged tree on a partial clone may differ from the full-clone merge by an empty directory entry. The differential test compares file sets, not ids. -### 8. Conflicts are deferred until their contract is specified +### 8. Conflicts: the contract (amended in 37c-iii) -A conflict needs a state contract this record does not yet fix, and the review showed why it is not a detail: +37c-i and 37c-ii refused every conflict because the state contract was unspecified, and the review showed why it is not a detail: - a conflicted merge's protected content merges produce plaintext that must live only on disk, with the CAS-safe decided root carrying everything else (`repo.rs:1420-1520`, `:1636`); - an in-view conflict can coexist with a **clean protected content merge outside the view**; today that clean plaintext is written straight to disk, and a decided root alone does not preserve it, so completion would lose it; @@ -96,7 +96,24 @@ A conflict needs a state contract this record does not yet fix, and the review s - presence does not imply the sparse view: a conflict outside the view needs the existing `sc sparse set` refusal, and abort must preflight and fill before it deletes sidecars (today it deletes them first, `repo.rs:1666`); - `sc conflicts` and `sc resolve` flatten whole trees and need path-directed lookup (`conflicts.rs:147`, `:263`, `:329`). -37c-i and 37c-ii therefore support **clean merges only**: any conflict on a partial clone is refused with a message saying so, before any merge-output, merge-state or working-tree write. This is implementable because `three_way_files` computes the conflict set without inserting objects (`merge.rs:191-438`), and a secret-registry conflict already fails before the file merge (`merge.rs:162`). 37c-iii amends this record with the conflict contract (a CAS-safe decided root, ephemeral conflict plaintext, persisted fresh wraps or an explicit refusal of the conflict-plus-out-of-view-protected-merge combination) before it builds anything. +37c-iii answers each, and the answers are mostly "the full-clone design already does this, so keep it and fix what a gap breaks". + +**State is unchanged.** A conflicted merge on a partial clone writes the same three files as a full clone (`MERGE_HEAD`, `MERGE_CONFLICTS`, `MERGE_DECIDED_ROOT`). The decided root is built by `write_tree_with_carried`: the CAS-safe enumerated entries (surviving ciphertext, plain files, marker text for plain conflicts) plus the carried subtrees verbatim. Nothing is sealed while the merge is open. Sealing happens at completion, from the plaintext on disk, which is why no fresh wrap needs to be persisted: every ciphertext in the decided root is an existing blob whose wraps sit in ours or theirs. (A blob carried from base alone cannot occur: a carried subtree equals the ours or theirs subtree, and a leaf keeps the side that changed.) Completion's policy is the union of both parents' prefixes and wraps, exactly as on a full clone, so the two engines agree by construction and need no second persistence format. + +**Two refusals, both before any merge-output, merge-state or working-tree write** (fetched objects persist, as in decision 5): + +1. A conflicted path outside the sparse view. This is the existing `materialize_conflict_state` refusal, and the widen hint stays: `sc sparse set` on a lazy clone, `sc backfill` on a non-lazy one. +2. A result that must be sealed at completion (protected plaintext from a clean content merge, or a plain file under a union rule) at a path outside the view. A full clone writes that plaintext to disk wherever it lives, and the completion commit reads it back. A partial clone completes from the view only, so the plaintext would be left behind and silently dropped. Writing it outside the view would also leave plaintext on disk where the user asked for none. The combination is refused with a message that names the path. Sealing it into the decided root at conflict time would lift the refusal, but it needs the fresh wraps persisted (a fourth state file), and nothing yet asks for it. + +**Completion** (`sc commit` with `MERGE_HEAD` set) is allowed on a partial clone only when a decided root is recorded, which every partial conflict writes. It builds `all` from the working tree and the in-view carry exactly as today, then splices every out-of-view subtree back in **from the decided root** by id (`graft_out_of_sparse` with the decided root as the parent), where a plain commit grafts from the tip. The wrap map keeps the complete union of both parents' wraps (decision 4's coarse rule: nothing is pruned when a subtree was grafted), since a grafted subtree's blobs never pass through the seal loop. A merge without a decided root, and every cherry-pick or rebase completion, still refuse; replay is 37c-iv. + +**The view is fixed across the merge.** `sc sparse set` already refuses while a merge is in progress, so the spec the conflict was checked against is the spec completion runs under. + +**The conflict materialization fills first.** `materialize_conflict_state` runs `ensure_view(conflict_root)` before the first working-tree write, as every other materializer does, because a carried subtree can sit inside the view. + +**Abort** checks the view before it removes anything. `merge --abort` runs `ensure_view(ours_root)` and the materialize preflight first and deletes the `.theirs` sidecars last, so a `GapInView` leaves the conflicted state intact and retryable. Its removal walk is the existing gap-tolerant present-only walk, so theirs' absent subtrees are skipped. + +**`sc conflicts` and `sc resolve`** look a path up by descending its components (`entry_at`) instead of flattening a whole tree. A missing tree on the way is a gap error that names the path, never `Absent`: an absent side means the path is not in that snapshot, and a gap must not read as a deletion. On a full clone the result is identical. ### 9. Acceptance: differential tests built from real operations @@ -118,7 +135,7 @@ A second differential runs each merge on a partial clone and a full clone. The n - **37c-i, engine (built).** The content/mode split and the collision check on the flatten merge (with the ADR-0012 amendment and tests), `merge_trees` returning `TreeMerge` or `Needs`, the policy-intersection test, `TreeMergeDone::merged_wraps` (the wrap-union rule; assembling and pruning the merged snapshot's `protection.wrapped` from it is 37c-ii), and the differential tests (file sets with modes and perms, conflicts, sidecars, wraps, carried-subtree policy, and fills on a store with objects removed). Nothing calls the new engine yet except tests. Decision 9's cases built from real operations (protect, grant, revoke, with decryption by each identity), equal plaintext sealed independently, an empty-directory entry on a carried side, merge then gc, and an interrupted fill then retry need the repo layer and land with 37c-ii. - **37c-ii, clean merge on a partial clone (built).** `merge_with_identity` uses the engine when the clone is partial: `Repo::tree_merge_filled` runs a pass under the store lock, fills what it asked for outside it (`Repo::fill_needs`, batched and split like every by-id fetch) and repeats; a conflict is refused with `PartialCloneUnsupported` before any write; the merged root is built by `Vfs::write_tree_with_carried` (enumerated files plus carried subtrees verbatim, or the carried root as is); `merged_wraps` is the wrap map, unpruned when anything was carried; `ensure_view(merged_root)` runs before the first ref, spec or working-tree write. The differing region's blobs are fetched too: the leaf merge returns file bytes even when one side wins by id, so a region both sides touched is read in full while an untouched subtree (identical on all three sides) is not. Cases from decision 9 that need real repo operations are covered by `partial_merge.rs`: disjoint carry without a fetch, the differing region (fill when lazy, gap error otherwise), a conflict refused, a carried protected subtree keeping its wraps, a keyless protected merge refusing before fetching (the merged result decrypts for the recipient and its plaintext never reaches the CAS), a carried subtree inside the view, and an interrupted fill followed by a retry. The carried wraps are checked byte for byte and by decrypting the carried ciphertext under the merged policy. Merge then gc is covered by the disjoint-carry test. Still open from decision 9: equal plaintext sealed independently, crossed revokes and regrants, an empty-directory entry on a carried side, and a randomized partial-versus-full differential. Demo: `demo/run_lazy_fetch_demo.sh` section 9. -- **37c-iii, conflicts.** First an amendment to this record that fixes decision 8's contract, then markers for present in-view conflicts, completion from the decided root, abort, `sc conflicts` and `sc resolve`. +- **37c-iii, conflicts (built).** The amendment to decision 8, then markers for in-view conflicts, completion from the decided root, abort, `sc conflicts` and `sc resolve`. A conflict outside the view, or a result that needs sealing outside it, is refused before any write. Tests in `partial_merge.rs` (conflict in view completes with the same tree as a full merge, carried subtree never fetched, abort, out-of-view refusals, carried protected subtree keeps its wraps through completion) and the CLI test `conflicting_merge_on_a_partial_clone_resolves_and_commits`. Demo section 10. - **37c-iv, replay.** `cherry-pick` and `rebase` call the same engine; `replay.rs:176` is the only guard that blocks them, and `three_way_files` is already their merge. Replay adds protected re-encryption under the replayed rules and the resumable-rebase state. - **Sessions** (`sc ws`, `sc work`): undecided. They materialize whole checkouts per agent and auto-merge on harvest. Whether they should work on a partial clone at all is a product question. diff --git a/docs/adr/README.md b/docs/adr/README.md index fd814c60..c33f3199 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -59,7 +59,7 @@ supersedes the old one and update the **Status** line of the superseded record. | [0046](0046-wal-bucket-remotes.md) | Bucket remotes via an immutable-pack WAL + one CAS'd manifest | Accepted | 36 | | [0047](0047-want-by-id-fetch-and-gap-fill.md) | Want-by-id fetch and explicit gap fill for partial clones | Accepted (37a built) | 37 | | [0048](0048-presence-based-partial-clones.md) | Presence-based walks for partial clones | Accepted (37b-i, 37b-ii built) | 37b | -| [0049](0049-merge-on-partial-clones.md) | Merge on partial clones | Proposed (37c-i, 37c-ii built) | 37c | +| [0049](0049-merge-on-partial-clones.md) | Merge on partial clones | Proposed (37c-i, 37c-ii, 37c-iii built) | 37c | See [`ROADMAP.md`](../../ROADMAP.md) for the phase sequence. ADR numbers are assigned in creation order, so 0017 — the secret scanner — sequences as phase P5, From 1379126e0fdef51e27c90409b051558f4c61f334 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Sun, 18 Oct 2026 16:05:20 +0300 Subject: [PATCH 2/4] fix(repo): address P37c-iii review findings - graft: carry an out-of-view file standing where the sparse spec expects a directory, instead of skipping it (a merge can swap a directory for a file) - abort: do not delete a tracked file named like a sidecar - conflict: refuse before any write when a sidecar would fall outside the view - completion: read parents with a presence-tolerant walk on a partial clone - resolve: do not read the base side it never uses - tests for each, plus enumerated out-of-view entries at completion, an in-view carried subtree filled during conflict, and a protected conflict in the view - ADR-0049: refusal list, wrap-set note (q13), sidecar next to an exact-file rule (q14), state durability (q15) Co-Authored-By: Claude Sonnet 5.5 --- crates/repo/src/conflicts.rs | 18 ++- crates/repo/src/partial_merge.rs | 185 +++++++++++++++++++++++ crates/repo/src/repo.rs | 46 +++++- crates/repo/src/worktree.rs | 71 ++++++++- docs/adr/0049-merge-on-partial-clones.md | 11 +- 5 files changed, 315 insertions(+), 16 deletions(-) diff --git a/crates/repo/src/conflicts.rs b/crates/repo/src/conflicts.rs index c62fae7f..5b807d01 100644 --- a/crates/repo/src/conflicts.rs +++ b/crates/repo/src/conflicts.rs @@ -170,14 +170,26 @@ impl Repo { &self, path: &str, identity: Option<&scl_crypto::SecretKey>, + ) -> Result { + self.versions_impl(path, identity, true) + } + + /// [`Repo::conflict_versions`], optionally leaving the base side + /// `Absent` unread. `resolve` takes ours or theirs and never needs the + /// base, which a partial clone may not hold. + fn versions_impl( + &self, + path: &str, + identity: Option<&scl_crypto::SecretKey>, + with_base: bool, ) -> Result { let triple = self.op_triple()?; let partial = self.promisor()?.is_some(); let store_arc = self.vfs.store(); let mut store = store_arc.lock().unwrap(); let base = match triple.base { - Some(id) => side_for(&mut store, id, identity, path, partial)?, - None => Side::Absent, + Some(id) if with_base => side_for(&mut store, id, identity, path, partial)?, + _ => Side::Absent, }; let ours = side_for(&mut store, triple.ours, identity, path, partial)?; let theirs = side_for(&mut store, triple.theirs, identity, path, partial)?; @@ -219,7 +231,7 @@ impl Repo { ))); } - let versions = self.conflict_versions(path, identity)?; + let versions = self.versions_impl(path, identity, false)?; let chosen = match side { ResolveSide::Ours => versions.ours, ResolveSide::Theirs => versions.theirs, diff --git a/crates/repo/src/partial_merge.rs b/crates/repo/src/partial_merge.rs index e0095fef..0a53a753 100644 --- a/crates/repo/src/partial_merge.rs +++ b/crates/repo/src/partial_merge.rs @@ -369,6 +369,30 @@ fn an_abort_that_hits_a_view_gap_changes_nothing() { fx.cleanup(); } +/// A file the user tracks under the name of a sidecar (`a.txt.theirs`) is not +/// this system's scratch: abort restores it and must not delete it. +#[test] +fn abort_keeps_a_tracked_file_that_is_named_like_a_sidecar() { + let fx = Fx::origin("abort-tracked", |root| { + std::fs::write(root.join("src/a.txt.theirs"), b"mine\n").unwrap(); + }); + fx.origin_branch("b1", &[("src/a.txt", b"ONE\ntwo\nthree\n")]); + let clone = fx.clone(false); + clone_edit_src(&fx, &clone, b"uno\ntwo\nthree\n"); + assert!(matches!( + clone.merge("origin/b1", "t").unwrap_err(), + Error::MergeConflicts(1) + )); + clone.merge_abort().unwrap(); + assert_eq!( + std::fs::read(fx.clone_root.join("src/a.txt.theirs")).unwrap(), + b"mine\n", + "the tracked file survives the abort" + ); + drop(clone); + fx.cleanup(); +} + /// A conflict outside the view is refused before any write, with the widen /// hint; nothing is marked and no state is recorded. #[test] @@ -695,3 +719,164 @@ fn a_conflict_with_sealing_outside_the_view_is_refused_before_any_write() { drop(clone); fx.cleanup(); } + +/// Out-of-view entries the merge READ and resolved cleanly (both sides edited +/// different `docs/` files) ride through a conflicted completion too: the +/// decided root holds enumerated entries there, not just carried subtrees, and +/// the graft must put them all back. Same tree as the full merge. +#[test] +fn a_conflict_completion_grafts_enumerated_out_of_view_entries() { + let fx = Fx::origin("conflict-enum", |_| {}); + fx.origin_branch( + "o", + &[("src/a.txt", b"uno\ntwo\nthree\n"), ("docs/b.md", b"b2\n")], + ); + fx.origin_branch( + "b1", + &[("src/a.txt", b"ONE\ntwo\nthree\n"), ("docs/a.md", b"a1\n")], + ); + let expect = full_merge_root( + &fx, + "b1", + &[("src/a.txt", b"uno\ntwo\nthree\n"), ("docs/b.md", b"b2\n")], + &["src/a.txt"], + ); + let clone = fx.clone(true); + clone.merge("origin/o", "t").unwrap(); // fast-forward + assert!(matches!( + clone.merge("origin/b1", "t").unwrap_err(), + Error::MergeConflicts(1) + )); + assert!(!fx.clone_root.join("docs").exists()); + clone + .resolve_path("src/a.txt", crate::conflicts::ResolveSide::Ours, None) + .unwrap(); + let merged = clone.commit("t", "merge").unwrap(); + assert_eq!(clone.snapshot(&merged).unwrap().root, expect); + drop(clone); + fx.cleanup(); +} + +/// A carried subtree that lies inside the view is filled before the conflict +/// is laid out: its file reaches the disk next to the markers. +#[test] +fn a_conflict_fills_a_carried_subtree_inside_the_view() { + let fx = Fx::origin("conflict-view", |_| {}); + fx.origin_branch( + "b1", + &[ + ("src/a.txt", b"ONE\ntwo\nthree\n"), + ("docs/api/guide.md", b"g changed\n"), + ], + ); + let clone = fx.clone(true); + clone + .set_sparse(&["src/".to_string(), "docs/".to_string()], None) + .unwrap(); + clone_edit_src(&fx, &clone, b"uno\ntwo\nthree\n"); + assert!(matches!( + clone.merge("origin/b1", "t").unwrap_err(), + Error::MergeConflicts(1) + )); + assert_eq!( + std::fs::read(fx.clone_root.join("docs/api/guide.md")).unwrap(), + b"g changed\n", + "the carried subtree was filled and laid out" + ); + clone.merge_abort().unwrap(); + assert_eq!( + std::fs::read(fx.clone_root.join("docs/api/guide.md")).unwrap(), + b"g\n", + "abort puts the view back" + ); + drop(clone); + fx.cleanup(); +} + +/// A protected path in the view conflicts: the markers are plaintext on disk +/// only, resolution and completion seal the result for the recipient, and no +/// plaintext reaches the store. +#[test] +fn a_protected_conflict_in_the_view_seals_at_completion() { + let (sk, pk) = scl_crypto::generate_keypair(); + let fx = protected_origin("conflict-protected", &pk); + fx.origin_branch("b1", &[("vault/s.txt", b"X\nl2\nl3\n")]); + fx.origin_branch("b2", &[("vault/s.txt", b"Y\nl2\nl3\n")]); + let clone = fx.clone(true); + clone + .set_sparse(&["src/".to_string(), "vault/".to_string()], Some(&sk)) + .unwrap(); + clone + .merge_with_identity("origin/b1", "t", Some(&sk)) + .unwrap(); // fast-forward + let err = clone + .merge_with_identity("origin/b2", "t", Some(&sk)) + .unwrap_err(); + assert!(matches!(err, Error::MergeConflicts(1)), "got {err:?}"); + let marked = std::fs::read(fx.clone_root.join("vault/s.txt")).unwrap(); + assert!(String::from_utf8_lossy(&marked).contains("<<<<<<<")); + assert!( + !has(&clone, &blob_id(&marked)), + "marker plaintext never reaches the store" + ); + + clone + .resolve_path( + "vault/s.txt", + crate::conflicts::ResolveSide::Ours, + Some(&sk), + ) + .unwrap(); + let merged = clone.commit("t", "merge").unwrap(); + let snap = clone.snapshot(&merged).unwrap(); + let arc = clone.vfs().store(); + let mut s = arc.lock().unwrap(); + let vault = s.get_tree(&snap.root).unwrap().get("vault").unwrap().id; + let entry = s.get_tree(&vault).unwrap().get("s.txt").unwrap().clone(); + assert_ne!(entry.perms & scl_core::PROTECTED, 0, "sealed again"); + let sealed = match s.get(&entry.id).unwrap() { + Object::Blob(b) => b, + other => panic!("expected a blob, got {}", other.kind_name()), + }; + let plain = + crate::protect::decrypt_with(&sealed, &entry.id, &[&snap.protection], &sk, "vault/s.txt") + .unwrap(); + assert_eq!(&plain[..], b"X\nl2\nl3\n"); + assert!( + !s.contains(&blob_id(b"X\nl2\nl3\n")), + "no plaintext in the CAS" + ); + drop(s); + drop(clone); + fx.cleanup(); +} + +/// A binary conflict writes `.theirs` beside the file. When the view is +/// the file itself the sidecar falls outside it, so the merge is refused before +/// any write instead of leaving plaintext where completion does not look. +#[test] +fn a_sidecar_outside_the_view_is_refused_before_any_write() { + let fx = Fx::origin("sidecar-out", |_| {}); + fx.origin_branch("b1", &[("src/a.txt", b"\xff\xfeTHEIRS\n")]); + let clone = fx.clone(false); + clone.set_sparse(&["src/a.txt".to_string()], None).unwrap(); + clone_edit_src(&fx, &clone, b"\xff\xfeOURS\n"); + let before = refs::read_branch_tip(clone.layout(), "main").unwrap(); + let err = clone.merge("origin/b1", "t").unwrap_err(); + assert!( + matches!(&err, Error::InvalidArgument(m) if m.contains("sidecar")), + "got {err:?}" + ); + assert!(!clone.merge_in_progress()); + assert!(!fx.clone_root.join("src/a.txt.theirs").exists()); + assert_eq!( + refs::read_branch_tip(clone.layout(), "main").unwrap(), + before + ); + assert_eq!( + std::fs::read(fx.clone_root.join("src/a.txt")).unwrap(), + b"\xff\xfeOURS\n" + ); + drop(clone); + fx.cleanup(); +} diff --git a/crates/repo/src/repo.rs b/crates/repo/src/repo.rs index b60acadd..4bb723dc 100644 --- a/crates/repo/src/repo.rs +++ b/crates/repo/src/repo.rs @@ -613,7 +613,14 @@ impl Repo { for parent in tip.into_iter().chain(merge_head) { let parent_root = store.get_snapshot(&parent)?.root; let mut entries = if partial { - worktree::tree_file_entries_with_perms_sparse(&mut store, parent_root, sparse)? + // A parent is only a fallback behind the decided root + // (merge completion); a subtree it lacks was never the + // only record of a path, so skip it, not fail. + worktree::tree_file_entries_with_perms_sparse_present( + &mut store, + parent_root, + sparse, + )? } else { worktree::tree_file_entries_with_perms(&mut store, parent_root)? }; @@ -1708,6 +1715,17 @@ impl Repo { ))); } } + // A sidecar is plaintext written beside a conflicted file. Outside the + // view it would sit where completion neither expects nor protects it, + // and a later commit would read it back as a plain file. + if self.promisor()?.is_some() { + if let Some((rel, _)) = sidecars.iter().find(|(r, _)| !sparse.matches(r)) { + return Err(Error::InvalidArgument(format!( + "merge has conflicts, and its sidecar {rel} would fall outside your sparse checkout; \ + run `sc sparse set` to include it, or `sc backfill --all`, then retry" + ))); + } + } let conflict_root = if carried_subtrees.is_empty() { self.vfs.write_tree_with_perms(carried)? } else { @@ -1783,6 +1801,16 @@ impl Repo { // the deletion pass drops files the conflicted merge pulled in from // theirs; materializing against ours==old would delete nothing. let mut skipped = Vec::new(); + // Sidecars (`.theirs`) this abort may remove: not a file either + // side tracks, which would be the user's own and was just restored. + // A lookup that fails is treated as tracked: a stray sidecar is + // harmless, deleting a tracked file is not. + let mut removable: Vec = Vec::new(); + let partial = self.promisor()?.is_some(); + let sidecars = crate::merge_state::read_conflicts(&self.layout)? + .into_iter() + .map(|p| format!("{p}.theirs")) + .collect::>(); let ours_tip = self.head_tip()?; if let Some(ours_id) = ours_tip { let store_arc = self.vfs.store(); @@ -1791,6 +1819,15 @@ impl Repo { let ours_root = ours_snap.root; let ours_protection = ours_snap.protection; let theirs_root = store.get_snapshot(&theirs_id)?.root; + for sidecar in sidecars { + let tracked = [ours_root, theirs_root].iter().any(|root| { + worktree::tree_entry_at(&mut store, *root, &sidecar, partial) + .map_or(true, |e| e.is_some()) + }); + if !tracked { + removable.push(sidecar); + } + } // No identity at abort time: protected files in ours are skipped // (not decrypted). With nothing decrypted, `materialize` records // nothing into a cache, so pass `None` — no cache to open or save @@ -1806,10 +1843,11 @@ impl Repo { &self.sparse_spec()?, None, )?; + } else { + removable = sidecars; } - // Remove any .theirs sidecars recorded as conflicts. - for path in crate::merge_state::read_conflicts(&self.layout)? { - let _ = std::fs::remove_file(self.layout.root.join(format!("{path}.theirs"))); + for sidecar in removable { + let _ = std::fs::remove_file(self.layout.root.join(sidecar)); } crate::merge_state::clear(&self.layout)?; Ok(skipped) diff --git a/crates/repo/src/worktree.rs b/crates/repo/src/worktree.rs index 4b05bfc5..6ee0c707 100644 --- a/crates/repo/src/worktree.rs +++ b/crates/repo/src/worktree.rs @@ -162,7 +162,23 @@ pub(crate) fn tree_file_entries_with_perms_sparse( return tree_file_entries_with_perms(store, root); } let mut out = BTreeMap::new(); - walk_entries_with_perms_sparse(store, root, String::new(), sparse, &mut out)?; + walk_entries_with_perms_sparse(store, root, String::new(), sparse, &mut out, false)?; + Ok(out) +} + +/// [`tree_file_entries_with_perms_sparse`] that skips a tree the store lacks +/// instead of failing. For a parent read only as a fallback on a partial clone +/// (ADR-0049 merge completion): a subtree it does not hold lies where the +/// decided root already decided, and is never the only record of a path. +pub(crate) fn tree_file_entries_with_perms_sparse_present( + store: &mut Store, + root: ObjectId, + sparse: &Sparse, +) -> Result> { + let mut out = BTreeMap::new(); + if store.contains(&root) { + walk_entries_with_perms_sparse(store, root, String::new(), sparse, &mut out, true)?; + } Ok(out) } @@ -172,6 +188,7 @@ fn walk_entries_with_perms_sparse( prefix: String, sparse: &Sparse, out: &mut BTreeMap, + skip_absent: bool, ) -> Result<()> { let tree: Tree = store.get_tree(&tree_id)?; for e in tree.entries { @@ -187,8 +204,8 @@ fn walk_entries_with_perms_sparse( } } EntryKind::Tree => { - if sparse.should_descend(&path) { - walk_entries_with_perms_sparse(store, e.id, path, sparse, out)?; + if sparse.should_descend(&path) && (!skip_absent || store.contains(&e.id)) { + walk_entries_with_perms_sparse(store, e.id, path, sparse, out, skip_absent)?; } } } @@ -305,9 +322,16 @@ pub(crate) fn graft_out_of_sparse( // An ancestor directory: some deeper prefix is in-sparse, so recurse // rather than grafting the whole thing whole. if pe.kind != EntryKind::Tree { - // A file sitting where the sparse spec expects a descendable - // directory (a genuinely malformed/unexpected shape) — leave - // whatever the built side already decided. + // A file where the sparse spec expects a descendable directory + // (a merge can swap a directory for a file, ADR-0049). A file + // that does not match the view is out of it, so it is grafted by + // id like any other out-of-sparse entry; dropping it would lose + // the path. Content the built side holds under the same name is + // refused for the same reason as above. + if by_name.contains_key(&pe.name) { + return Err(Error::GappedPathContent(path)); + } + by_name.insert(pe.name.clone(), pe); continue; } let child_built_id = match by_name.get(&pe.name) { @@ -984,6 +1008,41 @@ mod tests { std::fs::remove_dir_all(&layout.root).unwrap(); } + /// A merge can swap a directory for a file. When the file lies outside a + /// view whose prefixes sit below it, the graft must carry it by id, not + /// skip it as a malformed ancestor (ADR-0049 decision 8). + #[test] + fn graft_carries_an_out_of_view_file_standing_where_a_directory_was() { + let (layout, mut store) = tmp_objects("graft-dir-to-file"); + let file = |store: &mut Store, name: &str, bytes: &[u8]| TreeEntry { + name: name.into(), + kind: EntryKind::Blob, + id: store.put(Object::blob(bytes.to_vec())).unwrap(), + mode: FileMode::FILE, + perms: 0, + }; + let other = file(&mut store, "other.txt", b"o"); + // The decided tree replaced directory `src/` with the file `src`. + let src_file = file(&mut store, "src", b"now a file"); + let parent = store + .put(Object::Tree(Tree::new(vec![ + other.clone(), + src_file.clone(), + ]))) + .unwrap(); + // The built side comes from the view: only `other.txt` survives. + let built = store + .put(Object::Tree(Tree::new(vec![other.clone()]))) + .unwrap(); + let sparse = Sparse::new(vec!["src/a.txt".into(), "other.txt".into()]); + let promisor = crate::promisor::Promisor::new("origin", vec!["src/a.txt".into()]); + let out = graft_out_of_sparse(&mut store, built, parent, &sparse, &promisor, "").unwrap(); + let tree = store.get_tree(&out).unwrap(); + assert_eq!(tree.get("src").map(|e| e.id), Some(src_file.id)); + assert_eq!(tree.get("other.txt").map(|e| e.id), Some(other.id)); + let _ = std::fs::remove_dir_all(&layout.root); + } + #[test] fn materialize_rejects_path_traversal_entry() { let (layout, mut store) = tmp_objects("traversal"); diff --git a/docs/adr/0049-merge-on-partial-clones.md b/docs/adr/0049-merge-on-partial-clones.md index e56a95d7..42b1ee5a 100644 --- a/docs/adr/0049-merge-on-partial-clones.md +++ b/docs/adr/0049-merge-on-partial-clones.md @@ -98,12 +98,13 @@ Flatten drops empty directories and directory metadata and rebuilds directories 37c-iii answers each, and the answers are mostly "the full-clone design already does this, so keep it and fix what a gap breaks". -**State is unchanged.** A conflicted merge on a partial clone writes the same three files as a full clone (`MERGE_HEAD`, `MERGE_CONFLICTS`, `MERGE_DECIDED_ROOT`). The decided root is built by `write_tree_with_carried`: the CAS-safe enumerated entries (surviving ciphertext, plain files, marker text for plain conflicts) plus the carried subtrees verbatim. Nothing is sealed while the merge is open. Sealing happens at completion, from the plaintext on disk, which is why no fresh wrap needs to be persisted: every ciphertext in the decided root is an existing blob whose wraps sit in ours or theirs. (A blob carried from base alone cannot occur: a carried subtree equals the ours or theirs subtree, and a leaf keeps the side that changed.) Completion's policy is the union of both parents' prefixes and wraps, exactly as on a full clone, so the two engines agree by construction and need no second persistence format. +**State is unchanged.** A conflicted merge on a partial clone writes the same three files as a full clone (`MERGE_HEAD`, `MERGE_CONFLICTS`, `MERGE_DECIDED_ROOT`). The decided root is built by `write_tree_with_carried`: the CAS-safe enumerated entries (surviving ciphertext, plain files, marker text for plain conflicts) plus the carried subtrees verbatim. Nothing is sealed while the merge is open. Sealing happens at completion, from the plaintext on disk, which is why no fresh wrap needs to be persisted: every ciphertext in the decided root is an existing blob whose wraps sit in ours or theirs. A carried subtree equals the ours or theirs subtree, and a leaf keeps the side that changed, so every ciphertext in the decided root has its wraps in one of the two parents. Completion's policy is the union of both parents' prefixes and wraps, exactly as on a full clone, so the two engines agree by construction and need no second persistence format. One consequence is recorded in open question 13: a clean merge also unions the base's wraps and a conflicted completion does not. -**Two refusals, both before any merge-output, merge-state or working-tree write** (fetched objects persist, as in decision 5): +**Three refusals, all before any merge-output, merge-state or working-tree write** (fetched objects persist, as in decision 5): 1. A conflicted path outside the sparse view. This is the existing `materialize_conflict_state` refusal, and the widen hint stays: `sc sparse set` on a lazy clone, `sc backfill` on a non-lazy one. -2. A result that must be sealed at completion (protected plaintext from a clean content merge, or a plain file under a union rule) at a path outside the view. A full clone writes that plaintext to disk wherever it lives, and the completion commit reads it back. A partial clone completes from the view only, so the plaintext would be left behind and silently dropped. Writing it outside the view would also leave plaintext on disk where the user asked for none. The combination is refused with a message that names the path. Sealing it into the decided root at conflict time would lift the refusal, but it needs the fresh wraps persisted (a fourth state file), and nothing yet asks for it. +2. A sidecar (`.theirs`, written for a binary conflict) whose path is outside the view: it is plaintext beside the conflicted file, and outside the view completion neither expects nor protects it. +3. A result that must be sealed at completion (protected plaintext from a clean content merge, or a plain file under a union rule) at a path outside the view. A full clone writes that plaintext to disk wherever it lives, and the completion commit reads it back. A partial clone completes from the view only, so the plaintext would be left behind and silently dropped. Writing it outside the view would also leave plaintext on disk where the user asked for none. The combination is refused with a message that names the path. Sealing it into the decided root at conflict time would lift the refusal, but it needs the fresh wraps persisted (a fourth state file), and nothing yet asks for it. **Completion** (`sc commit` with `MERGE_HEAD` set) is allowed on a partial clone only when a decided root is recorded, which every partial conflict writes. It builds `all` from the working tree and the in-view carry exactly as today, then splices every out-of-view subtree back in **from the decided root** by id (`graft_out_of_sparse` with the decided root as the parent), where a plain commit grafts from the tip. The wrap map keeps the complete union of both parents' wraps (decision 4's coarse rule: nothing is pruned when a subtree was grafted), since a grafted subtree's blobs never pass through the seal loop. A merge without a decided root, and every cherry-pick or rebase completion, still refuse; replay is 37c-iv. @@ -162,6 +163,10 @@ A second differential runs each merge on a partial clone and a full clone. The n 11. **Gap error names one path.** On a non-lazy clone the error names the first missing path; a second gap shows only after the first `sc backfill`. Whether `sc backfill ` fetches the merge-base version of a file (the gap can be a blob path) is unverified. 12. **Plaintext copies across rounds.** A pass that returns `Needs` discards decrypted protected inputs held in plain `Vec`s that are not wiped. The full merge has the same exposure once; the fill loop repeats it per round. Memory hygiene only. +13. **Wrap sets differ between a clean and a conflicted completion.** A clean partial merge that carried a subtree keeps the union of ours, theirs and base wraps; a conflicted completion keeps ours and theirs, as a full clone does. No blob whose wrap exists only in base can reach the decided root, so nothing readable is lost, but the same merge can record a different `protection.wrapped` depending on whether it conflicted. A recipient revoked on both branches whose base wrap would otherwise come back is the visible case. Decide whether completion should read the merge base's wraps too. +14. **A sidecar next to an exact-file protection rule.** A protected binary conflict at `secret.bin` writes plaintext `secret.bin.theirs`; if the rule covers exactly `secret.bin`, the sidecar matches no rule and a later commit would read it back as a plain file. This predates partial clones (the scanner is the only guard). `sc resolve` removes it; a commit that leaves it behind should probably refuse. +15. **State durability.** `merge_state` renames without syncing, as the other state files do. A power loss between the decided-root write and `MERGE_HEAD` leaves a state that partial completion refuses and `merge --abort` clears. + ## Alternatives considered - **Fetch everything a merge might need first (`backfill --all`, then merge).** Correct and simple, and what the refusal message says today. It defeats the partial clone. From bd9d2e1539fa1adb908dad84cd15e9f9e62cea90 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Sun, 18 Oct 2026 16:07:39 +0300 Subject: [PATCH 3/4] fix(repo): keep the built entry when grafting a file over a view directory The graft refused a single-tip commit that replaced a tracked file with a directory holding an in-view file. Keep the built side's entry when it holds the name, and graft the parent's file only when it is absent. ADR-0049 records the protected delete-versus-modify parity with full clones (q16). Co-Authored-By: Claude Sonnet 5.5 --- crates/repo/src/worktree.rs | 54 +++++++++++++++++++++--- docs/adr/0049-merge-on-partial-clones.md | 2 + 2 files changed, 50 insertions(+), 6 deletions(-) diff --git a/crates/repo/src/worktree.rs b/crates/repo/src/worktree.rs index 6ee0c707..45e7528e 100644 --- a/crates/repo/src/worktree.rs +++ b/crates/repo/src/worktree.rs @@ -326,12 +326,10 @@ pub(crate) fn graft_out_of_sparse( // (a merge can swap a directory for a file, ADR-0049). A file // that does not match the view is out of it, so it is grafted by // id like any other out-of-sparse entry; dropping it would lose - // the path. Content the built side holds under the same name is - // refused for the same reason as above. - if by_name.contains_key(&pe.name) { - return Err(Error::GappedPathContent(path)); - } - by_name.insert(pe.name.clone(), pe); + // the path. When the built side already holds the name (a file + // replaced by a directory, or rewritten in place) it decided + // that path, so it stays. + by_name.entry(pe.name.clone()).or_insert(pe); continue; } let child_built_id = match by_name.get(&pe.name) { @@ -1043,6 +1041,50 @@ mod tests { let _ = std::fs::remove_dir_all(&layout.root); } + /// A single-tip commit that replaces the file `src` with a directory + /// holding an in-view file: the built side decided that path, so the + /// graft keeps it and does not resurrect the old file or refuse. + #[test] + fn graft_keeps_a_directory_the_built_side_put_where_a_file_was() { + let (layout, mut store) = tmp_objects("graft-file-to-dir"); + let blob = + |store: &mut Store, bytes: &[u8]| store.put(Object::blob(bytes.to_vec())).unwrap(); + let old_file = blob(&mut store, b"was a file"); + let a = blob(&mut store, b"a"); + let parent = store + .put(Object::Tree(Tree::new(vec![TreeEntry { + name: "src".into(), + kind: EntryKind::Blob, + id: old_file, + mode: FileMode::FILE, + perms: 0, + }]))) + .unwrap(); + let dir = store + .put(Object::Tree(Tree::new(vec![TreeEntry { + name: "a.txt".into(), + kind: EntryKind::Blob, + id: a, + mode: FileMode::FILE, + perms: 0, + }]))) + .unwrap(); + let built = store + .put(Object::Tree(Tree::new(vec![TreeEntry { + name: "src".into(), + kind: EntryKind::Tree, + id: dir, + mode: FileMode(0o755), + perms: 0, + }]))) + .unwrap(); + let sparse = Sparse::new(vec!["src/a.txt".into()]); + let promisor = crate::promisor::Promisor::new("origin", vec!["src/".into()]); + let out = graft_out_of_sparse(&mut store, built, parent, &sparse, &promisor, "").unwrap(); + assert_eq!(out, built, "the built directory stays"); + let _ = std::fs::remove_dir_all(&layout.root); + } + #[test] fn materialize_rejects_path_traversal_entry() { let (layout, mut store) = tmp_objects("traversal"); diff --git a/docs/adr/0049-merge-on-partial-clones.md b/docs/adr/0049-merge-on-partial-clones.md index 42b1ee5a..60226c38 100644 --- a/docs/adr/0049-merge-on-partial-clones.md +++ b/docs/adr/0049-merge-on-partial-clones.md @@ -166,6 +166,8 @@ A second differential runs each merge on a partial clone and a full clone. The n 13. **Wrap sets differ between a clean and a conflicted completion.** A clean partial merge that carried a subtree keeps the union of ours, theirs and base wraps; a conflicted completion keeps ours and theirs, as a full clone does. No blob whose wrap exists only in base can reach the decided root, so nothing readable is lost, but the same merge can record a different `protection.wrapped` depending on whether it conflicted. A recipient revoked on both branches whose base wrap would otherwise come back is the visible case. Decide whether completion should read the merge base's wraps too. 14. **A sidecar next to an exact-file protection rule.** A protected binary conflict at `secret.bin` writes plaintext `secret.bin.theirs`; if the rule covers exactly `secret.bin`, the sidecar matches no rule and a later commit would read it back as a plain file. This predates partial clones (the scanner is the only guard). `sc resolve` removes it; a commit that leaves it behind should probably refuse. 15. **State durability.** `merge_state` renames without syncing, as the other state files do. A power loss between the decided-root write and `MERGE_HEAD` leaves a state that partial completion refuses and `merge --abort` clears. +16. **A protected delete-versus-modify conflict resolved to the absent side.** The conflict path is not in the decided root (protected conflict plaintext lives on disk), so after `sc resolve` deletes the working file, completion's absent-protected carry restores it from the first parent. The full-clone completion runs the same code (`snapshot_files`, "explicit deletion of a protected file is a future operation"), so the partial path matches it. Recording a decided deletion in the merge state would fix both. +17. **Single-tip graft keeps what the built side decided.** `graft_out_of_sparse` carries an out-of-view file standing where the view expects a directory, unless the built side already holds that name (a file replaced by a directory in an ordinary commit). ## Alternatives considered From 40727e436e1ad34904c63980d8db0d57eb3adaee Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Sun, 18 Oct 2026 16:14:24 +0300 Subject: [PATCH 4/4] fix(repo): refuse a conflict sidecar that would overwrite a tracked file A binary conflict writes `.theirs`. If either tree already tracked that name, the write overwrote it and `resolve` kept the overwritten bytes. Refuse before any write, on full and partial clones alike. Co-Authored-By: Claude Sonnet 5.5 --- crates/repo/src/partial_merge.rs | 29 +++++++++++++++++++++++++++++ crates/repo/src/repo.rs | 21 +++++++++++++++++++++ 2 files changed, 50 insertions(+) diff --git a/crates/repo/src/partial_merge.rs b/crates/repo/src/partial_merge.rs index 0a53a753..38136307 100644 --- a/crates/repo/src/partial_merge.rs +++ b/crates/repo/src/partial_merge.rs @@ -880,3 +880,32 @@ fn a_sidecar_outside_the_view_is_refused_before_any_write() { drop(clone); fx.cleanup(); } + +/// A binary conflict's `.theirs` sidecar must not overwrite a tracked +/// file of that name: the merge is refused before any write. +#[test] +fn a_sidecar_that_would_overwrite_a_tracked_file_is_refused() { + let fx = Fx::origin("sidecar-tracked", |root| { + std::fs::write(root.join("src/a.txt.theirs"), b"mine\n").unwrap(); + }); + fx.origin_branch("b1", &[("src/a.txt", b"\xff\xfeTHEIRS\n")]); + let clone = fx.clone(false); + clone_edit_src(&fx, &clone, b"\xff\xfeOURS\n"); + let before = refs::read_branch_tip(clone.layout(), "main").unwrap(); + let err = clone.merge("origin/b1", "t").unwrap_err(); + assert!( + matches!(&err, Error::InvalidArgument(m) if m.contains("overwrite a tracked file")), + "got {err:?}" + ); + assert!(!clone.merge_in_progress()); + assert_eq!( + std::fs::read(fx.clone_root.join("src/a.txt.theirs")).unwrap(), + b"mine\n" + ); + assert_eq!( + refs::read_branch_tip(clone.layout(), "main").unwrap(), + before + ); + drop(clone); + fx.cleanup(); +} diff --git a/crates/repo/src/repo.rs b/crates/repo/src/repo.rs index 4bb723dc..56083a4e 100644 --- a/crates/repo/src/repo.rs +++ b/crates/repo/src/repo.rs @@ -1732,6 +1732,27 @@ impl Repo { self.vfs .write_tree_with_carried(carried, carried_subtrees)? }; + // A sidecar must not land on a file either tree tracks: the write + // below would overwrite it, and `resolve` leaves tracked sidecar-named + // files alone, so the overwritten bytes would be committed. A failed + // lookup counts as tracked. + if !sidecars.is_empty() { + let partial = self.promisor()?.is_some(); + let store_arc = self.vfs.store(); + let mut store = store_arc.lock().unwrap(); + for (rel, _) in sidecars { + let tracked = [old_root, conflict_root].iter().any(|root| { + worktree::tree_entry_at(&mut store, *root, rel, partial) + .map_or(true, |e| e.is_some()) + }); + if tracked { + return Err(Error::InvalidArgument(format!( + "merge has conflicts, and its sidecar {rel} would overwrite a tracked file; \ + rename or remove that file, then retry" + ))); + } + } + } // A carried subtree can sit inside the view: fill its closure before // the first working-tree write (lazy partial clones only), outside // the store lock.