From d3097d1aed0909ac139da47f9a0681091b761375 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 18:55:48 +0300 Subject: [PATCH 01/22] docs(spec): P33 randomized protected-path encryption design (#40) Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- ...-randomized-protected-encryption-design.md | 191 ++++++++++++++++++ 1 file changed, 191 insertions(+) create mode 100644 docs/superpowers/specs/2026-07-10-p33-randomized-protected-encryption-design.md diff --git a/docs/superpowers/specs/2026-07-10-p33-randomized-protected-encryption-design.md b/docs/superpowers/specs/2026-07-10-p33-randomized-protected-encryption-design.md new file mode 100644 index 00000000..cb8a9745 --- /dev/null +++ b/docs/superpowers/specs/2026-07-10-p33-randomized-protected-encryption-design.md @@ -0,0 +1,191 @@ +# P33: Randomized protected-path encryption (dual-read, randomized-write) + +- **Issue:** #40 (part of the #24 audit-response map; decisions locked in #30) +- **Date:** 2026-07-10 +- **Status:** Approved design, pre-implementation + +## Goal + +Close the convergent-equality boundary (ADR-0014's accepted dictionary / +equality-confirmation oracle on protected paths). All **new** protected +content is sealed under a fresh random DEK + random nonce, so two seals of +the same plaintext yield different ciphertext ids — the oracle is closed for +everything sealed from this phase on. Convergent ciphertext already in +history stays readable forever (dual-read); nothing is rewritten or forcibly +migrated. + +## Locked decisions honored (from #30 / issue #40) + +1. Randomized seals for all new protected content, with a format tag. +2. Dual-read, randomized-write; no hard break; a rewrap-shaped one-command + eager re-seal upgrades the live tip on demand. +3. Unchanged-detection at commit via a local stat cache + keyed-hash + fallback; commit still needs only public keys. +4. Accepted costs: (a) independent identical edits on two branches now + conflict; (b) `sc rewrap` is no longer policy-only/tree-identical when it + randomizes content; (c) identical plaintext at two paths no longer dedups. +5. Rotate-for-paths becomes meaningful; recorded as a follow-on, not built. + +## Design decisions made in this brainstorm + +- **Format tag = perms-byte flag.** A `RANDOMIZED` bit alongside `PROTECTED` + in the tree-entry perms byte. Blob layout and `decrypt_path` are untouched. +- **`sc rewrap` itself performs the eager upgrade** (no new command, no flag). +- **The stat cache is per-checkout, including ws workspaces**, keyed by one + per-repo local key. +- **The commit rule is format-dispatched, not cache-only** (see §3) — this is + what prevents a silent mass-migration commit on first use after upgrade. + +## 1. Seal primitive (`crates/crypto`) + +New `encrypt_path_randomized(plaintext) -> (blob, dek)`: fresh random DEK and +random nonce from `OsRng`, plus an `encrypt_path_randomized_with_rng` variant +for deterministic tests. Blob layout is the existing `nonce(24) ‖ +AEAD-ciphertext` with the existing `PATH_AAD` — both formats decrypt through +the unchanged `decrypt_path`, which is why dual-read needs zero read-path +code. `encrypt_path` (convergent) remains for the legacy-format +unchanged-comparison only; no new seal call site uses it. Wrapping/unwrapping +(`wrap_dek_for` / `unwrap_dek_with`) unchanged. The RustCrypto quarantine +holds: only `crates/crypto` grows crypto code. + +## 2. Format tag (`crates/core`) + +A new perms bit, `RANDOMIZED` (always set together with `PROTECTED`), on +every newly sealed tree entry. No encoding change and no snapshot-tag bump: a +pre-P33 store decodes with zero `RANDOMIZED` bits and behaves byte-for-byte +as today. Carry-by-id preserves the source entry's perms (the P24 +discipline), so the format bit rides through merge, replay, graft, and sparse +carry with no new plumbing. Format identification never requires fetching +blob bytes. + +Old binaries reading a new store mask-check `PROTECTED` and would misreport +randomized entries as modified in `status`; accepted (no deployed old peers, +same posture as prior phases). + +## 3. Unchanged detection (new `crates/repo/src/cache.rs`) + +- `.sc/local-key`: 32 random bytes, mode 0600, created lazily, never + committed, never transferred. +- Per-checkout cache files: `.sc/protected-cache` for the main working tree; + `.sc/ws/cache-` per ws workspace — beside the checkout dir, NOT inside + it (a file inside `.sc/ws//` would be read back as an untracked working + file at harvest), removed with the workspace at teardown. Entries: `path → (mtime, size, keyed_tag, + blob_id)` where `keyed_tag = BLAKE3-keyed(local_key, plaintext)`. + +Commit's per-protected-path rule (format-dispatched): + +| Prior tip entry | Rule | +|---|---| +| **Convergent** (no `RANDOMIZED` bit) | Today's convergent re-encrypt-and-compare. Unchanged → carry as-is (stays convergent, history stays quiet with no cache needed). Edited → seal randomized. | +| **Randomized** | Stat hit (mtime+size) → carry cached `blob_id`. Stat miss → keyed-tag compare; match → carry + refresh entry; mismatch or no entry → seal randomized, update entry. | +| **New protected path** | Seal randomized. | + +Degradation: a lost or stale cache produces spurious re-seals (new blob ids +for unchanged plaintext), never incorrectness. The cache file alone leaks +nothing — tags are PRF outputs under a key that never leaves `.sc/` — so the +cache does not reintroduce the equality oracle. Commit continues to need +only public keys; decrypt-and-compare is not used anywhere. + +## 4. Status / diff + +`diff_worktree` (`crates/repo/src/worktree.rs`) and `diff_unified` +(`crates/repo/src/repo.rs`) get the same format dispatch: convergent entries +keep the re-encrypt-and-compare trick; randomized entries consult the cache +**read-only** (stat, then keyed tag); a missing entry reports the path as +modified — spurious-but-safe, same degradation rule as commit. + +## 5. Cache population + +Every site that writes authorized plaintext to disk records a cache entry: + +- checkout / `sc switch` (materialize) +- merge / pick / rebase completion materialization +- `sc resolve` writes +- ws fork workspace materialization (per-workspace cache) +- sparse re-lay (`sc sparse set` / `disable`) +- commit itself (authoritative refresh) +- `sc rewrap` (it holds the plaintext while upgrading; see §7) + +Per-workspace caches preserve two P20 properties that a main-tree-only cache +would regress: untouched protected files in forked workspaces remain provably +unchanged (no false identical-edit conflicts at harvest, no spurious +`work-` fallbacks), and crash-recovery re-harvest still converges to the +idempotent `UpToDate` no-op because cache hits reproduce the same tree id. + +## 6. Merge / replay semantics (the accepted P15 adjustment) + +- Unchanged and one-side-changed fast paths survive unchanged via + carry-by-id. +- Both sides editing a protected path — **including identical edits, which + under convergent encryption used to id-match and fast-path** — now + conflict by construction. Resolution: P23 `sc conflicts` / `sc resolve`, + or diff3-with-`--identity`, which merges identical plaintexts cleanly and + re-seals the result randomized. +- The content-merge re-encryption site in `merge.rs` and replay's + `encrypt_protected` path switch to the randomized primitive. +- Protection-rule union, wrap union, and tombstone semantics (P15/P16) are + unchanged. Mixed convergent+randomized trees need no special casing: every + path is carried by id or re-sealed randomized. + +## 7. `sc rewrap` eager upgrade + +While unwrapping each protected blob's DEK (by wrap presence, as today), +rewrap additionally decrypts any still-convergent blob and re-seals it +randomized (fresh DEK + nonce, `RANDOMIZED` bit set), replaces the wrap list +with the rule's `granted_keys() + escrow` as before, and populates the +main-tree cache with the plaintext it already holds. Output gains a +`re-sealed N convergent blob(s)` line; the commit is no longer tree-identical +when any blob was upgraded (accepted cost 4b). Already-randomized blobs get +wrap-replacement only — ciphertext id unchanged — so a second rewrap +converges back to policy-only. One command, one commit, one oplog record, +skip-and-report semantics unchanged. + +## 8. Out of scope (recorded, not built) + +- Rotate-for-paths (unlocked follow-on; ADR records that ADR-0019's + objection dissolves for randomized content). +- Per-prefix convergent/randomized choice (rejected on #30). +- `sc secret` changes (already randomized) and the P28 low-entropy nudge + (stays — steering low-entropy values to `sc secret` remains right). +- History rewriting or forced migration. + +## 9. Error handling + +- Sealing to an empty granted set still fails loudly + (`encrypt_protected`'s existing guard). +- A corrupt cache file is treated as absent (spurious re-seal, warning on + stderr), never an error that blocks commit. +- A missing `.sc/local-key` is minted lazily; a permissions failure creating + it is a hard error (the cache cannot be safely keyed without it). +- `decrypt_path` failures keep the P15 corruption-vs-authorization + distinction (`Error::Crypto` vs `NotAuthorized`). + +## 10. Testing & proof + +Tests (per the acceptance criteria on #40): + +- Randomized seal round trip; equality-oracle regression (two seals of the + same plaintext yield different blob ids). +- Dual-read of pre-phase convergent objects (fixture built with + `encrypt_path`). +- Format-dispatched commit: unchanged convergent content carries convergent + with no cache; unchanged randomized content carries via stat hit, via + keyed-tag fallback, and re-seals (correctly, spuriously) on lost cache. +- Eager tip re-seal via rewrap: convergent→randomized, second rewrap + policy-only, skip-and-report on unopenable entries. +- The new identical-edit conflict resolving via `sc resolve` and via + diff3-`--identity`. +- Merge/replay/rewrap over mixed convergent+randomized trees. +- ws fork/harvest with untouched protected files: no false conflicts; the + per-workspace cache reproduces the base tree id. + +Demos: all existing protected-path demos stay green over mixed-format +stores; new `demo/run_randomized_demo.sh` proves the oracle is closed (same +plaintext, different ciphertext ids), old history still decrypts, and rewrap +upgrades the tip — run twice, zero residue. + +Docs: ADR-0043 (format change, dual-read posture, cache design and why it +doesn't reintroduce the oracle, accepted costs 4a–c, P15/P17 semantic +adjustments, rotate-for-paths follow-on); THREAT-MODEL and +CLAUDE.md/ARCHITECTURE.md updated — ADR-0014's caveat superseded for new +content. From 5d69ae2462458f40469a136c93ff04104ef7c527 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 19:05:43 +0300 Subject: [PATCH 02/22] docs(plan): P33 randomized protected-path encryption implementation plan (#40) Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- ...-10-p33-randomized-protected-encryption.md | 1142 +++++++++++++++++ 1 file changed, 1142 insertions(+) create mode 100644 docs/superpowers/plans/2026-07-10-p33-randomized-protected-encryption.md diff --git a/docs/superpowers/plans/2026-07-10-p33-randomized-protected-encryption.md b/docs/superpowers/plans/2026-07-10-p33-randomized-protected-encryption.md new file mode 100644 index 00000000..05aa9662 --- /dev/null +++ b/docs/superpowers/plans/2026-07-10-p33-randomized-protected-encryption.md @@ -0,0 +1,1142 @@ +# P33: Randomized Protected-Path Encryption — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Close the convergent-equality oracle (ADR-0014) — all new protected content seals under a fresh random DEK + nonce, old convergent ciphertext stays readable forever, and a per-checkout stat cache keeps unchanged content from re-sealing. + +**Architecture:** A `RANDOMIZED` perms bit tags the new format at the tree-entry level; the blob layout (`nonce(24) ‖ ct`) and `decrypt_path` are unchanged, so dual-read needs zero read-path code. The single production seal choke point (`protect::encrypt_protected`) flips to random seals, and a format-dispatched carry decision in `snapshot_files`/`diff_worktree` keeps unchanged content quiet: convergent priors use the existing re-encrypt-and-compare trick, randomized priors use a `.sc`-local stat cache + BLAKE3 keyed-tag fallback. `sc rewrap` eagerly upgrades convergent blobs at the tip. + +**Tech Stack:** Rust (stable, edition 2021). No new dependencies — `blake3` and `hex` are already workspace deps of `scl-repo`; randomness reaches `crates/repo` via `scl_crypto` re-exports (RustCrypto quarantine holds). + +**Spec:** `docs/superpowers/specs/2026-07-10-p33-randomized-protected-encryption-design.md` (read it first). Issue #40; decisions locked on #30. + +## Global Constraints + +- **RustCrypto quarantine:** all crypto code stays in `crates/crypto`. `crates/repo` may use `blake3` (already a dep, cf. `serve_tokens.rs`) and `scl_crypto::*` re-exports only. +- **No format break:** no snapshot-tag bump. A pre-P33 store must decode and behave byte-for-byte as today (zero `RANDOMIZED` bits ⇒ all-convergent behavior). +- **Commit needs only public keys.** Decrypt-and-compare is forbidden in commit/status/diff. +- **Degradation rule:** lost/stale/corrupt cache ⇒ spurious re-seals, never incorrectness, never a hard error. +- **Green at every task boundary:** `cargo test` passes after each task. Task order is load-bearing — the dispatch machinery (Tasks 4–5) must land before the seal flip (Task 6), or every commit would mass-re-seal. +- **Tests:** colocated `#[cfg(test)] mod tests`; disk-touching tests clean up and assert the path is gone. Reuse each file's existing test helpers (e.g. `worktree.rs`'s `tmp_objects`, `repo.rs`'s repo-builder helpers) instead of inventing parallel ones. +- **Errors:** per-crate `thiserror` enums; convert with `?`. +- Commit messages end with: + `Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB` + +--- + +### Task 1: `RANDOMIZED` perms bit in `scl-core` + +**Files:** +- Modify: `crates/core/src/object.rs:23` (beside `PROTECTED`) +- Modify: `crates/core/src/lib.rs:29` (re-export) + +**Interfaces:** +- Produces: `scl_core::RANDOMIZED: u8 = 0b0000_0010`. Every later task tests format with `perms & RANDOMIZED != 0` and mints new seals as `PROTECTED | RANDOMIZED`. + +- [ ] **Step 1: Write the failing test** in `crates/core/src/object.rs`'s existing `mod tests`: + +```rust +#[test] +fn randomized_bit_is_distinct_from_protected() { + assert_eq!(PROTECTED & RANDOMIZED, 0, "flags must not overlap"); + assert_ne!(RANDOMIZED, 0); + // A randomized entry is always also protected. + let perms = PROTECTED | RANDOMIZED; + assert!(perms & PROTECTED != 0 && perms & RANDOMIZED != 0); +} +``` + +- [ ] **Step 2: Run to verify it fails** + +Run: `cargo test -p scl-core randomized_bit -- --nocapture` +Expected: FAIL — `cannot find value RANDOMIZED`. + +- [ ] **Step 3: Implement** — in `crates/core/src/object.rs`, directly below `pub const PROTECTED: u8 = 0b0000_0001;`: + +```rust +/// Perms flag: this PROTECTED entry was sealed with a fresh random DEK+nonce +/// (P33) rather than convergently. Always set together with `PROTECTED`. +/// Format identification for dual-read lives here, in the tree entry, so no +/// caller ever needs to fetch blob bytes to know the seal format. +pub const RANDOMIZED: u8 = 0b0000_0010; +``` + +Add `RANDOMIZED` to the `pub use` list in `crates/core/src/lib.rs:29`. + +- [ ] **Step 4: Run to verify it passes** + +Run: `cargo test -p scl-core` +Expected: PASS (all). + +- [ ] **Step 5: Commit** + +```bash +git add crates/core/src/object.rs crates/core/src/lib.rs +git commit -m "P33: add RANDOMIZED perms flag to scl-core" +``` + +--- + +### Task 2: `encrypt_path_randomized` in `scl-crypto` + +**Files:** +- Modify: `crates/crypto/src/envelope.rs` (below `encrypt_path`, ~line 168) +- Modify: `crates/crypto/src/lib.rs` (export) + +**Interfaces:** +- Consumes: existing `decrypt_path`, `PATH_AAD`, `DEK_LEN`, `NONCE_LEN`. +- Produces: `pub fn encrypt_path_randomized(plaintext: &[u8]) -> (Vec, Zeroizing<[u8; 32]>)` and `pub fn encrypt_path_randomized_with_rng(plaintext: &[u8], rng: &mut R) -> (Vec, Zeroizing<[u8; 32]>)`. Blob layout identical to `encrypt_path` (`nonce(24) ‖ ct`, same `PATH_AAD`) so the existing `decrypt_path` opens both. + +- [ ] **Step 1: Write the failing tests** in `envelope.rs`'s `mod tests`: + +```rust +#[test] +fn randomized_seal_roundtrips_and_closes_the_oracle() { + let pt = b"the database password is hunter2"; + let (blob1, dek1) = encrypt_path_randomized(pt); + let (blob2, dek2) = encrypt_path_randomized(pt); + // THE phase invariant: same plaintext, different ciphertext bytes. + assert_ne!(blob1, blob2, "equality oracle must be closed"); + assert_ne!(dek1.as_slice(), dek2.as_slice()); + // Dual-read: the unchanged decrypt_path opens a randomized blob. + assert_eq!(&decrypt_path(&blob1, &dek1).unwrap()[..], pt); + assert_eq!(&decrypt_path(&blob2, &dek2).unwrap()[..], pt); +} + +#[test] +fn randomized_seal_is_deterministic_under_a_seeded_rng() { + let mut r1 = rng(7); + let mut r2 = rng(7); + let (b1, _) = encrypt_path_randomized_with_rng(b"x", &mut r1); + let (b2, _) = encrypt_path_randomized_with_rng(b"x", &mut r2); + assert_eq!(b1, b2); +} +``` + +- [ ] **Step 2: Run to verify they fail** + +Run: `cargo test -p scl-crypto randomized_seal -- --nocapture` +Expected: FAIL — function not found. + +- [ ] **Step 3: Implement** in `envelope.rs`, after `encrypt_path` (note the doc comment on `encrypt_path` should also gain one line: "Legacy convergent format — kept for dual-read comparison only; new seals use `encrypt_path_randomized` (P33)."): + +```rust +/// Randomized file encryption (P33): fresh random DEK + random nonce per +/// seal, so identical plaintext yields different ciphertext every time — +/// the ADR-0014 equality oracle is closed for content sealed this way. +/// Blob layout and AAD are identical to `encrypt_path` (`nonce ‖ ct`, +/// `PATH_AAD`), so `decrypt_path` opens both formats unchanged (dual-read). +pub fn encrypt_path_randomized(plaintext: &[u8]) -> (Vec, Zeroizing<[u8; DEK_LEN]>) { + encrypt_path_randomized_with_rng(plaintext, &mut OsRng) +} + +/// `encrypt_path_randomized` with a caller-supplied RNG (deterministic in tests). +pub fn encrypt_path_randomized_with_rng( + plaintext: &[u8], + rng: &mut R, +) -> (Vec, Zeroizing<[u8; DEK_LEN]>) { + let mut dek = Zeroizing::new([0u8; DEK_LEN]); + rng.fill_bytes(dek.as_mut_slice()); + let mut nonce = [0u8; NONCE_LEN]; + rng.fill_bytes(&mut nonce); + + let cipher = XChaCha20Poly1305::new_from_slice(dek.as_slice()).expect("32-byte DEK"); + let ciphertext = cipher + .encrypt( + XNonce::from_slice(&nonce), + Payload { + msg: plaintext, + aad: PATH_AAD, + }, + ) + .expect("aead encrypt is infallible for valid inputs"); + + let mut blob = Vec::with_capacity(NONCE_LEN + ciphertext.len()); + blob.extend_from_slice(&nonce); + blob.extend_from_slice(&ciphertext); + (blob, dek) +} +``` + +Export both from `crates/crypto/src/lib.rs` in the existing `pub use envelope::{...}` list. + +- [ ] **Step 4: Run to verify they pass** + +Run: `cargo test -p scl-crypto` +Expected: PASS (all). + +- [ ] **Step 5: Commit** + +```bash +git add crates/crypto/src/envelope.rs crates/crypto/src/lib.rs +git commit -m "P33: randomized path seal (fresh DEK+nonce, dual-read layout)" +``` + +--- + +### Task 3: `cache.rs` — local key + per-checkout unchanged-detection cache + +**Files:** +- Create: `crates/repo/src/cache.rs` +- Modify: `crates/repo/src/lib.rs` (add `pub(crate) mod cache;` — match the file's existing mod-list style) +- Modify: `crates/repo/src/layout.rs` (three path helpers) + +**Interfaces:** +- Consumes: `scl_crypto::random_hex` (for key minting), `blake3::keyed_hash`, `hex`. +- Produces (all `pub(crate)`, used by Tasks 4, 5, 7, 8, 10): + - `cache::local_key(layout: &Layout) -> Result<[u8; 32]>` — reads `.sc/local-key` (64 hex chars), lazily minting it 0600 if absent; hard error only on create/permission failure. + - `struct ProtectedCache` with: + - `ProtectedCache::open(key: [u8; 32], path: Option) -> ProtectedCache` — loads entries if the file exists; a corrupt file is treated as empty (stderr warning); `path: None` = ephemeral in-memory cache (used by `sc work`). + - `fn unchanged(&self, rel: &str, abs: &Path, plaintext: &[u8]) -> Option` — stat hit (mtime+size) or keyed-tag match returns the cached ciphertext blob id. + - `fn record(&mut self, rel: &str, abs: &Path, plaintext: &[u8], blob_id: ObjectId)` + - `fn save(&self) -> Result<()>` — atomic (write temp sibling + rename); no-op when `path` is `None`. + - `Layout::local_key_path()` → `.sc/local-key`; `Layout::protected_cache_path()` → `.sc/protected-cache`; `Layout::ws_cache_path(i: usize)` → `.sc/ws/cache-` (beside the checkout dir, NOT inside `.sc/ws//` — a file inside the checkout would be harvested as an untracked working file). + +- [ ] **Step 1: Add the Layout helpers** (no test needed — pure path joins, mirror the style of `sparse_path()` at `layout.rs:78`): + +```rust +/// `.sc/local-key` — per-repo random key for the P33 unchanged-detection +/// cache's keyed hashes. Never committed, never transferred. +pub fn local_key_path(&self) -> PathBuf { + self.dot_sc.join("local-key") +} + +/// `.sc/protected-cache` — the main working tree's P33 stat cache. +pub fn protected_cache_path(&self) -> PathBuf { + self.dot_sc.join("protected-cache") +} + +/// `.sc/ws/cache-` — workspace `i`'s P33 stat cache. Lives BESIDE the +/// checkout dir (`.sc/ws//`), never inside it, so harvest's worktree +/// read can't pick it up as an untracked file. +pub fn ws_cache_path(&self, i: usize) -> PathBuf { + self.dot_sc.join("ws").join(format!("cache-{i}")) +} +``` + +- [ ] **Step 2: Write the failing tests** — create `crates/repo/src/cache.rs` with the tests first (module skeleton + `mod tests`). Use a tempdir pattern like `worktree.rs`'s `tmp_objects`: + +```rust +#[cfg(test)] +mod tests { + use super::*; + + fn tmp(tag: &str) -> Layout { + let root = std::env::temp_dir().join(format!("scl-cache-{tag}-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&root); + std::fs::create_dir_all(root.join(".sc")).unwrap(); + Layout::at(&root) + } + + fn cleanup(layout: &Layout) { + std::fs::remove_dir_all(&layout.root).unwrap(); + assert!(!layout.root.exists()); + } + + #[test] + fn local_key_is_minted_once_and_stable() { + let layout = tmp("key"); + let k1 = local_key(&layout).unwrap(); + let k2 = local_key(&layout).unwrap(); + assert_eq!(k1, k2, "second read must return the same key"); + assert_ne!(k1, [0u8; 32]); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + let mode = std::fs::metadata(layout.local_key_path()) + .unwrap() + .permissions() + .mode(); + assert_eq!(mode & 0o777, 0o600, "key file must be 0600"); + } + cleanup(&layout); + } + + #[test] + fn stat_hit_tag_fallback_and_miss() { + let layout = tmp("hits"); + let key = local_key(&layout).unwrap(); + let abs = layout.root.join("secret.txt"); + std::fs::write(&abs, b"v1").unwrap(); + let id = ObjectId::of(b"cipher-of-v1"); + + let mut c = ProtectedCache::open(key, Some(layout.protected_cache_path())); + assert_eq!(c.unchanged("secret.txt", &abs, b"v1"), None, "empty cache misses"); + c.record("secret.txt", &abs, b"v1", id); + + // Stat hit: file untouched. + assert_eq!(c.unchanged("secret.txt", &abs, b"v1"), Some(id)); + + // Tag fallback: touch mtime without changing content. + std::fs::write(&abs, b"v1").unwrap(); + assert_eq!(c.unchanged("secret.txt", &abs, b"v1"), Some(id)); + + // Real change: miss. + std::fs::write(&abs, b"v2").unwrap(); + assert_eq!(c.unchanged("secret.txt", &abs, b"v2"), None); + cleanup(&layout); + } + + #[test] + fn save_load_roundtrip_and_corrupt_file_is_empty() { + let layout = tmp("persist"); + let key = local_key(&layout).unwrap(); + let abs = layout.root.join("a b/spaced name.txt"); // path with spaces + std::fs::create_dir_all(abs.parent().unwrap()).unwrap(); + std::fs::write(&abs, b"v").unwrap(); + let id = ObjectId::of(b"c"); + + let mut c = ProtectedCache::open(key, Some(layout.protected_cache_path())); + c.record("a b/spaced name.txt", &abs, b"v", id); + c.save().unwrap(); + + let c2 = ProtectedCache::open(key, Some(layout.protected_cache_path())); + assert_eq!(c2.unchanged("a b/spaced name.txt", &abs, b"v"), Some(id)); + + // Corrupt file: degrades to empty, never errors. + std::fs::write(layout.protected_cache_path(), b"garbage\nlines\n").unwrap(); + let c3 = ProtectedCache::open(key, Some(layout.protected_cache_path())); + assert_eq!(c3.unchanged("a b/spaced name.txt", &abs, b"v"), None); + cleanup(&layout); + } + + #[test] + fn ephemeral_cache_never_touches_disk() { + let layout = tmp("ephemeral"); + let key = local_key(&layout).unwrap(); + let abs = layout.root.join("f"); + std::fs::write(&abs, b"v").unwrap(); + let mut c = ProtectedCache::open(key, None); + c.record("f", &abs, b"v", ObjectId::of(b"c")); + c.save().unwrap(); + assert!(!layout.protected_cache_path().exists()); + cleanup(&layout); + } +} +``` + +- [ ] **Step 3: Run to verify they fail** + +Run: `cargo test -p scl-repo cache:: -- --nocapture` +Expected: FAIL to compile — types not defined. + +- [ ] **Step 4: Implement the module**: + +```rust +//! P33: local unchanged-detection cache for randomized protected paths. +//! +//! Randomized seals (ADR-0043) make re-encrypt-and-compare useless for +//! change detection, so commit/status/diff consult this `.sc`-local, +//! never-committed, never-transferred map instead: +//! `path -> (mtime_ns, size, keyed_tag, ciphertext blob id)`. +//! The tag is `blake3::keyed_hash(local_key, plaintext)` under a random +//! per-repo key (`.sc/local-key`, 0600), so the cache file alone leaks +//! nothing — it does not reintroduce the equality oracle this phase closes. +//! A lost/stale/corrupt cache degrades to spurious re-seals, never +//! incorrectness and never a hard error. + +use std::collections::BTreeMap; +use std::path::{Path, PathBuf}; + +use scl_core::ObjectId; + +use crate::error::Result; +use crate::layout::Layout; + +/// Load (minting if absent) the per-repo random cache key. The key file is +/// the only hard-error surface here: if it can't be created/read, the cache +/// cannot be safely keyed, so surface it rather than running unkeyed. +pub(crate) fn local_key(layout: &Layout) -> Result<[u8; 32]> { + let path = layout.local_key_path(); + let hex_str = match std::fs::read_to_string(&path) { + Ok(s) => s.trim().to_string(), + Err(e) if e.kind() == std::io::ErrorKind::NotFound => { + let s = scl_crypto::random_hex(32); // 32 random bytes -> 64 hex chars + // Mirror the serve-TLS key discipline: 0600 before content matters. + std::fs::write(&path, &s)?; + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o600))?; + } + s + } + Err(e) => return Err(e.into()), + }; + let bytes = hex::decode(&hex_str) + .map_err(|_| crate::error::Error::Corrupt("malformed .sc/local-key".into()))?; + bytes + .try_into() + .map_err(|_| crate::error::Error::Corrupt("malformed .sc/local-key".into())) +} +``` + +> NOTE: check `scl_crypto::random_hex`'s contract at `crates/crypto/src/lib.rs:37` — if `random_hex(32)` returns 32 hex *chars* (16 bytes) rather than 64, pass 64 or add a `random_bytes(n)` helper to `scl-crypto` instead. The test (`k1 != [0u8;32]`, 32-byte decode) will catch a mismatch. If `Error::Corrupt` doesn't exist in `crates/repo/src/error.rs`, use the closest existing variant (e.g. `Error::InvalidArgument`) — do not add a new variant for this. + +```rust +#[derive(Clone, Copy, Debug, PartialEq)] +struct CacheEntry { + mtime_ns: u128, + size: u64, + tag: [u8; 32], + blob_id: ObjectId, +} + +/// Per-checkout unchanged-detection cache (main tree, one ws workspace, or +/// an ephemeral in-memory one for `sc work` temp checkouts). +pub(crate) struct ProtectedCache { + key: [u8; 32], + /// `None` => ephemeral: `save()` is a no-op. + path: Option, + entries: BTreeMap, +} + +impl ProtectedCache { + pub(crate) fn open(key: [u8; 32], path: Option) -> ProtectedCache { + let mut entries = BTreeMap::new(); + if let Some(p) = &path { + if let Ok(text) = std::fs::read_to_string(p) { + for line in text.lines() { + // Format: ` ` + // (path last: it may contain spaces). + let mut it = line.splitn(5, ' '); + let parsed = (|| { + let mtime_ns: u128 = it.next()?.parse().ok()?; + let size: u64 = it.next()?.parse().ok()?; + let tag: [u8; 32] = hex::decode(it.next()?).ok()?.try_into().ok()?; + let blob_id = ObjectId::from_hex(it.next()?).ok()?; + Some((it.next()?.to_string(), CacheEntry { mtime_ns, size, tag, blob_id })) + })(); + match parsed { + Some((path, e)) => { + entries.insert(path, e); + } + None => { + eprintln!("warning: ignoring corrupt protected-cache line"); + entries.clear(); + break; // degrade to empty: spurious re-seals, never incorrectness + } + } + } + } + } + ProtectedCache { key, path, entries } + } + + fn tag(&self, plaintext: &[u8]) -> [u8; 32] { + *blake3::keyed_hash(&self.key, plaintext).as_bytes() + } + + fn stat(abs: &Path) -> Option<(u128, u64)> { + let md = std::fs::metadata(abs).ok()?; + let mtime = md.modified().ok()?; + let ns = mtime + .duration_since(std::time::UNIX_EPOCH) + .ok()? + .as_nanos(); + Some((ns, md.len())) + } + + /// The cached ciphertext blob id iff this exact plaintext is what last + /// sealed at `rel`: stat hit (mtime+size unchanged) short-circuits; + /// otherwise fall back to the keyed-tag comparison. + pub(crate) fn unchanged(&self, rel: &str, abs: &Path, plaintext: &[u8]) -> Option { + let e = self.entries.get(rel)?; + if let Some((mtime_ns, size)) = Self::stat(abs) { + if mtime_ns == e.mtime_ns && size == e.size { + return Some(e.blob_id); + } + } + (self.tag(plaintext) == e.tag).then_some(e.blob_id) + } + + /// Record that `plaintext` at `rel` seals to `blob_id`. Missing stat + /// (file vanished mid-operation) just skips the entry. + pub(crate) fn record(&mut self, rel: &str, abs: &Path, plaintext: &[u8], blob_id: ObjectId) { + if let Some((mtime_ns, size)) = Self::stat(abs) { + let tag = self.tag(plaintext); + self.entries + .insert(rel.to_string(), CacheEntry { mtime_ns, size, tag, blob_id }); + } + } + + /// Atomic write (temp sibling + rename); no-op for an ephemeral cache. + pub(crate) fn save(&self) -> Result<()> { + let Some(path) = &self.path else { return Ok(()) }; + let mut out = String::new(); + for (p, e) in &self.entries { + out.push_str(&format!( + "{} {} {} {} {}\n", + e.mtime_ns, + e.size, + hex::encode(e.tag), + e.blob_id.to_hex(), + p + )); + } + if let Some(parent) = path.parent() { + std::fs::create_dir_all(parent)?; + } + let tmp = path.with_extension(format!("tmp-{}", std::process::id())); + std::fs::write(&tmp, out)?; + std::fs::rename(&tmp, path)?; + Ok(()) + } +} +``` + +> NOTE: check `ObjectId`'s hex API in `crates/core` — `refs.rs` serializes branch tips to hex; use exactly the same to/from helpers it uses (adjust `to_hex`/`from_hex` names to what exists). + +- [ ] **Step 5: Run to verify they pass** + +Run: `cargo test -p scl-repo cache::` +Expected: PASS (4 tests). + +- [ ] **Step 6: Commit** + +```bash +git add crates/repo/src/cache.rs crates/repo/src/lib.rs crates/repo/src/layout.rs +git commit -m "P33: per-checkout protected-path stat cache + local key" +``` + +--- + +### Task 4: Format dispatch in `diff_worktree` and `diff_unified` + +**Files:** +- Modify: `crates/repo/src/worktree.rs:477-529` (`diff_worktree`) +- Modify: `crates/repo/src/repo.rs:881-891` (`diff_unified`'s protected arm) +- Modify: every `diff_worktree` caller (grep `diff_worktree(` — status, dirty-tree preflights in switch/merge/rebase, `harvest_workspace` at `workspace.rs:85`, ws probes) to pass the new parameter. + +**Interfaces:** +- Consumes: `RANDOMIZED` (Task 1), `ProtectedCache::unchanged` (Task 3). +- Produces: `diff_worktree(layout, store, head_root, protection, sparse, cache: Option<&ProtectedCache>)` — the new trailing parameter. Main-tree callers pass `Some(&cache)` where they already have a `Repo` (open via `cache::local_key(self.layout())` + `ProtectedCache::open(key, Some(self.layout().protected_cache_path()))`); `harvest_workspace` passes the workspace cache its caller threads in (Task 7 wires ws/`sc work`; until then pass `None` there). + +Behavior for a `PROTECTED` HEAD entry with disk bytes present: +- `RANDOMIZED` bit clear → today's convergent re-encrypt-and-compare (unchanged code). +- `RANDOMIZED` bit set → `cache.unchanged(path, abs, bytes)`: `Some(id)` equal to the HEAD id ⇒ clean; `Some(other)`/`None`/no cache ⇒ modified (spurious-but-safe). + +- [ ] **Step 1: Write the failing test** in `worktree.rs`'s `mod tests` (adapt to the file's helpers — `tmp_objects` exists at `worktree.rs:672`): + +```rust +#[test] +fn diff_dispatches_randomized_entries_through_the_cache() { + let (layout, mut store) = tmp_objects("p33-diff"); + // Build a HEAD tree with one RANDOMIZED protected entry whose plaintext + // sits on disk, exactly as an authorized checkout leaves it. + let pt = b"top secret".to_vec(); + let (cipher, _dek) = scl_crypto::encrypt_path_randomized(&pt); + let blob = Object::blob(cipher.clone()); + let blob_id = blob.id(); + store.put(blob).unwrap(); + let root = write_tree_with_perms_into( + &mut store, + &[("secret/x".to_string(), cipher, FileMode::FILE, PROTECTED | scl_core::RANDOMIZED)], + ); + let abs = layout.root.join("secret/x"); + std::fs::create_dir_all(abs.parent().unwrap()).unwrap(); + std::fs::write(&abs, &pt).unwrap(); + + let key = crate::cache::local_key(&layout).unwrap(); + let mut cache = crate::cache::ProtectedCache::open(key, None); + + // No cache entry: spurious-but-safe MODIFIED. + let d = diff_worktree(&layout, &mut store, Some(root), &Protection::default(), + &Sparse::default(), Some(&cache)).unwrap(); + assert_eq!(d.modified, vec!["secret/x"]); + + // Recorded: provably clean. + cache.record("secret/x", &abs, &pt, blob_id); + let d = diff_worktree(&layout, &mut store, Some(root), &Protection::default(), + &Sparse::default(), Some(&cache)).unwrap(); + assert!(d.modified.is_empty() && d.added.is_empty() && d.deleted.is_empty()); + + // Genuine edit: modified again (tag mismatch). + std::fs::write(&abs, b"edited").unwrap(); + let d = diff_worktree(&layout, &mut store, Some(root), &Protection::default(), + &Sparse::default(), Some(&cache)).unwrap(); + assert_eq!(d.modified, vec!["secret/x"]); + + std::fs::remove_dir_all(&layout.root).unwrap(); + assert!(!layout.root.exists()); +} +``` + +(If no `write_tree_with_perms_into` helper exists in these tests, build the tree the way the file's existing protected-diff tests do — grep `PROTECTED` in `worktree.rs`'s test mod and copy that construction.) + +- [ ] **Step 2: Run to verify it fails** + +Run: `cargo test -p scl-repo diff_dispatches_randomized -- --nocapture` +Expected: FAIL to compile (extra argument). + +- [ ] **Step 3: Implement.** In `diff_worktree`, add the parameter and replace the protected arm (`worktree.rs:501-506`): + +```rust +let disk_id = if perms & PROTECTED != 0 { + if perms & scl_core::RANDOMIZED == 0 { + // Convergent (legacy) entry: re-encryption yields the same id + // as the commit did — still exact, no cache needed. + Object::blob(scl_crypto::encrypt_path(bytes).0).id() + } else { + // Randomized entry (P33): re-encrypting proves nothing. The + // stat cache (stat hit, then keyed-tag fallback) is the only + // public-key-free unchanged proof; a missing entry reports + // modified — spurious-but-safe, never incorrect. + let abs = safe_join(&layout.root, p)?; + match cache.and_then(|c| c.unchanged(p, &abs, bytes)) { + Some(id) => id, + None => { + diff.modified.push(p.clone()); + continue; + } + } + } +} else { + Object::blob(bytes.clone()).id() +}; +``` + +Extend the doc comment's protected-path paragraph to describe the two formats. In `diff_unified` (`repo.rs:881-891`), apply the same dispatch: convergent → existing compare; randomized → open the main-tree cache (`cache::local_key` + `ProtectedCache::open(key, Some(self.layout().protected_cache_path()))` at the top of the function) and report `protected file changed: {path} (content not shown)` when `unchanged` misses or disagrees. Update all `diff_worktree` callers: `Repo` methods construct the main-tree cache; `workspace.rs:85` passes `None` for now (Task 7 threads the real one). + +- [ ] **Step 4: Run the full crate suite** + +Run: `cargo test -p scl-repo` +Expected: PASS — no existing test regresses (no `RANDOMIZED` entries exist yet anywhere else, so every existing path takes the convergent arm). + +- [ ] **Step 5: Commit** + +```bash +git add -A crates/repo +git commit -m "P33: format-dispatched unchanged detection in status/diff" +``` + +--- + +### Task 5: Format-dispatched carry in `snapshot_files` (commit path) + +**Files:** +- Modify: `crates/repo/src/repo.rs:339-360` (the plain/protected split + encrypt block inside `snapshot_files`) and the function signature at `repo.rs:256` +- Modify: all `snapshot_files` callers (grep `snapshot_files(` — `commit`, `amend`, `assemble_completion_snapshot`, `harvest_workspace` at `workspace.rs:100`, and any replay-side callers) to pass the new parameter. + +**Interfaces:** +- Consumes: `RANDOMIZED`, `ProtectedCache` (Tasks 1, 3). +- Produces: `snapshot_files(..., sparse: &Sparse, cache: Option<&mut ProtectedCache>, author, message)` — one new parameter, threaded explicitly like `sparse` (the P24 anti-ambient discipline). Main-tree callers open + pass the main cache and call `cache.save()` after the commit succeeds; `harvest_workspace` threads its caller's cache (Task 7). + +Per-protected-path commit rule (the spec §3 table): +1. Prior tip entry convergent → convergent-compare; equal ⇒ **carry** prior ciphertext + perms verbatim (stays convergent, no cache needed); else seal. +2. Prior tip entry randomized → `cache.unchanged` hit matching the prior blob id ⇒ carry; else seal. +3. No prior entry ⇒ seal. + +Carried entries reuse the existing absent-path carry mechanics (fetch ciphertext from store, push with source perms, `fresh_wrapped.entry(blob_id).or_insert_with(prior wraps)`). Sealed entries go through `encrypt_protected` exactly as today. After sealing, `record` each sealed path's new blob id (recompute `Object::blob(cipher).id()` from the returned entries) and each carried randomized path (refresh stat) into the cache. + +- [ ] **Step 1: Write the failing test** in `repo.rs`'s `mod tests`, using the file's existing persistent-repo + protect helpers (grep `fn protect` / `sc protect`-style tests, e.g. ones that call `repo.protect(...)` then `repo.commit(...)`; mirror their setup): + +```rust +#[test] +fn commit_carries_unchanged_convergent_content_without_reseal() { + // A pre-P33 store: protect + commit mints CONVERGENT entries (until + // Task 6 flips the seal, new commits still mint convergent too — this + // test pins the carry rule either way by asserting blob-id stability). + let (repo, _tmp) = test_repo_with_identity("p33-carry"); // adapt to existing helper + write_file(&repo, "secret/db.txt", b"hunter2"); + repo.protect("secret/", &[recipient_pk()]).unwrap(); + let c1 = repo.commit("alice", "one").unwrap(); + let id1 = blob_id_at(&repo, c1, "secret/db.txt"); + + // Unrelated edit; protected file untouched. + write_file(&repo, "readme.md", b"hello"); + let c2 = repo.commit("alice", "two").unwrap(); + let id2 = blob_id_at(&repo, c2, "secret/db.txt"); + assert_eq!(id1, id2, "unchanged protected content must carry, not re-seal"); +} +``` + +Add a second test for the randomized arm — build a tip whose protected entry is `PROTECTED | RANDOMIZED` (seal with `encrypt_path_randomized`, write the tree via the low-level helpers as in Task 4's test), materialize the plaintext, record it in the main cache, run `commit`, and assert the blob id is carried on a cache hit and re-sealed (different id, still decryptable) after the cache file is deleted: + +```rust +#[test] +fn commit_randomized_carry_via_cache_and_reseal_on_lost_cache() { + // ...setup as described above... + let c2 = repo.commit("alice", "unrelated").unwrap(); + assert_eq!(blob_id_at(&repo, c2, "secret/x"), randomized_id, "cache hit carries"); + + std::fs::remove_file(repo.layout().protected_cache_path()).unwrap(); + write_file(&repo, "readme.md", b"more"); + let c3 = repo.commit("alice", "unrelated2").unwrap(); + let id3 = blob_id_at(&repo, c3, "secret/x"); + assert_ne!(id3, randomized_id, "lost cache degrades to a spurious re-seal"); + // Never incorrectness: the new blob still decrypts to the same plaintext. + assert_eq!(decrypt_at(&repo, c3, "secret/x", &identity()), b"top secret"); +} +``` + +- [ ] **Step 2: Run to verify they fail** + +Run: `cargo test -p scl-repo commit_carries_unchanged commit_randomized_carry -- --nocapture` +Expected: FAIL (compile error on the new param, then assertion). + +- [ ] **Step 3: Implement.** Inside `snapshot_files`, after the plain/protected split (`repo.rs:341-348`) and the scanner gate, insert the dispatch before `encrypt_protected`: + +```rust +// P33 format-dispatched carry: decide per protected path whether the tip +// already holds this exact content (carry the prior ciphertext id — quiet +// history) or it must be sealed fresh (randomized). Convergent priors use +// the exact re-encrypt-and-compare proof; randomized priors use the local +// stat cache (stat hit, then keyed tag). This dispatch is what makes the +// upgrade non-mandatory: unchanged legacy content stays convergent forever. +let tip_entries: BTreeMap = match tip { + Some(t) => { + let store_arc = self.vfs.store(); + let mut store = store_arc.lock().unwrap(); + let parent_root = store.get_snapshot(&t)?.root; + if partial { + worktree::tree_file_entries_with_perms_sparse(&mut store, parent_root, sparse)? + } else { + worktree::tree_file_entries_with_perms(&mut store, parent_root)? + } + } + None => BTreeMap::new(), +}; +let mut to_seal: Vec = Vec::new(); +let mut sealed_plaintexts: Vec<(String, Vec)> = Vec::new(); +for (path, bytes, mode, granted) in protected { + let carried = tip_entries.get(&path).and_then(|(blob_id, _m, perms)| { + if perms & scl_core::PROTECTED == 0 { + return None; + } + let unchanged = if perms & scl_core::RANDOMIZED == 0 { + Object::blob(scl_crypto::encrypt_path(&bytes).0).id() == *blob_id + } else { + let abs = worktree::safe_join(&self.layout.root, &path).ok()?; + cache + .as_deref() + .and_then(|c| c.unchanged(&path, &abs, &bytes)) + == Some(*blob_id) + }; + unchanged.then_some((*blob_id, *perms)) + }); + match carried { + Some((blob_id, perms)) => { + let store_arc = self.vfs.store(); + let mut store = store_arc.lock().unwrap(); + let cipher = match store.get(&blob_id)? { + Object::Blob(b) => b.to_vec(), + _ => { + to_seal.push((path, bytes, mode, granted)); + continue; + } + }; + all.push((path.clone(), cipher, mode, perms)); + if let Some(prior_wks) = protection.wrapped.get(&blob_id) { + fresh_wrapped + .entry(blob_id) + .or_insert_with(|| prior_wks.clone()); + } + } + None => { + sealed_plaintexts.push((path.clone(), bytes.clone())); + to_seal.push((path, bytes, mode, granted)); + } + } +} +let (protected_all, sealed_wrapped) = crate::protect::encrypt_protected(to_seal)?; +// Record every freshly sealed path (its new blob id) and refresh carried +// randomized paths in the cache; the caller saves after the commit lands. +if let Some(c) = cache.as_deref_mut() { + for (path, cipher, _mode, _perms) in &protected_all { + if let Some((_, pt)) = sealed_plaintexts.iter().find(|(p, _)| p == path) { + if let Ok(abs) = worktree::safe_join(&self.layout.root, path) { + c.record(path, &abs, pt, Object::blob(cipher.clone()).id()); + } + } + } +} +all.extend(protected_all); +fresh_wrapped.extend(sealed_wrapped); +``` + +> Anchoring notes for the implementer: `all`/`fresh_wrapped` are declared at `repo.rs:357-359` — restructure that block so `all` starts from `plain` (as today) and the dispatch above replaces the single `encrypt_protected(protected)` call. `partial`/`promisor` are computed at `repo.rs:418-419` — hoist that pair above the dispatch (it's needed earlier now). `ProtectedFile` is the existing alias used at `repo.rs:342`. The `?` inside the `and_then` closure won't compile — use explicit `match`/`if let` there instead (the code above marks the one spot with `.ok()?` inside a closure returning `Option`, which is fine; `store.get(...)?` in the outer loop is fine). +> +> **Error-path discipline:** if the commit later fails (scanner already ran; tree-write or ref-move errors), the cache was mutated in memory but `save()` never runs — entries stay stale, which the degradation rule already covers (stale = spurious re-seal next time or a stat mismatch falling to tag compare; a *recorded-but-unlanded* blob id would only be returned on a later hit if that exact plaintext seals again, in which case the id is present in the store anyway — content-addressed stores make this benign). Do NOT save the cache inside `snapshot_files`; the committing caller saves only after the ref moves. + +Thread the parameter through every caller. In `Repo::commit`/`amend`/`assemble_completion_snapshot`: + +```rust +let key = crate::cache::local_key(&self.layout)?; +let mut cache = + crate::cache::ProtectedCache::open(key, Some(self.layout.protected_cache_path())); +let id = self.snapshot_files(files, ..., sparse, Some(&mut cache), author, message)?; +// only after the ref actually moved: +cache.save()?; +``` + +`workspace.rs:100` (`harvest_workspace`) passes `None` in this task; Task 7 threads the real per-workspace cache. + +- [ ] **Step 4: Run the full crate suite** + +Run: `cargo test -p scl-repo` +Expected: PASS — existing behavior unchanged (all existing protected entries are convergent; the convergent-compare carry produces the same blob ids the re-seal used to). + +- [ ] **Step 5: Commit** + +```bash +git add -A crates/repo +git commit -m "P33: format-dispatched carry in the commit pipeline" +``` + +--- + +### Task 6: Flip `encrypt_protected` to randomized seals + +**Files:** +- Modify: `crates/repo/src/protect.rs:125-153` (`encrypt_protected`) +- Modify: any test that asserts convergent behavior of *new* seals (see Step 4) + +**Interfaces:** +- Consumes: `encrypt_path_randomized` (Task 2), `RANDOMIZED` (Task 1). +- Produces: `encrypt_protected` unchanged in signature, but every returned entry now carries perms `PROTECTED | RANDOMIZED` and a fresh random seal. This is the single choke point — its three callers (`repo.rs:359` commit [now via Task 5's `to_seal`], `repo.rs:1217` merge completion, `replay.rs:222` replay) all switch at once. + +- [ ] **Step 1: Write the failing test** in `protect.rs`'s `mod tests`: + +```rust +#[test] +fn encrypt_protected_seals_randomized() { + let (_sk, pk) = scl_crypto::generate_keypair(); + let f = |v: &[u8]| { + encrypt_protected(vec![( + "secret/x".into(), v.to_vec(), FileMode::FILE, vec![pk.to_bytes()], + )]) + .unwrap() + }; + let (a1, _) = f(b"same"); + let (a2, _) = f(b"same"); + assert_ne!(a1[0].1, a2[0].1, "two seals of one plaintext must differ (oracle closed)"); + assert_eq!(a1[0].3, scl_core::PROTECTED | scl_core::RANDOMIZED); +} +``` + +- [ ] **Step 2: Run to verify it fails** + +Run: `cargo test -p scl-repo encrypt_protected_seals_randomized -- --nocapture` +Expected: FAIL — identical ciphertext, perms == `PROTECTED`. + +- [ ] **Step 3: Implement.** In `encrypt_protected`, change two lines: + +```rust +let (blob_bytes, dek) = scl_crypto::encrypt_path_randomized(&bytes); +``` + +and + +```rust +all.push((path, blob_bytes, mode, scl_core::PROTECTED | scl_core::RANDOMIZED)); +``` + +Update the function's doc comment: "Randomized encryption (P33, ADR-0043): each file seals under a fresh random DEK+nonce…". Also update `reuse_prior_wraps`'s doc comment (`protect.rs:155-172`): prior-wrap reuse now only fires for carried (id-stable) blobs — a freshly randomized blob id is never in `prior`; the function stays for exactly the carried case. + +- [ ] **Step 4: Run the full workspace suite and fix convergence-assuming tests** + +Run: `cargo test` +Expected: the new test passes; a small set of existing tests that assert *new-seal* convergence may fail (e.g. tests asserting two commits of identical protected content share a blob id, or dedup-across-paths). For each failure, decide by the spec: +- Unchanged-content id-stability tests → should still PASS via Task 5's carry; if one fails, the carry dispatch has a bug — fix the dispatch, not the test. +- Genuine new-seal convergence assertions (same plaintext at two paths dedups; re-protect mints identical ciphertext) → the spec deliberately breaks these (accepted cost 4c). Invert/rename the assertions to pin the NEW behavior (e.g. `identical_plaintext_at_two_paths_no_longer_dedups`). + +- [ ] **Step 5: Commit** + +```bash +git add -A crates/repo +git commit -m "P33: all new protected seals are randomized (oracle closed)" +``` + +--- + +### Task 7: Cache population at materialization sites + +**Files:** +- Modify: `crates/repo/src/worktree.rs:573-664` (`materialize` — new `cache` param, record on decrypted writes) +- Modify: every `materialize` caller (grep `materialize(` in `repo.rs` [switch, merge/pick/rebase completion + aborts, sparse set/disable], `workspace.rs:51`, `ws.rs:239`) +- Modify: `crates/repo/src/workspace.rs:39-119` (`materialize_workspace` + `harvest_workspace` gain a `cache` param) +- Modify: `crates/repo/src/ws.rs` (fork opens `.sc/ws/cache-` per workspace and saves it; harvest opens/passes/saves the same; abandon/teardown removes the cache file with the workspace) +- Modify: the `sc work` session driver in `workspace.rs` (ephemeral `ProtectedCache::open(key, None)` per temp checkout, threaded fork→harvest) + +**Interfaces:** +- Consumes: `ProtectedCache::record` (Task 3). +- Produces: `materialize(layout, store, target_root, old_root, protection, identity, sparse, cache: Option<&mut ProtectedCache>)`. Inside the protected-decrypt arm (`worktree.rs:634-641`), after `std::fs::write(&full, &pt[..])?`: + +```rust +if let Some(c) = cache.as_deref_mut() { + c.record(path, &full, &pt[..], *blob_id); +} +``` + +Callers: +- `Repo::switch` / sparse set/disable / merge-pick-rebase completion materializations: open the main cache, pass `Some(&mut cache)`, `cache.save()?` after the materialize succeeds. +- `materialize_workspace` (ws fork, `ws.rs:239`): open `ProtectedCache::open(key, Some(layout.ws_cache_path(i)))`, pass it down, save after. +- `harvest_workspace` (`workspace.rs:66`): new param `cache: Option<&mut ProtectedCache>`; pass it to both `diff_worktree` (`:85`, replacing Task 4's `None`) and `snapshot_files` (`:100`, replacing Task 5's `None`). `sc ws harvest` opens the workspace's persistent cache; `sc work` threads its ephemeral one. +- `ws_abandon`/teardown: remove `.sc/ws/cache-` alongside the workspace dir. + +- [ ] **Step 1: Write the failing test** in `worktree.rs`'s `mod tests`: materialize a tree containing one `PROTECTED | RANDOMIZED` entry with a valid wrap for a test identity, passing a cache; assert the cache proves the file unchanged afterwards: + +```rust +#[test] +fn materialize_populates_the_cache_for_decrypted_files() { + // build store+tree as in Task 4's test, with a real wrap for `sk` + let key = crate::cache::local_key(&layout).unwrap(); + let mut cache = crate::cache::ProtectedCache::open(key, None); + let skipped = materialize(&layout, &mut store, root, None, &prot, Some(&sk), + &Sparse::default(), Some(&mut cache)).unwrap(); + assert!(skipped.is_empty()); + let abs = layout.root.join("secret/x"); + assert_eq!(cache.unchanged("secret/x", &abs, b"top secret"), Some(blob_id)); + // cleanup + assert gone +} +``` + +- [ ] **Step 2: Run to verify it fails** — `cargo test -p scl-repo materialize_populates` → FAIL to compile. + +- [ ] **Step 3: Implement** the parameter + record call + all caller threading described above. + +- [ ] **Step 4: Add the ws end-to-end regression** in `ws.rs`'s `mod tests` (this is the false-conflict hazard the per-workspace cache exists to prevent — mirror the structure of the existing harvest tests around `ws.rs:900-1200`): + +```rust +#[test] +fn untouched_protected_files_do_not_conflict_across_workspaces() { + // repo with a protected file, committed (now RANDOMIZED); identity known. + // fork 2 workspaces WITH identity; edit only workspace 1's PLAIN file; + // edit only workspace 2's OTHER plain file; harvest both. + // Assert: both land cleanly (no work- fallback), and the landed + // snapshots' secret/x blob id equals the base's (carried, not re-sealed). +} +``` + +Write it fully (setup helpers exist in that test mod — reuse them). + +- [ ] **Step 5: Run the full crate suite** — `cargo test -p scl-repo` → PASS. + +- [ ] **Step 6: Commit** + +```bash +git add -A crates/repo +git commit -m "P33: populate protected-cache at every plaintext materialization" +``` + +--- + +### Task 8: Cache population in `sc resolve` + +**Files:** +- Modify: `crates/repo/src/conflicts.rs` (the `resolve` write path — grep `fn resolve` / where it writes the chosen side's content to the working file) + +**Interfaces:** +- Consumes: `ProtectedCache::record`. +- Produces: after `resolve --ours|--theirs` writes a protected path's decrypted content, the main-tree cache records it (so completion carries instead of re-sealing when the chosen side's content equals a parent's). Plain paths: no cache involvement. + +- [ ] **Step 1: Write the failing test** in `conflicts.rs`'s test mod: set up a protected conflict (reuse the existing P23 test scaffolding in that file), `resolve --ours` with identity, then assert the main cache holds an entry for the path (`unchanged(path, abs, ours_plaintext)` returns the ours-side blob id). + +- [ ] **Step 2: Run to verify it fails.** + +- [ ] **Step 3: Implement:** in the resolve arm that writes decrypted protected content, open the main cache, `record(path, abs, plaintext, chosen_side_blob_id)`, `save()`. + +- [ ] **Step 4: Run** `cargo test -p scl-repo` → PASS. + +- [ ] **Step 5: Commit** — `git commit -m "P33: sc resolve records resolved protected content in the cache"`. + +--- + +### Task 9: Merge/replay semantics over mixed trees (tests + the identical-edit conflict) + +**Files:** +- Modify: `crates/repo/src/merge.rs` (tests), `crates/repo/src/replay.rs` (tests) +- Production changes expected: NONE (id fast paths and diff3-with-identity already handle it) — this task pins that claim with tests and fixes whatever falls out. + +**Interfaces:** +- Consumes: everything above. +- Produces: pinned behavior later tasks and the demo rely on. + +- [ ] **Step 1: Write the tests** (in `merge.rs`'s P15 test section, reusing `enc_file`/`tree_with_perms` at `merge.rs:765-782`; add a randomized twin of `enc_file`): + +```rust +/// Randomized twin of `enc_file` (P33). +fn enc_file_rand(prot: &mut Protection, pk: &scl_crypto::PublicKey, content: &[u8]) -> Vec { + let (cipher, dek) = scl_crypto::encrypt_path_randomized(content); + let id = Object::blob(cipher.clone()).id(); + prot.wrapped.entry(id).or_default().push(scl_crypto::wrap_dek_for(&dek, pk)); + cipher +} + +#[test] +fn identical_independent_edits_now_conflict_and_resolve_with_identity() { + // base: secret/a.txt = "v1" (randomized). ours and theirs BOTH edit it + // to "v2" — independently, so two different ciphertexts/ids. + // Assert: three_way_files WITHOUT identity reports the protected + // conflict/needs-identity outcome (match the existing content-divergent + // test's expected error/conflict shape); WITH identity it merges + // cleanly (diff3 of identical plaintexts) and the result decrypts to "v2". +} + +#[test] +fn mixed_convergent_and_randomized_tree_merges_by_id_fast_paths() { + // base holds one convergent entry (enc_file) and one randomized entry + // (enc_file_rand). ours edits only the plain file; theirs edits only the + // convergent entry. Assert the merge is clean without identity, the + // untouched randomized entry carries its exact blob id AND its + // PROTECTED|RANDOMIZED perms, and the convergent entry takes theirs' id. +} +``` + +Write both fully against the existing test patterns (`protected_id_fast_paths_need_no_identity` at `merge.rs:785` is the template — copy its structure). + +- [ ] **Step 2: Run** — `cargo test -p scl-repo identical_independent mixed_convergent -- --nocapture`. Expected: ideally PASS (the machinery is format-agnostic). If the perms byte is dropped anywhere in the merge carry (fast-path arms in `three_way_files`), fix it to carry the source entry's perms verbatim. + +- [ ] **Step 3: Add a replay twin** in `replay.rs` tests: cherry-pick a commit touching a randomized protected path onto a branch where it's unchanged (fast path, no identity) and where it diverged (needs identity). Mirror the file's existing P15 replay-protected tests. + +- [ ] **Step 4: Run the full suite** — `cargo test` → PASS. + +- [ ] **Step 5: Commit** — `git commit -m "P33: pin merge/replay behavior over mixed convergent+randomized trees"`. + +--- + +### Task 10: `sc rewrap` eager convergent→randomized upgrade + +**Files:** +- Modify: `crates/repo/src/rewrap.rs` (paths half, `rewrap.rs:125-201`, and the commit tail) +- Modify: `crates/cli/src/main.rs` (rewrap output — grep `blobs_rewrapped` for the print site) + +**Interfaces:** +- Consumes: `encrypt_path_randomized`, `RANDOMIZED`, `decrypt_path`, `ProtectedCache`. +- Produces: `RewrapReport` gains `pub blobs_resealed: usize`. Rewrap behavior per protected entry: + - Randomized entry → wrap-replacement only (existing code, ciphertext id unchanged). + - Convergent entry → decrypt with the unwrapped DEK, re-seal via `encrypt_path_randomized`, store the new blob, retarget the tree entry to the new id with perms `PROTECTED | RANDOMIZED`, fresh wraps for `granted + escrow` under the NEW id, drop the OLD id's `protection.wrapped` entry, record the path in the main-tree cache, count in `blobs_resealed`. + - The commit tail rebuilds the root tree when any blob was re-sealed (collect every tree entry's `(path, bytes, mode, perms)` — bytes from the store — apply the id/perms replacements, `self.vfs.write_tree_with_perms(&all)`), and the snapshot uses the new root. When `blobs_resealed == 0` the root is untouched (policy-only, exactly as P17 — a second rewrap converges back to tree-identical). + +- [ ] **Step 1: Write the failing test** in `rewrap.rs`'s test mod (reuse its existing scaffolding): + +```rust +#[test] +fn rewrap_upgrades_convergent_blobs_and_second_run_is_policy_only() { + // repo with a CONVERGENT protected entry at the tip (build pre-P33-style: + // seal via encrypt_path + PROTECTED perms, or reuse a fixture committed + // before Task 6 semantics via the low-level tree helpers). + let r1 = repo.rewrap(&identity, &[], &known, false).unwrap(); + assert_eq!(r1.blobs_resealed, 1); + let tip1 = repo.head_tip().unwrap().unwrap(); + // entry is now randomized, new id, decrypts to the same plaintext: + let (id1, perms1) = entry_at(&repo, tip1, "secret/x"); + assert!(perms1 & scl_core::RANDOMIZED != 0); + assert_eq!(decrypt_at(&repo, tip1, "secret/x", &identity), b"top secret"); + + let r2 = repo.rewrap(&identity, &[], &known, false).unwrap(); + assert_eq!(r2.blobs_resealed, 0, "second rewrap is policy-only again"); + if let Some(c2) = r2.commit { + let tip2 = repo.head_tip().unwrap().unwrap(); + assert_eq!(entry_at(&repo, tip2, "secret/x").0, id1, "ciphertext id stable"); + } +} +``` + +Also extend an existing skip-and-report test to cover a convergent blob the identity cannot open: it lands in `skipped`, everything else still upgrades, exit stays non-zero (existing semantics). + +- [ ] **Step 2: Run to verify it fails** — `cargo test -p scl-repo rewrap_upgrades -- --nocapture` → FAIL (`blobs_resealed` not found). + +- [ ] **Step 3: Implement.** In the paths loop after `unwrap_dek_with` succeeds (`rewrap.rs:166-172`), branch on the format: + +```rust +if perms & scl_core::RANDOMIZED == 0 { + // Convergent (pre-P33): eager upgrade. Decrypt with the DEK we just + // unwrapped, re-seal randomized, retarget tree + wraps to the new id. + let cipher = { /* store.get(&blob_id) -> Object::Blob bytes */ }; + let pt = match scl_crypto::decrypt_path(&cipher, &dek) { + Ok(p) => p, + Err(e) => { + skipped.push((format!("path {path}"), format!("ciphertext failed to open: {e}"))); + continue; + } + }; + if dry_run { blobs_resealed += 1; continue; } + let (new_cipher, new_dek) = scl_crypto::encrypt_path_randomized(&pt); + let new_id = { /* store.put(Object::blob(new_cipher.clone()))? */ }; + let mut new_wks: Vec = target_pks + .iter() + .map(|pk| scl_crypto::wrap_dek_for(&new_dek, pk)) + .collect(); + new_wks.sort_by(|a, b| a.recipient_id.cmp(&b.recipient_id)); + protection.wrapped.remove(&blob_id); + protection.wrapped.insert(new_id, new_wks); + retargets.insert(path.clone(), (new_id, scl_core::PROTECTED | scl_core::RANDOMIZED)); + main_cache.record(&path, &self.layout().root.join(&path), &pt, new_id); + blobs_resealed += 1; + continue; +} +// randomized: existing wrap-replacement code, unchanged +``` + +(`target_pks` already exists in the loop at `rewrap.rs:184-194` — hoist its construction above the branch so both arms share it. `retargets: BTreeMap` is new, declared beside `blobs_rewrapped`. `main_cache` is opened once before the loop and `save()`d only when the commit lands.) + +In the commit tail: when `retargets` is non-empty, rebuild the root — walk `entries` again, for each path fetch bytes from the store (new blob for retargeted paths), apply retargeted perms, `write_tree_with_perms`, and build the snapshot on the new root. Update the emptiness check (`rewrap.rs:206`) to also consider `blobs_resealed`. In the CLI print site, add: `re-sealed {n} convergent blob(s) → randomized` when `blobs_resealed > 0`. + +- [ ] **Step 4: Run** — `cargo test -p scl-repo rewrap` then `cargo test` → PASS. + +- [ ] **Step 5: Commit** — `git commit -m "P33: sc rewrap eagerly re-seals convergent blobs randomized"`. + +--- + +### Task 11: Demo — `demo/run_randomized_demo.sh` + +**Files:** +- Create: `demo/run_randomized_demo.sh` (copy the header/conventions of `demo/run_rewrap_demo.sh` — build once, `set -euo pipefail`, temp workdir, run-twice loop, zero-residue asserts) + +**Interfaces:** consumes the finished CLI. The demo must prove, in order: +1. **Oracle closed:** commit the same plaintext at two protected paths, and the same plaintext twice across two commits (edit away, edit back); extract blob ids via `sc log --json`/object inspection and assert all ciphertext ids differ. +2. **Quiet history:** an unrelated commit does NOT change an untouched protected file's blob id (cache carry). +3. **Dual-read:** a repo fixture with a convergent (pre-P33) entry still decrypts; `sc rewrap` upgrades it (`re-sealed 1 convergent blob(s)`), plaintext identical after; a second `sc rewrap` prints no re-seal line. +4. **Old history still decrypts** after the upgrade (check out the pre-rewrap commit, decrypt). +5. Zero `.sc/tmp` residue; the workdir is removed; run the whole script twice. + +For the convergent fixture: keep a tiny `sc`-driven pre-seed impossible post-Task-6, so build it with a rust test-utility or simply commit the fixture repo bytes… **Do neither** — instead drive the upgrade path end to end: initialize the repo with the CURRENT binary, then flip one committed protected entry to convergent form using a small hidden debug command? Also no. The honest MVP: the demo proves oracle-closure, quiet history, rewrap idempotence (steps 1, 2, second half of 3, 5) with the current binary, and **dual-read of genuinely old stores is pinned by unit tests** (Tasks 4–6, 9, 10 all build convergent fixtures via the library). State this split in a comment at the top of the demo. + +- [ ] **Step 1: Write the script** (full content, following `run_rewrap_demo.sh`'s skeleton). +- [ ] **Step 2: Run it twice** — `bash demo/run_randomized_demo.sh && bash demo/run_randomized_demo.sh` → both green, zero residue. +- [ ] **Step 3: Run every existing protected demo** — `bash demo/run_protected_merge_demo.sh && bash demo/run_revoke_demo.sh && bash demo/run_rewrap_demo.sh && bash demo/run_lifecycle_demo.sh && bash demo/run_merge_ergonomics_demo.sh && bash demo/run_ws_demo.sh` → all green. +- [ ] **Step 4: Commit** — `git commit -m "P33: randomized-encryption demo (oracle closed, quiet history, rewrap upgrade)"`. + +--- + +### Task 12: Docs — ADR-0043, THREAT-MODEL, CLAUDE.md, ARCHITECTURE.md + +**Files:** +- Create: `docs/adr/0043-randomized-protected-encryption.md` (follow the ADR template in `docs/adr/README.md` / recent ADRs' structure: Status/Date/Phase, Context, Decision, Consequences, Alternatives) +- Modify: `THREAT-MODEL.md` (the convergent-equality entry: superseded for new content, still true for pre-P33 history until rewrapped) +- Modify: `CLAUDE.md` (a "Phase 33 is built" section in the established voice: what shipped, the format-dispatch rule, the cache, accepted costs 4a–c, boundaries; update the `sc rewrap` command comment; move the P28 "randomized protected mode" follow-on out of Remaining follow-ons, add rotate-for-paths) +- Modify: `ARCHITECTURE.md` (the protected-path encryption section: randomized-write/dual-read posture) +- Modify: `docs/adr/0014-*.md` — add a one-line pointer: "Superseded for newly sealed content by ADR-0043 (P33); convergent dual-read retained." + +**Content requirements for ADR-0043** (each gets a paragraph): +- The format change (RANDOMIZED perms bit; blob layout/AAD unchanged; why the tag lives in the tree entry, not the blob). +- Dual-read posture (no tag bump, no migration; the format-dispatched commit rule and why cache-only dispatch would have mass-migrated). +- Cache design and why it doesn't reintroduce the oracle (keyed PRF under a never-traveling local key; degradation rule). +- Accepted costs 4a–c verbatim from #40, plus the rewrap tree-identity change. +- P15/P17 semantic adjustments; the ws per-workspace cache rationale (false-conflict hazard). +- Unlocked follow-on: rotate-for-paths (ADR-0019's objection dissolves for randomized content) — recorded, not built. + +- [ ] **Step 1: Write ADR-0043.** +- [ ] **Step 2: Update THREAT-MODEL.md, CLAUDE.md, ARCHITECTURE.md, ADR-0014 pointer.** +- [ ] **Step 3: Self-check** — grep CLAUDE.md for "randomized" to make sure the P28 accepted-boundary line ("convergent encryption stays equality-confirmable") is updated to point at P33. +- [ ] **Step 4: Commit** — `git commit -m "P33: ADR-0043 + THREAT-MODEL/CLAUDE.md/ARCHITECTURE.md updates"`. + +--- + +### Task 13: Full validation sweep + +- [ ] **Step 1:** `cargo test` (whole workspace) → PASS. +- [ ] **Step 2:** `cargo clippy --workspace` → no new warnings. +- [ ] **Step 3:** Run ALL demos listed in CLAUDE.md's Commands section (there are ~20 `demo/run_*.sh` scripts — run each once; protected-related ones twice). Expected: all green, zero residue. +- [ ] **Step 4:** Equality-oracle end check: `cargo test -p scl-crypto randomized_seal_roundtrips_and_closes_the_oracle -p scl-repo encrypt_protected_seals_randomized` → PASS. +- [ ] **Step 5:** Final commit if anything moved; then hand off to the finishing-a-development-branch flow (PR per repo convention — squash-merged). + +--- + +## Self-Review Notes (already applied) + +- **Spec coverage:** §1→Task 2, §2→Task 1, §3→Tasks 3+5, §4→Task 4, §5→Tasks 7+8, §6→Task 9, §7→Task 10, §9 error handling→Tasks 3/5/10 inline, §10→Tasks 11–13. The `sc work` ephemeral cache (spec §5 by implication of "per-checkout") is Task 7. +- **Ordering is load-bearing:** dispatch (4–5) before the flip (6); anything reordered mass-re-seals and breaks the suite. +- **Known adaptation points** are marked with `NOTE:` blocks (ObjectId hex API, `random_hex` contract, error variant, test-helper names) — resolve them by reading the named anchor, not by inventing new surface. From c552cb62d3a8b4d48a4730cf833d7d76f478250e Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 19:10:31 +0300 Subject: [PATCH 03/22] P33: add RANDOMIZED perms flag to scl-core Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/core/src/lib.rs | 2 +- crates/core/src/object.rs | 15 +++++++++++++++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/crates/core/src/lib.rs b/crates/core/src/lib.rs index 806f7dfc..a8da2b33 100644 --- a/crates/core/src/lib.rs +++ b/crates/core/src/lib.rs @@ -26,6 +26,6 @@ pub use error::{Error, Result}; pub use id::ObjectId; pub use object::{ EntryKind, FileMode, Object, ProtectPrefix, Protection, RecipientEntry, RecipientState, Secret, - SignatureObj, Snapshot, Transcript, Tree, TreeEntry, WrappedKey, PROTECTED, + SignatureObj, Snapshot, Transcript, Tree, TreeEntry, WrappedKey, PROTECTED, RANDOMIZED, }; pub use store::{Backend, SpillPolicy, Store, StoreConfig, StoreStats}; diff --git a/crates/core/src/object.rs b/crates/core/src/object.rs index 7c8581e3..791daa9b 100644 --- a/crates/core/src/object.rs +++ b/crates/core/src/object.rs @@ -22,6 +22,12 @@ const TAG_TRANSCRIPT: u8 = 6; /// encrypted file), not plaintext. Set on protected-path entries (P7). pub const PROTECTED: u8 = 0b0000_0001; +/// Perms flag: this PROTECTED entry was sealed with a fresh random DEK+nonce +/// (P33) rather than convergently. Always set together with `PROTECTED`. +/// Format identification for dual-read lives here, in the tree entry, so no +/// caller ever needs to fetch blob bytes to know the seal format. +pub const RANDOMIZED: u8 = 0b0000_0010; + /// A recipient's standing on a protected prefix: active or tombstoned. #[derive(Clone, Copy, PartialEq, Eq, Debug)] pub enum RecipientState { @@ -1021,4 +1027,13 @@ mod tests { assert_eq!(rule.granted_keys(), vec![[1; 32]]); assert_eq!(rule.next_epoch(), 3); } + + #[test] + fn randomized_bit_is_distinct_from_protected() { + assert_eq!(PROTECTED & RANDOMIZED, 0, "flags must not overlap"); + assert_ne!(RANDOMIZED, 0); + // A randomized entry is always also protected. + let perms = PROTECTED | RANDOMIZED; + assert!(perms & PROTECTED != 0 && perms & RANDOMIZED != 0); + } } From 7c9cdaf9388affe242d327b18ded479adcfc07c3 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 19:14:01 +0300 Subject: [PATCH 04/22] P33: randomized path seal (fresh DEK+nonce, dual-read layout) Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/crypto/src/envelope.rs | 64 +++++++++++++++++++++++++++++++++-- crates/crypto/src/lib.rs | 3 +- 2 files changed, 64 insertions(+), 3 deletions(-) diff --git a/crates/crypto/src/envelope.rs b/crates/crypto/src/envelope.rs index 12e47894..8ace9d8a 100644 --- a/crates/crypto/src/envelope.rs +++ b/crates/crypto/src/envelope.rs @@ -136,8 +136,10 @@ const PATH_AAD: &[u8] = b"scl-path-v1"; /// Convergent file encryption: the data key and nonce are derived from the /// plaintext, so identical plaintext yields identical `nonce‖ciphertext` bytes -/// (stable content-addressed id, perfect dedup). Returns the blob bytes and the -/// DEK (to be wrapped per recipient and stored in the snapshot policy). +/// (stable content-addressed id, perfect dedup). Legacy convergent format — kept +/// for dual-read comparison only; new seals use `encrypt_path_randomized` (P33). +/// Returns the blob bytes and the DEK (to be wrapped per recipient and stored +/// in the snapshot policy). pub fn encrypt_path(plaintext: &[u8]) -> (Vec, Zeroizing<[u8; DEK_LEN]>) { let hash = blake3::hash(plaintext); let mut ikm = Zeroizing::new([0u8; 32]); @@ -167,6 +169,42 @@ pub fn encrypt_path(plaintext: &[u8]) -> (Vec, Zeroizing<[u8; DEK_LEN]>) { (blob, dek) } +/// Randomized file encryption (P33): fresh random DEK + random nonce per +/// seal, so identical plaintext yields different ciphertext every time — +/// the ADR-0014 equality oracle is closed for content sealed this way. +/// Blob layout and AAD are identical to `encrypt_path` (`nonce ‖ ct`, +/// `PATH_AAD`), so `decrypt_path` opens both formats unchanged (dual-read). +pub fn encrypt_path_randomized(plaintext: &[u8]) -> (Vec, Zeroizing<[u8; DEK_LEN]>) { + encrypt_path_randomized_with_rng(plaintext, &mut OsRng) +} + +/// `encrypt_path_randomized` with a caller-supplied RNG (deterministic in tests). +pub fn encrypt_path_randomized_with_rng( + plaintext: &[u8], + rng: &mut R, +) -> (Vec, Zeroizing<[u8; DEK_LEN]>) { + let mut dek = Zeroizing::new([0u8; DEK_LEN]); + rng.fill_bytes(dek.as_mut_slice()); + let mut nonce = [0u8; NONCE_LEN]; + rng.fill_bytes(&mut nonce); + + let cipher = XChaCha20Poly1305::new_from_slice(dek.as_slice()).expect("32-byte DEK"); + let ciphertext = cipher + .encrypt( + XNonce::from_slice(&nonce), + Payload { + msg: plaintext, + aad: PATH_AAD, + }, + ) + .expect("aead encrypt is infallible for valid inputs"); + + let mut blob = Vec::with_capacity(NONCE_LEN + ciphertext.len()); + blob.extend_from_slice(&nonce); + blob.extend_from_slice(&ciphertext); + (blob, dek) +} + /// Decrypt a `nonce‖ciphertext` blob with its DEK (AEAD-verified). pub fn decrypt_path(blob: &[u8], dek: &[u8; DEK_LEN]) -> Result>> { if blob.len() < NONCE_LEN { @@ -403,4 +441,26 @@ mod tests { let (other_sk, _) = generate_keypair_with_rng(&mut ChaCha20Rng::seed_from_u64(2)); assert!(unwrap_dek_with(&wk, &other_sk).is_err()); } + + #[test] + fn randomized_seal_roundtrips_and_closes_the_oracle() { + let pt = b"the database password is hunter2"; + let (blob1, dek1) = encrypt_path_randomized(pt); + let (blob2, dek2) = encrypt_path_randomized(pt); + // THE phase invariant: same plaintext, different ciphertext bytes. + assert_ne!(blob1, blob2, "equality oracle must be closed"); + assert_ne!(dek1.as_slice(), dek2.as_slice()); + // Dual-read: the unchanged decrypt_path opens a randomized blob. + assert_eq!(&decrypt_path(&blob1, &dek1).unwrap()[..], pt); + assert_eq!(&decrypt_path(&blob2, &dek2).unwrap()[..], pt); + } + + #[test] + fn randomized_seal_is_deterministic_under_a_seeded_rng() { + let mut r1 = rng(7); + let mut r2 = rng(7); + let (b1, _) = encrypt_path_randomized_with_rng(b"x", &mut r1); + let (b2, _) = encrypt_path_randomized_with_rng(b"x", &mut r2); + assert_eq!(b1, b2); + } } diff --git a/crates/crypto/src/lib.rs b/crates/crypto/src/lib.rs index ed03e286..d3e15d60 100644 --- a/crates/crypto/src/lib.rs +++ b/crates/crypto/src/lib.rs @@ -16,7 +16,8 @@ pub mod provider; pub mod signing; pub use envelope::{ - decrypt_path, encrypt_path, open, revoke, rewrap_for, seal, unwrap_dek_with, wrap_dek_for, + decrypt_path, encrypt_path, encrypt_path_randomized, encrypt_path_randomized_with_rng, open, + revoke, rewrap_for, seal, unwrap_dek_with, wrap_dek_for, }; pub use error::{Error, Result}; pub use key::{generate_keypair, PublicKey, RecipientId, SecretKey}; From fc5df13a9036fa8ab13588db860cd1af96db0297 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 19:19:44 +0300 Subject: [PATCH 05/22] P33: per-checkout protected-path stat cache + local key Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- Cargo.lock | 1 + crates/repo/Cargo.toml | 1 + crates/repo/src/cache.rs | 251 ++++++++++++++++++++++++++++++++++++++ crates/repo/src/layout.rs | 18 +++ crates/repo/src/lib.rs | 1 + 5 files changed, 272 insertions(+) create mode 100644 crates/repo/src/cache.rs diff --git a/Cargo.lock b/Cargo.lock index 5f2424dc..d56ad998 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2315,6 +2315,7 @@ dependencies = [ name = "scl-repo" version = "0.1.0" dependencies = [ + "blake3", "hex", "libc", "regex", diff --git a/crates/repo/Cargo.toml b/crates/repo/Cargo.toml index 471aa181..d30fa3b3 100644 --- a/crates/repo/Cargo.toml +++ b/crates/repo/Cargo.toml @@ -16,6 +16,7 @@ serde = { version = "1", features = ["derive"] } libc = "0.2.186" scl-tlsio = { version = "0.1.0", path = "../tlsio" } hex = "0.4.3" +blake3 = "1.8.5" [lints] workspace = true diff --git a/crates/repo/src/cache.rs b/crates/repo/src/cache.rs new file mode 100644 index 00000000..dff092a9 --- /dev/null +++ b/crates/repo/src/cache.rs @@ -0,0 +1,251 @@ +//! P33: local unchanged-detection cache for randomized protected paths. +//! +//! Randomized seals (ADR-0043) make re-encrypt-and-compare useless for +//! change detection, so commit/status/diff consult this `.sc`-local, +//! never-committed, never-transferred map instead: +//! `path -> (mtime_ns, size, keyed_tag, ciphertext blob id)`. +//! The tag is `blake3::keyed_hash(local_key, plaintext)` under a random +//! per-repo key (`.sc/local-key`, 0600), so the cache file alone leaks +//! nothing — it does not reintroduce the equality oracle this phase closes. +//! A lost/stale/corrupt cache degrades to spurious re-seals, never +//! incorrectness and never a hard error. + +use std::collections::BTreeMap; +use std::path::{Path, PathBuf}; +use std::str::FromStr; + +use scl_core::ObjectId; + +use crate::error::{Error, Result}; +use crate::layout::Layout; + +/// Load (minting if absent) the per-repo random cache key. The key file is +/// the only hard-error surface here: if it can't be created/read, the cache +/// cannot be safely keyed, so surface it rather than running unkeyed. +pub(crate) fn local_key(layout: &Layout) -> Result<[u8; 32]> { + let path = layout.local_key_path(); + let hex_str = match std::fs::read_to_string(&path) { + Ok(s) => s.trim().to_string(), + Err(e) if e.kind() == std::io::ErrorKind::NotFound => { + let s = scl_crypto::random_hex(32); // 32 random bytes -> 64 hex chars + // Mirror the serve-TLS key discipline: 0600 before content matters. + std::fs::write(&path, &s)?; + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o600))?; + } + s + } + Err(e) => return Err(e.into()), + }; + let bytes = hex::decode(&hex_str) + .map_err(|_| Error::InvalidArgument("malformed .sc/local-key".into()))?; + bytes + .try_into() + .map_err(|_| Error::InvalidArgument("malformed .sc/local-key".into())) +} + +#[derive(Clone, Copy, Debug, PartialEq)] +struct CacheEntry { + mtime_ns: u128, + size: u64, + tag: [u8; 32], + blob_id: ObjectId, +} + +/// Per-checkout unchanged-detection cache (main tree, one ws workspace, or +/// an ephemeral in-memory one for `sc work` temp checkouts). +pub(crate) struct ProtectedCache { + key: [u8; 32], + /// `None` => ephemeral: `save()` is a no-op. + path: Option, + entries: BTreeMap, +} + +impl ProtectedCache { + pub(crate) fn open(key: [u8; 32], path: Option) -> ProtectedCache { + let mut entries = BTreeMap::new(); + if let Some(p) = &path { + if let Ok(text) = std::fs::read_to_string(p) { + for line in text.lines() { + // Format: ` ` + // (path last: it may contain spaces). + let mut it = line.splitn(5, ' '); + let parsed = (|| { + let mtime_ns: u128 = it.next()?.parse().ok()?; + let size: u64 = it.next()?.parse().ok()?; + let tag: [u8; 32] = hex::decode(it.next()?).ok()?.try_into().ok()?; + let blob_id = ObjectId::from_str(it.next()?).ok()?; + Some((it.next()?.to_string(), CacheEntry { mtime_ns, size, tag, blob_id })) + })(); + match parsed { + Some((path, e)) => { + entries.insert(path, e); + } + None => { + eprintln!("warning: ignoring corrupt protected-cache line"); + entries.clear(); + break; // degrade to empty: spurious re-seals, never incorrectness + } + } + } + } + } + ProtectedCache { key, path, entries } + } + + fn tag(&self, plaintext: &[u8]) -> [u8; 32] { + *blake3::keyed_hash(&self.key, plaintext).as_bytes() + } + + fn stat(abs: &Path) -> Option<(u128, u64)> { + let md = std::fs::metadata(abs).ok()?; + let mtime = md.modified().ok()?; + let ns = mtime.duration_since(std::time::UNIX_EPOCH).ok()?.as_nanos(); + Some((ns, md.len())) + } + + /// The cached ciphertext blob id iff this exact plaintext is what last + /// sealed at `rel`: stat hit (mtime+size unchanged) short-circuits; + /// otherwise fall back to the keyed-tag comparison. + pub(crate) fn unchanged(&self, rel: &str, abs: &Path, plaintext: &[u8]) -> Option { + let e = self.entries.get(rel)?; + if let Some((mtime_ns, size)) = Self::stat(abs) { + if mtime_ns == e.mtime_ns && size == e.size { + return Some(e.blob_id); + } + } + (self.tag(plaintext) == e.tag).then_some(e.blob_id) + } + + /// Record that `plaintext` at `rel` seals to `blob_id`. Missing stat + /// (file vanished mid-operation) just skips the entry. + pub(crate) fn record(&mut self, rel: &str, abs: &Path, plaintext: &[u8], blob_id: ObjectId) { + if let Some((mtime_ns, size)) = Self::stat(abs) { + let tag = self.tag(plaintext); + self.entries + .insert(rel.to_string(), CacheEntry { mtime_ns, size, tag, blob_id }); + } + } + + /// Atomic write (temp sibling + rename); no-op for an ephemeral cache. + pub(crate) fn save(&self) -> Result<()> { + let Some(path) = &self.path else { return Ok(()) }; + let mut out = String::new(); + for (p, e) in &self.entries { + out.push_str(&format!( + "{} {} {} {} {}\n", + e.mtime_ns, + e.size, + hex::encode(e.tag), + e.blob_id.to_hex(), + p + )); + } + if let Some(parent) = path.parent() { + std::fs::create_dir_all(parent)?; + } + let tmp = path.with_extension(format!("tmp-{}", std::process::id())); + std::fs::write(&tmp, out)?; + std::fs::rename(&tmp, path)?; + Ok(()) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn tmp(tag: &str) -> Layout { + let root = std::env::temp_dir().join(format!("scl-cache-{tag}-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&root); + std::fs::create_dir_all(root.join(".sc")).unwrap(); + Layout::at(&root) + } + + fn cleanup(layout: &Layout) { + std::fs::remove_dir_all(&layout.root).unwrap(); + assert!(!layout.root.exists()); + } + + #[test] + fn local_key_is_minted_once_and_stable() { + let layout = tmp("key"); + let k1 = local_key(&layout).unwrap(); + let k2 = local_key(&layout).unwrap(); + assert_eq!(k1, k2, "second read must return the same key"); + assert_ne!(k1, [0u8; 32]); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + let mode = std::fs::metadata(layout.local_key_path()) + .unwrap() + .permissions() + .mode(); + assert_eq!(mode & 0o777, 0o600, "key file must be 0600"); + } + cleanup(&layout); + } + + #[test] + fn stat_hit_tag_fallback_and_miss() { + let layout = tmp("hits"); + let key = local_key(&layout).unwrap(); + let abs = layout.root.join("secret.txt"); + std::fs::write(&abs, b"v1").unwrap(); + let id = ObjectId::of(b"cipher-of-v1"); + + let mut c = ProtectedCache::open(key, Some(layout.protected_cache_path())); + assert_eq!(c.unchanged("secret.txt", &abs, b"v1"), None, "empty cache misses"); + c.record("secret.txt", &abs, b"v1", id); + + // Stat hit: file untouched. + assert_eq!(c.unchanged("secret.txt", &abs, b"v1"), Some(id)); + + // Tag fallback: touch mtime without changing content. + std::fs::write(&abs, b"v1").unwrap(); + assert_eq!(c.unchanged("secret.txt", &abs, b"v1"), Some(id)); + + // Real change: miss. + std::fs::write(&abs, b"v2").unwrap(); + assert_eq!(c.unchanged("secret.txt", &abs, b"v2"), None); + cleanup(&layout); + } + + #[test] + fn save_load_roundtrip_and_corrupt_file_is_empty() { + let layout = tmp("persist"); + let key = local_key(&layout).unwrap(); + let abs = layout.root.join("a b/spaced name.txt"); // path with spaces + std::fs::create_dir_all(abs.parent().unwrap()).unwrap(); + std::fs::write(&abs, b"v").unwrap(); + let id = ObjectId::of(b"c"); + + let mut c = ProtectedCache::open(key, Some(layout.protected_cache_path())); + c.record("a b/spaced name.txt", &abs, b"v", id); + c.save().unwrap(); + + let c2 = ProtectedCache::open(key, Some(layout.protected_cache_path())); + assert_eq!(c2.unchanged("a b/spaced name.txt", &abs, b"v"), Some(id)); + + // Corrupt file: degrades to empty, never errors. + std::fs::write(layout.protected_cache_path(), b"garbage\nlines\n").unwrap(); + let c3 = ProtectedCache::open(key, Some(layout.protected_cache_path())); + assert_eq!(c3.unchanged("a b/spaced name.txt", &abs, b"v"), None); + cleanup(&layout); + } + + #[test] + fn ephemeral_cache_never_touches_disk() { + let layout = tmp("ephemeral"); + let key = local_key(&layout).unwrap(); + let abs = layout.root.join("f"); + std::fs::write(&abs, b"v").unwrap(); + let mut c = ProtectedCache::open(key, None); + c.record("f", &abs, b"v", ObjectId::of(b"c")); + c.save().unwrap(); + assert!(!layout.protected_cache_path().exists()); + cleanup(&layout); + } +} diff --git a/crates/repo/src/layout.rs b/crates/repo/src/layout.rs index 25acf17d..fecc32d0 100644 --- a/crates/repo/src/layout.rs +++ b/crates/repo/src/layout.rs @@ -110,4 +110,22 @@ impl Layout { pub fn serve_tokens_path(&self) -> PathBuf { self.dot_sc.join("serve-tokens.toml") } + + /// `.sc/local-key` — per-repo random key for the P33 unchanged-detection + /// cache's keyed hashes. Never committed, never transferred. + pub fn local_key_path(&self) -> PathBuf { + self.dot_sc.join("local-key") + } + + /// `.sc/protected-cache` — the main working tree's P33 stat cache. + pub fn protected_cache_path(&self) -> PathBuf { + self.dot_sc.join("protected-cache") + } + + /// `.sc/ws/cache-` — workspace `i`'s P33 stat cache. Lives BESIDE the + /// checkout dir (`.sc/ws//`), never inside it, so harvest's worktree + /// read can't pick it up as an untracked file. + pub fn ws_cache_path(&self, i: usize) -> PathBuf { + self.dot_sc.join("ws").join(format!("cache-{i}")) + } } diff --git a/crates/repo/src/lib.rs b/crates/repo/src/lib.rs index 5095a5df..14152ca2 100644 --- a/crates/repo/src/lib.rs +++ b/crates/repo/src/lib.rs @@ -1,6 +1,7 @@ //! `scl-repo` — the durable on-disk repository: `.sc/` layout, refs/HEAD, //! named branches, a git-like working tree, and commit/secret orchestration. +pub(crate) mod cache; pub(crate) mod conflicts; pub mod diff3; pub mod error; From c81085ade3ec1f222844ef5cadade4dbdedaa099 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 19:25:33 +0300 Subject: [PATCH 06/22] P33: harden cache local-key perms + reuse atomic_write_durable local_key() now creates .sc/local-key 0600 from the first byte via create_new (mirroring crates/tlsio/src/identity.rs), closing the write-then-chmod window, and falls through to reading the winner's file on a concurrent create_new race instead of erroring or clobbering it. ProtectedCache::save() drops its hand-rolled temp+rename in favor of scl_core::fsutil::atomic_write_durable, the same fsync-backed helper every other .sc-local writer in this crate uses. Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/repo/src/cache.rs | 51 +++++++++++++++++++++++++++++++--------- 1 file changed, 40 insertions(+), 11 deletions(-) diff --git a/crates/repo/src/cache.rs b/crates/repo/src/cache.rs index dff092a9..a534d924 100644 --- a/crates/repo/src/cache.rs +++ b/crates/repo/src/cache.rs @@ -22,20 +22,27 @@ use crate::layout::Layout; /// Load (minting if absent) the per-repo random cache key. The key file is /// the only hard-error surface here: if it can't be created/read, the cache /// cannot be safely keyed, so surface it rather than running unkeyed. +/// +/// Mirrors the serve-TLS key discipline (`crates/tlsio/src/identity.rs`): +/// the file is created 0600 from its first byte via `create_new`, so there +/// is never a window where the key sits at default permissions. A +/// concurrent minter losing the `create_new` race falls through to reading +/// the winner's file rather than erroring or clobbering it. pub(crate) fn local_key(layout: &Layout) -> Result<[u8; 32]> { let path = layout.local_key_path(); let hex_str = match std::fs::read_to_string(&path) { Ok(s) => s.trim().to_string(), Err(e) if e.kind() == std::io::ErrorKind::NotFound => { let s = scl_crypto::random_hex(32); // 32 random bytes -> 64 hex chars - // Mirror the serve-TLS key discipline: 0600 before content matters. - std::fs::write(&path, &s)?; - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o600))?; + match write_key_0600(&path, s.as_bytes()) { + Ok(()) => s, + Err(e) if e.kind() == std::io::ErrorKind::AlreadyExists => { + // Lost the create_new race to a concurrent minter: read + // their file instead of clobbering it. + std::fs::read_to_string(&path)?.trim().to_string() + } + Err(e) => return Err(e.into()), } - s } Err(e) => return Err(e.into()), }; @@ -46,6 +53,28 @@ pub(crate) fn local_key(layout: &Layout) -> Result<[u8; 32]> { .map_err(|_| Error::InvalidArgument("malformed .sc/local-key".into())) } +#[cfg(unix)] +fn write_key_0600(path: &Path, bytes: &[u8]) -> std::io::Result<()> { + use std::io::Write; + use std::os::unix::fs::OpenOptionsExt; + let mut f = std::fs::OpenOptions::new() + .write(true) + .create_new(true) + .mode(0o600) + .open(path)?; + f.write_all(bytes) +} + +#[cfg(not(unix))] +fn write_key_0600(path: &Path, bytes: &[u8]) -> std::io::Result<()> { + use std::io::Write; + let mut f = std::fs::OpenOptions::new() + .write(true) + .create_new(true) + .open(path)?; + f.write_all(bytes) +} + #[derive(Clone, Copy, Debug, PartialEq)] struct CacheEntry { mtime_ns: u128, @@ -129,7 +158,9 @@ impl ProtectedCache { } } - /// Atomic write (temp sibling + rename); no-op for an ephemeral cache. + /// Durable atomic write via the shared `fsutil` helper (fsync temp + + /// rename + fsync parent dir, no residue on failure); no-op for an + /// ephemeral cache. pub(crate) fn save(&self) -> Result<()> { let Some(path) = &self.path else { return Ok(()) }; let mut out = String::new(); @@ -146,9 +177,7 @@ impl ProtectedCache { if let Some(parent) = path.parent() { std::fs::create_dir_all(parent)?; } - let tmp = path.with_extension(format!("tmp-{}", std::process::id())); - std::fs::write(&tmp, out)?; - std::fs::rename(&tmp, path)?; + scl_core::fsutil::atomic_write_durable(path, out.as_bytes())?; Ok(()) } } From d005b05b8951d4cd38de856ee7992158ae8c0846 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 19:33:02 +0300 Subject: [PATCH 07/22] P33: format-dispatched unchanged detection in status/diff Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/repo/src/repo.rs | 27 ++++++++- crates/repo/src/workspace.rs | 13 +++- crates/repo/src/worktree.rs | 114 +++++++++++++++++++++++++++++++++-- crates/repo/src/ws.rs | 13 +++- 4 files changed, 156 insertions(+), 11 deletions(-) diff --git a/crates/repo/src/repo.rs b/crates/repo/src/repo.rs index 52ba5e02..7a7a4258 100644 --- a/crates/repo/src/repo.rs +++ b/crates/repo/src/repo.rs @@ -810,12 +810,15 @@ impl Repo { }; let store_arc = self.vfs.store(); let mut store = store_arc.lock().unwrap(); + let key = crate::cache::local_key(&self.layout)?; + let cache = crate::cache::ProtectedCache::open(key, Some(self.layout.protected_cache_path())); worktree::diff_worktree( &self.layout, &mut store, head_root, &protection, &self.sparse_spec()?, + Some(&cache), ) } @@ -850,6 +853,14 @@ impl Repo { .map(|t| self.snapshot(&t)) .transpose()? .map(|s| s.root); + // Randomized (P33) protected entries need the local unchanged-cache to + // report clean at all (re-encryption proves nothing for them). A + // failure to load/mint `.sc/local-key` degrades to `None` here rather + // than failing the whole diff — spurious-but-safe: every randomized + // entry just reports "changed" instead of the diff erroring out. + let cache = crate::cache::local_key(&self.layout) + .ok() + .map(|key| crate::cache::ProtectedCache::open(key, Some(self.layout.protected_cache_path()))); let store_arc = self.vfs.store(); let mut store = store_arc.lock().unwrap(); let head = match head_root { @@ -881,8 +892,20 @@ impl Repo { Some((blob_id, _mode, perms)) if *perms & PROTECTED != 0 => { // Never show protected content; report the change status only. if let Some(bytes) = disk { - let disk_id = Object::blob(scl_crypto::encrypt_path(bytes).0).id(); - if disk_id != *blob_id { + let changed = if *perms & scl_core::RANDOMIZED == 0 { + // Convergent (legacy): re-encryption yields an exact + // compare, unchanged from pre-P33. + Object::blob(scl_crypto::encrypt_path(bytes).0).id() != *blob_id + } else { + // Randomized (P33): re-encryption proves nothing; the + // cache is the only public-key-free unchanged proof. + // A miss/disagreement/absent cache reports changed — + // spurious-but-safe, never incorrect. + let abs = worktree::safe_join(&self.layout.root, path)?; + cache.as_ref().and_then(|c| c.unchanged(path, &abs, bytes)) + != Some(*blob_id) + }; + if changed { out.push_str(&format!( "protected file changed: {path} (content not shown)\n" )); diff --git a/crates/repo/src/workspace.rs b/crates/repo/src/workspace.rs index 9f33f885..d922f9ff 100644 --- a/crates/repo/src/workspace.rs +++ b/crates/repo/src/workspace.rs @@ -81,8 +81,17 @@ pub(crate) fn harvest_workspace( worktree::tree_file_ids(&mut store, snap.root)? .into_keys() .collect(); - let d = - worktree::diff_worktree(&ws, &mut store, Some(snap.root), &snap.protection, sparse)?; + // No workspace-local cache is threaded in yet (P33 Task 7 wires it); + // until then a randomized protected path in a workspace always + // reports modified — spurious-but-safe. + let d = worktree::diff_worktree( + &ws, + &mut store, + Some(snap.root), + &snap.protection, + sparse, + None, + )?; ( tracked, !(d.added.is_empty() && d.modified.is_empty() && d.deleted.is_empty()), diff --git a/crates/repo/src/worktree.rs b/crates/repo/src/worktree.rs index bdade9c7..a3b93ade 100644 --- a/crates/repo/src/worktree.rs +++ b/crates/repo/src/worktree.rs @@ -5,6 +5,7 @@ use std::path::Path; use scl_core::{EntryKind, FileMode, Object, ObjectId, Protection, Store, Tree, PROTECTED}; +use crate::cache::ProtectedCache; use crate::error::{Error, Result}; use crate::ignore::Ignore; use crate::layout::Layout; @@ -449,9 +450,16 @@ pub struct Diff { /// path: /// - absent on disk => CLEAN (the expected state for an unauthorized/skipped /// checkout — not a user deletion); -/// - present on disk => re-encrypt the disk bytes convergently (`encrypt_path` -/// is a pure, keyless function) and compare the resulting ciphertext blob id -/// to the stored one: equal => CLEAN, different => MODIFIED (a genuine edit). +/// - present on disk, format dispatches on the `RANDOMIZED` bit (P33): +/// - clear (convergent/legacy) => re-encrypt the disk bytes convergently +/// (`encrypt_path` is a pure, keyless function) and compare the +/// resulting ciphertext blob id to the stored one: equal => CLEAN, +/// different => MODIFIED (a genuine edit). Unchanged from pre-P33. +/// - set (randomized) => re-encryption proves nothing (a fresh seal never +/// matches the old ciphertext even for identical plaintext), so the only +/// public-key-free unchanged proof is the local stat cache +/// (`cache::unchanged`, Task 3): a hit reports CLEAN, a miss/disagreement +/// reports MODIFIED — spurious-but-safe, never incorrect. /// /// Non-protected paths use the usual plaintext-blob-id comparison. /// @@ -483,6 +491,7 @@ pub fn diff_worktree( // a future richer diff (e.g. prefix-aware reporting) needs no signature change. _protection: &Protection, sparse: &Sparse, + cache: Option<&ProtectedCache>, ) -> Result { let head = match head_root { Some(root) => tree_file_entries_with_perms_sparse(store, root, sparse)?, @@ -499,8 +508,24 @@ pub fn diff_worktree( None => diff.added.push(p.clone()), Some((hid, _mode, perms)) => { let disk_id = if perms & PROTECTED != 0 { - // Convergent re-encryption yields the same id as the commit did. - Object::blob(scl_crypto::encrypt_path(bytes).0).id() + if perms & scl_core::RANDOMIZED == 0 { + // Convergent (legacy) entry: re-encryption yields the same + // id as the commit did — still exact, no cache needed. + Object::blob(scl_crypto::encrypt_path(bytes).0).id() + } else { + // Randomized entry (P33): re-encrypting proves nothing. The + // stat cache (stat hit, then keyed-tag fallback) is the only + // public-key-free unchanged proof; a missing entry reports + // modified — spurious-but-safe, never incorrect. + let abs = safe_join(&layout.root, p)?; + match cache.and_then(|c| c.unchanged(p, &abs, bytes)) { + Some(id) => id, + None => { + diff.modified.push(p.clone()); + continue; + } + } + } } else { Object::blob(bytes.clone()).id() }; @@ -817,4 +842,83 @@ mod tests { drop(store); std::fs::remove_dir_all(&layout.root).unwrap(); } + + #[test] + fn diff_dispatches_randomized_entries_through_the_cache() { + let (layout, mut store) = tmp_objects("p33-diff"); + // Build a HEAD tree with one RANDOMIZED protected entry whose plaintext + // sits on disk, exactly as an authorized checkout leaves it. + let pt = b"top secret".to_vec(); + let (cipher, _dek) = scl_crypto::encrypt_path_randomized(&pt); + let blob = Object::blob(cipher.clone()); + let blob_id = blob.id(); + store.put(blob).unwrap(); + let inner = store + .put(Object::Tree(Tree::new(vec![TreeEntry { + name: "x".into(), + kind: EntryKind::Blob, + id: blob_id, + mode: FileMode::FILE, + perms: PROTECTED | scl_core::RANDOMIZED, + }]))) + .unwrap(); + let root = store + .put(Object::Tree(Tree::new(vec![TreeEntry { + name: "secret".into(), + kind: EntryKind::Tree, + id: inner, + mode: FileMode::FILE, + perms: 0, + }]))) + .unwrap(); + + let abs = layout.root.join("secret/x"); + std::fs::create_dir_all(abs.parent().unwrap()).unwrap(); + std::fs::write(&abs, &pt).unwrap(); + + let key = crate::cache::local_key(&layout).unwrap(); + let mut cache = crate::cache::ProtectedCache::open(key, None); + + // No cache entry: spurious-but-safe MODIFIED. + let d = diff_worktree( + &layout, + &mut store, + Some(root), + &Protection::default(), + &Sparse::default(), + Some(&cache), + ) + .unwrap(); + assert_eq!(d.modified, vec!["secret/x"]); + + // Recorded: provably clean. + cache.record("secret/x", &abs, &pt, blob_id); + let d = diff_worktree( + &layout, + &mut store, + Some(root), + &Protection::default(), + &Sparse::default(), + Some(&cache), + ) + .unwrap(); + assert!(d.modified.is_empty() && d.added.is_empty() && d.deleted.is_empty()); + + // Genuine edit: modified again (tag mismatch). + std::fs::write(&abs, b"edited").unwrap(); + let d = diff_worktree( + &layout, + &mut store, + Some(root), + &Protection::default(), + &Sparse::default(), + Some(&cache), + ) + .unwrap(); + assert_eq!(d.modified, vec!["secret/x"]); + + drop(store); + std::fs::remove_dir_all(&layout.root).unwrap(); + assert!(!layout.root.exists()); + } } diff --git a/crates/repo/src/ws.rs b/crates/repo/src/ws.rs index bee1231e..878778d7 100644 --- a/crates/repo/src/ws.rs +++ b/crates/repo/src/ws.rs @@ -297,8 +297,17 @@ impl Repo { let sparse = crate::sparse::Sparse::new(session.sparse.clone()); let store_arc = self.vfs().store(); let mut store = store_arc.lock().unwrap(); - let d = - worktree::diff_worktree(&ws, &mut store, Some(base.root), &base.protection, &sparse)?; + // No workspace-local cache is threaded in yet (P33 Task 7 wires it); + // until then a randomized protected path in a workspace always + // reports modified — spurious-but-safe. + let d = worktree::diff_worktree( + &ws, + &mut store, + Some(base.root), + &base.protection, + &sparse, + None, + )?; Ok(!(d.added.is_empty() && d.modified.is_empty() && d.deleted.is_empty())) } From 53a53da355e0ec354081ef8b861da4b6ed25ea37 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 19:56:53 +0300 Subject: [PATCH 08/22] P33: format-dispatched carry in the commit pipeline Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/repo/src/repo.rs | 372 ++++++++++++++++++++++++++++++++--- crates/repo/src/workspace.rs | 7 + 2 files changed, 351 insertions(+), 28 deletions(-) diff --git a/crates/repo/src/repo.rs b/crates/repo/src/repo.rs index 7a7a4258..58e1521e 100644 --- a/crates/repo/src/repo.rs +++ b/crates/repo/src/repo.rs @@ -191,7 +191,16 @@ impl Repo { message: &str, ) -> Result { let files = worktree::read_worktree(&self.layout, &self.tracked_paths_at(Some(parent))?)?; - self.snapshot_files( + // Pick/rebase completion opens its own main cache. It saves right + // after `snapshot_files` (before the caller moves the ref): the sealed + // blobs are already in the store via `write_tree_with_perms`, so a + // recorded-but-unlanded id is benign (the cache degradation contract), + // and both callers (`commit`'s pick arm, `rebase_continue`) move the + // ref immediately after a successful return. + let key = crate::cache::local_key(&self.layout)?; + let mut cache = + crate::cache::ProtectedCache::open(key, Some(self.layout.protected_cache_path())); + let id = self.snapshot_files( files, Some(parent), None, @@ -200,9 +209,12 @@ impl Repo { pick_registry_base, None, &self.sparse_spec()?, + Some(&mut cache), author, message, - ) + )?; + cache.save()?; + Ok(id) } /// The commit pipeline minus ref movement: split protected/plaintext files, @@ -263,6 +275,7 @@ impl Repo { pick_registry_base: Option, parents_override: Option>, sparse: &crate::sparse::Sparse, + mut cache: Option<&mut crate::cache::ProtectedCache>, author: &str, message: &str, ) -> Result { @@ -353,11 +366,151 @@ impl Repo { return Err(Error::SecretDetected(report)); } - // Encrypt protected files; accumulate fresh wrapped DEKs keyed by blob id. + // Plaintext files land first (perms 0); protected files are decided + // per-path below (carry vs. seal) and appended. let mut all: Vec<(String, Vec, scl_core::FileMode, u8)> = plain.into_iter().map(|(p, b, m)| (p, b, m, 0u8)).collect(); - let (protected_all, mut fresh_wrapped) = crate::protect::encrypt_protected(protected)?; + let mut fresh_wrapped: BTreeMap> = BTreeMap::new(); + + // Partial-clone context is needed here now (the P33 dispatch flattens + // the tip tree, which must stay gap-tolerant on a partial clone) and + // again in the absent-path carry block below — computed once. + let promisor = self.promisor()?; + let partial = promisor.is_some(); + // Merge/pick completion guard (P27 Task 5, T5-I4): `graft_out_of_sparse` + // below only splices out-of-filter subtrees back in for a plain + // single-tip commit (`decided_root.is_none() && merge_head.is_none()`). + // A merge (`merge_head: Some`) or a CONFLICTED pick/rebase completion + // (`decided_root: Some`) builds `all`/`root` purely from in-filter + // content on a partial clone — with no graft to follow, that would + // silently DROP every out-of-filter subtree from the new snapshot. + // Refuse loudly instead of ever letting that happen; a clean + // (non-conflicted) pick/rebase fold still has `decided_root: None` + // and keeps using the single-tip graft path, unaffected. + if partial && (decided_root.is_some() || merge_head.is_some()) { + return Err(crate::promisor::partial_clone_unsupported( + "merge/pick completion", + )); + } + + // P33 format-dispatched carry (ADR-0043): decide per protected path + // whether the tip already holds this exact content — carry the prior + // ciphertext id verbatim (quiet history, no reseal) — or it must be + // sealed fresh. A CONVERGENT prior (no `RANDOMIZED` bit) uses the exact + // re-encrypt-and-compare proof and needs no cache: this is what keeps + // the format upgrade non-mandatory (unchanged legacy content stays + // convergent forever). A RANDOMIZED prior can't be compared by + // re-encrypting (a fresh seal differs every time), so it consults the + // `.sc`-local stat/tag cache; a lost/stale cache degrades to a spurious + // reseal, never to incorrectness. Everything else seals. + let tip_entries: BTreeMap = match tip { + Some(t) => { + let store_arc = self.vfs.store(); + let mut store = store_arc.lock().unwrap(); + let parent_root = store.get_snapshot(&t)?.root; + if partial { + worktree::tree_file_entries_with_perms_sparse(&mut store, parent_root, sparse)? + } else { + worktree::tree_file_entries_with_perms(&mut store, parent_root)? + } + } + None => BTreeMap::new(), + }; + + let mut to_seal: Vec = Vec::new(); + // (path, plaintext) for each sealed path, so its fresh blob id can be + // recorded into the cache after `encrypt_protected` returns. + let mut sealed_plaintexts: Vec<(String, Vec)> = Vec::new(); + for (path, bytes, mode, granted) in protected { + // Decide carry (Some(prior blob id + perms)) vs. seal (None). + let carry: Option<(ObjectId, u8)> = match tip_entries.get(&path) { + Some((blob_id, _m, perms)) if perms & scl_core::PROTECTED != 0 => { + let unchanged = if perms & scl_core::RANDOMIZED == 0 { + // Convergent prior: exact re-encrypt-and-compare. + Object::blob(scl_crypto::encrypt_path(&bytes).0).id() == *blob_id + } else { + // Randomized prior: the local stat/tag cache is the only + // way to prove "same plaintext". Carry iff the cached id + // matches the prior tip id (double-checked: same content + // AND same recorded seal). + match worktree::safe_join(&self.layout.root, &path) { + Ok(abs) => { + cache.as_deref().and_then(|c| c.unchanged(&path, &abs, &bytes)) + == Some(*blob_id) + } + Err(_) => false, + } + }; + if unchanged { + Some((*blob_id, *perms)) + } else { + None + } + } + _ => None, + }; + + match carry { + Some((blob_id, perms)) => { + // Carry the prior ciphertext + perms verbatim (same + // mechanics as the absent-path carry below). + let cipher = { + let store_arc = self.vfs.store(); + let mut store = store_arc.lock().unwrap(); + match store.get(&blob_id)? { + Object::Blob(b) => Some(b.to_vec()), + _ => None, + } + }; + match cipher { + Some(cipher) => { + all.push((path.clone(), cipher, mode, perms)); + if let Some(prior_wks) = protection.wrapped.get(&blob_id) { + fresh_wrapped + .entry(blob_id) + .or_insert_with(|| prior_wks.clone()); + } + // Refresh a carried randomized path's cache entry so + // the next commit gets a fast stat hit (optimization + // only; the caller persists after the ref moves). + if perms & scl_core::RANDOMIZED != 0 { + if let (Some(c), Ok(abs)) = ( + cache.as_deref_mut(), + worktree::safe_join(&self.layout.root, &path), + ) { + c.record(&path, &abs, &bytes, blob_id); + } + } + } + // Prior blob unexpectedly absent (e.g. not a blob) — + // fall back to sealing rather than dropping content. + None => { + sealed_plaintexts.push((path.clone(), bytes.clone())); + to_seal.push((path, bytes, mode, granted)); + } + } + } + None => { + sealed_plaintexts.push((path.clone(), bytes.clone())); + to_seal.push((path, bytes, mode, granted)); + } + } + } + + let (protected_all, sealed_wrapped) = crate::protect::encrypt_protected(to_seal)?; + // Record every freshly sealed path's new blob id into the cache; the + // committing caller persists it only after the ref actually moves. + if let Some(c) = cache.as_deref_mut() { + for (path, cipher, _mode, _perms) in &protected_all { + if let Some((_, pt)) = sealed_plaintexts.iter().find(|(p, _)| p == path) { + if let Ok(abs) = worktree::safe_join(&self.layout.root, path) { + c.record(path, &abs, pt, Object::blob(cipher.clone()).id()); + } + } + } + } all.extend(protected_all); + fresh_wrapped.extend(sealed_wrapped); // Safe-by-default: carry forward still-protected files that are absent // from the working tree. `commit` cannot distinguish "absent because the @@ -415,23 +568,8 @@ impl Repo { // with an active sparse spec still has every object, and its // per-blob byte-carry below (the existing P24 mechanism) must stay // exactly as it was to avoid any behavior change for that case. - let promisor = self.promisor()?; - let partial = promisor.is_some(); - // Merge/pick completion guard (P27 Task 5, T5-I4): `graft_out_of_sparse` - // below only splices out-of-filter subtrees back in for a plain - // single-tip commit (`decided_root.is_none() && merge_head.is_none()`). - // A merge (`merge_head: Some`) or a CONFLICTED pick/rebase completion - // (`decided_root: Some`) builds `all`/`root` purely from in-filter - // content on a partial clone — with no graft to follow, that would - // silently DROP every out-of-filter subtree from the new snapshot. - // Refuse loudly instead of ever letting that happen; a clean - // (non-conflicted) pick/rebase fold still has `decided_root: None` - // and keeps using the single-tip graft path, unaffected. - if partial && (decided_root.is_some() || merge_head.is_some()) { - return Err(crate::promisor::partial_clone_unsupported( - "merge/pick completion", - )); - } + // (`promisor`/`partial` and the merge/pick-completion partial-clone + // guard were hoisted above the P33 dispatch, which needs them too.) { let mut on_disk: std::collections::BTreeSet = all.iter().map(|(p, _, _, _)| p.clone()).collect(); @@ -555,8 +693,24 @@ impl Repo { // `wrap_dek_for` randomizes its ephemeral key — re-wrapping every commit // would change the `protection` encoding (and thus the snapshot id) for // identical content, breaking "same content -> stable history". Carrying the - // prior wrap forward keeps it stable; only a newly-added recipient (or a new - // blob) gets a fresh wrap, and a revoked recipient is already absent here. + // prior wrap forward keeps it stable. + // + // P33 wrap-lifecycle note: a SEALED blob (new or edited content) wraps + // for exactly `granted_keys()`, so a revoked recipient is absent from + // its fresh wraps as before. A CARRIED blob (P33 unchanged-content + // dispatch above), however, keeps the tip's prior wrap list VERBATIM — + // it never re-derives from `granted_keys()`. That matches how the P15 + // absent-path carry has always behaved, and it is correct: `revoke` + // itself eagerly strips the revoked wrap from the tip + // (`protect_ops::revoke`), so an ordinary post-revoke commit on + // unchanged content carries the already-stripped list. The one place + // carry diverges from the old always-reseal behavior is the ADR-0026 + // corollary — merging a pre-revoke branch re-attaches a revoked + // recipient's old wraps as historical facts; the old reseal stripped + // them back to `granted_keys()` on the next commit, whereas carry + // preserves them on unchanged content. This is not a new disclosure + // (the recipient already holds the key; the ciphertext is unchanged) + // and `sc rewrap` remains the deliberate cutover, per ADR-0027. let prior = std::mem::take(&mut protection.wrapped); crate::protect::reuse_prior_wraps(&mut fresh_wrapped, &prior); protection.wrapped = fresh_wrapped; @@ -617,9 +771,14 @@ impl Repo { let merging = merge_head.is_some(); let picking = pick_head.is_some(); // Pick completion (no merge in progress) is the extracted assembly - // (P19 Task 2 groundwork), shared with `rebase_continue`. Every - // other case (plain commit, merge completion) still calls - // `snapshot_files` directly with a freshly read working tree. + // (P19 Task 2 groundwork), shared with `rebase_continue`; it manages + // its own protected-path cache internally. Every other case (plain + // commit, merge completion) calls `snapshot_files` directly with a + // freshly read working tree and a cache this function opens here and + // saves only after the ref moves below — the P33 discipline. `None` + // in the pick arm avoids double-managing (and clobbering) the cache + // `assemble_completion_snapshot` already saved. + let mut cache: Option = None; let id = match (tip, merge_head, pick_head) { (Some(t), None, Some(ph)) => self.assemble_completion_snapshot( t, @@ -631,7 +790,12 @@ impl Repo { )?, _ => { let files = worktree::read_worktree(&self.layout, &self.tracked_paths()?)?; - self.snapshot_files( + let key = crate::cache::local_key(&self.layout)?; + let mut c = crate::cache::ProtectedCache::open( + key, + Some(self.layout.protected_cache_path()), + ); + let id = self.snapshot_files( files, tip, merge_head, @@ -640,14 +804,22 @@ impl Repo { pick_registry_base, None, &self.sparse_spec()?, + Some(&mut c), author, message, - )? + )?; + cache = Some(c); + id } }; refs::write_branch_tip(&self.layout, &head, &id)?; crate::merge_state::clear(&self.layout)?; crate::pick_state::clear(&self.layout)?; + // The ref moved: persist the cache (no-op in the pick arm, which + // manages its own). + if let Some(c) = cache { + c.save()?; + } let label = if merging { "commit (merge)" } else if picking { @@ -702,6 +874,9 @@ impl Repo { let before = refs::read_branch_tip(&self.layout, &head)?; let files = worktree::read_worktree(&self.layout, &self.tracked_paths()?)?; + let key = crate::cache::local_key(&self.layout)?; + let mut cache = + crate::cache::ProtectedCache::open(key, Some(self.layout.protected_cache_path())); let id = self.snapshot_files( files, Some(tip), @@ -711,11 +886,13 @@ impl Repo { None, Some(tip_snap.parents.clone()), &self.sparse_spec()?, + Some(&mut cache), author, &msg, )?; refs::write_branch_tip(&self.layout, &head, &id)?; + cache.save()?; crate::oplog::record( &self.layout, "amend", @@ -3470,6 +3647,144 @@ mod tests { std::fs::remove_dir_all(&root).unwrap(); } + // ---- P33 Task 5: format-dispatched carry in the commit pipeline ---- + + /// The ciphertext blob id recorded for `path` in `commit`'s tree. + fn p33_entry_id(repo: &Repo, commit: &ObjectId, path: &str) -> ObjectId { + let store_arc = repo.vfs.store(); + let mut s = store_arc.lock().unwrap(); + let root = s.get_snapshot(commit).unwrap().root; + let entries = worktree::tree_file_entries_with_perms(&mut s, root).unwrap(); + entries[path].0 + } + + fn p33_blob_bytes(repo: &Repo, id: &ObjectId) -> Vec { + let store_arc = repo.vfs.store(); + let mut s = store_arc.lock().unwrap(); + match s.get(id).unwrap() { + Object::Blob(b) => b.to_vec(), + _ => panic!("expected Blob"), + } + } + + #[test] + fn commit_carries_unchanged_convergent_content_without_reseal() { + // A pre-P33-style store: protect + commit mints a CONVERGENT protected + // entry. This pins the carry rule by blob-id stability across an + // unrelated commit — the protected file is never re-touched, so it + // must carry (not re-seal). (Under Task 5 carry and re-seal both yield + // the same convergent id, so this is a regression guard that survives + // the Task 6 seal flip precisely because unchanged content carries.) + let root = tmp_root("p33-conv-carry"); + let repo = Repo::init(&root).unwrap(); + let (_alice_sk, alice_pk) = scl_crypto::generate_keypair(); + repo.protect("secret/", &[alice_pk], None).unwrap(); + std::fs::create_dir_all(root.join("secret")).unwrap(); + std::fs::write(root.join("secret/db.txt"), b"hunter2").unwrap(); + let c1 = repo.commit("alice", "one").unwrap(); + let id1 = p33_entry_id(&repo, &c1, "secret/db.txt"); + + // Unrelated edit; the protected file is untouched on disk. + std::fs::write(root.join("readme.md"), b"hello").unwrap(); + let c2 = repo.commit("alice", "two").unwrap(); + let id2 = p33_entry_id(&repo, &c2, "secret/db.txt"); + assert_eq!(id1, id2, "unchanged protected content must carry, not re-seal"); + + drop(repo); + std::fs::remove_dir_all(&root).unwrap(); + } + + #[test] + fn commit_randomized_carry_via_cache_and_reseal_on_lost_cache() { + // Build a tip whose protected entry is `PROTECTED | RANDOMIZED` (as a + // post-Task-6 authorized checkout leaves it), record the plaintext in + // the on-disk cache, and prove: (a) a cache hit CARRIES the randomized + // blob id verbatim; (b) losing the cache degrades to a spurious + // re-seal (different id) — never to incorrectness (still decrypts). + let root = tmp_root("p33-rand-carry"); + let repo = Repo::init(&root).unwrap(); + let (alice_sk, alice_pk) = scl_crypto::generate_keypair(); + repo.protect("secret/", std::slice::from_ref(&alice_pk), None) + .unwrap(); + + // Seal a randomized protected entry and install it as the branch tip, + // keeping the `secret/` rule from the protect commit so the next + // commit routes `secret/x` through the protected split, not `plain`. + let pt = b"top secret".to_vec(); + let (cipher, dek) = scl_crypto::encrypt_path_randomized(&pt); + let randomized_id = Object::blob(cipher.clone()).id(); + let wks = vec![scl_crypto::wrap_dek_for(&dek, &alice_pk)]; + + let tip = repo.head_tip().unwrap().unwrap(); + let tip_snap = repo.snapshot(&tip).unwrap(); + let mut protection = tip_snap.protection.clone(); + protection.wrapped.insert(randomized_id, wks); + let tree = repo + .vfs + .write_tree_with_perms(&[( + "secret/x".to_string(), + cipher.clone(), + scl_core::FileMode::FILE, + scl_core::PROTECTED | scl_core::RANDOMIZED, + )]) + .unwrap(); + let rand_tip = repo + .commit_snapshot( + tree, + vec![tip], + tip_snap.secrets.clone(), + protection, + "system", + "randomized tip", + ) + .unwrap(); + assert_eq!(repo.head_tip().unwrap(), Some(rand_tip)); + + // Materialize the plaintext on disk and record it in the on-disk cache. + std::fs::create_dir_all(root.join("secret")).unwrap(); + std::fs::write(root.join("secret/x"), &pt).unwrap(); + { + let key = crate::cache::local_key(repo.layout()).unwrap(); + let mut cache = crate::cache::ProtectedCache::open( + key, + Some(repo.layout().protected_cache_path()), + ); + cache.record("secret/x", &root.join("secret/x"), &pt, randomized_id); + cache.save().unwrap(); + } + + // Unrelated edit; the randomized protected file is untouched. + std::fs::write(root.join("readme.md"), b"hello").unwrap(); + let c2 = repo.commit("alice", "unrelated").unwrap(); + assert_eq!( + p33_entry_id(&repo, &c2, "secret/x"), + randomized_id, + "cache hit carries" + ); + + // Lose the cache: the next commit cannot prove unchanged, so it re-seals. + std::fs::remove_file(repo.layout().protected_cache_path()).unwrap(); + std::fs::write(root.join("readme.md"), b"more").unwrap(); + let c3 = repo.commit("alice", "unrelated2").unwrap(); + let id3 = p33_entry_id(&repo, &c3, "secret/x"); + assert_ne!(id3, randomized_id, "lost cache degrades to a spurious re-seal"); + // Never incorrectness: the re-sealed blob still decrypts to the plaintext. + let bytes = p33_blob_bytes(&repo, &id3); + let snap3 = repo.snapshot(&c3).unwrap(); + let got = crate::protect::decrypt_with( + &bytes, + &id3, + &[&snap3.protection], + &alice_sk, + "secret/x", + ) + .unwrap(); + assert_eq!(&got[..], b"top secret"); + + drop(repo); + std::fs::remove_dir_all(&root).unwrap(); + } + #[test] fn protect_grant_and_revoke_append_oplog_records() { let root = tmp_root("oplog-protect"); @@ -5595,6 +5910,7 @@ mod tests { None, None, &dst.sparse_spec().unwrap(), + None, "t", "msg", ) diff --git a/crates/repo/src/workspace.rs b/crates/repo/src/workspace.rs index d922f9ff..a1c1b7e6 100644 --- a/crates/repo/src/workspace.rs +++ b/crates/repo/src/workspace.rs @@ -106,6 +106,12 @@ pub(crate) fn harvest_workspace( // fixed at materialize time (or, for `sc work`, is always full), so the // carry predicate must see that same view, not the host repo's current // `.sc/sparse` — see `snapshot_files`'s doc comment. + // No workspace-local cache is threaded in yet (P33 Task 7 wires it): pass + // `None`, so a randomized protected path in a workspace always re-seals + // (spurious-but-safe). Passing the HOST repo's cache here would be WRONG — + // the abs-path base in `snapshot_files` is the host root, but these files + // came from the workspace dir. Convergent priors still carry (content-only + // compare, no cache needed). match repo.snapshot_files( files, Some(tip), @@ -115,6 +121,7 @@ pub(crate) fn harvest_workspace( None, None, sparse, + None, author, message, ) { From cdde49bd859121f94f2bd7ff8fe222a59a6eb51d Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 20:31:18 +0300 Subject: [PATCH 09/22] P33: all new protected seals are randomized (oracle closed) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Flips encrypt_protected (crates/repo/src/protect.rs) to scl_crypto::encrypt_path_randomized, tagging every fresh seal PROTECTED | RANDOMIZED. This is the single choke point all three callers (commit's to_seal, merge completion, replay) go through. Flipping it surfaced that worktree::materialize never wrote to the P33 ProtectedCache: Task 3/4 wired the cache's read side into diff_worktree and commit's carry decision, but the write side was never wired into checkout/switch/merge/rebase materialization. Invisible pre-flip (convergent seals self-verify by re-encrypting), this became a real bug the moment seals stopped being convergent — a freshly checked-out, byte-identical randomized protected file had no cache entry, so status/diff spuriously reported it modified, which several callers (merge, rebase, sparse set/disable) treat as a hard "uncommitted changes" precondition failure. Fixed by threading a cache: Option<&mut ProtectedCache> through materialize (worktree.rs), recording each RANDOMIZED entry right after its plaintext write, and updating all 15 call sites across repo.rs, replay.rs, sparse.rs, oplog.rs, sync.rs, and workspace.rs (a new Repo::open_protected_cache() helper opens/saves the host-repo cache at each site that has an identity to decrypt with; sites with no identity pass None since the record call is unreachable there anyway). Full workspace suite: 14 of 15 failures were this same dispatch bug (fixed, no test changes); 1 was a genuine new-seal-convergence assertion the spec deliberately breaks (accepted cost 4c) — convergent_merge_ids_are_stable_across_repos renamed/inverted to merge_ids_no_longer_converge_across_repos, citing ADR-0043/P33. Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/repo/src/oplog.rs | 1 + crates/repo/src/protect.rs | 42 ++++++++++++++++++----- crates/repo/src/replay.rs | 20 +++++++++-- crates/repo/src/repo.rs | 65 ++++++++++++++++++++++++++++++------ crates/repo/src/sparse.rs | 16 ++++++--- crates/repo/src/sync.rs | 1 + crates/repo/src/workspace.rs | 7 ++++ crates/repo/src/worktree.rs | 19 ++++++++++- 8 files changed, 145 insertions(+), 26 deletions(-) diff --git a/crates/repo/src/oplog.rs b/crates/repo/src/oplog.rs index c69eaa7b..a783df5b 100644 --- a/crates/repo/src/oplog.rs +++ b/crates/repo/src/oplog.rs @@ -428,6 +428,7 @@ impl Repo { &protection, None, &self.sparse_spec()?, + None, )? } else { Vec::new() diff --git a/crates/repo/src/protect.rs b/crates/repo/src/protect.rs index 3dc3b8ae..acd4b863 100644 --- a/crates/repo/src/protect.rs +++ b/crates/repo/src/protect.rs @@ -116,12 +116,16 @@ pub(crate) fn decrypt_with( Err(Error::NotAuthorized(path.to_string())) } -/// Convergently encrypt `plaintexts` (path, bytes, mode, rule recipients), -/// wrapping each fresh DEK to its recipients. Returns the ciphertext write-set -/// entries (PROTECTED perms) and fresh wraps keyed by ciphertext blob id. -/// Errors `InvalidArgument` if any file's recipient list is empty: a rule whose -/// every recipient is tombstoned (e.g. crossed revokes merged together) must fail -/// the seal loudly — sealing to nobody mints permanently unreadable ciphertext. +/// Randomized encryption (P33, ADR-0043): each file seals under a fresh +/// random DEK+nonce, wrapping that DEK to its rule's recipients. Returns the +/// ciphertext write-set entries (PROTECTED | RANDOMIZED perms) and fresh +/// wraps keyed by ciphertext blob id. Because the seal is randomized, the +/// blob id is fresh on every call — identical plaintext no longer dedups +/// across paths or across re-seals of the same content; that oracle-closing +/// trade is deliberate (accepted cost 4c). Errors `InvalidArgument` if any +/// file's recipient list is empty: a rule whose every recipient is +/// tombstoned (e.g. crossed revokes merged together) must fail the seal +/// loudly — sealing to nobody mints permanently unreadable ciphertext. pub(crate) fn encrypt_protected( plaintexts: Vec<(String, Vec, FileMode, Vec<[u8; 32]>)>, ) -> Result<( @@ -138,7 +142,7 @@ pub(crate) fn encrypt_protected( (all revoked?); run `sc grant` before committing under this prefix" ))); } - let (blob_bytes, dek) = scl_crypto::encrypt_path(&bytes); + let (blob_bytes, dek) = scl_crypto::encrypt_path_randomized(&bytes); // Build the blob object once for its id; `write_tree_with_perms` does // the (idempotent) store insert below — no second explicit `put`. let blob_id = Object::blob(blob_bytes.clone()).id(); @@ -147,7 +151,7 @@ pub(crate) fn encrypt_protected( .map(|pk| scl_crypto::wrap_dek_for(&dek, &scl_crypto::PublicKey::from_bytes(*pk))) .collect(); fresh_wrapped.insert(blob_id, wks); - all.push((path, blob_bytes, mode, scl_core::PROTECTED)); + all.push((path, blob_bytes, mode, scl_core::PROTECTED | scl_core::RANDOMIZED)); } Ok((all, fresh_wrapped)) } @@ -155,6 +159,10 @@ pub(crate) fn encrypt_protected( /// Prior-wrap reuse: for each (blob_id, recipient_id) already wrapped in /// `prior`, keep the prior wrap bytes so unchanged content's protection /// encoding (and thus snapshot ids) stays stable. Mutates `fresh` in place. +/// Since P33 (ADR-0043), `encrypt_protected`'s seals are randomized, so a +/// freshly sealed blob's id is never a key in `prior` — this only fires for +/// the carried, id-stable case (Task 5's carry-forward of unchanged +/// protected content), never for a genuinely new seal. pub(crate) fn reuse_prior_wraps( fresh: &mut BTreeMap>, prior: &BTreeMap>, @@ -175,6 +183,24 @@ pub(crate) fn reuse_prior_wraps( mod tests { use super::*; + #[test] + fn encrypt_protected_seals_randomized() { + let (_sk, pk) = scl_crypto::generate_keypair(); + let f = |v: &[u8]| { + encrypt_protected(vec![( + "secret/x".into(), + v.to_vec(), + FileMode::FILE, + vec![pk.to_bytes()], + )]) + .unwrap() + }; + let (a1, _) = f(b"same"); + let (a2, _) = f(b"same"); + assert_ne!(a1[0].1, a2[0].1, "two seals of one plaintext must differ (oracle closed)"); + assert_eq!(a1[0].3, scl_core::PROTECTED | scl_core::RANDOMIZED); + } + fn prot(prefixes: &[&str]) -> Protection { Protection { prefixes: prefixes diff --git a/crates/repo/src/replay.rs b/crates/repo/src/replay.rs index e6a403fd..19ca032a 100644 --- a/crates/repo/src/replay.rs +++ b/crates/repo/src/replay.rs @@ -471,6 +471,7 @@ impl Repo { // Protection-aware materialize: a PROTECTED entry decrypts // for `identity` when possible, else is skipped (never // writes ciphertext to disk) — same as merge's clean path. + let mut cache = self.open_protected_cache()?; worktree::materialize( &self.layout, &mut store, @@ -479,7 +480,9 @@ impl Repo { &protection, identity, &self.sparse_spec()?, + Some(&mut cache), )?; + cache.save()?; } refs::write_branch_tip(&self.layout, &head, &id)?; crate::oplog::record( @@ -590,6 +593,7 @@ impl Repo { let snap = self.snapshot(&tip)?; let store_arc = self.vfs().store(); let mut store = store_arc.lock().unwrap(); + let mut cache = self.open_protected_cache()?; skipped = worktree::materialize( &self.layout, &mut store, @@ -598,7 +602,9 @@ impl Repo { &snap.protection, None, &self.sparse_spec()?, + Some(&mut cache), )?; + cache.save()?; } crate::pick_state::clear(&self.layout)?; Ok(skipped) @@ -697,6 +703,7 @@ impl Repo { let target_snap = store.get_snapshot(&target_tip)?; let target_root = target_snap.root; let target_protection = target_snap.protection; + let mut cache = self.open_protected_cache()?; worktree::materialize( &self.layout, &mut store, @@ -705,7 +712,9 @@ impl Repo { &target_protection, identity, &self.sparse_spec()?, + Some(&mut cache), )?; + cache.save()?; drop(store); refs::write_branch_tip(&self.layout, &head, &target_tip)?; crate::oplog::record( @@ -986,6 +995,7 @@ impl Repo { // (each step pruned its wrap map to its own merged tree): a // PROTECTED entry decrypts for `identity` when possible, else is // skipped — never writes ciphertext to disk. + let mut cache = self.open_protected_cache()?; worktree::materialize( &self.layout, &mut store, @@ -994,7 +1004,9 @@ impl Repo { &acc_protection, identity, &self.sparse_spec()?, + Some(&mut cache), )?; + cache.save()?; } refs::write_branch_tip(&self.layout, &head, &acc_tip)?; crate::oplog::record( @@ -1198,7 +1210,8 @@ impl Repo { let skipped = { let store_arc = self.vfs().store(); let mut store = store_arc.lock().unwrap(); - worktree::materialize( + let mut cache = self.open_protected_cache()?; + let skipped = worktree::materialize( &self.layout, &mut store, snap.root, @@ -1206,7 +1219,10 @@ impl Repo { &snap.protection, None, &self.sparse_spec()?, - )? + Some(&mut cache), + )?; + cache.save()?; + skipped }; crate::rebase_state::clear(&self.layout)?; Ok(skipped) diff --git a/crates/repo/src/repo.rs b/crates/repo/src/repo.rs index 58e1521e..cc7d1c0a 100644 --- a/crates/repo/src/repo.rs +++ b/crates/repo/src/repo.rs @@ -76,6 +76,22 @@ impl Repo { &self.layout } + /// Open this repo's main-checkout P33 unchanged-detection cache + /// (`.sc/protected-cache`), keyed by `.sc/local-key`. Every + /// `worktree::materialize` call against the host working tree opens one + /// of these and saves it right after materializing, so a randomized + /// protected file just decrypted to disk is recognized as unchanged by + /// the very next `status`/`diff` — without this, `diff_worktree`'s + /// randomized-entry branch has no stat-hit or keyed-tag to match and + /// spuriously reports the untouched file as modified. + pub(crate) fn open_protected_cache(&self) -> Result { + let key = crate::cache::local_key(&self.layout)?; + Ok(crate::cache::ProtectedCache::open( + key, + Some(self.layout.protected_cache_path()), + )) + } + /// The tip snapshot of the current branch (None if unborn). pub fn head_tip(&self) -> Result> { refs::head_tip(&self.layout) @@ -1234,6 +1250,7 @@ impl Repo { let theirs_snap = store.get_snapshot(&theirs)?; let theirs_root = theirs_snap.root; let theirs_protection = theirs_snap.protection; + let mut cache = self.open_protected_cache()?; let skipped = worktree::materialize( &self.layout, &mut store, @@ -1242,7 +1259,9 @@ impl Repo { &theirs_protection, identity, &self.sparse_spec()?, + Some(&mut cache), )?; + cache.save()?; drop(store); refs::write_branch_tip(&self.layout, &head, &theirs)?; crate::oplog::record( @@ -1269,6 +1288,7 @@ impl Repo { let theirs_root = theirs_snap.root; let theirs_protection = theirs_snap.protection; let ours_root = store.get_snapshot(&ours)?.root; + let mut cache = self.open_protected_cache()?; let skipped = worktree::materialize( &self.layout, &mut store, @@ -1277,7 +1297,9 @@ impl Repo { &theirs_protection, identity, &self.sparse_spec()?, + Some(&mut cache), )?; + cache.save()?; drop(store); refs::write_branch_tip(&self.layout, &head, &theirs)?; crate::oplog::record( @@ -1457,7 +1479,8 @@ impl Repo { let skipped = { let store_arc = self.vfs.store(); let mut store = store_arc.lock().unwrap(); - worktree::materialize( + let mut cache = self.open_protected_cache()?; + let skipped = worktree::materialize( &self.layout, &mut store, merged_root, @@ -1465,7 +1488,10 @@ impl Repo { &merged_protection, identity, &self.sparse_spec()?, - )? + Some(&mut cache), + )?; + cache.save()?; + skipped }; // Clean merge: two-parent snapshot now, carrying the merged @@ -1557,6 +1583,7 @@ impl Repo { // completion commit's decided-tree carry-forward preserves // skipped content, so nothing is lost by not surfacing the list // here (the `Err`/`Stopped` return can't carry it). + let mut cache = self.open_protected_cache()?; let _skipped = worktree::materialize( &self.layout, &mut store, @@ -1565,7 +1592,9 @@ impl Repo { conflict_prot, identity, &self.sparse_spec()?, + Some(&mut cache), )?; + cache.save()?; } for (path, bytes, _mode, _recipients) in to_encrypt { let full = worktree::safe_join(&self.layout.root, path)?; @@ -1614,6 +1643,7 @@ impl Repo { let ours_protection = ours_snap.protection; let theirs_root = store.get_snapshot(&theirs_id)?.root; // No identity at abort time: protected files in ours are skipped (not decrypted). + let mut cache = self.open_protected_cache()?; skipped = worktree::materialize( &self.layout, &mut store, @@ -1622,7 +1652,9 @@ impl Repo { &ours_protection, None, &self.sparse_spec()?, + Some(&mut cache), )?; + cache.save()?; } crate::merge_state::clear(&self.layout)?; Ok(skipped) @@ -1733,7 +1765,8 @@ impl Repo { let skipped = { let store_arc = self.vfs.store(); let mut store = store_arc.lock().unwrap(); - worktree::materialize( + let mut cache = self.open_protected_cache()?; + let skipped = worktree::materialize( &self.layout, &mut store, target_root, @@ -1741,7 +1774,10 @@ impl Repo { &target_protection, identity, &self.sparse_spec()?, - )? + Some(&mut cache), + )?; + cache.save()?; + skipped }; refs::write_head(&self.layout, name)?; crate::oplog::record( @@ -2846,13 +2882,20 @@ mod tests { } #[test] - fn convergent_merge_ids_are_stable_across_repos() { + fn merge_ids_no_longer_converge_across_repos() { + // P33/ADR-0043: `encrypt_protected`'s seals are now randomized, so + // this pre-P33 test's premise ("identical merged plaintext converges + // to the same blob id across independent repos") is deliberately + // broken — the oracle that convergence gave an equality-testing + // observer is exactly what randomization closes (accepted cost 4c). + // Renamed and inverted to pin the NEW behavior: two independent + // repos building the identical protected divergence and + // content-merging it now produce DIFFERENT ciphertext blob ids for + // the same plaintext. + // // Two independent repos build the identical protected divergence // (same base plaintext, same edits on both sides) and content-merge - // with their own (distinct) recipient identity. Convergent encryption - // means the merged plaintext is identical, so the resulting ciphertext - // blob id must be IDENTICAL across the two repos — merge output is - // deterministic content addressing, not per-repo randomness. + // with their own (distinct) recipient identity. fn build(tag: &str) -> (Repo, std::path::PathBuf, ObjectId, ObjectId) { let root = tmp_root(tag); let repo = Repo::init(&root).unwrap(); @@ -2884,9 +2927,9 @@ mod tests { let (repo1, root1, _id1, blob1) = build("p15-merge-convergent-a"); let (repo2, root2, _id2, blob2) = build("p15-merge-convergent-b"); - assert_eq!( + assert_ne!( blob1, blob2, - "identical merged plaintext must converge to the same blob id" + "randomized seals must NOT converge across independent repos (oracle closed, P33)" ); drop(repo1); drop(repo2); diff --git a/crates/repo/src/sparse.rs b/crates/repo/src/sparse.rs index 466db137..840442d5 100644 --- a/crates/repo/src/sparse.rs +++ b/crates/repo/src/sparse.rs @@ -181,7 +181,8 @@ impl Repo { let snap = self.snapshot(&tip)?; let store_arc = self.vfs().store(); let mut s = store_arc.lock().unwrap(); - crate::worktree::materialize( + let mut cache = self.open_protected_cache()?; + let skipped = crate::worktree::materialize( self.layout(), &mut s, snap.root, @@ -189,7 +190,10 @@ impl Repo { &snap.protection, identity, &spec, - ) + Some(&mut cache), + )?; + cache.save()?; + Ok(skipped) } /// Disable sparse checkout: clear the persisted spec and re-materialize @@ -238,7 +242,8 @@ impl Repo { let snap = self.snapshot(&tip)?; let store_arc = self.vfs().store(); let mut s = store_arc.lock().unwrap(); - crate::worktree::materialize( + let mut cache = self.open_protected_cache()?; + let skipped = crate::worktree::materialize( self.layout(), &mut s, snap.root, @@ -246,7 +251,10 @@ impl Repo { &snap.protection, identity, &Sparse::default(), - ) + Some(&mut cache), + )?; + cache.save()?; + Ok(skipped) } } diff --git a/crates/repo/src/sync.rs b/crates/repo/src/sync.rs index 1036d61d..4eb3ded2 100644 --- a/crates/repo/src/sync.rs +++ b/crates/repo/src/sync.rs @@ -163,6 +163,7 @@ impl Repo { &head_protection, None, &sparse_spec, + None, )?; } Ok(dst_repo) diff --git a/crates/repo/src/workspace.rs b/crates/repo/src/workspace.rs index a1c1b7e6..cdc62ee3 100644 --- a/crates/repo/src/workspace.rs +++ b/crates/repo/src/workspace.rs @@ -48,6 +48,12 @@ pub(crate) fn materialize_workspace( let ws = Layout::at(dir); let store_arc = repo.vfs().store(); let mut store = store_arc.lock().unwrap(); + // No cache threaded here: a workspace checkout's stat cache would need to + // live beside the checkout dir (`Layout::ws_cache_path`), not inside it, + // to avoid harvest's worktree read picking it up as an untracked file — + // wiring that per-workspace cache through is a follow-on (P33 doesn't + // require it: a miss here only ever degrades to a spurious reseal on the + // next harvest, never to incorrectness, per `cache.rs`'s contract). worktree::materialize( &ws, &mut store, @@ -56,6 +62,7 @@ pub(crate) fn materialize_workspace( &snap.protection, identity, sparse, + None, ) } diff --git a/crates/repo/src/worktree.rs b/crates/repo/src/worktree.rs index a3b93ade..18642089 100644 --- a/crates/repo/src/worktree.rs +++ b/crates/repo/src/worktree.rs @@ -603,6 +603,7 @@ pub fn materialize( protection: &Protection, identity: Option<&scl_crypto::SecretKey>, sparse: &Sparse, + mut cache: Option<&mut ProtectedCache>, ) -> Result> { // Gap-tolerant when sparse is active (P27 Task 4): a promisor-filtered // clone never fetched out-of-filter subtrees at all, so an unfiltered @@ -662,6 +663,19 @@ pub fn materialize( std::fs::create_dir_all(parent)?; } std::fs::write(&full, &pt[..])?; + // Record a randomized entry's cache entry right after the + // write (P33): without this, the next `diff_worktree`/ + // `status` on this exact, unmodified plaintext has no + // stat-hit or keyed-tag to match and spuriously reports + // it as modified — since re-encryption can't prove + // unchanged-ness for a randomized seal. Mirrors the + // record call `commit`'s carry/fresh-seal paths already + // make (`repo.rs`'s `snapshot_files`). + if perms & scl_core::RANDOMIZED != 0 { + if let Some(c) = cache.as_deref_mut() { + c.record(path, &full, &pt, *blob_id); + } + } // `pt` (Zeroizing>) is dropped and zeroed here. } None => { @@ -769,6 +783,7 @@ mod tests { &Default::default(), None, &Sparse::default(), + None, ); // Restore perms before asserting so cleanup always works. @@ -807,6 +822,7 @@ mod tests { &Default::default(), None, &Sparse::default(), + None, ) .unwrap_err(); assert!(matches!(err, crate::error::Error::BadRef(_)), "got {err:?}"); @@ -835,7 +851,8 @@ mod tests { None, &Default::default(), None, - &Sparse::default() + &Sparse::default(), + None ) .is_err()); From afae35b9232d8b61953e9302e9ce009eb62da528 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 21:15:05 +0300 Subject: [PATCH 10/22] P33: per-workspace caches + cache-save discipline cleanup Thread a per-checkout ProtectedCache through the workspace fork/harvest seam so an untouched randomized protected path carries the base ciphertext blob instead of re-sealing (the false-conflict hazard across ws workspaces). ProtectedCache now carries its own abs-path root (set at open), so snapshot_files stats the workspace's files, not the host's. Normalize every cache save to best-effort after the ref moves; abort paths pass no cache; narrow materialize/diff_worktree to pub(crate). Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/repo/src/cache.rs | 80 ++++++++++++++++------- crates/repo/src/replay.rs | 33 ++++++---- crates/repo/src/repo.rs | 119 +++++++++++++++++----------------- crates/repo/src/sparse.rs | 10 ++- crates/repo/src/workspace.rs | 70 ++++++++++++-------- crates/repo/src/worktree.rs | 83 +++++++++++++++++++++--- crates/repo/src/ws.rs | 121 +++++++++++++++++++++++++++++++++-- 7 files changed, 380 insertions(+), 136 deletions(-) diff --git a/crates/repo/src/cache.rs b/crates/repo/src/cache.rs index a534d924..9db25137 100644 --- a/crates/repo/src/cache.rs +++ b/crates/repo/src/cache.rs @@ -85,15 +85,31 @@ struct CacheEntry { /// Per-checkout unchanged-detection cache (main tree, one ws workspace, or /// an ephemeral in-memory one for `sc work` temp checkouts). +/// +/// The cache owns its `root`: the absolute base directory of the checkout it +/// serves, against which relative tree paths are resolved for the stat +/// (mtime+size) short-circuit. This is the P33 Task 7 design choice (over +/// threading an explicit worktree-root through `snapshot_files`): a stat cache +/// is inherently tied to one checkout, so it should carry that checkout's root +/// rather than have every `unchanged`/`record` caller pass an absolute path. +/// **Invariant:** the `root` passed at `open` must equal `layout.root` of the +/// checkout that `materialize`/`diff_worktree`/`snapshot_files` operate on — +/// the main tree opens with `layout.root`, a `sc ws`/`sc work` workspace with +/// its own checkout dir. A mismatched root only ever loses the stat +/// short-circuit (the keyed-tag comparison still proves unchanged-ness, and a +/// miss degrades to a spurious reseal) — never incorrectness. pub(crate) struct ProtectedCache { key: [u8; 32], + /// Absolute base dir of the served checkout; `root.join(rel)` is the file + /// a relative tree path stats to. + root: PathBuf, /// `None` => ephemeral: `save()` is a no-op. path: Option, entries: BTreeMap, } impl ProtectedCache { - pub(crate) fn open(key: [u8; 32], path: Option) -> ProtectedCache { + pub(crate) fn open(key: [u8; 32], root: PathBuf, path: Option) -> ProtectedCache { let mut entries = BTreeMap::new(); if let Some(p) = &path { if let Ok(text) = std::fs::read_to_string(p) { @@ -121,7 +137,7 @@ impl ProtectedCache { } } } - ProtectedCache { key, path, entries } + ProtectedCache { key, root, path, entries } } fn tag(&self, plaintext: &[u8]) -> [u8; 32] { @@ -137,10 +153,11 @@ impl ProtectedCache { /// The cached ciphertext blob id iff this exact plaintext is what last /// sealed at `rel`: stat hit (mtime+size unchanged) short-circuits; - /// otherwise fall back to the keyed-tag comparison. - pub(crate) fn unchanged(&self, rel: &str, abs: &Path, plaintext: &[u8]) -> Option { + /// otherwise fall back to the keyed-tag comparison. The absolute file is + /// resolved against the cache's own `root` (`root.join(rel)`). + pub(crate) fn unchanged(&self, rel: &str, plaintext: &[u8]) -> Option { let e = self.entries.get(rel)?; - if let Some((mtime_ns, size)) = Self::stat(abs) { + if let Some((mtime_ns, size)) = Self::stat(&self.root.join(rel)) { if mtime_ns == e.mtime_ns && size == e.size { return Some(e.blob_id); } @@ -148,10 +165,11 @@ impl ProtectedCache { (self.tag(plaintext) == e.tag).then_some(e.blob_id) } - /// Record that `plaintext` at `rel` seals to `blob_id`. Missing stat - /// (file vanished mid-operation) just skips the entry. - pub(crate) fn record(&mut self, rel: &str, abs: &Path, plaintext: &[u8], blob_id: ObjectId) { - if let Some((mtime_ns, size)) = Self::stat(abs) { + /// Record that `plaintext` at `rel` seals to `blob_id`. The absolute file + /// is resolved against the cache's own `root`. Missing stat (file vanished + /// mid-operation) just skips the entry. + pub(crate) fn record(&mut self, rel: &str, plaintext: &[u8], blob_id: ObjectId) { + if let Some((mtime_ns, size)) = Self::stat(&self.root.join(rel)) { let tag = self.tag(plaintext); self.entries .insert(rel.to_string(), CacheEntry { mtime_ns, size, tag, blob_id }); @@ -180,6 +198,18 @@ impl ProtectedCache { scl_core::fsutil::atomic_write_durable(path, out.as_bytes())?; Ok(()) } + + /// Persist best-effort: on failure, log to stderr and swallow rather than + /// propagate. Every ref-moving op that populates the cache saves through + /// this AFTER its ref has moved (P33 Task 7), so cache trouble can never + /// abort an operation that has already logically succeeded — a lost/stale + /// cache only ever degrades to a spurious reseal next time, never to + /// incorrectness. + pub(crate) fn save_best_effort(&self) { + if let Err(e) = self.save() { + eprintln!("warning: failed to persist protected-cache: {e}"); + } + } } #[cfg(test)] @@ -225,20 +255,21 @@ mod tests { std::fs::write(&abs, b"v1").unwrap(); let id = ObjectId::of(b"cipher-of-v1"); - let mut c = ProtectedCache::open(key, Some(layout.protected_cache_path())); - assert_eq!(c.unchanged("secret.txt", &abs, b"v1"), None, "empty cache misses"); - c.record("secret.txt", &abs, b"v1", id); + let mut c = + ProtectedCache::open(key, layout.root.clone(), Some(layout.protected_cache_path())); + assert_eq!(c.unchanged("secret.txt", b"v1"), None, "empty cache misses"); + c.record("secret.txt", b"v1", id); // Stat hit: file untouched. - assert_eq!(c.unchanged("secret.txt", &abs, b"v1"), Some(id)); + assert_eq!(c.unchanged("secret.txt", b"v1"), Some(id)); // Tag fallback: touch mtime without changing content. std::fs::write(&abs, b"v1").unwrap(); - assert_eq!(c.unchanged("secret.txt", &abs, b"v1"), Some(id)); + assert_eq!(c.unchanged("secret.txt", b"v1"), Some(id)); // Real change: miss. std::fs::write(&abs, b"v2").unwrap(); - assert_eq!(c.unchanged("secret.txt", &abs, b"v2"), None); + assert_eq!(c.unchanged("secret.txt", b"v2"), None); cleanup(&layout); } @@ -251,17 +282,20 @@ mod tests { std::fs::write(&abs, b"v").unwrap(); let id = ObjectId::of(b"c"); - let mut c = ProtectedCache::open(key, Some(layout.protected_cache_path())); - c.record("a b/spaced name.txt", &abs, b"v", id); + let mut c = + ProtectedCache::open(key, layout.root.clone(), Some(layout.protected_cache_path())); + c.record("a b/spaced name.txt", b"v", id); c.save().unwrap(); - let c2 = ProtectedCache::open(key, Some(layout.protected_cache_path())); - assert_eq!(c2.unchanged("a b/spaced name.txt", &abs, b"v"), Some(id)); + let c2 = + ProtectedCache::open(key, layout.root.clone(), Some(layout.protected_cache_path())); + assert_eq!(c2.unchanged("a b/spaced name.txt", b"v"), Some(id)); // Corrupt file: degrades to empty, never errors. std::fs::write(layout.protected_cache_path(), b"garbage\nlines\n").unwrap(); - let c3 = ProtectedCache::open(key, Some(layout.protected_cache_path())); - assert_eq!(c3.unchanged("a b/spaced name.txt", &abs, b"v"), None); + let c3 = + ProtectedCache::open(key, layout.root.clone(), Some(layout.protected_cache_path())); + assert_eq!(c3.unchanged("a b/spaced name.txt", b"v"), None); cleanup(&layout); } @@ -271,8 +305,8 @@ mod tests { let key = local_key(&layout).unwrap(); let abs = layout.root.join("f"); std::fs::write(&abs, b"v").unwrap(); - let mut c = ProtectedCache::open(key, None); - c.record("f", &abs, b"v", ObjectId::of(b"c")); + let mut c = ProtectedCache::open(key, layout.root.clone(), None); + c.record("f", b"v", ObjectId::of(b"c")); c.save().unwrap(); assert!(!layout.protected_cache_path().exists()); cleanup(&layout); diff --git a/crates/repo/src/replay.rs b/crates/repo/src/replay.rs index 19ca032a..9c96a7c4 100644 --- a/crates/repo/src/replay.rs +++ b/crates/repo/src/replay.rs @@ -465,13 +465,15 @@ impl Repo { author, &message, )?; + // Cache hoisted above the block so its save follows the ref + // move below, matching commit/amend's discipline (P33 Task 7). + let mut cache = self.open_protected_cache()?; { let store_arc = self.vfs().store(); let mut store = store_arc.lock().unwrap(); // Protection-aware materialize: a PROTECTED entry decrypts // for `identity` when possible, else is skipped (never // writes ciphertext to disk) — same as merge's clean path. - let mut cache = self.open_protected_cache()?; worktree::materialize( &self.layout, &mut store, @@ -482,9 +484,9 @@ impl Repo { &self.sparse_spec()?, Some(&mut cache), )?; - cache.save()?; } refs::write_branch_tip(&self.layout, &head, &id)?; + cache.save_best_effort(); crate::oplog::record( &self.layout, &format!("cherry-pick {refname}"), @@ -593,7 +595,9 @@ impl Repo { let snap = self.snapshot(&tip)?; let store_arc = self.vfs().store(); let mut store = store_arc.lock().unwrap(); - let mut cache = self.open_protected_cache()?; + // No identity at abort time: nothing is decrypted, so `materialize` + // records nothing — pass `None`, no cache to open or save + // (P33 Task 7 item 7). skipped = worktree::materialize( &self.layout, &mut store, @@ -602,9 +606,8 @@ impl Repo { &snap.protection, None, &self.sparse_spec()?, - Some(&mut cache), + None, )?; - cache.save()?; } crate::pick_state::clear(&self.layout)?; Ok(skipped) @@ -714,9 +717,9 @@ impl Repo { &self.sparse_spec()?, Some(&mut cache), )?; - cache.save()?; drop(store); refs::write_branch_tip(&self.layout, &head, &target_tip)?; + cache.save_best_effort(); crate::oplog::record( &self.layout, &format!("rebase onto {target} (ff)"), @@ -988,6 +991,9 @@ impl Repo { } } + // Cache hoisted above the block so its save follows the ref move + // below, matching commit/amend's discipline (P33 Task 7). + let mut cache = self.open_protected_cache()?; { let store_arc = self.vfs().store(); let mut store = store_arc.lock().unwrap(); @@ -995,7 +1001,6 @@ impl Repo { // (each step pruned its wrap map to its own merged tree): a // PROTECTED entry decrypts for `identity` when possible, else is // skipped — never writes ciphertext to disk. - let mut cache = self.open_protected_cache()?; worktree::materialize( &self.layout, &mut store, @@ -1006,9 +1011,9 @@ impl Repo { &self.sparse_spec()?, Some(&mut cache), )?; - cache.save()?; } refs::write_branch_tip(&self.layout, &head, &acc_tip)?; + cache.save_best_effort(); crate::oplog::record( &self.layout, &format!("rebase onto {target} ({replayed} replayed, {skipped} skipped)"), @@ -1210,8 +1215,10 @@ impl Repo { let skipped = { let store_arc = self.vfs().store(); let mut store = store_arc.lock().unwrap(); - let mut cache = self.open_protected_cache()?; - let skipped = worktree::materialize( + // No identity at abort time: nothing is decrypted, so `materialize` + // records nothing — pass `None`, no cache to open or save + // (P33 Task 7 item 7). + worktree::materialize( &self.layout, &mut store, snap.root, @@ -1219,10 +1226,8 @@ impl Repo { &snap.protection, None, &self.sparse_spec()?, - Some(&mut cache), - )?; - cache.save()?; - skipped + None, + )? }; crate::rebase_state::clear(&self.layout)?; Ok(skipped) diff --git a/crates/repo/src/repo.rs b/crates/repo/src/repo.rs index cc7d1c0a..ff0d0590 100644 --- a/crates/repo/src/repo.rs +++ b/crates/repo/src/repo.rs @@ -88,6 +88,7 @@ impl Repo { let key = crate::cache::local_key(&self.layout)?; Ok(crate::cache::ProtectedCache::open( key, + self.layout.root.clone(), Some(self.layout.protected_cache_path()), )) } @@ -213,9 +214,7 @@ impl Repo { // recorded-but-unlanded id is benign (the cache degradation contract), // and both callers (`commit`'s pick arm, `rebase_continue`) move the // ref immediately after a successful return. - let key = crate::cache::local_key(&self.layout)?; - let mut cache = - crate::cache::ProtectedCache::open(key, Some(self.layout.protected_cache_path())); + let mut cache = self.open_protected_cache()?; let id = self.snapshot_files( files, Some(parent), @@ -229,7 +228,11 @@ impl Repo { author, message, )?; - cache.save()?; + // Best-effort (P33 Task 7): this function does not own the ref move + // (both callers move it right after a successful return), and its + // save-before-ref is already justified above — a recorded-but-unlanded + // id is benign. Never let cache trouble abort the completion. + cache.save_best_effort(); Ok(id) } @@ -448,14 +451,14 @@ impl Repo { // Randomized prior: the local stat/tag cache is the only // way to prove "same plaintext". Carry iff the cached id // matches the prior tip id (double-checked: same content - // AND same recorded seal). - match worktree::safe_join(&self.layout.root, &path) { - Ok(abs) => { - cache.as_deref().and_then(|c| c.unchanged(&path, &abs, &bytes)) - == Some(*blob_id) - } - Err(_) => false, - } + // AND same recorded seal). The cache resolves the abs + // stat path against its own checkout root (P33 Task 7), + // so a workspace commit stats the workspace file, not + // the host's. + cache + .as_deref() + .and_then(|c| c.unchanged(&path, &bytes)) + == Some(*blob_id) }; if unchanged { Some((*blob_id, *perms)) @@ -490,11 +493,8 @@ impl Repo { // the next commit gets a fast stat hit (optimization // only; the caller persists after the ref moves). if perms & scl_core::RANDOMIZED != 0 { - if let (Some(c), Ok(abs)) = ( - cache.as_deref_mut(), - worktree::safe_join(&self.layout.root, &path), - ) { - c.record(&path, &abs, &bytes, blob_id); + if let Some(c) = cache.as_deref_mut() { + c.record(&path, &bytes, blob_id); } } } @@ -516,12 +516,10 @@ impl Repo { let (protected_all, sealed_wrapped) = crate::protect::encrypt_protected(to_seal)?; // Record every freshly sealed path's new blob id into the cache; the // committing caller persists it only after the ref actually moves. - if let Some(c) = cache.as_deref_mut() { + if let Some(c) = cache { for (path, cipher, _mode, _perms) in &protected_all { if let Some((_, pt)) = sealed_plaintexts.iter().find(|(p, _)| p == path) { - if let Ok(abs) = worktree::safe_join(&self.layout.root, path) { - c.record(path, &abs, pt, Object::blob(cipher.clone()).id()); - } + c.record(path, pt, Object::blob(cipher.clone()).id()); } } } @@ -806,11 +804,7 @@ impl Repo { )?, _ => { let files = worktree::read_worktree(&self.layout, &self.tracked_paths()?)?; - let key = crate::cache::local_key(&self.layout)?; - let mut c = crate::cache::ProtectedCache::open( - key, - Some(self.layout.protected_cache_path()), - ); + let mut c = self.open_protected_cache()?; let id = self.snapshot_files( files, tip, @@ -831,10 +825,11 @@ impl Repo { refs::write_branch_tip(&self.layout, &head, &id)?; crate::merge_state::clear(&self.layout)?; crate::pick_state::clear(&self.layout)?; - // The ref moved: persist the cache (no-op in the pick arm, which - // manages its own). + // The ref moved: persist the cache best-effort (no-op in the pick arm, + // which manages its own). Cache trouble must never abort a commit + // whose ref already moved (P33 Task 7). if let Some(c) = cache { - c.save()?; + c.save_best_effort(); } let label = if merging { "commit (merge)" @@ -890,9 +885,7 @@ impl Repo { let before = refs::read_branch_tip(&self.layout, &head)?; let files = worktree::read_worktree(&self.layout, &self.tracked_paths()?)?; - let key = crate::cache::local_key(&self.layout)?; - let mut cache = - crate::cache::ProtectedCache::open(key, Some(self.layout.protected_cache_path())); + let mut cache = self.open_protected_cache()?; let id = self.snapshot_files( files, Some(tip), @@ -908,7 +901,7 @@ impl Repo { )?; refs::write_branch_tip(&self.layout, &head, &id)?; - cache.save()?; + cache.save_best_effort(); crate::oplog::record( &self.layout, "amend", @@ -1003,8 +996,7 @@ impl Repo { }; let store_arc = self.vfs.store(); let mut store = store_arc.lock().unwrap(); - let key = crate::cache::local_key(&self.layout)?; - let cache = crate::cache::ProtectedCache::open(key, Some(self.layout.protected_cache_path())); + let cache = self.open_protected_cache()?; worktree::diff_worktree( &self.layout, &mut store, @@ -1051,9 +1043,13 @@ impl Repo { // failure to load/mint `.sc/local-key` degrades to `None` here rather // than failing the whole diff — spurious-but-safe: every randomized // entry just reports "changed" instead of the diff erroring out. - let cache = crate::cache::local_key(&self.layout) - .ok() - .map(|key| crate::cache::ProtectedCache::open(key, Some(self.layout.protected_cache_path()))); + let cache = crate::cache::local_key(&self.layout).ok().map(|key| { + crate::cache::ProtectedCache::open( + key, + self.layout.root.clone(), + Some(self.layout.protected_cache_path()), + ) + }); let store_arc = self.vfs.store(); let mut store = store_arc.lock().unwrap(); let head = match head_root { @@ -1094,9 +1090,7 @@ impl Repo { // cache is the only public-key-free unchanged proof. // A miss/disagreement/absent cache reports changed — // spurious-but-safe, never incorrect. - let abs = worktree::safe_join(&self.layout.root, path)?; - cache.as_ref().and_then(|c| c.unchanged(path, &abs, bytes)) - != Some(*blob_id) + cache.as_ref().and_then(|c| c.unchanged(path, bytes)) != Some(*blob_id) }; if changed { out.push_str(&format!( @@ -1261,9 +1255,9 @@ impl Repo { &self.sparse_spec()?, Some(&mut cache), )?; - cache.save()?; drop(store); refs::write_branch_tip(&self.layout, &head, &theirs)?; + cache.save_best_effort(); crate::oplog::record( &self.layout, &format!("merge {branch} (adopt)"), @@ -1299,9 +1293,9 @@ impl Repo { &self.sparse_spec()?, Some(&mut cache), )?; - cache.save()?; drop(store); refs::write_branch_tip(&self.layout, &head, &theirs)?; + cache.save_best_effort(); crate::oplog::record( &self.layout, &format!("merge {branch} (ff)"), @@ -1476,11 +1470,14 @@ impl Repo { // a merged PROTECTED entry decrypts for `identity` when possible, else // is skipped (never writes ciphertext to disk) and reported to the // caller. (Sidecars exist only on the conflict path, handled above.) + // The cache is hoisted above the block so its save can follow the ref + // move (`commit_snapshot` below), matching commit/amend's discipline + // (P33 Task 7). + let mut cache = self.open_protected_cache()?; let skipped = { let store_arc = self.vfs.store(); let mut store = store_arc.lock().unwrap(); - let mut cache = self.open_protected_cache()?; - let skipped = worktree::materialize( + worktree::materialize( &self.layout, &mut store, merged_root, @@ -1489,9 +1486,7 @@ impl Repo { identity, &self.sparse_spec()?, Some(&mut cache), - )?; - cache.save()?; - skipped + )? }; // Clean merge: two-parent snapshot now, carrying the merged @@ -1504,6 +1499,7 @@ impl Repo { author, &format!("merge {branch}"), )?; + cache.save_best_effort(); crate::oplog::record( &self.layout, &format!("merge {branch}"), @@ -1594,7 +1590,9 @@ impl Repo { &self.sparse_spec()?, Some(&mut cache), )?; - cache.save()?; + // Conflict materialization moves no branch ref; best-effort save + // so cache trouble never aborts the conflict write-out (P33 Task 7). + cache.save_best_effort(); } for (path, bytes, _mode, _recipients) in to_encrypt { let full = worktree::safe_join(&self.layout.root, path)?; @@ -1642,8 +1640,10 @@ impl Repo { let ours_root = ours_snap.root; let ours_protection = ours_snap.protection; let theirs_root = store.get_snapshot(&theirs_id)?.root; - // No identity at abort time: protected files in ours are skipped (not decrypted). - let mut cache = self.open_protected_cache()?; + // No identity at abort time: protected files in ours are skipped + // (not decrypted). With nothing decrypted, `materialize` records + // nothing into a cache, so pass `None` — no cache to open or save + // (P33 Task 7 item 7). skipped = worktree::materialize( &self.layout, &mut store, @@ -1652,9 +1652,8 @@ impl Repo { &ours_protection, None, &self.sparse_spec()?, - Some(&mut cache), + None, )?; - cache.save()?; } crate::merge_state::clear(&self.layout)?; Ok(skipped) @@ -1762,11 +1761,13 @@ impl Repo { let snap = store_arc.lock().unwrap().get_snapshot(&target_tip)?; (snap.root, snap.protection) }; + // Cache hoisted above the block so its save follows the ref move + // (`write_head` below), matching commit/amend's discipline (P33 Task 7). + let mut cache = self.open_protected_cache()?; let skipped = { let store_arc = self.vfs.store(); let mut store = store_arc.lock().unwrap(); - let mut cache = self.open_protected_cache()?; - let skipped = worktree::materialize( + worktree::materialize( &self.layout, &mut store, target_root, @@ -1775,11 +1776,10 @@ impl Repo { identity, &self.sparse_spec()?, Some(&mut cache), - )?; - cache.save()?; - skipped + )? }; refs::write_head(&self.layout, name)?; + cache.save_best_effort(); crate::oplog::record( &self.layout, &format!("switch {name}"), @@ -3790,9 +3790,10 @@ mod tests { let key = crate::cache::local_key(repo.layout()).unwrap(); let mut cache = crate::cache::ProtectedCache::open( key, + repo.layout().root.clone(), Some(repo.layout().protected_cache_path()), ); - cache.record("secret/x", &root.join("secret/x"), &pt, randomized_id); + cache.record("secret/x", &pt, randomized_id); cache.save().unwrap(); } diff --git a/crates/repo/src/sparse.rs b/crates/repo/src/sparse.rs index 840442d5..86d94750 100644 --- a/crates/repo/src/sparse.rs +++ b/crates/repo/src/sparse.rs @@ -192,7 +192,11 @@ impl Repo { &spec, Some(&mut cache), )?; - cache.save()?; + // The sparse spec (the operation's durable commit point) is already + // persisted above; this re-lay moves no branch ref, so the cache save + // is best-effort — never abort a sparse change on cache trouble + // (P33 Task 7). + cache.save_best_effort(); Ok(skipped) } @@ -253,7 +257,9 @@ impl Repo { &Sparse::default(), Some(&mut cache), )?; - cache.save()?; + // Best-effort: the cleared sparse spec is already persisted above and + // this re-lay moves no branch ref (P33 Task 7). + cache.save_best_effort(); Ok(skipped) } } diff --git a/crates/repo/src/workspace.rs b/crates/repo/src/workspace.rs index cdc62ee3..70d4cd37 100644 --- a/crates/repo/src/workspace.rs +++ b/crates/repo/src/workspace.rs @@ -42,18 +42,20 @@ pub(crate) fn materialize_workspace( dir: &Path, identity: Option<&scl_crypto::SecretKey>, sparse: &crate::sparse::Sparse, + cache: Option<&mut crate::cache::ProtectedCache>, ) -> Result> { std::fs::create_dir_all(dir)?; let snap = repo.snapshot(&tip)?; let ws = Layout::at(dir); let store_arc = repo.vfs().store(); let mut store = store_arc.lock().unwrap(); - // No cache threaded here: a workspace checkout's stat cache would need to - // live beside the checkout dir (`Layout::ws_cache_path`), not inside it, - // to avoid harvest's worktree read picking it up as an untracked file — - // wiring that per-workspace cache through is a follow-on (P33 doesn't - // require it: a miss here only ever degrades to a spurious reseal on the - // next harvest, never to incorrectness, per `cache.rs`'s contract). + // The per-workspace cache (P33 Task 7) is opened by the caller with its + // `root` set to THIS checkout dir and its file at `Layout::ws_cache_path` + // (beside the checkout dir, never inside it, so harvest's worktree read + // can't pick it up). `sc ws fork` passes a persistent cache; `sc work` + // passes an ephemeral one threaded fork->harvest. A `None` cache still + // works — a workspace's randomized protected path just re-seals on the + // next harvest instead of carrying (spurious-but-safe, per `cache.rs`). worktree::materialize( &ws, &mut store, @@ -62,7 +64,7 @@ pub(crate) fn materialize_workspace( &snap.protection, identity, sparse, - None, + cache, ) } @@ -78,6 +80,7 @@ pub(crate) fn harvest_workspace( author: &str, message: &str, sparse: &crate::sparse::Sparse, + cache: Option<&mut crate::cache::ProtectedCache>, ) -> Result { let snap = repo.snapshot(&tip)?; let ws = Layout::at(dir); @@ -88,16 +91,19 @@ pub(crate) fn harvest_workspace( worktree::tree_file_ids(&mut store, snap.root)? .into_keys() .collect(); - // No workspace-local cache is threaded in yet (P33 Task 7 wires it); - // until then a randomized protected path in a workspace always - // reports modified — spurious-but-safe. + // The workspace-local cache (P33 Task 7) proves an untouched randomized + // protected path unchanged, so it isn't spuriously reported modified + // here (which would send it into the fresh-seal path in `snapshot_files` + // below). Its `root` is this checkout dir, so its stat lookups resolve + // against the workspace's own files — unlike the host cache, which + // would stat the wrong tree. let d = worktree::diff_worktree( &ws, &mut store, Some(snap.root), &snap.protection, sparse, - None, + cache.as_deref(), )?; ( tracked, @@ -113,12 +119,11 @@ pub(crate) fn harvest_workspace( // fixed at materialize time (or, for `sc work`, is always full), so the // carry predicate must see that same view, not the host repo's current // `.sc/sparse` — see `snapshot_files`'s doc comment. - // No workspace-local cache is threaded in yet (P33 Task 7 wires it): pass - // `None`, so a randomized protected path in a workspace always re-seals - // (spurious-but-safe). Passing the HOST repo's cache here would be WRONG — - // the abs-path base in `snapshot_files` is the host root, but these files - // came from the workspace dir. Convergent priors still carry (content-only - // compare, no cache needed). + // The workspace-local cache is threaded into the commit too (P33 Task 7): + // its `root` is this checkout dir, so a randomized protected path the + // workspace never touched carries the base's exact ciphertext blob id + // (proven unchanged by the cache) instead of re-sealing. Convergent priors + // still carry via content-only compare, no cache needed. match repo.snapshot_files( files, Some(tip), @@ -128,7 +133,7 @@ pub(crate) fn harvest_workspace( None, None, sparse, - None, + cache, author, message, ) { @@ -286,25 +291,36 @@ impl Repo { // guarantees no early return can occur while children are running — // an interleaved failure would orphan live agents and race the // teardown guard's remove_dir_all underneath them. + // One EPHEMERAL protected-cache per temp checkout (P33 Task 7): `root` + // is the checkout dir, `path` is `None` so nothing persists (a `sc + // work` session leaves zero residue outside `.sc/`). Threaded from + // this materialize loop into the harvest loop below so an untouched + // randomized protected path carries the base blob instead of resealing. + let key = crate::cache::local_key(&self.layout)?; let mut dirs = Vec::with_capacity(labels.len()); + let mut caches = Vec::with_capacity(labels.len()); for label in &labels { let dir = session_root.join(label); + let mut cache = crate::cache::ProtectedCache::open(key, dir.clone(), None); let skipped = materialize_workspace( self, tip, &dir, opts.identity.as_ref(), &crate::sparse::Sparse::default(), + Some(&mut cache), )?; for path in &skipped { eprintln!("workspace {label}: skipped (no key): {path}"); } dirs.push(dir); + caches.push(cache); } - // Spawn all agents first (they run concurrently), then await each. + // Spawn all agents first (they run concurrently), then await each. The + // ephemeral cache travels alongside each (label, dir) into the harvest. let mut children = Vec::with_capacity(labels.len()); - for (label, dir) in labels.iter().zip(dirs) { + for ((label, dir), cache) in labels.iter().zip(dirs).zip(caches) { let mut c = std::process::Command::new(exe); c.args(args) .current_dir(&dir) @@ -313,11 +329,11 @@ impl Repo { for (k, v) in &secret_envs { c.env(k, v); } - children.push((label.clone(), dir, c.spawn())); + children.push((label.clone(), dir, cache, c.spawn())); } let mut outcomes = Vec::with_capacity(children.len()); - for (label, dir, spawn) in children { + for (label, dir, mut cache, spawn) in children { let agent_exit = match spawn { Ok(mut child) => child.wait().ok().and_then(|s| s.code()), Err(e) => { @@ -333,6 +349,7 @@ impl Repo { &opts.author, &message, &crate::sparse::Sparse::default(), + Some(&mut cache), ); outcomes.push(WorkspaceOutcome { label, @@ -398,7 +415,7 @@ mod tests { let tip = repo.head_tip().unwrap().unwrap(); let dir = scratch.join("ws1"); let skipped = - materialize_workspace(&repo, tip, &dir, None, &crate::sparse::Sparse::default()) + materialize_workspace(&repo, tip, &dir, None, &crate::sparse::Sparse::default(), None) .unwrap(); assert!(skipped.is_empty()); assert_eq!( @@ -415,6 +432,7 @@ mod tests { "test", "msg", &crate::sparse::Sparse::default(), + None, ) .unwrap(); let id = match res { @@ -438,7 +456,7 @@ mod tests { let repo = Repo::open(&root).unwrap(); let tip = repo.head_tip().unwrap().unwrap(); let dir = scratch.join("ws1"); - materialize_workspace(&repo, tip, &dir, None, &crate::sparse::Sparse::default()).unwrap(); + materialize_workspace(&repo, tip, &dir, None, &crate::sparse::Sparse::default(), None).unwrap(); let res = harvest_workspace( &repo, tip, @@ -447,6 +465,7 @@ mod tests { "test", "msg", &crate::sparse::Sparse::default(), + None, ) .unwrap(); assert!(matches!(res, HarvestResult::Unchanged)); @@ -464,7 +483,7 @@ mod tests { let repo = Repo::open(&root).unwrap(); let tip = repo.head_tip().unwrap().unwrap(); let dir = scratch.join("ws1"); - materialize_workspace(&repo, tip, &dir, None, &crate::sparse::Sparse::default()).unwrap(); + materialize_workspace(&repo, tip, &dir, None, &crate::sparse::Sparse::default(), None).unwrap(); // An AWS-style key id trips the P5 pattern rules. std::fs::write(dir.join("leak.txt"), "AKIAIOSFODNN7EXAMPLE\n").unwrap(); let res = harvest_workspace( @@ -475,6 +494,7 @@ mod tests { "test", "msg", &crate::sparse::Sparse::default(), + None, ) .unwrap(); assert!(matches!(res, HarvestResult::Rejected(_))); diff --git a/crates/repo/src/worktree.rs b/crates/repo/src/worktree.rs index 18642089..36c97b2d 100644 --- a/crates/repo/src/worktree.rs +++ b/crates/repo/src/worktree.rs @@ -482,7 +482,7 @@ pub struct Diff { /// `disable_sparse`'s preflight), so this walk never touches a gapped /// out-of-filter object — an out-of-filter path was never materialized to /// disk either, so it's expected-absent, not a deletion. -pub fn diff_worktree( +pub(crate) fn diff_worktree( layout: &Layout, store: &mut Store, head_root: Option, @@ -516,9 +516,9 @@ pub fn diff_worktree( // Randomized entry (P33): re-encrypting proves nothing. The // stat cache (stat hit, then keyed-tag fallback) is the only // public-key-free unchanged proof; a missing entry reports - // modified — spurious-but-safe, never incorrect. - let abs = safe_join(&layout.root, p)?; - match cache.and_then(|c| c.unchanged(p, &abs, bytes)) { + // modified — spurious-but-safe, never incorrect. The cache + // resolves the abs stat path against its own checkout root. + match cache.and_then(|c| c.unchanged(p, bytes)) { Some(id) => id, None => { diff.modified.push(p.clone()); @@ -595,7 +595,7 @@ pub(crate) fn safe_join(root: &Path, rel: &str) -> Result { /// paths `sparse` no longer matches. This closes the crash a bare /// `set_sparse`/`disable_sparse` narrowing (or a `switch`) used to hit on /// any partial clone with a genuine out-of-filter gap elsewhere in the tree. -pub fn materialize( +pub(crate) fn materialize( layout: &Layout, store: &mut Store, target_root: ObjectId, @@ -673,7 +673,7 @@ pub fn materialize( // make (`repo.rs`'s `snapshot_files`). if perms & scl_core::RANDOMIZED != 0 { if let Some(c) = cache.as_deref_mut() { - c.record(path, &full, &pt, *blob_id); + c.record(path, &pt, *blob_id); } } // `pt` (Zeroizing>) is dropped and zeroed here. @@ -894,7 +894,7 @@ mod tests { std::fs::write(&abs, &pt).unwrap(); let key = crate::cache::local_key(&layout).unwrap(); - let mut cache = crate::cache::ProtectedCache::open(key, None); + let mut cache = crate::cache::ProtectedCache::open(key, layout.root.clone(), None); // No cache entry: spurious-but-safe MODIFIED. let d = diff_worktree( @@ -909,7 +909,7 @@ mod tests { assert_eq!(d.modified, vec!["secret/x"]); // Recorded: provably clean. - cache.record("secret/x", &abs, &pt, blob_id); + cache.record("secret/x", &pt, blob_id); let d = diff_worktree( &layout, &mut store, @@ -938,4 +938,71 @@ mod tests { std::fs::remove_dir_all(&layout.root).unwrap(); assert!(!layout.root.exists()); } + + #[test] + fn materialize_populates_the_cache_for_decrypted_files() { + // P33 Task 7 Step 1 pin: materializing a RANDOMIZED protected entry + // for an authorized identity must record the decrypted plaintext into + // the threaded cache, so the very next `unchanged` on the untouched + // on-disk plaintext proves it clean (a randomized seal can't be proven + // unchanged by re-encrypting). + let (layout, mut store) = tmp_objects("p33-mat-cache"); + let (sk, pk) = scl_crypto::generate_keypair(); + let pt = b"top secret".to_vec(); + let (cipher, dek) = scl_crypto::encrypt_path_randomized(&pt); + let blob = Object::blob(cipher); + let blob_id = blob.id(); + store.put(blob).unwrap(); + let inner = store + .put(Object::Tree(Tree::new(vec![TreeEntry { + name: "x".into(), + kind: EntryKind::Blob, + id: blob_id, + mode: FileMode::FILE, + perms: PROTECTED | scl_core::RANDOMIZED, + }]))) + .unwrap(); + let root = store + .put(Object::Tree(Tree::new(vec![TreeEntry { + name: "secret".into(), + kind: EntryKind::Tree, + id: inner, + mode: FileMode::FILE, + perms: 0, + }]))) + .unwrap(); + + let mut wrapped = BTreeMap::new(); + wrapped.insert(blob_id, vec![scl_crypto::wrap_dek_for(&dek, &pk)]); + let prot = Protection { + prefixes: Vec::new(), + wrapped, + }; + + let key = crate::cache::local_key(&layout).unwrap(); + let mut cache = crate::cache::ProtectedCache::open(key, layout.root.clone(), None); + let skipped = materialize( + &layout, + &mut store, + root, + None, + &prot, + Some(&sk), + &Sparse::default(), + Some(&mut cache), + ) + .unwrap(); + assert!(skipped.is_empty(), "authorized identity decrypts, nothing skipped"); + assert_eq!( + std::fs::read(layout.root.join("secret/x")).unwrap(), + pt, + "plaintext materialized on disk" + ); + // The cache now proves the untouched plaintext unchanged. + assert_eq!(cache.unchanged("secret/x", &pt), Some(blob_id)); + + drop(store); + std::fs::remove_dir_all(&layout.root).unwrap(); + assert!(!layout.root.exists()); + } } diff --git a/crates/repo/src/ws.rs b/crates/repo/src/ws.rs index 878778d7..04719b18 100644 --- a/crates/repo/src/ws.rs +++ b/crates/repo/src/ws.rs @@ -232,18 +232,37 @@ impl Repo { // No manifest proves any .sc/ws content is crash residue from a // killed fork — always safe to clear before materializing. let _ = std::fs::remove_dir_all(&root); + // Per-workspace P33 cache key (host's `.sc/local-key`), minted once. + let key = crate::cache::local_key(self.layout())?; let mut workspaces = Vec::with_capacity(agents as usize); for i in 1..=agents { let dir = root.join(i.to_string()); - if let Err(e) = - crate::workspace::materialize_workspace(self, tip, &dir, identity, &sparse) - { + // Persistent per-workspace cache (P33 Task 7): `root` is THIS + // checkout dir, file at `.sc/ws/cache-` (beside the checkout, + // under `ws_dir`, so the whole-session teardown removes it too). + // Records every randomized protected path decrypted at fork so a + // later `sc ws harvest` (a separate invocation) carries the base + // blob instead of re-sealing an untouched file. + let mut cache = crate::cache::ProtectedCache::open( + key, + dir.clone(), + Some(self.layout().ws_cache_path(i as usize)), + ); + if let Err(e) = crate::workspace::materialize_workspace( + self, + tip, + &dir, + identity, + &sparse, + Some(&mut cache), + ) { // Nothing announced yet (no manifest written) — tear down // whatever partial checkouts exist so a failed fork leaves // no residue under .sc/ws. let _ = std::fs::remove_dir_all(&root); return Err(e); } + cache.save_best_effort(); workspaces.push(WsEntry { index: i, dir, @@ -374,6 +393,9 @@ impl Repo { .ok_or_else(|| Error::InvalidArgument(format!("no such workspace: {i}")))?; if entry.live { let _ = std::fs::remove_dir_all(&entry.dir); + // Drop the workspace's P33 stat cache too (sibling file, + // not under the checkout dir) — P33 Task 7 item 3. + let _ = std::fs::remove_file(self.layout().ws_cache_path(entry.index as usize)); entry.live = false; } } @@ -622,6 +644,8 @@ impl Repo { } } + // Per-workspace P33 cache key (host's `.sc/local-key`), minted once. + let key = crate::cache::local_key(self.layout())?; let mut outcomes = Vec::with_capacity(live_indices.len()); for i in live_indices { let dir = session @@ -666,7 +690,19 @@ impl Repo { // `ws_changed_for` above, now threaded into the commit carry too // via `harvest_workspace` -> `snapshot_files`. let fork_time_sparse = crate::sparse::Sparse::new(session.sparse.clone()); - match crate::workspace::harvest_workspace( + // The workspace's persistent P33 cache (`root` = this checkout dir, + // file at `.sc/ws/cache-`), threaded through the harvest so an + // untouched randomized protected path carries the base blob instead + // of re-sealing. Saved best-effort right after harvest: if the entry + // resolves (teardown removes the cache file too), the save is + // immediately undone; if it stays live (Rejected/FallbackBranch), + // the next harvest gets a warm cache. + let mut cache = crate::cache::ProtectedCache::open( + key, + dir.clone(), + Some(self.layout().ws_cache_path(i as usize)), + ); + let harvested = crate::workspace::harvest_workspace( self, session.base_snapshot, &dir, @@ -674,7 +710,10 @@ impl Repo { author, &msg, &fork_time_sparse, - )? { + Some(&mut cache), + )?; + cache.save_best_effort(); + match harvested { crate::workspace::HarvestResult::Rejected(report) => { // DESIGN DECISION (spec precision note, Task 5): a // scanner-rejected workspace stays LIVE so the offending @@ -823,6 +862,10 @@ fn resolve_and_teardown( // io error with no recorded recovery path). write_manifest(layout, session)?; let _ = std::fs::remove_dir_all(dir); + // Drop this workspace's P33 stat cache alongside its checkout dir (P33 + // Task 7 item 3): the cache file lives beside the dir, not inside it, so + // removing the dir doesn't remove it. + let _ = std::fs::remove_file(layout.ws_cache_path(index as usize)); Ok(()) } @@ -1179,6 +1222,74 @@ mod tests { std::fs::remove_dir_all(&root).unwrap(); } + #[test] + fn untouched_protected_files_do_not_conflict_across_workspaces() { + // P33 Task 7 Step 4 — the false-conflict hazard the per-workspace + // cache exists to prevent. Two workspaces forked WITH identity, each + // editing a different PLAIN file, both leaving a RANDOMIZED protected + // file untouched. Both must land cleanly (no `work-` fallback) AND + // carry the base's EXACT protected blob id. Without a per-workspace + // cache, harvest would re-seal the untouched randomized path to a + // fresh ciphertext id (a randomized seal differs every time), which + // the merge would then adopt as a spurious "change". + let root = tmp_root("ws-untouched-protected"); + let repo = init(&root); + std::fs::write(root.join("p1.txt"), "one\n").unwrap(); + std::fs::write(root.join("p2.txt"), "two\n").unwrap(); + let (sk, pk) = scl_crypto::generate_keypair(); + repo.protect("secret/", &[pk], None).unwrap(); + std::fs::create_dir_all(root.join("secret")).unwrap(); + std::fs::write(root.join("secret/x"), "the secret\n").unwrap(); + repo.commit("t", "add plain + protected").unwrap(); + let base = repo.head_tip().unwrap().unwrap(); + + // Base's protected blob id, and PROOF it is RANDOMIZED — otherwise a + // convergent carry-by-re-encrypt would pass this test without ever + // touching the cache (the whole point of the regression). + let base_secret_id = { + let a = repo.vfs().store(); + let mut s = a.lock().unwrap(); + let base_root = s.get_snapshot(&base).unwrap().root; + let entries = worktree::tree_file_entries_with_perms(&mut s, base_root).unwrap(); + let (id, _mode, perms) = *entries.get("secret/x").expect("protected file present"); + assert!( + perms & scl_core::PROTECTED != 0 && perms & scl_core::RANDOMIZED != 0, + "the protected file must be RANDOMIZED for this regression to exercise the cache" + ); + id + }; + + // Fork two workspaces WITH identity: the protected file materializes + // decrypted AND is recorded in each workspace's own cache at fork. + let session = repo.ws_fork(2, "t", Some(&sk)).unwrap(); + let dir1 = session.workspaces[0].dir.clone(); + let dir2 = session.workspaces[1].dir.clone(); + // Each workspace edits ONLY its own plain file; secret/x is untouched. + std::fs::write(dir1.join("p1.txt"), "one edited\n").unwrap(); + std::fs::write(dir2.join("p2.txt"), "two edited\n").unwrap(); + + let outcomes = repo.ws_harvest(None, "t", Some(&sk)).unwrap(); + assert_eq!(outcomes.len(), 2); + for o in &outcomes { + let merged_tip = match o { + WsHarvestOutcome::Landed { merged_tip, .. } => *merged_tip, + other => panic!("expected Landed (no work- fallback), got {other:?}"), + }; + let a = repo.vfs().store(); + let mut s = a.lock().unwrap(); + let mroot = s.get_snapshot(&merged_tip).unwrap().root; + let entries = worktree::tree_file_entries_with_perms(&mut s, mroot).unwrap(); + let (id, _m, _p) = *entries.get("secret/x").unwrap(); + assert_eq!( + id, base_secret_id, + "an untouched randomized protected file must carry the base blob id, not re-seal" + ); + } + + drop(repo); + std::fs::remove_dir_all(&root).unwrap(); + } + #[test] fn gc_roots_ws_base_snapshot() { let root = tmp_root("gcroot"); From 7f1018a91c02e57584556617233628b0173980b1 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 21:23:26 +0300 Subject: [PATCH 11/22] P33: thread workspace cache into ws_changed_for ws_changed_for passed None as the ProtectedCache to diff_worktree, so an untouched RANDOMIZED protected file in a workspace always reported "changed". That fed any_changed in ws_harvest's preflight, refusing the whole harvest for a dirty host tree even when every workspace was a genuine zero-edit no-op, defeating the documented affordance. Now opens the workspace's own persistent cache (same open-call shape as ws_harvest), degrading to None only if local_key can't be loaded. Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/repo/src/ws.rs | 82 +++++++++++++++++++++++++++++++++++++++---- 1 file changed, 75 insertions(+), 7 deletions(-) diff --git a/crates/repo/src/ws.rs b/crates/repo/src/ws.rs index 04719b18..2ced0ccc 100644 --- a/crates/repo/src/ws.rs +++ b/crates/repo/src/ws.rs @@ -316,16 +316,29 @@ impl Repo { let sparse = crate::sparse::Sparse::new(session.sparse.clone()); let store_arc = self.vfs().store(); let mut store = store_arc.lock().unwrap(); - // No workspace-local cache is threaded in yet (P33 Task 7 wires it); - // until then a randomized protected path in a workspace always - // reports modified — spurious-but-safe. + // The workspace's own persistent P33 cache, read-only (this is a + // status/list/preflight query, never a harvest, so nothing is + // recorded or saved back) — same open-call shape as `ws_harvest` + // (below), rooted at the workspace's own checkout dir so an + // untouched RANDOMIZED protected path is recognized as unchanged + // instead of always reporting modified. Degrades to `None` (never + // errors) if the host's cache key can't be loaded/minted: this must + // stay a non-fatal query path, and a lost cache only ever costs the + // stat/tag short-circuit, never correctness. + let cache = crate::cache::local_key(self.layout()).ok().map(|key| { + crate::cache::ProtectedCache::open( + key, + entry.dir.clone(), + Some(self.layout().ws_cache_path(entry.index as usize)), + ) + }); let d = worktree::diff_worktree( &ws, &mut store, Some(base.root), &base.protection, &sparse, - None, + cache.as_ref(), )?; Ok(!(d.added.is_empty() && d.modified.is_empty() && d.deleted.is_empty())) } @@ -729,11 +742,18 @@ impl Repo { }); } crate::workspace::HarvestResult::Unchanged => { - // `ws_changed` already confirmed a diff, but + // `ws_changed_for` already confirmed a diff above, and // `harvest_workspace` re-diffs independently against its // own `tip` argument (session.base_snapshot, same value - // here) — so this arm is unreachable in practice, kept - // only so the match is exhaustive without a panic. + // here) using the SAME per-workspace cache — so the two + // checks agree in the common case, but this arm is + // genuinely reachable: a cold or lost cache (e.g. this + // workspace's cache file was never written, or the host's + // `.sc/local-key` changed) can make `ws_changed_for` + // spuriously report "changed" for an untouched + // RANDOMIZED protected path while `harvest_workspace`'s + // own diff — same inputs, same cache lookup — still + // resolves it as unchanged. Handled, not a panic guard. resolve_and_teardown(self.layout(), &mut session, i, &dir, None)?; outcomes.push(WsHarvestOutcome::Unchanged { index: i }); } @@ -1290,6 +1310,54 @@ mod tests { std::fs::remove_dir_all(&root).unwrap(); } + #[test] + fn ws_changed_for_ignores_untouched_randomized_protected_file() { + // P33 Task 7 (ws_changed_for arm): before this fix, `ws_changed_for` + // always passed `None` for the cache, so a RANDOMIZED protected + // path — never bit-for-bit reproducible by re-encrypting the same + // plaintext — always compared as "different" even when genuinely + // untouched, reporting the workspace "changed" for no real edit. + // With the workspace's own persistent cache threaded in (recorded + // at fork time, same as `harvest_workspace` uses), an untouched + // randomized protected file must resolve as unchanged. + let root = tmp_root("ws-changed-for-randomized"); + let repo = init(&root); + let (sk, pk) = scl_crypto::generate_keypair(); + repo.protect("secret/", &[pk], None).unwrap(); + std::fs::create_dir_all(root.join("secret")).unwrap(); + std::fs::write(root.join("secret/x"), "the secret\n").unwrap(); + repo.commit("t", "add protected").unwrap(); + let base = repo.head_tip().unwrap().unwrap(); + + // Confirm RANDOMIZED, mirroring the sibling harvest-level regression + // — otherwise this test would pass even without the fix. + { + let a = repo.vfs().store(); + let mut s = a.lock().unwrap(); + let base_root = s.get_snapshot(&base).unwrap().root; + let entries = worktree::tree_file_entries_with_perms(&mut s, base_root).unwrap(); + let (_id, _mode, perms) = *entries.get("secret/x").expect("protected file present"); + assert!( + perms & scl_core::PROTECTED != 0 && perms & scl_core::RANDOMIZED != 0, + "the protected file must be RANDOMIZED for this regression to exercise the cache" + ); + } + + // Fork WITH identity so the protected file materializes decrypted + // and is recorded in the workspace's own cache at fork time; leave + // it completely untouched. + let session = repo.ws_fork(1, "t", Some(&sk)).unwrap(); + assert!( + !repo + .ws_changed_for(&session, &session.workspaces[0]) + .unwrap(), + "an untouched RANDOMIZED protected file must not report the workspace as changed" + ); + + drop(repo); + std::fs::remove_dir_all(&root).unwrap(); + } + #[test] fn gc_roots_ws_base_snapshot() { let root = tmp_root("gcroot"); From 49f171b0edc810a3624493bf3fe9a60c009bdd71 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 21:32:33 +0300 Subject: [PATCH 12/22] P33: sc resolve records resolved protected content in the cache Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/repo/src/conflicts.rs | 85 +++++++++++++++++++++++++++++++++++- 1 file changed, 84 insertions(+), 1 deletion(-) diff --git a/crates/repo/src/conflicts.rs b/crates/repo/src/conflicts.rs index 7d7b24f7..bc27a815 100644 --- a/crates/repo/src/conflicts.rs +++ b/crates/repo/src/conflicts.rs @@ -221,6 +221,10 @@ impl Repo { ResolveSide::Ours => versions.ours, ResolveSide::Theirs => versions.theirs, }; + // Computed once and reused below for the sidecar-cleanup check too — + // `conflict_kind` re-derives from the DAG and is safe to call before + // the write. + let kind = self.conflict_kind(path)?; let full = safe_join(&self.layout.root, path)?; match chosen { @@ -229,6 +233,35 @@ impl Repo { std::fs::create_dir_all(parent)?; } std::fs::write(&full, &bytes)?; + + // P33 Task 8: a resolved PROTECTED path's plaintext is now + // sitting on disk exactly as the chosen side's ciphertext + // decrypted it — record it in the main-tree cache so the + // completing `sc commit`/`sc rebase --continue` recognizes it + // as unchanged (`unchanged(path, plaintext) == Some(blob_id)`) + // and carries the existing ciphertext instead of re-sealing + // under a fresh random nonce. Plain paths never touch the + // cache — it exists only to skip re-sealing convergent-free + // randomized ciphertext. + if kind == ConflictKind::Protected { + let triple = self.op_triple()?; + let chosen_snapshot = match side { + ResolveSide::Ours => triple.ours, + ResolveSide::Theirs => triple.theirs, + }; + let store_arc = self.vfs.store(); + let blob_id = { + let mut store = store_arc.lock().unwrap(); + let snap = store.get_snapshot(&chosen_snapshot)?; + let entries = tree_file_entries_with_perms(&mut store, snap.root)?; + entries.get(path).map(|(id, _, _)| *id) + }; + if let Some(blob_id) = blob_id { + let mut cache = self.open_protected_cache()?; + cache.record(path, &bytes, blob_id); + cache.save_best_effort(); + } + } } Side::Absent => match std::fs::remove_file(&full) { Ok(()) => {} @@ -255,7 +288,7 @@ impl Repo { // remove it when it is NOT a tracked path — a genuine sidecar is // untracked scratch by construction, so a tracked `{path}.theirs` // must be a user's real file and is left alone. - if self.conflict_kind(path)? != ConflictKind::Text { + if kind != ConflictKind::Text { let theirs_sidecar_path = format!("{path}.theirs"); if !self.tracked_paths()?.contains(&theirs_sidecar_path) { let sidecar = safe_join(&self.layout.root, &theirs_sidecar_path)?; @@ -1005,6 +1038,56 @@ mod tests { std::fs::remove_dir_all(&root).ok(); } + #[test] + fn resolve_ours_records_protected_content_in_main_cache() { + let root = tmp_root("resolve-cache"); + let repo = Repo::init(&root).unwrap(); + let (alice_sk, alice_pk) = scl_crypto::generate_keypair(); + repo.protect("secret/", &[alice_pk], None).unwrap(); + std::fs::create_dir_all(root.join("secret")).unwrap(); + std::fs::write(root.join("secret/a.txt"), b"base\n").unwrap(); + repo.commit("me", "base").unwrap(); + repo.branch("feature").unwrap(); + + std::fs::write(root.join("secret/a.txt"), b"ours\n").unwrap(); + repo.commit("me", "ours edits secret").unwrap(); + repo.switch_with_identity("feature", Some(&alice_sk)) + .unwrap(); + std::fs::write(root.join("secret/a.txt"), b"theirs\n").unwrap(); + repo.commit("me", "theirs edits secret").unwrap(); + repo.switch_with_identity("main", Some(&alice_sk)).unwrap(); + + let err = repo + .merge_with_identity("feature", "me", Some(&alice_sk)) + .unwrap_err(); + assert!(matches!(err, Error::MergeConflicts(1)), "got {err:?}"); + + repo.resolve_path("secret/a.txt", ResolveSide::Ours, Some(&alice_sk)) + .unwrap(); + + // The ours-side ciphertext blob id, re-derived straight from the DAG + // (HEAD is still the "ours" commit — a merge in progress never moves + // it) — this is what the cache must map "ours\n" to. + let ours_tip = repo.head_tip().unwrap().unwrap(); + let ours_blob_id = { + let store_arc = repo.vfs().store(); + let mut store = store_arc.lock().unwrap(); + let snap = store.get_snapshot(&ours_tip).unwrap(); + let entries = tree_file_entries_with_perms(&mut store, snap.root).unwrap(); + entries.get("secret/a.txt").unwrap().0 + }; + + let cache = repo.open_protected_cache().unwrap(); + assert_eq!( + cache.unchanged("secret/a.txt", b"ours\n"), + Some(ours_blob_id), + "resolve must record the resolved plaintext -> chosen ciphertext id in the main cache" + ); + + drop(repo); + std::fs::remove_dir_all(&root).ok(); + } + #[test] fn resolved_merge_completes_via_commit() { let root = tmp_root("resolve-completes"); From ce24679319b7f0ebb32c896c691b3bf0a6e28b1d Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 21:41:29 +0300 Subject: [PATCH 13/22] P33: keyed tag is authoritative in cache lookups (racy-stat hardening) ProtectedCache::unchanged() previously returned a cache hit on a bare mtime+size stat match without checking the keyed tag. An edit that keeps the same file size and lands within the filesystem's mtime granularity (the classic git "racy stat" problem) was misread as unchanged, silently carrying the old ciphertext into the commit and dropping the user's edit. The stat short-circuit is removed; every unchanged() call now requires keyed-tag equality, which every caller can afford since the plaintext is already in memory. mtime/size are still recorded per entry as advisory bookkeeping only. Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/repo/src/cache.rs | 95 ++++++++++++++++++++++++++++++---------- 1 file changed, 73 insertions(+), 22 deletions(-) diff --git a/crates/repo/src/cache.rs b/crates/repo/src/cache.rs index 9db25137..2ac1213f 100644 --- a/crates/repo/src/cache.rs +++ b/crates/repo/src/cache.rs @@ -9,6 +9,18 @@ //! nothing — it does not reintroduce the equality oracle this phase closes. //! A lost/stale/corrupt cache degrades to spurious re-seals, never //! incorrectness and never a hard error. +//! +//! **The keyed tag is the sole authority for a hit.** `mtime_ns`/`size` are +//! recorded for documentation (when the entry was observed) and as a +//! possible future input to a paths-only walk that doesn't already hold the +//! plaintext in RAM, but they are NEVER sufficient on their own: a bare +//! stat match is the classic git "racy stat" problem (an edit that lands +//! within the filesystem's mtime granularity and happens to keep the same +//! size is indistinguishable from no edit at all by stat alone), and every +//! `unchanged()` caller already has the plaintext in memory (`read_worktree` +//! loads all bytes before commit/diff), so there is no reason to trust a +//! cheaper, unsound signal — keyed blake3 over in-memory bytes runs at +//! GB/s. use std::collections::BTreeMap; use std::path::{Path, PathBuf}; @@ -87,17 +99,19 @@ struct CacheEntry { /// an ephemeral in-memory one for `sc work` temp checkouts). /// /// The cache owns its `root`: the absolute base directory of the checkout it -/// serves, against which relative tree paths are resolved for the stat -/// (mtime+size) short-circuit. This is the P33 Task 7 design choice (over -/// threading an explicit worktree-root through `snapshot_files`): a stat cache -/// is inherently tied to one checkout, so it should carry that checkout's root -/// rather than have every `unchanged`/`record` caller pass an absolute path. -/// **Invariant:** the `root` passed at `open` must equal `layout.root` of the -/// checkout that `materialize`/`diff_worktree`/`snapshot_files` operate on — -/// the main tree opens with `layout.root`, a `sc ws`/`sc work` workspace with -/// its own checkout dir. A mismatched root only ever loses the stat -/// short-circuit (the keyed-tag comparison still proves unchanged-ness, and a -/// miss degrades to a spurious reseal) — never incorrectness. +/// serves, against which relative tree paths are resolved to record the +/// advisory mtime+size fields alongside each entry. This is the P33 Task 7 +/// design choice (over threading an explicit worktree-root through +/// `snapshot_files`): the cache is inherently tied to one checkout, so it +/// should carry that checkout's root rather than have every +/// `unchanged`/`record` caller pass an absolute path. **Invariant:** the +/// `root` passed at `open` must equal `layout.root` of the checkout that +/// `materialize`/`diff_worktree`/`snapshot_files` operate on — the main tree +/// opens with `layout.root`, a `sc ws`/`sc work` workspace with its own +/// checkout dir. The keyed-tag comparison that decides every `unchanged()` +/// hit does not depend on `root` at all (it hashes the caller's in-memory +/// plaintext), so a mismatched root only ever loses the advisory stat +/// bookkeeping — never incorrectness. pub(crate) struct ProtectedCache { key: [u8; 32], /// Absolute base dir of the served checkout; `root.join(rel)` is the file @@ -152,16 +166,17 @@ impl ProtectedCache { } /// The cached ciphertext blob id iff this exact plaintext is what last - /// sealed at `rel`: stat hit (mtime+size unchanged) short-circuits; - /// otherwise fall back to the keyed-tag comparison. The absolute file is - /// resolved against the cache's own `root` (`root.join(rel)`). + /// sealed at `rel`, decided ENTIRELY by the keyed-tag comparison — never + /// by a bare mtime+size stat match. A stat-only hit is the classic git + /// "racy stat" problem: an edit that keeps the same file size and lands + /// within the filesystem's mtime granularity is indistinguishable from + /// no edit at all by stat alone, and misreading it as unchanged would + /// silently carry the OLD ciphertext into the commit, dropping the + /// user's edit — exactly the incorrectness this cache must never cause. + /// Every caller already holds `plaintext` in memory before this is + /// called, so the keyed tag is cheap enough to be the sole authority. pub(crate) fn unchanged(&self, rel: &str, plaintext: &[u8]) -> Option { let e = self.entries.get(rel)?; - if let Some((mtime_ns, size)) = Self::stat(&self.root.join(rel)) { - if mtime_ns == e.mtime_ns && size == e.size { - return Some(e.blob_id); - } - } (self.tag(plaintext) == e.tag).then_some(e.blob_id) } @@ -248,7 +263,7 @@ mod tests { } #[test] - fn stat_hit_tag_fallback_and_miss() { + fn tag_is_authoritative_for_hits_and_misses() { let layout = tmp("hits"); let key = local_key(&layout).unwrap(); let abs = layout.root.join("secret.txt"); @@ -260,10 +275,10 @@ mod tests { assert_eq!(c.unchanged("secret.txt", b"v1"), None, "empty cache misses"); c.record("secret.txt", b"v1", id); - // Stat hit: file untouched. + // Untouched file, same plaintext: hit. assert_eq!(c.unchanged("secret.txt", b"v1"), Some(id)); - // Tag fallback: touch mtime without changing content. + // Touch mtime without changing content: still a hit (tag matches). std::fs::write(&abs, b"v1").unwrap(); assert_eq!(c.unchanged("secret.txt", b"v1"), Some(id)); @@ -273,6 +288,42 @@ mod tests { cleanup(&layout); } + /// Regression for the racy-stat data-loss finding: same-length content + /// written back with the SAME recorded mtime (the classic git "racy + /// stat" condition — an edit that lands within the filesystem's mtime + /// granularity) must still miss, because a stat-only hit would silently + /// carry the old ciphertext and drop the user's edit from the commit. + #[test] + fn racy_stat_same_size_same_mtime_still_misses() { + let layout = tmp("racy"); + let key = local_key(&layout).unwrap(); + let abs = layout.root.join("secret.txt"); + std::fs::write(&abs, b"aaaa").unwrap(); + let id = ObjectId::of(b"cipher-of-aaaa"); + let original_mtime = std::fs::metadata(&abs).unwrap().modified().unwrap(); + + let mut c = + ProtectedCache::open(key, layout.root.clone(), Some(layout.protected_cache_path())); + c.record("secret.txt", b"aaaa", id); + + // Overwrite with same-length content, then force the racy condition + // by resetting mtime back to what was recorded. + std::fs::write(&abs, b"bbbb").unwrap(); + std::fs::File::options() + .write(true) + .open(&abs) + .unwrap() + .set_modified(original_mtime) + .unwrap(); + + assert_eq!( + c.unchanged("secret.txt", b"bbbb"), + None, + "racy stat (same mtime, same size, different content) must not be misread as unchanged" + ); + cleanup(&layout); + } + #[test] fn save_load_roundtrip_and_corrupt_file_is_empty() { let layout = tmp("persist"); From 3a0a2962b70fb8b9944580b38075c41548af29c5 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 21:54:45 +0300 Subject: [PATCH 14/22] P33: pin merge/replay behavior over mixed convergent+randomized trees Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/repo/src/merge.rs | 154 +++++++++++++++++++++++++++++++++- crates/repo/src/replay.rs | 171 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 324 insertions(+), 1 deletion(-) diff --git a/crates/repo/src/merge.rs b/crates/repo/src/merge.rs index 2c3232d1..cd4cfdf3 100644 --- a/crates/repo/src/merge.rs +++ b/crates/repo/src/merge.rs @@ -758,7 +758,7 @@ mod tests { // ---- P15: perms-aware three_way_files ---- - use scl_core::{Protection, PROTECTED}; + use scl_core::{Protection, PROTECTED, RANDOMIZED}; /// Convergently encrypt `content`, wrap its DEK for `pk`, and register the /// wrap in `prot` under the ciphertext blob's id. Returns the ciphertext. @@ -772,6 +772,18 @@ mod tests { cipher } + /// Randomized twin of `enc_file` (P33): seals via `encrypt_path_randomized`, + /// so identical plaintext yields a fresh ciphertext/id on every call. + fn enc_file_rand(prot: &mut Protection, pk: &scl_crypto::PublicKey, content: &[u8]) -> Vec { + let (cipher, dek) = scl_crypto::encrypt_path_randomized(content); + let id = Object::blob(cipher.clone()).id(); + prot.wrapped + .entry(id) + .or_default() + .push(scl_crypto::wrap_dek_for(&dek, pk)); + cipher + } + /// Write a tree whose files carry explicit `perms` bytes; returns its root id. fn tree_with_perms(repo: &VfsRepo, files: &[(&str, Vec, u8)]) -> ObjectId { let fs: Vec<(String, Vec, FileMode, u8)> = files @@ -917,6 +929,146 @@ mod tests { assert_eq!(f.bytes, b"L1\nl2\nL3\n", "both edits present in plaintext"); } + // ---- P33: randomized protected-path encryption ---- + + #[test] + fn identical_independent_edits_now_conflict_and_resolve_with_identity() { + // base: secret/a.txt = "v1" (randomized). ours and theirs BOTH edit it + // to "v2" — independently, so each seal is fresh and the two + // ciphertexts/ids differ even though the plaintexts are identical. + // Pre-P33 convergent sealing would have id-matched this as a clean + // fast path; randomized sealing makes it a genuine content-divergent + // protected case. + let (alice_sk, alice_pk) = scl_crypto::generate_keypair(); + let (mut bp, mut op, mut tp) = ( + Protection::default(), + Protection::default(), + Protection::default(), + ); + let cb = enc_file_rand(&mut bp, &alice_pk, b"v1\n"); + let co = enc_file_rand(&mut op, &alice_pk, b"v2\n"); + let ct = enc_file_rand(&mut tp, &alice_pk, b"v2\n"); + assert_ne!(co, ct, "independent randomized seals of identical plaintext differ"); + + let repo = VfsRepo::new(Store::with_budget(1 << 20)); + let base = tree_with_perms(&repo, &[("secret/a.txt", cb, PROTECTED | RANDOMIZED)]); + let ours = tree_with_perms(&repo, &[("secret/a.txt", co, PROTECTED | RANDOMIZED)]); + let theirs = tree_with_perms(&repo, &[("secret/a.txt", ct, PROTECTED | RANDOMIZED)]); + let arc = repo.store(); + let mut s = arc.lock().unwrap(); + + // Without identity: the same protected-conflict/needs-identity + // outcome the existing content-divergent test expects. + let err = three_way_files(&mut s, Some((base, &bp)), (ours, &op), (theirs, &tp), None) + .unwrap_err(); + assert!( + matches!(&err, Error::ProtectedMergeNeedsIdentity(p) if p == "secret/a.txt"), + "got {err:?}" + ); + + // With identity: diff3 of identical plaintexts merges cleanly and + // the winning content re-seals randomized (needs_encrypt is set). + let fm = three_way_files( + &mut s, + Some((base, &bp)), + (ours, &op), + (theirs, &tp), + Some(&alice_sk), + ) + .unwrap(); + assert!(fm.conflicts.is_empty()); + assert_eq!(fm.files.len(), 1); + let f = &fm.files[0]; + assert_eq!(f.path, "secret/a.txt"); + assert_eq!(f.bytes, b"v2\n", "identical plaintexts merge to v2 with no conflict"); + assert!(f.needs_encrypt, "merged plaintext pending re-encryption"); + assert!(f.perms & PROTECTED != 0); + } + + #[test] + fn mixed_convergent_and_randomized_tree_merges_by_id_fast_paths() { + // base holds one plain file, one convergent protected entry + // (enc_file), and one randomized protected entry (enc_file_rand). + // ours edits only the plain file; theirs edits only the convergent + // entry. Assert the merge is clean without identity, the untouched + // randomized entry carries its exact blob id AND its + // PROTECTED|RANDOMIZED perms verbatim, and the convergent entry + // takes theirs' id. + let repo = VfsRepo::new(Store::with_budget(1 << 20)); + let (_alice_sk, alice_pk) = scl_crypto::generate_keypair(); + let (mut bp, op, mut tp) = ( + Protection::default(), + Protection::default(), + Protection::default(), + ); + let conv_base = enc_file(&mut bp, &alice_pk, b"c1\n"); + let rand_base = enc_file_rand(&mut bp, &alice_pk, b"r1\n"); + let conv_theirs = enc_file(&mut tp, &alice_pk, b"c2\n"); + + let base = tree_with_perms( + &repo, + &[ + ("plain.txt", b"p1\n".to_vec(), 0), + ("secret/conv.txt", conv_base.clone(), PROTECTED), + ("secret/rand.txt", rand_base.clone(), PROTECTED | RANDOMIZED), + ], + ); + let ours = tree_with_perms( + &repo, + &[ + ("plain.txt", b"p2\n".to_vec(), 0), + ("secret/conv.txt", conv_base.clone(), PROTECTED), + ("secret/rand.txt", rand_base.clone(), PROTECTED | RANDOMIZED), + ], + ); + let theirs = tree_with_perms( + &repo, + &[ + ("plain.txt", b"p1\n".to_vec(), 0), + ("secret/conv.txt", conv_theirs.clone(), PROTECTED), + ("secret/rand.txt", rand_base.clone(), PROTECTED | RANDOMIZED), + ], + ); + + let arc = repo.store(); + let mut s = arc.lock().unwrap(); + let fm = + three_way_files(&mut s, Some((base, &bp)), (ours, &op), (theirs, &tp), None).unwrap(); + assert!(fm.conflicts.is_empty()); + assert!( + fm.files.iter().all(|f| !f.needs_encrypt), + "every entry resolved by an id fast path -- no decryption occurred" + ); + + let plain = fm.files.iter().find(|f| f.path == "plain.txt").unwrap(); + assert_eq!(plain.bytes, b"p2\n", "ours' plain edit wins"); + + let conv = fm.files.iter().find(|f| f.path == "secret/conv.txt").unwrap(); + assert_eq!(conv.bytes, conv_theirs, "convergent entry takes theirs' id"); + assert!(conv.perms & PROTECTED != 0); + let conv_theirs_id = Object::blob(conv_theirs).id(); + assert!( + fm.wrapped_carry.contains_key(&conv_theirs_id), + "surviving convergent blob's wraps carried" + ); + + let rand = fm.files.iter().find(|f| f.path == "secret/rand.txt").unwrap(); + assert_eq!( + rand.bytes, rand_base, + "untouched randomized entry carries its exact blob id" + ); + assert_eq!( + rand.perms, + PROTECTED | RANDOMIZED, + "untouched randomized entry carries PROTECTED|RANDOMIZED perms verbatim" + ); + let rand_base_id = Object::blob(rand_base).id(); + assert!( + fm.wrapped_carry.contains_key(&rand_base_id), + "untouched randomized blob's wraps carried -- recipients can still decrypt" + ); + } + #[test] fn protected_conflict_carries_plaintext_markers() { let (alice_sk, alice_pk) = scl_crypto::generate_keypair(); diff --git a/crates/repo/src/replay.rs b/crates/repo/src/replay.rs index 9c96a7c4..5af2339e 100644 --- a/crates/repo/src/replay.rs +++ b/crates/repo/src/replay.rs @@ -2236,6 +2236,177 @@ mod tests { std::fs::remove_dir_all(&root).ok(); } + // ---- P33: randomized protected-path encryption, replay twin ---- + // + // `Repo::protect`/`commit` seal every protected file via + // `encrypt_protected`, which has sealed randomized (PROTECTED|RANDOMIZED) + // since P33 (ADR-0043) — so every real cherry-pick over protected content + // already exercises randomized ciphertext. These twins of + // `cherry_pick_disjoint_protected_commit_needs_no_identity` and + // `cherry_pick_content_divergent_requires_identity_and_reencrypts` pin + // that explicitly: the RANDOMIZED bit rides the fast path verbatim, and + // a content-divergent pick re-seals randomized on completion. + + #[test] + fn cherry_pick_randomized_protected_unchanged_fast_path_needs_no_identity() { + // The picked commit updates a protected file ours never touched: the + // ciphertext-id fast path carries the picked (randomized) blob + // verbatim — no identity needed — and the RANDOMIZED bit rides + // along with PROTECTED. + let root = tmp_root("cp-prot-rand-disjoint"); + let repo = Repo::init(&root).unwrap(); + let (alice_sk, alice_pk) = scl_crypto::generate_keypair(); + repo.protect("secret/", &[alice_pk], None).unwrap(); + std::fs::create_dir_all(root.join("secret")).unwrap(); + std::fs::write(root.join("secret/db.txt"), b"v1").unwrap(); + repo.commit("me", "base").unwrap(); + repo.branch("work").unwrap(); + + // work updates the protected file (fresh randomized seal). + repo.switch_with_identity("work", Some(&alice_sk)).unwrap(); + std::fs::write(root.join("secret/db.txt"), b"v2").unwrap(); + let picked = repo.commit("me", "work updates secret").unwrap(); + let (v2_id, v2_perms) = tree_entry(&repo, &picked, "secret/db.txt"); + assert_eq!( + v2_perms, + scl_core::PROTECTED | scl_core::RANDOMIZED, + "encrypt_protected seals PROTECTED|RANDOMIZED" + ); + + // main gains an unrelated plain commit (keyless hop back). + repo.switch("main").unwrap(); + std::fs::write(root.join("main.txt"), b"m\n").unwrap(); + let main_tip = repo.commit("me", "main adds main.txt").unwrap(); + + // KEYLESS pick: unchanged-on-ours fast path, no identity needed. + let outcome = repo.cherry_pick("work", "me", None, None).unwrap(); + let id = match outcome { + PickResult::Picked(id) => id, + other => panic!("expected Picked, got {other:?}"), + }; + assert_eq!(repo.head_tip().unwrap(), Some(id), "branch advanced"); + let snap = repo.snapshot(&id).unwrap(); + assert_eq!(snap.parents, vec![main_tip]); + + // The picked ciphertext blob is carried byte-for-byte, with its + // RANDOMIZED bit intact. + let (got_id, perms) = tree_entry(&repo, &id, "secret/db.txt"); + assert_eq!(got_id, v2_id, "ciphertext carried verbatim"); + assert_eq!( + perms, + scl_core::PROTECTED | scl_core::RANDOMIZED, + "PROTECTED|RANDOMIZED carried verbatim through the fast path" + ); + let bytes = blob_bytes_of(&repo, &got_id); + let pt = crate::protect::decrypt_with( + &bytes, + &got_id, + &[&snap.protection], + &alice_sk, + "secret/db.txt", + ) + .unwrap(); + assert_eq!(&pt[..], b"v2", "recipient decrypts the picked update"); + + drop(repo); + std::fs::remove_dir_all(&root).ok(); + } + + #[test] + fn cherry_pick_randomized_protected_content_divergent_requires_identity() { + // secret/a.txt diverged in content on both sides (mergeable lines), + // each side's seal randomized and distinct: a keyless pick fails + // with the typed error and moves nothing; with an identity the + // plaintexts are diff3'd and the merged content is re-sealed + // randomized for ALL rule recipients. + let root = tmp_root("cp-prot-rand-divergent"); + let repo = Repo::init(&root).unwrap(); + let (alice_sk, alice_pk) = scl_crypto::generate_keypair(); + let (bob_sk, bob_pk) = scl_crypto::generate_keypair(); + repo.protect("secret/", &[alice_pk, bob_pk], None).unwrap(); + std::fs::create_dir_all(root.join("secret")).unwrap(); + std::fs::write(root.join("secret/a.txt"), b"l1\nl2\nl3\n").unwrap(); + repo.commit("me", "base").unwrap(); + repo.branch("work").unwrap(); + + // ours edits line 1. + std::fs::write(root.join("secret/a.txt"), b"OURS\nl2\nl3\n").unwrap(); + let main_tip = repo.commit("me", "main edits line 1").unwrap(); + let (ours_id, _) = tree_entry(&repo, &main_tip, "secret/a.txt"); + + // work edits line 3 (mergeable divergence) — a fresh, distinct seal. + repo.switch_with_identity("work", Some(&alice_sk)).unwrap(); + std::fs::write(root.join("secret/a.txt"), b"l1\nl2\nTHEIRS\n").unwrap(); + let picked = repo.commit("me", "work edits line 3").unwrap(); + let (theirs_id, _) = tree_entry(&repo, &picked, "secret/a.txt"); + assert_ne!( + ours_id, theirs_id, + "independent randomized seals of divergent content differ" + ); + repo.switch_with_identity("main", Some(&alice_sk)).unwrap(); + + // Keyless: typed error, refs untouched, no pick state. + let err = repo.cherry_pick("work", "me", None, None).unwrap_err(); + assert!( + matches!(err, Error::ProtectedMergeNeedsIdentity(ref p) if p == "secret/a.txt"), + "got {err:?}" + ); + assert_eq!( + repo.head_tip().unwrap(), + Some(main_tip), + "tip must not move" + ); + assert!( + !repo.pick_in_progress(), + "no pick state on the identity refusal" + ); + + // With identity: clean pick, merged content re-sealed randomized. + let outcome = repo + .cherry_pick("work", "me", Some(&alice_sk), None) + .unwrap(); + let id = match outcome { + PickResult::Picked(id) => id, + other => panic!("expected Picked, got {other:?}"), + }; + let snap = repo.snapshot(&id).unwrap(); + assert_eq!(snap.parents, vec![main_tip]); + let (blob_id, perms) = tree_entry(&repo, &id, "secret/a.txt"); + assert_eq!( + perms, + scl_core::PROTECTED | scl_core::RANDOMIZED, + "re-encryption seals PROTECTED|RANDOMIZED" + ); + let bytes = blob_bytes_of(&repo, &blob_id); + assert!( + !bytes.windows(4).any(|w| w == b"OURS"), + "plaintext leaked into the CAS blob" + ); + for (who, sk) in [("alice", &alice_sk), ("bob", &bob_sk)] { + let pt = crate::protect::decrypt_with( + &bytes, + &blob_id, + &[&snap.protection], + sk, + "secret/a.txt", + ) + .unwrap(); + assert_eq!( + &pt[..], + b"OURS\nl2\nTHEIRS\n", + "{who} must decrypt the merged content" + ); + } + // The identity-holder gets the merged plaintext on disk. + assert_eq!( + std::fs::read(root.join("secret/a.txt")).unwrap(), + b"OURS\nl2\nTHEIRS\n".to_vec() + ); + + drop(repo); + std::fs::remove_dir_all(&root).ok(); + } + #[test] fn cherry_pick_protected_conflict_writes_plaintext_markers_worktree_only() { // Same-line edits of secret/a.txt on both sides: the pick conflicts. From d51cc12e2240c54ab0ef78c1a4c805a3aab23876 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 22:18:13 +0300 Subject: [PATCH 15/22] P33: sc rewrap eagerly re-seals convergent blobs randomized Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/cli/src/main.rs | 11 ++ crates/repo/src/rewrap.rs | 382 +++++++++++++++++++++++++++++++++++--- 2 files changed, 367 insertions(+), 26 deletions(-) diff --git a/crates/cli/src/main.rs b/crates/cli/src/main.rs index 5d612b0e..26b9253b 100644 --- a/crates/cli/src/main.rs +++ b/crates/cli/src/main.rs @@ -3825,6 +3825,17 @@ fn run_rewrap(identity: Option, dry_run: bool) -> Result<()> { report.secrets_rewrapped.len(), report.blobs_rewrapped ); + if report.blobs_resealed > 0 { + let reseal_verb = if dry_run { + "would re-seal" + } else { + "re-sealed" + }; + println!( + "{reseal_verb} {} convergent blob(s) → randomized", + report.blobs_resealed + ); + } if let Some(id) = &report.commit { println!("commit: {}", id.short()); } diff --git a/crates/repo/src/rewrap.rs b/crates/repo/src/rewrap.rs index 126857d2..5988c1bb 100644 --- a/crates/repo/src/rewrap.rs +++ b/crates/repo/src/rewrap.rs @@ -14,6 +14,9 @@ use crate::repo::Repo; pub struct RewrapReport { pub secrets_rewrapped: Vec, pub blobs_rewrapped: usize, + /// Convergent (pre-P33) protected blobs eagerly re-sealed randomized + /// (fresh DEK + nonce, new ciphertext id) during this run (P33 Task 10). + pub blobs_resealed: usize, /// (entry label e.g. "secret db-pass" / "path secret/db.txt", reason) pub skipped: Vec<(String, String)>, /// The commit id; None on --dry-run or when nothing needed rewrapping. @@ -24,10 +27,16 @@ impl Repo { /// Re-seal every secret (fresh DEK, current recipients + escrow) and /// replace every protected blob's wrap list (rule's granted set + escrow) /// at the tip, as ONE commit and ONE oplog record. Entries `identity` - /// cannot open are skipped and reported. Policy/registry-only: the root - /// tree id is untouched. Cuts the LIVE TIP only — history keeps old wraps - /// and old secret objects (content addressing; same boundary as rotation, - /// ADR-0019). + /// cannot open are skipped and reported. Still-convergent (pre-P33) + /// protected blobs are eagerly upgraded while we're here: decrypted with + /// the unwrapped DEK, re-sealed randomized (`encrypt_path_randomized`), + /// and the tree entry retargeted to the new ciphertext id with the + /// `RANDOMIZED` perms bit — so the root tree changes exactly when an + /// upgrade happened. Once every entry is randomized the run is + /// policy/registry-only again and the root tree id is untouched (the P17 + /// property). Cuts the LIVE TIP only — history keeps old wraps, old + /// convergent ciphertext, and old secret objects (content addressing; + /// same boundary as rotation, ADR-0019). pub fn rewrap( &self, identity: &SecretKey, @@ -121,19 +130,35 @@ impl Repo { new_secret_objs.push((name, Object::Secret(sealed))); } - // ---- Paths half: replace wrap lists with granted + escrow. ---- + // ---- Paths half: replace wrap lists with granted + escrow; eagerly + // upgrade still-convergent (pre-P33) blobs to randomized. ---- let mut protection = snap.protection.clone(); let mut blobs_rewrapped = 0usize; + let mut blobs_resealed = 0usize; + // path -> (new blob id, new perms) for eagerly-upgraded entries; the + // commit tail rebuilds the root iff this is non-empty. + let mut retargets: std::collections::BTreeMap = + std::collections::BTreeMap::new(); + // Main-tree protected cache: recording (path, plaintext, new id) for + // each upgraded path is what keeps the next `sc status`/`commit` + // quiet — rewrap never rematerializes the working tree, so the + // on-disk plaintext (and its stat) is untouched while its sealed id + // changes. Never opened on dry-run (no mutation, nothing to record). + let mut main_cache = if dry_run { + None + } else { + Some(self.open_protected_cache()?) + }; let entries = { let arc = self.store_arc(); let mut store = arc.lock().unwrap(); crate::worktree::tree_file_entries_with_perms(&mut store, snap.root)? }; - for (path, (blob_id, _mode, perms)) in entries { + for (path, (blob_id, _mode, perms)) in &entries { if perms & scl_core::PROTECTED == 0 { continue; } - let Some(rule) = crate::protect::matching_prefix(&protection, &path) else { + let Some(rule) = crate::protect::matching_prefix(&protection, path) else { skipped.push(( format!("path {path}"), "no governing rule (bit/rule mismatch)".into(), @@ -149,7 +174,7 @@ impl Repo { )); continue; } - let Some(wks) = protection.wrapped.get(&blob_id) else { + let Some(wks) = protection.wrapped.get(blob_id) else { skipped.push(( format!("path {path}"), "no wrapped DEKs recorded for blob".into(), @@ -171,15 +196,8 @@ impl Repo { continue; } }; - if dry_run { - blobs_rewrapped += 1; - continue; - } - // Rebuild the wrap list: exactly granted + escrow, reusing prior - // wrap bytes for recipients already present (id-stability), fresh - // wraps for the rest. Tombstoned/stale wraps are dropped. - let prior = wks.clone(); - let mut new_wks: Vec = Vec::new(); + // The target recipient set — exactly granted + escrow — shared + // by both arms below. let mut target_pks: Vec = granted.iter().map(|b| PublicKey::from_bytes(*b)).collect(); for e in escrows { @@ -190,6 +208,75 @@ impl Repo { target_pks.push(e.clone()); } } + if perms & scl_core::RANDOMIZED == 0 { + // Convergent (pre-P33): eager upgrade. Decrypt with the DEK + // we just unwrapped, re-seal randomized, retarget the tree + // entry + wraps to the fresh ciphertext id. + let cipher = { + let arc = self.store_arc(); + let obj = arc.lock().unwrap().get(blob_id)?; + match obj { + Object::Blob(b) => b.to_vec(), + _ => { + skipped + .push((format!("path {path}"), "tree entry is not a blob".into())); + continue; + } + } + }; + let pt = match scl_crypto::decrypt_path(&cipher, &dek) { + Ok(p) => p, + Err(e) => { + // Corruption-shaped, not an authorization failure + // (the wrap itself opened): named, not fatal. + skipped.push(( + format!("path {path}"), + format!("ciphertext failed to open: {e}"), + )); + continue; + } + }; + if dry_run { + blobs_resealed += 1; + continue; + } + let (new_cipher, new_dek) = scl_crypto::encrypt_path_randomized(&pt); + let new_id = { + let arc = self.store_arc(); + let i = arc.lock().unwrap().put(Object::blob(new_cipher))?; + i + }; + let mut new_wks: Vec = target_pks + .iter() + .map(|pk| scl_crypto::wrap_dek_for(&new_dek, pk)) + .collect(); + new_wks.sort_by(|a, b| a.recipient_id.cmp(&b.recipient_id)); + protection.wrapped.remove(blob_id); + protection.wrapped.insert(new_id, new_wks); + retargets.insert( + path.clone(), + (new_id, scl_core::PROTECTED | scl_core::RANDOMIZED), + ); + if let Some(c) = main_cache.as_mut() { + // Recorded unconditionally for upgraded paths: if the + // working file diverged locally, the keyed tag simply + // won't match its plaintext — a benign spurious reseal + // at the next commit, never incorrectness. + c.record(path, &pt, new_id); + } + blobs_resealed += 1; + continue; + } + if dry_run { + blobs_rewrapped += 1; + continue; + } + // Randomized: rebuild the wrap list only — exactly granted + + // escrow, reusing prior wrap bytes for recipients already + // present (id-stability), fresh wraps for the rest. + // Tombstoned/stale wraps are dropped. Ciphertext id unchanged. + let prior = wks.clone(); + let mut new_wks: Vec = Vec::new(); for pk in &target_pks { let rid = pk.recipient_id().to_string(); match prior.iter().find(|w| w.recipient_id == rid) { @@ -198,17 +285,21 @@ impl Repo { } } new_wks.sort_by(|a, b| a.recipient_id.cmp(&b.recipient_id)); - protection.wrapped.insert(blob_id, new_wks); + protection.wrapped.insert(*blob_id, new_wks); blobs_rewrapped += 1; } // ---- Nothing to do / dry-run: report only. ---- if dry_run - || (secrets_rewrapped.is_empty() && new_secret_objs.is_empty() && blobs_rewrapped == 0) + || (secrets_rewrapped.is_empty() + && new_secret_objs.is_empty() + && blobs_rewrapped == 0 + && blobs_resealed == 0) { return Ok(RewrapReport { secrets_rewrapped, blobs_rewrapped, + blobs_resealed, skipped, commit: None, }); @@ -224,15 +315,52 @@ impl Repo { registry.insert(name.clone(), id); secrets_rewrapped.push(name); } + // Rebuild the root tree iff any convergent blob was re-sealed: walk + // the tip's entries again, swapping in each retargeted path's new + // ciphertext id + `RANDOMIZED` perms and keeping every other entry's + // bytes/mode/perms verbatim. With no retargets the root is untouched + // byte-for-byte (policy-only, the P17 property — a second rewrap + // converges back to tree-identical). + let new_root = if retargets.is_empty() { + snap.root + } else { + let mut all: Vec<(String, Vec, scl_core::FileMode, u8)> = + Vec::with_capacity(entries.len()); + { + let arc = self.store_arc(); + let mut store = arc.lock().unwrap(); + for (path, (blob_id, mode, perms)) in &entries { + let (id, p) = match retargets.get(path) { + Some((new_id, new_perms)) => (*new_id, *new_perms), + None => (*blob_id, *perms), + }; + let bytes = match store.get(&id)? { + Object::Blob(b) => b.to_vec(), + _ => return Err(Error::CorruptObject(id)), + }; + all.push((path.clone(), bytes, *mode, p)); + } + // Store lock dropped here: write_tree_with_perms locks it. + } + self.vfs.write_tree_with_perms(&all)? + }; let head = crate::refs::current_branch(self.layout())?; let before = crate::refs::read_branch_tip(self.layout(), &head)?; - let msg = format!( - "rewrap: {} secret(s), {} blob(s)", - secrets_rewrapped.len(), - blobs_rewrapped - ); - let id = - self.commit_snapshot(snap.root, vec![tip], registry, protection, "system", &msg)?; + let msg = if blobs_resealed > 0 { + format!( + "rewrap: {} secret(s), {} blob(s), {} re-sealed randomized", + secrets_rewrapped.len(), + blobs_rewrapped, + blobs_resealed + ) + } else { + format!( + "rewrap: {} secret(s), {} blob(s)", + secrets_rewrapped.len(), + blobs_rewrapped + ) + }; + let id = self.commit_snapshot(new_root, vec![tip], registry, protection, "system", &msg)?; crate::oplog::record( self.layout(), "rewrap", @@ -240,9 +368,18 @@ impl Repo { &head, &[(head.clone(), before, Some(id))], )?; + // Persist the cache only after the commit has landed (never `?` a + // save — cache trouble must not fail a rewrap that already + // succeeded; worst case is a spurious reseal next commit). + if !retargets.is_empty() { + if let Some(c) = &main_cache { + c.save_best_effort(); + } + } Ok(RewrapReport { secrets_rewrapped, blobs_rewrapped, + blobs_resealed, skipped, commit: Some(id), }) @@ -500,6 +637,199 @@ mod tests { std::fs::remove_dir_all(&root).unwrap(); } + // ---- P33 Task 10: eager convergent → randomized upgrade ---- + + use scl_core::{ + FileMode, Object, ObjectId, ProtectPrefix, Protection, RecipientEntry, RecipientState, + PROTECTED, RANDOMIZED, + }; + + /// Seed the tip with pre-P33 CONVERGENT protected entries under + /// `secret/`. The CLI can no longer mint convergent ciphertext (commit + /// seals randomized since Task 6), so this seals via the low-level + /// `encrypt_path` and writes the tree + protection registry directly. + /// Each entry: (path, plaintext, recipient the DEK is wrapped for, + /// wrap_wrong_dek — when true, the wrap holds a DEK that does NOT open + /// the ciphertext, simulating corruption the unwrap can't detect). + fn seed_convergent_tip( + repo: &Repo, + rule_pk: &scl_crypto::PublicKey, + entries: &[(&str, &[u8], &scl_crypto::PublicKey, bool)], + ) { + let mut prot = Protection::default(); + prot.prefixes.push(ProtectPrefix { + prefix: "secret/".into(), + recipients: vec![RecipientEntry { + key: rule_pk.to_bytes(), + epoch: 1, + state: RecipientState::Granted, + }], + }); + let mut files = Vec::new(); + for (path, pt, pk, wrap_wrong) in entries { + let (cipher, dek) = scl_crypto::encrypt_path(pt); + let id = Object::blob(cipher.clone()).id(); + let wrapped_dek = if *wrap_wrong { + let (_c, other) = scl_crypto::encrypt_path(b"a different plaintext entirely"); + other + } else { + dek + }; + prot.wrapped + .entry(id) + .or_default() + .push(scl_crypto::wrap_dek_for(&wrapped_dek, pk)); + files.push((path.to_string(), cipher, FileMode::FILE, PROTECTED)); + } + let root = repo.vfs.write_tree_with_perms(&files).unwrap(); + let parents = repo.head_tip().unwrap().into_iter().collect(); + repo.commit_snapshot( + root, + parents, + Default::default(), + prot, + "test", + "seed convergent", + ) + .unwrap(); + } + + /// The tip tree's (blob id, perms) at `path`. + fn entry_at(repo: &Repo, tip: ObjectId, path: &str) -> (ObjectId, u8) { + let snap = repo.snapshot(&tip).unwrap(); + let arc = repo.store_arc(); + let mut store = arc.lock().unwrap(); + let entries = crate::worktree::tree_file_entries_with_perms(&mut store, snap.root).unwrap(); + let (id, _mode, perms) = *entries.get(path).expect("entry present"); + (id, perms) + } + + /// Decrypt `path`'s ciphertext at `tip` via wrap presence (as `sc run`/ + /// `decrypt_with` do): unwrap the DEK with `sk`, open the blob. + fn decrypt_at(repo: &Repo, tip: ObjectId, path: &str, sk: &scl_crypto::SecretKey) -> Vec { + let snap = repo.snapshot(&tip).unwrap(); + let (id, _perms) = entry_at(repo, tip, path); + let wks = snap.protection.wrapped.get(&id).expect("wraps recorded"); + let me = sk.public().recipient_id().to_string(); + let wk = wks + .iter() + .find(|w| w.recipient_id == me) + .expect("identity has a wrap"); + let dek = scl_crypto::unwrap_dek_with(wk, sk).unwrap(); + let cipher = { + let arc = repo.store_arc(); + let obj = arc.lock().unwrap().get(&id).unwrap(); + match obj { + Object::Blob(b) => b.to_vec(), + _ => panic!("tree entry is not a blob"), + } + }; + scl_crypto::decrypt_path(&cipher, &dek).unwrap().to_vec() + } + + #[test] + fn rewrap_upgrades_convergent_blobs_and_second_run_is_policy_only() { + let root = tmp_root("upgrade"); + let repo = Repo::init(&root).unwrap(); + let (alice_sk, alice_pk) = scl_crypto::generate_keypair(); + seed_convergent_tip( + &repo, + &alice_pk, + &[("secret/x", b"top secret", &alice_pk, false)], + ); + let tip0 = repo.head_tip().unwrap().unwrap(); + let (old_id, old_perms) = entry_at(&repo, tip0, "secret/x"); + assert_eq!(old_perms, PROTECTED, "fixture must be convergent"); + let known = [alice_pk.clone()]; + + // Dry-run counts the upgrade without mutating anything. + let dry = repo.rewrap(&alice_sk, &[], &known, true).unwrap(); + assert_eq!(dry.blobs_resealed, 1, "dry-run still REPORTS the upgrade"); + assert!(dry.commit.is_none()); + assert_eq!(repo.head_tip().unwrap().unwrap(), tip0, "tip must not move"); + + let r1 = repo.rewrap(&alice_sk, &[], &known, false).unwrap(); + assert_eq!(r1.blobs_resealed, 1); + assert!(r1.skipped.is_empty()); + let tip1 = repo.head_tip().unwrap().unwrap(); + let (id1, perms1) = entry_at(&repo, tip1, "secret/x"); + assert!(perms1 & RANDOMIZED != 0, "entry must be flagged randomized"); + assert!(perms1 & PROTECTED != 0, "entry must stay protected"); + assert_ne!(id1, old_id, "randomized reseal mints a fresh ciphertext id"); + assert_eq!( + decrypt_at(&repo, tip1, "secret/x", &alice_sk), + b"top secret".to_vec() + ); + // Old convergent id's wrap entry is dropped from the live tip; the + // new id carries the fresh wraps. + let snap1 = repo.snapshot(&tip1).unwrap(); + assert!(!snap1.protection.wrapped.contains_key(&old_id)); + assert!(snap1.protection.wrapped.contains_key(&id1)); + + let r2 = repo.rewrap(&alice_sk, &[], &known, false).unwrap(); + assert_eq!(r2.blobs_resealed, 0, "second rewrap is policy-only again"); + if r2.commit.is_some() { + let tip2 = repo.head_tip().unwrap().unwrap(); + assert_eq!( + entry_at(&repo, tip2, "secret/x").0, + id1, + "ciphertext id stable" + ); + assert_eq!( + repo.snapshot(&tip2).unwrap().root, + snap1.root, + "root untouched byte-for-byte (P17 policy-only property)" + ); + } + drop(repo); + std::fs::remove_dir_all(&root).unwrap(); + } + + #[test] + fn rewrap_skips_unopenable_convergent_blob_and_upgrades_the_rest() { + // Skip-and-report for the new decrypt step: a convergent blob whose + // wrap opens but whose ciphertext does NOT (corruption-shaped) lands + // in `skipped`, named; the openable one still upgrades; the run + // still commits (existing partial-success semantics). + let root = tmp_root("convergent-skip"); + let repo = Repo::init(&root).unwrap(); + let (alice_sk, alice_pk) = scl_crypto::generate_keypair(); + seed_convergent_tip( + &repo, + &alice_pk, + &[ + ("secret/good", b"ok", &alice_pk, false), + ("secret/bad", b"nope", &alice_pk, true), + ], + ); + let known = [alice_pk.clone()]; + let report = repo.rewrap(&alice_sk, &[], &known, false).unwrap(); + assert_eq!(report.blobs_resealed, 1, "the openable blob still upgrades"); + assert_eq!(report.skipped.len(), 1); + assert!(report.skipped[0].0.contains("secret/bad")); + assert!( + report.skipped[0].1.contains("ciphertext failed to open"), + "got {:?}", + report.skipped[0] + ); + assert!(report.commit.is_some(), "partial success still commits"); + let tip = repo.head_tip().unwrap().unwrap(); + let (_gid, gperms) = entry_at(&repo, tip, "secret/good"); + assert!(gperms & RANDOMIZED != 0); + // The unopenable entry is left exactly as it was: convergent, same + // id, wraps intact — nothing silently dropped. + let (bid, bperms) = entry_at(&repo, tip, "secret/bad"); + assert_eq!(bperms & RANDOMIZED, 0); + assert!(repo + .snapshot(&tip) + .unwrap() + .protection + .wrapped + .contains_key(&bid)); + drop(repo); + std::fs::remove_dir_all(&root).unwrap(); + } + #[test] fn rewrap_refuses_during_in_progress_merge() { // final-review I1: rewrap must not cut over while a conflicted From 2e1e1a31b32cf57a482b30c1ee45d9131ceacc2d Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 22:33:09 +0300 Subject: [PATCH 16/22] =?UTF-8?q?P33:=20rewrap=20fix=20=E2=80=94=20shared?= =?UTF-8?q?=20convergent=20blob=20upgrade=20(review=20Critical)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/repo/src/rewrap.rs | 155 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 150 insertions(+), 5 deletions(-) diff --git a/crates/repo/src/rewrap.rs b/crates/repo/src/rewrap.rs index 5988c1bb..1c57d11c 100644 --- a/crates/repo/src/rewrap.rs +++ b/crates/repo/src/rewrap.rs @@ -139,6 +139,16 @@ impl Repo { // commit tail rebuilds the root iff this is non-empty. let mut retargets: std::collections::BTreeMap = std::collections::BTreeMap::new(); + // Old convergent blob ids some path upgraded FROM. Their wrap-map + // entries are removed only AFTER the loop, and only if no tree entry + // still references them (review Critical): two paths can share ONE + // convergent blob id (pre-P33 dedup of identical plaintext), and + // removing the shared entry inside the loop stripped the wraps out + // from under the second path — it was then skipped with "no wrapped + // DEKs recorded for blob" and left pointing at a wrap-orphaned blob + // at the tip: silent access loss no re-run could repair. + let mut upgraded_from: std::collections::BTreeSet = + std::collections::BTreeSet::new(); // Main-tree protected cache: recording (path, plaintext, new id) for // each upgraded path is what keeps the next `sc status`/`commit` // quiet — rewrap never rematerializes the working tree, so the @@ -251,8 +261,12 @@ impl Repo { .map(|pk| scl_crypto::wrap_dek_for(&new_dek, pk)) .collect(); new_wks.sort_by(|a, b| a.recipient_id.cmp(&b.recipient_id)); - protection.wrapped.remove(blob_id); + // The OLD id's wrap entry is NOT removed here — another tree + // path may share this convergent blob id and still needs the + // wraps to upgrade itself (or, if it skips, to stay + // decryptable). Deferred cleanup below the loop. protection.wrapped.insert(new_id, new_wks); + upgraded_from.insert(*blob_id); retargets.insert( path.clone(), (new_id, scl_core::PROTECTED | scl_core::RANDOMIZED), @@ -289,6 +303,24 @@ impl Repo { blobs_rewrapped += 1; } + // Deferred old-id wrap cleanup: drop an upgraded-from convergent id's + // wrap entry only when NO tree entry still points at it after the + // retargets are applied — a path that shared the blob but was skipped + // (no rule / not a recipient / …) keeps its wraps, so no entry is + // ever left wrap-orphaned at the tip. + if !upgraded_from.is_empty() { + let still_referenced: std::collections::BTreeSet = entries + .iter() + .filter(|(p, _)| !retargets.contains_key(p.as_str())) + .map(|(_, (id, _, _))| *id) + .collect(); + for old in &upgraded_from { + if !still_referenced.contains(old) { + protection.wrapped.remove(old); + } + } + } + // ---- Nothing to do / dry-run: report only. ---- if dry_run || (secrets_rewrapped.is_empty() @@ -675,10 +707,14 @@ mod tests { } else { dek }; - prot.wrapped - .entry(id) - .or_default() - .push(scl_crypto::wrap_dek_for(&wrapped_dek, pk)); + // One wrap per recipient per blob id, as a real pre-P33 commit's + // reuse_prior_wraps produced — identical plaintext at two paths + // dedups to ONE shared blob id with ONE wrap list. + let wks = prot.wrapped.entry(id).or_default(); + let rid = pk.recipient_id().to_string(); + if !wks.iter().any(|w| w.recipient_id == rid) { + wks.push(scl_crypto::wrap_dek_for(&wrapped_dek, pk)); + } files.push((path.to_string(), cipher, FileMode::FILE, PROTECTED)); } let root = repo.vfs.write_tree_with_perms(&files).unwrap(); @@ -785,6 +821,115 @@ mod tests { std::fs::remove_dir_all(&root).unwrap(); } + #[test] + fn rewrap_upgrades_two_paths_sharing_one_convergent_blob() { + // Review Critical: identical plaintext at two paths pre-P33 dedups + // to ONE convergent blob id shared by both tree entries. The old + // in-loop wrap removal stripped the shared entry after the FIRST + // path upgraded, so the second was skipped ("no wrapped DEKs") and + // left pointing at a wrap-orphaned blob — silent access loss no + // re-run could repair. Both paths must upgrade, decryptable, and no + // tree entry may reference a wrapless blob at the new tip. + let root = tmp_root("shared-blob"); + let repo = Repo::init(&root).unwrap(); + let (alice_sk, alice_pk) = scl_crypto::generate_keypair(); + seed_convergent_tip( + &repo, + &alice_pk, + &[ + ("secret/a", b"same plaintext", &alice_pk, false), + ("secret/b", b"same plaintext", &alice_pk, false), + ], + ); + let tip0 = repo.head_tip().unwrap().unwrap(); + let (old_a, _) = entry_at(&repo, tip0, "secret/a"); + let (old_b, _) = entry_at(&repo, tip0, "secret/b"); + assert_eq!(old_a, old_b, "fixture must share ONE convergent blob id"); + + let known = [alice_pk.clone()]; + let report = repo.rewrap(&alice_sk, &[], &known, false).unwrap(); + assert!(report.skipped.is_empty(), "got {:?}", report.skipped); + assert_eq!(report.blobs_resealed, 2, "both paths re-sealed"); + + let tip1 = repo.head_tip().unwrap().unwrap(); + let snap1 = repo.snapshot(&tip1).unwrap(); + let (id_a, perms_a) = entry_at(&repo, tip1, "secret/a"); + let (id_b, perms_b) = entry_at(&repo, tip1, "secret/b"); + assert!(perms_a & RANDOMIZED != 0 && perms_b & RANDOMIZED != 0); + assert_ne!( + id_a, id_b, + "per-path randomized reseal also closes the intra-tip equality oracle" + ); + assert_eq!( + decrypt_at(&repo, tip1, "secret/a", &alice_sk), + b"same plaintext".to_vec() + ); + assert_eq!( + decrypt_at(&repo, tip1, "secret/b", &alice_sk), + b"same plaintext".to_vec() + ); + // The shared old id is unreferenced after both retargets → removed. + assert!(!snap1.protection.wrapped.contains_key(&old_a)); + // Invariant the Critical violated: every PROTECTED tree entry's id + // has wraps at the tip — nothing is wrap-orphaned. + let arc = repo.store_arc(); + let entries = { + let mut store = arc.lock().unwrap(); + crate::worktree::tree_file_entries_with_perms(&mut store, snap1.root).unwrap() + }; + for (p, (id, _m, perms)) in &entries { + if perms & PROTECTED != 0 { + assert!( + snap1.protection.wrapped.contains_key(id), + "{p} points at a wrap-orphaned blob" + ); + } + } + drop(repo); + std::fs::remove_dir_all(&root).unwrap(); + } + + #[test] + fn undo_of_reseal_commit_restores_convergent_tip_without_spurious_status() { + // Minor (same review): `sc undo` of a reseal commit must restore the + // convergent tip and report a clean status — the convergent diff + // branch re-encrypts the on-disk plaintext keylessly, so it needs no + // cache entry, and the stale randomized cache tag is harmless. + let root = tmp_root("undo-reseal"); + let repo = Repo::init(&root).unwrap(); + let (alice_sk, alice_pk) = scl_crypto::generate_keypair(); + std::fs::create_dir_all(root.join("secret")).unwrap(); + std::fs::write(root.join("secret/x"), b"top secret").unwrap(); + seed_convergent_tip( + &repo, + &alice_pk, + &[("secret/x", b"top secret", &alice_pk, false)], + ); + let tip0 = repo.head_tip().unwrap().unwrap(); + let known = [alice_pk.clone()]; + + let r = repo.rewrap(&alice_sk, &[], &known, false).unwrap(); + assert_eq!(r.blobs_resealed, 1); + // Post-reseal, the cache keeps the untouched on-disk plaintext quiet. + let s = repo.status().unwrap(); + assert!(s.modified.is_empty(), "post-reseal: got {:?}", s.modified); + + repo.undo().unwrap(); + assert_eq!( + repo.head_tip().unwrap().unwrap(), + tip0, + "one undo reverts the whole reseal commit" + ); + let (_id, perms) = entry_at(&repo, tip0, "secret/x"); + assert_eq!(perms & RANDOMIZED, 0, "tip entry is convergent again"); + let s = repo.status().unwrap(); + assert!(s.modified.is_empty(), "post-undo: got {:?}", s.modified); + assert!(s.added.is_empty(), "post-undo: got {:?}", s.added); + assert!(s.deleted.is_empty(), "post-undo: got {:?}", s.deleted); + drop(repo); + std::fs::remove_dir_all(&root).unwrap(); + } + #[test] fn rewrap_skips_unopenable_convergent_blob_and_upgrades_the_rest() { // Skip-and-report for the new decrypt step: a convergent blob whose From 56b3edef56ecbdd6a583cd3bd694ffec2cd168f9 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 22:52:45 +0300 Subject: [PATCH 17/22] P33: randomized-encryption demo (oracle closed, quiet history, rewrap upgrade) Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- demo/run_randomized_demo.sh | 255 ++++++++++++++++++++++++++++++++++++ 1 file changed, 255 insertions(+) create mode 100755 demo/run_randomized_demo.sh diff --git a/demo/run_randomized_demo.sh b/demo/run_randomized_demo.sh new file mode 100755 index 00000000..06facd3e --- /dev/null +++ b/demo/run_randomized_demo.sh @@ -0,0 +1,255 @@ +#!/usr/bin/env bash +# P33 demo: randomized protected-path encryption. Proves that the oracle +# ADR-0014 accepted (equality-confirmable protected content: same plaintext +# -> same ciphertext id) is CLOSED for everything sealed from this phase on, +# that untouched protected files stay quiet across unrelated commits (the +# local stat/keyed-hash cache carries them instead of re-sealing), that +# identical independent edits on two branches now require --identity to +# merge (accepted cost 4a — a real UX regression vs. the old id-fast-path, +# even though the merge itself completes cleanly once given the key), and +# that `sc rewrap` on an already-fully-randomized tip is a true no-op +# policy-wise (no re-seal line, no new tree/blob objects — only a fresh +# commit record). +# +# Scope split (deliberate, stated plainly): this script drives the CURRENT +# binary only, which — since Task 6 flipped `encrypt_protected` to always +# randomize — can no longer mint a convergent (pre-P33) seal at all. So this +# demo proves oracle-closure, quiet history, the accepted-cost-4a merge +# behavior, and rewrap's policy-only idempotence on an all-randomized tip +# (steps 1, 2, 3, and the "second run reports nothing to upgrade" half of +# the rewrap-upgrade story). Dual-read of a GENUINELY convergent (pre-P33) +# store, and `sc rewrap`'s eager convergent->randomized upgrade path itself +# (the "re-sealed N convergent blob(s)" line firing), are pinned instead by +# unit tests built with the library directly (crates/repo/src/rewrap.rs, +# crates/repo/src/{merge,replay}.rs) — Tasks 4-6, 9, 10 all build convergent +# fixtures that way, which is impossible to reproduce honestly with a CLI +# that can only ever mint randomized ciphertext. +# +# Technique note: there is no `sc cat-file`/`sc ls-tree` exposing a path's +# blob id, so ciphertext-id (in)equality is proven indirectly but rigorously +# via the content-addressed object store's file count: every object is one +# file named by its own hash, so "how many NEW object files appeared" is an +# exact count of how many genuinely new (blob/tree/snapshot) objects a +# commit created. Two operations of identical *shape* (e.g. "add one new +# flat file") cost identical object-count deltas whenever their content is +# equally novel to the store — so comparing deltas across a controlled pair +# of same-shape commits proves id (in)equality without ever needing to read +# a specific id. +# +# Self-checking: every claim is an assertion; any failure exits non-zero +# before the RESULT line. +set -euo pipefail + +cd "$(dirname "$0")/.." +cargo build --quiet --bin sc +SC="$PWD/target/debug/sc" + +tmp_base="${TMPDIR:-/tmp}" +tmp_base="${tmp_base%/}" + +fail() { echo "FAIL: $1"; exit 1; } + +# Count loose objects in the CAS. Called only before `sc gc` ever runs in +# this script, so every object stays loose (one file per object) and the +# count is an exact reachable-object tally. +objcount() { find .sc/objects -type f | wc -l | tr -d ' '; } + +run_demo() { + local run_label="$1" + local work + work=$(mktemp -d "$tmp_base/sc-randomized-demo.XXXXXX") + trap 'rm -rf "$work"' RETURN + + local keys="$work/keys" + mkdir -p "$keys" + local repo="$work/repo" + mkdir -p "$repo" + + # Keyless by construction (the P15/P23 demos' pattern): point SC_IDENTITY + # at a path that can never exist, so every command below is keyless unless + # it passes an explicit --identity. + export SC_IDENTITY="$work/no-such-identity" + + local alice_out alice_pk + alice_out=$("$SC" keygen --out "$keys/alice") + alice_pk=$(echo "$alice_out" | grep 'public key' | awk '{print $3}') + + cd "$repo" + "$SC" init >/dev/null + printf '[recipients]\nalice = "%s"\n' "$alice_pk" > .sc/recipients.toml + "$SC" protect secret/ --to alice >/dev/null + mkdir -p secret + + echo + echo "=== [$run_label] 1a: oracle closed — same plaintext, two different protected paths ===" + echo "repeatable-plaintext-1" > secret/a.txt + "$SC" commit -m "base: secret/a.txt" --author demo >/dev/null + local c_a + c_a=$(objcount) + + # ADD secret/b.txt with the SAME plaintext as secret/a.txt — same "shape" + # of change (one new flat file under secret/) as the next commit below. + echo "repeatable-plaintext-1" > secret/b.txt + "$SC" commit -m "secret/b.txt: same plaintext as a.txt" --author demo >/dev/null + local c_b delta_same + c_b=$(objcount) + delta_same=$((c_b - c_a)) + + # ADD secret/c.txt with a genuinely NOVEL plaintext — identical shape + # (one new flat file under secret/), the "definitely a fresh blob" baseline. + echo "genuinely-novel-plaintext-2" > secret/c.txt + "$SC" commit -m "secret/c.txt: novel plaintext" --author demo >/dev/null + local c_c delta_diff + c_c=$(objcount) + delta_diff=$((c_c - c_b)) + + [ "$delta_same" -eq "$delta_diff" ] \ + || fail "adding secret/b.txt (repeat of a.txt's plaintext) cost $delta_same new object(s), but adding secret/c.txt (novel plaintext) cost $delta_diff — a convergent seal would have cost LESS for the repeat (blob dedup), so the oracle would still be open" + [ "$delta_same" -ge 3 ] || fail "expected at least 3 new objects (blob+tree+snapshot) for a one-file add, got $delta_same" + echo "secret/b.txt (repeat of a.txt's plaintext) cost $delta_same new object(s), identical to secret/c.txt's (genuinely novel) $delta_diff — a fresh, distinct ciphertext blob was minted for the repeat ✔" + + echo + echo "=== [$run_label] 1b: oracle closed — same plaintext, edited away then back on ONE path ===" + echo "edit-value-one" > secret/e.txt + "$SC" commit -m "base: secret/e.txt = one" --author demo >/dev/null + local c_e0 + c_e0=$(objcount) + + echo "edit-value-two" > secret/e.txt + "$SC" commit -m "secret/e.txt: edited away" --author demo >/dev/null + local c_e1 delta_away + c_e1=$(objcount) + delta_away=$((c_e1 - c_e0)) + + echo "edit-value-one" > secret/e.txt + "$SC" commit -m "secret/e.txt: edited back to its original value" --author demo >/dev/null + local c_e2 delta_back + c_e2=$(objcount) + delta_back=$((c_e2 - c_e1)) + + [ "$delta_away" -eq "$delta_back" ] \ + || fail "editing secret/e.txt away cost $delta_away new object(s), editing it back to the SAME original plaintext cost $delta_back — a convergent seal would have reused the original commit's blob for the 'back' edit (fewer new objects), so the oracle would still be open across commits" + echo "editing secret/e.txt away ($delta_away new object(s)) and back to its ORIGINAL plaintext ($delta_back) cost the same — the 'back' edit minted a fresh ciphertext blob rather than reconverging on the first commit's blob ✔" + + echo + echo "=== [$run_label] 2: quiet history — an unrelated commit doesn't touch untouched protected files ===" + local c_before_unrelated + c_before_unrelated=$(objcount) + # Nested one level deep (like secret/c.txt) so the object-count "shape" of + # this add matches 1a's delta_diff baseline exactly: root tree + one new + # subtree + snapshot + blob = 4. A flat root-level add would cost one + # fewer object (no subtree) and make the comparison apples-to-oranges. + mkdir -p misc + echo "unrelated noise" > misc/note.txt + "$SC" commit -m "unrelated: add misc/note.txt" --author demo >/dev/null + local c_after_unrelated delta_unrelated + c_after_unrelated=$(objcount) + delta_unrelated=$((c_after_unrelated - c_before_unrelated)) + + # Same shape as 1a's "add one new subtree with one novel-content flat file" + # baseline (delta_diff): if it were EQUAL, this commit cost exactly as much + # as adding one new file in one new subdirectory and nothing else touched. + # If secret/{a,b,c,e}.txt had been needlessly re-encrypted too, this delta + # would be strictly larger (an extra new secret/ subtree object plus one + # new blob per re-sealed file). + [ "$delta_unrelated" -eq "$delta_diff" ] \ + || fail "an unrelated top-level commit cost $delta_unrelated new object(s), vs. $delta_diff for adding one novel flat file alone — the extra objects mean an untouched protected file was needlessly re-sealed (cache-carry regression)" + echo "unrelated commit (new subdir + 1 file) cost exactly $delta_unrelated new object(s), matching the 1a novel-add baseline — no protected file's ciphertext id changed (cache carried them all) ✔" + + local status_out + status_out=$("$SC" status) + case "$status_out" in *"clean"*) ;; *) fail "sc status must stay clean after an unrelated commit, got: $status_out" ;; esac + echo "sc status: clean ✔" + + echo + echo "=== [$run_label] 3: accepted cost 4a — identical independent edits on two branches now need --identity ===" + echo "shared-base-value" > secret/x.txt + "$SC" commit -m "base: secret/x.txt" --author demo >/dev/null + "$SC" branch feature >/dev/null + + echo "identical-new-value" > secret/x.txt + "$SC" commit -m "main: edit secret/x.txt" --author demo >/dev/null + + "$SC" switch feature --identity "$keys/alice" >/dev/null + echo "identical-new-value" > secret/x.txt + "$SC" commit -m "feature: edit secret/x.txt to the SAME value" --author demo >/dev/null + + "$SC" switch main --identity "$keys/alice" >/dev/null + + set +e + no_id_out=$("$SC" merge feature --author demo 2>&1) + no_id_rc=$? + set -e + [ "$no_id_rc" -eq 1 ] || fail "merging identical independent edits without --identity must exit 1 (accepted cost 4a), got rc=$no_id_rc: $no_id_out" + case "$no_id_out" in *"--identity"*) ;; *) fail "expected a --identity hint in the refusal, got: $no_id_out" ;; esac + echo "sc merge feature (no --identity) refuses even though both sides wrote the SAME plaintext — pre-P33 this would have id-fast-pathed with no key needed at all ✔" + + status_out=$("$SC" status) + case "$status_out" in *"clean"*) ;; *) fail "a refused (not attempted) merge must leave the working tree clean, got: $status_out" ;; esac + echo "refusal left refs/working tree untouched (no MERGE_HEAD, no markers) ✔" + + local merge_out + merge_out=$("$SC" merge feature --identity "$keys/alice" --author demo) + case "$merge_out" in *"merged feature"*) ;; *) fail "merge with --identity must succeed cleanly, got: $merge_out" ;; esac + echo "sc merge feature --identity alice completes cleanly — no conflict markers (diff3 of identical plaintext has nothing to conflict over) ✔" + + local top_line + top_line=$("$SC" log | head -1) + case "$top_line" in *"(merge)"*) ;; *) fail "expected a merge marker on the tip, got: $top_line" ;; esac + + "$SC" switch main --identity "$keys/alice" >/dev/null + [ "$(cat secret/x.txt)" = "identical-new-value" ] || fail "post-merge secret/x.txt must decrypt to the agreed value" + echo "merged tip decrypts to the agreed value, re-encrypted through the ordinary commit path ✔" + + echo + echo "=== [$run_label] 4: rewrap on an all-randomized tip is policy-only (no re-seal line) ===" + local c_before_rewrap + c_before_rewrap=$(objcount) + local rewrap_out + rewrap_out=$("$SC" rewrap --identity "$keys/alice") + echo "$rewrap_out" + case "$rewrap_out" in *"re-sealed"*"convergent blob"*) fail "rewrap on an all-randomized tip must NOT print a convergent re-seal line, got: $rewrap_out" ;; esac + case "$rewrap_out" in *"rewrapped"*"protected blob(s)"*) ;; *) fail "expected a rewrapped-blobs summary line, got: $rewrap_out" ;; esac + local c_after_rewrap delta_rewrap + c_after_rewrap=$(objcount) + delta_rewrap=$((c_after_rewrap - c_before_rewrap)) + # Nothing to upgrade: the root tree and every blob are byte-identical + # (the P17 policy-only property), so the ONLY new object is the fresh + # commit snapshot itself — no new tree, no new blob. + [ "$delta_rewrap" -eq 1 ] \ + || fail "rewrap on an all-randomized tip must add exactly 1 new object (the commit snapshot only — root/blobs unchanged), got $delta_rewrap" + echo "rewrap added exactly 1 new object (the commit record) — root tree and every blob stayed byte-identical ✔" + + "$SC" switch main --identity "$keys/alice" >/dev/null + [ "$(cat secret/a.txt)" = "repeatable-plaintext-1" ] || fail "content must still decrypt correctly after the policy-only rewrap" + [ "$(cat secret/x.txt)" = "identical-new-value" ] || fail "merged content must still decrypt correctly after the policy-only rewrap" + echo "all protected content still decrypts correctly after rewrap ✔" + + echo + echo "=== [$run_label] 5: zero residue — .sc/tmp is empty, nothing lives outside \$work ===" + [ -d "$repo/.sc" ] || fail "repo/.sc must exist while the repo is alive" + if [ -d "$repo/.sc/tmp" ] && [ -n "$(ls -A "$repo/.sc/tmp" 2>/dev/null)" ]; then + fail ".sc/tmp is not empty after the run" + fi + echo "no .sc/tmp residue ✔" +} + +run_demo "run 1" +run_demo "run 2 (repeatable, not a one-shot fluke)" + +echo +echo "RESULT: the equality-confirmation oracle is closed for randomized-write" +echo "protected content — same plaintext at two paths, and the same plaintext" +echo "edited away and back on one path, both mint a fresh distinct ciphertext" +echo "blob (proven via matched object-count deltas, since the CLI exposes no" +echo "cat-file); an unrelated commit costs exactly the baseline object delta," +echo "meaning every untouched protected file was cache-carried, not re-sealed," +echo "and sc status stays clean; two branches writing the IDENTICAL plaintext" +echo "to a protected path now refuse to merge without --identity (accepted" +echo "cost 4a) but merge cleanly (no markers) once given the key; and sc" +echo "rewrap on an already-fully-randomized tip is genuinely policy-only (no" +echo "re-seal line, exactly one new object — the commit record). Dual-read of" +echo "a genuinely convergent (pre-P33) store and rewrap's eager upgrade path" +echo "itself are pinned by unit tests instead, since this binary can no" +echo "longer mint a convergent seal to fixture one honestly. Verified across" +echo "two independent runs, zero residue." From 7dc0ae73d82901c3bbb7ffd4e116ce135f85d312 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 23:09:08 +0300 Subject: [PATCH 18/22] P33: ADR-0043 + THREAT-MODEL/CLAUDE.md/ARCHITECTURE.md updates Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- ARCHITECTURE.md | 60 ++-- CLAUDE.md | 142 ++++++++- ROADMAP.md | 52 +++- docs/THREAT-MODEL.md | 24 +- ...14-per-file-permissions-encrypted-paths.md | 5 +- .../0043-randomized-protected-encryption.md | 274 ++++++++++++++++++ docs/adr/README.md | 1 + 7 files changed, 513 insertions(+), 45 deletions(-) create mode 100644 docs/adr/0043-randomized-protected-encryption.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index eefb76d8..4b232f91 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -105,8 +105,10 @@ a retired pre-P16 snapshot encoding, refused with a clear error): - **Blob** — raw file contents. - **Tree** — a sorted directory listing mapping a name to `(kind, ObjectId, mode, permissions)`. The per-entry `permissions` byte carries the `PROTECTED` - bit (P7): a protected entry's blob is convergently-encrypted ciphertext, and - the bit is what makes "protected" and "ciphertext" synonymous in every tree. + bit (P7): a protected entry's blob is encrypted ciphertext — convergent for + pre-P33 seals, randomized (a companion `RANDOMIZED` bit, P33/ADR-0043) for + new ones — and the bit is what makes "protected" and "ciphertext" synonymous + in every tree. - **Snapshot** — the Jujutsu-inspired analogue of a commit: a root tree id plus metadata (parent snapshot ids, author, timestamp, message), a `secrets` side-registry (name → secret-object id, so secrets are env vars, not files), @@ -202,8 +204,9 @@ transport: `repo` speaks a framed wire protocol over ssh:// (P12) and sc+http:// All cryptography is quarantined in `crates/crypto`; `core` and `repo` see only opaque bytes. The crate owns three surfaces, all built on the RustCrypto stack: -**envelope encryption** (secrets, below), **convergent encryption** for protected -paths, and **Ed25519 signing** for provenance. +**envelope encryption** (secrets, below), **encryption for protected paths** +(randomized DEK/nonce for new content since P33, convergent dual-read for +pre-P33 history — see below), and **Ed25519 signing** for provenance. Committed secrets use **envelope encryption**. Each secret object carries ciphertext encrypted under a fresh per-secret data-encryption key (DEK) using @@ -231,14 +234,27 @@ does not require rotating the secret itself. This is the same envelope model use by age and cloud KMS, chosen because it is well-understood and auditable. **Protected paths (P7)** encrypt designated file *content* rather than named -env-var secrets. They use **convergent encryption** — the DEK and nonce derive -from `BLAKE3(plaintext)` — so identical protected content dedups and an -unauthorized clone gets ciphertext it cannot read. The tradeoff (equality is -confirmable) is deliberate and documented; the `sc protect` CLI nudges genuine -low-entropy secrets toward `sc secret` instead (P28). Per-prefix recipient rules -are last-writer-wins epoch registers so a revoke is durable across merges (P16), -and `sc grant`/`revoke`/`rewrap` manage the recipient set without re-encrypting -convergent content. See ADR-0014/0026. +env-var secrets. As of **P33 (ADR-0043)** all new protected content is sealed +under a **fresh random DEK + random nonce** (`RANDOMIZED` perms bit alongside +`PROTECTED`), so two seals of the same plaintext yield different ciphertext ids +— closing the convergent equality-confirmation oracle for everything sealed +from P33 on. The store is **dual-read, randomized-write**: pre-P33 *convergent* +ciphertext (DEK/nonce derived from `BLAKE3(plaintext)`) still decrypts through +the unchanged `decrypt_path`, so no snapshot-tag bump and no forced migration — +a pre-P33 store reads byte-for-byte identically. Commit's per-path rule is +**format-dispatched** on the prior tip entry (convergent → re-encrypt-and- +compare, carry-if-unchanged; randomized → keyed-PRF stat cache under a never- +committed `.sc/local-key`; new → seal randomized), so unchanged convergent +content stays convergent until an edit or an explicit `sc rewrap` upgrades it. +An unauthorized clone still gets ciphertext it cannot read; the `sc protect` CLI +still nudges genuine low-entropy secrets toward `sc secret` (P28). Per-prefix +recipient rules are last-writer-wins epoch registers so a revoke is durable +across merges (P16). `sc grant`/`revoke` manage the recipient set without +touching ciphertext; **`sc rewrap` now eagerly re-seals still-convergent blobs +randomized** as it re-wraps the tip — so a rewrap that upgrades content is no +longer tree-identical (a convergent→randomized reseal changes the blob id), +though a second rewrap over an all-randomized tip converges back to policy-only. +See ADR-0014/0026/0043. **Lifecycle (P11/P17):** `sc secret rotate` re-seals a secret under a fresh DEK (the real cryptographic cutover, distinct from metadata-only revoke); `sc escrow` @@ -476,12 +492,15 @@ is recovered via `--identity` and re-sealed. Recipients default to the secret's current set (reverse `recipient_id` lookup against `.sc/recipients.toml`), overridable with `--to`. -Rotation is **secrets-only**: protected paths use convergent encryption -(`DEK = HKDF(BLAKE3(plaintext))`), so a recipient who checked the file out can -re-derive the key — "rotating" a path's DEK is either dedup-breaking or -security-meaningless. Path lifecycle stays on recipient re-wrap -(`grant`/`revoke`). `sc secret revoke` remains metadata-only and now hints to -run `rotate` for the actual cryptographic cutover. +Rotation is **secrets-only**: pre-P33 protected paths used convergent +encryption (`DEK = HKDF(BLAKE3(plaintext))`), so a recipient who checked the +file out could re-derive the key — "rotating" a convergent path's DEK is either +dedup-breaking or security-meaningless. Path lifecycle stays on recipient +re-wrap (`grant`/`revoke`). `sc secret revoke` remains metadata-only and now +hints to run `rotate` for the actual cryptographic cutover. (**P33/ADR-0043:** +the security-meaningless objection dissolves for **randomized** content — a +random DEK is independent of the plaintext — so a genuine rotate-for-paths +cutover is now coherent and recorded as an unlocked follow-on, not yet built.) `sc escrow set ` / `sc escrow show` configure a single break-glass recipient key (`[escrow]` in `.sc/recipients.toml`) that is @@ -566,7 +585,10 @@ three-way cases resolve on ciphertext ids alone: unchanged, one-side-changed, and clean-delete protected paths merge with no identity at all, carrying ciphertext blobs plus a union of wrapped DEKs. Only a **content-divergent** protected path — both sides edited the plaintext — needs an authorized -`--identity`: the two plaintexts are decrypted, diff3-merged, and the result +`--identity` (**→ P33/ADR-0043 adjustment:** under randomized sealing, both +sides editing to even *identical* plaintext no longer id-match, so that case +now also conflicts and needs an identity — accepted cost 4a): the two +plaintexts are decrypted, diff3-merged, and the result is re-encrypted through the same `encrypt_protected`/`reuse_prior_wraps` helpers `commit` already used (extracted, single-sourced), so plaintext is never written to the CAS — conflict markers and sidecars for protected paths diff --git a/CLAUDE.md b/CLAUDE.md index b75ec144..9f505d18 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -169,11 +169,25 @@ bash demo/run_lifecycle_demo.sh # rotation + escr cargo run --bin sc -- rewrap [--identity ] [--dry-run] # one-commit bulk reseal of all # secrets + protected wrap lists to current # recipient/escrow sets (skip-and-report; - # exits 1 when entries were skipped) + # exits 1 when entries were skipped); also + # eagerly re-seals any still-CONVERGENT (pre-P33) + # protected blob RANDOMIZED (P33) — so a rewrap + # that upgrades content is no longer tree-identical, + # while a second rewrap over an all-randomized tip + # converges back to policy-only cargo run --bin sc -- escrow add # append a break-glass key (list) cargo run --bin sc -- escrow remove cargo run --bin sc -- escrow show # lists all escrow keys bash demo/run_rewrap_demo.sh # bulk rewrap + escrow-list proof +bash demo/run_randomized_demo.sh # randomized protected-encryption proof + # (P33): same plaintext → different ciphertext ids + # (oracle closed), quiet history for unchanged + # content, cost-4a identical-edit conflict, and + # sc rewrap policy-only on an all-randomized tip — + # run twice, zero residue (the convergent→randomized + # rewrap UPGRADE and pre-P33 dual-read are unit- + # test-pinned, since the current binary can no + # longer write a convergent blob) cargo run --bin sc -- work --agents 3 -- # fork agent workspaces, run in each, # harvest changed ones to work- branches # (--with-secrets --identity injects @@ -435,7 +449,11 @@ default to the secret's current set (resolved by reverse `recipient_id` lookup against `.sc/recipients.toml`) or `--to`. This is secrets-only — per-file protected paths use convergent encryption, where DEK "rotation" is either dedup-breaking or security-meaningless, so path lifecycle stays on -recipient re-wrap (`grant`/`revoke`). `sc secret revoke` remains +recipient re-wrap (`grant`/`revoke`). **(→ P33/ADR-0043: this objection +dissolves for randomized content — a random DEK is independent of the +plaintext, so re-sealing a protected path under a fresh DEK becomes a genuine +cutover; a rotate-for-paths surface is now an unlocked follow-on, recorded not +built.)** `sc secret revoke` remains metadata-only (unchanged behavior), now printing a hint to run `rotate` for an actual cryptographic cutover. `sc escrow {add,remove,show}` / `sc escrow set` (single break-glass key → managed list in P17; `set` kept as replace-with-one sugar) in @@ -588,7 +606,11 @@ presence and its wrap list **replaced** with exactly the governing rule's `granted_keys() + escrow`, so a tombstoned recipient's wraps re-attached by a pre-revoke merge do not survive the sweep. Convergent DEKs keep ciphertext ids unchanged, so the commit is policy-only (root tree -byte-identical). **Skip-and-report:** entries the identity cannot open are +byte-identical). **(→ P33/ADR-0043: no longer unconditionally true — rewrap +now also eagerly re-seals any still-convergent blob RANDOMIZED, changing its +id and the root tree, so a rewrap that upgrades content is content-changing; +only a rewrap over an all-randomized tip stays policy-only.)** +**Skip-and-report:** entries the identity cannot open are skipped and named (not silently dropped); the command commits whatever it could and exits non-zero when anything was skipped, printing a hint to re-run with an identity that can open the rest. **Honesty caveat (same @@ -1187,7 +1209,9 @@ isolation," and a compile-time pin locks in that `scl_crypto::open`'s `Zeroizing>` plaintext rides unchanged through to the unavoidable `OsString` child-env hand-off. **Two accepted boundaries, unchanged by design:** convergent encryption stays equality-confirmable (ADR-0014 — -randomized protected mode is deferred, not a bug); the secret's child-env +**closed for new content in P33/ADR-0043**, which seals all new protected +content randomized; the caveat holds only for pre-P33 convergent ciphertext in +history until `sc rewrap` upgrades it); the secret's child-env copy is fundamental and un-zeroizable (fd/stdin injection is deferred, not a bug) — the parent's own decrypted buffer is what gets zeroized. Every prior demo stays green plus new pinned regression tests across all four @@ -1374,6 +1398,105 @@ first connection to any host is unverified by construction (the TOFU trade, same as SSH's first connect); a loopback bind with no tokens configured stays unauthenticated by design, unchanged from P29. See ADR-0042. +**Phase 33 is built.** Randomized protected-path encryption closes ADR-0014's +convergent equality-confirmation oracle (the audit's remaining protected-path +High, ticket #40) for all **new** content, dual-read for the old. All new +protected content seals under a **fresh random DEK + random nonce** +(`encrypt_path_randomized`, `crates/crypto`), so two seals of the same +plaintext yield different ciphertext ids — the dictionary/equality oracle is +closed from this phase on. **The format tag is a perms-byte flag, not a +blob-layout change:** a new `RANDOMIZED` bit (`0b0000_0010`, always set with +`PROTECTED`) rides the tree entry; the blob is still `nonce(24) ‖ ciphertext` +under the same `PATH_AAD`, decrypted by the unchanged `decrypt_path`, so +dual-read needs ZERO read-path code. The tag lives in the tree entry (not the +blob) so format identification never fetches or decrypts bytes, and so a +format discriminator never perturbs a blob id. **No snapshot-tag bump +(`TAG_SNAPSHOT` stays 4), no `PROTOCOL_VERSION` change (stays 3)** — a pre-P33 +store reads byte-for-byte identically, and carry-by-id (the P15/P24 discipline, +preserving the source entry's own perms) rides the bit through merge, replay, +graft, and sparse carry with no new plumbing. **The commit rule is +format-dispatched, not cache-only** (spec §3 table): a prior-tip **convergent** +entry uses today's re-encrypt-and-compare (unchanged → carry as convergent +forever with no cache; edited → seal randomized), a prior-tip **randomized** +entry consults the stat cache, a **new** path seals randomized — a cache-only +rule would have mass-migrated every unchanged protected file on the first +commit after upgrade; format dispatch means unchanged convergent content stays +convergent until an edit or `sc rewrap` touches it. **Unchanged detection is a +per-checkout stat cache** (`crates/repo/src/cache.rs`) keyed by +`.sc/local-key` (32 random bytes, **0600 from the first byte**, lazily minted, +**never committed, never transferred**): entries are `path → (mtime, size, +keyed_tag, blob_id)` with `keyed_tag = BLAKE3-keyed(local_key, plaintext)`. +Commit still needs only **public keys** — decrypt-and-compare is used nowhere. +The cache **does not reintroduce the oracle** because the tag is a PRF output +under a key that never leaves `.sc/`, so the cache file alone leaks nothing; +its worst failure mode is a spurious re-seal (fresh id for unchanged +plaintext), never incorrectness — a corrupt line is treated as absent with a +warning, a missing key is minted, a key that can't be created is a hard error. +**DEVIATION from #30 decision-3's letter (and the spec §3 table): the stat hit +is NEVER trusted alone — the keyed BLAKE3 tag is authoritative in every cache +lookup.** The recorded `(mtime, size)` are advisory; the tag is recomputed and +compared every time. This closes the git racy-stat class (a same-size edit +within mtime granularity, at the recorded mtime, would otherwise be silently +dropped from a commit — pinned by `racy_stat_same_size_same_mtime_still_misses` +and `tag_is_authoritative_for_hits_and_misses`). Rationale: the plaintext is +already in memory at every call site, keyed hashing is ~free against it, and +"never incorrectness" outranks the stat shortcut. **`sc rewrap` performs the +eager upgrade** (no new command, no flag): while re-wrapping each blob's DEK by +wrap presence, it additionally decrypts any still-convergent blob and re-seals +it randomized to `granted_keys() + escrow`, recording the plaintext it already +holds into the main-tree cache (so the next `sc status`/`commit` stays quiet — +rewrap never rematerializes the working tree). Output gains a `re-sealed N +convergent blob(s)` line; a rewrap that upgrades content is **no longer +tree-identical** (accepted cost 4b), while a second rewrap over an +all-randomized tip converges back to policy-only. **A review Critical, fixed:** +two paths sharing ONE convergent blob (pre-P33 dedup of identical plaintext) — +old-id wrap removal is **DEFERRED past the path loop** (removed only when no +kept tree entry still references it); removing it in-loop stripped the shared +entry out from under the second path, leaving it wrap-orphaned at the tip with +no re-run repair. Each shared path mints its own randomized blob, which also +closes the intra-tip oracle between those two paths. **Accepted costs 4a–c all +landed as predicted:** (4a) independent identical edits on two branches now +conflict — both sides editing to even trivially-identical plaintext no longer +id-matches, so the merge needs `--identity` (or P23 `sc conflicts`/`sc +resolve`) even to merge identical content; (4b) rewrap is tree-changing when it +upgrades; (4c) identical plaintext at two paths no longer dedups. **P15/P17 +semantic adjustments:** merge/replay's content-divergent re-encryption switches +to the randomized primitive (union/tombstone semantics unchanged; mixed +convergent+randomized trees need no special casing — every path is carried by +id or re-sealed randomized), and `sc rewrap` gains the upgrade above. +**`sc protect --to ` no longer wraps UNCHANGED content +at the next commit** (review-confirmed real behavior change): unchanged files +carry their prior wrap list verbatim, so a bare re-protect does not grant the +new recipient access to un-re-sealed files. This fails **safe** — it +under-grants, never over-grants — and `sc grant` / `sc rewrap` are the covering +flows (operator guidance: use `sc grant`, not a second `sc protect`, to add a +recipient to already-committed content). **The cache is per-checkout:** the +main tree uses `.sc/protected-cache`; a ws workspace uses `.sc/ws/cache-` +**beside** the checkout dir, never inside it (a file inside `.sc/ws//` would +be harvested as an untracked working file), removed with the workspace at +teardown; `sc work`'s one-shot agents use ephemeral in-memory caches. Rationale +— without per-workspace caches, untouched protected files in forked workspaces +would have no cache entry, be re-sealed randomized at harvest with a fresh id, +and identical-edit-**conflict** (cost-4a fired spuriously) into a `work-` +fallback instead of a clean auto-merge; per-workspace caches reproduce the base +tree id and preserve P20's clean-auto-merge and crash-recovery `UpToDate` +idempotency. All production cache saves are **best-effort and ordered after ref +moves** — the ref move stays the atomic commit point; a save failure warns, +never errors. **Unlocked follow-on: rotate-for-paths** — ADR-0019 called +per-path DEK rotation security-meaningless under convergent encryption; that +objection **dissolves for randomized content** (a random DEK is independent of +the plaintext), so a `sc protect … rotate` genuine cutover is now coherent — +**recorded, not built.** **Proof splits honestly:** dual-read of genuine +pre-P33 stores is pinned by **library-built convergent-tree fixtures** in the +`rewrap`/`merge`/`replay` unit tests (the current binary can no longer *write* +a convergent blob), while `demo/run_randomized_demo.sh` proves oracle-closure, +quiet history, cost-4a conflicts, and rewrap policy-only on an all-randomized +tip against the current binary (the convergent→randomized rewrap upgrade path +is unit-pinned, not demo-pinned, for the same reason) — run twice, zero +residue. Zero new dependencies (`blake3`/`hex` +were already workspace deps; `scl-repo` now links `blake3` directly for the +cache tag). See ADR-0043. + Remaining follow-ons: operation objects in the CAS, oplog entries for remote-tracking refs, extending the sparse gate to `materialize_conflict_state`'s `to_encrypt`/sidecar write loops (see the @@ -1381,10 +1504,13 @@ P24 boundary note above), the three P27 items named above (transparent lazy-fetch as a deferred alternative to explicit `sc backfill`, per-case gap-tolerant merge/rebase/`ws harvest`/`sc work` instead of blanket refusal, and blob-size/object-count clone filters alongside the -prefix-only filter shipped here), the three P28 items named above -(randomized protected mode for equality-hiding, fd/stdin secret injection -as an alternative to env vars, and a `--max-object-size` operator config -knob), and the P28 final-review follow-ons (a one-line +prefix-only filter shipped here), the two remaining P28 items named above +(fd/stdin secret injection as an alternative to env vars, and a +`--max-object-size` operator config knob — randomized protected mode is now +SHIPPED in P33/ADR-0043), the P33 item (**rotate-for-paths** — ADR-0019's +"path DEK rotation is security-meaningless" objection dissolves for randomized +content, so a `sc protect … rotate`-style genuine per-path cutover is now +coherent; recorded, not built), and the P28 final-review follow-ons (a one-line `validate_branch_name` call in `refs::write_head`/`refs::delete_branch` for ref-validation class completeness — not exploitable today, since HEAD's path is fixed and `delete_branch` only takes internally-generated diff --git a/ROADMAP.md b/ROADMAP.md index 14c770bd..bc51f3ed 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -63,8 +63,10 @@ across every phase. (deduped, forward-only) whenever a secret is sealed or a path is protected. `revoke` stays metadata-only, now with a hint pointing at `rotate`. Rotation is secrets-only — convergent encryption makes protected-path DEK rotation - security-meaningless (ADR-0014) — and rotation is a future-reads cutover, - not erasure: old ciphertext remains reachable from history. (ADR-0019.) + security-meaningless (ADR-0014; → this objection dissolves for randomized + content in P33/ADR-0043, unlocking rotate-for-paths as a follow-on) — and + rotation is a future-reads cutover, not erasure: old ciphertext remains + reachable from history. (ADR-0019.) - **Phase 12 — Network transport over SSH.** A framed stdio wire protocol mirrors the 8 `Transport` verbs; `sc serve --stdio` dispatches onto the existing `LocalTransport` (CAS, pack verification reused verbatim); the @@ -394,7 +396,8 @@ across every phase. the P5 content scanner) prints one stderr warning steering low-entropy secret basenames (`.env`/`*.key`/`*credentials*`…) toward `sc secret`, citing ADR-0014 — warning-only, convergent encryption's - equality-confirmability stays accepted by design. Secret env-var + equality-confirmability stayed accepted by design (→ closed for new + content in P33/ADR-0043; the nudge itself stays right). Secret env-var boundary: the threat model is tightened to "authorized local process context, NOT strong isolation," and a compile-time pin locks in that `scl_crypto::open`'s `Zeroizing>` plaintext rides unchanged to @@ -506,6 +509,30 @@ across every phase. carrying a signed chunked blob byte-for-byte, the TOFU pin/mismatch/strict/pre-pin lifecycle, and the tightened plaintext gate — run twice, zero residue. (ADR-0042.) +- **Phase 33 — Randomized protected-path encryption.** Closes ADR-0014's + convergent equality-confirmation oracle (the audit's remaining protected-path + High, ticket #40) for all **new** content: every new protected seal uses a + **fresh random DEK + random nonce** (`encrypt_path_randomized`, + `crates/crypto`), so two seals of the same plaintext yield different + ciphertext ids. The store is **dual-read, randomized-write**: a `RANDOMIZED` + perms-byte flag (alongside `PROTECTED`) tags new entries, the blob layout and + `decrypt_path` are unchanged, and there is **no snapshot-tag bump and no + `PROTOCOL_VERSION` change** — a pre-P33 store reads byte-for-byte identically. + Commit's per-path rule is **format-dispatched** (convergent → carry-if- + unchanged, stays convergent; randomized → keyed-PRF stat cache; new → seal + randomized), so no forced mass-migration. Unchanged detection is a + per-checkout cache (`.sc/protected-cache`, ws `.sc/ws/cache-`, `sc work` + in-memory) keyed by a never-committed `.sc/local-key`; the keyed BLAKE3 tag is + authoritative in every lookup (stat is advisory — closes the git racy-stat + class), and the cache leaks nothing (PRF under a local-only key). `sc rewrap` + eagerly re-seals still-convergent blobs randomized (so an upgrading rewrap is + no longer tree-identical). Accepted costs: identical independent edits now + conflict (4a), rewrap is content-changing when it upgrades (4b), identical + plaintext at two paths no longer dedups (4c). Zero new dependencies. Proven by + `demo/run_randomized_demo.sh` (oracle closed, quiet history, cost-4a conflict, + rewrap policy-only on an all-randomized tip — run twice); dual-read of pre-P33 + convergent stores and the convergent→randomized rewrap upgrade are unit-pinned + (the current binary can no longer write a convergent blob). (ADR-0043.) ## Active @@ -709,12 +736,19 @@ scale-&-reach horizon): prefix-only, matching the sparse path-boundary machinery already built; git's `--filter=blob:limit` size-based filtering is a separate axis, deferred. -- **Randomized protected mode (P28).** `sc protect` stays convergent - (deterministic DEK/nonce per plaintext), so ciphertext equality still - confirms plaintext equality without decryption — accepted by design in - ADR-0014, surfaced rather than closed by P28's filename nudge. A - non-convergent DEK/nonce mode for equality-hiding on high-sensitivity - paths is the follow-on. +- **Randomized protected mode — SHIPPED in P33 (ADR-0043).** `sc protect` + used to stay convergent (deterministic DEK/nonce per plaintext), so + ciphertext equality confirmed plaintext equality without decryption — + accepted by design in ADR-0014, surfaced by P28's filename nudge. P33 closes + it: all new protected content seals under a fresh random DEK + nonce + (dual-read for pre-P33 convergent history; `sc rewrap` upgrades a tip on + demand). Moved from deferred to Done above. +- **Rotate-for-paths (P33 follow-on).** ADR-0019 declared per-path DEK rotation + "security-meaningless" under convergent encryption. That objection dissolves + for randomized content (a random DEK is independent of the plaintext), so a + `sc protect … rotate`-style genuine per-path cryptographic cutover — the + protected-path analogue of `sc secret rotate` — is now coherent. Recorded in + ADR-0043, not built. - **fd/stdin secret injection (P28).** `sc run` hands decrypted secrets to the child via its environment — observable by same-user processes, crash dumps, and shell wrappers, per the tightened threat-model wording. diff --git a/docs/THREAT-MODEL.md b/docs/THREAT-MODEL.md index 7a80f4d8..b3196bdd 100644 --- a/docs/THREAT-MODEL.md +++ b/docs/THREAT-MODEL.md @@ -46,17 +46,25 @@ too, not just the src-control-side metadata. revoke` is metadata-only. Real security requires rotating the underlying external credential. -## Protected paths (`sc protect`) — ADR-0014/0026/0027 +## Protected paths (`sc protect`) — ADR-0014/0026/0027/0043 - **Defends:** read-confidentiality of designated file content for a chosen recipient set; an unauthorized clone gets ciphertext it cannot read. -- **Does NOT defend — equality is confirmable:** protected paths use *convergent - encryption* (DEK and nonce derive from `BLAKE3(plaintext)`), so identical - plaintext dedups to identical ciphertext. An observer of the ciphertext can - therefore confirm that two protected files are identical, or that a protected - file matches a **guessed** plaintext. This is a deliberate tradeoff for dedup, - documented in ADR-0014. Low-entropy secrets (API keys, `.env`) belong in - `sc secret`, not `sc protect` — `sc protect` prints a nudge to that effect. +- **Equality-confirmation oracle — closed for new content (P33, ADR-0043), + still true for pre-P33 history until rewrapped.** Protected paths *used to* + use *convergent encryption* (DEK and nonce derive from `BLAKE3(plaintext)`), + so identical plaintext dedups to identical ciphertext and an observer could + confirm that two protected files are identical, or that a protected file + matches a **guessed** plaintext. As of P33, **all content sealed from this + phase on uses a fresh random DEK + random nonce** — two seals of the same + plaintext yield different ciphertext ids, closing the oracle. Convergent + ciphertext already written into history stays readable (dual-read) and stays + equality-confirmable **until `sc rewrap` upgrades the live tip**, which + eagerly re-seals any still-convergent protected blob randomized. Low-entropy + secrets (API keys, `.env`) still belong in `sc secret`, not `sc protect` — + `sc protect` prints a nudge to that effect, and the P28 rationale is + unchanged (governed secret lifecycle beats a protected file even with + randomized sealing). - **Revocation is durable but not retroactive:** `sc revoke` survives merges of pre-revoke branches (revocation tombstones, ADR-0026), so a revoked recipient never seals a *fresh* DEK again. But a recipient who already held a wrap can diff --git a/docs/adr/0014-per-file-permissions-encrypted-paths.md b/docs/adr/0014-per-file-permissions-encrypted-paths.md index a654bf7a..1c8e8bca 100644 --- a/docs/adr/0014-per-file-permissions-encrypted-paths.md +++ b/docs/adr/0014-per-file-permissions-encrypted-paths.md @@ -56,7 +56,10 @@ makes the same plaintext encrypt to different ciphertext every commit — new /id is a deterministic function of the plaintext). This is the standard trade-off for encrypted content-addressed stores and is acceptable for source content; it is documented so users with low-entropy secret files choose Phase 2 secrets - instead. + instead. **Superseded for newly sealed content by ADR-0043 (P33):** all + content sealed from P33 on uses a fresh random DEK + nonce, closing this + oracle; convergent dual-read is retained, so the caveat remains true for + pre-P33 ciphertext still in history until a `sc rewrap` upgrades the tip. - Reuses `scl-crypto` and the recipient model wholesale; no new crypto primitives. - The protection policy is a snapshot-level object like the secrets registry, so it versions and travels with the code. diff --git a/docs/adr/0043-randomized-protected-encryption.md b/docs/adr/0043-randomized-protected-encryption.md new file mode 100644 index 00000000..f518f5ca --- /dev/null +++ b/docs/adr/0043-randomized-protected-encryption.md @@ -0,0 +1,274 @@ +# ADR-0043: Randomized protected-path encryption (dual-read, randomized-write) + +- **Status:** Accepted +- **Date:** 2026-07-10 +- **Phase:** 33 +- **Builds on:** ADR-0014 (P7 convergent protected paths — superseded *for + newly sealed content only*, dual-read retained), ADR-0025 (P15 carry-by-id + discipline the format bit rides through merge/replay), ADR-0027 (P17 + `sc rewrap` — the vehicle for the eager tip upgrade), ADR-0034 (P24 + perms-preserving carry predicate) +- **Issue:** #40 (part of the #24 audit-response map; decisions locked in #30) +- **Spec:** `docs/superpowers/specs/2026-07-10-p33-randomized-protected-encryption-design.md` + +## Context + +ADR-0014's protected paths use **convergent** encryption: the DEK and nonce +derive from `BLAKE3(plaintext)`, so identical plaintext seals to an identical +ciphertext blob id. That preserves the project's bedrock content-addressing +and dedup invariant, but it carries an accepted caveat (ADR-0014, consequence +"Accepted caveat of convergent encryption"): an observer who already holds a +candidate plaintext can confirm whether it is present in the repo, or that two +protected files are identical, from the ciphertext id alone — a +dictionary/equality-confirmation oracle. The 2026-07-09 security audit (ticket +#24's map) flagged this as the remaining protected-path confidentiality +boundary. ADR-0014 itself named the fix — "random per-encryption (fresh DEK + +nonce each time)… may be offered as an opt-in per-policy mode" — and deferred +it as the cost of dedup. P28's `sc protect` low-entropy nudge surfaced the +boundary to operators but did not close it. + +## Decision + +All **new** protected content is sealed under a **fresh random DEK + random +nonce**, so two seals of the same plaintext yield different ciphertext ids — +the oracle is closed for everything sealed from P33 on. Convergent ciphertext +already in history stays readable forever (**dual-read**); nothing is rewritten +or forcibly migrated, and no operator is forced through an upgrade. `sc rewrap` +gains an **eager, on-demand** upgrade of the live tip for operators who want +the old ciphertext gone from the current snapshot. + +### The format change: a perms-byte flag, not a blob-layout change + +The blob layout is **unchanged** — a randomized protected blob is still +`nonce(24) ‖ AEAD-ciphertext` under the same `PATH_AAD` (`b"scl-path-v1"`), +decrypted by the same `decrypt_path`. The only new bit is a format **tag**: +`RANDOMIZED` (`0b0000_0010`, `crates/core/src/object.rs`), always set together +with `PROTECTED` on a newly sealed `TreeEntry.perms` byte. `encrypt_path_randomized` +(`crates/crypto/src/envelope.rs`, plus an `_with_rng` variant for deterministic +tests) is the one new seal primitive; `encrypt_path` (convergent) survives only +for the legacy-format unchanged-comparison described below — no new seal call +site uses it. The tag lives in the **tree entry, not the blob**, for two +reasons: (1) format identification never requires fetching or decrypting blob +bytes — `status`, `diff`, commit, merge, and rewrap all read the perms byte +they already hold; and (2) putting a format discriminator *inside* the blob +would change the blob's bytes and therefore its id, reintroducing exactly the +determinism the change exists to break. The RustCrypto quarantine holds — only +`crates/crypto` grew crypto code; `crates/repo` links `blake3` directly (already +a workspace dependency) for the cache tag below. + +### Dual-read posture: no tag bump, no migration, format-dispatched commit + +There is **no snapshot-tag bump** (`TAG_SNAPSHOT` stays at P16's value 4) and +**no `PROTOCOL_VERSION` change** (stays 3). A pre-P33 store decodes with zero +`RANDOMIZED` bits and behaves byte-for-byte as it did before — a genuine +pre-P33 store reads identically, pinned by library-built convergent-tree +fixtures in the `rewrap`/`merge`/`replay` unit tests. Carry-by-id (the ADR-0025 +P15 discipline, generalized by ADR-0034 to preserve the *source* entry's own +perms) rides the format bit through merge, replay, graft, and sparse carry with +no new plumbing. + +Commit's per-protected-path rule is **format-dispatched, not cache-only** — the +load-bearing choice that prevents a silent mass-migration commit on first use +after upgrade. Dispatched on the *prior tip entry's* format: + +| Prior tip entry | Rule | +|---|---| +| **Convergent** (no `RANDOMIZED` bit) | Convergent re-encrypt-and-compare. Unchanged → carry as-is (stays convergent, history stays quiet, no cache needed). Edited → seal randomized. | +| **Randomized** | Consult the unchanged-detection cache (below); a hit carries the cached id, a miss seals randomized. | +| **New protected path** | Seal randomized. | + +Had dispatch been cache-only — "no cache entry ⇒ re-seal" — the first commit +after upgrading to a P33 binary would have found an empty cache and re-sealed +**every** unchanged protected file randomized, a mass-migration commit nobody +asked for. Format dispatch means unchanged convergent content stays convergent +**forever** until an edit or an explicit `sc rewrap` touches it; the store +migrates only what actually changes. + +### Unchanged detection: a keyed-PRF cache that does not reintroduce the oracle + +Commit still needs only **public keys** — it never decrypts to decide whether a +file changed. Detection is a per-checkout stat cache (`crates/repo/src/cache.rs`): + +- `.sc/local-key`: 32 random bytes, mode **0600 from first byte** (via + `atomic_write_durable`), created lazily, **never committed, never + transferred**. +- Per-checkout cache files: `.sc/protected-cache` for the main working tree; + `.sc/ws/cache-` per ws workspace (see below). Each entry is + `path → (mtime, size, keyed_tag, blob_id)` where + `keyed_tag = BLAKE3-keyed(local_key, plaintext)`. + +**The cache does not reintroduce the equality oracle** precisely because the +tag is a PRF output under a key that never leaves `.sc/` and never travels: the +cache file alone leaks nothing — two identical plaintexts have identical tags +only to a holder of the local key, who by construction already has the working +tree. **Degradation rule:** a lost, stale, or corrupt cache produces spurious +re-seals (fresh randomized ids for unchanged plaintext) — never incorrectness, +never a wrong tree. A corrupt cache line is treated as absent with a stderr +warning; a `.sc/local-key` that cannot be created is a hard error (the cache +cannot be safely keyed without it). + +### Accepted costs (from #40, all landed as predicted) + +- **(4a) Independent identical edits on two branches now conflict.** Under + convergent encryption, both sides editing a protected path to the *same* + plaintext produced the same ciphertext id and merged with an id-level fast + path. Randomized seals give the two edits different ids, so the merge now + conflicts **by construction — even when the plaintexts are trivially + identical** and needs `--identity` (or P23 `sc conflicts`/`sc resolve`) to + resolve, at which point diff3 merges the identical plaintexts cleanly and + re-seals the result randomized. +- **(4b) `sc rewrap` is no longer policy-only / tree-identical when it + upgrades.** A rewrap that re-seals any convergent blob randomized changes + that blob's id and therefore the root tree — the commit is content-changing, + not the byte-identical policy-only commit ADR-0027 described. A second rewrap + over an all-randomized tip converges back to policy-only (wrap-replacement + only, ids unchanged). +- **(4c) Identical plaintext at two paths no longer dedups.** Two protected + paths with byte-identical plaintext seal to two different randomized blob ids + — the intended cost of closing the intra-tip equality oracle between them. + +### P15 merge / P17 rewrap semantic adjustments + +- **P15 (ADR-0025):** unchanged and one-side-changed protected fast paths + survive unchanged (carry-by-id). Both-sides-edited paths now conflict by + construction (cost 4a). The content-merge re-encryption site in `merge.rs` + and replay's `encrypt_protected` path switch to the randomized primitive. + Protection-rule union, wrap union, and revocation-tombstone semantics + (P15/P16) are unchanged; mixed convergent+randomized trees need no special + casing — every path is either carried by id or re-sealed randomized. +- **P17 (ADR-0027):** `sc rewrap` performs the eager upgrade with no new + command or flag. While unwrapping each protected blob's DEK by wrap presence + (as today), it additionally decrypts any still-**convergent** blob and + re-seals it randomized, replaces the wrap list with the rule's + `granted_keys() + escrow`, and records the plaintext it already holds into + the main-tree cache (so the next `sc status`/`commit` stays quiet — rewrap + never rematerializes the working tree). Output gains a `re-sealed N + convergent blob(s)` line. Already-randomized blobs get wrap-replacement only. + One command, one commit, one oplog record, skip-and-report semantics + unchanged; sealing to an empty granted set still fails loudly. + +### Unlocked follow-on: rotate-for-paths (recorded, not built) + +ADR-0019 declared per-path DEK rotation "security-meaningless" under convergent +encryption — the DEK is a deterministic function of the plaintext, so +"rotating" it is either dedup-breaking or a no-op. **That objection dissolves +for randomized content:** a randomized DEK is independent of the plaintext, so +re-sealing a protected path under a fresh DEK is a genuine cryptographic +cutover, exactly as `sc secret rotate` already is for secrets. A +`sc protect … rotate`-style surface is now a coherent follow-on, recorded here +and in `ROADMAP.md`, **not built** in this phase. + +## Consequences + +- The convergent-equality oracle (ADR-0014) is closed for all content sealed + from P33 on; it remains a true property of pre-P33 convergent ciphertext in + history until a `sc rewrap` upgrades that tip. THREAT-MODEL and ARCHITECTURE + are updated to state this split honestly. +- Zero new dependencies: `blake3`/`hex` were already workspace deps; `scl-repo` + now links `blake3` directly for the keyed cache tag. +- Old binaries reading a new store mask-check `PROTECTED` and would misreport + randomized entries as modified in `status` — accepted (no deployed old peers, + same posture as prior format-touching phases). +- The proof splits by concern, recorded honestly: dual-read of genuine pre-P33 + stores is pinned by **library-built convergent-tree fixtures** in the + `rewrap`/`merge`/`replay` unit tests (the current binary can no longer *write* + a convergent blob, so the fixtures build them directly via `encrypt_path`), + while `demo/run_randomized_demo.sh` proves oracle-closure (same plaintext → + different ciphertext ids), quiet history, cost-4a conflicts, and `sc rewrap` + policy-only idempotence on an all-randomized tip against the current binary — + run twice, zero residue. The convergent→randomized rewrap **upgrade** path + (the `re-sealed N convergent blob(s)` line firing) is unit-pinned rather than + demo-pinned, for the same reason the dual-read fixtures are library-built: the + current binary cannot mint a convergent blob to feed a demo. + +## Alternatives considered + +- **Cache-only dispatch (no format tag).** Rejected: an empty cache after a + binary upgrade would mass-migrate every unchanged protected file to + randomized on the first commit. The perms-byte format tag is what makes + "unchanged convergent content stays convergent" hold. See Decision §Dual-read. +- **A snapshot-tag bump / forced migration.** Rejected: dual-read needs zero + read-path changes and lets pre-P33 stores decode byte-for-byte identically; a + forced migration would rewrite history and break dedup for every existing + protected repo with no consent. +- **Per-prefix convergent/randomized choice.** Rejected on #30 — one format for + all new content, upgraded in bulk by `sc rewrap`, is simpler than a per-rule + mode the merge/replay/graft paths would each have to branch on. +- **A separate `sc rewrap --randomize` flag or a new upgrade command.** + Rejected: `sc rewrap` already holds each protected plaintext while re-wrapping + and already commits one policy cutover; folding the re-seal into it needs no + new surface. + +## Refinements discovered during the build + +Every prior phase's Refinements section holds this one to the same bar: every +claim below is checked against the shipped code, not the plan. + +1. **The keyed tag is authoritative in every cache lookup — a deliberate + deviation from #30 decision-3's letter AND the spec §3 table.** Both #30 and + the spec described a stat-hit shortcut (mtime+size match ⇒ carry the cached + id without hashing). The shipped `ProtectedCache::unchanged` + (`crates/repo/src/cache.rs`) **never trusts the stat alone** — the recorded + `(mtime, size)` fields are advisory, and the keyed BLAKE3 tag is recomputed + and compared on every lookup. This closes the classic git "racy-stat" data- + loss class: a same-size edit that lands within the filesystem's mtime + granularity, with the recorded mtime, would otherwise be misread as unchanged + and silently dropped from the commit (pinned by + `racy_stat_same_size_same_mtime_still_misses` and + `tag_is_authoritative_for_hits_and_misses`). The stat shortcut was traded + away because the plaintext is already in memory at every call site (commit, + status, diff all hold the working-tree bytes to seal or compare), keyed + hashing is ~free against that, and "never incorrectness" outranks the stat + shortcut. Stat fields are still recorded for a future fast-path that can + afford the risk; today nothing relies on them. + +2. **`sc protect --to ` no longer wraps UNCHANGED content + at the next commit — an accepted semantic adjustment.** Under convergent + encryption, re-protecting an already-protected prefix for a new recipient + caused the next commit's re-encrypt-and-compare to re-derive each unchanged + file's convergent DEK and wrap it for the new recipient. Under P33, unchanged + protected files carry their **prior wrap list verbatim** (carry-by-id), so a + bare `sc protect … --to ` does not grant the new recipient access to + files that are not re-sealed. This fails **safe** — it under-grants, never + over-grants — and `sc grant --to --identity` (re-wraps every + affected file's existing DEK) and `sc rewrap` (bulk cutover) are the covering + flows. Operator guidance: use `sc grant`, not a second `sc protect`, to add a + recipient to already-committed protected content. + +3. **Rewrap's shared-blob upgrade: deferred old-id wrap removal (review + Critical).** Two protected paths can share ONE convergent blob id (pre-P33 + dedup of identical plaintext at two paths). An earlier draft removed the old + blob's wrap-map entry **inside** the per-path upgrade loop; when the first + path re-sealed and stripped the shared entry, the second path was then found + with "no wrapped DEKs recorded for blob," skipped, and left pointing at a + wrap-orphaned blob at the tip — silent access loss no re-run could repair + (`crates/repo/src/rewrap.rs`). The fix **defers** old-id wrap removal past + the path loop: an old id's wraps are removed only once no kept tree entry + still references it. Each shared path mints its **own** randomized blob, which + also closes the intra-tip equality oracle between those two paths (cost 4c + applied intra-tip). Pinned by the shared-blob rewrap regression test + asserting `blobs_resealed == 2` and two distinct randomized ids. + +4. **The cache is per-checkout, and ws workspaces get their own — a false- + conflict hazard, not a nicety.** The main tree uses `.sc/protected-cache`; a + ws workspace uses `.sc/ws/cache-`, placed **beside** the checkout dir, + never inside `.sc/ws//` (a file inside the checkout would be read back as + an untracked working file at harvest and land in the branch), and removed + with the workspace at teardown; `sc work`'s one-shot ephemeral agents use an + **in-memory** cache (`ProtectedCache::open(..., None)`, never touches disk). + Without per-workspace caches, an untouched protected file in a forked + workspace has no cache entry, so harvest's format-dispatched detection would + re-seal it randomized — giving it a different id from the base and turning + every untouched protected file into an identical-edit **conflict** at harvest + (the cost-4a mechanism, fired spuriously), with a `work-` fallback instead + of a clean auto-merge. The per-workspace cache reproduces the base tree id on + an untouched file, preserving P20's clean-auto-merge and crash-recovery + `UpToDate` idempotency. + +5. **All production cache saves are best-effort and ordered after ref moves.** + A cache save never gates a commit or a ref update: the ref move is the atomic + commit point (unchanged from every prior phase), and the cache is refreshed + afterward on a best-effort basis (a save failure warns on stderr, never + errors). Combined with the 0600-from-first-byte local key and the keyed-PRF + tag, this is why the cache is pure optimization — its worst failure mode is a + spurious re-seal, never a lost edit or a leaked oracle. diff --git a/docs/adr/README.md b/docs/adr/README.md index 975d580c..d650ec36 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -53,6 +53,7 @@ supersedes the old one and update the **Status** line of the superseded record. | [0040](0040-sc-http-access-control.md) | sc+http access control (read-only, fail-closed bind, bearer tokens) | Accepted | 29 | | [0041](0041-listener-resource-limits.md) | Listener resource limits (connection cap, session timeout, spool cap, accept backoff) | Accepted | 31 | | [0042](0042-in-binary-tls-sc-https.md) | In-binary TLS — `sc+https://` via rustls | Accepted | 32 | +| [0043](0043-randomized-protected-encryption.md) | Randomized protected-path encryption (dual-read, randomized-write) | Accepted | 33 | See [`ROADMAP.md`](../../ROADMAP.md) for the phase sequence. ADR numbers are assigned in creation order, so 0017 — the secret scanner — sequences as phase P5, From b136d7868c0b89de38029af9c9f9f8ec07f8ee64 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Sat, 11 Jul 2026 08:34:34 +0300 Subject: [PATCH 19/22] P33: fmt pass + review-minor cleanups MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - cargo fmt --all across ~6 files left non-canonical by earlier P33 commits - merge.rs: tighten the content-divergent re-seal test's final perms assertion from a loose bitmask check to an exact PROTECTED|RANDOMIZED equality, pinning the RANDOMIZED bit no other test covers - ws.rs: correct the HarvestResult::Unchanged arm's comment — it is unreachable in practice (harvest re-diffs the same persisted cache state moments after ws_changed_for already confirmed a diff), kept only for match exhaustiveness - run_randomized_demo.sh: note that the RETURN trap removes the cwd, so later paths in run_demo must stay absolute Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/repo/src/cache.rs | 67 ++++++++++++++++++++++++++++-------- crates/repo/src/merge.rs | 28 ++++++++++++--- crates/repo/src/protect.rs | 12 +++++-- crates/repo/src/repo.rs | 26 +++++++------- crates/repo/src/workspace.rs | 32 ++++++++++++++--- crates/repo/src/worktree.rs | 5 ++- crates/repo/src/ws.rs | 19 ++++------ demo/run_randomized_demo.sh | 4 +++ 8 files changed, 139 insertions(+), 54 deletions(-) diff --git a/crates/repo/src/cache.rs b/crates/repo/src/cache.rs index 2ac1213f..c65a7cca 100644 --- a/crates/repo/src/cache.rs +++ b/crates/repo/src/cache.rs @@ -136,7 +136,15 @@ impl ProtectedCache { let size: u64 = it.next()?.parse().ok()?; let tag: [u8; 32] = hex::decode(it.next()?).ok()?.try_into().ok()?; let blob_id = ObjectId::from_str(it.next()?).ok()?; - Some((it.next()?.to_string(), CacheEntry { mtime_ns, size, tag, blob_id })) + Some(( + it.next()?.to_string(), + CacheEntry { + mtime_ns, + size, + tag, + blob_id, + }, + )) })(); match parsed { Some((path, e)) => { @@ -151,7 +159,12 @@ impl ProtectedCache { } } } - ProtectedCache { key, root, path, entries } + ProtectedCache { + key, + root, + path, + entries, + } } fn tag(&self, plaintext: &[u8]) -> [u8; 32] { @@ -186,8 +199,15 @@ impl ProtectedCache { pub(crate) fn record(&mut self, rel: &str, plaintext: &[u8], blob_id: ObjectId) { if let Some((mtime_ns, size)) = Self::stat(&self.root.join(rel)) { let tag = self.tag(plaintext); - self.entries - .insert(rel.to_string(), CacheEntry { mtime_ns, size, tag, blob_id }); + self.entries.insert( + rel.to_string(), + CacheEntry { + mtime_ns, + size, + tag, + blob_id, + }, + ); } } @@ -195,7 +215,9 @@ impl ProtectedCache { /// rename + fsync parent dir, no residue on failure); no-op for an /// ephemeral cache. pub(crate) fn save(&self) -> Result<()> { - let Some(path) = &self.path else { return Ok(()) }; + let Some(path) = &self.path else { + return Ok(()); + }; let mut out = String::new(); for (p, e) in &self.entries { out.push_str(&format!( @@ -270,8 +292,11 @@ mod tests { std::fs::write(&abs, b"v1").unwrap(); let id = ObjectId::of(b"cipher-of-v1"); - let mut c = - ProtectedCache::open(key, layout.root.clone(), Some(layout.protected_cache_path())); + let mut c = ProtectedCache::open( + key, + layout.root.clone(), + Some(layout.protected_cache_path()), + ); assert_eq!(c.unchanged("secret.txt", b"v1"), None, "empty cache misses"); c.record("secret.txt", b"v1", id); @@ -302,8 +327,11 @@ mod tests { let id = ObjectId::of(b"cipher-of-aaaa"); let original_mtime = std::fs::metadata(&abs).unwrap().modified().unwrap(); - let mut c = - ProtectedCache::open(key, layout.root.clone(), Some(layout.protected_cache_path())); + let mut c = ProtectedCache::open( + key, + layout.root.clone(), + Some(layout.protected_cache_path()), + ); c.record("secret.txt", b"aaaa", id); // Overwrite with same-length content, then force the racy condition @@ -333,19 +361,28 @@ mod tests { std::fs::write(&abs, b"v").unwrap(); let id = ObjectId::of(b"c"); - let mut c = - ProtectedCache::open(key, layout.root.clone(), Some(layout.protected_cache_path())); + let mut c = ProtectedCache::open( + key, + layout.root.clone(), + Some(layout.protected_cache_path()), + ); c.record("a b/spaced name.txt", b"v", id); c.save().unwrap(); - let c2 = - ProtectedCache::open(key, layout.root.clone(), Some(layout.protected_cache_path())); + let c2 = ProtectedCache::open( + key, + layout.root.clone(), + Some(layout.protected_cache_path()), + ); assert_eq!(c2.unchanged("a b/spaced name.txt", b"v"), Some(id)); // Corrupt file: degrades to empty, never errors. std::fs::write(layout.protected_cache_path(), b"garbage\nlines\n").unwrap(); - let c3 = - ProtectedCache::open(key, layout.root.clone(), Some(layout.protected_cache_path())); + let c3 = ProtectedCache::open( + key, + layout.root.clone(), + Some(layout.protected_cache_path()), + ); assert_eq!(c3.unchanged("a b/spaced name.txt", b"v"), None); cleanup(&layout); } diff --git a/crates/repo/src/merge.rs b/crates/repo/src/merge.rs index cd4cfdf3..559201d3 100644 --- a/crates/repo/src/merge.rs +++ b/crates/repo/src/merge.rs @@ -948,7 +948,10 @@ mod tests { let cb = enc_file_rand(&mut bp, &alice_pk, b"v1\n"); let co = enc_file_rand(&mut op, &alice_pk, b"v2\n"); let ct = enc_file_rand(&mut tp, &alice_pk, b"v2\n"); - assert_ne!(co, ct, "independent randomized seals of identical plaintext differ"); + assert_ne!( + co, ct, + "independent randomized seals of identical plaintext differ" + ); let repo = VfsRepo::new(Store::with_budget(1 << 20)); let base = tree_with_perms(&repo, &[("secret/a.txt", cb, PROTECTED | RANDOMIZED)]); @@ -980,9 +983,16 @@ mod tests { assert_eq!(fm.files.len(), 1); let f = &fm.files[0]; assert_eq!(f.path, "secret/a.txt"); - assert_eq!(f.bytes, b"v2\n", "identical plaintexts merge to v2 with no conflict"); + assert_eq!( + f.bytes, b"v2\n", + "identical plaintexts merge to v2 with no conflict" + ); assert!(f.needs_encrypt, "merged plaintext pending re-encryption"); - assert!(f.perms & PROTECTED != 0); + assert_eq!( + f.perms, + PROTECTED | RANDOMIZED, + "content-divergent re-seal path pins the RANDOMIZED bit" + ); } #[test] @@ -1043,7 +1053,11 @@ mod tests { let plain = fm.files.iter().find(|f| f.path == "plain.txt").unwrap(); assert_eq!(plain.bytes, b"p2\n", "ours' plain edit wins"); - let conv = fm.files.iter().find(|f| f.path == "secret/conv.txt").unwrap(); + let conv = fm + .files + .iter() + .find(|f| f.path == "secret/conv.txt") + .unwrap(); assert_eq!(conv.bytes, conv_theirs, "convergent entry takes theirs' id"); assert!(conv.perms & PROTECTED != 0); let conv_theirs_id = Object::blob(conv_theirs).id(); @@ -1052,7 +1066,11 @@ mod tests { "surviving convergent blob's wraps carried" ); - let rand = fm.files.iter().find(|f| f.path == "secret/rand.txt").unwrap(); + let rand = fm + .files + .iter() + .find(|f| f.path == "secret/rand.txt") + .unwrap(); assert_eq!( rand.bytes, rand_base, "untouched randomized entry carries its exact blob id" diff --git a/crates/repo/src/protect.rs b/crates/repo/src/protect.rs index acd4b863..75bbac49 100644 --- a/crates/repo/src/protect.rs +++ b/crates/repo/src/protect.rs @@ -151,7 +151,12 @@ pub(crate) fn encrypt_protected( .map(|pk| scl_crypto::wrap_dek_for(&dek, &scl_crypto::PublicKey::from_bytes(*pk))) .collect(); fresh_wrapped.insert(blob_id, wks); - all.push((path, blob_bytes, mode, scl_core::PROTECTED | scl_core::RANDOMIZED)); + all.push(( + path, + blob_bytes, + mode, + scl_core::PROTECTED | scl_core::RANDOMIZED, + )); } Ok((all, fresh_wrapped)) } @@ -197,7 +202,10 @@ mod tests { }; let (a1, _) = f(b"same"); let (a2, _) = f(b"same"); - assert_ne!(a1[0].1, a2[0].1, "two seals of one plaintext must differ (oracle closed)"); + assert_ne!( + a1[0].1, a2[0].1, + "two seals of one plaintext must differ (oracle closed)" + ); assert_eq!(a1[0].3, scl_core::PROTECTED | scl_core::RANDOMIZED); } diff --git a/crates/repo/src/repo.rs b/crates/repo/src/repo.rs index ff0d0590..c38bdb36 100644 --- a/crates/repo/src/repo.rs +++ b/crates/repo/src/repo.rs @@ -455,10 +455,7 @@ impl Repo { // stat path against its own checkout root (P33 Task 7), // so a workspace commit stats the workspace file, not // the host's. - cache - .as_deref() - .and_then(|c| c.unchanged(&path, &bytes)) - == Some(*blob_id) + cache.as_deref().and_then(|c| c.unchanged(&path, &bytes)) == Some(*blob_id) }; if unchanged { Some((*blob_id, *perms)) @@ -3731,7 +3728,10 @@ mod tests { std::fs::write(root.join("readme.md"), b"hello").unwrap(); let c2 = repo.commit("alice", "two").unwrap(); let id2 = p33_entry_id(&repo, &c2, "secret/db.txt"); - assert_eq!(id1, id2, "unchanged protected content must carry, not re-seal"); + assert_eq!( + id1, id2, + "unchanged protected content must carry, not re-seal" + ); drop(repo); std::fs::remove_dir_all(&root).unwrap(); @@ -3811,18 +3811,16 @@ mod tests { std::fs::write(root.join("readme.md"), b"more").unwrap(); let c3 = repo.commit("alice", "unrelated2").unwrap(); let id3 = p33_entry_id(&repo, &c3, "secret/x"); - assert_ne!(id3, randomized_id, "lost cache degrades to a spurious re-seal"); + assert_ne!( + id3, randomized_id, + "lost cache degrades to a spurious re-seal" + ); // Never incorrectness: the re-sealed blob still decrypts to the plaintext. let bytes = p33_blob_bytes(&repo, &id3); let snap3 = repo.snapshot(&c3).unwrap(); - let got = crate::protect::decrypt_with( - &bytes, - &id3, - &[&snap3.protection], - &alice_sk, - "secret/x", - ) - .unwrap(); + let got = + crate::protect::decrypt_with(&bytes, &id3, &[&snap3.protection], &alice_sk, "secret/x") + .unwrap(); assert_eq!(&got[..], b"top secret"); drop(repo); diff --git a/crates/repo/src/workspace.rs b/crates/repo/src/workspace.rs index 70d4cd37..a5b2cb46 100644 --- a/crates/repo/src/workspace.rs +++ b/crates/repo/src/workspace.rs @@ -414,9 +414,15 @@ mod tests { let repo = Repo::open(&root).unwrap(); let tip = repo.head_tip().unwrap().unwrap(); let dir = scratch.join("ws1"); - let skipped = - materialize_workspace(&repo, tip, &dir, None, &crate::sparse::Sparse::default(), None) - .unwrap(); + let skipped = materialize_workspace( + &repo, + tip, + &dir, + None, + &crate::sparse::Sparse::default(), + None, + ) + .unwrap(); assert!(skipped.is_empty()); assert_eq!( std::fs::read_to_string(dir.join("a.txt")).unwrap(), @@ -456,7 +462,15 @@ mod tests { let repo = Repo::open(&root).unwrap(); let tip = repo.head_tip().unwrap().unwrap(); let dir = scratch.join("ws1"); - materialize_workspace(&repo, tip, &dir, None, &crate::sparse::Sparse::default(), None).unwrap(); + materialize_workspace( + &repo, + tip, + &dir, + None, + &crate::sparse::Sparse::default(), + None, + ) + .unwrap(); let res = harvest_workspace( &repo, tip, @@ -483,7 +497,15 @@ mod tests { let repo = Repo::open(&root).unwrap(); let tip = repo.head_tip().unwrap().unwrap(); let dir = scratch.join("ws1"); - materialize_workspace(&repo, tip, &dir, None, &crate::sparse::Sparse::default(), None).unwrap(); + materialize_workspace( + &repo, + tip, + &dir, + None, + &crate::sparse::Sparse::default(), + None, + ) + .unwrap(); // An AWS-style key id trips the P5 pattern rules. std::fs::write(dir.join("leak.txt"), "AKIAIOSFODNN7EXAMPLE\n").unwrap(); let res = harvest_workspace( diff --git a/crates/repo/src/worktree.rs b/crates/repo/src/worktree.rs index 36c97b2d..07cabe6c 100644 --- a/crates/repo/src/worktree.rs +++ b/crates/repo/src/worktree.rs @@ -992,7 +992,10 @@ mod tests { Some(&mut cache), ) .unwrap(); - assert!(skipped.is_empty(), "authorized identity decrypts, nothing skipped"); + assert!( + skipped.is_empty(), + "authorized identity decrypts, nothing skipped" + ); assert_eq!( std::fs::read(layout.root.join("secret/x")).unwrap(), pt, diff --git a/crates/repo/src/ws.rs b/crates/repo/src/ws.rs index 2ced0ccc..3990ff8b 100644 --- a/crates/repo/src/ws.rs +++ b/crates/repo/src/ws.rs @@ -742,18 +742,13 @@ impl Repo { }); } crate::workspace::HarvestResult::Unchanged => { - // `ws_changed_for` already confirmed a diff above, and - // `harvest_workspace` re-diffs independently against its - // own `tip` argument (session.base_snapshot, same value - // here) using the SAME per-workspace cache — so the two - // checks agree in the common case, but this arm is - // genuinely reachable: a cold or lost cache (e.g. this - // workspace's cache file was never written, or the host's - // `.sc/local-key` changed) can make `ws_changed_for` - // spuriously report "changed" for an untouched - // RANDOMIZED protected path while `harvest_workspace`'s - // own diff — same inputs, same cache lookup — still - // resolves it as unchanged. Handled, not a panic guard. + // Unreachable in practice: `ws_changed_for` already + // confirmed a diff above, and `harvest_workspace` + // re-diffs moments later against the SAME persisted + // cache state (same `tip` argument, same per-workspace + // cache, no intervening write) — the two checks cannot + // disagree. Kept for match exhaustiveness, not as a + // reachable panic guard. resolve_and_teardown(self.layout(), &mut session, i, &dir, None)?; outcomes.push(WsHarvestOutcome::Unchanged { index: i }); } diff --git a/demo/run_randomized_demo.sh b/demo/run_randomized_demo.sh index 06facd3e..67765eaa 100755 --- a/demo/run_randomized_demo.sh +++ b/demo/run_randomized_demo.sh @@ -74,6 +74,10 @@ run_demo() { alice_out=$("$SC" keygen --out "$keys/alice") alice_pk=$(echo "$alice_out" | grep 'public key' | awk '{print $3}') + # NB: the RETURN trap above removes $work (and therefore $repo, our new + # cwd) on function exit, so every path used below this point must stay + # absolute — a relative path minted here would resolve against a + # directory that's already gone by the time it's read. cd "$repo" "$SC" init >/dev/null printf '[recipients]\nalice = "%s"\n' "$alice_pk" > .sc/recipients.toml From 871c2b22128871766fd42d636b22b5a764df6d73 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Sat, 11 Jul 2026 08:38:56 +0300 Subject: [PATCH 20/22] =?UTF-8?q?P33:=20fix=20merge.rs=20test=20assertion?= =?UTF-8?q?=20=E2=80=94=20pin=20the=20field=20three=5Fway=5Ffiles=20actual?= =?UTF-8?q?ly=20sets?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The prior cleanup commit tightened this test's final perms assertion to PROTECTED | RANDOMIZED, assuming three_way_files surfaces the RANDOMIZED bit for a content-divergent (needs_encrypt) protected entry. It doesn't: repo.rs's completion path never reads f.perms for a needs_encrypt entry (only bytes/mode/recipients), so RANDOMIZED is applied downstream by encrypt_protected at actual re-encryption time. f.perms here only ever carries the PROTECTED marker bit (see the content-divergent match arm's mask). Corrected the assertion to PROTECTED, with a comment explaining why RANDOMIZED can't be pinned at this call site — that coverage would need a commit-then-inspect-snapshot test instead. Caught by cargo test during the P33 validation sweep. Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/repo/src/merge.rs | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/crates/repo/src/merge.rs b/crates/repo/src/merge.rs index 559201d3..695afd6e 100644 --- a/crates/repo/src/merge.rs +++ b/crates/repo/src/merge.rs @@ -988,10 +988,17 @@ mod tests { "identical plaintexts merge to v2 with no conflict" ); assert!(f.needs_encrypt, "merged plaintext pending re-encryption"); + // `three_way_files` never surfaces RANDOMIZED here: for a + // needs_encrypt entry the caller (repo.rs's completion path) reads + // `f.bytes`/`f.mode`/the matched rule's recipients, never `f.perms` + // — the RANDOMIZED bit is applied downstream by `encrypt_protected` + // at actual re-encryption time, not by this merge step. `f.perms` + // here only ever carries the PROTECTED marker bit (see the mask at + // the content-divergent match arm above). assert_eq!( - f.perms, - PROTECTED | RANDOMIZED, - "content-divergent re-seal path pins the RANDOMIZED bit" + f.perms, PROTECTED, + "three_way_files marks the path PROTECTED; RANDOMIZED is applied \ + at downstream re-encryption, not surfaced here" ); } From 7df7b88dd8caee3c8bb66a26139f09b7c296f7cb Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Sat, 11 Jul 2026 09:01:46 +0300 Subject: [PATCH 21/22] =?UTF-8?q?P33:=20final-review=20fixes=20=E2=80=94?= =?UTF-8?q?=20threat-model=20precision=20(F1)=20+=20doc/comment=20correcti?= =?UTF-8?q?ons?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- ROADMAP.md | 20 ++++++++++ crates/repo/src/conflicts.rs | 21 +++++++---- crates/repo/src/repo.rs | 13 ++++--- docs/THREAT-MODEL.md | 14 +++++-- .../0043-randomized-protected-encryption.md | 37 +++++++++++++------ 5 files changed, 78 insertions(+), 27 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index bc51f3ed..b7a3745f 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -749,6 +749,26 @@ scale-&-reach horizon): `sc protect … rotate`-style genuine per-path cryptographic cutover — the protected-path analogue of `sc secret rotate` — is now coherent. Recorded in ADR-0043, not built. +- **`sc rewrap` on partial clones (P33 final-review follow-on).** `sc rewrap` + loads each protected plaintext whole into RAM and has no + `PartialCloneUnsupported` guard, unlike merge/rebase/ws-fork/ws-harvest/`sc + work` (P27); a paths-only/streaming tree rebuild (or the same refusal-first + posture) is the follow-on. +- **Protected-cache hygiene (P33 final-review follow-on).** `.sc/protected-cache` + and `.sc/ws/cache-` entries for a deleted or renamed path are never + pruned — unbounded but benign growth (a stale entry just goes unmatched). + Periodic or commit-time pruning of dead entries is the follow-on. +- **Theirs-side resolve carry (P33 final-review follow-on).** `sc resolve + --theirs` on a PROTECTED path records into the main-tree cache today, but + completion's format-dispatched carry only consults the cache against the + prior TIP (ours') entry, so a theirs-side recording never matches and + completion re-seals randomized anyway (spurious-but-safe). Extending the + carry-candidate map to also check `merge_head`/decided-root entries would + let a theirs-side resolution skip the re-seal too. +- **Non-unix local-key fallback (P33 final-review follow-on).** `.sc/local-key` + is minted 0600-from-first-byte via a unix-only `OpenOptions::mode` call; + there is no non-unix fallback today, so the key file is left world-readable + on a non-unix target. A non-unix permission-hardening path is deferred. - **fd/stdin secret injection (P28).** `sc run` hands decrypted secrets to the child via its environment — observable by same-user processes, crash dumps, and shell wrappers, per the tightened threat-model wording. diff --git a/crates/repo/src/conflicts.rs b/crates/repo/src/conflicts.rs index bc27a815..6a603e39 100644 --- a/crates/repo/src/conflicts.rs +++ b/crates/repo/src/conflicts.rs @@ -236,13 +236,20 @@ impl Repo { // P33 Task 8: a resolved PROTECTED path's plaintext is now // sitting on disk exactly as the chosen side's ciphertext - // decrypted it — record it in the main-tree cache so the - // completing `sc commit`/`sc rebase --continue` recognizes it - // as unchanged (`unchanged(path, plaintext) == Some(blob_id)`) - // and carries the existing ciphertext instead of re-sealing - // under a fresh random nonce. Plain paths never touch the - // cache — it exists only to skip re-sealing convergent-free - // randomized ciphertext. + // decrypted it — record it in the main-tree cache regardless + // of which side was chosen. This only pays off for an + // `--ours` resolution: completion's format-dispatched carry + // (`snapshot_files`) consults the cache against the prior + // TIP entry's blob id, so an `--ours` recording matches and + // the completing `sc commit`/`sc rebase --continue` carries + // the existing ciphertext instead of re-sealing under a + // fresh random nonce. A `--theirs` recording keys the cache + // to a blob id completion never looks up against (the tip is + // still ours), so it goes unmatched and completion re-seals + // randomized anyway — spurious cache-population, but benign + // (never incorrect, just a wasted write). Plain paths never + // touch the cache — it exists only to skip re-sealing + // convergent-free randomized ciphertext. if kind == ConflictKind::Protected { let triple = self.op_triple()?; let chosen_snapshot = match side { diff --git a/crates/repo/src/repo.rs b/crates/repo/src/repo.rs index c38bdb36..95c0c695 100644 --- a/crates/repo/src/repo.rs +++ b/crates/repo/src/repo.rs @@ -687,11 +687,14 @@ impl Repo { // here is still the tip's own full map (`prior` is taken // below, after this point) and already carries every wrap // the tip itself could offer; union in any entry - // `fresh_wrapped` doesn't already know about. Convergent - // encryption keeps a blob's id stable regardless of who - // grafted it, so reusing the prior wrap bytes verbatim is - // correct, not just convenient — no re-encryption, no key - // material touched. + // `fresh_wrapped` doesn't already know about. The graft + // splices the subtree in BY ID and never re-derives or + // touches the blob's DEK, so reusing the prior wrap bytes + // verbatim is correct — this holds for a RANDOMIZED blob + // exactly as it does for a convergent one (carry-by-id, not + // convergent-id-stability, is the load-bearing property; a + // randomized blob's id is just as stable across a + // by-id graft as a convergent one's). for (id, wks) in &protection.wrapped { fresh_wrapped.entry(*id).or_insert_with(|| wks.clone()); } diff --git a/docs/THREAT-MODEL.md b/docs/THREAT-MODEL.md index b3196bdd..734a6599 100644 --- a/docs/THREAT-MODEL.md +++ b/docs/THREAT-MODEL.md @@ -58,9 +58,17 @@ too, not just the src-control-side metadata. matches a **guessed** plaintext. As of P33, **all content sealed from this phase on uses a fresh random DEK + random nonce** — two seals of the same plaintext yield different ciphertext ids, closing the oracle. Convergent - ciphertext already written into history stays readable (dual-read) and stays - equality-confirmable **until `sc rewrap` upgrades the live tip**, which - eagerly re-seals any still-convergent protected blob randomized. Low-entropy + ciphertext already written into history stays readable (dual-read) and + remains equality-confirmable **forever** to anyone holding a clone — this is + the same rotation ≠ erasure boundary ADR-0019 already names for secrets: + content addressing means the old convergent object stays reachable in + history regardless of what happens at the tip. `sc rewrap` re-seals a + still-convergent blob randomized **at the live tip**, which stops that + plaintext's convergent form from propagating into *future* snapshots built + on top — it does not erase the historical convergent object, which any clone + that already fetched it (or fetches it later) can still equality-confirm + against. Real cutover of guessable content means changing the content (or + underlying credential) itself, not just re-sealing at the tip. Low-entropy secrets (API keys, `.env`) still belong in `sc secret`, not `sc protect` — `sc protect` prints a nudge to that effect, and the P28 rationale is unchanged (governed secret lifecycle beats a protected file even with diff --git a/docs/adr/0043-randomized-protected-encryption.md b/docs/adr/0043-randomized-protected-encryption.md index f518f5ca..513ed912 100644 --- a/docs/adr/0043-randomized-protected-encryption.md +++ b/docs/adr/0043-randomized-protected-encryption.md @@ -90,8 +90,10 @@ Commit still needs only **public keys** — it never decrypts to decide whether file changed. Detection is a per-checkout stat cache (`crates/repo/src/cache.rs`): - `.sc/local-key`: 32 random bytes, mode **0600 from first byte** (via - `atomic_write_durable`), created lazily, **never committed, never - transferred**. + `OpenOptions::create_new` + `mode(0o600)`, so the file is never briefly + world-readable between creation and permission-setting), created lazily, + **never committed, never transferred**. The per-checkout cache file itself + (below) is written via `atomic_write_durable`. - Per-checkout cache files: `.sc/protected-cache` for the main working tree; `.sc/ws/cache-` per ws workspace (see below). Each entry is `path → (mtime, size, keyed_tag, blob_id)` where @@ -161,9 +163,15 @@ and in `ROADMAP.md`, **not built** in this phase. ## Consequences - The convergent-equality oracle (ADR-0014) is closed for all content sealed - from P33 on; it remains a true property of pre-P33 convergent ciphertext in - history until a `sc rewrap` upgrades that tip. THREAT-MODEL and ARCHITECTURE - are updated to state this split honestly. + from P33 on; pre-P33 convergent ciphertext already in history stays + equality-confirmable **forever** to anyone holding a clone — the same + rotation ≠ erasure boundary ADR-0019 names for secrets, since content + addressing keeps the old object reachable regardless of what happens at the + tip. `sc rewrap` stops a still-convergent blob's plaintext from propagating + into *future* snapshots at the live tip; it does not erase the historical + convergent object. Real cutover of guessable content means changing the + content (or underlying credential) itself. THREAT-MODEL and ARCHITECTURE are + updated to state this split honestly. - Zero new dependencies: `blake3`/`hex` were already workspace deps; `scl-repo` now links `blake3` directly for the keyed cache tag. - Old binaries reading a new store mask-check `PROTECTED` and would misreport @@ -265,10 +273,15 @@ claim below is checked against the shipped code, not the plan. an untouched file, preserving P20's clean-auto-merge and crash-recovery `UpToDate` idempotency. -5. **All production cache saves are best-effort and ordered after ref moves.** - A cache save never gates a commit or a ref update: the ref move is the atomic - commit point (unchanged from every prior phase), and the cache is refreshed - afterward on a best-effort basis (a save failure warns on stderr, never - errors). Combined with the 0600-from-first-byte local key and the keyed-PRF - tag, this is why the cache is pure optimization — its worst failure mode is a - spurious re-seal, never a lost edit or a leaked oracle. +5. **A cache save never gates an operation, and the ref move stays the commit + point.** This is not "every save is strictly ordered after every ref move" + — `assemble_completion_snapshot` deliberately saves the cache *before* its + callers move the ref (documented in-code; benign, since a crash between the + two just costs a spurious re-seal next time, not a lost edit) and several + call sites move no ref at all. The property that actually holds everywhere: + a cache save is best-effort (a save failure warns on stderr, never errors) + and never blocks or gates the operation it's attached to, while the ref + move — wherever one occurs — remains the sole atomic commit point, unchanged + from every prior phase. Combined with the 0600-from-first-byte local key and + the keyed-PRF tag, this is why the cache is pure optimization — its worst + failure mode is a spurious re-seal, never a lost edit or a leaked oracle. From bfca3c4277ba98d9c1b2cf7880bb1a6280461224 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Sat, 11 Jul 2026 09:04:30 +0300 Subject: [PATCH 22/22] =?UTF-8?q?P33:=20F1=20residuals=20=E2=80=94=20purge?= =?UTF-8?q?=20the=20remaining=20'until=20rewrap'=20overclaims=20(THREAT-MO?= =?UTF-8?q?DEL=20heading,=20ADR-0014=20pointer,=20CLAUDE.md=20P28=20bounda?= =?UTF-8?q?ry)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- CLAUDE.md | 5 +++-- docs/THREAT-MODEL.md | 5 +++-- docs/adr/0014-per-file-permissions-encrypted-paths.md | 7 +++++-- 3 files changed, 11 insertions(+), 6 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 9f505d18..a435bc77 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1210,8 +1210,9 @@ isolation," and a compile-time pin locks in that `scl_crypto::open`'s `OsString` child-env hand-off. **Two accepted boundaries, unchanged by design:** convergent encryption stays equality-confirmable (ADR-0014 — **closed for new content in P33/ADR-0043**, which seals all new protected -content randomized; the caveat holds only for pre-P33 convergent ciphertext in -history until `sc rewrap` upgrades it); the secret's child-env +content randomized; the caveat holds forever for pre-P33 convergent ciphertext +in history — `sc rewrap` only stops it propagating into future snapshots, +rotation ≠ erasure per ADR-0019); the secret's child-env copy is fundamental and un-zeroizable (fd/stdin injection is deferred, not a bug) — the parent's own decrypted buffer is what gets zeroized. Every prior demo stays green plus new pinned regression tests across all four diff --git a/docs/THREAT-MODEL.md b/docs/THREAT-MODEL.md index 734a6599..96ae7689 100644 --- a/docs/THREAT-MODEL.md +++ b/docs/THREAT-MODEL.md @@ -50,8 +50,9 @@ too, not just the src-control-side metadata. - **Defends:** read-confidentiality of designated file content for a chosen recipient set; an unauthorized clone gets ciphertext it cannot read. -- **Equality-confirmation oracle — closed for new content (P33, ADR-0043), - still true for pre-P33 history until rewrapped.** Protected paths *used to* +- **Equality-confirmation oracle — closed for new content (P33, ADR-0043); + pre-P33 convergent ciphertext in history stays confirmable forever + (rotation ≠ erasure, ADR-0019).** Protected paths *used to* use *convergent encryption* (DEK and nonce derive from `BLAKE3(plaintext)`), so identical plaintext dedups to identical ciphertext and an observer could confirm that two protected files are identical, or that a protected file diff --git a/docs/adr/0014-per-file-permissions-encrypted-paths.md b/docs/adr/0014-per-file-permissions-encrypted-paths.md index 1c8e8bca..b3173e58 100644 --- a/docs/adr/0014-per-file-permissions-encrypted-paths.md +++ b/docs/adr/0014-per-file-permissions-encrypted-paths.md @@ -58,8 +58,11 @@ makes the same plaintext encrypt to different ciphertext every commit — new is documented so users with low-entropy secret files choose Phase 2 secrets instead. **Superseded for newly sealed content by ADR-0043 (P33):** all content sealed from P33 on uses a fresh random DEK + nonce, closing this - oracle; convergent dual-read is retained, so the caveat remains true for - pre-P33 ciphertext still in history until a `sc rewrap` upgrades the tip. + oracle; convergent dual-read is retained, so the caveat remains true forever + for pre-P33 ciphertext in history — `sc rewrap` re-seals the live tip, which + stops the convergent form propagating into future snapshots, but the + historical convergent objects stay reachable and confirmable (rotation ≠ + erasure, ADR-0019). - Reuses `scl-crypto` and the recipient model wholesale; no new crypto primitives. - The protection policy is a snapshot-level object like the secrets registry, so it versions and travels with the code.