diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index eefb76d8..4b232f91 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -105,8 +105,10 @@ a retired pre-P16 snapshot encoding, refused with a clear error): - **Blob** — raw file contents. - **Tree** — a sorted directory listing mapping a name to `(kind, ObjectId, mode, permissions)`. The per-entry `permissions` byte carries the `PROTECTED` - bit (P7): a protected entry's blob is convergently-encrypted ciphertext, and - the bit is what makes "protected" and "ciphertext" synonymous in every tree. + bit (P7): a protected entry's blob is encrypted ciphertext — convergent for + pre-P33 seals, randomized (a companion `RANDOMIZED` bit, P33/ADR-0043) for + new ones — and the bit is what makes "protected" and "ciphertext" synonymous + in every tree. - **Snapshot** — the Jujutsu-inspired analogue of a commit: a root tree id plus metadata (parent snapshot ids, author, timestamp, message), a `secrets` side-registry (name → secret-object id, so secrets are env vars, not files), @@ -202,8 +204,9 @@ transport: `repo` speaks a framed wire protocol over ssh:// (P12) and sc+http:// All cryptography is quarantined in `crates/crypto`; `core` and `repo` see only opaque bytes. The crate owns three surfaces, all built on the RustCrypto stack: -**envelope encryption** (secrets, below), **convergent encryption** for protected -paths, and **Ed25519 signing** for provenance. +**envelope encryption** (secrets, below), **encryption for protected paths** +(randomized DEK/nonce for new content since P33, convergent dual-read for +pre-P33 history — see below), and **Ed25519 signing** for provenance. Committed secrets use **envelope encryption**. Each secret object carries ciphertext encrypted under a fresh per-secret data-encryption key (DEK) using @@ -231,14 +234,27 @@ does not require rotating the secret itself. This is the same envelope model use by age and cloud KMS, chosen because it is well-understood and auditable. **Protected paths (P7)** encrypt designated file *content* rather than named -env-var secrets. They use **convergent encryption** — the DEK and nonce derive -from `BLAKE3(plaintext)` — so identical protected content dedups and an -unauthorized clone gets ciphertext it cannot read. The tradeoff (equality is -confirmable) is deliberate and documented; the `sc protect` CLI nudges genuine -low-entropy secrets toward `sc secret` instead (P28). Per-prefix recipient rules -are last-writer-wins epoch registers so a revoke is durable across merges (P16), -and `sc grant`/`revoke`/`rewrap` manage the recipient set without re-encrypting -convergent content. See ADR-0014/0026. +env-var secrets. As of **P33 (ADR-0043)** all new protected content is sealed +under a **fresh random DEK + random nonce** (`RANDOMIZED` perms bit alongside +`PROTECTED`), so two seals of the same plaintext yield different ciphertext ids +— closing the convergent equality-confirmation oracle for everything sealed +from P33 on. The store is **dual-read, randomized-write**: pre-P33 *convergent* +ciphertext (DEK/nonce derived from `BLAKE3(plaintext)`) still decrypts through +the unchanged `decrypt_path`, so no snapshot-tag bump and no forced migration — +a pre-P33 store reads byte-for-byte identically. Commit's per-path rule is +**format-dispatched** on the prior tip entry (convergent → re-encrypt-and- +compare, carry-if-unchanged; randomized → keyed-PRF stat cache under a never- +committed `.sc/local-key`; new → seal randomized), so unchanged convergent +content stays convergent until an edit or an explicit `sc rewrap` upgrades it. +An unauthorized clone still gets ciphertext it cannot read; the `sc protect` CLI +still nudges genuine low-entropy secrets toward `sc secret` (P28). Per-prefix +recipient rules are last-writer-wins epoch registers so a revoke is durable +across merges (P16). `sc grant`/`revoke` manage the recipient set without +touching ciphertext; **`sc rewrap` now eagerly re-seals still-convergent blobs +randomized** as it re-wraps the tip — so a rewrap that upgrades content is no +longer tree-identical (a convergent→randomized reseal changes the blob id), +though a second rewrap over an all-randomized tip converges back to policy-only. +See ADR-0014/0026/0043. **Lifecycle (P11/P17):** `sc secret rotate` re-seals a secret under a fresh DEK (the real cryptographic cutover, distinct from metadata-only revoke); `sc escrow` @@ -476,12 +492,15 @@ is recovered via `--identity` and re-sealed. Recipients default to the secret's current set (reverse `recipient_id` lookup against `.sc/recipients.toml`), overridable with `--to`. -Rotation is **secrets-only**: protected paths use convergent encryption -(`DEK = HKDF(BLAKE3(plaintext))`), so a recipient who checked the file out can -re-derive the key — "rotating" a path's DEK is either dedup-breaking or -security-meaningless. Path lifecycle stays on recipient re-wrap -(`grant`/`revoke`). `sc secret revoke` remains metadata-only and now hints to -run `rotate` for the actual cryptographic cutover. +Rotation is **secrets-only**: pre-P33 protected paths used convergent +encryption (`DEK = HKDF(BLAKE3(plaintext))`), so a recipient who checked the +file out could re-derive the key — "rotating" a convergent path's DEK is either +dedup-breaking or security-meaningless. Path lifecycle stays on recipient +re-wrap (`grant`/`revoke`). `sc secret revoke` remains metadata-only and now +hints to run `rotate` for the actual cryptographic cutover. (**P33/ADR-0043:** +the security-meaningless objection dissolves for **randomized** content — a +random DEK is independent of the plaintext — so a genuine rotate-for-paths +cutover is now coherent and recorded as an unlocked follow-on, not yet built.) `sc escrow set ` / `sc escrow show` configure a single break-glass recipient key (`[escrow]` in `.sc/recipients.toml`) that is @@ -566,7 +585,10 @@ three-way cases resolve on ciphertext ids alone: unchanged, one-side-changed, and clean-delete protected paths merge with no identity at all, carrying ciphertext blobs plus a union of wrapped DEKs. Only a **content-divergent** protected path — both sides edited the plaintext — needs an authorized -`--identity`: the two plaintexts are decrypted, diff3-merged, and the result +`--identity` (**→ P33/ADR-0043 adjustment:** under randomized sealing, both +sides editing to even *identical* plaintext no longer id-match, so that case +now also conflicts and needs an identity — accepted cost 4a): the two +plaintexts are decrypted, diff3-merged, and the result is re-encrypted through the same `encrypt_protected`/`reuse_prior_wraps` helpers `commit` already used (extracted, single-sourced), so plaintext is never written to the CAS — conflict markers and sidecars for protected paths diff --git a/CLAUDE.md b/CLAUDE.md index b75ec144..a435bc77 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -169,11 +169,25 @@ bash demo/run_lifecycle_demo.sh # rotation + escr cargo run --bin sc -- rewrap [--identity ] [--dry-run] # one-commit bulk reseal of all # secrets + protected wrap lists to current # recipient/escrow sets (skip-and-report; - # exits 1 when entries were skipped) + # exits 1 when entries were skipped); also + # eagerly re-seals any still-CONVERGENT (pre-P33) + # protected blob RANDOMIZED (P33) — so a rewrap + # that upgrades content is no longer tree-identical, + # while a second rewrap over an all-randomized tip + # converges back to policy-only cargo run --bin sc -- escrow add # append a break-glass key (list) cargo run --bin sc -- escrow remove cargo run --bin sc -- escrow show # lists all escrow keys bash demo/run_rewrap_demo.sh # bulk rewrap + escrow-list proof +bash demo/run_randomized_demo.sh # randomized protected-encryption proof + # (P33): same plaintext → different ciphertext ids + # (oracle closed), quiet history for unchanged + # content, cost-4a identical-edit conflict, and + # sc rewrap policy-only on an all-randomized tip — + # run twice, zero residue (the convergent→randomized + # rewrap UPGRADE and pre-P33 dual-read are unit- + # test-pinned, since the current binary can no + # longer write a convergent blob) cargo run --bin sc -- work --agents 3 -- # fork agent workspaces, run in each, # harvest changed ones to work- branches # (--with-secrets --identity injects @@ -435,7 +449,11 @@ default to the secret's current set (resolved by reverse `recipient_id` lookup against `.sc/recipients.toml`) or `--to`. This is secrets-only — per-file protected paths use convergent encryption, where DEK "rotation" is either dedup-breaking or security-meaningless, so path lifecycle stays on -recipient re-wrap (`grant`/`revoke`). `sc secret revoke` remains +recipient re-wrap (`grant`/`revoke`). **(→ P33/ADR-0043: this objection +dissolves for randomized content — a random DEK is independent of the +plaintext, so re-sealing a protected path under a fresh DEK becomes a genuine +cutover; a rotate-for-paths surface is now an unlocked follow-on, recorded not +built.)** `sc secret revoke` remains metadata-only (unchanged behavior), now printing a hint to run `rotate` for an actual cryptographic cutover. `sc escrow {add,remove,show}` / `sc escrow set` (single break-glass key → managed list in P17; `set` kept as replace-with-one sugar) in @@ -588,7 +606,11 @@ presence and its wrap list **replaced** with exactly the governing rule's `granted_keys() + escrow`, so a tombstoned recipient's wraps re-attached by a pre-revoke merge do not survive the sweep. Convergent DEKs keep ciphertext ids unchanged, so the commit is policy-only (root tree -byte-identical). **Skip-and-report:** entries the identity cannot open are +byte-identical). **(→ P33/ADR-0043: no longer unconditionally true — rewrap +now also eagerly re-seals any still-convergent blob RANDOMIZED, changing its +id and the root tree, so a rewrap that upgrades content is content-changing; +only a rewrap over an all-randomized tip stays policy-only.)** +**Skip-and-report:** entries the identity cannot open are skipped and named (not silently dropped); the command commits whatever it could and exits non-zero when anything was skipped, printing a hint to re-run with an identity that can open the rest. **Honesty caveat (same @@ -1187,7 +1209,10 @@ isolation," and a compile-time pin locks in that `scl_crypto::open`'s `Zeroizing>` plaintext rides unchanged through to the unavoidable `OsString` child-env hand-off. **Two accepted boundaries, unchanged by design:** convergent encryption stays equality-confirmable (ADR-0014 — -randomized protected mode is deferred, not a bug); the secret's child-env +**closed for new content in P33/ADR-0043**, which seals all new protected +content randomized; the caveat holds forever for pre-P33 convergent ciphertext +in history — `sc rewrap` only stops it propagating into future snapshots, +rotation ≠ erasure per ADR-0019); the secret's child-env copy is fundamental and un-zeroizable (fd/stdin injection is deferred, not a bug) — the parent's own decrypted buffer is what gets zeroized. Every prior demo stays green plus new pinned regression tests across all four @@ -1374,6 +1399,105 @@ first connection to any host is unverified by construction (the TOFU trade, same as SSH's first connect); a loopback bind with no tokens configured stays unauthenticated by design, unchanged from P29. See ADR-0042. +**Phase 33 is built.** Randomized protected-path encryption closes ADR-0014's +convergent equality-confirmation oracle (the audit's remaining protected-path +High, ticket #40) for all **new** content, dual-read for the old. All new +protected content seals under a **fresh random DEK + random nonce** +(`encrypt_path_randomized`, `crates/crypto`), so two seals of the same +plaintext yield different ciphertext ids — the dictionary/equality oracle is +closed from this phase on. **The format tag is a perms-byte flag, not a +blob-layout change:** a new `RANDOMIZED` bit (`0b0000_0010`, always set with +`PROTECTED`) rides the tree entry; the blob is still `nonce(24) ‖ ciphertext` +under the same `PATH_AAD`, decrypted by the unchanged `decrypt_path`, so +dual-read needs ZERO read-path code. The tag lives in the tree entry (not the +blob) so format identification never fetches or decrypts bytes, and so a +format discriminator never perturbs a blob id. **No snapshot-tag bump +(`TAG_SNAPSHOT` stays 4), no `PROTOCOL_VERSION` change (stays 3)** — a pre-P33 +store reads byte-for-byte identically, and carry-by-id (the P15/P24 discipline, +preserving the source entry's own perms) rides the bit through merge, replay, +graft, and sparse carry with no new plumbing. **The commit rule is +format-dispatched, not cache-only** (spec §3 table): a prior-tip **convergent** +entry uses today's re-encrypt-and-compare (unchanged → carry as convergent +forever with no cache; edited → seal randomized), a prior-tip **randomized** +entry consults the stat cache, a **new** path seals randomized — a cache-only +rule would have mass-migrated every unchanged protected file on the first +commit after upgrade; format dispatch means unchanged convergent content stays +convergent until an edit or `sc rewrap` touches it. **Unchanged detection is a +per-checkout stat cache** (`crates/repo/src/cache.rs`) keyed by +`.sc/local-key` (32 random bytes, **0600 from the first byte**, lazily minted, +**never committed, never transferred**): entries are `path → (mtime, size, +keyed_tag, blob_id)` with `keyed_tag = BLAKE3-keyed(local_key, plaintext)`. +Commit still needs only **public keys** — decrypt-and-compare is used nowhere. +The cache **does not reintroduce the oracle** because the tag is a PRF output +under a key that never leaves `.sc/`, so the cache file alone leaks nothing; +its worst failure mode is a spurious re-seal (fresh id for unchanged +plaintext), never incorrectness — a corrupt line is treated as absent with a +warning, a missing key is minted, a key that can't be created is a hard error. +**DEVIATION from #30 decision-3's letter (and the spec §3 table): the stat hit +is NEVER trusted alone — the keyed BLAKE3 tag is authoritative in every cache +lookup.** The recorded `(mtime, size)` are advisory; the tag is recomputed and +compared every time. This closes the git racy-stat class (a same-size edit +within mtime granularity, at the recorded mtime, would otherwise be silently +dropped from a commit — pinned by `racy_stat_same_size_same_mtime_still_misses` +and `tag_is_authoritative_for_hits_and_misses`). Rationale: the plaintext is +already in memory at every call site, keyed hashing is ~free against it, and +"never incorrectness" outranks the stat shortcut. **`sc rewrap` performs the +eager upgrade** (no new command, no flag): while re-wrapping each blob's DEK by +wrap presence, it additionally decrypts any still-convergent blob and re-seals +it randomized to `granted_keys() + escrow`, recording the plaintext it already +holds into the main-tree cache (so the next `sc status`/`commit` stays quiet — +rewrap never rematerializes the working tree). Output gains a `re-sealed N +convergent blob(s)` line; a rewrap that upgrades content is **no longer +tree-identical** (accepted cost 4b), while a second rewrap over an +all-randomized tip converges back to policy-only. **A review Critical, fixed:** +two paths sharing ONE convergent blob (pre-P33 dedup of identical plaintext) — +old-id wrap removal is **DEFERRED past the path loop** (removed only when no +kept tree entry still references it); removing it in-loop stripped the shared +entry out from under the second path, leaving it wrap-orphaned at the tip with +no re-run repair. Each shared path mints its own randomized blob, which also +closes the intra-tip oracle between those two paths. **Accepted costs 4a–c all +landed as predicted:** (4a) independent identical edits on two branches now +conflict — both sides editing to even trivially-identical plaintext no longer +id-matches, so the merge needs `--identity` (or P23 `sc conflicts`/`sc +resolve`) even to merge identical content; (4b) rewrap is tree-changing when it +upgrades; (4c) identical plaintext at two paths no longer dedups. **P15/P17 +semantic adjustments:** merge/replay's content-divergent re-encryption switches +to the randomized primitive (union/tombstone semantics unchanged; mixed +convergent+randomized trees need no special casing — every path is carried by +id or re-sealed randomized), and `sc rewrap` gains the upgrade above. +**`sc protect --to ` no longer wraps UNCHANGED content +at the next commit** (review-confirmed real behavior change): unchanged files +carry their prior wrap list verbatim, so a bare re-protect does not grant the +new recipient access to un-re-sealed files. This fails **safe** — it +under-grants, never over-grants — and `sc grant` / `sc rewrap` are the covering +flows (operator guidance: use `sc grant`, not a second `sc protect`, to add a +recipient to already-committed content). **The cache is per-checkout:** the +main tree uses `.sc/protected-cache`; a ws workspace uses `.sc/ws/cache-` +**beside** the checkout dir, never inside it (a file inside `.sc/ws//` would +be harvested as an untracked working file), removed with the workspace at +teardown; `sc work`'s one-shot agents use ephemeral in-memory caches. Rationale +— without per-workspace caches, untouched protected files in forked workspaces +would have no cache entry, be re-sealed randomized at harvest with a fresh id, +and identical-edit-**conflict** (cost-4a fired spuriously) into a `work-` +fallback instead of a clean auto-merge; per-workspace caches reproduce the base +tree id and preserve P20's clean-auto-merge and crash-recovery `UpToDate` +idempotency. All production cache saves are **best-effort and ordered after ref +moves** — the ref move stays the atomic commit point; a save failure warns, +never errors. **Unlocked follow-on: rotate-for-paths** — ADR-0019 called +per-path DEK rotation security-meaningless under convergent encryption; that +objection **dissolves for randomized content** (a random DEK is independent of +the plaintext), so a `sc protect … rotate` genuine cutover is now coherent — +**recorded, not built.** **Proof splits honestly:** dual-read of genuine +pre-P33 stores is pinned by **library-built convergent-tree fixtures** in the +`rewrap`/`merge`/`replay` unit tests (the current binary can no longer *write* +a convergent blob), while `demo/run_randomized_demo.sh` proves oracle-closure, +quiet history, cost-4a conflicts, and rewrap policy-only on an all-randomized +tip against the current binary (the convergent→randomized rewrap upgrade path +is unit-pinned, not demo-pinned, for the same reason) — run twice, zero +residue. Zero new dependencies (`blake3`/`hex` +were already workspace deps; `scl-repo` now links `blake3` directly for the +cache tag). See ADR-0043. + Remaining follow-ons: operation objects in the CAS, oplog entries for remote-tracking refs, extending the sparse gate to `materialize_conflict_state`'s `to_encrypt`/sidecar write loops (see the @@ -1381,10 +1505,13 @@ P24 boundary note above), the three P27 items named above (transparent lazy-fetch as a deferred alternative to explicit `sc backfill`, per-case gap-tolerant merge/rebase/`ws harvest`/`sc work` instead of blanket refusal, and blob-size/object-count clone filters alongside the -prefix-only filter shipped here), the three P28 items named above -(randomized protected mode for equality-hiding, fd/stdin secret injection -as an alternative to env vars, and a `--max-object-size` operator config -knob), and the P28 final-review follow-ons (a one-line +prefix-only filter shipped here), the two remaining P28 items named above +(fd/stdin secret injection as an alternative to env vars, and a +`--max-object-size` operator config knob — randomized protected mode is now +SHIPPED in P33/ADR-0043), the P33 item (**rotate-for-paths** — ADR-0019's +"path DEK rotation is security-meaningless" objection dissolves for randomized +content, so a `sc protect … rotate`-style genuine per-path cutover is now +coherent; recorded, not built), and the P28 final-review follow-ons (a one-line `validate_branch_name` call in `refs::write_head`/`refs::delete_branch` for ref-validation class completeness — not exploitable today, since HEAD's path is fixed and `delete_branch` only takes internally-generated diff --git a/Cargo.lock b/Cargo.lock index 5f2424dc..d56ad998 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2315,6 +2315,7 @@ dependencies = [ name = "scl-repo" version = "0.1.0" dependencies = [ + "blake3", "hex", "libc", "regex", diff --git a/ROADMAP.md b/ROADMAP.md index 14c770bd..b7a3745f 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -63,8 +63,10 @@ across every phase. (deduped, forward-only) whenever a secret is sealed or a path is protected. `revoke` stays metadata-only, now with a hint pointing at `rotate`. Rotation is secrets-only — convergent encryption makes protected-path DEK rotation - security-meaningless (ADR-0014) — and rotation is a future-reads cutover, - not erasure: old ciphertext remains reachable from history. (ADR-0019.) + security-meaningless (ADR-0014; → this objection dissolves for randomized + content in P33/ADR-0043, unlocking rotate-for-paths as a follow-on) — and + rotation is a future-reads cutover, not erasure: old ciphertext remains + reachable from history. (ADR-0019.) - **Phase 12 — Network transport over SSH.** A framed stdio wire protocol mirrors the 8 `Transport` verbs; `sc serve --stdio` dispatches onto the existing `LocalTransport` (CAS, pack verification reused verbatim); the @@ -394,7 +396,8 @@ across every phase. the P5 content scanner) prints one stderr warning steering low-entropy secret basenames (`.env`/`*.key`/`*credentials*`…) toward `sc secret`, citing ADR-0014 — warning-only, convergent encryption's - equality-confirmability stays accepted by design. Secret env-var + equality-confirmability stayed accepted by design (→ closed for new + content in P33/ADR-0043; the nudge itself stays right). Secret env-var boundary: the threat model is tightened to "authorized local process context, NOT strong isolation," and a compile-time pin locks in that `scl_crypto::open`'s `Zeroizing>` plaintext rides unchanged to @@ -506,6 +509,30 @@ across every phase. carrying a signed chunked blob byte-for-byte, the TOFU pin/mismatch/strict/pre-pin lifecycle, and the tightened plaintext gate — run twice, zero residue. (ADR-0042.) +- **Phase 33 — Randomized protected-path encryption.** Closes ADR-0014's + convergent equality-confirmation oracle (the audit's remaining protected-path + High, ticket #40) for all **new** content: every new protected seal uses a + **fresh random DEK + random nonce** (`encrypt_path_randomized`, + `crates/crypto`), so two seals of the same plaintext yield different + ciphertext ids. The store is **dual-read, randomized-write**: a `RANDOMIZED` + perms-byte flag (alongside `PROTECTED`) tags new entries, the blob layout and + `decrypt_path` are unchanged, and there is **no snapshot-tag bump and no + `PROTOCOL_VERSION` change** — a pre-P33 store reads byte-for-byte identically. + Commit's per-path rule is **format-dispatched** (convergent → carry-if- + unchanged, stays convergent; randomized → keyed-PRF stat cache; new → seal + randomized), so no forced mass-migration. Unchanged detection is a + per-checkout cache (`.sc/protected-cache`, ws `.sc/ws/cache-`, `sc work` + in-memory) keyed by a never-committed `.sc/local-key`; the keyed BLAKE3 tag is + authoritative in every lookup (stat is advisory — closes the git racy-stat + class), and the cache leaks nothing (PRF under a local-only key). `sc rewrap` + eagerly re-seals still-convergent blobs randomized (so an upgrading rewrap is + no longer tree-identical). Accepted costs: identical independent edits now + conflict (4a), rewrap is content-changing when it upgrades (4b), identical + plaintext at two paths no longer dedups (4c). Zero new dependencies. Proven by + `demo/run_randomized_demo.sh` (oracle closed, quiet history, cost-4a conflict, + rewrap policy-only on an all-randomized tip — run twice); dual-read of pre-P33 + convergent stores and the convergent→randomized rewrap upgrade are unit-pinned + (the current binary can no longer write a convergent blob). (ADR-0043.) ## Active @@ -709,12 +736,39 @@ scale-&-reach horizon): prefix-only, matching the sparse path-boundary machinery already built; git's `--filter=blob:limit` size-based filtering is a separate axis, deferred. -- **Randomized protected mode (P28).** `sc protect` stays convergent - (deterministic DEK/nonce per plaintext), so ciphertext equality still - confirms plaintext equality without decryption — accepted by design in - ADR-0014, surfaced rather than closed by P28's filename nudge. A - non-convergent DEK/nonce mode for equality-hiding on high-sensitivity - paths is the follow-on. +- **Randomized protected mode — SHIPPED in P33 (ADR-0043).** `sc protect` + used to stay convergent (deterministic DEK/nonce per plaintext), so + ciphertext equality confirmed plaintext equality without decryption — + accepted by design in ADR-0014, surfaced by P28's filename nudge. P33 closes + it: all new protected content seals under a fresh random DEK + nonce + (dual-read for pre-P33 convergent history; `sc rewrap` upgrades a tip on + demand). Moved from deferred to Done above. +- **Rotate-for-paths (P33 follow-on).** ADR-0019 declared per-path DEK rotation + "security-meaningless" under convergent encryption. That objection dissolves + for randomized content (a random DEK is independent of the plaintext), so a + `sc protect … rotate`-style genuine per-path cryptographic cutover — the + protected-path analogue of `sc secret rotate` — is now coherent. Recorded in + ADR-0043, not built. +- **`sc rewrap` on partial clones (P33 final-review follow-on).** `sc rewrap` + loads each protected plaintext whole into RAM and has no + `PartialCloneUnsupported` guard, unlike merge/rebase/ws-fork/ws-harvest/`sc + work` (P27); a paths-only/streaming tree rebuild (or the same refusal-first + posture) is the follow-on. +- **Protected-cache hygiene (P33 final-review follow-on).** `.sc/protected-cache` + and `.sc/ws/cache-` entries for a deleted or renamed path are never + pruned — unbounded but benign growth (a stale entry just goes unmatched). + Periodic or commit-time pruning of dead entries is the follow-on. +- **Theirs-side resolve carry (P33 final-review follow-on).** `sc resolve + --theirs` on a PROTECTED path records into the main-tree cache today, but + completion's format-dispatched carry only consults the cache against the + prior TIP (ours') entry, so a theirs-side recording never matches and + completion re-seals randomized anyway (spurious-but-safe). Extending the + carry-candidate map to also check `merge_head`/decided-root entries would + let a theirs-side resolution skip the re-seal too. +- **Non-unix local-key fallback (P33 final-review follow-on).** `.sc/local-key` + is minted 0600-from-first-byte via a unix-only `OpenOptions::mode` call; + there is no non-unix fallback today, so the key file is left world-readable + on a non-unix target. A non-unix permission-hardening path is deferred. - **fd/stdin secret injection (P28).** `sc run` hands decrypted secrets to the child via its environment — observable by same-user processes, crash dumps, and shell wrappers, per the tightened threat-model wording. diff --git a/crates/cli/src/main.rs b/crates/cli/src/main.rs index 5d612b0e..26b9253b 100644 --- a/crates/cli/src/main.rs +++ b/crates/cli/src/main.rs @@ -3825,6 +3825,17 @@ fn run_rewrap(identity: Option, dry_run: bool) -> Result<()> { report.secrets_rewrapped.len(), report.blobs_rewrapped ); + if report.blobs_resealed > 0 { + let reseal_verb = if dry_run { + "would re-seal" + } else { + "re-sealed" + }; + println!( + "{reseal_verb} {} convergent blob(s) → randomized", + report.blobs_resealed + ); + } if let Some(id) = &report.commit { println!("commit: {}", id.short()); } diff --git a/crates/core/src/lib.rs b/crates/core/src/lib.rs index 806f7dfc..a8da2b33 100644 --- a/crates/core/src/lib.rs +++ b/crates/core/src/lib.rs @@ -26,6 +26,6 @@ pub use error::{Error, Result}; pub use id::ObjectId; pub use object::{ EntryKind, FileMode, Object, ProtectPrefix, Protection, RecipientEntry, RecipientState, Secret, - SignatureObj, Snapshot, Transcript, Tree, TreeEntry, WrappedKey, PROTECTED, + SignatureObj, Snapshot, Transcript, Tree, TreeEntry, WrappedKey, PROTECTED, RANDOMIZED, }; pub use store::{Backend, SpillPolicy, Store, StoreConfig, StoreStats}; diff --git a/crates/core/src/object.rs b/crates/core/src/object.rs index 7c8581e3..791daa9b 100644 --- a/crates/core/src/object.rs +++ b/crates/core/src/object.rs @@ -22,6 +22,12 @@ const TAG_TRANSCRIPT: u8 = 6; /// encrypted file), not plaintext. Set on protected-path entries (P7). pub const PROTECTED: u8 = 0b0000_0001; +/// Perms flag: this PROTECTED entry was sealed with a fresh random DEK+nonce +/// (P33) rather than convergently. Always set together with `PROTECTED`. +/// Format identification for dual-read lives here, in the tree entry, so no +/// caller ever needs to fetch blob bytes to know the seal format. +pub const RANDOMIZED: u8 = 0b0000_0010; + /// A recipient's standing on a protected prefix: active or tombstoned. #[derive(Clone, Copy, PartialEq, Eq, Debug)] pub enum RecipientState { @@ -1021,4 +1027,13 @@ mod tests { assert_eq!(rule.granted_keys(), vec![[1; 32]]); assert_eq!(rule.next_epoch(), 3); } + + #[test] + fn randomized_bit_is_distinct_from_protected() { + assert_eq!(PROTECTED & RANDOMIZED, 0, "flags must not overlap"); + assert_ne!(RANDOMIZED, 0); + // A randomized entry is always also protected. + let perms = PROTECTED | RANDOMIZED; + assert!(perms & PROTECTED != 0 && perms & RANDOMIZED != 0); + } } diff --git a/crates/crypto/src/envelope.rs b/crates/crypto/src/envelope.rs index 12e47894..8ace9d8a 100644 --- a/crates/crypto/src/envelope.rs +++ b/crates/crypto/src/envelope.rs @@ -136,8 +136,10 @@ const PATH_AAD: &[u8] = b"scl-path-v1"; /// Convergent file encryption: the data key and nonce are derived from the /// plaintext, so identical plaintext yields identical `nonce‖ciphertext` bytes -/// (stable content-addressed id, perfect dedup). Returns the blob bytes and the -/// DEK (to be wrapped per recipient and stored in the snapshot policy). +/// (stable content-addressed id, perfect dedup). Legacy convergent format — kept +/// for dual-read comparison only; new seals use `encrypt_path_randomized` (P33). +/// Returns the blob bytes and the DEK (to be wrapped per recipient and stored +/// in the snapshot policy). pub fn encrypt_path(plaintext: &[u8]) -> (Vec, Zeroizing<[u8; DEK_LEN]>) { let hash = blake3::hash(plaintext); let mut ikm = Zeroizing::new([0u8; 32]); @@ -167,6 +169,42 @@ pub fn encrypt_path(plaintext: &[u8]) -> (Vec, Zeroizing<[u8; DEK_LEN]>) { (blob, dek) } +/// Randomized file encryption (P33): fresh random DEK + random nonce per +/// seal, so identical plaintext yields different ciphertext every time — +/// the ADR-0014 equality oracle is closed for content sealed this way. +/// Blob layout and AAD are identical to `encrypt_path` (`nonce ‖ ct`, +/// `PATH_AAD`), so `decrypt_path` opens both formats unchanged (dual-read). +pub fn encrypt_path_randomized(plaintext: &[u8]) -> (Vec, Zeroizing<[u8; DEK_LEN]>) { + encrypt_path_randomized_with_rng(plaintext, &mut OsRng) +} + +/// `encrypt_path_randomized` with a caller-supplied RNG (deterministic in tests). +pub fn encrypt_path_randomized_with_rng( + plaintext: &[u8], + rng: &mut R, +) -> (Vec, Zeroizing<[u8; DEK_LEN]>) { + let mut dek = Zeroizing::new([0u8; DEK_LEN]); + rng.fill_bytes(dek.as_mut_slice()); + let mut nonce = [0u8; NONCE_LEN]; + rng.fill_bytes(&mut nonce); + + let cipher = XChaCha20Poly1305::new_from_slice(dek.as_slice()).expect("32-byte DEK"); + let ciphertext = cipher + .encrypt( + XNonce::from_slice(&nonce), + Payload { + msg: plaintext, + aad: PATH_AAD, + }, + ) + .expect("aead encrypt is infallible for valid inputs"); + + let mut blob = Vec::with_capacity(NONCE_LEN + ciphertext.len()); + blob.extend_from_slice(&nonce); + blob.extend_from_slice(&ciphertext); + (blob, dek) +} + /// Decrypt a `nonce‖ciphertext` blob with its DEK (AEAD-verified). pub fn decrypt_path(blob: &[u8], dek: &[u8; DEK_LEN]) -> Result>> { if blob.len() < NONCE_LEN { @@ -403,4 +441,26 @@ mod tests { let (other_sk, _) = generate_keypair_with_rng(&mut ChaCha20Rng::seed_from_u64(2)); assert!(unwrap_dek_with(&wk, &other_sk).is_err()); } + + #[test] + fn randomized_seal_roundtrips_and_closes_the_oracle() { + let pt = b"the database password is hunter2"; + let (blob1, dek1) = encrypt_path_randomized(pt); + let (blob2, dek2) = encrypt_path_randomized(pt); + // THE phase invariant: same plaintext, different ciphertext bytes. + assert_ne!(blob1, blob2, "equality oracle must be closed"); + assert_ne!(dek1.as_slice(), dek2.as_slice()); + // Dual-read: the unchanged decrypt_path opens a randomized blob. + assert_eq!(&decrypt_path(&blob1, &dek1).unwrap()[..], pt); + assert_eq!(&decrypt_path(&blob2, &dek2).unwrap()[..], pt); + } + + #[test] + fn randomized_seal_is_deterministic_under_a_seeded_rng() { + let mut r1 = rng(7); + let mut r2 = rng(7); + let (b1, _) = encrypt_path_randomized_with_rng(b"x", &mut r1); + let (b2, _) = encrypt_path_randomized_with_rng(b"x", &mut r2); + assert_eq!(b1, b2); + } } diff --git a/crates/crypto/src/lib.rs b/crates/crypto/src/lib.rs index ed03e286..d3e15d60 100644 --- a/crates/crypto/src/lib.rs +++ b/crates/crypto/src/lib.rs @@ -16,7 +16,8 @@ pub mod provider; pub mod signing; pub use envelope::{ - decrypt_path, encrypt_path, open, revoke, rewrap_for, seal, unwrap_dek_with, wrap_dek_for, + decrypt_path, encrypt_path, encrypt_path_randomized, encrypt_path_randomized_with_rng, open, + revoke, rewrap_for, seal, unwrap_dek_with, wrap_dek_for, }; pub use error::{Error, Result}; pub use key::{generate_keypair, PublicKey, RecipientId, SecretKey}; diff --git a/crates/repo/Cargo.toml b/crates/repo/Cargo.toml index 471aa181..d30fa3b3 100644 --- a/crates/repo/Cargo.toml +++ b/crates/repo/Cargo.toml @@ -16,6 +16,7 @@ serde = { version = "1", features = ["derive"] } libc = "0.2.186" scl-tlsio = { version = "0.1.0", path = "../tlsio" } hex = "0.4.3" +blake3 = "1.8.5" [lints] workspace = true diff --git a/crates/repo/src/cache.rs b/crates/repo/src/cache.rs new file mode 100644 index 00000000..c65a7cca --- /dev/null +++ b/crates/repo/src/cache.rs @@ -0,0 +1,402 @@ +//! P33: local unchanged-detection cache for randomized protected paths. +//! +//! Randomized seals (ADR-0043) make re-encrypt-and-compare useless for +//! change detection, so commit/status/diff consult this `.sc`-local, +//! never-committed, never-transferred map instead: +//! `path -> (mtime_ns, size, keyed_tag, ciphertext blob id)`. +//! The tag is `blake3::keyed_hash(local_key, plaintext)` under a random +//! per-repo key (`.sc/local-key`, 0600), so the cache file alone leaks +//! nothing — it does not reintroduce the equality oracle this phase closes. +//! A lost/stale/corrupt cache degrades to spurious re-seals, never +//! incorrectness and never a hard error. +//! +//! **The keyed tag is the sole authority for a hit.** `mtime_ns`/`size` are +//! recorded for documentation (when the entry was observed) and as a +//! possible future input to a paths-only walk that doesn't already hold the +//! plaintext in RAM, but they are NEVER sufficient on their own: a bare +//! stat match is the classic git "racy stat" problem (an edit that lands +//! within the filesystem's mtime granularity and happens to keep the same +//! size is indistinguishable from no edit at all by stat alone), and every +//! `unchanged()` caller already has the plaintext in memory (`read_worktree` +//! loads all bytes before commit/diff), so there is no reason to trust a +//! cheaper, unsound signal — keyed blake3 over in-memory bytes runs at +//! GB/s. + +use std::collections::BTreeMap; +use std::path::{Path, PathBuf}; +use std::str::FromStr; + +use scl_core::ObjectId; + +use crate::error::{Error, Result}; +use crate::layout::Layout; + +/// Load (minting if absent) the per-repo random cache key. The key file is +/// the only hard-error surface here: if it can't be created/read, the cache +/// cannot be safely keyed, so surface it rather than running unkeyed. +/// +/// Mirrors the serve-TLS key discipline (`crates/tlsio/src/identity.rs`): +/// the file is created 0600 from its first byte via `create_new`, so there +/// is never a window where the key sits at default permissions. A +/// concurrent minter losing the `create_new` race falls through to reading +/// the winner's file rather than erroring or clobbering it. +pub(crate) fn local_key(layout: &Layout) -> Result<[u8; 32]> { + let path = layout.local_key_path(); + let hex_str = match std::fs::read_to_string(&path) { + Ok(s) => s.trim().to_string(), + Err(e) if e.kind() == std::io::ErrorKind::NotFound => { + let s = scl_crypto::random_hex(32); // 32 random bytes -> 64 hex chars + match write_key_0600(&path, s.as_bytes()) { + Ok(()) => s, + Err(e) if e.kind() == std::io::ErrorKind::AlreadyExists => { + // Lost the create_new race to a concurrent minter: read + // their file instead of clobbering it. + std::fs::read_to_string(&path)?.trim().to_string() + } + Err(e) => return Err(e.into()), + } + } + Err(e) => return Err(e.into()), + }; + let bytes = hex::decode(&hex_str) + .map_err(|_| Error::InvalidArgument("malformed .sc/local-key".into()))?; + bytes + .try_into() + .map_err(|_| Error::InvalidArgument("malformed .sc/local-key".into())) +} + +#[cfg(unix)] +fn write_key_0600(path: &Path, bytes: &[u8]) -> std::io::Result<()> { + use std::io::Write; + use std::os::unix::fs::OpenOptionsExt; + let mut f = std::fs::OpenOptions::new() + .write(true) + .create_new(true) + .mode(0o600) + .open(path)?; + f.write_all(bytes) +} + +#[cfg(not(unix))] +fn write_key_0600(path: &Path, bytes: &[u8]) -> std::io::Result<()> { + use std::io::Write; + let mut f = std::fs::OpenOptions::new() + .write(true) + .create_new(true) + .open(path)?; + f.write_all(bytes) +} + +#[derive(Clone, Copy, Debug, PartialEq)] +struct CacheEntry { + mtime_ns: u128, + size: u64, + tag: [u8; 32], + blob_id: ObjectId, +} + +/// Per-checkout unchanged-detection cache (main tree, one ws workspace, or +/// an ephemeral in-memory one for `sc work` temp checkouts). +/// +/// The cache owns its `root`: the absolute base directory of the checkout it +/// serves, against which relative tree paths are resolved to record the +/// advisory mtime+size fields alongside each entry. This is the P33 Task 7 +/// design choice (over threading an explicit worktree-root through +/// `snapshot_files`): the cache is inherently tied to one checkout, so it +/// should carry that checkout's root rather than have every +/// `unchanged`/`record` caller pass an absolute path. **Invariant:** the +/// `root` passed at `open` must equal `layout.root` of the checkout that +/// `materialize`/`diff_worktree`/`snapshot_files` operate on — the main tree +/// opens with `layout.root`, a `sc ws`/`sc work` workspace with its own +/// checkout dir. The keyed-tag comparison that decides every `unchanged()` +/// hit does not depend on `root` at all (it hashes the caller's in-memory +/// plaintext), so a mismatched root only ever loses the advisory stat +/// bookkeeping — never incorrectness. +pub(crate) struct ProtectedCache { + key: [u8; 32], + /// Absolute base dir of the served checkout; `root.join(rel)` is the file + /// a relative tree path stats to. + root: PathBuf, + /// `None` => ephemeral: `save()` is a no-op. + path: Option, + entries: BTreeMap, +} + +impl ProtectedCache { + pub(crate) fn open(key: [u8; 32], root: PathBuf, path: Option) -> ProtectedCache { + let mut entries = BTreeMap::new(); + if let Some(p) = &path { + if let Ok(text) = std::fs::read_to_string(p) { + for line in text.lines() { + // Format: ` ` + // (path last: it may contain spaces). + let mut it = line.splitn(5, ' '); + let parsed = (|| { + let mtime_ns: u128 = it.next()?.parse().ok()?; + let size: u64 = it.next()?.parse().ok()?; + let tag: [u8; 32] = hex::decode(it.next()?).ok()?.try_into().ok()?; + let blob_id = ObjectId::from_str(it.next()?).ok()?; + Some(( + it.next()?.to_string(), + CacheEntry { + mtime_ns, + size, + tag, + blob_id, + }, + )) + })(); + match parsed { + Some((path, e)) => { + entries.insert(path, e); + } + None => { + eprintln!("warning: ignoring corrupt protected-cache line"); + entries.clear(); + break; // degrade to empty: spurious re-seals, never incorrectness + } + } + } + } + } + ProtectedCache { + key, + root, + path, + entries, + } + } + + fn tag(&self, plaintext: &[u8]) -> [u8; 32] { + *blake3::keyed_hash(&self.key, plaintext).as_bytes() + } + + fn stat(abs: &Path) -> Option<(u128, u64)> { + let md = std::fs::metadata(abs).ok()?; + let mtime = md.modified().ok()?; + let ns = mtime.duration_since(std::time::UNIX_EPOCH).ok()?.as_nanos(); + Some((ns, md.len())) + } + + /// The cached ciphertext blob id iff this exact plaintext is what last + /// sealed at `rel`, decided ENTIRELY by the keyed-tag comparison — never + /// by a bare mtime+size stat match. A stat-only hit is the classic git + /// "racy stat" problem: an edit that keeps the same file size and lands + /// within the filesystem's mtime granularity is indistinguishable from + /// no edit at all by stat alone, and misreading it as unchanged would + /// silently carry the OLD ciphertext into the commit, dropping the + /// user's edit — exactly the incorrectness this cache must never cause. + /// Every caller already holds `plaintext` in memory before this is + /// called, so the keyed tag is cheap enough to be the sole authority. + pub(crate) fn unchanged(&self, rel: &str, plaintext: &[u8]) -> Option { + let e = self.entries.get(rel)?; + (self.tag(plaintext) == e.tag).then_some(e.blob_id) + } + + /// Record that `plaintext` at `rel` seals to `blob_id`. The absolute file + /// is resolved against the cache's own `root`. Missing stat (file vanished + /// mid-operation) just skips the entry. + pub(crate) fn record(&mut self, rel: &str, plaintext: &[u8], blob_id: ObjectId) { + if let Some((mtime_ns, size)) = Self::stat(&self.root.join(rel)) { + let tag = self.tag(plaintext); + self.entries.insert( + rel.to_string(), + CacheEntry { + mtime_ns, + size, + tag, + blob_id, + }, + ); + } + } + + /// Durable atomic write via the shared `fsutil` helper (fsync temp + + /// rename + fsync parent dir, no residue on failure); no-op for an + /// ephemeral cache. + pub(crate) fn save(&self) -> Result<()> { + let Some(path) = &self.path else { + return Ok(()); + }; + let mut out = String::new(); + for (p, e) in &self.entries { + out.push_str(&format!( + "{} {} {} {} {}\n", + e.mtime_ns, + e.size, + hex::encode(e.tag), + e.blob_id.to_hex(), + p + )); + } + if let Some(parent) = path.parent() { + std::fs::create_dir_all(parent)?; + } + scl_core::fsutil::atomic_write_durable(path, out.as_bytes())?; + Ok(()) + } + + /// Persist best-effort: on failure, log to stderr and swallow rather than + /// propagate. Every ref-moving op that populates the cache saves through + /// this AFTER its ref has moved (P33 Task 7), so cache trouble can never + /// abort an operation that has already logically succeeded — a lost/stale + /// cache only ever degrades to a spurious reseal next time, never to + /// incorrectness. + pub(crate) fn save_best_effort(&self) { + if let Err(e) = self.save() { + eprintln!("warning: failed to persist protected-cache: {e}"); + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn tmp(tag: &str) -> Layout { + let root = std::env::temp_dir().join(format!("scl-cache-{tag}-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&root); + std::fs::create_dir_all(root.join(".sc")).unwrap(); + Layout::at(&root) + } + + fn cleanup(layout: &Layout) { + std::fs::remove_dir_all(&layout.root).unwrap(); + assert!(!layout.root.exists()); + } + + #[test] + fn local_key_is_minted_once_and_stable() { + let layout = tmp("key"); + let k1 = local_key(&layout).unwrap(); + let k2 = local_key(&layout).unwrap(); + assert_eq!(k1, k2, "second read must return the same key"); + assert_ne!(k1, [0u8; 32]); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + let mode = std::fs::metadata(layout.local_key_path()) + .unwrap() + .permissions() + .mode(); + assert_eq!(mode & 0o777, 0o600, "key file must be 0600"); + } + cleanup(&layout); + } + + #[test] + fn tag_is_authoritative_for_hits_and_misses() { + let layout = tmp("hits"); + let key = local_key(&layout).unwrap(); + let abs = layout.root.join("secret.txt"); + std::fs::write(&abs, b"v1").unwrap(); + let id = ObjectId::of(b"cipher-of-v1"); + + let mut c = ProtectedCache::open( + key, + layout.root.clone(), + Some(layout.protected_cache_path()), + ); + assert_eq!(c.unchanged("secret.txt", b"v1"), None, "empty cache misses"); + c.record("secret.txt", b"v1", id); + + // Untouched file, same plaintext: hit. + assert_eq!(c.unchanged("secret.txt", b"v1"), Some(id)); + + // Touch mtime without changing content: still a hit (tag matches). + std::fs::write(&abs, b"v1").unwrap(); + assert_eq!(c.unchanged("secret.txt", b"v1"), Some(id)); + + // Real change: miss. + std::fs::write(&abs, b"v2").unwrap(); + assert_eq!(c.unchanged("secret.txt", b"v2"), None); + cleanup(&layout); + } + + /// Regression for the racy-stat data-loss finding: same-length content + /// written back with the SAME recorded mtime (the classic git "racy + /// stat" condition — an edit that lands within the filesystem's mtime + /// granularity) must still miss, because a stat-only hit would silently + /// carry the old ciphertext and drop the user's edit from the commit. + #[test] + fn racy_stat_same_size_same_mtime_still_misses() { + let layout = tmp("racy"); + let key = local_key(&layout).unwrap(); + let abs = layout.root.join("secret.txt"); + std::fs::write(&abs, b"aaaa").unwrap(); + let id = ObjectId::of(b"cipher-of-aaaa"); + let original_mtime = std::fs::metadata(&abs).unwrap().modified().unwrap(); + + let mut c = ProtectedCache::open( + key, + layout.root.clone(), + Some(layout.protected_cache_path()), + ); + c.record("secret.txt", b"aaaa", id); + + // Overwrite with same-length content, then force the racy condition + // by resetting mtime back to what was recorded. + std::fs::write(&abs, b"bbbb").unwrap(); + std::fs::File::options() + .write(true) + .open(&abs) + .unwrap() + .set_modified(original_mtime) + .unwrap(); + + assert_eq!( + c.unchanged("secret.txt", b"bbbb"), + None, + "racy stat (same mtime, same size, different content) must not be misread as unchanged" + ); + cleanup(&layout); + } + + #[test] + fn save_load_roundtrip_and_corrupt_file_is_empty() { + let layout = tmp("persist"); + let key = local_key(&layout).unwrap(); + let abs = layout.root.join("a b/spaced name.txt"); // path with spaces + std::fs::create_dir_all(abs.parent().unwrap()).unwrap(); + std::fs::write(&abs, b"v").unwrap(); + let id = ObjectId::of(b"c"); + + let mut c = ProtectedCache::open( + key, + layout.root.clone(), + Some(layout.protected_cache_path()), + ); + c.record("a b/spaced name.txt", b"v", id); + c.save().unwrap(); + + let c2 = ProtectedCache::open( + key, + layout.root.clone(), + Some(layout.protected_cache_path()), + ); + assert_eq!(c2.unchanged("a b/spaced name.txt", b"v"), Some(id)); + + // Corrupt file: degrades to empty, never errors. + std::fs::write(layout.protected_cache_path(), b"garbage\nlines\n").unwrap(); + let c3 = ProtectedCache::open( + key, + layout.root.clone(), + Some(layout.protected_cache_path()), + ); + assert_eq!(c3.unchanged("a b/spaced name.txt", b"v"), None); + cleanup(&layout); + } + + #[test] + fn ephemeral_cache_never_touches_disk() { + let layout = tmp("ephemeral"); + let key = local_key(&layout).unwrap(); + let abs = layout.root.join("f"); + std::fs::write(&abs, b"v").unwrap(); + let mut c = ProtectedCache::open(key, layout.root.clone(), None); + c.record("f", b"v", ObjectId::of(b"c")); + c.save().unwrap(); + assert!(!layout.protected_cache_path().exists()); + cleanup(&layout); + } +} diff --git a/crates/repo/src/conflicts.rs b/crates/repo/src/conflicts.rs index 7d7b24f7..6a603e39 100644 --- a/crates/repo/src/conflicts.rs +++ b/crates/repo/src/conflicts.rs @@ -221,6 +221,10 @@ impl Repo { ResolveSide::Ours => versions.ours, ResolveSide::Theirs => versions.theirs, }; + // Computed once and reused below for the sidecar-cleanup check too — + // `conflict_kind` re-derives from the DAG and is safe to call before + // the write. + let kind = self.conflict_kind(path)?; let full = safe_join(&self.layout.root, path)?; match chosen { @@ -229,6 +233,42 @@ impl Repo { std::fs::create_dir_all(parent)?; } std::fs::write(&full, &bytes)?; + + // P33 Task 8: a resolved PROTECTED path's plaintext is now + // sitting on disk exactly as the chosen side's ciphertext + // decrypted it — record it in the main-tree cache regardless + // of which side was chosen. This only pays off for an + // `--ours` resolution: completion's format-dispatched carry + // (`snapshot_files`) consults the cache against the prior + // TIP entry's blob id, so an `--ours` recording matches and + // the completing `sc commit`/`sc rebase --continue` carries + // the existing ciphertext instead of re-sealing under a + // fresh random nonce. A `--theirs` recording keys the cache + // to a blob id completion never looks up against (the tip is + // still ours), so it goes unmatched and completion re-seals + // randomized anyway — spurious cache-population, but benign + // (never incorrect, just a wasted write). Plain paths never + // touch the cache — it exists only to skip re-sealing + // convergent-free randomized ciphertext. + if kind == ConflictKind::Protected { + let triple = self.op_triple()?; + let chosen_snapshot = match side { + ResolveSide::Ours => triple.ours, + ResolveSide::Theirs => triple.theirs, + }; + let store_arc = self.vfs.store(); + let blob_id = { + let mut store = store_arc.lock().unwrap(); + let snap = store.get_snapshot(&chosen_snapshot)?; + let entries = tree_file_entries_with_perms(&mut store, snap.root)?; + entries.get(path).map(|(id, _, _)| *id) + }; + if let Some(blob_id) = blob_id { + let mut cache = self.open_protected_cache()?; + cache.record(path, &bytes, blob_id); + cache.save_best_effort(); + } + } } Side::Absent => match std::fs::remove_file(&full) { Ok(()) => {} @@ -255,7 +295,7 @@ impl Repo { // remove it when it is NOT a tracked path — a genuine sidecar is // untracked scratch by construction, so a tracked `{path}.theirs` // must be a user's real file and is left alone. - if self.conflict_kind(path)? != ConflictKind::Text { + if kind != ConflictKind::Text { let theirs_sidecar_path = format!("{path}.theirs"); if !self.tracked_paths()?.contains(&theirs_sidecar_path) { let sidecar = safe_join(&self.layout.root, &theirs_sidecar_path)?; @@ -1005,6 +1045,56 @@ mod tests { std::fs::remove_dir_all(&root).ok(); } + #[test] + fn resolve_ours_records_protected_content_in_main_cache() { + let root = tmp_root("resolve-cache"); + let repo = Repo::init(&root).unwrap(); + let (alice_sk, alice_pk) = scl_crypto::generate_keypair(); + repo.protect("secret/", &[alice_pk], None).unwrap(); + std::fs::create_dir_all(root.join("secret")).unwrap(); + std::fs::write(root.join("secret/a.txt"), b"base\n").unwrap(); + repo.commit("me", "base").unwrap(); + repo.branch("feature").unwrap(); + + std::fs::write(root.join("secret/a.txt"), b"ours\n").unwrap(); + repo.commit("me", "ours edits secret").unwrap(); + repo.switch_with_identity("feature", Some(&alice_sk)) + .unwrap(); + std::fs::write(root.join("secret/a.txt"), b"theirs\n").unwrap(); + repo.commit("me", "theirs edits secret").unwrap(); + repo.switch_with_identity("main", Some(&alice_sk)).unwrap(); + + let err = repo + .merge_with_identity("feature", "me", Some(&alice_sk)) + .unwrap_err(); + assert!(matches!(err, Error::MergeConflicts(1)), "got {err:?}"); + + repo.resolve_path("secret/a.txt", ResolveSide::Ours, Some(&alice_sk)) + .unwrap(); + + // The ours-side ciphertext blob id, re-derived straight from the DAG + // (HEAD is still the "ours" commit — a merge in progress never moves + // it) — this is what the cache must map "ours\n" to. + let ours_tip = repo.head_tip().unwrap().unwrap(); + let ours_blob_id = { + let store_arc = repo.vfs().store(); + let mut store = store_arc.lock().unwrap(); + let snap = store.get_snapshot(&ours_tip).unwrap(); + let entries = tree_file_entries_with_perms(&mut store, snap.root).unwrap(); + entries.get("secret/a.txt").unwrap().0 + }; + + let cache = repo.open_protected_cache().unwrap(); + assert_eq!( + cache.unchanged("secret/a.txt", b"ours\n"), + Some(ours_blob_id), + "resolve must record the resolved plaintext -> chosen ciphertext id in the main cache" + ); + + drop(repo); + std::fs::remove_dir_all(&root).ok(); + } + #[test] fn resolved_merge_completes_via_commit() { let root = tmp_root("resolve-completes"); diff --git a/crates/repo/src/layout.rs b/crates/repo/src/layout.rs index 25acf17d..fecc32d0 100644 --- a/crates/repo/src/layout.rs +++ b/crates/repo/src/layout.rs @@ -110,4 +110,22 @@ impl Layout { pub fn serve_tokens_path(&self) -> PathBuf { self.dot_sc.join("serve-tokens.toml") } + + /// `.sc/local-key` — per-repo random key for the P33 unchanged-detection + /// cache's keyed hashes. Never committed, never transferred. + pub fn local_key_path(&self) -> PathBuf { + self.dot_sc.join("local-key") + } + + /// `.sc/protected-cache` — the main working tree's P33 stat cache. + pub fn protected_cache_path(&self) -> PathBuf { + self.dot_sc.join("protected-cache") + } + + /// `.sc/ws/cache-` — workspace `i`'s P33 stat cache. Lives BESIDE the + /// checkout dir (`.sc/ws//`), never inside it, so harvest's worktree + /// read can't pick it up as an untracked file. + pub fn ws_cache_path(&self, i: usize) -> PathBuf { + self.dot_sc.join("ws").join(format!("cache-{i}")) + } } diff --git a/crates/repo/src/lib.rs b/crates/repo/src/lib.rs index 5095a5df..14152ca2 100644 --- a/crates/repo/src/lib.rs +++ b/crates/repo/src/lib.rs @@ -1,6 +1,7 @@ //! `scl-repo` — the durable on-disk repository: `.sc/` layout, refs/HEAD, //! named branches, a git-like working tree, and commit/secret orchestration. +pub(crate) mod cache; pub(crate) mod conflicts; pub mod diff3; pub mod error; diff --git a/crates/repo/src/merge.rs b/crates/repo/src/merge.rs index 2c3232d1..695afd6e 100644 --- a/crates/repo/src/merge.rs +++ b/crates/repo/src/merge.rs @@ -758,7 +758,7 @@ mod tests { // ---- P15: perms-aware three_way_files ---- - use scl_core::{Protection, PROTECTED}; + use scl_core::{Protection, PROTECTED, RANDOMIZED}; /// Convergently encrypt `content`, wrap its DEK for `pk`, and register the /// wrap in `prot` under the ciphertext blob's id. Returns the ciphertext. @@ -772,6 +772,18 @@ mod tests { cipher } + /// Randomized twin of `enc_file` (P33): seals via `encrypt_path_randomized`, + /// so identical plaintext yields a fresh ciphertext/id on every call. + fn enc_file_rand(prot: &mut Protection, pk: &scl_crypto::PublicKey, content: &[u8]) -> Vec { + let (cipher, dek) = scl_crypto::encrypt_path_randomized(content); + let id = Object::blob(cipher.clone()).id(); + prot.wrapped + .entry(id) + .or_default() + .push(scl_crypto::wrap_dek_for(&dek, pk)); + cipher + } + /// Write a tree whose files carry explicit `perms` bytes; returns its root id. fn tree_with_perms(repo: &VfsRepo, files: &[(&str, Vec, u8)]) -> ObjectId { let fs: Vec<(String, Vec, FileMode, u8)> = files @@ -917,6 +929,171 @@ mod tests { assert_eq!(f.bytes, b"L1\nl2\nL3\n", "both edits present in plaintext"); } + // ---- P33: randomized protected-path encryption ---- + + #[test] + fn identical_independent_edits_now_conflict_and_resolve_with_identity() { + // base: secret/a.txt = "v1" (randomized). ours and theirs BOTH edit it + // to "v2" — independently, so each seal is fresh and the two + // ciphertexts/ids differ even though the plaintexts are identical. + // Pre-P33 convergent sealing would have id-matched this as a clean + // fast path; randomized sealing makes it a genuine content-divergent + // protected case. + let (alice_sk, alice_pk) = scl_crypto::generate_keypair(); + let (mut bp, mut op, mut tp) = ( + Protection::default(), + Protection::default(), + Protection::default(), + ); + let cb = enc_file_rand(&mut bp, &alice_pk, b"v1\n"); + let co = enc_file_rand(&mut op, &alice_pk, b"v2\n"); + let ct = enc_file_rand(&mut tp, &alice_pk, b"v2\n"); + assert_ne!( + co, ct, + "independent randomized seals of identical plaintext differ" + ); + + let repo = VfsRepo::new(Store::with_budget(1 << 20)); + let base = tree_with_perms(&repo, &[("secret/a.txt", cb, PROTECTED | RANDOMIZED)]); + let ours = tree_with_perms(&repo, &[("secret/a.txt", co, PROTECTED | RANDOMIZED)]); + let theirs = tree_with_perms(&repo, &[("secret/a.txt", ct, PROTECTED | RANDOMIZED)]); + let arc = repo.store(); + let mut s = arc.lock().unwrap(); + + // Without identity: the same protected-conflict/needs-identity + // outcome the existing content-divergent test expects. + let err = three_way_files(&mut s, Some((base, &bp)), (ours, &op), (theirs, &tp), None) + .unwrap_err(); + assert!( + matches!(&err, Error::ProtectedMergeNeedsIdentity(p) if p == "secret/a.txt"), + "got {err:?}" + ); + + // With identity: diff3 of identical plaintexts merges cleanly and + // the winning content re-seals randomized (needs_encrypt is set). + let fm = three_way_files( + &mut s, + Some((base, &bp)), + (ours, &op), + (theirs, &tp), + Some(&alice_sk), + ) + .unwrap(); + assert!(fm.conflicts.is_empty()); + assert_eq!(fm.files.len(), 1); + let f = &fm.files[0]; + assert_eq!(f.path, "secret/a.txt"); + assert_eq!( + f.bytes, b"v2\n", + "identical plaintexts merge to v2 with no conflict" + ); + assert!(f.needs_encrypt, "merged plaintext pending re-encryption"); + // `three_way_files` never surfaces RANDOMIZED here: for a + // needs_encrypt entry the caller (repo.rs's completion path) reads + // `f.bytes`/`f.mode`/the matched rule's recipients, never `f.perms` + // — the RANDOMIZED bit is applied downstream by `encrypt_protected` + // at actual re-encryption time, not by this merge step. `f.perms` + // here only ever carries the PROTECTED marker bit (see the mask at + // the content-divergent match arm above). + assert_eq!( + f.perms, PROTECTED, + "three_way_files marks the path PROTECTED; RANDOMIZED is applied \ + at downstream re-encryption, not surfaced here" + ); + } + + #[test] + fn mixed_convergent_and_randomized_tree_merges_by_id_fast_paths() { + // base holds one plain file, one convergent protected entry + // (enc_file), and one randomized protected entry (enc_file_rand). + // ours edits only the plain file; theirs edits only the convergent + // entry. Assert the merge is clean without identity, the untouched + // randomized entry carries its exact blob id AND its + // PROTECTED|RANDOMIZED perms verbatim, and the convergent entry + // takes theirs' id. + let repo = VfsRepo::new(Store::with_budget(1 << 20)); + let (_alice_sk, alice_pk) = scl_crypto::generate_keypair(); + let (mut bp, op, mut tp) = ( + Protection::default(), + Protection::default(), + Protection::default(), + ); + let conv_base = enc_file(&mut bp, &alice_pk, b"c1\n"); + let rand_base = enc_file_rand(&mut bp, &alice_pk, b"r1\n"); + let conv_theirs = enc_file(&mut tp, &alice_pk, b"c2\n"); + + let base = tree_with_perms( + &repo, + &[ + ("plain.txt", b"p1\n".to_vec(), 0), + ("secret/conv.txt", conv_base.clone(), PROTECTED), + ("secret/rand.txt", rand_base.clone(), PROTECTED | RANDOMIZED), + ], + ); + let ours = tree_with_perms( + &repo, + &[ + ("plain.txt", b"p2\n".to_vec(), 0), + ("secret/conv.txt", conv_base.clone(), PROTECTED), + ("secret/rand.txt", rand_base.clone(), PROTECTED | RANDOMIZED), + ], + ); + let theirs = tree_with_perms( + &repo, + &[ + ("plain.txt", b"p1\n".to_vec(), 0), + ("secret/conv.txt", conv_theirs.clone(), PROTECTED), + ("secret/rand.txt", rand_base.clone(), PROTECTED | RANDOMIZED), + ], + ); + + let arc = repo.store(); + let mut s = arc.lock().unwrap(); + let fm = + three_way_files(&mut s, Some((base, &bp)), (ours, &op), (theirs, &tp), None).unwrap(); + assert!(fm.conflicts.is_empty()); + assert!( + fm.files.iter().all(|f| !f.needs_encrypt), + "every entry resolved by an id fast path -- no decryption occurred" + ); + + let plain = fm.files.iter().find(|f| f.path == "plain.txt").unwrap(); + assert_eq!(plain.bytes, b"p2\n", "ours' plain edit wins"); + + let conv = fm + .files + .iter() + .find(|f| f.path == "secret/conv.txt") + .unwrap(); + assert_eq!(conv.bytes, conv_theirs, "convergent entry takes theirs' id"); + assert!(conv.perms & PROTECTED != 0); + let conv_theirs_id = Object::blob(conv_theirs).id(); + assert!( + fm.wrapped_carry.contains_key(&conv_theirs_id), + "surviving convergent blob's wraps carried" + ); + + let rand = fm + .files + .iter() + .find(|f| f.path == "secret/rand.txt") + .unwrap(); + assert_eq!( + rand.bytes, rand_base, + "untouched randomized entry carries its exact blob id" + ); + assert_eq!( + rand.perms, + PROTECTED | RANDOMIZED, + "untouched randomized entry carries PROTECTED|RANDOMIZED perms verbatim" + ); + let rand_base_id = Object::blob(rand_base).id(); + assert!( + fm.wrapped_carry.contains_key(&rand_base_id), + "untouched randomized blob's wraps carried -- recipients can still decrypt" + ); + } + #[test] fn protected_conflict_carries_plaintext_markers() { let (alice_sk, alice_pk) = scl_crypto::generate_keypair(); diff --git a/crates/repo/src/oplog.rs b/crates/repo/src/oplog.rs index c69eaa7b..a783df5b 100644 --- a/crates/repo/src/oplog.rs +++ b/crates/repo/src/oplog.rs @@ -428,6 +428,7 @@ impl Repo { &protection, None, &self.sparse_spec()?, + None, )? } else { Vec::new() diff --git a/crates/repo/src/protect.rs b/crates/repo/src/protect.rs index 3dc3b8ae..75bbac49 100644 --- a/crates/repo/src/protect.rs +++ b/crates/repo/src/protect.rs @@ -116,12 +116,16 @@ pub(crate) fn decrypt_with( Err(Error::NotAuthorized(path.to_string())) } -/// Convergently encrypt `plaintexts` (path, bytes, mode, rule recipients), -/// wrapping each fresh DEK to its recipients. Returns the ciphertext write-set -/// entries (PROTECTED perms) and fresh wraps keyed by ciphertext blob id. -/// Errors `InvalidArgument` if any file's recipient list is empty: a rule whose -/// every recipient is tombstoned (e.g. crossed revokes merged together) must fail -/// the seal loudly — sealing to nobody mints permanently unreadable ciphertext. +/// Randomized encryption (P33, ADR-0043): each file seals under a fresh +/// random DEK+nonce, wrapping that DEK to its rule's recipients. Returns the +/// ciphertext write-set entries (PROTECTED | RANDOMIZED perms) and fresh +/// wraps keyed by ciphertext blob id. Because the seal is randomized, the +/// blob id is fresh on every call — identical plaintext no longer dedups +/// across paths or across re-seals of the same content; that oracle-closing +/// trade is deliberate (accepted cost 4c). Errors `InvalidArgument` if any +/// file's recipient list is empty: a rule whose every recipient is +/// tombstoned (e.g. crossed revokes merged together) must fail the seal +/// loudly — sealing to nobody mints permanently unreadable ciphertext. pub(crate) fn encrypt_protected( plaintexts: Vec<(String, Vec, FileMode, Vec<[u8; 32]>)>, ) -> Result<( @@ -138,7 +142,7 @@ pub(crate) fn encrypt_protected( (all revoked?); run `sc grant` before committing under this prefix" ))); } - let (blob_bytes, dek) = scl_crypto::encrypt_path(&bytes); + let (blob_bytes, dek) = scl_crypto::encrypt_path_randomized(&bytes); // Build the blob object once for its id; `write_tree_with_perms` does // the (idempotent) store insert below — no second explicit `put`. let blob_id = Object::blob(blob_bytes.clone()).id(); @@ -147,7 +151,12 @@ pub(crate) fn encrypt_protected( .map(|pk| scl_crypto::wrap_dek_for(&dek, &scl_crypto::PublicKey::from_bytes(*pk))) .collect(); fresh_wrapped.insert(blob_id, wks); - all.push((path, blob_bytes, mode, scl_core::PROTECTED)); + all.push(( + path, + blob_bytes, + mode, + scl_core::PROTECTED | scl_core::RANDOMIZED, + )); } Ok((all, fresh_wrapped)) } @@ -155,6 +164,10 @@ pub(crate) fn encrypt_protected( /// Prior-wrap reuse: for each (blob_id, recipient_id) already wrapped in /// `prior`, keep the prior wrap bytes so unchanged content's protection /// encoding (and thus snapshot ids) stays stable. Mutates `fresh` in place. +/// Since P33 (ADR-0043), `encrypt_protected`'s seals are randomized, so a +/// freshly sealed blob's id is never a key in `prior` — this only fires for +/// the carried, id-stable case (Task 5's carry-forward of unchanged +/// protected content), never for a genuinely new seal. pub(crate) fn reuse_prior_wraps( fresh: &mut BTreeMap>, prior: &BTreeMap>, @@ -175,6 +188,27 @@ pub(crate) fn reuse_prior_wraps( mod tests { use super::*; + #[test] + fn encrypt_protected_seals_randomized() { + let (_sk, pk) = scl_crypto::generate_keypair(); + let f = |v: &[u8]| { + encrypt_protected(vec![( + "secret/x".into(), + v.to_vec(), + FileMode::FILE, + vec![pk.to_bytes()], + )]) + .unwrap() + }; + let (a1, _) = f(b"same"); + let (a2, _) = f(b"same"); + assert_ne!( + a1[0].1, a2[0].1, + "two seals of one plaintext must differ (oracle closed)" + ); + assert_eq!(a1[0].3, scl_core::PROTECTED | scl_core::RANDOMIZED); + } + fn prot(prefixes: &[&str]) -> Protection { Protection { prefixes: prefixes diff --git a/crates/repo/src/replay.rs b/crates/repo/src/replay.rs index e6a403fd..5af2339e 100644 --- a/crates/repo/src/replay.rs +++ b/crates/repo/src/replay.rs @@ -465,6 +465,9 @@ impl Repo { author, &message, )?; + // Cache hoisted above the block so its save follows the ref + // move below, matching commit/amend's discipline (P33 Task 7). + let mut cache = self.open_protected_cache()?; { let store_arc = self.vfs().store(); let mut store = store_arc.lock().unwrap(); @@ -479,9 +482,11 @@ impl Repo { &protection, identity, &self.sparse_spec()?, + Some(&mut cache), )?; } refs::write_branch_tip(&self.layout, &head, &id)?; + cache.save_best_effort(); crate::oplog::record( &self.layout, &format!("cherry-pick {refname}"), @@ -590,6 +595,9 @@ impl Repo { let snap = self.snapshot(&tip)?; let store_arc = self.vfs().store(); let mut store = store_arc.lock().unwrap(); + // No identity at abort time: nothing is decrypted, so `materialize` + // records nothing — pass `None`, no cache to open or save + // (P33 Task 7 item 7). skipped = worktree::materialize( &self.layout, &mut store, @@ -598,6 +606,7 @@ impl Repo { &snap.protection, None, &self.sparse_spec()?, + None, )?; } crate::pick_state::clear(&self.layout)?; @@ -697,6 +706,7 @@ impl Repo { let target_snap = store.get_snapshot(&target_tip)?; let target_root = target_snap.root; let target_protection = target_snap.protection; + let mut cache = self.open_protected_cache()?; worktree::materialize( &self.layout, &mut store, @@ -705,9 +715,11 @@ impl Repo { &target_protection, identity, &self.sparse_spec()?, + Some(&mut cache), )?; drop(store); refs::write_branch_tip(&self.layout, &head, &target_tip)?; + cache.save_best_effort(); crate::oplog::record( &self.layout, &format!("rebase onto {target} (ff)"), @@ -979,6 +991,9 @@ impl Repo { } } + // Cache hoisted above the block so its save follows the ref move + // below, matching commit/amend's discipline (P33 Task 7). + let mut cache = self.open_protected_cache()?; { let store_arc = self.vfs().store(); let mut store = store_arc.lock().unwrap(); @@ -994,9 +1009,11 @@ impl Repo { &acc_protection, identity, &self.sparse_spec()?, + Some(&mut cache), )?; } refs::write_branch_tip(&self.layout, &head, &acc_tip)?; + cache.save_best_effort(); crate::oplog::record( &self.layout, &format!("rebase onto {target} ({replayed} replayed, {skipped} skipped)"), @@ -1198,6 +1215,9 @@ impl Repo { let skipped = { let store_arc = self.vfs().store(); let mut store = store_arc.lock().unwrap(); + // No identity at abort time: nothing is decrypted, so `materialize` + // records nothing — pass `None`, no cache to open or save + // (P33 Task 7 item 7). worktree::materialize( &self.layout, &mut store, @@ -1206,6 +1226,7 @@ impl Repo { &snap.protection, None, &self.sparse_spec()?, + None, )? }; crate::rebase_state::clear(&self.layout)?; @@ -2215,6 +2236,177 @@ mod tests { std::fs::remove_dir_all(&root).ok(); } + // ---- P33: randomized protected-path encryption, replay twin ---- + // + // `Repo::protect`/`commit` seal every protected file via + // `encrypt_protected`, which has sealed randomized (PROTECTED|RANDOMIZED) + // since P33 (ADR-0043) — so every real cherry-pick over protected content + // already exercises randomized ciphertext. These twins of + // `cherry_pick_disjoint_protected_commit_needs_no_identity` and + // `cherry_pick_content_divergent_requires_identity_and_reencrypts` pin + // that explicitly: the RANDOMIZED bit rides the fast path verbatim, and + // a content-divergent pick re-seals randomized on completion. + + #[test] + fn cherry_pick_randomized_protected_unchanged_fast_path_needs_no_identity() { + // The picked commit updates a protected file ours never touched: the + // ciphertext-id fast path carries the picked (randomized) blob + // verbatim — no identity needed — and the RANDOMIZED bit rides + // along with PROTECTED. + let root = tmp_root("cp-prot-rand-disjoint"); + let repo = Repo::init(&root).unwrap(); + let (alice_sk, alice_pk) = scl_crypto::generate_keypair(); + repo.protect("secret/", &[alice_pk], None).unwrap(); + std::fs::create_dir_all(root.join("secret")).unwrap(); + std::fs::write(root.join("secret/db.txt"), b"v1").unwrap(); + repo.commit("me", "base").unwrap(); + repo.branch("work").unwrap(); + + // work updates the protected file (fresh randomized seal). + repo.switch_with_identity("work", Some(&alice_sk)).unwrap(); + std::fs::write(root.join("secret/db.txt"), b"v2").unwrap(); + let picked = repo.commit("me", "work updates secret").unwrap(); + let (v2_id, v2_perms) = tree_entry(&repo, &picked, "secret/db.txt"); + assert_eq!( + v2_perms, + scl_core::PROTECTED | scl_core::RANDOMIZED, + "encrypt_protected seals PROTECTED|RANDOMIZED" + ); + + // main gains an unrelated plain commit (keyless hop back). + repo.switch("main").unwrap(); + std::fs::write(root.join("main.txt"), b"m\n").unwrap(); + let main_tip = repo.commit("me", "main adds main.txt").unwrap(); + + // KEYLESS pick: unchanged-on-ours fast path, no identity needed. + let outcome = repo.cherry_pick("work", "me", None, None).unwrap(); + let id = match outcome { + PickResult::Picked(id) => id, + other => panic!("expected Picked, got {other:?}"), + }; + assert_eq!(repo.head_tip().unwrap(), Some(id), "branch advanced"); + let snap = repo.snapshot(&id).unwrap(); + assert_eq!(snap.parents, vec![main_tip]); + + // The picked ciphertext blob is carried byte-for-byte, with its + // RANDOMIZED bit intact. + let (got_id, perms) = tree_entry(&repo, &id, "secret/db.txt"); + assert_eq!(got_id, v2_id, "ciphertext carried verbatim"); + assert_eq!( + perms, + scl_core::PROTECTED | scl_core::RANDOMIZED, + "PROTECTED|RANDOMIZED carried verbatim through the fast path" + ); + let bytes = blob_bytes_of(&repo, &got_id); + let pt = crate::protect::decrypt_with( + &bytes, + &got_id, + &[&snap.protection], + &alice_sk, + "secret/db.txt", + ) + .unwrap(); + assert_eq!(&pt[..], b"v2", "recipient decrypts the picked update"); + + drop(repo); + std::fs::remove_dir_all(&root).ok(); + } + + #[test] + fn cherry_pick_randomized_protected_content_divergent_requires_identity() { + // secret/a.txt diverged in content on both sides (mergeable lines), + // each side's seal randomized and distinct: a keyless pick fails + // with the typed error and moves nothing; with an identity the + // plaintexts are diff3'd and the merged content is re-sealed + // randomized for ALL rule recipients. + let root = tmp_root("cp-prot-rand-divergent"); + let repo = Repo::init(&root).unwrap(); + let (alice_sk, alice_pk) = scl_crypto::generate_keypair(); + let (bob_sk, bob_pk) = scl_crypto::generate_keypair(); + repo.protect("secret/", &[alice_pk, bob_pk], None).unwrap(); + std::fs::create_dir_all(root.join("secret")).unwrap(); + std::fs::write(root.join("secret/a.txt"), b"l1\nl2\nl3\n").unwrap(); + repo.commit("me", "base").unwrap(); + repo.branch("work").unwrap(); + + // ours edits line 1. + std::fs::write(root.join("secret/a.txt"), b"OURS\nl2\nl3\n").unwrap(); + let main_tip = repo.commit("me", "main edits line 1").unwrap(); + let (ours_id, _) = tree_entry(&repo, &main_tip, "secret/a.txt"); + + // work edits line 3 (mergeable divergence) — a fresh, distinct seal. + repo.switch_with_identity("work", Some(&alice_sk)).unwrap(); + std::fs::write(root.join("secret/a.txt"), b"l1\nl2\nTHEIRS\n").unwrap(); + let picked = repo.commit("me", "work edits line 3").unwrap(); + let (theirs_id, _) = tree_entry(&repo, &picked, "secret/a.txt"); + assert_ne!( + ours_id, theirs_id, + "independent randomized seals of divergent content differ" + ); + repo.switch_with_identity("main", Some(&alice_sk)).unwrap(); + + // Keyless: typed error, refs untouched, no pick state. + let err = repo.cherry_pick("work", "me", None, None).unwrap_err(); + assert!( + matches!(err, Error::ProtectedMergeNeedsIdentity(ref p) if p == "secret/a.txt"), + "got {err:?}" + ); + assert_eq!( + repo.head_tip().unwrap(), + Some(main_tip), + "tip must not move" + ); + assert!( + !repo.pick_in_progress(), + "no pick state on the identity refusal" + ); + + // With identity: clean pick, merged content re-sealed randomized. + let outcome = repo + .cherry_pick("work", "me", Some(&alice_sk), None) + .unwrap(); + let id = match outcome { + PickResult::Picked(id) => id, + other => panic!("expected Picked, got {other:?}"), + }; + let snap = repo.snapshot(&id).unwrap(); + assert_eq!(snap.parents, vec![main_tip]); + let (blob_id, perms) = tree_entry(&repo, &id, "secret/a.txt"); + assert_eq!( + perms, + scl_core::PROTECTED | scl_core::RANDOMIZED, + "re-encryption seals PROTECTED|RANDOMIZED" + ); + let bytes = blob_bytes_of(&repo, &blob_id); + assert!( + !bytes.windows(4).any(|w| w == b"OURS"), + "plaintext leaked into the CAS blob" + ); + for (who, sk) in [("alice", &alice_sk), ("bob", &bob_sk)] { + let pt = crate::protect::decrypt_with( + &bytes, + &blob_id, + &[&snap.protection], + sk, + "secret/a.txt", + ) + .unwrap(); + assert_eq!( + &pt[..], + b"OURS\nl2\nTHEIRS\n", + "{who} must decrypt the merged content" + ); + } + // The identity-holder gets the merged plaintext on disk. + assert_eq!( + std::fs::read(root.join("secret/a.txt")).unwrap(), + b"OURS\nl2\nTHEIRS\n".to_vec() + ); + + drop(repo); + std::fs::remove_dir_all(&root).ok(); + } + #[test] fn cherry_pick_protected_conflict_writes_plaintext_markers_worktree_only() { // Same-line edits of secret/a.txt on both sides: the pick conflicts. diff --git a/crates/repo/src/repo.rs b/crates/repo/src/repo.rs index 52ba5e02..95c0c695 100644 --- a/crates/repo/src/repo.rs +++ b/crates/repo/src/repo.rs @@ -76,6 +76,23 @@ impl Repo { &self.layout } + /// Open this repo's main-checkout P33 unchanged-detection cache + /// (`.sc/protected-cache`), keyed by `.sc/local-key`. Every + /// `worktree::materialize` call against the host working tree opens one + /// of these and saves it right after materializing, so a randomized + /// protected file just decrypted to disk is recognized as unchanged by + /// the very next `status`/`diff` — without this, `diff_worktree`'s + /// randomized-entry branch has no stat-hit or keyed-tag to match and + /// spuriously reports the untouched file as modified. + pub(crate) fn open_protected_cache(&self) -> Result { + let key = crate::cache::local_key(&self.layout)?; + Ok(crate::cache::ProtectedCache::open( + key, + self.layout.root.clone(), + Some(self.layout.protected_cache_path()), + )) + } + /// The tip snapshot of the current branch (None if unborn). pub fn head_tip(&self) -> Result> { refs::head_tip(&self.layout) @@ -191,7 +208,14 @@ impl Repo { message: &str, ) -> Result { let files = worktree::read_worktree(&self.layout, &self.tracked_paths_at(Some(parent))?)?; - self.snapshot_files( + // Pick/rebase completion opens its own main cache. It saves right + // after `snapshot_files` (before the caller moves the ref): the sealed + // blobs are already in the store via `write_tree_with_perms`, so a + // recorded-but-unlanded id is benign (the cache degradation contract), + // and both callers (`commit`'s pick arm, `rebase_continue`) move the + // ref immediately after a successful return. + let mut cache = self.open_protected_cache()?; + let id = self.snapshot_files( files, Some(parent), None, @@ -200,9 +224,16 @@ impl Repo { pick_registry_base, None, &self.sparse_spec()?, + Some(&mut cache), author, message, - ) + )?; + // Best-effort (P33 Task 7): this function does not own the ref move + // (both callers move it right after a successful return), and its + // save-before-ref is already justified above — a recorded-but-unlanded + // id is benign. Never let cache trouble abort the completion. + cache.save_best_effort(); + Ok(id) } /// The commit pipeline minus ref movement: split protected/plaintext files, @@ -263,6 +294,7 @@ impl Repo { pick_registry_base: Option, parents_override: Option>, sparse: &crate::sparse::Sparse, + mut cache: Option<&mut crate::cache::ProtectedCache>, author: &str, message: &str, ) -> Result { @@ -353,11 +385,143 @@ impl Repo { return Err(Error::SecretDetected(report)); } - // Encrypt protected files; accumulate fresh wrapped DEKs keyed by blob id. + // Plaintext files land first (perms 0); protected files are decided + // per-path below (carry vs. seal) and appended. let mut all: Vec<(String, Vec, scl_core::FileMode, u8)> = plain.into_iter().map(|(p, b, m)| (p, b, m, 0u8)).collect(); - let (protected_all, mut fresh_wrapped) = crate::protect::encrypt_protected(protected)?; + let mut fresh_wrapped: BTreeMap> = BTreeMap::new(); + + // Partial-clone context is needed here now (the P33 dispatch flattens + // the tip tree, which must stay gap-tolerant on a partial clone) and + // again in the absent-path carry block below — computed once. + let promisor = self.promisor()?; + let partial = promisor.is_some(); + // Merge/pick completion guard (P27 Task 5, T5-I4): `graft_out_of_sparse` + // below only splices out-of-filter subtrees back in for a plain + // single-tip commit (`decided_root.is_none() && merge_head.is_none()`). + // A merge (`merge_head: Some`) or a CONFLICTED pick/rebase completion + // (`decided_root: Some`) builds `all`/`root` purely from in-filter + // content on a partial clone — with no graft to follow, that would + // silently DROP every out-of-filter subtree from the new snapshot. + // Refuse loudly instead of ever letting that happen; a clean + // (non-conflicted) pick/rebase fold still has `decided_root: None` + // and keeps using the single-tip graft path, unaffected. + if partial && (decided_root.is_some() || merge_head.is_some()) { + return Err(crate::promisor::partial_clone_unsupported( + "merge/pick completion", + )); + } + + // P33 format-dispatched carry (ADR-0043): decide per protected path + // whether the tip already holds this exact content — carry the prior + // ciphertext id verbatim (quiet history, no reseal) — or it must be + // sealed fresh. A CONVERGENT prior (no `RANDOMIZED` bit) uses the exact + // re-encrypt-and-compare proof and needs no cache: this is what keeps + // the format upgrade non-mandatory (unchanged legacy content stays + // convergent forever). A RANDOMIZED prior can't be compared by + // re-encrypting (a fresh seal differs every time), so it consults the + // `.sc`-local stat/tag cache; a lost/stale cache degrades to a spurious + // reseal, never to incorrectness. Everything else seals. + let tip_entries: BTreeMap = match tip { + Some(t) => { + let store_arc = self.vfs.store(); + let mut store = store_arc.lock().unwrap(); + let parent_root = store.get_snapshot(&t)?.root; + if partial { + worktree::tree_file_entries_with_perms_sparse(&mut store, parent_root, sparse)? + } else { + worktree::tree_file_entries_with_perms(&mut store, parent_root)? + } + } + None => BTreeMap::new(), + }; + + let mut to_seal: Vec = Vec::new(); + // (path, plaintext) for each sealed path, so its fresh blob id can be + // recorded into the cache after `encrypt_protected` returns. + let mut sealed_plaintexts: Vec<(String, Vec)> = Vec::new(); + for (path, bytes, mode, granted) in protected { + // Decide carry (Some(prior blob id + perms)) vs. seal (None). + let carry: Option<(ObjectId, u8)> = match tip_entries.get(&path) { + Some((blob_id, _m, perms)) if perms & scl_core::PROTECTED != 0 => { + let unchanged = if perms & scl_core::RANDOMIZED == 0 { + // Convergent prior: exact re-encrypt-and-compare. + Object::blob(scl_crypto::encrypt_path(&bytes).0).id() == *blob_id + } else { + // Randomized prior: the local stat/tag cache is the only + // way to prove "same plaintext". Carry iff the cached id + // matches the prior tip id (double-checked: same content + // AND same recorded seal). The cache resolves the abs + // stat path against its own checkout root (P33 Task 7), + // so a workspace commit stats the workspace file, not + // the host's. + cache.as_deref().and_then(|c| c.unchanged(&path, &bytes)) == Some(*blob_id) + }; + if unchanged { + Some((*blob_id, *perms)) + } else { + None + } + } + _ => None, + }; + + match carry { + Some((blob_id, perms)) => { + // Carry the prior ciphertext + perms verbatim (same + // mechanics as the absent-path carry below). + let cipher = { + let store_arc = self.vfs.store(); + let mut store = store_arc.lock().unwrap(); + match store.get(&blob_id)? { + Object::Blob(b) => Some(b.to_vec()), + _ => None, + } + }; + match cipher { + Some(cipher) => { + all.push((path.clone(), cipher, mode, perms)); + if let Some(prior_wks) = protection.wrapped.get(&blob_id) { + fresh_wrapped + .entry(blob_id) + .or_insert_with(|| prior_wks.clone()); + } + // Refresh a carried randomized path's cache entry so + // the next commit gets a fast stat hit (optimization + // only; the caller persists after the ref moves). + if perms & scl_core::RANDOMIZED != 0 { + if let Some(c) = cache.as_deref_mut() { + c.record(&path, &bytes, blob_id); + } + } + } + // Prior blob unexpectedly absent (e.g. not a blob) — + // fall back to sealing rather than dropping content. + None => { + sealed_plaintexts.push((path.clone(), bytes.clone())); + to_seal.push((path, bytes, mode, granted)); + } + } + } + None => { + sealed_plaintexts.push((path.clone(), bytes.clone())); + to_seal.push((path, bytes, mode, granted)); + } + } + } + + let (protected_all, sealed_wrapped) = crate::protect::encrypt_protected(to_seal)?; + // Record every freshly sealed path's new blob id into the cache; the + // committing caller persists it only after the ref actually moves. + if let Some(c) = cache { + for (path, cipher, _mode, _perms) in &protected_all { + if let Some((_, pt)) = sealed_plaintexts.iter().find(|(p, _)| p == path) { + c.record(path, pt, Object::blob(cipher.clone()).id()); + } + } + } all.extend(protected_all); + fresh_wrapped.extend(sealed_wrapped); // Safe-by-default: carry forward still-protected files that are absent // from the working tree. `commit` cannot distinguish "absent because the @@ -415,23 +579,8 @@ impl Repo { // with an active sparse spec still has every object, and its // per-blob byte-carry below (the existing P24 mechanism) must stay // exactly as it was to avoid any behavior change for that case. - let promisor = self.promisor()?; - let partial = promisor.is_some(); - // Merge/pick completion guard (P27 Task 5, T5-I4): `graft_out_of_sparse` - // below only splices out-of-filter subtrees back in for a plain - // single-tip commit (`decided_root.is_none() && merge_head.is_none()`). - // A merge (`merge_head: Some`) or a CONFLICTED pick/rebase completion - // (`decided_root: Some`) builds `all`/`root` purely from in-filter - // content on a partial clone — with no graft to follow, that would - // silently DROP every out-of-filter subtree from the new snapshot. - // Refuse loudly instead of ever letting that happen; a clean - // (non-conflicted) pick/rebase fold still has `decided_root: None` - // and keeps using the single-tip graft path, unaffected. - if partial && (decided_root.is_some() || merge_head.is_some()) { - return Err(crate::promisor::partial_clone_unsupported( - "merge/pick completion", - )); - } + // (`promisor`/`partial` and the merge/pick-completion partial-clone + // guard were hoisted above the P33 dispatch, which needs them too.) { let mut on_disk: std::collections::BTreeSet = all.iter().map(|(p, _, _, _)| p.clone()).collect(); @@ -538,11 +687,14 @@ impl Repo { // here is still the tip's own full map (`prior` is taken // below, after this point) and already carries every wrap // the tip itself could offer; union in any entry - // `fresh_wrapped` doesn't already know about. Convergent - // encryption keeps a blob's id stable regardless of who - // grafted it, so reusing the prior wrap bytes verbatim is - // correct, not just convenient — no re-encryption, no key - // material touched. + // `fresh_wrapped` doesn't already know about. The graft + // splices the subtree in BY ID and never re-derives or + // touches the blob's DEK, so reusing the prior wrap bytes + // verbatim is correct — this holds for a RANDOMIZED blob + // exactly as it does for a convergent one (carry-by-id, not + // convergent-id-stability, is the load-bearing property; a + // randomized blob's id is just as stable across a + // by-id graft as a convergent one's). for (id, wks) in &protection.wrapped { fresh_wrapped.entry(*id).or_insert_with(|| wks.clone()); } @@ -555,8 +707,24 @@ impl Repo { // `wrap_dek_for` randomizes its ephemeral key — re-wrapping every commit // would change the `protection` encoding (and thus the snapshot id) for // identical content, breaking "same content -> stable history". Carrying the - // prior wrap forward keeps it stable; only a newly-added recipient (or a new - // blob) gets a fresh wrap, and a revoked recipient is already absent here. + // prior wrap forward keeps it stable. + // + // P33 wrap-lifecycle note: a SEALED blob (new or edited content) wraps + // for exactly `granted_keys()`, so a revoked recipient is absent from + // its fresh wraps as before. A CARRIED blob (P33 unchanged-content + // dispatch above), however, keeps the tip's prior wrap list VERBATIM — + // it never re-derives from `granted_keys()`. That matches how the P15 + // absent-path carry has always behaved, and it is correct: `revoke` + // itself eagerly strips the revoked wrap from the tip + // (`protect_ops::revoke`), so an ordinary post-revoke commit on + // unchanged content carries the already-stripped list. The one place + // carry diverges from the old always-reseal behavior is the ADR-0026 + // corollary — merging a pre-revoke branch re-attaches a revoked + // recipient's old wraps as historical facts; the old reseal stripped + // them back to `granted_keys()` on the next commit, whereas carry + // preserves them on unchanged content. This is not a new disclosure + // (the recipient already holds the key; the ciphertext is unchanged) + // and `sc rewrap` remains the deliberate cutover, per ADR-0027. let prior = std::mem::take(&mut protection.wrapped); crate::protect::reuse_prior_wraps(&mut fresh_wrapped, &prior); protection.wrapped = fresh_wrapped; @@ -617,9 +785,14 @@ impl Repo { let merging = merge_head.is_some(); let picking = pick_head.is_some(); // Pick completion (no merge in progress) is the extracted assembly - // (P19 Task 2 groundwork), shared with `rebase_continue`. Every - // other case (plain commit, merge completion) still calls - // `snapshot_files` directly with a freshly read working tree. + // (P19 Task 2 groundwork), shared with `rebase_continue`; it manages + // its own protected-path cache internally. Every other case (plain + // commit, merge completion) calls `snapshot_files` directly with a + // freshly read working tree and a cache this function opens here and + // saves only after the ref moves below — the P33 discipline. `None` + // in the pick arm avoids double-managing (and clobbering) the cache + // `assemble_completion_snapshot` already saved. + let mut cache: Option = None; let id = match (tip, merge_head, pick_head) { (Some(t), None, Some(ph)) => self.assemble_completion_snapshot( t, @@ -631,7 +804,8 @@ impl Repo { )?, _ => { let files = worktree::read_worktree(&self.layout, &self.tracked_paths()?)?; - self.snapshot_files( + let mut c = self.open_protected_cache()?; + let id = self.snapshot_files( files, tip, merge_head, @@ -640,14 +814,23 @@ impl Repo { pick_registry_base, None, &self.sparse_spec()?, + Some(&mut c), author, message, - )? + )?; + cache = Some(c); + id } }; refs::write_branch_tip(&self.layout, &head, &id)?; crate::merge_state::clear(&self.layout)?; crate::pick_state::clear(&self.layout)?; + // The ref moved: persist the cache best-effort (no-op in the pick arm, + // which manages its own). Cache trouble must never abort a commit + // whose ref already moved (P33 Task 7). + if let Some(c) = cache { + c.save_best_effort(); + } let label = if merging { "commit (merge)" } else if picking { @@ -702,6 +885,7 @@ impl Repo { let before = refs::read_branch_tip(&self.layout, &head)?; let files = worktree::read_worktree(&self.layout, &self.tracked_paths()?)?; + let mut cache = self.open_protected_cache()?; let id = self.snapshot_files( files, Some(tip), @@ -711,11 +895,13 @@ impl Repo { None, Some(tip_snap.parents.clone()), &self.sparse_spec()?, + Some(&mut cache), author, &msg, )?; refs::write_branch_tip(&self.layout, &head, &id)?; + cache.save_best_effort(); crate::oplog::record( &self.layout, "amend", @@ -810,12 +996,14 @@ impl Repo { }; let store_arc = self.vfs.store(); let mut store = store_arc.lock().unwrap(); + let cache = self.open_protected_cache()?; worktree::diff_worktree( &self.layout, &mut store, head_root, &protection, &self.sparse_spec()?, + Some(&cache), ) } @@ -850,6 +1038,18 @@ impl Repo { .map(|t| self.snapshot(&t)) .transpose()? .map(|s| s.root); + // Randomized (P33) protected entries need the local unchanged-cache to + // report clean at all (re-encryption proves nothing for them). A + // failure to load/mint `.sc/local-key` degrades to `None` here rather + // than failing the whole diff — spurious-but-safe: every randomized + // entry just reports "changed" instead of the diff erroring out. + let cache = crate::cache::local_key(&self.layout).ok().map(|key| { + crate::cache::ProtectedCache::open( + key, + self.layout.root.clone(), + Some(self.layout.protected_cache_path()), + ) + }); let store_arc = self.vfs.store(); let mut store = store_arc.lock().unwrap(); let head = match head_root { @@ -881,8 +1081,18 @@ impl Repo { Some((blob_id, _mode, perms)) if *perms & PROTECTED != 0 => { // Never show protected content; report the change status only. if let Some(bytes) = disk { - let disk_id = Object::blob(scl_crypto::encrypt_path(bytes).0).id(); - if disk_id != *blob_id { + let changed = if *perms & scl_core::RANDOMIZED == 0 { + // Convergent (legacy): re-encryption yields an exact + // compare, unchanged from pre-P33. + Object::blob(scl_crypto::encrypt_path(bytes).0).id() != *blob_id + } else { + // Randomized (P33): re-encryption proves nothing; the + // cache is the only public-key-free unchanged proof. + // A miss/disagreement/absent cache reports changed — + // spurious-but-safe, never incorrect. + cache.as_ref().and_then(|c| c.unchanged(path, bytes)) != Some(*blob_id) + }; + if changed { out.push_str(&format!( "protected file changed: {path} (content not shown)\n" )); @@ -1034,6 +1244,7 @@ impl Repo { let theirs_snap = store.get_snapshot(&theirs)?; let theirs_root = theirs_snap.root; let theirs_protection = theirs_snap.protection; + let mut cache = self.open_protected_cache()?; let skipped = worktree::materialize( &self.layout, &mut store, @@ -1042,9 +1253,11 @@ impl Repo { &theirs_protection, identity, &self.sparse_spec()?, + Some(&mut cache), )?; drop(store); refs::write_branch_tip(&self.layout, &head, &theirs)?; + cache.save_best_effort(); crate::oplog::record( &self.layout, &format!("merge {branch} (adopt)"), @@ -1069,6 +1282,7 @@ impl Repo { let theirs_root = theirs_snap.root; let theirs_protection = theirs_snap.protection; let ours_root = store.get_snapshot(&ours)?.root; + let mut cache = self.open_protected_cache()?; let skipped = worktree::materialize( &self.layout, &mut store, @@ -1077,9 +1291,11 @@ impl Repo { &theirs_protection, identity, &self.sparse_spec()?, + Some(&mut cache), )?; drop(store); refs::write_branch_tip(&self.layout, &head, &theirs)?; + cache.save_best_effort(); crate::oplog::record( &self.layout, &format!("merge {branch} (ff)"), @@ -1254,6 +1470,10 @@ impl Repo { // a merged PROTECTED entry decrypts for `identity` when possible, else // is skipped (never writes ciphertext to disk) and reported to the // caller. (Sidecars exist only on the conflict path, handled above.) + // The cache is hoisted above the block so its save can follow the ref + // move (`commit_snapshot` below), matching commit/amend's discipline + // (P33 Task 7). + let mut cache = self.open_protected_cache()?; let skipped = { let store_arc = self.vfs.store(); let mut store = store_arc.lock().unwrap(); @@ -1265,6 +1485,7 @@ impl Repo { &merged_protection, identity, &self.sparse_spec()?, + Some(&mut cache), )? }; @@ -1278,6 +1499,7 @@ impl Repo { author, &format!("merge {branch}"), )?; + cache.save_best_effort(); crate::oplog::record( &self.layout, &format!("merge {branch}"), @@ -1357,6 +1579,7 @@ impl Repo { // completion commit's decided-tree carry-forward preserves // skipped content, so nothing is lost by not surfacing the list // here (the `Err`/`Stopped` return can't carry it). + let mut cache = self.open_protected_cache()?; let _skipped = worktree::materialize( &self.layout, &mut store, @@ -1365,7 +1588,11 @@ impl Repo { conflict_prot, identity, &self.sparse_spec()?, + Some(&mut cache), )?; + // Conflict materialization moves no branch ref; best-effort save + // so cache trouble never aborts the conflict write-out (P33 Task 7). + cache.save_best_effort(); } for (path, bytes, _mode, _recipients) in to_encrypt { let full = worktree::safe_join(&self.layout.root, path)?; @@ -1413,7 +1640,10 @@ impl Repo { let ours_root = ours_snap.root; let ours_protection = ours_snap.protection; let theirs_root = store.get_snapshot(&theirs_id)?.root; - // No identity at abort time: protected files in ours are skipped (not decrypted). + // No identity at abort time: protected files in ours are skipped + // (not decrypted). With nothing decrypted, `materialize` records + // nothing into a cache, so pass `None` — no cache to open or save + // (P33 Task 7 item 7). skipped = worktree::materialize( &self.layout, &mut store, @@ -1422,6 +1652,7 @@ impl Repo { &ours_protection, None, &self.sparse_spec()?, + None, )?; } crate::merge_state::clear(&self.layout)?; @@ -1530,6 +1761,9 @@ impl Repo { let snap = store_arc.lock().unwrap().get_snapshot(&target_tip)?; (snap.root, snap.protection) }; + // Cache hoisted above the block so its save follows the ref move + // (`write_head` below), matching commit/amend's discipline (P33 Task 7). + let mut cache = self.open_protected_cache()?; let skipped = { let store_arc = self.vfs.store(); let mut store = store_arc.lock().unwrap(); @@ -1541,9 +1775,11 @@ impl Repo { &target_protection, identity, &self.sparse_spec()?, + Some(&mut cache), )? }; refs::write_head(&self.layout, name)?; + cache.save_best_effort(); crate::oplog::record( &self.layout, &format!("switch {name}"), @@ -2646,13 +2882,20 @@ mod tests { } #[test] - fn convergent_merge_ids_are_stable_across_repos() { + fn merge_ids_no_longer_converge_across_repos() { + // P33/ADR-0043: `encrypt_protected`'s seals are now randomized, so + // this pre-P33 test's premise ("identical merged plaintext converges + // to the same blob id across independent repos") is deliberately + // broken — the oracle that convergence gave an equality-testing + // observer is exactly what randomization closes (accepted cost 4c). + // Renamed and inverted to pin the NEW behavior: two independent + // repos building the identical protected divergence and + // content-merging it now produce DIFFERENT ciphertext blob ids for + // the same plaintext. + // // Two independent repos build the identical protected divergence // (same base plaintext, same edits on both sides) and content-merge - // with their own (distinct) recipient identity. Convergent encryption - // means the merged plaintext is identical, so the resulting ciphertext - // blob id must be IDENTICAL across the two repos — merge output is - // deterministic content addressing, not per-repo randomness. + // with their own (distinct) recipient identity. fn build(tag: &str) -> (Repo, std::path::PathBuf, ObjectId, ObjectId) { let root = tmp_root(tag); let repo = Repo::init(&root).unwrap(); @@ -2684,9 +2927,9 @@ mod tests { let (repo1, root1, _id1, blob1) = build("p15-merge-convergent-a"); let (repo2, root2, _id2, blob2) = build("p15-merge-convergent-b"); - assert_eq!( + assert_ne!( blob1, blob2, - "identical merged plaintext must converge to the same blob id" + "randomized seals must NOT converge across independent repos (oracle closed, P33)" ); drop(repo1); drop(repo2); @@ -3447,6 +3690,146 @@ mod tests { std::fs::remove_dir_all(&root).unwrap(); } + // ---- P33 Task 5: format-dispatched carry in the commit pipeline ---- + + /// The ciphertext blob id recorded for `path` in `commit`'s tree. + fn p33_entry_id(repo: &Repo, commit: &ObjectId, path: &str) -> ObjectId { + let store_arc = repo.vfs.store(); + let mut s = store_arc.lock().unwrap(); + let root = s.get_snapshot(commit).unwrap().root; + let entries = worktree::tree_file_entries_with_perms(&mut s, root).unwrap(); + entries[path].0 + } + + fn p33_blob_bytes(repo: &Repo, id: &ObjectId) -> Vec { + let store_arc = repo.vfs.store(); + let mut s = store_arc.lock().unwrap(); + match s.get(id).unwrap() { + Object::Blob(b) => b.to_vec(), + _ => panic!("expected Blob"), + } + } + + #[test] + fn commit_carries_unchanged_convergent_content_without_reseal() { + // A pre-P33-style store: protect + commit mints a CONVERGENT protected + // entry. This pins the carry rule by blob-id stability across an + // unrelated commit — the protected file is never re-touched, so it + // must carry (not re-seal). (Under Task 5 carry and re-seal both yield + // the same convergent id, so this is a regression guard that survives + // the Task 6 seal flip precisely because unchanged content carries.) + let root = tmp_root("p33-conv-carry"); + let repo = Repo::init(&root).unwrap(); + let (_alice_sk, alice_pk) = scl_crypto::generate_keypair(); + repo.protect("secret/", &[alice_pk], None).unwrap(); + std::fs::create_dir_all(root.join("secret")).unwrap(); + std::fs::write(root.join("secret/db.txt"), b"hunter2").unwrap(); + let c1 = repo.commit("alice", "one").unwrap(); + let id1 = p33_entry_id(&repo, &c1, "secret/db.txt"); + + // Unrelated edit; the protected file is untouched on disk. + std::fs::write(root.join("readme.md"), b"hello").unwrap(); + let c2 = repo.commit("alice", "two").unwrap(); + let id2 = p33_entry_id(&repo, &c2, "secret/db.txt"); + assert_eq!( + id1, id2, + "unchanged protected content must carry, not re-seal" + ); + + drop(repo); + std::fs::remove_dir_all(&root).unwrap(); + } + + #[test] + fn commit_randomized_carry_via_cache_and_reseal_on_lost_cache() { + // Build a tip whose protected entry is `PROTECTED | RANDOMIZED` (as a + // post-Task-6 authorized checkout leaves it), record the plaintext in + // the on-disk cache, and prove: (a) a cache hit CARRIES the randomized + // blob id verbatim; (b) losing the cache degrades to a spurious + // re-seal (different id) — never to incorrectness (still decrypts). + let root = tmp_root("p33-rand-carry"); + let repo = Repo::init(&root).unwrap(); + let (alice_sk, alice_pk) = scl_crypto::generate_keypair(); + repo.protect("secret/", std::slice::from_ref(&alice_pk), None) + .unwrap(); + + // Seal a randomized protected entry and install it as the branch tip, + // keeping the `secret/` rule from the protect commit so the next + // commit routes `secret/x` through the protected split, not `plain`. + let pt = b"top secret".to_vec(); + let (cipher, dek) = scl_crypto::encrypt_path_randomized(&pt); + let randomized_id = Object::blob(cipher.clone()).id(); + let wks = vec![scl_crypto::wrap_dek_for(&dek, &alice_pk)]; + + let tip = repo.head_tip().unwrap().unwrap(); + let tip_snap = repo.snapshot(&tip).unwrap(); + let mut protection = tip_snap.protection.clone(); + protection.wrapped.insert(randomized_id, wks); + let tree = repo + .vfs + .write_tree_with_perms(&[( + "secret/x".to_string(), + cipher.clone(), + scl_core::FileMode::FILE, + scl_core::PROTECTED | scl_core::RANDOMIZED, + )]) + .unwrap(); + let rand_tip = repo + .commit_snapshot( + tree, + vec![tip], + tip_snap.secrets.clone(), + protection, + "system", + "randomized tip", + ) + .unwrap(); + assert_eq!(repo.head_tip().unwrap(), Some(rand_tip)); + + // Materialize the plaintext on disk and record it in the on-disk cache. + std::fs::create_dir_all(root.join("secret")).unwrap(); + std::fs::write(root.join("secret/x"), &pt).unwrap(); + { + let key = crate::cache::local_key(repo.layout()).unwrap(); + let mut cache = crate::cache::ProtectedCache::open( + key, + repo.layout().root.clone(), + Some(repo.layout().protected_cache_path()), + ); + cache.record("secret/x", &pt, randomized_id); + cache.save().unwrap(); + } + + // Unrelated edit; the randomized protected file is untouched. + std::fs::write(root.join("readme.md"), b"hello").unwrap(); + let c2 = repo.commit("alice", "unrelated").unwrap(); + assert_eq!( + p33_entry_id(&repo, &c2, "secret/x"), + randomized_id, + "cache hit carries" + ); + + // Lose the cache: the next commit cannot prove unchanged, so it re-seals. + std::fs::remove_file(repo.layout().protected_cache_path()).unwrap(); + std::fs::write(root.join("readme.md"), b"more").unwrap(); + let c3 = repo.commit("alice", "unrelated2").unwrap(); + let id3 = p33_entry_id(&repo, &c3, "secret/x"); + assert_ne!( + id3, randomized_id, + "lost cache degrades to a spurious re-seal" + ); + // Never incorrectness: the re-sealed blob still decrypts to the plaintext. + let bytes = p33_blob_bytes(&repo, &id3); + let snap3 = repo.snapshot(&c3).unwrap(); + let got = + crate::protect::decrypt_with(&bytes, &id3, &[&snap3.protection], &alice_sk, "secret/x") + .unwrap(); + assert_eq!(&got[..], b"top secret"); + + drop(repo); + std::fs::remove_dir_all(&root).unwrap(); + } + #[test] fn protect_grant_and_revoke_append_oplog_records() { let root = tmp_root("oplog-protect"); @@ -5572,6 +5955,7 @@ mod tests { None, None, &dst.sparse_spec().unwrap(), + None, "t", "msg", ) diff --git a/crates/repo/src/rewrap.rs b/crates/repo/src/rewrap.rs index 126857d2..1c57d11c 100644 --- a/crates/repo/src/rewrap.rs +++ b/crates/repo/src/rewrap.rs @@ -14,6 +14,9 @@ use crate::repo::Repo; pub struct RewrapReport { pub secrets_rewrapped: Vec, pub blobs_rewrapped: usize, + /// Convergent (pre-P33) protected blobs eagerly re-sealed randomized + /// (fresh DEK + nonce, new ciphertext id) during this run (P33 Task 10). + pub blobs_resealed: usize, /// (entry label e.g. "secret db-pass" / "path secret/db.txt", reason) pub skipped: Vec<(String, String)>, /// The commit id; None on --dry-run or when nothing needed rewrapping. @@ -24,10 +27,16 @@ impl Repo { /// Re-seal every secret (fresh DEK, current recipients + escrow) and /// replace every protected blob's wrap list (rule's granted set + escrow) /// at the tip, as ONE commit and ONE oplog record. Entries `identity` - /// cannot open are skipped and reported. Policy/registry-only: the root - /// tree id is untouched. Cuts the LIVE TIP only — history keeps old wraps - /// and old secret objects (content addressing; same boundary as rotation, - /// ADR-0019). + /// cannot open are skipped and reported. Still-convergent (pre-P33) + /// protected blobs are eagerly upgraded while we're here: decrypted with + /// the unwrapped DEK, re-sealed randomized (`encrypt_path_randomized`), + /// and the tree entry retargeted to the new ciphertext id with the + /// `RANDOMIZED` perms bit — so the root tree changes exactly when an + /// upgrade happened. Once every entry is randomized the run is + /// policy/registry-only again and the root tree id is untouched (the P17 + /// property). Cuts the LIVE TIP only — history keeps old wraps, old + /// convergent ciphertext, and old secret objects (content addressing; + /// same boundary as rotation, ADR-0019). pub fn rewrap( &self, identity: &SecretKey, @@ -121,19 +130,45 @@ impl Repo { new_secret_objs.push((name, Object::Secret(sealed))); } - // ---- Paths half: replace wrap lists with granted + escrow. ---- + // ---- Paths half: replace wrap lists with granted + escrow; eagerly + // upgrade still-convergent (pre-P33) blobs to randomized. ---- let mut protection = snap.protection.clone(); let mut blobs_rewrapped = 0usize; + let mut blobs_resealed = 0usize; + // path -> (new blob id, new perms) for eagerly-upgraded entries; the + // commit tail rebuilds the root iff this is non-empty. + let mut retargets: std::collections::BTreeMap = + std::collections::BTreeMap::new(); + // Old convergent blob ids some path upgraded FROM. Their wrap-map + // entries are removed only AFTER the loop, and only if no tree entry + // still references them (review Critical): two paths can share ONE + // convergent blob id (pre-P33 dedup of identical plaintext), and + // removing the shared entry inside the loop stripped the wraps out + // from under the second path — it was then skipped with "no wrapped + // DEKs recorded for blob" and left pointing at a wrap-orphaned blob + // at the tip: silent access loss no re-run could repair. + let mut upgraded_from: std::collections::BTreeSet = + std::collections::BTreeSet::new(); + // Main-tree protected cache: recording (path, plaintext, new id) for + // each upgraded path is what keeps the next `sc status`/`commit` + // quiet — rewrap never rematerializes the working tree, so the + // on-disk plaintext (and its stat) is untouched while its sealed id + // changes. Never opened on dry-run (no mutation, nothing to record). + let mut main_cache = if dry_run { + None + } else { + Some(self.open_protected_cache()?) + }; let entries = { let arc = self.store_arc(); let mut store = arc.lock().unwrap(); crate::worktree::tree_file_entries_with_perms(&mut store, snap.root)? }; - for (path, (blob_id, _mode, perms)) in entries { + for (path, (blob_id, _mode, perms)) in &entries { if perms & scl_core::PROTECTED == 0 { continue; } - let Some(rule) = crate::protect::matching_prefix(&protection, &path) else { + let Some(rule) = crate::protect::matching_prefix(&protection, path) else { skipped.push(( format!("path {path}"), "no governing rule (bit/rule mismatch)".into(), @@ -149,7 +184,7 @@ impl Repo { )); continue; } - let Some(wks) = protection.wrapped.get(&blob_id) else { + let Some(wks) = protection.wrapped.get(blob_id) else { skipped.push(( format!("path {path}"), "no wrapped DEKs recorded for blob".into(), @@ -171,15 +206,8 @@ impl Repo { continue; } }; - if dry_run { - blobs_rewrapped += 1; - continue; - } - // Rebuild the wrap list: exactly granted + escrow, reusing prior - // wrap bytes for recipients already present (id-stability), fresh - // wraps for the rest. Tombstoned/stale wraps are dropped. - let prior = wks.clone(); - let mut new_wks: Vec = Vec::new(); + // The target recipient set — exactly granted + escrow — shared + // by both arms below. let mut target_pks: Vec = granted.iter().map(|b| PublicKey::from_bytes(*b)).collect(); for e in escrows { @@ -190,6 +218,79 @@ impl Repo { target_pks.push(e.clone()); } } + if perms & scl_core::RANDOMIZED == 0 { + // Convergent (pre-P33): eager upgrade. Decrypt with the DEK + // we just unwrapped, re-seal randomized, retarget the tree + // entry + wraps to the fresh ciphertext id. + let cipher = { + let arc = self.store_arc(); + let obj = arc.lock().unwrap().get(blob_id)?; + match obj { + Object::Blob(b) => b.to_vec(), + _ => { + skipped + .push((format!("path {path}"), "tree entry is not a blob".into())); + continue; + } + } + }; + let pt = match scl_crypto::decrypt_path(&cipher, &dek) { + Ok(p) => p, + Err(e) => { + // Corruption-shaped, not an authorization failure + // (the wrap itself opened): named, not fatal. + skipped.push(( + format!("path {path}"), + format!("ciphertext failed to open: {e}"), + )); + continue; + } + }; + if dry_run { + blobs_resealed += 1; + continue; + } + let (new_cipher, new_dek) = scl_crypto::encrypt_path_randomized(&pt); + let new_id = { + let arc = self.store_arc(); + let i = arc.lock().unwrap().put(Object::blob(new_cipher))?; + i + }; + let mut new_wks: Vec = target_pks + .iter() + .map(|pk| scl_crypto::wrap_dek_for(&new_dek, pk)) + .collect(); + new_wks.sort_by(|a, b| a.recipient_id.cmp(&b.recipient_id)); + // The OLD id's wrap entry is NOT removed here — another tree + // path may share this convergent blob id and still needs the + // wraps to upgrade itself (or, if it skips, to stay + // decryptable). Deferred cleanup below the loop. + protection.wrapped.insert(new_id, new_wks); + upgraded_from.insert(*blob_id); + retargets.insert( + path.clone(), + (new_id, scl_core::PROTECTED | scl_core::RANDOMIZED), + ); + if let Some(c) = main_cache.as_mut() { + // Recorded unconditionally for upgraded paths: if the + // working file diverged locally, the keyed tag simply + // won't match its plaintext — a benign spurious reseal + // at the next commit, never incorrectness. + c.record(path, &pt, new_id); + } + blobs_resealed += 1; + continue; + } + if dry_run { + blobs_rewrapped += 1; + continue; + } + // Randomized: rebuild the wrap list only — exactly granted + + // escrow, reusing prior wrap bytes for recipients already + // present (id-stability), fresh wraps for the rest. + // Tombstoned/stale wraps are dropped. Ciphertext id unchanged. + let prior = wks.clone(); + let mut new_wks: Vec = Vec::new(); for pk in &target_pks { let rid = pk.recipient_id().to_string(); match prior.iter().find(|w| w.recipient_id == rid) { @@ -198,17 +299,39 @@ impl Repo { } } new_wks.sort_by(|a, b| a.recipient_id.cmp(&b.recipient_id)); - protection.wrapped.insert(blob_id, new_wks); + protection.wrapped.insert(*blob_id, new_wks); blobs_rewrapped += 1; } + // Deferred old-id wrap cleanup: drop an upgraded-from convergent id's + // wrap entry only when NO tree entry still points at it after the + // retargets are applied — a path that shared the blob but was skipped + // (no rule / not a recipient / …) keeps its wraps, so no entry is + // ever left wrap-orphaned at the tip. + if !upgraded_from.is_empty() { + let still_referenced: std::collections::BTreeSet = entries + .iter() + .filter(|(p, _)| !retargets.contains_key(p.as_str())) + .map(|(_, (id, _, _))| *id) + .collect(); + for old in &upgraded_from { + if !still_referenced.contains(old) { + protection.wrapped.remove(old); + } + } + } + // ---- Nothing to do / dry-run: report only. ---- if dry_run - || (secrets_rewrapped.is_empty() && new_secret_objs.is_empty() && blobs_rewrapped == 0) + || (secrets_rewrapped.is_empty() + && new_secret_objs.is_empty() + && blobs_rewrapped == 0 + && blobs_resealed == 0) { return Ok(RewrapReport { secrets_rewrapped, blobs_rewrapped, + blobs_resealed, skipped, commit: None, }); @@ -224,15 +347,52 @@ impl Repo { registry.insert(name.clone(), id); secrets_rewrapped.push(name); } + // Rebuild the root tree iff any convergent blob was re-sealed: walk + // the tip's entries again, swapping in each retargeted path's new + // ciphertext id + `RANDOMIZED` perms and keeping every other entry's + // bytes/mode/perms verbatim. With no retargets the root is untouched + // byte-for-byte (policy-only, the P17 property — a second rewrap + // converges back to tree-identical). + let new_root = if retargets.is_empty() { + snap.root + } else { + let mut all: Vec<(String, Vec, scl_core::FileMode, u8)> = + Vec::with_capacity(entries.len()); + { + let arc = self.store_arc(); + let mut store = arc.lock().unwrap(); + for (path, (blob_id, mode, perms)) in &entries { + let (id, p) = match retargets.get(path) { + Some((new_id, new_perms)) => (*new_id, *new_perms), + None => (*blob_id, *perms), + }; + let bytes = match store.get(&id)? { + Object::Blob(b) => b.to_vec(), + _ => return Err(Error::CorruptObject(id)), + }; + all.push((path.clone(), bytes, *mode, p)); + } + // Store lock dropped here: write_tree_with_perms locks it. + } + self.vfs.write_tree_with_perms(&all)? + }; let head = crate::refs::current_branch(self.layout())?; let before = crate::refs::read_branch_tip(self.layout(), &head)?; - let msg = format!( - "rewrap: {} secret(s), {} blob(s)", - secrets_rewrapped.len(), - blobs_rewrapped - ); - let id = - self.commit_snapshot(snap.root, vec![tip], registry, protection, "system", &msg)?; + let msg = if blobs_resealed > 0 { + format!( + "rewrap: {} secret(s), {} blob(s), {} re-sealed randomized", + secrets_rewrapped.len(), + blobs_rewrapped, + blobs_resealed + ) + } else { + format!( + "rewrap: {} secret(s), {} blob(s)", + secrets_rewrapped.len(), + blobs_rewrapped + ) + }; + let id = self.commit_snapshot(new_root, vec![tip], registry, protection, "system", &msg)?; crate::oplog::record( self.layout(), "rewrap", @@ -240,9 +400,18 @@ impl Repo { &head, &[(head.clone(), before, Some(id))], )?; + // Persist the cache only after the commit has landed (never `?` a + // save — cache trouble must not fail a rewrap that already + // succeeded; worst case is a spurious reseal next commit). + if !retargets.is_empty() { + if let Some(c) = &main_cache { + c.save_best_effort(); + } + } Ok(RewrapReport { secrets_rewrapped, blobs_rewrapped, + blobs_resealed, skipped, commit: Some(id), }) @@ -500,6 +669,312 @@ mod tests { std::fs::remove_dir_all(&root).unwrap(); } + // ---- P33 Task 10: eager convergent → randomized upgrade ---- + + use scl_core::{ + FileMode, Object, ObjectId, ProtectPrefix, Protection, RecipientEntry, RecipientState, + PROTECTED, RANDOMIZED, + }; + + /// Seed the tip with pre-P33 CONVERGENT protected entries under + /// `secret/`. The CLI can no longer mint convergent ciphertext (commit + /// seals randomized since Task 6), so this seals via the low-level + /// `encrypt_path` and writes the tree + protection registry directly. + /// Each entry: (path, plaintext, recipient the DEK is wrapped for, + /// wrap_wrong_dek — when true, the wrap holds a DEK that does NOT open + /// the ciphertext, simulating corruption the unwrap can't detect). + fn seed_convergent_tip( + repo: &Repo, + rule_pk: &scl_crypto::PublicKey, + entries: &[(&str, &[u8], &scl_crypto::PublicKey, bool)], + ) { + let mut prot = Protection::default(); + prot.prefixes.push(ProtectPrefix { + prefix: "secret/".into(), + recipients: vec![RecipientEntry { + key: rule_pk.to_bytes(), + epoch: 1, + state: RecipientState::Granted, + }], + }); + let mut files = Vec::new(); + for (path, pt, pk, wrap_wrong) in entries { + let (cipher, dek) = scl_crypto::encrypt_path(pt); + let id = Object::blob(cipher.clone()).id(); + let wrapped_dek = if *wrap_wrong { + let (_c, other) = scl_crypto::encrypt_path(b"a different plaintext entirely"); + other + } else { + dek + }; + // One wrap per recipient per blob id, as a real pre-P33 commit's + // reuse_prior_wraps produced — identical plaintext at two paths + // dedups to ONE shared blob id with ONE wrap list. + let wks = prot.wrapped.entry(id).or_default(); + let rid = pk.recipient_id().to_string(); + if !wks.iter().any(|w| w.recipient_id == rid) { + wks.push(scl_crypto::wrap_dek_for(&wrapped_dek, pk)); + } + files.push((path.to_string(), cipher, FileMode::FILE, PROTECTED)); + } + let root = repo.vfs.write_tree_with_perms(&files).unwrap(); + let parents = repo.head_tip().unwrap().into_iter().collect(); + repo.commit_snapshot( + root, + parents, + Default::default(), + prot, + "test", + "seed convergent", + ) + .unwrap(); + } + + /// The tip tree's (blob id, perms) at `path`. + fn entry_at(repo: &Repo, tip: ObjectId, path: &str) -> (ObjectId, u8) { + let snap = repo.snapshot(&tip).unwrap(); + let arc = repo.store_arc(); + let mut store = arc.lock().unwrap(); + let entries = crate::worktree::tree_file_entries_with_perms(&mut store, snap.root).unwrap(); + let (id, _mode, perms) = *entries.get(path).expect("entry present"); + (id, perms) + } + + /// Decrypt `path`'s ciphertext at `tip` via wrap presence (as `sc run`/ + /// `decrypt_with` do): unwrap the DEK with `sk`, open the blob. + fn decrypt_at(repo: &Repo, tip: ObjectId, path: &str, sk: &scl_crypto::SecretKey) -> Vec { + let snap = repo.snapshot(&tip).unwrap(); + let (id, _perms) = entry_at(repo, tip, path); + let wks = snap.protection.wrapped.get(&id).expect("wraps recorded"); + let me = sk.public().recipient_id().to_string(); + let wk = wks + .iter() + .find(|w| w.recipient_id == me) + .expect("identity has a wrap"); + let dek = scl_crypto::unwrap_dek_with(wk, sk).unwrap(); + let cipher = { + let arc = repo.store_arc(); + let obj = arc.lock().unwrap().get(&id).unwrap(); + match obj { + Object::Blob(b) => b.to_vec(), + _ => panic!("tree entry is not a blob"), + } + }; + scl_crypto::decrypt_path(&cipher, &dek).unwrap().to_vec() + } + + #[test] + fn rewrap_upgrades_convergent_blobs_and_second_run_is_policy_only() { + let root = tmp_root("upgrade"); + let repo = Repo::init(&root).unwrap(); + let (alice_sk, alice_pk) = scl_crypto::generate_keypair(); + seed_convergent_tip( + &repo, + &alice_pk, + &[("secret/x", b"top secret", &alice_pk, false)], + ); + let tip0 = repo.head_tip().unwrap().unwrap(); + let (old_id, old_perms) = entry_at(&repo, tip0, "secret/x"); + assert_eq!(old_perms, PROTECTED, "fixture must be convergent"); + let known = [alice_pk.clone()]; + + // Dry-run counts the upgrade without mutating anything. + let dry = repo.rewrap(&alice_sk, &[], &known, true).unwrap(); + assert_eq!(dry.blobs_resealed, 1, "dry-run still REPORTS the upgrade"); + assert!(dry.commit.is_none()); + assert_eq!(repo.head_tip().unwrap().unwrap(), tip0, "tip must not move"); + + let r1 = repo.rewrap(&alice_sk, &[], &known, false).unwrap(); + assert_eq!(r1.blobs_resealed, 1); + assert!(r1.skipped.is_empty()); + let tip1 = repo.head_tip().unwrap().unwrap(); + let (id1, perms1) = entry_at(&repo, tip1, "secret/x"); + assert!(perms1 & RANDOMIZED != 0, "entry must be flagged randomized"); + assert!(perms1 & PROTECTED != 0, "entry must stay protected"); + assert_ne!(id1, old_id, "randomized reseal mints a fresh ciphertext id"); + assert_eq!( + decrypt_at(&repo, tip1, "secret/x", &alice_sk), + b"top secret".to_vec() + ); + // Old convergent id's wrap entry is dropped from the live tip; the + // new id carries the fresh wraps. + let snap1 = repo.snapshot(&tip1).unwrap(); + assert!(!snap1.protection.wrapped.contains_key(&old_id)); + assert!(snap1.protection.wrapped.contains_key(&id1)); + + let r2 = repo.rewrap(&alice_sk, &[], &known, false).unwrap(); + assert_eq!(r2.blobs_resealed, 0, "second rewrap is policy-only again"); + if r2.commit.is_some() { + let tip2 = repo.head_tip().unwrap().unwrap(); + assert_eq!( + entry_at(&repo, tip2, "secret/x").0, + id1, + "ciphertext id stable" + ); + assert_eq!( + repo.snapshot(&tip2).unwrap().root, + snap1.root, + "root untouched byte-for-byte (P17 policy-only property)" + ); + } + drop(repo); + std::fs::remove_dir_all(&root).unwrap(); + } + + #[test] + fn rewrap_upgrades_two_paths_sharing_one_convergent_blob() { + // Review Critical: identical plaintext at two paths pre-P33 dedups + // to ONE convergent blob id shared by both tree entries. The old + // in-loop wrap removal stripped the shared entry after the FIRST + // path upgraded, so the second was skipped ("no wrapped DEKs") and + // left pointing at a wrap-orphaned blob — silent access loss no + // re-run could repair. Both paths must upgrade, decryptable, and no + // tree entry may reference a wrapless blob at the new tip. + let root = tmp_root("shared-blob"); + let repo = Repo::init(&root).unwrap(); + let (alice_sk, alice_pk) = scl_crypto::generate_keypair(); + seed_convergent_tip( + &repo, + &alice_pk, + &[ + ("secret/a", b"same plaintext", &alice_pk, false), + ("secret/b", b"same plaintext", &alice_pk, false), + ], + ); + let tip0 = repo.head_tip().unwrap().unwrap(); + let (old_a, _) = entry_at(&repo, tip0, "secret/a"); + let (old_b, _) = entry_at(&repo, tip0, "secret/b"); + assert_eq!(old_a, old_b, "fixture must share ONE convergent blob id"); + + let known = [alice_pk.clone()]; + let report = repo.rewrap(&alice_sk, &[], &known, false).unwrap(); + assert!(report.skipped.is_empty(), "got {:?}", report.skipped); + assert_eq!(report.blobs_resealed, 2, "both paths re-sealed"); + + let tip1 = repo.head_tip().unwrap().unwrap(); + let snap1 = repo.snapshot(&tip1).unwrap(); + let (id_a, perms_a) = entry_at(&repo, tip1, "secret/a"); + let (id_b, perms_b) = entry_at(&repo, tip1, "secret/b"); + assert!(perms_a & RANDOMIZED != 0 && perms_b & RANDOMIZED != 0); + assert_ne!( + id_a, id_b, + "per-path randomized reseal also closes the intra-tip equality oracle" + ); + assert_eq!( + decrypt_at(&repo, tip1, "secret/a", &alice_sk), + b"same plaintext".to_vec() + ); + assert_eq!( + decrypt_at(&repo, tip1, "secret/b", &alice_sk), + b"same plaintext".to_vec() + ); + // The shared old id is unreferenced after both retargets → removed. + assert!(!snap1.protection.wrapped.contains_key(&old_a)); + // Invariant the Critical violated: every PROTECTED tree entry's id + // has wraps at the tip — nothing is wrap-orphaned. + let arc = repo.store_arc(); + let entries = { + let mut store = arc.lock().unwrap(); + crate::worktree::tree_file_entries_with_perms(&mut store, snap1.root).unwrap() + }; + for (p, (id, _m, perms)) in &entries { + if perms & PROTECTED != 0 { + assert!( + snap1.protection.wrapped.contains_key(id), + "{p} points at a wrap-orphaned blob" + ); + } + } + drop(repo); + std::fs::remove_dir_all(&root).unwrap(); + } + + #[test] + fn undo_of_reseal_commit_restores_convergent_tip_without_spurious_status() { + // Minor (same review): `sc undo` of a reseal commit must restore the + // convergent tip and report a clean status — the convergent diff + // branch re-encrypts the on-disk plaintext keylessly, so it needs no + // cache entry, and the stale randomized cache tag is harmless. + let root = tmp_root("undo-reseal"); + let repo = Repo::init(&root).unwrap(); + let (alice_sk, alice_pk) = scl_crypto::generate_keypair(); + std::fs::create_dir_all(root.join("secret")).unwrap(); + std::fs::write(root.join("secret/x"), b"top secret").unwrap(); + seed_convergent_tip( + &repo, + &alice_pk, + &[("secret/x", b"top secret", &alice_pk, false)], + ); + let tip0 = repo.head_tip().unwrap().unwrap(); + let known = [alice_pk.clone()]; + + let r = repo.rewrap(&alice_sk, &[], &known, false).unwrap(); + assert_eq!(r.blobs_resealed, 1); + // Post-reseal, the cache keeps the untouched on-disk plaintext quiet. + let s = repo.status().unwrap(); + assert!(s.modified.is_empty(), "post-reseal: got {:?}", s.modified); + + repo.undo().unwrap(); + assert_eq!( + repo.head_tip().unwrap().unwrap(), + tip0, + "one undo reverts the whole reseal commit" + ); + let (_id, perms) = entry_at(&repo, tip0, "secret/x"); + assert_eq!(perms & RANDOMIZED, 0, "tip entry is convergent again"); + let s = repo.status().unwrap(); + assert!(s.modified.is_empty(), "post-undo: got {:?}", s.modified); + assert!(s.added.is_empty(), "post-undo: got {:?}", s.added); + assert!(s.deleted.is_empty(), "post-undo: got {:?}", s.deleted); + drop(repo); + std::fs::remove_dir_all(&root).unwrap(); + } + + #[test] + fn rewrap_skips_unopenable_convergent_blob_and_upgrades_the_rest() { + // Skip-and-report for the new decrypt step: a convergent blob whose + // wrap opens but whose ciphertext does NOT (corruption-shaped) lands + // in `skipped`, named; the openable one still upgrades; the run + // still commits (existing partial-success semantics). + let root = tmp_root("convergent-skip"); + let repo = Repo::init(&root).unwrap(); + let (alice_sk, alice_pk) = scl_crypto::generate_keypair(); + seed_convergent_tip( + &repo, + &alice_pk, + &[ + ("secret/good", b"ok", &alice_pk, false), + ("secret/bad", b"nope", &alice_pk, true), + ], + ); + let known = [alice_pk.clone()]; + let report = repo.rewrap(&alice_sk, &[], &known, false).unwrap(); + assert_eq!(report.blobs_resealed, 1, "the openable blob still upgrades"); + assert_eq!(report.skipped.len(), 1); + assert!(report.skipped[0].0.contains("secret/bad")); + assert!( + report.skipped[0].1.contains("ciphertext failed to open"), + "got {:?}", + report.skipped[0] + ); + assert!(report.commit.is_some(), "partial success still commits"); + let tip = repo.head_tip().unwrap().unwrap(); + let (_gid, gperms) = entry_at(&repo, tip, "secret/good"); + assert!(gperms & RANDOMIZED != 0); + // The unopenable entry is left exactly as it was: convergent, same + // id, wraps intact — nothing silently dropped. + let (bid, bperms) = entry_at(&repo, tip, "secret/bad"); + assert_eq!(bperms & RANDOMIZED, 0); + assert!(repo + .snapshot(&tip) + .unwrap() + .protection + .wrapped + .contains_key(&bid)); + drop(repo); + std::fs::remove_dir_all(&root).unwrap(); + } + #[test] fn rewrap_refuses_during_in_progress_merge() { // final-review I1: rewrap must not cut over while a conflicted diff --git a/crates/repo/src/sparse.rs b/crates/repo/src/sparse.rs index 466db137..86d94750 100644 --- a/crates/repo/src/sparse.rs +++ b/crates/repo/src/sparse.rs @@ -181,7 +181,8 @@ impl Repo { let snap = self.snapshot(&tip)?; let store_arc = self.vfs().store(); let mut s = store_arc.lock().unwrap(); - crate::worktree::materialize( + let mut cache = self.open_protected_cache()?; + let skipped = crate::worktree::materialize( self.layout(), &mut s, snap.root, @@ -189,7 +190,14 @@ impl Repo { &snap.protection, identity, &spec, - ) + Some(&mut cache), + )?; + // The sparse spec (the operation's durable commit point) is already + // persisted above; this re-lay moves no branch ref, so the cache save + // is best-effort — never abort a sparse change on cache trouble + // (P33 Task 7). + cache.save_best_effort(); + Ok(skipped) } /// Disable sparse checkout: clear the persisted spec and re-materialize @@ -238,7 +246,8 @@ impl Repo { let snap = self.snapshot(&tip)?; let store_arc = self.vfs().store(); let mut s = store_arc.lock().unwrap(); - crate::worktree::materialize( + let mut cache = self.open_protected_cache()?; + let skipped = crate::worktree::materialize( self.layout(), &mut s, snap.root, @@ -246,7 +255,12 @@ impl Repo { &snap.protection, identity, &Sparse::default(), - ) + Some(&mut cache), + )?; + // Best-effort: the cleared sparse spec is already persisted above and + // this re-lay moves no branch ref (P33 Task 7). + cache.save_best_effort(); + Ok(skipped) } } diff --git a/crates/repo/src/sync.rs b/crates/repo/src/sync.rs index 1036d61d..4eb3ded2 100644 --- a/crates/repo/src/sync.rs +++ b/crates/repo/src/sync.rs @@ -163,6 +163,7 @@ impl Repo { &head_protection, None, &sparse_spec, + None, )?; } Ok(dst_repo) diff --git a/crates/repo/src/workspace.rs b/crates/repo/src/workspace.rs index 9f33f885..a5b2cb46 100644 --- a/crates/repo/src/workspace.rs +++ b/crates/repo/src/workspace.rs @@ -42,12 +42,20 @@ pub(crate) fn materialize_workspace( dir: &Path, identity: Option<&scl_crypto::SecretKey>, sparse: &crate::sparse::Sparse, + cache: Option<&mut crate::cache::ProtectedCache>, ) -> Result> { std::fs::create_dir_all(dir)?; let snap = repo.snapshot(&tip)?; let ws = Layout::at(dir); let store_arc = repo.vfs().store(); let mut store = store_arc.lock().unwrap(); + // The per-workspace cache (P33 Task 7) is opened by the caller with its + // `root` set to THIS checkout dir and its file at `Layout::ws_cache_path` + // (beside the checkout dir, never inside it, so harvest's worktree read + // can't pick it up). `sc ws fork` passes a persistent cache; `sc work` + // passes an ephemeral one threaded fork->harvest. A `None` cache still + // works — a workspace's randomized protected path just re-seals on the + // next harvest instead of carrying (spurious-but-safe, per `cache.rs`). worktree::materialize( &ws, &mut store, @@ -56,6 +64,7 @@ pub(crate) fn materialize_workspace( &snap.protection, identity, sparse, + cache, ) } @@ -71,6 +80,7 @@ pub(crate) fn harvest_workspace( author: &str, message: &str, sparse: &crate::sparse::Sparse, + cache: Option<&mut crate::cache::ProtectedCache>, ) -> Result { let snap = repo.snapshot(&tip)?; let ws = Layout::at(dir); @@ -81,8 +91,20 @@ pub(crate) fn harvest_workspace( worktree::tree_file_ids(&mut store, snap.root)? .into_keys() .collect(); - let d = - worktree::diff_worktree(&ws, &mut store, Some(snap.root), &snap.protection, sparse)?; + // The workspace-local cache (P33 Task 7) proves an untouched randomized + // protected path unchanged, so it isn't spuriously reported modified + // here (which would send it into the fresh-seal path in `snapshot_files` + // below). Its `root` is this checkout dir, so its stat lookups resolve + // against the workspace's own files — unlike the host cache, which + // would stat the wrong tree. + let d = worktree::diff_worktree( + &ws, + &mut store, + Some(snap.root), + &snap.protection, + sparse, + cache.as_deref(), + )?; ( tracked, !(d.added.is_empty() && d.modified.is_empty() && d.deleted.is_empty()), @@ -97,6 +119,11 @@ pub(crate) fn harvest_workspace( // fixed at materialize time (or, for `sc work`, is always full), so the // carry predicate must see that same view, not the host repo's current // `.sc/sparse` — see `snapshot_files`'s doc comment. + // The workspace-local cache is threaded into the commit too (P33 Task 7): + // its `root` is this checkout dir, so a randomized protected path the + // workspace never touched carries the base's exact ciphertext blob id + // (proven unchanged by the cache) instead of re-sealing. Convergent priors + // still carry via content-only compare, no cache needed. match repo.snapshot_files( files, Some(tip), @@ -106,6 +133,7 @@ pub(crate) fn harvest_workspace( None, None, sparse, + cache, author, message, ) { @@ -263,25 +291,36 @@ impl Repo { // guarantees no early return can occur while children are running — // an interleaved failure would orphan live agents and race the // teardown guard's remove_dir_all underneath them. + // One EPHEMERAL protected-cache per temp checkout (P33 Task 7): `root` + // is the checkout dir, `path` is `None` so nothing persists (a `sc + // work` session leaves zero residue outside `.sc/`). Threaded from + // this materialize loop into the harvest loop below so an untouched + // randomized protected path carries the base blob instead of resealing. + let key = crate::cache::local_key(&self.layout)?; let mut dirs = Vec::with_capacity(labels.len()); + let mut caches = Vec::with_capacity(labels.len()); for label in &labels { let dir = session_root.join(label); + let mut cache = crate::cache::ProtectedCache::open(key, dir.clone(), None); let skipped = materialize_workspace( self, tip, &dir, opts.identity.as_ref(), &crate::sparse::Sparse::default(), + Some(&mut cache), )?; for path in &skipped { eprintln!("workspace {label}: skipped (no key): {path}"); } dirs.push(dir); + caches.push(cache); } - // Spawn all agents first (they run concurrently), then await each. + // Spawn all agents first (they run concurrently), then await each. The + // ephemeral cache travels alongside each (label, dir) into the harvest. let mut children = Vec::with_capacity(labels.len()); - for (label, dir) in labels.iter().zip(dirs) { + for ((label, dir), cache) in labels.iter().zip(dirs).zip(caches) { let mut c = std::process::Command::new(exe); c.args(args) .current_dir(&dir) @@ -290,11 +329,11 @@ impl Repo { for (k, v) in &secret_envs { c.env(k, v); } - children.push((label.clone(), dir, c.spawn())); + children.push((label.clone(), dir, cache, c.spawn())); } let mut outcomes = Vec::with_capacity(children.len()); - for (label, dir, spawn) in children { + for (label, dir, mut cache, spawn) in children { let agent_exit = match spawn { Ok(mut child) => child.wait().ok().and_then(|s| s.code()), Err(e) => { @@ -310,6 +349,7 @@ impl Repo { &opts.author, &message, &crate::sparse::Sparse::default(), + Some(&mut cache), ); outcomes.push(WorkspaceOutcome { label, @@ -374,9 +414,15 @@ mod tests { let repo = Repo::open(&root).unwrap(); let tip = repo.head_tip().unwrap().unwrap(); let dir = scratch.join("ws1"); - let skipped = - materialize_workspace(&repo, tip, &dir, None, &crate::sparse::Sparse::default()) - .unwrap(); + let skipped = materialize_workspace( + &repo, + tip, + &dir, + None, + &crate::sparse::Sparse::default(), + None, + ) + .unwrap(); assert!(skipped.is_empty()); assert_eq!( std::fs::read_to_string(dir.join("a.txt")).unwrap(), @@ -392,6 +438,7 @@ mod tests { "test", "msg", &crate::sparse::Sparse::default(), + None, ) .unwrap(); let id = match res { @@ -415,7 +462,15 @@ mod tests { let repo = Repo::open(&root).unwrap(); let tip = repo.head_tip().unwrap().unwrap(); let dir = scratch.join("ws1"); - materialize_workspace(&repo, tip, &dir, None, &crate::sparse::Sparse::default()).unwrap(); + materialize_workspace( + &repo, + tip, + &dir, + None, + &crate::sparse::Sparse::default(), + None, + ) + .unwrap(); let res = harvest_workspace( &repo, tip, @@ -424,6 +479,7 @@ mod tests { "test", "msg", &crate::sparse::Sparse::default(), + None, ) .unwrap(); assert!(matches!(res, HarvestResult::Unchanged)); @@ -441,7 +497,15 @@ mod tests { let repo = Repo::open(&root).unwrap(); let tip = repo.head_tip().unwrap().unwrap(); let dir = scratch.join("ws1"); - materialize_workspace(&repo, tip, &dir, None, &crate::sparse::Sparse::default()).unwrap(); + materialize_workspace( + &repo, + tip, + &dir, + None, + &crate::sparse::Sparse::default(), + None, + ) + .unwrap(); // An AWS-style key id trips the P5 pattern rules. std::fs::write(dir.join("leak.txt"), "AKIAIOSFODNN7EXAMPLE\n").unwrap(); let res = harvest_workspace( @@ -452,6 +516,7 @@ mod tests { "test", "msg", &crate::sparse::Sparse::default(), + None, ) .unwrap(); assert!(matches!(res, HarvestResult::Rejected(_))); diff --git a/crates/repo/src/worktree.rs b/crates/repo/src/worktree.rs index bdade9c7..07cabe6c 100644 --- a/crates/repo/src/worktree.rs +++ b/crates/repo/src/worktree.rs @@ -5,6 +5,7 @@ use std::path::Path; use scl_core::{EntryKind, FileMode, Object, ObjectId, Protection, Store, Tree, PROTECTED}; +use crate::cache::ProtectedCache; use crate::error::{Error, Result}; use crate::ignore::Ignore; use crate::layout::Layout; @@ -449,9 +450,16 @@ pub struct Diff { /// path: /// - absent on disk => CLEAN (the expected state for an unauthorized/skipped /// checkout — not a user deletion); -/// - present on disk => re-encrypt the disk bytes convergently (`encrypt_path` -/// is a pure, keyless function) and compare the resulting ciphertext blob id -/// to the stored one: equal => CLEAN, different => MODIFIED (a genuine edit). +/// - present on disk, format dispatches on the `RANDOMIZED` bit (P33): +/// - clear (convergent/legacy) => re-encrypt the disk bytes convergently +/// (`encrypt_path` is a pure, keyless function) and compare the +/// resulting ciphertext blob id to the stored one: equal => CLEAN, +/// different => MODIFIED (a genuine edit). Unchanged from pre-P33. +/// - set (randomized) => re-encryption proves nothing (a fresh seal never +/// matches the old ciphertext even for identical plaintext), so the only +/// public-key-free unchanged proof is the local stat cache +/// (`cache::unchanged`, Task 3): a hit reports CLEAN, a miss/disagreement +/// reports MODIFIED — spurious-but-safe, never incorrect. /// /// Non-protected paths use the usual plaintext-blob-id comparison. /// @@ -474,7 +482,7 @@ pub struct Diff { /// `disable_sparse`'s preflight), so this walk never touches a gapped /// out-of-filter object — an out-of-filter path was never materialized to /// disk either, so it's expected-absent, not a deletion. -pub fn diff_worktree( +pub(crate) fn diff_worktree( layout: &Layout, store: &mut Store, head_root: Option, @@ -483,6 +491,7 @@ pub fn diff_worktree( // a future richer diff (e.g. prefix-aware reporting) needs no signature change. _protection: &Protection, sparse: &Sparse, + cache: Option<&ProtectedCache>, ) -> Result { let head = match head_root { Some(root) => tree_file_entries_with_perms_sparse(store, root, sparse)?, @@ -499,8 +508,24 @@ pub fn diff_worktree( None => diff.added.push(p.clone()), Some((hid, _mode, perms)) => { let disk_id = if perms & PROTECTED != 0 { - // Convergent re-encryption yields the same id as the commit did. - Object::blob(scl_crypto::encrypt_path(bytes).0).id() + if perms & scl_core::RANDOMIZED == 0 { + // Convergent (legacy) entry: re-encryption yields the same + // id as the commit did — still exact, no cache needed. + Object::blob(scl_crypto::encrypt_path(bytes).0).id() + } else { + // Randomized entry (P33): re-encrypting proves nothing. The + // stat cache (stat hit, then keyed-tag fallback) is the only + // public-key-free unchanged proof; a missing entry reports + // modified — spurious-but-safe, never incorrect. The cache + // resolves the abs stat path against its own checkout root. + match cache.and_then(|c| c.unchanged(p, bytes)) { + Some(id) => id, + None => { + diff.modified.push(p.clone()); + continue; + } + } + } } else { Object::blob(bytes.clone()).id() }; @@ -570,7 +595,7 @@ pub(crate) fn safe_join(root: &Path, rel: &str) -> Result { /// paths `sparse` no longer matches. This closes the crash a bare /// `set_sparse`/`disable_sparse` narrowing (or a `switch`) used to hit on /// any partial clone with a genuine out-of-filter gap elsewhere in the tree. -pub fn materialize( +pub(crate) fn materialize( layout: &Layout, store: &mut Store, target_root: ObjectId, @@ -578,6 +603,7 @@ pub fn materialize( protection: &Protection, identity: Option<&scl_crypto::SecretKey>, sparse: &Sparse, + mut cache: Option<&mut ProtectedCache>, ) -> Result> { // Gap-tolerant when sparse is active (P27 Task 4): a promisor-filtered // clone never fetched out-of-filter subtrees at all, so an unfiltered @@ -637,6 +663,19 @@ pub fn materialize( std::fs::create_dir_all(parent)?; } std::fs::write(&full, &pt[..])?; + // Record a randomized entry's cache entry right after the + // write (P33): without this, the next `diff_worktree`/ + // `status` on this exact, unmodified plaintext has no + // stat-hit or keyed-tag to match and spuriously reports + // it as modified — since re-encryption can't prove + // unchanged-ness for a randomized seal. Mirrors the + // record call `commit`'s carry/fresh-seal paths already + // make (`repo.rs`'s `snapshot_files`). + if perms & scl_core::RANDOMIZED != 0 { + if let Some(c) = cache.as_deref_mut() { + c.record(path, &pt, *blob_id); + } + } // `pt` (Zeroizing>) is dropped and zeroed here. } None => { @@ -744,6 +783,7 @@ mod tests { &Default::default(), None, &Sparse::default(), + None, ); // Restore perms before asserting so cleanup always works. @@ -782,6 +822,7 @@ mod tests { &Default::default(), None, &Sparse::default(), + None, ) .unwrap_err(); assert!(matches!(err, crate::error::Error::BadRef(_)), "got {err:?}"); @@ -810,11 +851,161 @@ mod tests { None, &Default::default(), None, - &Sparse::default() + &Sparse::default(), + None ) .is_err()); drop(store); std::fs::remove_dir_all(&layout.root).unwrap(); } + + #[test] + fn diff_dispatches_randomized_entries_through_the_cache() { + let (layout, mut store) = tmp_objects("p33-diff"); + // Build a HEAD tree with one RANDOMIZED protected entry whose plaintext + // sits on disk, exactly as an authorized checkout leaves it. + let pt = b"top secret".to_vec(); + let (cipher, _dek) = scl_crypto::encrypt_path_randomized(&pt); + let blob = Object::blob(cipher.clone()); + let blob_id = blob.id(); + store.put(blob).unwrap(); + let inner = store + .put(Object::Tree(Tree::new(vec![TreeEntry { + name: "x".into(), + kind: EntryKind::Blob, + id: blob_id, + mode: FileMode::FILE, + perms: PROTECTED | scl_core::RANDOMIZED, + }]))) + .unwrap(); + let root = store + .put(Object::Tree(Tree::new(vec![TreeEntry { + name: "secret".into(), + kind: EntryKind::Tree, + id: inner, + mode: FileMode::FILE, + perms: 0, + }]))) + .unwrap(); + + let abs = layout.root.join("secret/x"); + std::fs::create_dir_all(abs.parent().unwrap()).unwrap(); + std::fs::write(&abs, &pt).unwrap(); + + let key = crate::cache::local_key(&layout).unwrap(); + let mut cache = crate::cache::ProtectedCache::open(key, layout.root.clone(), None); + + // No cache entry: spurious-but-safe MODIFIED. + let d = diff_worktree( + &layout, + &mut store, + Some(root), + &Protection::default(), + &Sparse::default(), + Some(&cache), + ) + .unwrap(); + assert_eq!(d.modified, vec!["secret/x"]); + + // Recorded: provably clean. + cache.record("secret/x", &pt, blob_id); + let d = diff_worktree( + &layout, + &mut store, + Some(root), + &Protection::default(), + &Sparse::default(), + Some(&cache), + ) + .unwrap(); + assert!(d.modified.is_empty() && d.added.is_empty() && d.deleted.is_empty()); + + // Genuine edit: modified again (tag mismatch). + std::fs::write(&abs, b"edited").unwrap(); + let d = diff_worktree( + &layout, + &mut store, + Some(root), + &Protection::default(), + &Sparse::default(), + Some(&cache), + ) + .unwrap(); + assert_eq!(d.modified, vec!["secret/x"]); + + drop(store); + std::fs::remove_dir_all(&layout.root).unwrap(); + assert!(!layout.root.exists()); + } + + #[test] + fn materialize_populates_the_cache_for_decrypted_files() { + // P33 Task 7 Step 1 pin: materializing a RANDOMIZED protected entry + // for an authorized identity must record the decrypted plaintext into + // the threaded cache, so the very next `unchanged` on the untouched + // on-disk plaintext proves it clean (a randomized seal can't be proven + // unchanged by re-encrypting). + let (layout, mut store) = tmp_objects("p33-mat-cache"); + let (sk, pk) = scl_crypto::generate_keypair(); + let pt = b"top secret".to_vec(); + let (cipher, dek) = scl_crypto::encrypt_path_randomized(&pt); + let blob = Object::blob(cipher); + let blob_id = blob.id(); + store.put(blob).unwrap(); + let inner = store + .put(Object::Tree(Tree::new(vec![TreeEntry { + name: "x".into(), + kind: EntryKind::Blob, + id: blob_id, + mode: FileMode::FILE, + perms: PROTECTED | scl_core::RANDOMIZED, + }]))) + .unwrap(); + let root = store + .put(Object::Tree(Tree::new(vec![TreeEntry { + name: "secret".into(), + kind: EntryKind::Tree, + id: inner, + mode: FileMode::FILE, + perms: 0, + }]))) + .unwrap(); + + let mut wrapped = BTreeMap::new(); + wrapped.insert(blob_id, vec![scl_crypto::wrap_dek_for(&dek, &pk)]); + let prot = Protection { + prefixes: Vec::new(), + wrapped, + }; + + let key = crate::cache::local_key(&layout).unwrap(); + let mut cache = crate::cache::ProtectedCache::open(key, layout.root.clone(), None); + let skipped = materialize( + &layout, + &mut store, + root, + None, + &prot, + Some(&sk), + &Sparse::default(), + Some(&mut cache), + ) + .unwrap(); + assert!( + skipped.is_empty(), + "authorized identity decrypts, nothing skipped" + ); + assert_eq!( + std::fs::read(layout.root.join("secret/x")).unwrap(), + pt, + "plaintext materialized on disk" + ); + // The cache now proves the untouched plaintext unchanged. + assert_eq!(cache.unchanged("secret/x", &pt), Some(blob_id)); + + drop(store); + std::fs::remove_dir_all(&layout.root).unwrap(); + assert!(!layout.root.exists()); + } } diff --git a/crates/repo/src/ws.rs b/crates/repo/src/ws.rs index bee1231e..3990ff8b 100644 --- a/crates/repo/src/ws.rs +++ b/crates/repo/src/ws.rs @@ -232,18 +232,37 @@ impl Repo { // No manifest proves any .sc/ws content is crash residue from a // killed fork — always safe to clear before materializing. let _ = std::fs::remove_dir_all(&root); + // Per-workspace P33 cache key (host's `.sc/local-key`), minted once. + let key = crate::cache::local_key(self.layout())?; let mut workspaces = Vec::with_capacity(agents as usize); for i in 1..=agents { let dir = root.join(i.to_string()); - if let Err(e) = - crate::workspace::materialize_workspace(self, tip, &dir, identity, &sparse) - { + // Persistent per-workspace cache (P33 Task 7): `root` is THIS + // checkout dir, file at `.sc/ws/cache-` (beside the checkout, + // under `ws_dir`, so the whole-session teardown removes it too). + // Records every randomized protected path decrypted at fork so a + // later `sc ws harvest` (a separate invocation) carries the base + // blob instead of re-sealing an untouched file. + let mut cache = crate::cache::ProtectedCache::open( + key, + dir.clone(), + Some(self.layout().ws_cache_path(i as usize)), + ); + if let Err(e) = crate::workspace::materialize_workspace( + self, + tip, + &dir, + identity, + &sparse, + Some(&mut cache), + ) { // Nothing announced yet (no manifest written) — tear down // whatever partial checkouts exist so a failed fork leaves // no residue under .sc/ws. let _ = std::fs::remove_dir_all(&root); return Err(e); } + cache.save_best_effort(); workspaces.push(WsEntry { index: i, dir, @@ -297,8 +316,30 @@ impl Repo { let sparse = crate::sparse::Sparse::new(session.sparse.clone()); let store_arc = self.vfs().store(); let mut store = store_arc.lock().unwrap(); - let d = - worktree::diff_worktree(&ws, &mut store, Some(base.root), &base.protection, &sparse)?; + // The workspace's own persistent P33 cache, read-only (this is a + // status/list/preflight query, never a harvest, so nothing is + // recorded or saved back) — same open-call shape as `ws_harvest` + // (below), rooted at the workspace's own checkout dir so an + // untouched RANDOMIZED protected path is recognized as unchanged + // instead of always reporting modified. Degrades to `None` (never + // errors) if the host's cache key can't be loaded/minted: this must + // stay a non-fatal query path, and a lost cache only ever costs the + // stat/tag short-circuit, never correctness. + let cache = crate::cache::local_key(self.layout()).ok().map(|key| { + crate::cache::ProtectedCache::open( + key, + entry.dir.clone(), + Some(self.layout().ws_cache_path(entry.index as usize)), + ) + }); + let d = worktree::diff_worktree( + &ws, + &mut store, + Some(base.root), + &base.protection, + &sparse, + cache.as_ref(), + )?; Ok(!(d.added.is_empty() && d.modified.is_empty() && d.deleted.is_empty())) } @@ -365,6 +406,9 @@ impl Repo { .ok_or_else(|| Error::InvalidArgument(format!("no such workspace: {i}")))?; if entry.live { let _ = std::fs::remove_dir_all(&entry.dir); + // Drop the workspace's P33 stat cache too (sibling file, + // not under the checkout dir) — P33 Task 7 item 3. + let _ = std::fs::remove_file(self.layout().ws_cache_path(entry.index as usize)); entry.live = false; } } @@ -613,6 +657,8 @@ impl Repo { } } + // Per-workspace P33 cache key (host's `.sc/local-key`), minted once. + let key = crate::cache::local_key(self.layout())?; let mut outcomes = Vec::with_capacity(live_indices.len()); for i in live_indices { let dir = session @@ -657,7 +703,19 @@ impl Repo { // `ws_changed_for` above, now threaded into the commit carry too // via `harvest_workspace` -> `snapshot_files`. let fork_time_sparse = crate::sparse::Sparse::new(session.sparse.clone()); - match crate::workspace::harvest_workspace( + // The workspace's persistent P33 cache (`root` = this checkout dir, + // file at `.sc/ws/cache-`), threaded through the harvest so an + // untouched randomized protected path carries the base blob instead + // of re-sealing. Saved best-effort right after harvest: if the entry + // resolves (teardown removes the cache file too), the save is + // immediately undone; if it stays live (Rejected/FallbackBranch), + // the next harvest gets a warm cache. + let mut cache = crate::cache::ProtectedCache::open( + key, + dir.clone(), + Some(self.layout().ws_cache_path(i as usize)), + ); + let harvested = crate::workspace::harvest_workspace( self, session.base_snapshot, &dir, @@ -665,7 +723,10 @@ impl Repo { author, &msg, &fork_time_sparse, - )? { + Some(&mut cache), + )?; + cache.save_best_effort(); + match harvested { crate::workspace::HarvestResult::Rejected(report) => { // DESIGN DECISION (spec precision note, Task 5): a // scanner-rejected workspace stays LIVE so the offending @@ -681,11 +742,13 @@ impl Repo { }); } crate::workspace::HarvestResult::Unchanged => { - // `ws_changed` already confirmed a diff, but - // `harvest_workspace` re-diffs independently against its - // own `tip` argument (session.base_snapshot, same value - // here) — so this arm is unreachable in practice, kept - // only so the match is exhaustive without a panic. + // Unreachable in practice: `ws_changed_for` already + // confirmed a diff above, and `harvest_workspace` + // re-diffs moments later against the SAME persisted + // cache state (same `tip` argument, same per-workspace + // cache, no intervening write) — the two checks cannot + // disagree. Kept for match exhaustiveness, not as a + // reachable panic guard. resolve_and_teardown(self.layout(), &mut session, i, &dir, None)?; outcomes.push(WsHarvestOutcome::Unchanged { index: i }); } @@ -814,6 +877,10 @@ fn resolve_and_teardown( // io error with no recorded recovery path). write_manifest(layout, session)?; let _ = std::fs::remove_dir_all(dir); + // Drop this workspace's P33 stat cache alongside its checkout dir (P33 + // Task 7 item 3): the cache file lives beside the dir, not inside it, so + // removing the dir doesn't remove it. + let _ = std::fs::remove_file(layout.ws_cache_path(index as usize)); Ok(()) } @@ -1170,6 +1237,122 @@ mod tests { std::fs::remove_dir_all(&root).unwrap(); } + #[test] + fn untouched_protected_files_do_not_conflict_across_workspaces() { + // P33 Task 7 Step 4 — the false-conflict hazard the per-workspace + // cache exists to prevent. Two workspaces forked WITH identity, each + // editing a different PLAIN file, both leaving a RANDOMIZED protected + // file untouched. Both must land cleanly (no `work-` fallback) AND + // carry the base's EXACT protected blob id. Without a per-workspace + // cache, harvest would re-seal the untouched randomized path to a + // fresh ciphertext id (a randomized seal differs every time), which + // the merge would then adopt as a spurious "change". + let root = tmp_root("ws-untouched-protected"); + let repo = init(&root); + std::fs::write(root.join("p1.txt"), "one\n").unwrap(); + std::fs::write(root.join("p2.txt"), "two\n").unwrap(); + let (sk, pk) = scl_crypto::generate_keypair(); + repo.protect("secret/", &[pk], None).unwrap(); + std::fs::create_dir_all(root.join("secret")).unwrap(); + std::fs::write(root.join("secret/x"), "the secret\n").unwrap(); + repo.commit("t", "add plain + protected").unwrap(); + let base = repo.head_tip().unwrap().unwrap(); + + // Base's protected blob id, and PROOF it is RANDOMIZED — otherwise a + // convergent carry-by-re-encrypt would pass this test without ever + // touching the cache (the whole point of the regression). + let base_secret_id = { + let a = repo.vfs().store(); + let mut s = a.lock().unwrap(); + let base_root = s.get_snapshot(&base).unwrap().root; + let entries = worktree::tree_file_entries_with_perms(&mut s, base_root).unwrap(); + let (id, _mode, perms) = *entries.get("secret/x").expect("protected file present"); + assert!( + perms & scl_core::PROTECTED != 0 && perms & scl_core::RANDOMIZED != 0, + "the protected file must be RANDOMIZED for this regression to exercise the cache" + ); + id + }; + + // Fork two workspaces WITH identity: the protected file materializes + // decrypted AND is recorded in each workspace's own cache at fork. + let session = repo.ws_fork(2, "t", Some(&sk)).unwrap(); + let dir1 = session.workspaces[0].dir.clone(); + let dir2 = session.workspaces[1].dir.clone(); + // Each workspace edits ONLY its own plain file; secret/x is untouched. + std::fs::write(dir1.join("p1.txt"), "one edited\n").unwrap(); + std::fs::write(dir2.join("p2.txt"), "two edited\n").unwrap(); + + let outcomes = repo.ws_harvest(None, "t", Some(&sk)).unwrap(); + assert_eq!(outcomes.len(), 2); + for o in &outcomes { + let merged_tip = match o { + WsHarvestOutcome::Landed { merged_tip, .. } => *merged_tip, + other => panic!("expected Landed (no work- fallback), got {other:?}"), + }; + let a = repo.vfs().store(); + let mut s = a.lock().unwrap(); + let mroot = s.get_snapshot(&merged_tip).unwrap().root; + let entries = worktree::tree_file_entries_with_perms(&mut s, mroot).unwrap(); + let (id, _m, _p) = *entries.get("secret/x").unwrap(); + assert_eq!( + id, base_secret_id, + "an untouched randomized protected file must carry the base blob id, not re-seal" + ); + } + + drop(repo); + std::fs::remove_dir_all(&root).unwrap(); + } + + #[test] + fn ws_changed_for_ignores_untouched_randomized_protected_file() { + // P33 Task 7 (ws_changed_for arm): before this fix, `ws_changed_for` + // always passed `None` for the cache, so a RANDOMIZED protected + // path — never bit-for-bit reproducible by re-encrypting the same + // plaintext — always compared as "different" even when genuinely + // untouched, reporting the workspace "changed" for no real edit. + // With the workspace's own persistent cache threaded in (recorded + // at fork time, same as `harvest_workspace` uses), an untouched + // randomized protected file must resolve as unchanged. + let root = tmp_root("ws-changed-for-randomized"); + let repo = init(&root); + let (sk, pk) = scl_crypto::generate_keypair(); + repo.protect("secret/", &[pk], None).unwrap(); + std::fs::create_dir_all(root.join("secret")).unwrap(); + std::fs::write(root.join("secret/x"), "the secret\n").unwrap(); + repo.commit("t", "add protected").unwrap(); + let base = repo.head_tip().unwrap().unwrap(); + + // Confirm RANDOMIZED, mirroring the sibling harvest-level regression + // — otherwise this test would pass even without the fix. + { + let a = repo.vfs().store(); + let mut s = a.lock().unwrap(); + let base_root = s.get_snapshot(&base).unwrap().root; + let entries = worktree::tree_file_entries_with_perms(&mut s, base_root).unwrap(); + let (_id, _mode, perms) = *entries.get("secret/x").expect("protected file present"); + assert!( + perms & scl_core::PROTECTED != 0 && perms & scl_core::RANDOMIZED != 0, + "the protected file must be RANDOMIZED for this regression to exercise the cache" + ); + } + + // Fork WITH identity so the protected file materializes decrypted + // and is recorded in the workspace's own cache at fork time; leave + // it completely untouched. + let session = repo.ws_fork(1, "t", Some(&sk)).unwrap(); + assert!( + !repo + .ws_changed_for(&session, &session.workspaces[0]) + .unwrap(), + "an untouched RANDOMIZED protected file must not report the workspace as changed" + ); + + drop(repo); + std::fs::remove_dir_all(&root).unwrap(); + } + #[test] fn gc_roots_ws_base_snapshot() { let root = tmp_root("gcroot"); diff --git a/demo/run_randomized_demo.sh b/demo/run_randomized_demo.sh new file mode 100755 index 00000000..67765eaa --- /dev/null +++ b/demo/run_randomized_demo.sh @@ -0,0 +1,259 @@ +#!/usr/bin/env bash +# P33 demo: randomized protected-path encryption. Proves that the oracle +# ADR-0014 accepted (equality-confirmable protected content: same plaintext +# -> same ciphertext id) is CLOSED for everything sealed from this phase on, +# that untouched protected files stay quiet across unrelated commits (the +# local stat/keyed-hash cache carries them instead of re-sealing), that +# identical independent edits on two branches now require --identity to +# merge (accepted cost 4a — a real UX regression vs. the old id-fast-path, +# even though the merge itself completes cleanly once given the key), and +# that `sc rewrap` on an already-fully-randomized tip is a true no-op +# policy-wise (no re-seal line, no new tree/blob objects — only a fresh +# commit record). +# +# Scope split (deliberate, stated plainly): this script drives the CURRENT +# binary only, which — since Task 6 flipped `encrypt_protected` to always +# randomize — can no longer mint a convergent (pre-P33) seal at all. So this +# demo proves oracle-closure, quiet history, the accepted-cost-4a merge +# behavior, and rewrap's policy-only idempotence on an all-randomized tip +# (steps 1, 2, 3, and the "second run reports nothing to upgrade" half of +# the rewrap-upgrade story). Dual-read of a GENUINELY convergent (pre-P33) +# store, and `sc rewrap`'s eager convergent->randomized upgrade path itself +# (the "re-sealed N convergent blob(s)" line firing), are pinned instead by +# unit tests built with the library directly (crates/repo/src/rewrap.rs, +# crates/repo/src/{merge,replay}.rs) — Tasks 4-6, 9, 10 all build convergent +# fixtures that way, which is impossible to reproduce honestly with a CLI +# that can only ever mint randomized ciphertext. +# +# Technique note: there is no `sc cat-file`/`sc ls-tree` exposing a path's +# blob id, so ciphertext-id (in)equality is proven indirectly but rigorously +# via the content-addressed object store's file count: every object is one +# file named by its own hash, so "how many NEW object files appeared" is an +# exact count of how many genuinely new (blob/tree/snapshot) objects a +# commit created. Two operations of identical *shape* (e.g. "add one new +# flat file") cost identical object-count deltas whenever their content is +# equally novel to the store — so comparing deltas across a controlled pair +# of same-shape commits proves id (in)equality without ever needing to read +# a specific id. +# +# Self-checking: every claim is an assertion; any failure exits non-zero +# before the RESULT line. +set -euo pipefail + +cd "$(dirname "$0")/.." +cargo build --quiet --bin sc +SC="$PWD/target/debug/sc" + +tmp_base="${TMPDIR:-/tmp}" +tmp_base="${tmp_base%/}" + +fail() { echo "FAIL: $1"; exit 1; } + +# Count loose objects in the CAS. Called only before `sc gc` ever runs in +# this script, so every object stays loose (one file per object) and the +# count is an exact reachable-object tally. +objcount() { find .sc/objects -type f | wc -l | tr -d ' '; } + +run_demo() { + local run_label="$1" + local work + work=$(mktemp -d "$tmp_base/sc-randomized-demo.XXXXXX") + trap 'rm -rf "$work"' RETURN + + local keys="$work/keys" + mkdir -p "$keys" + local repo="$work/repo" + mkdir -p "$repo" + + # Keyless by construction (the P15/P23 demos' pattern): point SC_IDENTITY + # at a path that can never exist, so every command below is keyless unless + # it passes an explicit --identity. + export SC_IDENTITY="$work/no-such-identity" + + local alice_out alice_pk + alice_out=$("$SC" keygen --out "$keys/alice") + alice_pk=$(echo "$alice_out" | grep 'public key' | awk '{print $3}') + + # NB: the RETURN trap above removes $work (and therefore $repo, our new + # cwd) on function exit, so every path used below this point must stay + # absolute — a relative path minted here would resolve against a + # directory that's already gone by the time it's read. + cd "$repo" + "$SC" init >/dev/null + printf '[recipients]\nalice = "%s"\n' "$alice_pk" > .sc/recipients.toml + "$SC" protect secret/ --to alice >/dev/null + mkdir -p secret + + echo + echo "=== [$run_label] 1a: oracle closed — same plaintext, two different protected paths ===" + echo "repeatable-plaintext-1" > secret/a.txt + "$SC" commit -m "base: secret/a.txt" --author demo >/dev/null + local c_a + c_a=$(objcount) + + # ADD secret/b.txt with the SAME plaintext as secret/a.txt — same "shape" + # of change (one new flat file under secret/) as the next commit below. + echo "repeatable-plaintext-1" > secret/b.txt + "$SC" commit -m "secret/b.txt: same plaintext as a.txt" --author demo >/dev/null + local c_b delta_same + c_b=$(objcount) + delta_same=$((c_b - c_a)) + + # ADD secret/c.txt with a genuinely NOVEL plaintext — identical shape + # (one new flat file under secret/), the "definitely a fresh blob" baseline. + echo "genuinely-novel-plaintext-2" > secret/c.txt + "$SC" commit -m "secret/c.txt: novel plaintext" --author demo >/dev/null + local c_c delta_diff + c_c=$(objcount) + delta_diff=$((c_c - c_b)) + + [ "$delta_same" -eq "$delta_diff" ] \ + || fail "adding secret/b.txt (repeat of a.txt's plaintext) cost $delta_same new object(s), but adding secret/c.txt (novel plaintext) cost $delta_diff — a convergent seal would have cost LESS for the repeat (blob dedup), so the oracle would still be open" + [ "$delta_same" -ge 3 ] || fail "expected at least 3 new objects (blob+tree+snapshot) for a one-file add, got $delta_same" + echo "secret/b.txt (repeat of a.txt's plaintext) cost $delta_same new object(s), identical to secret/c.txt's (genuinely novel) $delta_diff — a fresh, distinct ciphertext blob was minted for the repeat ✔" + + echo + echo "=== [$run_label] 1b: oracle closed — same plaintext, edited away then back on ONE path ===" + echo "edit-value-one" > secret/e.txt + "$SC" commit -m "base: secret/e.txt = one" --author demo >/dev/null + local c_e0 + c_e0=$(objcount) + + echo "edit-value-two" > secret/e.txt + "$SC" commit -m "secret/e.txt: edited away" --author demo >/dev/null + local c_e1 delta_away + c_e1=$(objcount) + delta_away=$((c_e1 - c_e0)) + + echo "edit-value-one" > secret/e.txt + "$SC" commit -m "secret/e.txt: edited back to its original value" --author demo >/dev/null + local c_e2 delta_back + c_e2=$(objcount) + delta_back=$((c_e2 - c_e1)) + + [ "$delta_away" -eq "$delta_back" ] \ + || fail "editing secret/e.txt away cost $delta_away new object(s), editing it back to the SAME original plaintext cost $delta_back — a convergent seal would have reused the original commit's blob for the 'back' edit (fewer new objects), so the oracle would still be open across commits" + echo "editing secret/e.txt away ($delta_away new object(s)) and back to its ORIGINAL plaintext ($delta_back) cost the same — the 'back' edit minted a fresh ciphertext blob rather than reconverging on the first commit's blob ✔" + + echo + echo "=== [$run_label] 2: quiet history — an unrelated commit doesn't touch untouched protected files ===" + local c_before_unrelated + c_before_unrelated=$(objcount) + # Nested one level deep (like secret/c.txt) so the object-count "shape" of + # this add matches 1a's delta_diff baseline exactly: root tree + one new + # subtree + snapshot + blob = 4. A flat root-level add would cost one + # fewer object (no subtree) and make the comparison apples-to-oranges. + mkdir -p misc + echo "unrelated noise" > misc/note.txt + "$SC" commit -m "unrelated: add misc/note.txt" --author demo >/dev/null + local c_after_unrelated delta_unrelated + c_after_unrelated=$(objcount) + delta_unrelated=$((c_after_unrelated - c_before_unrelated)) + + # Same shape as 1a's "add one new subtree with one novel-content flat file" + # baseline (delta_diff): if it were EQUAL, this commit cost exactly as much + # as adding one new file in one new subdirectory and nothing else touched. + # If secret/{a,b,c,e}.txt had been needlessly re-encrypted too, this delta + # would be strictly larger (an extra new secret/ subtree object plus one + # new blob per re-sealed file). + [ "$delta_unrelated" -eq "$delta_diff" ] \ + || fail "an unrelated top-level commit cost $delta_unrelated new object(s), vs. $delta_diff for adding one novel flat file alone — the extra objects mean an untouched protected file was needlessly re-sealed (cache-carry regression)" + echo "unrelated commit (new subdir + 1 file) cost exactly $delta_unrelated new object(s), matching the 1a novel-add baseline — no protected file's ciphertext id changed (cache carried them all) ✔" + + local status_out + status_out=$("$SC" status) + case "$status_out" in *"clean"*) ;; *) fail "sc status must stay clean after an unrelated commit, got: $status_out" ;; esac + echo "sc status: clean ✔" + + echo + echo "=== [$run_label] 3: accepted cost 4a — identical independent edits on two branches now need --identity ===" + echo "shared-base-value" > secret/x.txt + "$SC" commit -m "base: secret/x.txt" --author demo >/dev/null + "$SC" branch feature >/dev/null + + echo "identical-new-value" > secret/x.txt + "$SC" commit -m "main: edit secret/x.txt" --author demo >/dev/null + + "$SC" switch feature --identity "$keys/alice" >/dev/null + echo "identical-new-value" > secret/x.txt + "$SC" commit -m "feature: edit secret/x.txt to the SAME value" --author demo >/dev/null + + "$SC" switch main --identity "$keys/alice" >/dev/null + + set +e + no_id_out=$("$SC" merge feature --author demo 2>&1) + no_id_rc=$? + set -e + [ "$no_id_rc" -eq 1 ] || fail "merging identical independent edits without --identity must exit 1 (accepted cost 4a), got rc=$no_id_rc: $no_id_out" + case "$no_id_out" in *"--identity"*) ;; *) fail "expected a --identity hint in the refusal, got: $no_id_out" ;; esac + echo "sc merge feature (no --identity) refuses even though both sides wrote the SAME plaintext — pre-P33 this would have id-fast-pathed with no key needed at all ✔" + + status_out=$("$SC" status) + case "$status_out" in *"clean"*) ;; *) fail "a refused (not attempted) merge must leave the working tree clean, got: $status_out" ;; esac + echo "refusal left refs/working tree untouched (no MERGE_HEAD, no markers) ✔" + + local merge_out + merge_out=$("$SC" merge feature --identity "$keys/alice" --author demo) + case "$merge_out" in *"merged feature"*) ;; *) fail "merge with --identity must succeed cleanly, got: $merge_out" ;; esac + echo "sc merge feature --identity alice completes cleanly — no conflict markers (diff3 of identical plaintext has nothing to conflict over) ✔" + + local top_line + top_line=$("$SC" log | head -1) + case "$top_line" in *"(merge)"*) ;; *) fail "expected a merge marker on the tip, got: $top_line" ;; esac + + "$SC" switch main --identity "$keys/alice" >/dev/null + [ "$(cat secret/x.txt)" = "identical-new-value" ] || fail "post-merge secret/x.txt must decrypt to the agreed value" + echo "merged tip decrypts to the agreed value, re-encrypted through the ordinary commit path ✔" + + echo + echo "=== [$run_label] 4: rewrap on an all-randomized tip is policy-only (no re-seal line) ===" + local c_before_rewrap + c_before_rewrap=$(objcount) + local rewrap_out + rewrap_out=$("$SC" rewrap --identity "$keys/alice") + echo "$rewrap_out" + case "$rewrap_out" in *"re-sealed"*"convergent blob"*) fail "rewrap on an all-randomized tip must NOT print a convergent re-seal line, got: $rewrap_out" ;; esac + case "$rewrap_out" in *"rewrapped"*"protected blob(s)"*) ;; *) fail "expected a rewrapped-blobs summary line, got: $rewrap_out" ;; esac + local c_after_rewrap delta_rewrap + c_after_rewrap=$(objcount) + delta_rewrap=$((c_after_rewrap - c_before_rewrap)) + # Nothing to upgrade: the root tree and every blob are byte-identical + # (the P17 policy-only property), so the ONLY new object is the fresh + # commit snapshot itself — no new tree, no new blob. + [ "$delta_rewrap" -eq 1 ] \ + || fail "rewrap on an all-randomized tip must add exactly 1 new object (the commit snapshot only — root/blobs unchanged), got $delta_rewrap" + echo "rewrap added exactly 1 new object (the commit record) — root tree and every blob stayed byte-identical ✔" + + "$SC" switch main --identity "$keys/alice" >/dev/null + [ "$(cat secret/a.txt)" = "repeatable-plaintext-1" ] || fail "content must still decrypt correctly after the policy-only rewrap" + [ "$(cat secret/x.txt)" = "identical-new-value" ] || fail "merged content must still decrypt correctly after the policy-only rewrap" + echo "all protected content still decrypts correctly after rewrap ✔" + + echo + echo "=== [$run_label] 5: zero residue — .sc/tmp is empty, nothing lives outside \$work ===" + [ -d "$repo/.sc" ] || fail "repo/.sc must exist while the repo is alive" + if [ -d "$repo/.sc/tmp" ] && [ -n "$(ls -A "$repo/.sc/tmp" 2>/dev/null)" ]; then + fail ".sc/tmp is not empty after the run" + fi + echo "no .sc/tmp residue ✔" +} + +run_demo "run 1" +run_demo "run 2 (repeatable, not a one-shot fluke)" + +echo +echo "RESULT: the equality-confirmation oracle is closed for randomized-write" +echo "protected content — same plaintext at two paths, and the same plaintext" +echo "edited away and back on one path, both mint a fresh distinct ciphertext" +echo "blob (proven via matched object-count deltas, since the CLI exposes no" +echo "cat-file); an unrelated commit costs exactly the baseline object delta," +echo "meaning every untouched protected file was cache-carried, not re-sealed," +echo "and sc status stays clean; two branches writing the IDENTICAL plaintext" +echo "to a protected path now refuse to merge without --identity (accepted" +echo "cost 4a) but merge cleanly (no markers) once given the key; and sc" +echo "rewrap on an already-fully-randomized tip is genuinely policy-only (no" +echo "re-seal line, exactly one new object — the commit record). Dual-read of" +echo "a genuinely convergent (pre-P33) store and rewrap's eager upgrade path" +echo "itself are pinned by unit tests instead, since this binary can no" +echo "longer mint a convergent seal to fixture one honestly. Verified across" +echo "two independent runs, zero residue." diff --git a/docs/THREAT-MODEL.md b/docs/THREAT-MODEL.md index 7a80f4d8..96ae7689 100644 --- a/docs/THREAT-MODEL.md +++ b/docs/THREAT-MODEL.md @@ -46,17 +46,34 @@ too, not just the src-control-side metadata. revoke` is metadata-only. Real security requires rotating the underlying external credential. -## Protected paths (`sc protect`) — ADR-0014/0026/0027 +## Protected paths (`sc protect`) — ADR-0014/0026/0027/0043 - **Defends:** read-confidentiality of designated file content for a chosen recipient set; an unauthorized clone gets ciphertext it cannot read. -- **Does NOT defend — equality is confirmable:** protected paths use *convergent - encryption* (DEK and nonce derive from `BLAKE3(plaintext)`), so identical - plaintext dedups to identical ciphertext. An observer of the ciphertext can - therefore confirm that two protected files are identical, or that a protected - file matches a **guessed** plaintext. This is a deliberate tradeoff for dedup, - documented in ADR-0014. Low-entropy secrets (API keys, `.env`) belong in - `sc secret`, not `sc protect` — `sc protect` prints a nudge to that effect. +- **Equality-confirmation oracle — closed for new content (P33, ADR-0043); + pre-P33 convergent ciphertext in history stays confirmable forever + (rotation ≠ erasure, ADR-0019).** Protected paths *used to* + use *convergent encryption* (DEK and nonce derive from `BLAKE3(plaintext)`), + so identical plaintext dedups to identical ciphertext and an observer could + confirm that two protected files are identical, or that a protected file + matches a **guessed** plaintext. As of P33, **all content sealed from this + phase on uses a fresh random DEK + random nonce** — two seals of the same + plaintext yield different ciphertext ids, closing the oracle. Convergent + ciphertext already written into history stays readable (dual-read) and + remains equality-confirmable **forever** to anyone holding a clone — this is + the same rotation ≠ erasure boundary ADR-0019 already names for secrets: + content addressing means the old convergent object stays reachable in + history regardless of what happens at the tip. `sc rewrap` re-seals a + still-convergent blob randomized **at the live tip**, which stops that + plaintext's convergent form from propagating into *future* snapshots built + on top — it does not erase the historical convergent object, which any clone + that already fetched it (or fetches it later) can still equality-confirm + against. Real cutover of guessable content means changing the content (or + underlying credential) itself, not just re-sealing at the tip. Low-entropy + secrets (API keys, `.env`) still belong in `sc secret`, not `sc protect` — + `sc protect` prints a nudge to that effect, and the P28 rationale is + unchanged (governed secret lifecycle beats a protected file even with + randomized sealing). - **Revocation is durable but not retroactive:** `sc revoke` survives merges of pre-revoke branches (revocation tombstones, ADR-0026), so a revoked recipient never seals a *fresh* DEK again. But a recipient who already held a wrap can diff --git a/docs/adr/0014-per-file-permissions-encrypted-paths.md b/docs/adr/0014-per-file-permissions-encrypted-paths.md index a654bf7a..b3173e58 100644 --- a/docs/adr/0014-per-file-permissions-encrypted-paths.md +++ b/docs/adr/0014-per-file-permissions-encrypted-paths.md @@ -56,7 +56,13 @@ makes the same plaintext encrypt to different ciphertext every commit — new /id is a deterministic function of the plaintext). This is the standard trade-off for encrypted content-addressed stores and is acceptable for source content; it is documented so users with low-entropy secret files choose Phase 2 secrets - instead. + instead. **Superseded for newly sealed content by ADR-0043 (P33):** all + content sealed from P33 on uses a fresh random DEK + nonce, closing this + oracle; convergent dual-read is retained, so the caveat remains true forever + for pre-P33 ciphertext in history — `sc rewrap` re-seals the live tip, which + stops the convergent form propagating into future snapshots, but the + historical convergent objects stay reachable and confirmable (rotation ≠ + erasure, ADR-0019). - Reuses `scl-crypto` and the recipient model wholesale; no new crypto primitives. - The protection policy is a snapshot-level object like the secrets registry, so it versions and travels with the code. diff --git a/docs/adr/0043-randomized-protected-encryption.md b/docs/adr/0043-randomized-protected-encryption.md new file mode 100644 index 00000000..513ed912 --- /dev/null +++ b/docs/adr/0043-randomized-protected-encryption.md @@ -0,0 +1,287 @@ +# ADR-0043: Randomized protected-path encryption (dual-read, randomized-write) + +- **Status:** Accepted +- **Date:** 2026-07-10 +- **Phase:** 33 +- **Builds on:** ADR-0014 (P7 convergent protected paths — superseded *for + newly sealed content only*, dual-read retained), ADR-0025 (P15 carry-by-id + discipline the format bit rides through merge/replay), ADR-0027 (P17 + `sc rewrap` — the vehicle for the eager tip upgrade), ADR-0034 (P24 + perms-preserving carry predicate) +- **Issue:** #40 (part of the #24 audit-response map; decisions locked in #30) +- **Spec:** `docs/superpowers/specs/2026-07-10-p33-randomized-protected-encryption-design.md` + +## Context + +ADR-0014's protected paths use **convergent** encryption: the DEK and nonce +derive from `BLAKE3(plaintext)`, so identical plaintext seals to an identical +ciphertext blob id. That preserves the project's bedrock content-addressing +and dedup invariant, but it carries an accepted caveat (ADR-0014, consequence +"Accepted caveat of convergent encryption"): an observer who already holds a +candidate plaintext can confirm whether it is present in the repo, or that two +protected files are identical, from the ciphertext id alone — a +dictionary/equality-confirmation oracle. The 2026-07-09 security audit (ticket +#24's map) flagged this as the remaining protected-path confidentiality +boundary. ADR-0014 itself named the fix — "random per-encryption (fresh DEK + +nonce each time)… may be offered as an opt-in per-policy mode" — and deferred +it as the cost of dedup. P28's `sc protect` low-entropy nudge surfaced the +boundary to operators but did not close it. + +## Decision + +All **new** protected content is sealed under a **fresh random DEK + random +nonce**, so two seals of the same plaintext yield different ciphertext ids — +the oracle is closed for everything sealed from P33 on. Convergent ciphertext +already in history stays readable forever (**dual-read**); nothing is rewritten +or forcibly migrated, and no operator is forced through an upgrade. `sc rewrap` +gains an **eager, on-demand** upgrade of the live tip for operators who want +the old ciphertext gone from the current snapshot. + +### The format change: a perms-byte flag, not a blob-layout change + +The blob layout is **unchanged** — a randomized protected blob is still +`nonce(24) ‖ AEAD-ciphertext` under the same `PATH_AAD` (`b"scl-path-v1"`), +decrypted by the same `decrypt_path`. The only new bit is a format **tag**: +`RANDOMIZED` (`0b0000_0010`, `crates/core/src/object.rs`), always set together +with `PROTECTED` on a newly sealed `TreeEntry.perms` byte. `encrypt_path_randomized` +(`crates/crypto/src/envelope.rs`, plus an `_with_rng` variant for deterministic +tests) is the one new seal primitive; `encrypt_path` (convergent) survives only +for the legacy-format unchanged-comparison described below — no new seal call +site uses it. The tag lives in the **tree entry, not the blob**, for two +reasons: (1) format identification never requires fetching or decrypting blob +bytes — `status`, `diff`, commit, merge, and rewrap all read the perms byte +they already hold; and (2) putting a format discriminator *inside* the blob +would change the blob's bytes and therefore its id, reintroducing exactly the +determinism the change exists to break. The RustCrypto quarantine holds — only +`crates/crypto` grew crypto code; `crates/repo` links `blake3` directly (already +a workspace dependency) for the cache tag below. + +### Dual-read posture: no tag bump, no migration, format-dispatched commit + +There is **no snapshot-tag bump** (`TAG_SNAPSHOT` stays at P16's value 4) and +**no `PROTOCOL_VERSION` change** (stays 3). A pre-P33 store decodes with zero +`RANDOMIZED` bits and behaves byte-for-byte as it did before — a genuine +pre-P33 store reads identically, pinned by library-built convergent-tree +fixtures in the `rewrap`/`merge`/`replay` unit tests. Carry-by-id (the ADR-0025 +P15 discipline, generalized by ADR-0034 to preserve the *source* entry's own +perms) rides the format bit through merge, replay, graft, and sparse carry with +no new plumbing. + +Commit's per-protected-path rule is **format-dispatched, not cache-only** — the +load-bearing choice that prevents a silent mass-migration commit on first use +after upgrade. Dispatched on the *prior tip entry's* format: + +| Prior tip entry | Rule | +|---|---| +| **Convergent** (no `RANDOMIZED` bit) | Convergent re-encrypt-and-compare. Unchanged → carry as-is (stays convergent, history stays quiet, no cache needed). Edited → seal randomized. | +| **Randomized** | Consult the unchanged-detection cache (below); a hit carries the cached id, a miss seals randomized. | +| **New protected path** | Seal randomized. | + +Had dispatch been cache-only — "no cache entry ⇒ re-seal" — the first commit +after upgrading to a P33 binary would have found an empty cache and re-sealed +**every** unchanged protected file randomized, a mass-migration commit nobody +asked for. Format dispatch means unchanged convergent content stays convergent +**forever** until an edit or an explicit `sc rewrap` touches it; the store +migrates only what actually changes. + +### Unchanged detection: a keyed-PRF cache that does not reintroduce the oracle + +Commit still needs only **public keys** — it never decrypts to decide whether a +file changed. Detection is a per-checkout stat cache (`crates/repo/src/cache.rs`): + +- `.sc/local-key`: 32 random bytes, mode **0600 from first byte** (via + `OpenOptions::create_new` + `mode(0o600)`, so the file is never briefly + world-readable between creation and permission-setting), created lazily, + **never committed, never transferred**. The per-checkout cache file itself + (below) is written via `atomic_write_durable`. +- Per-checkout cache files: `.sc/protected-cache` for the main working tree; + `.sc/ws/cache-` per ws workspace (see below). Each entry is + `path → (mtime, size, keyed_tag, blob_id)` where + `keyed_tag = BLAKE3-keyed(local_key, plaintext)`. + +**The cache does not reintroduce the equality oracle** precisely because the +tag is a PRF output under a key that never leaves `.sc/` and never travels: the +cache file alone leaks nothing — two identical plaintexts have identical tags +only to a holder of the local key, who by construction already has the working +tree. **Degradation rule:** a lost, stale, or corrupt cache produces spurious +re-seals (fresh randomized ids for unchanged plaintext) — never incorrectness, +never a wrong tree. A corrupt cache line is treated as absent with a stderr +warning; a `.sc/local-key` that cannot be created is a hard error (the cache +cannot be safely keyed without it). + +### Accepted costs (from #40, all landed as predicted) + +- **(4a) Independent identical edits on two branches now conflict.** Under + convergent encryption, both sides editing a protected path to the *same* + plaintext produced the same ciphertext id and merged with an id-level fast + path. Randomized seals give the two edits different ids, so the merge now + conflicts **by construction — even when the plaintexts are trivially + identical** and needs `--identity` (or P23 `sc conflicts`/`sc resolve`) to + resolve, at which point diff3 merges the identical plaintexts cleanly and + re-seals the result randomized. +- **(4b) `sc rewrap` is no longer policy-only / tree-identical when it + upgrades.** A rewrap that re-seals any convergent blob randomized changes + that blob's id and therefore the root tree — the commit is content-changing, + not the byte-identical policy-only commit ADR-0027 described. A second rewrap + over an all-randomized tip converges back to policy-only (wrap-replacement + only, ids unchanged). +- **(4c) Identical plaintext at two paths no longer dedups.** Two protected + paths with byte-identical plaintext seal to two different randomized blob ids + — the intended cost of closing the intra-tip equality oracle between them. + +### P15 merge / P17 rewrap semantic adjustments + +- **P15 (ADR-0025):** unchanged and one-side-changed protected fast paths + survive unchanged (carry-by-id). Both-sides-edited paths now conflict by + construction (cost 4a). The content-merge re-encryption site in `merge.rs` + and replay's `encrypt_protected` path switch to the randomized primitive. + Protection-rule union, wrap union, and revocation-tombstone semantics + (P15/P16) are unchanged; mixed convergent+randomized trees need no special + casing — every path is either carried by id or re-sealed randomized. +- **P17 (ADR-0027):** `sc rewrap` performs the eager upgrade with no new + command or flag. While unwrapping each protected blob's DEK by wrap presence + (as today), it additionally decrypts any still-**convergent** blob and + re-seals it randomized, replaces the wrap list with the rule's + `granted_keys() + escrow`, and records the plaintext it already holds into + the main-tree cache (so the next `sc status`/`commit` stays quiet — rewrap + never rematerializes the working tree). Output gains a `re-sealed N + convergent blob(s)` line. Already-randomized blobs get wrap-replacement only. + One command, one commit, one oplog record, skip-and-report semantics + unchanged; sealing to an empty granted set still fails loudly. + +### Unlocked follow-on: rotate-for-paths (recorded, not built) + +ADR-0019 declared per-path DEK rotation "security-meaningless" under convergent +encryption — the DEK is a deterministic function of the plaintext, so +"rotating" it is either dedup-breaking or a no-op. **That objection dissolves +for randomized content:** a randomized DEK is independent of the plaintext, so +re-sealing a protected path under a fresh DEK is a genuine cryptographic +cutover, exactly as `sc secret rotate` already is for secrets. A +`sc protect … rotate`-style surface is now a coherent follow-on, recorded here +and in `ROADMAP.md`, **not built** in this phase. + +## Consequences + +- The convergent-equality oracle (ADR-0014) is closed for all content sealed + from P33 on; pre-P33 convergent ciphertext already in history stays + equality-confirmable **forever** to anyone holding a clone — the same + rotation ≠ erasure boundary ADR-0019 names for secrets, since content + addressing keeps the old object reachable regardless of what happens at the + tip. `sc rewrap` stops a still-convergent blob's plaintext from propagating + into *future* snapshots at the live tip; it does not erase the historical + convergent object. Real cutover of guessable content means changing the + content (or underlying credential) itself. THREAT-MODEL and ARCHITECTURE are + updated to state this split honestly. +- Zero new dependencies: `blake3`/`hex` were already workspace deps; `scl-repo` + now links `blake3` directly for the keyed cache tag. +- Old binaries reading a new store mask-check `PROTECTED` and would misreport + randomized entries as modified in `status` — accepted (no deployed old peers, + same posture as prior format-touching phases). +- The proof splits by concern, recorded honestly: dual-read of genuine pre-P33 + stores is pinned by **library-built convergent-tree fixtures** in the + `rewrap`/`merge`/`replay` unit tests (the current binary can no longer *write* + a convergent blob, so the fixtures build them directly via `encrypt_path`), + while `demo/run_randomized_demo.sh` proves oracle-closure (same plaintext → + different ciphertext ids), quiet history, cost-4a conflicts, and `sc rewrap` + policy-only idempotence on an all-randomized tip against the current binary — + run twice, zero residue. The convergent→randomized rewrap **upgrade** path + (the `re-sealed N convergent blob(s)` line firing) is unit-pinned rather than + demo-pinned, for the same reason the dual-read fixtures are library-built: the + current binary cannot mint a convergent blob to feed a demo. + +## Alternatives considered + +- **Cache-only dispatch (no format tag).** Rejected: an empty cache after a + binary upgrade would mass-migrate every unchanged protected file to + randomized on the first commit. The perms-byte format tag is what makes + "unchanged convergent content stays convergent" hold. See Decision §Dual-read. +- **A snapshot-tag bump / forced migration.** Rejected: dual-read needs zero + read-path changes and lets pre-P33 stores decode byte-for-byte identically; a + forced migration would rewrite history and break dedup for every existing + protected repo with no consent. +- **Per-prefix convergent/randomized choice.** Rejected on #30 — one format for + all new content, upgraded in bulk by `sc rewrap`, is simpler than a per-rule + mode the merge/replay/graft paths would each have to branch on. +- **A separate `sc rewrap --randomize` flag or a new upgrade command.** + Rejected: `sc rewrap` already holds each protected plaintext while re-wrapping + and already commits one policy cutover; folding the re-seal into it needs no + new surface. + +## Refinements discovered during the build + +Every prior phase's Refinements section holds this one to the same bar: every +claim below is checked against the shipped code, not the plan. + +1. **The keyed tag is authoritative in every cache lookup — a deliberate + deviation from #30 decision-3's letter AND the spec §3 table.** Both #30 and + the spec described a stat-hit shortcut (mtime+size match ⇒ carry the cached + id without hashing). The shipped `ProtectedCache::unchanged` + (`crates/repo/src/cache.rs`) **never trusts the stat alone** — the recorded + `(mtime, size)` fields are advisory, and the keyed BLAKE3 tag is recomputed + and compared on every lookup. This closes the classic git "racy-stat" data- + loss class: a same-size edit that lands within the filesystem's mtime + granularity, with the recorded mtime, would otherwise be misread as unchanged + and silently dropped from the commit (pinned by + `racy_stat_same_size_same_mtime_still_misses` and + `tag_is_authoritative_for_hits_and_misses`). The stat shortcut was traded + away because the plaintext is already in memory at every call site (commit, + status, diff all hold the working-tree bytes to seal or compare), keyed + hashing is ~free against that, and "never incorrectness" outranks the stat + shortcut. Stat fields are still recorded for a future fast-path that can + afford the risk; today nothing relies on them. + +2. **`sc protect --to ` no longer wraps UNCHANGED content + at the next commit — an accepted semantic adjustment.** Under convergent + encryption, re-protecting an already-protected prefix for a new recipient + caused the next commit's re-encrypt-and-compare to re-derive each unchanged + file's convergent DEK and wrap it for the new recipient. Under P33, unchanged + protected files carry their **prior wrap list verbatim** (carry-by-id), so a + bare `sc protect … --to ` does not grant the new recipient access to + files that are not re-sealed. This fails **safe** — it under-grants, never + over-grants — and `sc grant --to --identity` (re-wraps every + affected file's existing DEK) and `sc rewrap` (bulk cutover) are the covering + flows. Operator guidance: use `sc grant`, not a second `sc protect`, to add a + recipient to already-committed protected content. + +3. **Rewrap's shared-blob upgrade: deferred old-id wrap removal (review + Critical).** Two protected paths can share ONE convergent blob id (pre-P33 + dedup of identical plaintext at two paths). An earlier draft removed the old + blob's wrap-map entry **inside** the per-path upgrade loop; when the first + path re-sealed and stripped the shared entry, the second path was then found + with "no wrapped DEKs recorded for blob," skipped, and left pointing at a + wrap-orphaned blob at the tip — silent access loss no re-run could repair + (`crates/repo/src/rewrap.rs`). The fix **defers** old-id wrap removal past + the path loop: an old id's wraps are removed only once no kept tree entry + still references it. Each shared path mints its **own** randomized blob, which + also closes the intra-tip equality oracle between those two paths (cost 4c + applied intra-tip). Pinned by the shared-blob rewrap regression test + asserting `blobs_resealed == 2` and two distinct randomized ids. + +4. **The cache is per-checkout, and ws workspaces get their own — a false- + conflict hazard, not a nicety.** The main tree uses `.sc/protected-cache`; a + ws workspace uses `.sc/ws/cache-`, placed **beside** the checkout dir, + never inside `.sc/ws//` (a file inside the checkout would be read back as + an untracked working file at harvest and land in the branch), and removed + with the workspace at teardown; `sc work`'s one-shot ephemeral agents use an + **in-memory** cache (`ProtectedCache::open(..., None)`, never touches disk). + Without per-workspace caches, an untouched protected file in a forked + workspace has no cache entry, so harvest's format-dispatched detection would + re-seal it randomized — giving it a different id from the base and turning + every untouched protected file into an identical-edit **conflict** at harvest + (the cost-4a mechanism, fired spuriously), with a `work-` fallback instead + of a clean auto-merge. The per-workspace cache reproduces the base tree id on + an untouched file, preserving P20's clean-auto-merge and crash-recovery + `UpToDate` idempotency. + +5. **A cache save never gates an operation, and the ref move stays the commit + point.** This is not "every save is strictly ordered after every ref move" + — `assemble_completion_snapshot` deliberately saves the cache *before* its + callers move the ref (documented in-code; benign, since a crash between the + two just costs a spurious re-seal next time, not a lost edit) and several + call sites move no ref at all. The property that actually holds everywhere: + a cache save is best-effort (a save failure warns on stderr, never errors) + and never blocks or gates the operation it's attached to, while the ref + move — wherever one occurs — remains the sole atomic commit point, unchanged + from every prior phase. Combined with the 0600-from-first-byte local key and + the keyed-PRF tag, this is why the cache is pure optimization — its worst + failure mode is a spurious re-seal, never a lost edit or a leaked oracle. diff --git a/docs/adr/README.md b/docs/adr/README.md index 975d580c..d650ec36 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -53,6 +53,7 @@ supersedes the old one and update the **Status** line of the superseded record. | [0040](0040-sc-http-access-control.md) | sc+http access control (read-only, fail-closed bind, bearer tokens) | Accepted | 29 | | [0041](0041-listener-resource-limits.md) | Listener resource limits (connection cap, session timeout, spool cap, accept backoff) | Accepted | 31 | | [0042](0042-in-binary-tls-sc-https.md) | In-binary TLS — `sc+https://` via rustls | Accepted | 32 | +| [0043](0043-randomized-protected-encryption.md) | Randomized protected-path encryption (dual-read, randomized-write) | Accepted | 33 | See [`ROADMAP.md`](../../ROADMAP.md) for the phase sequence. ADR numbers are assigned in creation order, so 0017 — the secret scanner — sequences as phase P5, diff --git a/docs/superpowers/plans/2026-07-10-p33-randomized-protected-encryption.md b/docs/superpowers/plans/2026-07-10-p33-randomized-protected-encryption.md new file mode 100644 index 00000000..05aa9662 --- /dev/null +++ b/docs/superpowers/plans/2026-07-10-p33-randomized-protected-encryption.md @@ -0,0 +1,1142 @@ +# P33: Randomized Protected-Path Encryption — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Close the convergent-equality oracle (ADR-0014) — all new protected content seals under a fresh random DEK + nonce, old convergent ciphertext stays readable forever, and a per-checkout stat cache keeps unchanged content from re-sealing. + +**Architecture:** A `RANDOMIZED` perms bit tags the new format at the tree-entry level; the blob layout (`nonce(24) ‖ ct`) and `decrypt_path` are unchanged, so dual-read needs zero read-path code. The single production seal choke point (`protect::encrypt_protected`) flips to random seals, and a format-dispatched carry decision in `snapshot_files`/`diff_worktree` keeps unchanged content quiet: convergent priors use the existing re-encrypt-and-compare trick, randomized priors use a `.sc`-local stat cache + BLAKE3 keyed-tag fallback. `sc rewrap` eagerly upgrades convergent blobs at the tip. + +**Tech Stack:** Rust (stable, edition 2021). No new dependencies — `blake3` and `hex` are already workspace deps of `scl-repo`; randomness reaches `crates/repo` via `scl_crypto` re-exports (RustCrypto quarantine holds). + +**Spec:** `docs/superpowers/specs/2026-07-10-p33-randomized-protected-encryption-design.md` (read it first). Issue #40; decisions locked on #30. + +## Global Constraints + +- **RustCrypto quarantine:** all crypto code stays in `crates/crypto`. `crates/repo` may use `blake3` (already a dep, cf. `serve_tokens.rs`) and `scl_crypto::*` re-exports only. +- **No format break:** no snapshot-tag bump. A pre-P33 store must decode and behave byte-for-byte as today (zero `RANDOMIZED` bits ⇒ all-convergent behavior). +- **Commit needs only public keys.** Decrypt-and-compare is forbidden in commit/status/diff. +- **Degradation rule:** lost/stale/corrupt cache ⇒ spurious re-seals, never incorrectness, never a hard error. +- **Green at every task boundary:** `cargo test` passes after each task. Task order is load-bearing — the dispatch machinery (Tasks 4–5) must land before the seal flip (Task 6), or every commit would mass-re-seal. +- **Tests:** colocated `#[cfg(test)] mod tests`; disk-touching tests clean up and assert the path is gone. Reuse each file's existing test helpers (e.g. `worktree.rs`'s `tmp_objects`, `repo.rs`'s repo-builder helpers) instead of inventing parallel ones. +- **Errors:** per-crate `thiserror` enums; convert with `?`. +- Commit messages end with: + `Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB` + +--- + +### Task 1: `RANDOMIZED` perms bit in `scl-core` + +**Files:** +- Modify: `crates/core/src/object.rs:23` (beside `PROTECTED`) +- Modify: `crates/core/src/lib.rs:29` (re-export) + +**Interfaces:** +- Produces: `scl_core::RANDOMIZED: u8 = 0b0000_0010`. Every later task tests format with `perms & RANDOMIZED != 0` and mints new seals as `PROTECTED | RANDOMIZED`. + +- [ ] **Step 1: Write the failing test** in `crates/core/src/object.rs`'s existing `mod tests`: + +```rust +#[test] +fn randomized_bit_is_distinct_from_protected() { + assert_eq!(PROTECTED & RANDOMIZED, 0, "flags must not overlap"); + assert_ne!(RANDOMIZED, 0); + // A randomized entry is always also protected. + let perms = PROTECTED | RANDOMIZED; + assert!(perms & PROTECTED != 0 && perms & RANDOMIZED != 0); +} +``` + +- [ ] **Step 2: Run to verify it fails** + +Run: `cargo test -p scl-core randomized_bit -- --nocapture` +Expected: FAIL — `cannot find value RANDOMIZED`. + +- [ ] **Step 3: Implement** — in `crates/core/src/object.rs`, directly below `pub const PROTECTED: u8 = 0b0000_0001;`: + +```rust +/// Perms flag: this PROTECTED entry was sealed with a fresh random DEK+nonce +/// (P33) rather than convergently. Always set together with `PROTECTED`. +/// Format identification for dual-read lives here, in the tree entry, so no +/// caller ever needs to fetch blob bytes to know the seal format. +pub const RANDOMIZED: u8 = 0b0000_0010; +``` + +Add `RANDOMIZED` to the `pub use` list in `crates/core/src/lib.rs:29`. + +- [ ] **Step 4: Run to verify it passes** + +Run: `cargo test -p scl-core` +Expected: PASS (all). + +- [ ] **Step 5: Commit** + +```bash +git add crates/core/src/object.rs crates/core/src/lib.rs +git commit -m "P33: add RANDOMIZED perms flag to scl-core" +``` + +--- + +### Task 2: `encrypt_path_randomized` in `scl-crypto` + +**Files:** +- Modify: `crates/crypto/src/envelope.rs` (below `encrypt_path`, ~line 168) +- Modify: `crates/crypto/src/lib.rs` (export) + +**Interfaces:** +- Consumes: existing `decrypt_path`, `PATH_AAD`, `DEK_LEN`, `NONCE_LEN`. +- Produces: `pub fn encrypt_path_randomized(plaintext: &[u8]) -> (Vec, Zeroizing<[u8; 32]>)` and `pub fn encrypt_path_randomized_with_rng(plaintext: &[u8], rng: &mut R) -> (Vec, Zeroizing<[u8; 32]>)`. Blob layout identical to `encrypt_path` (`nonce(24) ‖ ct`, same `PATH_AAD`) so the existing `decrypt_path` opens both. + +- [ ] **Step 1: Write the failing tests** in `envelope.rs`'s `mod tests`: + +```rust +#[test] +fn randomized_seal_roundtrips_and_closes_the_oracle() { + let pt = b"the database password is hunter2"; + let (blob1, dek1) = encrypt_path_randomized(pt); + let (blob2, dek2) = encrypt_path_randomized(pt); + // THE phase invariant: same plaintext, different ciphertext bytes. + assert_ne!(blob1, blob2, "equality oracle must be closed"); + assert_ne!(dek1.as_slice(), dek2.as_slice()); + // Dual-read: the unchanged decrypt_path opens a randomized blob. + assert_eq!(&decrypt_path(&blob1, &dek1).unwrap()[..], pt); + assert_eq!(&decrypt_path(&blob2, &dek2).unwrap()[..], pt); +} + +#[test] +fn randomized_seal_is_deterministic_under_a_seeded_rng() { + let mut r1 = rng(7); + let mut r2 = rng(7); + let (b1, _) = encrypt_path_randomized_with_rng(b"x", &mut r1); + let (b2, _) = encrypt_path_randomized_with_rng(b"x", &mut r2); + assert_eq!(b1, b2); +} +``` + +- [ ] **Step 2: Run to verify they fail** + +Run: `cargo test -p scl-crypto randomized_seal -- --nocapture` +Expected: FAIL — function not found. + +- [ ] **Step 3: Implement** in `envelope.rs`, after `encrypt_path` (note the doc comment on `encrypt_path` should also gain one line: "Legacy convergent format — kept for dual-read comparison only; new seals use `encrypt_path_randomized` (P33)."): + +```rust +/// Randomized file encryption (P33): fresh random DEK + random nonce per +/// seal, so identical plaintext yields different ciphertext every time — +/// the ADR-0014 equality oracle is closed for content sealed this way. +/// Blob layout and AAD are identical to `encrypt_path` (`nonce ‖ ct`, +/// `PATH_AAD`), so `decrypt_path` opens both formats unchanged (dual-read). +pub fn encrypt_path_randomized(plaintext: &[u8]) -> (Vec, Zeroizing<[u8; DEK_LEN]>) { + encrypt_path_randomized_with_rng(plaintext, &mut OsRng) +} + +/// `encrypt_path_randomized` with a caller-supplied RNG (deterministic in tests). +pub fn encrypt_path_randomized_with_rng( + plaintext: &[u8], + rng: &mut R, +) -> (Vec, Zeroizing<[u8; DEK_LEN]>) { + let mut dek = Zeroizing::new([0u8; DEK_LEN]); + rng.fill_bytes(dek.as_mut_slice()); + let mut nonce = [0u8; NONCE_LEN]; + rng.fill_bytes(&mut nonce); + + let cipher = XChaCha20Poly1305::new_from_slice(dek.as_slice()).expect("32-byte DEK"); + let ciphertext = cipher + .encrypt( + XNonce::from_slice(&nonce), + Payload { + msg: plaintext, + aad: PATH_AAD, + }, + ) + .expect("aead encrypt is infallible for valid inputs"); + + let mut blob = Vec::with_capacity(NONCE_LEN + ciphertext.len()); + blob.extend_from_slice(&nonce); + blob.extend_from_slice(&ciphertext); + (blob, dek) +} +``` + +Export both from `crates/crypto/src/lib.rs` in the existing `pub use envelope::{...}` list. + +- [ ] **Step 4: Run to verify they pass** + +Run: `cargo test -p scl-crypto` +Expected: PASS (all). + +- [ ] **Step 5: Commit** + +```bash +git add crates/crypto/src/envelope.rs crates/crypto/src/lib.rs +git commit -m "P33: randomized path seal (fresh DEK+nonce, dual-read layout)" +``` + +--- + +### Task 3: `cache.rs` — local key + per-checkout unchanged-detection cache + +**Files:** +- Create: `crates/repo/src/cache.rs` +- Modify: `crates/repo/src/lib.rs` (add `pub(crate) mod cache;` — match the file's existing mod-list style) +- Modify: `crates/repo/src/layout.rs` (three path helpers) + +**Interfaces:** +- Consumes: `scl_crypto::random_hex` (for key minting), `blake3::keyed_hash`, `hex`. +- Produces (all `pub(crate)`, used by Tasks 4, 5, 7, 8, 10): + - `cache::local_key(layout: &Layout) -> Result<[u8; 32]>` — reads `.sc/local-key` (64 hex chars), lazily minting it 0600 if absent; hard error only on create/permission failure. + - `struct ProtectedCache` with: + - `ProtectedCache::open(key: [u8; 32], path: Option) -> ProtectedCache` — loads entries if the file exists; a corrupt file is treated as empty (stderr warning); `path: None` = ephemeral in-memory cache (used by `sc work`). + - `fn unchanged(&self, rel: &str, abs: &Path, plaintext: &[u8]) -> Option` — stat hit (mtime+size) or keyed-tag match returns the cached ciphertext blob id. + - `fn record(&mut self, rel: &str, abs: &Path, plaintext: &[u8], blob_id: ObjectId)` + - `fn save(&self) -> Result<()>` — atomic (write temp sibling + rename); no-op when `path` is `None`. + - `Layout::local_key_path()` → `.sc/local-key`; `Layout::protected_cache_path()` → `.sc/protected-cache`; `Layout::ws_cache_path(i: usize)` → `.sc/ws/cache-` (beside the checkout dir, NOT inside `.sc/ws//` — a file inside the checkout would be harvested as an untracked working file). + +- [ ] **Step 1: Add the Layout helpers** (no test needed — pure path joins, mirror the style of `sparse_path()` at `layout.rs:78`): + +```rust +/// `.sc/local-key` — per-repo random key for the P33 unchanged-detection +/// cache's keyed hashes. Never committed, never transferred. +pub fn local_key_path(&self) -> PathBuf { + self.dot_sc.join("local-key") +} + +/// `.sc/protected-cache` — the main working tree's P33 stat cache. +pub fn protected_cache_path(&self) -> PathBuf { + self.dot_sc.join("protected-cache") +} + +/// `.sc/ws/cache-` — workspace `i`'s P33 stat cache. Lives BESIDE the +/// checkout dir (`.sc/ws//`), never inside it, so harvest's worktree +/// read can't pick it up as an untracked file. +pub fn ws_cache_path(&self, i: usize) -> PathBuf { + self.dot_sc.join("ws").join(format!("cache-{i}")) +} +``` + +- [ ] **Step 2: Write the failing tests** — create `crates/repo/src/cache.rs` with the tests first (module skeleton + `mod tests`). Use a tempdir pattern like `worktree.rs`'s `tmp_objects`: + +```rust +#[cfg(test)] +mod tests { + use super::*; + + fn tmp(tag: &str) -> Layout { + let root = std::env::temp_dir().join(format!("scl-cache-{tag}-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&root); + std::fs::create_dir_all(root.join(".sc")).unwrap(); + Layout::at(&root) + } + + fn cleanup(layout: &Layout) { + std::fs::remove_dir_all(&layout.root).unwrap(); + assert!(!layout.root.exists()); + } + + #[test] + fn local_key_is_minted_once_and_stable() { + let layout = tmp("key"); + let k1 = local_key(&layout).unwrap(); + let k2 = local_key(&layout).unwrap(); + assert_eq!(k1, k2, "second read must return the same key"); + assert_ne!(k1, [0u8; 32]); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + let mode = std::fs::metadata(layout.local_key_path()) + .unwrap() + .permissions() + .mode(); + assert_eq!(mode & 0o777, 0o600, "key file must be 0600"); + } + cleanup(&layout); + } + + #[test] + fn stat_hit_tag_fallback_and_miss() { + let layout = tmp("hits"); + let key = local_key(&layout).unwrap(); + let abs = layout.root.join("secret.txt"); + std::fs::write(&abs, b"v1").unwrap(); + let id = ObjectId::of(b"cipher-of-v1"); + + let mut c = ProtectedCache::open(key, Some(layout.protected_cache_path())); + assert_eq!(c.unchanged("secret.txt", &abs, b"v1"), None, "empty cache misses"); + c.record("secret.txt", &abs, b"v1", id); + + // Stat hit: file untouched. + assert_eq!(c.unchanged("secret.txt", &abs, b"v1"), Some(id)); + + // Tag fallback: touch mtime without changing content. + std::fs::write(&abs, b"v1").unwrap(); + assert_eq!(c.unchanged("secret.txt", &abs, b"v1"), Some(id)); + + // Real change: miss. + std::fs::write(&abs, b"v2").unwrap(); + assert_eq!(c.unchanged("secret.txt", &abs, b"v2"), None); + cleanup(&layout); + } + + #[test] + fn save_load_roundtrip_and_corrupt_file_is_empty() { + let layout = tmp("persist"); + let key = local_key(&layout).unwrap(); + let abs = layout.root.join("a b/spaced name.txt"); // path with spaces + std::fs::create_dir_all(abs.parent().unwrap()).unwrap(); + std::fs::write(&abs, b"v").unwrap(); + let id = ObjectId::of(b"c"); + + let mut c = ProtectedCache::open(key, Some(layout.protected_cache_path())); + c.record("a b/spaced name.txt", &abs, b"v", id); + c.save().unwrap(); + + let c2 = ProtectedCache::open(key, Some(layout.protected_cache_path())); + assert_eq!(c2.unchanged("a b/spaced name.txt", &abs, b"v"), Some(id)); + + // Corrupt file: degrades to empty, never errors. + std::fs::write(layout.protected_cache_path(), b"garbage\nlines\n").unwrap(); + let c3 = ProtectedCache::open(key, Some(layout.protected_cache_path())); + assert_eq!(c3.unchanged("a b/spaced name.txt", &abs, b"v"), None); + cleanup(&layout); + } + + #[test] + fn ephemeral_cache_never_touches_disk() { + let layout = tmp("ephemeral"); + let key = local_key(&layout).unwrap(); + let abs = layout.root.join("f"); + std::fs::write(&abs, b"v").unwrap(); + let mut c = ProtectedCache::open(key, None); + c.record("f", &abs, b"v", ObjectId::of(b"c")); + c.save().unwrap(); + assert!(!layout.protected_cache_path().exists()); + cleanup(&layout); + } +} +``` + +- [ ] **Step 3: Run to verify they fail** + +Run: `cargo test -p scl-repo cache:: -- --nocapture` +Expected: FAIL to compile — types not defined. + +- [ ] **Step 4: Implement the module**: + +```rust +//! P33: local unchanged-detection cache for randomized protected paths. +//! +//! Randomized seals (ADR-0043) make re-encrypt-and-compare useless for +//! change detection, so commit/status/diff consult this `.sc`-local, +//! never-committed, never-transferred map instead: +//! `path -> (mtime_ns, size, keyed_tag, ciphertext blob id)`. +//! The tag is `blake3::keyed_hash(local_key, plaintext)` under a random +//! per-repo key (`.sc/local-key`, 0600), so the cache file alone leaks +//! nothing — it does not reintroduce the equality oracle this phase closes. +//! A lost/stale/corrupt cache degrades to spurious re-seals, never +//! incorrectness and never a hard error. + +use std::collections::BTreeMap; +use std::path::{Path, PathBuf}; + +use scl_core::ObjectId; + +use crate::error::Result; +use crate::layout::Layout; + +/// Load (minting if absent) the per-repo random cache key. The key file is +/// the only hard-error surface here: if it can't be created/read, the cache +/// cannot be safely keyed, so surface it rather than running unkeyed. +pub(crate) fn local_key(layout: &Layout) -> Result<[u8; 32]> { + let path = layout.local_key_path(); + let hex_str = match std::fs::read_to_string(&path) { + Ok(s) => s.trim().to_string(), + Err(e) if e.kind() == std::io::ErrorKind::NotFound => { + let s = scl_crypto::random_hex(32); // 32 random bytes -> 64 hex chars + // Mirror the serve-TLS key discipline: 0600 before content matters. + std::fs::write(&path, &s)?; + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o600))?; + } + s + } + Err(e) => return Err(e.into()), + }; + let bytes = hex::decode(&hex_str) + .map_err(|_| crate::error::Error::Corrupt("malformed .sc/local-key".into()))?; + bytes + .try_into() + .map_err(|_| crate::error::Error::Corrupt("malformed .sc/local-key".into())) +} +``` + +> NOTE: check `scl_crypto::random_hex`'s contract at `crates/crypto/src/lib.rs:37` — if `random_hex(32)` returns 32 hex *chars* (16 bytes) rather than 64, pass 64 or add a `random_bytes(n)` helper to `scl-crypto` instead. The test (`k1 != [0u8;32]`, 32-byte decode) will catch a mismatch. If `Error::Corrupt` doesn't exist in `crates/repo/src/error.rs`, use the closest existing variant (e.g. `Error::InvalidArgument`) — do not add a new variant for this. + +```rust +#[derive(Clone, Copy, Debug, PartialEq)] +struct CacheEntry { + mtime_ns: u128, + size: u64, + tag: [u8; 32], + blob_id: ObjectId, +} + +/// Per-checkout unchanged-detection cache (main tree, one ws workspace, or +/// an ephemeral in-memory one for `sc work` temp checkouts). +pub(crate) struct ProtectedCache { + key: [u8; 32], + /// `None` => ephemeral: `save()` is a no-op. + path: Option, + entries: BTreeMap, +} + +impl ProtectedCache { + pub(crate) fn open(key: [u8; 32], path: Option) -> ProtectedCache { + let mut entries = BTreeMap::new(); + if let Some(p) = &path { + if let Ok(text) = std::fs::read_to_string(p) { + for line in text.lines() { + // Format: ` ` + // (path last: it may contain spaces). + let mut it = line.splitn(5, ' '); + let parsed = (|| { + let mtime_ns: u128 = it.next()?.parse().ok()?; + let size: u64 = it.next()?.parse().ok()?; + let tag: [u8; 32] = hex::decode(it.next()?).ok()?.try_into().ok()?; + let blob_id = ObjectId::from_hex(it.next()?).ok()?; + Some((it.next()?.to_string(), CacheEntry { mtime_ns, size, tag, blob_id })) + })(); + match parsed { + Some((path, e)) => { + entries.insert(path, e); + } + None => { + eprintln!("warning: ignoring corrupt protected-cache line"); + entries.clear(); + break; // degrade to empty: spurious re-seals, never incorrectness + } + } + } + } + } + ProtectedCache { key, path, entries } + } + + fn tag(&self, plaintext: &[u8]) -> [u8; 32] { + *blake3::keyed_hash(&self.key, plaintext).as_bytes() + } + + fn stat(abs: &Path) -> Option<(u128, u64)> { + let md = std::fs::metadata(abs).ok()?; + let mtime = md.modified().ok()?; + let ns = mtime + .duration_since(std::time::UNIX_EPOCH) + .ok()? + .as_nanos(); + Some((ns, md.len())) + } + + /// The cached ciphertext blob id iff this exact plaintext is what last + /// sealed at `rel`: stat hit (mtime+size unchanged) short-circuits; + /// otherwise fall back to the keyed-tag comparison. + pub(crate) fn unchanged(&self, rel: &str, abs: &Path, plaintext: &[u8]) -> Option { + let e = self.entries.get(rel)?; + if let Some((mtime_ns, size)) = Self::stat(abs) { + if mtime_ns == e.mtime_ns && size == e.size { + return Some(e.blob_id); + } + } + (self.tag(plaintext) == e.tag).then_some(e.blob_id) + } + + /// Record that `plaintext` at `rel` seals to `blob_id`. Missing stat + /// (file vanished mid-operation) just skips the entry. + pub(crate) fn record(&mut self, rel: &str, abs: &Path, plaintext: &[u8], blob_id: ObjectId) { + if let Some((mtime_ns, size)) = Self::stat(abs) { + let tag = self.tag(plaintext); + self.entries + .insert(rel.to_string(), CacheEntry { mtime_ns, size, tag, blob_id }); + } + } + + /// Atomic write (temp sibling + rename); no-op for an ephemeral cache. + pub(crate) fn save(&self) -> Result<()> { + let Some(path) = &self.path else { return Ok(()) }; + let mut out = String::new(); + for (p, e) in &self.entries { + out.push_str(&format!( + "{} {} {} {} {}\n", + e.mtime_ns, + e.size, + hex::encode(e.tag), + e.blob_id.to_hex(), + p + )); + } + if let Some(parent) = path.parent() { + std::fs::create_dir_all(parent)?; + } + let tmp = path.with_extension(format!("tmp-{}", std::process::id())); + std::fs::write(&tmp, out)?; + std::fs::rename(&tmp, path)?; + Ok(()) + } +} +``` + +> NOTE: check `ObjectId`'s hex API in `crates/core` — `refs.rs` serializes branch tips to hex; use exactly the same to/from helpers it uses (adjust `to_hex`/`from_hex` names to what exists). + +- [ ] **Step 5: Run to verify they pass** + +Run: `cargo test -p scl-repo cache::` +Expected: PASS (4 tests). + +- [ ] **Step 6: Commit** + +```bash +git add crates/repo/src/cache.rs crates/repo/src/lib.rs crates/repo/src/layout.rs +git commit -m "P33: per-checkout protected-path stat cache + local key" +``` + +--- + +### Task 4: Format dispatch in `diff_worktree` and `diff_unified` + +**Files:** +- Modify: `crates/repo/src/worktree.rs:477-529` (`diff_worktree`) +- Modify: `crates/repo/src/repo.rs:881-891` (`diff_unified`'s protected arm) +- Modify: every `diff_worktree` caller (grep `diff_worktree(` — status, dirty-tree preflights in switch/merge/rebase, `harvest_workspace` at `workspace.rs:85`, ws probes) to pass the new parameter. + +**Interfaces:** +- Consumes: `RANDOMIZED` (Task 1), `ProtectedCache::unchanged` (Task 3). +- Produces: `diff_worktree(layout, store, head_root, protection, sparse, cache: Option<&ProtectedCache>)` — the new trailing parameter. Main-tree callers pass `Some(&cache)` where they already have a `Repo` (open via `cache::local_key(self.layout())` + `ProtectedCache::open(key, Some(self.layout().protected_cache_path()))`); `harvest_workspace` passes the workspace cache its caller threads in (Task 7 wires ws/`sc work`; until then pass `None` there). + +Behavior for a `PROTECTED` HEAD entry with disk bytes present: +- `RANDOMIZED` bit clear → today's convergent re-encrypt-and-compare (unchanged code). +- `RANDOMIZED` bit set → `cache.unchanged(path, abs, bytes)`: `Some(id)` equal to the HEAD id ⇒ clean; `Some(other)`/`None`/no cache ⇒ modified (spurious-but-safe). + +- [ ] **Step 1: Write the failing test** in `worktree.rs`'s `mod tests` (adapt to the file's helpers — `tmp_objects` exists at `worktree.rs:672`): + +```rust +#[test] +fn diff_dispatches_randomized_entries_through_the_cache() { + let (layout, mut store) = tmp_objects("p33-diff"); + // Build a HEAD tree with one RANDOMIZED protected entry whose plaintext + // sits on disk, exactly as an authorized checkout leaves it. + let pt = b"top secret".to_vec(); + let (cipher, _dek) = scl_crypto::encrypt_path_randomized(&pt); + let blob = Object::blob(cipher.clone()); + let blob_id = blob.id(); + store.put(blob).unwrap(); + let root = write_tree_with_perms_into( + &mut store, + &[("secret/x".to_string(), cipher, FileMode::FILE, PROTECTED | scl_core::RANDOMIZED)], + ); + let abs = layout.root.join("secret/x"); + std::fs::create_dir_all(abs.parent().unwrap()).unwrap(); + std::fs::write(&abs, &pt).unwrap(); + + let key = crate::cache::local_key(&layout).unwrap(); + let mut cache = crate::cache::ProtectedCache::open(key, None); + + // No cache entry: spurious-but-safe MODIFIED. + let d = diff_worktree(&layout, &mut store, Some(root), &Protection::default(), + &Sparse::default(), Some(&cache)).unwrap(); + assert_eq!(d.modified, vec!["secret/x"]); + + // Recorded: provably clean. + cache.record("secret/x", &abs, &pt, blob_id); + let d = diff_worktree(&layout, &mut store, Some(root), &Protection::default(), + &Sparse::default(), Some(&cache)).unwrap(); + assert!(d.modified.is_empty() && d.added.is_empty() && d.deleted.is_empty()); + + // Genuine edit: modified again (tag mismatch). + std::fs::write(&abs, b"edited").unwrap(); + let d = diff_worktree(&layout, &mut store, Some(root), &Protection::default(), + &Sparse::default(), Some(&cache)).unwrap(); + assert_eq!(d.modified, vec!["secret/x"]); + + std::fs::remove_dir_all(&layout.root).unwrap(); + assert!(!layout.root.exists()); +} +``` + +(If no `write_tree_with_perms_into` helper exists in these tests, build the tree the way the file's existing protected-diff tests do — grep `PROTECTED` in `worktree.rs`'s test mod and copy that construction.) + +- [ ] **Step 2: Run to verify it fails** + +Run: `cargo test -p scl-repo diff_dispatches_randomized -- --nocapture` +Expected: FAIL to compile (extra argument). + +- [ ] **Step 3: Implement.** In `diff_worktree`, add the parameter and replace the protected arm (`worktree.rs:501-506`): + +```rust +let disk_id = if perms & PROTECTED != 0 { + if perms & scl_core::RANDOMIZED == 0 { + // Convergent (legacy) entry: re-encryption yields the same id + // as the commit did — still exact, no cache needed. + Object::blob(scl_crypto::encrypt_path(bytes).0).id() + } else { + // Randomized entry (P33): re-encrypting proves nothing. The + // stat cache (stat hit, then keyed-tag fallback) is the only + // public-key-free unchanged proof; a missing entry reports + // modified — spurious-but-safe, never incorrect. + let abs = safe_join(&layout.root, p)?; + match cache.and_then(|c| c.unchanged(p, &abs, bytes)) { + Some(id) => id, + None => { + diff.modified.push(p.clone()); + continue; + } + } + } +} else { + Object::blob(bytes.clone()).id() +}; +``` + +Extend the doc comment's protected-path paragraph to describe the two formats. In `diff_unified` (`repo.rs:881-891`), apply the same dispatch: convergent → existing compare; randomized → open the main-tree cache (`cache::local_key` + `ProtectedCache::open(key, Some(self.layout().protected_cache_path()))` at the top of the function) and report `protected file changed: {path} (content not shown)` when `unchanged` misses or disagrees. Update all `diff_worktree` callers: `Repo` methods construct the main-tree cache; `workspace.rs:85` passes `None` for now (Task 7 threads the real one). + +- [ ] **Step 4: Run the full crate suite** + +Run: `cargo test -p scl-repo` +Expected: PASS — no existing test regresses (no `RANDOMIZED` entries exist yet anywhere else, so every existing path takes the convergent arm). + +- [ ] **Step 5: Commit** + +```bash +git add -A crates/repo +git commit -m "P33: format-dispatched unchanged detection in status/diff" +``` + +--- + +### Task 5: Format-dispatched carry in `snapshot_files` (commit path) + +**Files:** +- Modify: `crates/repo/src/repo.rs:339-360` (the plain/protected split + encrypt block inside `snapshot_files`) and the function signature at `repo.rs:256` +- Modify: all `snapshot_files` callers (grep `snapshot_files(` — `commit`, `amend`, `assemble_completion_snapshot`, `harvest_workspace` at `workspace.rs:100`, and any replay-side callers) to pass the new parameter. + +**Interfaces:** +- Consumes: `RANDOMIZED`, `ProtectedCache` (Tasks 1, 3). +- Produces: `snapshot_files(..., sparse: &Sparse, cache: Option<&mut ProtectedCache>, author, message)` — one new parameter, threaded explicitly like `sparse` (the P24 anti-ambient discipline). Main-tree callers open + pass the main cache and call `cache.save()` after the commit succeeds; `harvest_workspace` threads its caller's cache (Task 7). + +Per-protected-path commit rule (the spec §3 table): +1. Prior tip entry convergent → convergent-compare; equal ⇒ **carry** prior ciphertext + perms verbatim (stays convergent, no cache needed); else seal. +2. Prior tip entry randomized → `cache.unchanged` hit matching the prior blob id ⇒ carry; else seal. +3. No prior entry ⇒ seal. + +Carried entries reuse the existing absent-path carry mechanics (fetch ciphertext from store, push with source perms, `fresh_wrapped.entry(blob_id).or_insert_with(prior wraps)`). Sealed entries go through `encrypt_protected` exactly as today. After sealing, `record` each sealed path's new blob id (recompute `Object::blob(cipher).id()` from the returned entries) and each carried randomized path (refresh stat) into the cache. + +- [ ] **Step 1: Write the failing test** in `repo.rs`'s `mod tests`, using the file's existing persistent-repo + protect helpers (grep `fn protect` / `sc protect`-style tests, e.g. ones that call `repo.protect(...)` then `repo.commit(...)`; mirror their setup): + +```rust +#[test] +fn commit_carries_unchanged_convergent_content_without_reseal() { + // A pre-P33 store: protect + commit mints CONVERGENT entries (until + // Task 6 flips the seal, new commits still mint convergent too — this + // test pins the carry rule either way by asserting blob-id stability). + let (repo, _tmp) = test_repo_with_identity("p33-carry"); // adapt to existing helper + write_file(&repo, "secret/db.txt", b"hunter2"); + repo.protect("secret/", &[recipient_pk()]).unwrap(); + let c1 = repo.commit("alice", "one").unwrap(); + let id1 = blob_id_at(&repo, c1, "secret/db.txt"); + + // Unrelated edit; protected file untouched. + write_file(&repo, "readme.md", b"hello"); + let c2 = repo.commit("alice", "two").unwrap(); + let id2 = blob_id_at(&repo, c2, "secret/db.txt"); + assert_eq!(id1, id2, "unchanged protected content must carry, not re-seal"); +} +``` + +Add a second test for the randomized arm — build a tip whose protected entry is `PROTECTED | RANDOMIZED` (seal with `encrypt_path_randomized`, write the tree via the low-level helpers as in Task 4's test), materialize the plaintext, record it in the main cache, run `commit`, and assert the blob id is carried on a cache hit and re-sealed (different id, still decryptable) after the cache file is deleted: + +```rust +#[test] +fn commit_randomized_carry_via_cache_and_reseal_on_lost_cache() { + // ...setup as described above... + let c2 = repo.commit("alice", "unrelated").unwrap(); + assert_eq!(blob_id_at(&repo, c2, "secret/x"), randomized_id, "cache hit carries"); + + std::fs::remove_file(repo.layout().protected_cache_path()).unwrap(); + write_file(&repo, "readme.md", b"more"); + let c3 = repo.commit("alice", "unrelated2").unwrap(); + let id3 = blob_id_at(&repo, c3, "secret/x"); + assert_ne!(id3, randomized_id, "lost cache degrades to a spurious re-seal"); + // Never incorrectness: the new blob still decrypts to the same plaintext. + assert_eq!(decrypt_at(&repo, c3, "secret/x", &identity()), b"top secret"); +} +``` + +- [ ] **Step 2: Run to verify they fail** + +Run: `cargo test -p scl-repo commit_carries_unchanged commit_randomized_carry -- --nocapture` +Expected: FAIL (compile error on the new param, then assertion). + +- [ ] **Step 3: Implement.** Inside `snapshot_files`, after the plain/protected split (`repo.rs:341-348`) and the scanner gate, insert the dispatch before `encrypt_protected`: + +```rust +// P33 format-dispatched carry: decide per protected path whether the tip +// already holds this exact content (carry the prior ciphertext id — quiet +// history) or it must be sealed fresh (randomized). Convergent priors use +// the exact re-encrypt-and-compare proof; randomized priors use the local +// stat cache (stat hit, then keyed tag). This dispatch is what makes the +// upgrade non-mandatory: unchanged legacy content stays convergent forever. +let tip_entries: BTreeMap = match tip { + Some(t) => { + let store_arc = self.vfs.store(); + let mut store = store_arc.lock().unwrap(); + let parent_root = store.get_snapshot(&t)?.root; + if partial { + worktree::tree_file_entries_with_perms_sparse(&mut store, parent_root, sparse)? + } else { + worktree::tree_file_entries_with_perms(&mut store, parent_root)? + } + } + None => BTreeMap::new(), +}; +let mut to_seal: Vec = Vec::new(); +let mut sealed_plaintexts: Vec<(String, Vec)> = Vec::new(); +for (path, bytes, mode, granted) in protected { + let carried = tip_entries.get(&path).and_then(|(blob_id, _m, perms)| { + if perms & scl_core::PROTECTED == 0 { + return None; + } + let unchanged = if perms & scl_core::RANDOMIZED == 0 { + Object::blob(scl_crypto::encrypt_path(&bytes).0).id() == *blob_id + } else { + let abs = worktree::safe_join(&self.layout.root, &path).ok()?; + cache + .as_deref() + .and_then(|c| c.unchanged(&path, &abs, &bytes)) + == Some(*blob_id) + }; + unchanged.then_some((*blob_id, *perms)) + }); + match carried { + Some((blob_id, perms)) => { + let store_arc = self.vfs.store(); + let mut store = store_arc.lock().unwrap(); + let cipher = match store.get(&blob_id)? { + Object::Blob(b) => b.to_vec(), + _ => { + to_seal.push((path, bytes, mode, granted)); + continue; + } + }; + all.push((path.clone(), cipher, mode, perms)); + if let Some(prior_wks) = protection.wrapped.get(&blob_id) { + fresh_wrapped + .entry(blob_id) + .or_insert_with(|| prior_wks.clone()); + } + } + None => { + sealed_plaintexts.push((path.clone(), bytes.clone())); + to_seal.push((path, bytes, mode, granted)); + } + } +} +let (protected_all, sealed_wrapped) = crate::protect::encrypt_protected(to_seal)?; +// Record every freshly sealed path (its new blob id) and refresh carried +// randomized paths in the cache; the caller saves after the commit lands. +if let Some(c) = cache.as_deref_mut() { + for (path, cipher, _mode, _perms) in &protected_all { + if let Some((_, pt)) = sealed_plaintexts.iter().find(|(p, _)| p == path) { + if let Ok(abs) = worktree::safe_join(&self.layout.root, path) { + c.record(path, &abs, pt, Object::blob(cipher.clone()).id()); + } + } + } +} +all.extend(protected_all); +fresh_wrapped.extend(sealed_wrapped); +``` + +> Anchoring notes for the implementer: `all`/`fresh_wrapped` are declared at `repo.rs:357-359` — restructure that block so `all` starts from `plain` (as today) and the dispatch above replaces the single `encrypt_protected(protected)` call. `partial`/`promisor` are computed at `repo.rs:418-419` — hoist that pair above the dispatch (it's needed earlier now). `ProtectedFile` is the existing alias used at `repo.rs:342`. The `?` inside the `and_then` closure won't compile — use explicit `match`/`if let` there instead (the code above marks the one spot with `.ok()?` inside a closure returning `Option`, which is fine; `store.get(...)?` in the outer loop is fine). +> +> **Error-path discipline:** if the commit later fails (scanner already ran; tree-write or ref-move errors), the cache was mutated in memory but `save()` never runs — entries stay stale, which the degradation rule already covers (stale = spurious re-seal next time or a stat mismatch falling to tag compare; a *recorded-but-unlanded* blob id would only be returned on a later hit if that exact plaintext seals again, in which case the id is present in the store anyway — content-addressed stores make this benign). Do NOT save the cache inside `snapshot_files`; the committing caller saves only after the ref moves. + +Thread the parameter through every caller. In `Repo::commit`/`amend`/`assemble_completion_snapshot`: + +```rust +let key = crate::cache::local_key(&self.layout)?; +let mut cache = + crate::cache::ProtectedCache::open(key, Some(self.layout.protected_cache_path())); +let id = self.snapshot_files(files, ..., sparse, Some(&mut cache), author, message)?; +// only after the ref actually moved: +cache.save()?; +``` + +`workspace.rs:100` (`harvest_workspace`) passes `None` in this task; Task 7 threads the real per-workspace cache. + +- [ ] **Step 4: Run the full crate suite** + +Run: `cargo test -p scl-repo` +Expected: PASS — existing behavior unchanged (all existing protected entries are convergent; the convergent-compare carry produces the same blob ids the re-seal used to). + +- [ ] **Step 5: Commit** + +```bash +git add -A crates/repo +git commit -m "P33: format-dispatched carry in the commit pipeline" +``` + +--- + +### Task 6: Flip `encrypt_protected` to randomized seals + +**Files:** +- Modify: `crates/repo/src/protect.rs:125-153` (`encrypt_protected`) +- Modify: any test that asserts convergent behavior of *new* seals (see Step 4) + +**Interfaces:** +- Consumes: `encrypt_path_randomized` (Task 2), `RANDOMIZED` (Task 1). +- Produces: `encrypt_protected` unchanged in signature, but every returned entry now carries perms `PROTECTED | RANDOMIZED` and a fresh random seal. This is the single choke point — its three callers (`repo.rs:359` commit [now via Task 5's `to_seal`], `repo.rs:1217` merge completion, `replay.rs:222` replay) all switch at once. + +- [ ] **Step 1: Write the failing test** in `protect.rs`'s `mod tests`: + +```rust +#[test] +fn encrypt_protected_seals_randomized() { + let (_sk, pk) = scl_crypto::generate_keypair(); + let f = |v: &[u8]| { + encrypt_protected(vec![( + "secret/x".into(), v.to_vec(), FileMode::FILE, vec![pk.to_bytes()], + )]) + .unwrap() + }; + let (a1, _) = f(b"same"); + let (a2, _) = f(b"same"); + assert_ne!(a1[0].1, a2[0].1, "two seals of one plaintext must differ (oracle closed)"); + assert_eq!(a1[0].3, scl_core::PROTECTED | scl_core::RANDOMIZED); +} +``` + +- [ ] **Step 2: Run to verify it fails** + +Run: `cargo test -p scl-repo encrypt_protected_seals_randomized -- --nocapture` +Expected: FAIL — identical ciphertext, perms == `PROTECTED`. + +- [ ] **Step 3: Implement.** In `encrypt_protected`, change two lines: + +```rust +let (blob_bytes, dek) = scl_crypto::encrypt_path_randomized(&bytes); +``` + +and + +```rust +all.push((path, blob_bytes, mode, scl_core::PROTECTED | scl_core::RANDOMIZED)); +``` + +Update the function's doc comment: "Randomized encryption (P33, ADR-0043): each file seals under a fresh random DEK+nonce…". Also update `reuse_prior_wraps`'s doc comment (`protect.rs:155-172`): prior-wrap reuse now only fires for carried (id-stable) blobs — a freshly randomized blob id is never in `prior`; the function stays for exactly the carried case. + +- [ ] **Step 4: Run the full workspace suite and fix convergence-assuming tests** + +Run: `cargo test` +Expected: the new test passes; a small set of existing tests that assert *new-seal* convergence may fail (e.g. tests asserting two commits of identical protected content share a blob id, or dedup-across-paths). For each failure, decide by the spec: +- Unchanged-content id-stability tests → should still PASS via Task 5's carry; if one fails, the carry dispatch has a bug — fix the dispatch, not the test. +- Genuine new-seal convergence assertions (same plaintext at two paths dedups; re-protect mints identical ciphertext) → the spec deliberately breaks these (accepted cost 4c). Invert/rename the assertions to pin the NEW behavior (e.g. `identical_plaintext_at_two_paths_no_longer_dedups`). + +- [ ] **Step 5: Commit** + +```bash +git add -A crates/repo +git commit -m "P33: all new protected seals are randomized (oracle closed)" +``` + +--- + +### Task 7: Cache population at materialization sites + +**Files:** +- Modify: `crates/repo/src/worktree.rs:573-664` (`materialize` — new `cache` param, record on decrypted writes) +- Modify: every `materialize` caller (grep `materialize(` in `repo.rs` [switch, merge/pick/rebase completion + aborts, sparse set/disable], `workspace.rs:51`, `ws.rs:239`) +- Modify: `crates/repo/src/workspace.rs:39-119` (`materialize_workspace` + `harvest_workspace` gain a `cache` param) +- Modify: `crates/repo/src/ws.rs` (fork opens `.sc/ws/cache-` per workspace and saves it; harvest opens/passes/saves the same; abandon/teardown removes the cache file with the workspace) +- Modify: the `sc work` session driver in `workspace.rs` (ephemeral `ProtectedCache::open(key, None)` per temp checkout, threaded fork→harvest) + +**Interfaces:** +- Consumes: `ProtectedCache::record` (Task 3). +- Produces: `materialize(layout, store, target_root, old_root, protection, identity, sparse, cache: Option<&mut ProtectedCache>)`. Inside the protected-decrypt arm (`worktree.rs:634-641`), after `std::fs::write(&full, &pt[..])?`: + +```rust +if let Some(c) = cache.as_deref_mut() { + c.record(path, &full, &pt[..], *blob_id); +} +``` + +Callers: +- `Repo::switch` / sparse set/disable / merge-pick-rebase completion materializations: open the main cache, pass `Some(&mut cache)`, `cache.save()?` after the materialize succeeds. +- `materialize_workspace` (ws fork, `ws.rs:239`): open `ProtectedCache::open(key, Some(layout.ws_cache_path(i)))`, pass it down, save after. +- `harvest_workspace` (`workspace.rs:66`): new param `cache: Option<&mut ProtectedCache>`; pass it to both `diff_worktree` (`:85`, replacing Task 4's `None`) and `snapshot_files` (`:100`, replacing Task 5's `None`). `sc ws harvest` opens the workspace's persistent cache; `sc work` threads its ephemeral one. +- `ws_abandon`/teardown: remove `.sc/ws/cache-` alongside the workspace dir. + +- [ ] **Step 1: Write the failing test** in `worktree.rs`'s `mod tests`: materialize a tree containing one `PROTECTED | RANDOMIZED` entry with a valid wrap for a test identity, passing a cache; assert the cache proves the file unchanged afterwards: + +```rust +#[test] +fn materialize_populates_the_cache_for_decrypted_files() { + // build store+tree as in Task 4's test, with a real wrap for `sk` + let key = crate::cache::local_key(&layout).unwrap(); + let mut cache = crate::cache::ProtectedCache::open(key, None); + let skipped = materialize(&layout, &mut store, root, None, &prot, Some(&sk), + &Sparse::default(), Some(&mut cache)).unwrap(); + assert!(skipped.is_empty()); + let abs = layout.root.join("secret/x"); + assert_eq!(cache.unchanged("secret/x", &abs, b"top secret"), Some(blob_id)); + // cleanup + assert gone +} +``` + +- [ ] **Step 2: Run to verify it fails** — `cargo test -p scl-repo materialize_populates` → FAIL to compile. + +- [ ] **Step 3: Implement** the parameter + record call + all caller threading described above. + +- [ ] **Step 4: Add the ws end-to-end regression** in `ws.rs`'s `mod tests` (this is the false-conflict hazard the per-workspace cache exists to prevent — mirror the structure of the existing harvest tests around `ws.rs:900-1200`): + +```rust +#[test] +fn untouched_protected_files_do_not_conflict_across_workspaces() { + // repo with a protected file, committed (now RANDOMIZED); identity known. + // fork 2 workspaces WITH identity; edit only workspace 1's PLAIN file; + // edit only workspace 2's OTHER plain file; harvest both. + // Assert: both land cleanly (no work- fallback), and the landed + // snapshots' secret/x blob id equals the base's (carried, not re-sealed). +} +``` + +Write it fully (setup helpers exist in that test mod — reuse them). + +- [ ] **Step 5: Run the full crate suite** — `cargo test -p scl-repo` → PASS. + +- [ ] **Step 6: Commit** + +```bash +git add -A crates/repo +git commit -m "P33: populate protected-cache at every plaintext materialization" +``` + +--- + +### Task 8: Cache population in `sc resolve` + +**Files:** +- Modify: `crates/repo/src/conflicts.rs` (the `resolve` write path — grep `fn resolve` / where it writes the chosen side's content to the working file) + +**Interfaces:** +- Consumes: `ProtectedCache::record`. +- Produces: after `resolve --ours|--theirs` writes a protected path's decrypted content, the main-tree cache records it (so completion carries instead of re-sealing when the chosen side's content equals a parent's). Plain paths: no cache involvement. + +- [ ] **Step 1: Write the failing test** in `conflicts.rs`'s test mod: set up a protected conflict (reuse the existing P23 test scaffolding in that file), `resolve --ours` with identity, then assert the main cache holds an entry for the path (`unchanged(path, abs, ours_plaintext)` returns the ours-side blob id). + +- [ ] **Step 2: Run to verify it fails.** + +- [ ] **Step 3: Implement:** in the resolve arm that writes decrypted protected content, open the main cache, `record(path, abs, plaintext, chosen_side_blob_id)`, `save()`. + +- [ ] **Step 4: Run** `cargo test -p scl-repo` → PASS. + +- [ ] **Step 5: Commit** — `git commit -m "P33: sc resolve records resolved protected content in the cache"`. + +--- + +### Task 9: Merge/replay semantics over mixed trees (tests + the identical-edit conflict) + +**Files:** +- Modify: `crates/repo/src/merge.rs` (tests), `crates/repo/src/replay.rs` (tests) +- Production changes expected: NONE (id fast paths and diff3-with-identity already handle it) — this task pins that claim with tests and fixes whatever falls out. + +**Interfaces:** +- Consumes: everything above. +- Produces: pinned behavior later tasks and the demo rely on. + +- [ ] **Step 1: Write the tests** (in `merge.rs`'s P15 test section, reusing `enc_file`/`tree_with_perms` at `merge.rs:765-782`; add a randomized twin of `enc_file`): + +```rust +/// Randomized twin of `enc_file` (P33). +fn enc_file_rand(prot: &mut Protection, pk: &scl_crypto::PublicKey, content: &[u8]) -> Vec { + let (cipher, dek) = scl_crypto::encrypt_path_randomized(content); + let id = Object::blob(cipher.clone()).id(); + prot.wrapped.entry(id).or_default().push(scl_crypto::wrap_dek_for(&dek, pk)); + cipher +} + +#[test] +fn identical_independent_edits_now_conflict_and_resolve_with_identity() { + // base: secret/a.txt = "v1" (randomized). ours and theirs BOTH edit it + // to "v2" — independently, so two different ciphertexts/ids. + // Assert: three_way_files WITHOUT identity reports the protected + // conflict/needs-identity outcome (match the existing content-divergent + // test's expected error/conflict shape); WITH identity it merges + // cleanly (diff3 of identical plaintexts) and the result decrypts to "v2". +} + +#[test] +fn mixed_convergent_and_randomized_tree_merges_by_id_fast_paths() { + // base holds one convergent entry (enc_file) and one randomized entry + // (enc_file_rand). ours edits only the plain file; theirs edits only the + // convergent entry. Assert the merge is clean without identity, the + // untouched randomized entry carries its exact blob id AND its + // PROTECTED|RANDOMIZED perms, and the convergent entry takes theirs' id. +} +``` + +Write both fully against the existing test patterns (`protected_id_fast_paths_need_no_identity` at `merge.rs:785` is the template — copy its structure). + +- [ ] **Step 2: Run** — `cargo test -p scl-repo identical_independent mixed_convergent -- --nocapture`. Expected: ideally PASS (the machinery is format-agnostic). If the perms byte is dropped anywhere in the merge carry (fast-path arms in `three_way_files`), fix it to carry the source entry's perms verbatim. + +- [ ] **Step 3: Add a replay twin** in `replay.rs` tests: cherry-pick a commit touching a randomized protected path onto a branch where it's unchanged (fast path, no identity) and where it diverged (needs identity). Mirror the file's existing P15 replay-protected tests. + +- [ ] **Step 4: Run the full suite** — `cargo test` → PASS. + +- [ ] **Step 5: Commit** — `git commit -m "P33: pin merge/replay behavior over mixed convergent+randomized trees"`. + +--- + +### Task 10: `sc rewrap` eager convergent→randomized upgrade + +**Files:** +- Modify: `crates/repo/src/rewrap.rs` (paths half, `rewrap.rs:125-201`, and the commit tail) +- Modify: `crates/cli/src/main.rs` (rewrap output — grep `blobs_rewrapped` for the print site) + +**Interfaces:** +- Consumes: `encrypt_path_randomized`, `RANDOMIZED`, `decrypt_path`, `ProtectedCache`. +- Produces: `RewrapReport` gains `pub blobs_resealed: usize`. Rewrap behavior per protected entry: + - Randomized entry → wrap-replacement only (existing code, ciphertext id unchanged). + - Convergent entry → decrypt with the unwrapped DEK, re-seal via `encrypt_path_randomized`, store the new blob, retarget the tree entry to the new id with perms `PROTECTED | RANDOMIZED`, fresh wraps for `granted + escrow` under the NEW id, drop the OLD id's `protection.wrapped` entry, record the path in the main-tree cache, count in `blobs_resealed`. + - The commit tail rebuilds the root tree when any blob was re-sealed (collect every tree entry's `(path, bytes, mode, perms)` — bytes from the store — apply the id/perms replacements, `self.vfs.write_tree_with_perms(&all)`), and the snapshot uses the new root. When `blobs_resealed == 0` the root is untouched (policy-only, exactly as P17 — a second rewrap converges back to tree-identical). + +- [ ] **Step 1: Write the failing test** in `rewrap.rs`'s test mod (reuse its existing scaffolding): + +```rust +#[test] +fn rewrap_upgrades_convergent_blobs_and_second_run_is_policy_only() { + // repo with a CONVERGENT protected entry at the tip (build pre-P33-style: + // seal via encrypt_path + PROTECTED perms, or reuse a fixture committed + // before Task 6 semantics via the low-level tree helpers). + let r1 = repo.rewrap(&identity, &[], &known, false).unwrap(); + assert_eq!(r1.blobs_resealed, 1); + let tip1 = repo.head_tip().unwrap().unwrap(); + // entry is now randomized, new id, decrypts to the same plaintext: + let (id1, perms1) = entry_at(&repo, tip1, "secret/x"); + assert!(perms1 & scl_core::RANDOMIZED != 0); + assert_eq!(decrypt_at(&repo, tip1, "secret/x", &identity), b"top secret"); + + let r2 = repo.rewrap(&identity, &[], &known, false).unwrap(); + assert_eq!(r2.blobs_resealed, 0, "second rewrap is policy-only again"); + if let Some(c2) = r2.commit { + let tip2 = repo.head_tip().unwrap().unwrap(); + assert_eq!(entry_at(&repo, tip2, "secret/x").0, id1, "ciphertext id stable"); + } +} +``` + +Also extend an existing skip-and-report test to cover a convergent blob the identity cannot open: it lands in `skipped`, everything else still upgrades, exit stays non-zero (existing semantics). + +- [ ] **Step 2: Run to verify it fails** — `cargo test -p scl-repo rewrap_upgrades -- --nocapture` → FAIL (`blobs_resealed` not found). + +- [ ] **Step 3: Implement.** In the paths loop after `unwrap_dek_with` succeeds (`rewrap.rs:166-172`), branch on the format: + +```rust +if perms & scl_core::RANDOMIZED == 0 { + // Convergent (pre-P33): eager upgrade. Decrypt with the DEK we just + // unwrapped, re-seal randomized, retarget tree + wraps to the new id. + let cipher = { /* store.get(&blob_id) -> Object::Blob bytes */ }; + let pt = match scl_crypto::decrypt_path(&cipher, &dek) { + Ok(p) => p, + Err(e) => { + skipped.push((format!("path {path}"), format!("ciphertext failed to open: {e}"))); + continue; + } + }; + if dry_run { blobs_resealed += 1; continue; } + let (new_cipher, new_dek) = scl_crypto::encrypt_path_randomized(&pt); + let new_id = { /* store.put(Object::blob(new_cipher.clone()))? */ }; + let mut new_wks: Vec = target_pks + .iter() + .map(|pk| scl_crypto::wrap_dek_for(&new_dek, pk)) + .collect(); + new_wks.sort_by(|a, b| a.recipient_id.cmp(&b.recipient_id)); + protection.wrapped.remove(&blob_id); + protection.wrapped.insert(new_id, new_wks); + retargets.insert(path.clone(), (new_id, scl_core::PROTECTED | scl_core::RANDOMIZED)); + main_cache.record(&path, &self.layout().root.join(&path), &pt, new_id); + blobs_resealed += 1; + continue; +} +// randomized: existing wrap-replacement code, unchanged +``` + +(`target_pks` already exists in the loop at `rewrap.rs:184-194` — hoist its construction above the branch so both arms share it. `retargets: BTreeMap` is new, declared beside `blobs_rewrapped`. `main_cache` is opened once before the loop and `save()`d only when the commit lands.) + +In the commit tail: when `retargets` is non-empty, rebuild the root — walk `entries` again, for each path fetch bytes from the store (new blob for retargeted paths), apply retargeted perms, `write_tree_with_perms`, and build the snapshot on the new root. Update the emptiness check (`rewrap.rs:206`) to also consider `blobs_resealed`. In the CLI print site, add: `re-sealed {n} convergent blob(s) → randomized` when `blobs_resealed > 0`. + +- [ ] **Step 4: Run** — `cargo test -p scl-repo rewrap` then `cargo test` → PASS. + +- [ ] **Step 5: Commit** — `git commit -m "P33: sc rewrap eagerly re-seals convergent blobs randomized"`. + +--- + +### Task 11: Demo — `demo/run_randomized_demo.sh` + +**Files:** +- Create: `demo/run_randomized_demo.sh` (copy the header/conventions of `demo/run_rewrap_demo.sh` — build once, `set -euo pipefail`, temp workdir, run-twice loop, zero-residue asserts) + +**Interfaces:** consumes the finished CLI. The demo must prove, in order: +1. **Oracle closed:** commit the same plaintext at two protected paths, and the same plaintext twice across two commits (edit away, edit back); extract blob ids via `sc log --json`/object inspection and assert all ciphertext ids differ. +2. **Quiet history:** an unrelated commit does NOT change an untouched protected file's blob id (cache carry). +3. **Dual-read:** a repo fixture with a convergent (pre-P33) entry still decrypts; `sc rewrap` upgrades it (`re-sealed 1 convergent blob(s)`), plaintext identical after; a second `sc rewrap` prints no re-seal line. +4. **Old history still decrypts** after the upgrade (check out the pre-rewrap commit, decrypt). +5. Zero `.sc/tmp` residue; the workdir is removed; run the whole script twice. + +For the convergent fixture: keep a tiny `sc`-driven pre-seed impossible post-Task-6, so build it with a rust test-utility or simply commit the fixture repo bytes… **Do neither** — instead drive the upgrade path end to end: initialize the repo with the CURRENT binary, then flip one committed protected entry to convergent form using a small hidden debug command? Also no. The honest MVP: the demo proves oracle-closure, quiet history, rewrap idempotence (steps 1, 2, second half of 3, 5) with the current binary, and **dual-read of genuinely old stores is pinned by unit tests** (Tasks 4–6, 9, 10 all build convergent fixtures via the library). State this split in a comment at the top of the demo. + +- [ ] **Step 1: Write the script** (full content, following `run_rewrap_demo.sh`'s skeleton). +- [ ] **Step 2: Run it twice** — `bash demo/run_randomized_demo.sh && bash demo/run_randomized_demo.sh` → both green, zero residue. +- [ ] **Step 3: Run every existing protected demo** — `bash demo/run_protected_merge_demo.sh && bash demo/run_revoke_demo.sh && bash demo/run_rewrap_demo.sh && bash demo/run_lifecycle_demo.sh && bash demo/run_merge_ergonomics_demo.sh && bash demo/run_ws_demo.sh` → all green. +- [ ] **Step 4: Commit** — `git commit -m "P33: randomized-encryption demo (oracle closed, quiet history, rewrap upgrade)"`. + +--- + +### Task 12: Docs — ADR-0043, THREAT-MODEL, CLAUDE.md, ARCHITECTURE.md + +**Files:** +- Create: `docs/adr/0043-randomized-protected-encryption.md` (follow the ADR template in `docs/adr/README.md` / recent ADRs' structure: Status/Date/Phase, Context, Decision, Consequences, Alternatives) +- Modify: `THREAT-MODEL.md` (the convergent-equality entry: superseded for new content, still true for pre-P33 history until rewrapped) +- Modify: `CLAUDE.md` (a "Phase 33 is built" section in the established voice: what shipped, the format-dispatch rule, the cache, accepted costs 4a–c, boundaries; update the `sc rewrap` command comment; move the P28 "randomized protected mode" follow-on out of Remaining follow-ons, add rotate-for-paths) +- Modify: `ARCHITECTURE.md` (the protected-path encryption section: randomized-write/dual-read posture) +- Modify: `docs/adr/0014-*.md` — add a one-line pointer: "Superseded for newly sealed content by ADR-0043 (P33); convergent dual-read retained." + +**Content requirements for ADR-0043** (each gets a paragraph): +- The format change (RANDOMIZED perms bit; blob layout/AAD unchanged; why the tag lives in the tree entry, not the blob). +- Dual-read posture (no tag bump, no migration; the format-dispatched commit rule and why cache-only dispatch would have mass-migrated). +- Cache design and why it doesn't reintroduce the oracle (keyed PRF under a never-traveling local key; degradation rule). +- Accepted costs 4a–c verbatim from #40, plus the rewrap tree-identity change. +- P15/P17 semantic adjustments; the ws per-workspace cache rationale (false-conflict hazard). +- Unlocked follow-on: rotate-for-paths (ADR-0019's objection dissolves for randomized content) — recorded, not built. + +- [ ] **Step 1: Write ADR-0043.** +- [ ] **Step 2: Update THREAT-MODEL.md, CLAUDE.md, ARCHITECTURE.md, ADR-0014 pointer.** +- [ ] **Step 3: Self-check** — grep CLAUDE.md for "randomized" to make sure the P28 accepted-boundary line ("convergent encryption stays equality-confirmable") is updated to point at P33. +- [ ] **Step 4: Commit** — `git commit -m "P33: ADR-0043 + THREAT-MODEL/CLAUDE.md/ARCHITECTURE.md updates"`. + +--- + +### Task 13: Full validation sweep + +- [ ] **Step 1:** `cargo test` (whole workspace) → PASS. +- [ ] **Step 2:** `cargo clippy --workspace` → no new warnings. +- [ ] **Step 3:** Run ALL demos listed in CLAUDE.md's Commands section (there are ~20 `demo/run_*.sh` scripts — run each once; protected-related ones twice). Expected: all green, zero residue. +- [ ] **Step 4:** Equality-oracle end check: `cargo test -p scl-crypto randomized_seal_roundtrips_and_closes_the_oracle -p scl-repo encrypt_protected_seals_randomized` → PASS. +- [ ] **Step 5:** Final commit if anything moved; then hand off to the finishing-a-development-branch flow (PR per repo convention — squash-merged). + +--- + +## Self-Review Notes (already applied) + +- **Spec coverage:** §1→Task 2, §2→Task 1, §3→Tasks 3+5, §4→Task 4, §5→Tasks 7+8, §6→Task 9, §7→Task 10, §9 error handling→Tasks 3/5/10 inline, §10→Tasks 11–13. The `sc work` ephemeral cache (spec §5 by implication of "per-checkout") is Task 7. +- **Ordering is load-bearing:** dispatch (4–5) before the flip (6); anything reordered mass-re-seals and breaks the suite. +- **Known adaptation points** are marked with `NOTE:` blocks (ObjectId hex API, `random_hex` contract, error variant, test-helper names) — resolve them by reading the named anchor, not by inventing new surface. diff --git a/docs/superpowers/specs/2026-07-10-p33-randomized-protected-encryption-design.md b/docs/superpowers/specs/2026-07-10-p33-randomized-protected-encryption-design.md new file mode 100644 index 00000000..cb8a9745 --- /dev/null +++ b/docs/superpowers/specs/2026-07-10-p33-randomized-protected-encryption-design.md @@ -0,0 +1,191 @@ +# P33: Randomized protected-path encryption (dual-read, randomized-write) + +- **Issue:** #40 (part of the #24 audit-response map; decisions locked in #30) +- **Date:** 2026-07-10 +- **Status:** Approved design, pre-implementation + +## Goal + +Close the convergent-equality boundary (ADR-0014's accepted dictionary / +equality-confirmation oracle on protected paths). All **new** protected +content is sealed under a fresh random DEK + random nonce, so two seals of +the same plaintext yield different ciphertext ids — the oracle is closed for +everything sealed from this phase on. Convergent ciphertext already in +history stays readable forever (dual-read); nothing is rewritten or forcibly +migrated. + +## Locked decisions honored (from #30 / issue #40) + +1. Randomized seals for all new protected content, with a format tag. +2. Dual-read, randomized-write; no hard break; a rewrap-shaped one-command + eager re-seal upgrades the live tip on demand. +3. Unchanged-detection at commit via a local stat cache + keyed-hash + fallback; commit still needs only public keys. +4. Accepted costs: (a) independent identical edits on two branches now + conflict; (b) `sc rewrap` is no longer policy-only/tree-identical when it + randomizes content; (c) identical plaintext at two paths no longer dedups. +5. Rotate-for-paths becomes meaningful; recorded as a follow-on, not built. + +## Design decisions made in this brainstorm + +- **Format tag = perms-byte flag.** A `RANDOMIZED` bit alongside `PROTECTED` + in the tree-entry perms byte. Blob layout and `decrypt_path` are untouched. +- **`sc rewrap` itself performs the eager upgrade** (no new command, no flag). +- **The stat cache is per-checkout, including ws workspaces**, keyed by one + per-repo local key. +- **The commit rule is format-dispatched, not cache-only** (see §3) — this is + what prevents a silent mass-migration commit on first use after upgrade. + +## 1. Seal primitive (`crates/crypto`) + +New `encrypt_path_randomized(plaintext) -> (blob, dek)`: fresh random DEK and +random nonce from `OsRng`, plus an `encrypt_path_randomized_with_rng` variant +for deterministic tests. Blob layout is the existing `nonce(24) ‖ +AEAD-ciphertext` with the existing `PATH_AAD` — both formats decrypt through +the unchanged `decrypt_path`, which is why dual-read needs zero read-path +code. `encrypt_path` (convergent) remains for the legacy-format +unchanged-comparison only; no new seal call site uses it. Wrapping/unwrapping +(`wrap_dek_for` / `unwrap_dek_with`) unchanged. The RustCrypto quarantine +holds: only `crates/crypto` grows crypto code. + +## 2. Format tag (`crates/core`) + +A new perms bit, `RANDOMIZED` (always set together with `PROTECTED`), on +every newly sealed tree entry. No encoding change and no snapshot-tag bump: a +pre-P33 store decodes with zero `RANDOMIZED` bits and behaves byte-for-byte +as today. Carry-by-id preserves the source entry's perms (the P24 +discipline), so the format bit rides through merge, replay, graft, and sparse +carry with no new plumbing. Format identification never requires fetching +blob bytes. + +Old binaries reading a new store mask-check `PROTECTED` and would misreport +randomized entries as modified in `status`; accepted (no deployed old peers, +same posture as prior phases). + +## 3. Unchanged detection (new `crates/repo/src/cache.rs`) + +- `.sc/local-key`: 32 random bytes, mode 0600, created lazily, never + committed, never transferred. +- Per-checkout cache files: `.sc/protected-cache` for the main working tree; + `.sc/ws/cache-` per ws workspace — beside the checkout dir, NOT inside + it (a file inside `.sc/ws//` would be read back as an untracked working + file at harvest), removed with the workspace at teardown. Entries: `path → (mtime, size, keyed_tag, + blob_id)` where `keyed_tag = BLAKE3-keyed(local_key, plaintext)`. + +Commit's per-protected-path rule (format-dispatched): + +| Prior tip entry | Rule | +|---|---| +| **Convergent** (no `RANDOMIZED` bit) | Today's convergent re-encrypt-and-compare. Unchanged → carry as-is (stays convergent, history stays quiet with no cache needed). Edited → seal randomized. | +| **Randomized** | Stat hit (mtime+size) → carry cached `blob_id`. Stat miss → keyed-tag compare; match → carry + refresh entry; mismatch or no entry → seal randomized, update entry. | +| **New protected path** | Seal randomized. | + +Degradation: a lost or stale cache produces spurious re-seals (new blob ids +for unchanged plaintext), never incorrectness. The cache file alone leaks +nothing — tags are PRF outputs under a key that never leaves `.sc/` — so the +cache does not reintroduce the equality oracle. Commit continues to need +only public keys; decrypt-and-compare is not used anywhere. + +## 4. Status / diff + +`diff_worktree` (`crates/repo/src/worktree.rs`) and `diff_unified` +(`crates/repo/src/repo.rs`) get the same format dispatch: convergent entries +keep the re-encrypt-and-compare trick; randomized entries consult the cache +**read-only** (stat, then keyed tag); a missing entry reports the path as +modified — spurious-but-safe, same degradation rule as commit. + +## 5. Cache population + +Every site that writes authorized plaintext to disk records a cache entry: + +- checkout / `sc switch` (materialize) +- merge / pick / rebase completion materialization +- `sc resolve` writes +- ws fork workspace materialization (per-workspace cache) +- sparse re-lay (`sc sparse set` / `disable`) +- commit itself (authoritative refresh) +- `sc rewrap` (it holds the plaintext while upgrading; see §7) + +Per-workspace caches preserve two P20 properties that a main-tree-only cache +would regress: untouched protected files in forked workspaces remain provably +unchanged (no false identical-edit conflicts at harvest, no spurious +`work-` fallbacks), and crash-recovery re-harvest still converges to the +idempotent `UpToDate` no-op because cache hits reproduce the same tree id. + +## 6. Merge / replay semantics (the accepted P15 adjustment) + +- Unchanged and one-side-changed fast paths survive unchanged via + carry-by-id. +- Both sides editing a protected path — **including identical edits, which + under convergent encryption used to id-match and fast-path** — now + conflict by construction. Resolution: P23 `sc conflicts` / `sc resolve`, + or diff3-with-`--identity`, which merges identical plaintexts cleanly and + re-seals the result randomized. +- The content-merge re-encryption site in `merge.rs` and replay's + `encrypt_protected` path switch to the randomized primitive. +- Protection-rule union, wrap union, and tombstone semantics (P15/P16) are + unchanged. Mixed convergent+randomized trees need no special casing: every + path is carried by id or re-sealed randomized. + +## 7. `sc rewrap` eager upgrade + +While unwrapping each protected blob's DEK (by wrap presence, as today), +rewrap additionally decrypts any still-convergent blob and re-seals it +randomized (fresh DEK + nonce, `RANDOMIZED` bit set), replaces the wrap list +with the rule's `granted_keys() + escrow` as before, and populates the +main-tree cache with the plaintext it already holds. Output gains a +`re-sealed N convergent blob(s)` line; the commit is no longer tree-identical +when any blob was upgraded (accepted cost 4b). Already-randomized blobs get +wrap-replacement only — ciphertext id unchanged — so a second rewrap +converges back to policy-only. One command, one commit, one oplog record, +skip-and-report semantics unchanged. + +## 8. Out of scope (recorded, not built) + +- Rotate-for-paths (unlocked follow-on; ADR records that ADR-0019's + objection dissolves for randomized content). +- Per-prefix convergent/randomized choice (rejected on #30). +- `sc secret` changes (already randomized) and the P28 low-entropy nudge + (stays — steering low-entropy values to `sc secret` remains right). +- History rewriting or forced migration. + +## 9. Error handling + +- Sealing to an empty granted set still fails loudly + (`encrypt_protected`'s existing guard). +- A corrupt cache file is treated as absent (spurious re-seal, warning on + stderr), never an error that blocks commit. +- A missing `.sc/local-key` is minted lazily; a permissions failure creating + it is a hard error (the cache cannot be safely keyed without it). +- `decrypt_path` failures keep the P15 corruption-vs-authorization + distinction (`Error::Crypto` vs `NotAuthorized`). + +## 10. Testing & proof + +Tests (per the acceptance criteria on #40): + +- Randomized seal round trip; equality-oracle regression (two seals of the + same plaintext yield different blob ids). +- Dual-read of pre-phase convergent objects (fixture built with + `encrypt_path`). +- Format-dispatched commit: unchanged convergent content carries convergent + with no cache; unchanged randomized content carries via stat hit, via + keyed-tag fallback, and re-seals (correctly, spuriously) on lost cache. +- Eager tip re-seal via rewrap: convergent→randomized, second rewrap + policy-only, skip-and-report on unopenable entries. +- The new identical-edit conflict resolving via `sc resolve` and via + diff3-`--identity`. +- Merge/replay/rewrap over mixed convergent+randomized trees. +- ws fork/harvest with untouched protected files: no false conflicts; the + per-workspace cache reproduces the base tree id. + +Demos: all existing protected-path demos stay green over mixed-format +stores; new `demo/run_randomized_demo.sh` proves the oracle is closed (same +plaintext, different ciphertext ids), old history still decrypts, and rewrap +upgrades the tip — run twice, zero residue. + +Docs: ADR-0043 (format change, dual-read posture, cache design and why it +doesn't reintroduce the oracle, accepted costs 4a–c, P15/P17 semantic +adjustments, rotate-for-paths follow-on); THREAT-MODEL and +CLAUDE.md/ARCHITECTURE.md updated — ADR-0014's caveat superseded for new +content.