Skip to content

P33: randomized protected-path encryption (dual-read, randomized-write) - #43

Merged
tonibergholm merged 22 commits into
mainfrom
p33-randomized
Jul 11, 2026
Merged

tonibergholm merged 22 commits into
mainfrom
p33-randomized

Conversation

@tonibergholm

Copy link
Copy Markdown
Member

Closes #40. Implements the locked decisions from #30: the ADR-0014 convergent-equality oracle is closed for all newly sealed protected content.

What shipped

  • Randomized seals everywhere new content is sealed: encrypt_protected (the single production choke point — commit, merge completion, replay) now seals under a fresh random DEK + nonce via encrypt_path_randomized. Blob layout (nonce ‖ ct) and decrypt_path are unchanged, so dual-read costs zero read-path code.
  • Format tag = RANDOMIZED perms bit beside PROTECTED in the tree entry. No snapshot-tag bump, no wire change (PROTOCOL_VERSION stays 3) — a pre-P33 store behaves byte-for-byte as before.
  • Format-dispatched commit rule (the anti-mass-migration mechanism): unchanged convergent priors carry via the exact re-encrypt-and-compare proof and stay convergent forever; unchanged randomized priors carry via a .sc-local cache; everything else seals randomized.
  • Per-checkout unchanged-detection cache (.sc/protected-cache, .sc/ws/cache-<i>, ephemeral for sc work) keyed by a never-committed, never-transferred 0600 local key. The keyed BLAKE3 tag is authoritative — a bare stat hit is never trusted (closes the git racy-stat silent-edit-drop class; deliberate tightening of Decide: convergent-equality boundary — randomized protected mode or defer #30 decision 3's letter, recorded in ADR-0043). Lost/corrupt cache degrades to spurious re-seals, never incorrectness, never a hard error.
  • sc rewrap eagerly upgrades still-convergent tip blobs to randomized (re-sealed N convergent blob(s) → randomized); a second rewrap converges back to P17's policy-only/tree-identical property.
  • Accepted costs landed as specified (ADR-0043): independent identical edits now conflict (--identity resolves); rewrap is tree-changing when it upgrades; identical plaintext at two paths no longer dedups.

Review process highlights

  • Task-review Critical fixed: rewrap wrap-orphaned the second of two paths sharing one convergent blob (pre-P33 dedup) — closed via deferred old-id wrap removal; covering test proven non-vacuous by re-introducing the bug.
  • Security-review finding fixed: local-key file now 0600-from-first-byte (create_new + mode, tlsio discipline).
  • Review-confirmed behavior change, documented with operator guidance: sc protect <existing> --to <new> no longer wraps unchanged content at the next commit (under-grants only; sc grant/sc rewrap are the covering flows).
  • Final whole-branch review (no Criticals): READY TO MERGE; its one Important (THREAT-MODEL wording implying rewrap ends equality-confirmability of historical convergent content — it cannot; ADR-0019 boundary) fixed.

Proof

  • 694 workspace tests green, clippy zero warnings, fmt clean.
  • New demo/run_randomized_demo.sh (oracle closed via matched-pair object-count deltas, quiet history, cost-4a end to end, rewrap policy-only, zero residue, run-twice). All existing protected-path demos green. (run_protect_demo.sh fails pre-existing on main — stale pre-P8 objects glob; separate ticket.)
  • Dual-read of genuine convergent stores is pinned by library-built unit fixtures (the CLI can no longer mint convergent seals — proof split recorded in ADR-0043).

Docs: ADR-0043, THREAT-MODEL, CLAUDE.md Phase 33 section, ARCHITECTURE.md, ROADMAP (+4 named follow-ons, incl. rotate-for-paths now unlocked).

https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB

local_key() now creates .sc/local-key 0600 from the first byte via
create_new (mirroring crates/tlsio/src/identity.rs), closing the
write-then-chmod window, and falls through to reading the winner's
file on a concurrent create_new race instead of erroring or
clobbering it. ProtectedCache::save() drops its hand-rolled
temp+rename in favor of scl_core::fsutil::atomic_write_durable, the
same fsync-backed helper every other .sc-local writer in this crate
uses.

Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
Flips encrypt_protected (crates/repo/src/protect.rs) to
scl_crypto::encrypt_path_randomized, tagging every fresh seal
PROTECTED | RANDOMIZED. This is the single choke point all three
callers (commit's to_seal, merge completion, replay) go through.

Flipping it surfaced that worktree::materialize never wrote to the
P33 ProtectedCache: Task 3/4 wired the cache's read side into
diff_worktree and commit's carry decision, but the write side was
never wired into checkout/switch/merge/rebase materialization.
Invisible pre-flip (convergent seals self-verify by re-encrypting),
this became a real bug the moment seals stopped being convergent —
a freshly checked-out, byte-identical randomized protected file had
no cache entry, so status/diff spuriously reported it modified,
which several callers (merge, rebase, sparse set/disable) treat as a
hard "uncommitted changes" precondition failure.

Fixed by threading a cache: Option<&mut ProtectedCache> through
materialize (worktree.rs), recording each RANDOMIZED entry right
after its plaintext write, and updating all 15 call sites across
repo.rs, replay.rs, sparse.rs, oplog.rs, sync.rs, and workspace.rs
(a new Repo::open_protected_cache() helper opens/saves the
host-repo cache at each site that has an identity to decrypt with;
sites with no identity pass None since the record call is
unreachable there anyway).

Full workspace suite: 14 of 15 failures were this same dispatch bug
(fixed, no test changes); 1 was a genuine new-seal-convergence
assertion the spec deliberately breaks (accepted cost 4c) —
convergent_merge_ids_are_stable_across_repos renamed/inverted to
merge_ids_no_longer_converge_across_repos, citing ADR-0043/P33.

Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
Thread a per-checkout ProtectedCache through the workspace fork/harvest
seam so an untouched randomized protected path carries the base ciphertext
blob instead of re-sealing (the false-conflict hazard across ws workspaces).
ProtectedCache now carries its own abs-path root (set at open), so
snapshot_files stats the workspace's files, not the host's. Normalize every
cache save to best-effort after the ref moves; abort paths pass no cache;
narrow materialize/diff_worktree to pub(crate).

Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
ws_changed_for passed None as the ProtectedCache to diff_worktree, so an
untouched RANDOMIZED protected file in a workspace always reported
"changed". That fed any_changed in ws_harvest's preflight, refusing the
whole harvest for a dirty host tree even when every workspace was a
genuine zero-edit no-op, defeating the documented affordance. Now opens
the workspace's own persistent cache (same open-call shape as
ws_harvest), degrading to None only if local_key can't be loaded.

Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
ProtectedCache::unchanged() previously returned a cache hit on a bare
mtime+size stat match without checking the keyed tag. An edit that keeps
the same file size and lands within the filesystem's mtime granularity
(the classic git "racy stat" problem) was misread as unchanged, silently
carrying the old ciphertext into the commit and dropping the user's edit.
The stat short-circuit is removed; every unchanged() call now requires
keyed-tag equality, which every caller can afford since the plaintext is
already in memory. mtime/size are still recorded per entry as advisory
bookkeeping only.

Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
- cargo fmt --all across ~6 files left non-canonical by earlier P33 commits
- merge.rs: tighten the content-divergent re-seal test's final perms
  assertion from a loose bitmask check to an exact PROTECTED|RANDOMIZED
  equality, pinning the RANDOMIZED bit no other test covers
- ws.rs: correct the HarvestResult::Unchanged arm's comment — it is
  unreachable in practice (harvest re-diffs the same persisted cache
  state moments after ws_changed_for already confirmed a diff), kept
  only for match exhaustiveness
- run_randomized_demo.sh: note that the RETURN trap removes the cwd,
  so later paths in run_demo must stay absolute

Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
…ally sets

The prior cleanup commit tightened this test's final perms assertion to
PROTECTED | RANDOMIZED, assuming three_way_files surfaces the RANDOMIZED
bit for a content-divergent (needs_encrypt) protected entry. It doesn't:
repo.rs's completion path never reads f.perms for a needs_encrypt entry
(only bytes/mode/recipients), so RANDOMIZED is applied downstream by
encrypt_protected at actual re-encryption time. f.perms here only ever
carries the PROTECTED marker bit (see the content-divergent match arm's
mask). Corrected the assertion to PROTECTED, with a comment explaining
why RANDOMIZED can't be pinned at this call site — that coverage would
need a commit-then-inspect-snapshot test instead.

Caught by cargo test during the P33 validation sweep.

Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB
@tonibergholm
tonibergholm merged commit bb6d435 into main Jul 11, 2026
6 checks passed
@tonibergholm
tonibergholm deleted the p33-randomized branch July 11, 2026 06:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

P33: randomized protected-path encryption (dual-read, randomized-write)

1 participant