From a11da0688f3846b0c008e203bfee4b450e4b1ce6 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Wed, 21 Oct 2026 10:08:13 +0300 Subject: [PATCH 1/3] docs: final sync with the implementation (pass 3) Audit of the living docs against the code at origin/main (through #283): ROADMAP Done/Deferred now match the open issues (#281 only), the #278/#280 file/directory collision guard replaces the stale open-hazard text, the DirBucket OS lock, compact --dry-run cost, ws run lock release and a single downgrade-hazard list are written down, demo count and ws/refresh/signature index/index lock/ADR status lines are corrected, and the sc-cli-reference skill is verified against sc --help. --- .claude/skills/sc-cli-reference/SKILL.md | 51 +++++-- ARCHITECTURE.md | 68 +++++++--- CLAUDE.md | 47 ++++++- CONTRIBUTING.md | 9 +- README.md | 26 ++-- ROADMAP.md | 80 +++++++++-- SECURITY.md | 4 +- docs/THREAT-MODEL.md | 166 ++++++++++++++++++++--- docs/adr/README.md | 10 +- 9 files changed, 371 insertions(+), 90 deletions(-) diff --git a/.claude/skills/sc-cli-reference/SKILL.md b/.claude/skills/sc-cli-reference/SKILL.md index bc98adc7..72e376d1 100644 --- a/.claude/skills/sc-cli-reference/SKILL.md +++ b/.claude/skills/sc-cli-reference/SKILL.md @@ -13,7 +13,7 @@ the ADRs under `docs/adr/` and the design in `ARCHITECTURE.md`. ```sh cd apps/desktop && npm run tauri dev # native desktop development app cd apps/desktop && npm run tauri build -- --bundles app # validated macOS production bundle -cargo run --bin sc -- demo --agents 4 # parallel-agent demo +cargo run --bin sc -- demo --agents 4 # parallel-agent demo (defaults: --agents 4, --budget-mb 8) cargo run --bin sc -- demo --agents 6 --budget-mb 4 --spill # exercise eviction cargo run --bin sc -- demo --agents 4 --checkout # also materialize each worktree to a session # temp dir (the one disk-writing path), then clean up @@ -24,6 +24,8 @@ cargo run --bin sc -- keygen [--out ] # (default ~/.sc/identity; 0600 on # a pre-P22 v1 (scl-sk-) identity still parses and # still encrypts, but cannot sign cargo run --bin sc -- secret-demo # committed-secrets authorize/deny/grant proof +# Identity default everywhere a command takes --identity: the flag, then $SC_IDENTITY, +# then ~/.sc/identity (a missing file just skips protected paths where decryption is optional). # Persistent repo commands (sc init creates .sc/ in the working directory): cargo run --bin sc -- init # initialize a new persistent repo @@ -107,7 +109,7 @@ cargo run --bin sc -- run [--secrets-fd N] -- [args…] # inject decrypt # NUL-terminated NAME=VALUE records instead of env vars # (SC_SECRETS_FD names N); ADR-0052 cargo run --bin sc -- gc # pack reachable objects + prune unreachable -cargo run --bin sc -- gc --prune-expire 7d # custom grace window +cargo run --bin sc -- gc --prune-expire 7d # custom grace window (default 24h) cargo run --bin sc -- export --to # write current branch history to Git cargo run --bin sc -- export --to --ref # update this Git ref instead of # refs/heads/ @@ -117,7 +119,7 @@ bash demo/run_remote_demo.sh # local-path remote round trip (P6) # fetch+merge and push between two local repos cargo run --bin sc -- remote list # name, URL and kind ([sc] / [git]) of each remote cargo run --bin sc -- remote add --git # register a git-backed remote -cargo run --bin sc -- fetch # import git history -> remote-tracking ref +cargo run --bin sc -- fetch [] # import git history -> remote-tracking ref (remote defaults to origin; same for push) cargo run --bin sc -- push [--include-encrypted] # export sc history -> git ref (ff-only) bash demo/run_git_remote_demo.sh # git-as-a-remote round-trip proof cargo run --bin sc -- remote add ssh://[user@]host[:port]/path # ssh-native remote @@ -153,13 +155,17 @@ bash demo/run_randomized_demo.sh # randomized protected-enc # rewrap UPGRADE and pre-P33 dual-read are unit- # test-pinned, since the current binary can no # longer write a convergent blob) -cargo run --bin sc -- work --agents 3 [--name ] -- # fork agent workspaces, run - # in each, harvest changed ones to - # - branches (default base `work`) +cargo run --bin sc -- work --agents 3 [--name ] [--budget-mb ] -- # fork agent + # workspaces (default 2), run in each, + # harvest changed ones to - branches + # (default base `work`); --budget-mb bounds the + # session's shared object cache # (--with-secrets --identity injects # decrypted secrets into each agent env) bash demo/run_work_demo.sh # parallel-agents round-trip proof -cargo run --bin sc -- ws fork --agents N [--name ] [--identity ] # durable multi-invocation +cargo run --bin sc -- ws fork --agents N [--name ] [--identity ] [--author ] # durable multi-invocation + # (--agents default 2; --author is recorded on any + # commit a later harvest makes) # session: materializes N checkouts under # .sc/ws// and persists a manifest (P20). # --name opens a further named session in @@ -168,7 +174,9 @@ cargo run --bin sc -- ws fork --agents N [--name ] [--identity ] # dura cargo run --bin sc -- ws list [--session ] # every open session's workspaces, changed/unchanged # (or just session 's) cargo run --bin sc -- ws run [--session ] [--with-secrets --identity ] -- # run in - # workspace i (SC_WORKSPACE/SC_WORKSPACE_DIR set) + # workspace i (SC_WORKSPACE/SC_WORKSPACE_DIR set); + # the repo lock is released while runs + # (ADR-0056), so other sc commands stay usable cargo run --bin sc -- ws harvest [--session ] [--into ] [--identity ] # probe + auto-merge # every live workspace onto the landing branch # (default the session's base branch, which must @@ -191,7 +199,12 @@ cargo run --bin sc -- ws refresh [--session ] [--onto ] [--identity < # An interrupted refresh leaves a marker: harvest # refuses until `ws refresh` is run again to finish it cargo run --bin sc -- ws abandon [--session ] [] # drop one workspace or the whole session -bash demo/run_ws_demo.sh # multi-invocation session + auto-merge + refresh proof (P20) +bash demo/run_ws_demo.sh # durable-session proof (P20): fork, edit across separate + # invocations, cumulative auto-merge harvest with a + # conflicting fallback branch, undo/redo of landings, + # two named sessions at once (ADR-0056), `ws refresh` + # onto a newer tip keeping edits; ends with zero + # `.sc/ws*` residue cargo run --bin sc -- cherry-pick [--identity ] # replay one commit onto the # current branch (--identity as above) cargo run --bin sc -- cherry-pick --mainline # replay a merge commit relative @@ -372,7 +385,12 @@ cargo run --bin sc -- clone sc+http://host[:port]/repo # clone over sc+h # accept the same sc+http:// URL form cargo run --bin sc -- remote add sc+wal:// # bucket WAL remote on a local # directory (P36, ADR-0046); `clone` takes the URL - # directly, `fetch`/`push` use the remote's name + # directly, `fetch`/`push` use the remote's name. + # Local filesystem, one owner: the CAS is an OS file + # lock on /.cas-lock that waits without a + # timeout (not reliable on NFS; binaries from before + # #264 do not exclude new ones; use S3 for several + # machines) cargo run --bin sc -- remote add sc+s3:/// # same, on an S3-compatible # store; a malformed URL (e.g. no bucket) is rejected # at `remote add` time. Public content is plaintext at @@ -387,7 +405,8 @@ cargo run --bin sc -- remote compact [--dry-run] [--prune-expire 24h] [-- # are older than --prune-expire (same flag and default # as `sc gc`; keep it well above any push's duration). # Needs bucket list + delete rights. --dry-run prints - # the plan and writes nothing. Safe beside live + # the plan and writes nothing, but still downloads + # and decodes every candidate pack (S3 egress). Safe beside live # pushes/fetches within --prune-expire (a writer # stalled longer than that between uploading a pack # and committing it can lose the pack); run one at a time. A --prune-expire under 1h is refused @@ -442,7 +461,8 @@ cargo run --bin sc -- clone --filter --lazy cargo run --bin sc -- backfill --lazy [prefix…] # mark an existing partial clone lazy (P37); with # prefixes, also backfills them; conflicts --all -cargo run --bin sc -- cat : # print one file as of a ref (P37). On a lazy +cargo run --bin sc -- cat : # print one file as of a ref (P37; is a branch + # or remote-tracking ref like origin/main, not HEAD). On a lazy # partial clone fetches only the absent trees and # the blob on that path (one GetObjects round trip # per level). SC_OFFLINE=1 forces the gap error. @@ -454,7 +474,8 @@ bash demo/run_lazy_fetch_demo.sh # by-id fetch proof (P37a): cat of # GetObjects bash demo/run_blob_limit_demo.sh # blob-size filter proof (ADR-0055): old large # blob left out, tip checkout whole, verify/gc/ - # push fine, --lazy, prefix+limit, backfill repair + # push fine, --lazy, prefix+limit, backfill repair, + # a later fetch keeps the limit unless --no-blob-limit bash demo/run_partial_clone_demo.sh # partial clone proof (P27): sc clone --filter # src/ fetches fewer objects than a full clone # (object-store count + sc verify's gap report); @@ -488,6 +509,7 @@ cargo run --bin sc -- serve fingerprint [] # print (minting if absent) t # (.sc/) — found from the cwd, or cargo run --bin sc -- clone sc+https://host[:port]/repo # TLS clone (P32); accept-new # TOFU: first connect pins into + # $XDG_CONFIG_HOME/sc/known_hosts, else # ~/.config/sc/known_hosts (SC_HTTPS_KNOWN_HOSTS # overrides), mismatch always hard-fails; # SC_HTTPS_FINGERPRINT= pre-pins (CI), @@ -497,7 +519,8 @@ cargo run --bin sc -- clone sc+https://host[:port]/repo # TLS clone (P32 # pinning (no pin file read/written; ADR-0054); # remote add/fetch/push accept the same URL form cargo run --bin sc -- tls list [--json] # pins in the known_hosts file (ADR-0053) -cargo run --bin sc -- tls forget # drop a host's pin (changed server key); a later +cargo run --bin sc -- tls forget # drop every pin for a host (port defaults to 8730; + # changed server key); a later # accept-new connect (no SC_HTTPS_STRICT, no # SC_HTTPS_FINGERPRINT) re-pins; errors if nothing # was pinned diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 5492e886..e301a77e 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -92,7 +92,7 @@ src-control/ │ reference is the sc-cli-reference skill) ├── apps/desktop/ Tauri v2 top-level adapter (`scl-desktop`) + bundled React/TypeScript │ renderer; native read model only, with no UI dependency in domain crates -├── demo/ 27 end-to-end proof scripts (`run_*.sh`) +├── demo/ 28 end-to-end proof scripts (`run_*.sh`) ├── fuzz/ cargo-fuzz target for the wire decoder (`wire_decode`) └── ARCHITECTURE.md ``` @@ -333,10 +333,16 @@ phases but the shape is stable: scanner (P5). - `oplog` — append-only operation log; every ref-moving op records before/after so `sc undo`/redo can invert it (P14). Local-only, like a reflog. -- `signatures` — snapshot→signature-object index (P22); `transcripts` — - snapshot→transcript-object index, one-to-many (P30). Both gc-rooted, local-only. - `transcript-drops` — the derived set of dropped transcripts (`sc transcript - drop`), rebuilt from the store on clone. +- `signatures.v2` — the signature index, one slot per (snapshot, signer) (P22, + #257); `signatures` is the pre-#257 file, read only by the one-time import + (retried on later ingests if it failed) and left alone so an older binary keeps + a file it understands. `transcripts` — snapshot→transcript-object index, + one-to-many (P30). All gc-rooted, local-only. `transcript-drops` — the derived + set of dropped transcripts (`sc transcript drop`), rebuilt from the store on + clone; `transcript-pending` (+ `transcript-pending-trust`) — signed tombstones + held but not yet honored, re-judged when the trusted-signer set changes (#230). +- `index.lock` — advisory lock around one update of these derived indexes, taken + by serve-path ingests and local commands; distinct from `lock` (#260). - `sparse` — the local, uncommitted sparse-checkout prefix spec (P24); `promisor` — a partial clone's fetch-filter + origin (P27); `promisor-blob-limit` — its blob-size limit and whole-tree-or-prefix scope (ADR-0055). @@ -351,7 +357,9 @@ phases but the shape is stable: for protected paths; never committed, never transferred. - `tmp/` — guard-cleaned scratch for streaming pack transfer (P25). - `ws/` — durable agent-session workspaces: `session.toml` manifest + per-agent - checkouts (P20); `ws-/` — the same for a named session (ADR-0056). `git-remotes//` — the `git_oid ↔ sc_id` marks map and a + checkouts (P20); `ws-/` — the same for a named session (ADR-0056); the + manifest also carries the `pending_refresh` marker of an unfinished `sc ws + refresh`. `git-remotes//` — the `git_oid ↔ sc_id` marks map and a lazily-built bare `mirror.git` for network Git (P10/P18). - Transient in-progress state (removed on completion/abort): `MERGE_HEAD`, `PICK_HEAD`, `REBASE_STATE`, and their `*_DECIDED_ROOT` records, plus @@ -379,6 +387,19 @@ plain file, and setuid/setgid/sticky are always cleared (#242, #243). `sc resolve` applies the chosen side's mode after rewriting a deleted conflicted file. A chmod the filesystem refuses is tolerated with a one-time warning. +`materialize` treats the disk as the user's wherever it cannot prove otherwise. +A forward operation (merge, switch, cherry-pick, rebase, undo, sparse-spec change) +refuses, before any write, to lay a file or directory onto an untracked or ignored +one. A tracked path is deleted or overwritten only when the disk copy is a regular +file with exactly the tracked bytes, so an edit, a replacement or a symlink +survives; an emptied directory is cleared so a file and a directory can swap at one +path, and every such directory is checked before the first removal, so a refusal +changes nothing. An abort (`merge_abort`, `cherry_pick_abort`, `rebase_abort`) +always completes and returns an `AbortReport`: it removes what the stop laid, +overwrites a file only if it is still the stopped tree's bytes or the stop owns it +(recorded plaintext, a conflicted path), and lists what it left in `kept` (`sc` +prints `left in place (changed since the stop)`). See ADR-0034 refinement 8. + ### Mode-scoped disk invariant **Ephemeral mode** (Phase 1 agents, `sc demo`, `sc secret-demo`) keeps the @@ -402,7 +423,11 @@ Remote ref updates are compare-and-swap: `Transport::update_ref` revalidates the expected old tip under the remote's own lock, so two racing pushes cannot silently clobber each other. The single-writer lock file records the holder's PID and is broken automatically when that process is provably dead, so a -SIGKILLed `sc` doesn't brick the repo. See ADR-0021. +SIGKILLed `sc` doesn't brick the repo. The derived indexes (signatures, +transcripts and their drop state) take a separate, short `index.lock`, held only +around one update and never while waiting on the repo lock, so a serve-path ingest +and a clone's reindex cannot lose each other's appended lines (#260, #273, #277). +See ADR-0021. ## Phase 1 deliverable and proof @@ -566,8 +591,10 @@ requires rotating the underlying external credential too. See ADR-0019. ## Phase 12 — Network transport over SSH (built) `sc clone / fetch / push` work against `ssh://[user@]host[:port]/path` -remotes. The wire protocol mirrors the 8 `Transport` verbs over length- -prefixed frames with a version handshake; the server (`sc serve --stdio`) is +remotes. The wire protocol mirrors the `Transport` verbs (eight here; the batched +`GetObjects` joined in wire v5) over length-prefixed frames with a version +handshake (`PROTOCOL_VERSION` is 6 today; each bump is noted in the phase that +made it, and a mismatched pair fails closed at the handshake); the server (`sc serve --stdio`) is a dispatch loop around `LocalTransport`, so CAS ref updates and pack verification apply verbatim server-side. The client spawns the user's `ssh` (overridable via `SC_SSH`, Git's `GIT_SSH` pattern — the demo and tests drive @@ -586,8 +613,8 @@ blob cache and eviction never needs a spill backend — `.sc/objects` is the reconstruction source. Checkout reuses the P7-aware `materialize`; harvest reuses the commit pipeline (`snapshot_files`, extracted from `commit`), so the P5 scanner and `.scignore` gate agent output exactly like a human -commit. Each changed workspace becomes a flat `work-` branch (the ref -grammar reserves `/` for remote-tracking refs); merge is the ordinary P4 +commit. Each changed workspace becomes a flat `work-` branch (`-` with +`sc work --name `; the ref grammar reserves `/` for remote-tracking refs); merge is the ordinary P4 path. The session holds the single-writer lock end to end, and teardown is Drop-guarded: zero residue outside `.sc/` on success, error, or panic. @@ -807,8 +834,11 @@ a write. Applying touches only differing paths, each written atomically and re-checked against what the candidate read right before it is touched; a `pending_refresh` marker (target, its branch and, per workspace, the candidate and result snapshots) is written to the manifest before the first file, and the -manifest with the new base and no marker last. A crash leaves the marker: -harvest and run refuse, and `sc ws refresh` resumes from the recorded pairs +manifest with the new base and no marker last. While the marker is set the +manifest's `base_snapshot` holds the placeholder `refresh-unfinished` (the real +base is inside the marker), so a binary that predates refresh cannot read the base +and fails closed instead of harvesting a half-moved session. A crash leaves the +marker: harvest and run refuse, and `sc ws refresh` resumes from the recorded pairs (accepting at each path the candidate's bytes or the result's, never re-reading the half-written checkouts) before moving to the requested tip. Session-wide only. @@ -893,7 +923,12 @@ newly in-view files from the stop's decided root, leaves protected ones absent, and the interrupted write is journaled in `.sc/sparse-widen` (#224). Sparse is CHECKOUT-only — every object stays in the CAS regardless of the spec, so `sc gc`'s reachability walk is unaffected; not fetching out-of-prefix objects at all is deferred to P27. -See ADR-0034. +A stopped operation records a decided root (public branches), and completion +carries an out-of-view or still-protected path from it alone, so a path the other +side deleted cleanly is not resurrected from a parent (#238/#239). A spec change +never counts a case alias of a laid-out path as tracked, since on a +case-insensitive filesystem the newly visible alias would be the laid-out file +itself (#241). See ADR-0034. ## Phase 25 — streaming pack transfer (built) @@ -1190,12 +1225,13 @@ as a write-ahead log of **immutable, content-addressed packs** plus **one compare-and-swapped manifest**. A push uploads a pack, appends a parent-linked `log/` entry carrying its ref updates, then CASes the manifest's `head_seq` forward (`Bucket::put_if_tag`). A lost CAS refreshes and retries (up -to 16 times), so pushes to different branches both land; a push whose target +to 16 times, with jittered exponential backoff between attempts, 2 ms doubling to +a 250 ms cap), so pushes to different branches both land; a push whose target branch no longer matches its expected old tip fails with `NonFastForward`, exactly as on any other transport, and a loser's orphaned log entry is invisible to readers because they walk `parent_seq` back from the manifest. There is no leader, lock file or lease. The new leaf crate `objio` holds the `Bucket` trait -and its two backends (`DirBucket` for `sc+wal://`, `S3Bucket` for +and its two backends (`DirBucket` for `sc+wal://`, whose CAS takes an OS advisory file lock (single host, local filesystem; see THREAT-MODEL), `S3Bucket` for `sc+s3:///`), quarantining the S3 SDK the way `gix` and RustCrypto are quarantined. diff --git a/CLAUDE.md b/CLAUDE.md index 374a56a8..53156b57 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -142,17 +142,17 @@ the code, those win. | P24 | Sparse checkouts: `.sc/sparse` local prefix spec; checkout-only (CAS keeps everything) | [0034](docs/adr/0034-sparse-checkouts.md) | | P25 | Streaming pack transfer: bounded RAM on both sides, `ST_PACK_CHUNK` frames (`SC_PACK_CHUNK` tunes), two-pass atomic-after-verify ingest | [0035](docs/adr/0035-streaming-transfer.md) | | P26 | `sc+http://` transport: `sc serve --http`, thread-per-connection, status-line-before-handshake opening | [0036](docs/adr/0036-http-transport.md) | -| P27 | Partial clone: `sc clone --filter `, `.sc/promisor`, `sc backfill [--all]`; wire `PROTOCOL_VERSION` 3. Blob-size filter (#217): `--filter blob:limit=` (combinable with prefixes; `blob:` is reserved) leaves out blobs of at least that size unless a ref tip reaches them, evaluated by the server (`GetPack.blob_limit`, wire v6); the gap is absence as in ADR-0048, `sc backfill ` or a lazy fill repairs it, `.sc/promisor-blob-limit` records limit and scope; `sc fetch` from the promisor origin sends the recorded limit again (`--no-blob-limit` opts out for one fetch); no object-count limit (rejected) | [0037](docs/adr/0037-partial-clone.md), [0055](docs/adr/0055-blob-size-clone-filter.md) | +| P27 | Partial clone: `sc clone --filter `, `.sc/promisor`, `sc backfill [--all]`; wire `PROTOCOL_VERSION` 3 when added (6 now). Blob-size filter (#217): `--filter blob:limit=` (combinable with prefixes; `blob:` is reserved) leaves out blobs of at least that size unless a ref tip reaches them, evaluated by the server (`GetPack.blob_limit`, wire v6); the gap is absence as in ADR-0048, `sc backfill ` or a lazy fill repairs it, `.sc/promisor-blob-limit` records limit and scope; `sc fetch` from the promisor origin sends the recorded limit again (`--no-blob-limit` opts out for one fetch); no object-count limit (rejected) | [0037](docs/adr/0037-partial-clone.md), [0055](docs/adr/0055-blob-size-clone-filter.md) | | P28 | Security hardening: a max object size (default 256 MiB; `--max-object-size` / `SC_MAX_OBJECT_SIZE`, 2 MiB to 1 GiB) caps every untrusted transfer length; strict ref-name validation | [0039](docs/adr/0039-security-hardening-sweep.md) | | P29 | `sc serve --http` access control: bearer tokens (`sc serve token add [--expires]`/`rotate`/`remove`/`list`; an expired token gets the same 401 as an unknown one), `--read-only`, fail-closed non-loopback bind gate | [0040](docs/adr/0040-sc-http-access-control.md) | | P30 | Sealed agent-session transcripts: `sc transcript attach/show/list/sign/drop`, always-encrypted body, gc-coupled index; `--no-transcripts` on fetch/clone/push (client-side, per command); drop = grow-only tombstone (a bare `Transcript` object, no wire change), kept by gc, hides the body from fetch/reindex; retained signed tombstones ride a transfer when their snapshot is in it or was collected by the sender, only verifying signatures are indexed, and held-but-untrusted tombstones are re-judged from `.sc/transcript-pending` when the trusted-signer set changes (#230); the signature index (`.sc/signatures.v2`) holds one slot per (target, signer) so attacker-made keys cannot crowd out a drop (#257), and serve-path index updates take a short `.sc/index.lock` distinct from the repo lock (#260) | [0038](docs/adr/0038-agent-session-transcripts.md) | | P31 | Listener resource limits: `--max-connections`, `--timeout`, `--max-pack-size` (both transports), capped read-only drain, accept backoff; later `--max-conn-bytes` per-connection write budget (#233) | [0041](docs/adr/0041-listener-resource-limits.md) | | P32 | In-binary TLS: `sc+https://` via leaf crate `tlsio`; accept-new TOFU pinning (`sc tls list/forget/pin/path` manages the pin file); `SC_HTTPS_CA_FILE`/`SC_HTTPS_CA=system` validate against a CA instead; client connect/handshake bounded; public plaintext bind no longer justified by tokens alone | [0042](docs/adr/0042-in-binary-tls-sc-https.md), [0053](docs/adr/0053-tls-pin-management.md), [0054](docs/adr/0054-tls-ca-validation.md) | | P33 | Randomized protected sealing (fresh DEK + nonce; `RANDOMIZED` perms bit); dual-read of pre-P33 convergent ciphertext; per-checkout keyed stat cache; `sc rewrap` upgrades convergent blobs at the tip | [0043](docs/adr/0043-randomized-protected-encryption.md) | -| P34 | Private branches: ref points at a sealed-branch manifest; every commit/tree/blob individually sealed (copy-on-write) under a per-branch KEK wrapped per recipient + escrow; `sc branch --private/grant/revoke/publish`; opaque to non-recipients (content, paths, messages); grant O(1), revoke rotates the KEK; publish replays to public with a scanner gate; git bridge + private→public integration refused; `PROTOCOL_VERSION` 4 | [0044](docs/adr/0044-per-branch-access-control.md) | +| P34 | Private branches: ref points at a sealed-branch manifest; every commit/tree/blob individually sealed (copy-on-write) under a per-branch KEK wrapped per recipient + escrow; `sc branch --private/grant/revoke/publish`; opaque to non-recipients (content, paths, messages); grant O(1), revoke rotates the KEK; publish replays to public with a scanner gate; git bridge + private→public integration refused; `PROTOCOL_VERSION` 4 when added | [0044](docs/adr/0044-per-branch-access-control.md) | | P35 | Native Tauri desktop browser: opens `.sc` repositories through `scl-repo`, shows local/remote refs, all-parent snapshot DAG + provenance, public trees and first-parent diffs; protected content is locked and private branches remain opaque; no mutation or identity surface | [0045](docs/adr/0045-native-desktop-read-model.md) | | P36 | Bucket WAL remotes (sc+wal://, sc+s3://): immutable packs + CAS'd manifest, checkpoints + log-tail cold start, bucket-backed `sc serve --store` with disposable instances; `get_pack` serves signatures, transcripts and signed drop tombstones by decoding the on-chain packs (#229, no format change); `sc remote compact [--dry-run] [--prune-expire 24h]` (#216) merges small packs behind one manifest CAS, then deletes log entries, checkpoints and unreferenced packs behind an aged-checkpoint cutoff, readers restart on a moved manifest, no format or wire change | [0046](docs/adr/0046-wal-bucket-remotes.md), [0057](docs/adr/0057-bucket-compaction.md) | -| P37a | Want-by-id fetch: batched `GetObjects` verb (wire v5), opt-in `.sc/promisor-lazy`, `sc cat :` fills only the absent trees and blob on its path; `Store::get` stays network-free; sealed objects refused by kind; sparse widen and the lifted P27 refusals are P37b | [0047](docs/adr/0047-want-by-id-fetch-and-gap-fill.md) | +| P37a | Want-by-id fetch: batched `GetObjects` verb (added in wire v5), opt-in `.sc/promisor-lazy`, `sc cat :` fills only the absent trees and blob on its path; `Store::get` stays network-free; sealed objects refused by kind; sparse widen and the lifted P27 refusals are P37b | [0047](docs/adr/0047-want-by-id-fetch-and-gap-fill.md) | | P37b-i | Presence-based partial clones: push, verify and gc walk what the store holds (`reachable_objects_partial`), not the fetch filter; push checks every gap id against the destination before sending; a view gap fails with `GapInView` before any write (preflight at the top of `materialize`, plus `undo` and `sparse set`); a partial clone fetches from its origin only. 37b-ii: on a lazy, online clone `ensure_view` fills the view closure by id before switch, fast-forward, undo and `sparse set`, `sparse set` may widen past the filter, a by-id reply is capped at `min(--max-pack-size, 2 GiB)` (a constant since #213, independent of the object cap) and a refused batch is split. | [0048](docs/adr/0048-presence-based-partial-clones.md) | | P37c | Merge, cherry-pick and rebase on a partial clone: a tree-recursive engine (`merge_trees`) carries a subtree by id when the sides agree and no protection rule applies, reads only where they differ or a rule intersects, and on a lazy, online clone fills what it asks for by id outside the store lock (a non-lazy clone gets the gap error with `sc backfill `); a carried protected subtree keeps the complete input wrap union; a conflict inside the view is marked and resolved like on a full clone (`sc conflicts`/`sc resolve`/`merge --abort`/`sc commit`; the out-of-view part is grafted from the decided root), while a conflict outside the view, or a result that must be sealed outside it, is refused before any ref, merge-state or working-tree change (fetched objects stay). A subtree identical on all three sides is carried even under a protection rule. Cherry-pick and rebase use the same engine (`replay_commit`): the fold's accumulator may hold absent subtrees, the view is filled before the final write, and a stopped pick or rebase keeps its carried subtrees in the decided root. Mode-only edits now merge and a file/directory collision fails before any write, on full clones too. | [0049](docs/adr/0049-merge-on-partial-clones.md) | | P37d | Agent sessions on a partial clone: `sc ws fork`/`run`/`harvest`/`abandon` and `sc work` work; a workspace or agent sees the host's sparse view only (`sc work` no longer gets a complete tree there), the view closure is filled (lazy, online) or refused with the gap error before anything is written, a workspace commit grafts the out-of-view subtrees from the base without reading them, and harvest lands through the ADR-0049 merge (any conflict falls back to a `work-` branch as on a full clone) | [0050](docs/adr/0050-sessions-on-partial-clones.md) | @@ -187,7 +187,7 @@ transport-adjacent. The rest, imperatively: identity material. Opening a repository can write the derived transcript-drop indexes (`.sc/transcript-drops`, `.sc/transcript-pending`) when the trusted signer set changed; no key material is involved (#230). -- **Partial clones refuse `sc export`, `sparse disable`, an empty `sparse set` (and `sparse disable`; both are allowed on a size-only `AllPaths` clone, ADR-0055), on a non-lazy clone a `sparse set` outside the filter, `sc rewrap`, +- **Partial clones refuse `sc export`, `sparse disable` and an empty `sparse set` (both are allowed on a size-only `AllPaths` clone, ADR-0055), on a non-lazy clone a `sparse set` outside the filter, `sc rewrap`, `sc grant`/`revoke`, creating, switching to or leaving a private branch, a merge, cherry-pick or rebase whose conflict lies outside the view (a `.theirs` sidecar sits beside its conflicted path, so it is in view whenever that path is; ADR-0049) (no ref, state or working-tree change; on every clone, a conflict stop that needs sealing outside the view is refused the same way), `sc push` to a git remote, and `sc fetch` from any native remote but the promisor origin.** `sc merge`, `sc cherry-pick` and `sc rebase` work, including conflicts inside the view (P37c). `sc ws` and `sc work` work too (ADR-0050): workspaces and agents see the clone's view only, and harvest lands through the same merge. `sc backfill --all` converts to a genuine full clone and re-enables them. The filter records what @@ -212,12 +212,25 @@ transport-adjacent. The rest, imperatively: - **Bucket remotes hold public content plaintext at rest** — bucket ACL is the perimeter (sealed content stays ciphertext, unchanged); partial-clone `filter` against bucket remotes is refused. +- **`DirBucket` (`sc+wal://`) is a single-host, single-owner backend (#264).** + Its CAS takes an OS advisory file lock (`File::lock`: `flock` on unix, + `LockFileEx` on Windows) on `/.cas-lock`, a file that is never removed and + is opened for writing. `lock()` waits without a timeout, so a holder that + stalls (a suspended process, a hung filesystem) blocks every writer of that + directory until it lets go; the OS drops the lock if the holder dies. Do not + run binaries from before and after #264 against one directory at once: the old + create-exclusive lock and the new lock do not exclude each other (an update can + be lost, and an old binary can fail with "cas lock stuck"). `flock` on NFS and + similar is unreliable, so keep it on a local filesystem; use `sc+s3://` when + several machines write (ADR-0046, THREAT-MODEL). - **`sc remote compact` deletes bucket objects** (ADR-0057): it needs bucket list and delete rights, which pushers and `sc serve --store` instances should not hold. It advances the manifest only through the normal CAS (one no-op entry plus a checkpoint) and never deletes what the current chain names. Keep `--prune-expire` well above any push's duration (default 24h): a writer stalled longer than that between uploading a pack and its commit can still lose the pack (the leases follow-on closes it). + `--dry-run` writes and deletes nothing but still downloads and decodes every + candidate pack, so on S3 it costs the same egress and requests as a real run. - **`sc serve --store` still requires a local serve home with `.sc/`** — tokens, TLS identity, and pack spills live there; the bucket holds all served content. @@ -245,7 +258,8 @@ transport-adjacent. The rest, imperatively: `sc rewrap` only replaces wrap lists) plus rotating the underlying credential. History keeps the old ciphertext (rotation is not erasure). - **The signature index is `.sc/signatures.v2` (#257).** `.sc/signatures` is the - pre-#257 file, kept only for the one-time import. Do not run an older binary's + pre-#257 file, kept only for the one-time import (an import that failed is retried on later + ingests). Do not run an older binary's `sc gc` on a repo this build has signed in or ingested into: it roots only the legacy file and sweeps the signatures v2 lists (ADR-0038). - **The oplog is local-only** (like a reflog) — it never travels over @@ -298,7 +312,12 @@ transport-adjacent. The rest, imperatively: `rebase --continue` refusal is fixed by widening and rerunning it (#224). A binary conflict's `.theirs` sidecar sits beside its in-view file and may not match a file-exact spec (partial clones check it by its base path); - completion and abort remove it. + completion and abort remove it. With a decided root recorded (public branches), + completion carries out-of-view and still-protected paths from it alone, never + from a parent, so a path the other side deleted cleanly stays deleted (#238, + #239). A spec change never counts a case alias of a laid-out path as tracked + (#241): on a case-insensitive filesystem the newly visible alias would be the + laid-out file and writing it would replace that file's bytes. - **`.sc/git-remotes//marks` carries `git_oid ↔ sc_id` identity — deleting it is destructive.** Deleting the sibling `mirror.git` is always safe (reconstructed on next use). Stale marks self-heal on push/export. @@ -352,6 +371,22 @@ transport-adjacent. The rest, imperatively: policy ops) is guarded against an in-progress merge/pick/rebase** — finish or abort the in-progress op first. A push into a repo with a stopped op is refused for that repo's checked-out branch (other branches still accept it). +- **`sc ws run` releases the repo lock while the agent command runs + (ADR-0056).** It resolves the workspace and secret environment under the lock, + then spawns the command without it, so other sessions and `sc` commands stay + usable; an `sc` the command runs from inside the workspace finds the host repo + and can change it. `sc work` keeps the lock for its whole session. +- **Downgrade hazards (an older binary on a repo or peer this build has + touched):** (1) a binary from before ADR-0056 does not root a named session's + base (`.sc/ws-`) in `sc gc`; (2) while a `sc ws refresh` is unfinished the + manifest's `base_snapshot` holds the placeholder `refresh-unfinished`, so an + older binary fails to parse the manifest (and its gc fails) instead of + harvesting a half-refreshed checkout; finish the refresh with this build; (3) + an older `sc gc` roots only the legacy `.sc/signatures` and sweeps what + `.sc/signatures.v2` alone lists; (4) wire v6 is matched exactly, so every + mixed client/server pair, ssh remotes included, fails at the handshake; (5) + `DirBucket` binaries from before and after #264 do not exclude each other (see + above). Upgrade every user of a repo, serve home or bucket directory together. - **A `sc work`/`ws` session is the only sanctioned ephemeral+persistent mix:** temp checkouts are removed on teardown; durable writes go only through the normal commit path. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 88a1d2b9..a66bd4dd 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -43,10 +43,11 @@ CI (`ci.yml`) runs `fmt --check`, `clippy -D warnings`, `cargo test --workspace An advisory `windows` job (`cargo check` and `clippy` for the workspace, plus the `scl-repo` `cache::` tests, which verify the owner-only ACL on `.sc/local-key`) is not a required check; keep it green when you touch -Windows-specific code. Your PR must be green on all required jobs. Dependency changes also run the RustSec -`audit` workflow, and CodeQL, Scorecard, the SBOM generation, the -`vendor-provenance` check and a nightly `wire_decode` fuzz run live in their own -workflows. In addition: +Windows-specific code. Your PR must be green on all required jobs. Changes to +`Cargo.toml` or `Cargo.lock` also run the RustSec `audit` workflow (it runs daily +as well). CodeQL, Scorecard, the SBOM generation, the `vendor-provenance` check +(vendored glib only) and a nightly `wire_decode` fuzz run (also on PRs that touch +`fuzz/`) live in their own workflows. In addition: 1. **Every new behavior ships with a test.** Tests live in `#[cfg(test)] mod tests` next to the code. A test that materializes to disk must clean up after diff --git a/README.md b/README.md index cb371b30..868c9c7a 100644 --- a/README.md +++ b/README.md @@ -74,7 +74,8 @@ apps/ desktop/ read-only Tauri v2 + React/TypeScript repository browser (see apps/desktop/README.md); calls `repo` directly, never the `sc` binary demo/ - 27 end-to-end proof scripts covering everything below, e.g.: + 28 end-to-end proof scripts covering most capabilities (bucket remotes are + covered by tests only), e.g.: run_demo.sh independent before/after filesystem diff, zero residue run_repo_demo.sh persistent repo end-to-end demo run_ssh_remote_demo.sh ssh-native transport round trip @@ -93,8 +94,9 @@ keeping the rest of the system Git-, crypto-, TLS- and object-store-agnostic. ## Quick start ```sh -# Build and test the whole workspace -cargo test +# Build and test the whole workspace (on Linux the desktop crate needs the Tauri +# system packages; see CONTRIBUTING.md) +cargo test --workspace # Run the parallel-agent demo (4 agents, default 8 MiB blob budget) cargo run --bin sc -- demo --agents 4 @@ -128,11 +130,12 @@ bash demo/run_demo.sh Run `sc --help` (and `sc --help`) for the full command surface: merge, history editing, secrets lifecycle, sparse/partial clone, ssh/HTTP(S)/ bucket remotes, session transcripts, and more. The annotated reference lives in -the `sc-cli-reference` skill (`.claude/skills/sc-cli-reference/SKILL.md`), which -does not yet cover bucket remotes. Those take `sc remote add -sc+wal://` or `sc+s3:///` (see ADR-0046), and `sc serve ---http 127.0.0.1:8730 --store ` serves a bucket from a -disposable instance (`--stdio` also works; `sc serve --help`). +the `sc-cli-reference` skill (`.claude/skills/sc-cli-reference/SKILL.md`). +Bucket remotes take `sc remote add sc+wal://` or +`sc+s3:///` (ADR-0046); `sc remote compact ` prunes them +(ADR-0057), and `sc serve --http 127.0.0.1:8730 --store ` +serves a bucket from a disposable instance (`--stdio` also works; +`sc serve --help`). ## How Phase 1 works @@ -169,8 +172,7 @@ via `SC_HTTPS_CA_FILE`/`SC_HTTPS_CA=system`), bucket (`sc+wal://`, `sc+s3://`), and Git (including hosted GitHub) network transports, plus private branches sealed to a recipient set until publish and a read-only Tauri desktop browser. All 36 phases and P37a-37d are implemented and tested; `demo/` holds -28 independent end-to-end proofs, one per major capability. Remaining -follow-ons (transcript drop delivery gaps, `sc ws` -workspace refresh and similar hardening -items) are tracked in [ROADMAP.md](ROADMAP.md)'s Deferred section and as open +28 independent end-to-end proofs covering most major capabilities. Remaining +follow-ons (richer signer trust models, per-path token ACLs, transcript +rewrap/grant/revoke and similar items) are tracked in [ROADMAP.md](ROADMAP.md)'s Deferred section and as open GitHub issues rather than being open MVP scope. diff --git a/ROADMAP.md b/ROADMAP.md index 73b2d17b..ba17dd40 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -73,8 +73,9 @@ across every phase. client spawns the user's `ssh` for `ssh://` remotes, overridable via `SC_SSH` (GIT_SSH pattern) so tests and `demo/run_ssh_remote_demo.sh` drive the full ssh:// code path with no sshd. Zero new dependencies. - Accepted limitations: 4 GiB frame cap, no repo paths with spaces over - real ssh, `sc` must be on the server's PATH. (ADR-0022.) + Accepted limitations: 4 GiB frame cap (since replaced by the configurable + object cap, P28 and #213), no repo paths with spaces over real ssh, `sc` must + be on the server's PATH. (ADR-0022.) - **Phase 13 — Agent workspaces (`sc work`).** Fused the two halves of the project: fork N in-RAM copy-on-write workspaces from a persistent repo's HEAD (the repo's budget-bounded store is the backing tier — eviction is @@ -202,8 +203,19 @@ across every phase. workspace's state (dirty files included) is committed to a throwaway candidate and replayed with the ADR-0049 engine; any conflict, running command, scanner hit, view gap or user file in the way refuses the whole refresh before a - write; only the differing files are re-laid in place and the manifest base - moves last. Session-wide only; per-index refresh is not built. + write; only the differing files are re-laid in place (each re-checked against + what was read, right before it is touched) and the manifest base moves last. + It also refuses a tip that drops the protection of a path the workspace holds + as plaintext, and an `.scignore` change that would hide an added file or expose + an untracked one. A crash mid-refresh leaves a `pending_refresh` marker + (target and per-workspace candidate and result snapshots, all gc roots): + `harvest` and `ws run` refuse until `sc ws refresh` is run again and finishes + it from the recorded pair. While the marker is set the manifest's + `base_snapshot` reads `refresh-unfinished`, so an older binary fails to parse + the manifest (and its gc fails) instead of harvesting a half-refreshed + checkout. A binary from before ADR-0056 also does not root a named session's + base in `sc gc`; do not run one against a repo with named sessions open. + Session-wide only; per-index refresh and `harvest --all` are not built. - **Phase 21 — Hardening & consolidation.** Closes the P16–P20 review tail with no new capability axis. Every commit-creating policy op (`protect`/`grant`/`revoke`/`secret add`/`secret rotate`/`secret @@ -376,11 +388,12 @@ across every phase. ` widens the filter from the promisor origin on demand, offline everywhere else (explicit backfill, not transparent lazy-fetch — a deliberate MVP choice to keep network I/O out of every read path). - **Coarse but honest limitation:** merge, cherry-pick/rebase replay, `sc - ws harvest`, and `sc work` are refused entirely on a partial clone - (`Error::PartialCloneUnsupported`) — backfill to a full clone first, not - a per-case gap-tolerant reimplementation. `sc export` refuses too (Git - needs full trees). Proven by `demo/run_partial_clone_demo.sh`: a + **Coarse but honest limitation at the time:** merge, cherry-pick/rebase + replay, `sc ws harvest`, and `sc work` were refused entirely on a partial + clone (`Error::PartialCloneUnsupported`) — backfill to a full clone first, + not a per-case gap-tolerant reimplementation (P37c and P37d lifted the merge, + replay and session refusals). `sc export` still refuses (Git needs full + trees). Proven by `demo/run_partial_clone_demo.sh`: a `--filter src/` clone holds fewer objects than a full clone (both by object-store count and `sc verify`'s gap report), docs/lib/ are never fetched or materialized, a src/ edit committed and pushed from the @@ -399,7 +412,8 @@ across every phase. undo, ws) — and `is_unsafe_ref_component` (the distinct, `/`-permitting remote-tracking validator) is upgraded to also reject whitespace/control, closing an oplog-corruption gap via a hostile git remote's branch name. - DoS caps: a single `MAX_OBJECT_SIZE` (256 MiB) in `crates/core` bounds + DoS caps: a single max object size (256 MiB by default; configurable from 2 MiB + to 1 GiB since #213) in `crates/core` bounds the wire frame length, the pack-record compressed length, AND the zstd decompressed output via a decode-WITH-LIMIT reader (never decode-then- check, so a decompression bomb never fully materializes); four @@ -500,8 +514,15 @@ across every phase. and a remote that held a tombstone before trusting the signer re-judges it from a capped `.sc/transcript-pending` index when the trusted-signer set changes. Valid signatures from attacker keys cannot crowd out a drop (#257: the - signature index holds one slot per (target, signer)), and serve-path index - updates take a short index lock (#260). + signature index holds one slot per (target, signer), now in + `.sc/signatures.v2`; the old `.sc/signatures` is imported once, re-verified), + and serve-path index updates take a short `.sc/index.lock`, waited for at most + 30 s (#260). Only signatures that verify for a held target are indexed, so a + forged one reads as `Unsigned`, never `Invalid`; an older binary's `sc gc` + sweeps signatures only the v2 file lists, so do not run one on such a repo + (#265). The migration retries on every ingest (#270, #273), and both + `signatures::reindex` and `transcripts::reindex` scan outside the index lock + and merge late arrivals under it (#273, #277). - **Phase 32 — In-binary TLS (`sc+https://`).** Closes the audit's High #1 (plaintext bearer tokens/traffic on `sc+http://`) and ADR-0036's "no TLS" @@ -586,7 +607,16 @@ across every phase. signed drop tombstones by decoding the on-chain packs (#229, no format change). `sc remote compact` (#216) merges small packs and deletes the log entries, checkpoints and unreferenced packs behind an aged cutoff; readers - restart on a moved manifest; no format change. (ADR-0046, ADR-0057.) + restart on a moved manifest; no format change; `--dry-run` writes nothing but + downloads and decodes the same candidate packs as a real run. A lost manifest + CAS is retried up to 16 times with jittered backoff (#272). The `DirBucket` + CAS lock is an OS advisory file lock on `/.cas-lock` (#264): it queues + waiters and is dropped when its holder dies, but `lock()` waits without a + timeout, the file is never removed, binaries from before and after #264 do not + exclude each other on one directory (an old binary can also fail with "cas lock + stuck"), and `flock` on NFS is unreliable, so a directory bucket is a + single-host backend; use S3 when several machines write. (ADR-0046, + ADR-0057.) - **Phase 37a — Want-by-id fetch.** A batched `GetObjects` verb (wire v5), an opt-in `.sc/promisor-lazy` marker, and `sc cat :`, which fills only the absent trees and the blob on its path. `Store::get` stays network-free. @@ -639,8 +669,22 @@ across every phase. path, and `sc sparse set` may widen while a merge, pick or rebase is stopped (#224, #236); a stopped completion no longer resurrects out-of-view deletions (#239). - - Checkout restores the executable bit on the write-out paths (#242, #243); - `sc resolve` after a deleted conflicted file restores it too. + - Checkout restores the executable bit on the write-out paths (#242, #243, + #253); `sc resolve` after a deleted conflicted file restores it too. + - Working-tree safety (ADR-0034 refinement 8): a stop's `materialize` and every + abort delete or overwrite only what is provably the stop's own (a regular + file with exactly the tracked bytes, or a path the stop recorded); an abort + finishes and reports what it kept through `AbortReport` (#255, #256, #259). + A file/directory change at one path now crosses switch, undo, abort and + `--continue`, clearing emptied directories (#247, #250); an untracked file at + an out-of-view path that a result adds is refused (#248, #251); a stopped + merge into a private branch aborts and completes with `--identity` (#246, + #252); a sparse view change never counts a case alias of a laid-out path as + tracked (#241, #264). + - A snapshot that would hold a path as both a file and a directory (a + directory made where an absent protected or out-of-view file is carried back) + is refused by both tree builders before any tree is written (#278, #280; + ADR-0034 refinement 9, ADR-0025 refinement 8). - A warning when the protected-cache key cannot be permission-restricted on a non-unix target (#225), and an owner-only Windows ACL on `.sc/local-key`, verified by the advisory `windows` CI job (#214). Windows keys created @@ -789,6 +833,12 @@ smaller ones (protected-cache hygiene, theirs-side resolve carry, sparse and P28 review follow-ons, the port de-flake, the clippy allow-list) are recorded in `CLAUDE.md`, `ARCHITECTURE.md` and the ADRs. +- **Windows owner-only protection for the `sc keygen` identity file and the TLS + serve key (#281, follow-on to #214).** `.sc/local-key` gets an owner-only DACL; + the identity file (`sc keygen --out`, `0600` on unix only) and the serve-home + TLS key still get default permissions on non-unix. The helper lives in + `scl-repo`, while `cli` and `tlsio` must not depend on it (leaf rule), so it + moves to `core` or arrives as a callback. - **Remaining history-editing depth:** operation objects in the CAS (Jujutsu-deep upgrade to the file oplog), oplog entries for remote-tracking refs. diff --git a/SECURITY.md b/SECURITY.md index 7a0b9932..ceab0113 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -29,7 +29,9 @@ it are consolidated in [`docs/THREAT-MODEL.md`](docs/THREAT-MODEL.md). In brief: - **Bucket remotes (`sc+wal://`, `sc+s3://`) put the bucket's ACL on the perimeter.** Public content sits in the bucket as plaintext at rest and a writer with raw bucket access can move refs outside the `sc` client; sealed - content stays ciphertext. A bucket cannot judge a transcript drop tombstone, so + content stays ciphertext. `sc remote compact` deletes bucket objects, so it + needs list and delete rights that ordinary pushers and `sc serve --store` + instances should not hold. A bucket cannot judge a transcript drop tombstone, so it serves a dropped transcript's body next to the tombstone; the receiver honors the drop only for a signer in its `[signers] trusted` list. - Committed secrets injected by `sc run` live in an **authorized local process diff --git a/docs/THREAT-MODEL.md b/docs/THREAT-MODEL.md index 796e5505..a5621e2c 100644 --- a/docs/THREAT-MODEL.md +++ b/docs/THREAT-MODEL.md @@ -4,7 +4,7 @@ This document consolidates, in one place, the **deliberate security boundaries** of src-control's cryptographic features — the things it does *not* defend against by design. They are drawn from the per-phase ADRs (linked below); this page exists so a newcomer evaluating whether to trust real data to the system does not -have to read 54 ADRs to find the caveats. +have to read 57 ADRs to find the caveats. > **Status: pre-1.0, not independently audited.** These are MVP implementations > reviewed by the project's own process only. Do not commit production secrets to @@ -172,6 +172,11 @@ too, not just the src-control-side metadata. structure); or plaintext copied elsewhere by another authorized local tool. Phase 35 has no identity import or decryption path, so adding either requires a new threat-model review rather than widening these commands in place. +- **"Read-only" means no refs, objects or working-tree files.** Each query + briefly takes the repository's single-writer lock, and opening the repository + can update derived indexes (`.sc/transcript-drops`, `.sc/transcript-pending` + when the trusted-signer set changed, and the one-time import into + `.sc/signatures.v2`), best effort; no key material is involved. ## Signed commits & provenance (`sc commit --sign` / `sc verify`) — ADR-0032 @@ -197,6 +202,19 @@ too, not just the src-control-side metadata. verified at fetch/clone) is its own trust-model effort, not a corner of an unrelated map. Revisit triggers: the first real multi-writer or hosted `sc serve` deployment, or hosting sc repos on untrusted infrastructure. +- **Received signatures are verified before they are indexed (ADR-0038 addenda, + #230/#257).** The index (`.sc/signatures.v2`) takes a signature only when its + target is held and it verifies under the key it names, in the signing domain of + its target kind. A forged or dangling signature that arrives in a pack stays in + the object store until gc but is never indexed, so ingest cannot turn a + snapshot `Invalid`: it reads `Unsigned`, which `sc verify --require` also + rejects. Trust fails closed, but a forgery attempt is reported as an absent + signature, not as an invalid one. A bucket + remote forwards the signatures it holds without checking them; the receiver + filters. The index keeps one slot per (target, signer). **Downgrade:** an + older binary's `sc gc` roots only the legacy `.sc/signatures` file and sweeps + signatures that only the v2 file lists, so do not run one on a repo this build + has signed in or ingested into. ## Agent session transcripts (`sc transcript`) — ADR-0038 @@ -233,7 +251,12 @@ too, not just the src-control-side metadata. cannot crowd out the real one (resolved, #257); they cost a line and a check each. Index updates on the serve path take a short `.sc/index.lock`, so concurrent pushes and local commands do not lose index or pending entries - (resolved, #260). Writers + (resolved, #260). The lock is an advisory `flock` on a file that is never + removed, held only for single index updates, and waited for at most 30 + seconds; past that the ingest fails with `IndexBusy` after its objects are + stored, and they stay unindexed on that remote until a reindex (`sc clone` of + it). Advisory locking is only as reliable as the filesystem's `flock` (not + guaranteed on NFS). Writers can also evict pending entries from a push-only remote (trust the signer before the push). Trust is checked at evaluation and is not retroactive. A tombstone a remote already held before upgrading to a build with the pending index is not remembered: trust the @@ -390,8 +413,14 @@ too, not just the src-control-side metadata. `--max-pack-size`, `--max-conn-bytes` and `--max-object-size` are the bounds that also apply to `--stdio` (the connection, timeout and TLS knobs are `--http` only). +- **The wire protocol is version 6, matched exactly.** Each side refuses a peer + of another version at the handshake, in both directions, so a stale client or + server cannot be talked into an older, weaker framing. This is compatibility, + not authentication: any peer that speaks version 6 and passes the access gates + is served. An `ssh://` remote runs the remote host's own `sc`, so that binary + must match too. -## Bucket remotes (`sc+wal://` / `sc+s3://`) — ADR-0046 +## Bucket remotes (`sc+wal://` / `sc+s3://`) — ADR-0046/0057 - **The bucket is untrusted storage with cooperative writers, not a trusted peer.** Unlike `sc serve`, no process authenticates a push or enforces @@ -451,7 +480,34 @@ too, not just the src-control-side metadata. leases as the real fix); (b) a hostile write that fills `log/` or `checkpoints/` with junk is reclaimed once it falls behind the cutoff; junk at a far-future seq never falls behind it, and an undecodable one stops pack - deletion (fail closed, the report names the key to remove by hand). Use `--dry-run` first on a new bucket. + deletion (fail closed, the report names the key to remove by hand). Use + `--dry-run` first on a new bucket: it deletes and uploads nothing, but it + downloads and decodes the same candidate packs a real run would, so on S3 it + costs the same egress and requests. A `--prune-expire` under 1h is refused + unless `--allow-short-grace` is given, which is only safe while nothing + pushes to or fetches from the bucket. +- **Manifest CAS retries are bounded, and contention is a denial of service, + not a bypass.** `update_ref` retries a lost compare-and-swap up to 16 times + with a jittered, capped exponential backoff (about 2.5 s in total at worst), + then fails with "manifest cas contention". A busy fleet or a hostile writer + can starve a pusher this way and force a rerun; it cannot move a ref the + pusher did not ask for, and every retry re-reads the manifest it races. +- **`DirBucket` (`sc+wal://`) is a single-host, local-filesystem backend, + and its CAS lock has limits (#264).** The lock is an OS advisory file lock + (`File::lock`, `flock` on unix) on `/.cas-lock`; it replaced a + create-exclusive spin file. (a) The file is never removed, and `list` skips + it. (b) `lock()` blocks without a timeout: a holder that stalls while holding + it (a suspended process, a hung filesystem) blocks every writer of that + directory until it is released; the OS drops the lock when the holder dies. + (c) Binaries from before and after #264 do not exclude each other: the old + one does not take the OS lock and the new one ignores its create-exclusive + file, so running both against one bucket directory at the same time can lose + an update, and an old binary that finds a `.cas-lock` left by a new one fails + with "cas lock stuck". Upgrade every user of a directory bucket together. (d) + Advisory locks are only as reliable as the filesystem: `flock` on NFS and + some other network filesystems is not dependable, so do not share a + `DirBucket` across hosts. Use S3, whose CAS is a server-side precondition and + is not affected, when more than one machine writes. - **Sealed content stays end-to-end sealed, unchanged.** Protected-path ciphertext (ADR-0014/0043) and committed secrets travel through a bucket remote exactly as through any other transport: a bucket reader without the @@ -517,17 +573,18 @@ too, not just the src-control-side metadata. read is capped at 4 GiB, a constant independent of the object cap (it bounds a disk spool, not RAM). - **Server cost is bounded per request:** 1024 ids (checked while decoding, - before any id is read), the server's `--max-pack-size` over the summed - encoded size, and 1 GiB (the largest configurable object cap) per object. The - server's own object cap is not applied to replies, since the cap is - receiver-only: a lower-cap server serves a larger local object to a client - whose cap admits it, as `GetPack` does. The reply ceiling is a constant - 2 GiB. + before any id is read), a ceiling on the summed encoded size of + `min(--max-pack-size, 2 GiB)` (the 2 GiB is a constant, independent of the + object cap; `--max-pack-size 0` leaves only the 2 GiB), and 1 GiB (the largest + configurable object cap) per object. A batch over the ceiling is refused and + the client splits it. The server's own object cap is not applied to replies, + since the cap is receiver-only: a lower-cap server serves a larger local + object to a client whose cap admits it, as `GetPack` does. - **Laziness is a local opt-in.** `.sc/promisor-lazy` is never committed or transferred; without it a partial clone never dials the remote after clone, and `SC_OFFLINE=1` forces that per process. -## Partial clones (`--filter`, `--lazy`, merge and sessions on a clone) — ADR-0037/0047/0048/0049/0050 +## Partial clones (`--filter`, `--lazy`, merge and sessions on a clone) — ADR-0037/0047/0048/0049/0050/0055/0056 - **A partial clone is a store plus one promise, and the promise is about the origin (ADR-0048).** Every id referenced by an object the clone holds, that @@ -571,7 +628,12 @@ too, not just the src-control-side metadata. injection). A command run by `sc ws run` no longer holds the repo lock (ADR-0056), so an `sc` it runs from inside the workspace finds the host repo and can change it, as a hand-started agent always - could. + could. Several sessions can be open at once (the unnamed one in `.sc/ws/`, + named ones in `.sc/ws-/`); their harvests are serialized by the repo + lock and the first to land wins, a later conflicting one falls back to a + `-` branch. Do not run a binary from before ADR-0056 against a repo + with a named session open: its `sc gc` does not root that session's base + snapshot. - **`sc ws refresh` writes into a workspace the user or an agent may be using (ADR-0056).** It overwrites or removes a file only if it still holds what the refresh read, checked again right before each write (plain files by bytes, @@ -580,9 +642,21 @@ too, not just the src-control-side metadata. one is in the way. An edit in the instant between that check and the write can still be lost. It decrypts a protected path only with the identity passed to it and refuses rather than leave a stale plaintext copy; the throwaway - candidate seals protected edits like a harvest does and is left for gc. An - interrupted refresh leaves a `pending_refresh` marker so harvest cannot treat - half-refreshed files as the agent's own work. + candidate seals protected edits like a harvest does and is left for gc. It + also refuses, before any write, a tip that drops the protection of a path the + workspace holds as plaintext (a later harvest would otherwise publish it), and + an `.scignore` change that would hide a file the agent added or expose an + untracked one a harvest would then commit. An interrupted refresh leaves a + `pending_refresh` marker, kept with the target and the per-workspace + candidate and result snapshots as gc roots, so harvest and `sc ws run` refuse + and a rerun of `sc ws refresh` finishes it; a retry needs `--identity` to + resume a protected file it cannot otherwise verify. While the marker is set + the manifest's `base_snapshot` reads `refresh-unfinished`, so a binary from + before this change fails to parse the manifest, and so does its gc. + Known limits: the marker records no per-file progress, so a file + the interrupted refresh added and the agent then deleted comes back on the + retry, and a power loss (not a process crash) can leave a mode or a deletion + to redo. - **A blob-size limit (`--filter blob:limit=`, ADR-0055) is request scope like the prefix filter, and the server evaluates it.** It hides nothing: any client that can read the repository can fetch the left-out blobs (`sc @@ -611,6 +685,63 @@ too, not just the src-control-side metadata. - **Laziness is a local opt-in** (see By-id object reads above), and bucket remotes refuse `--filter` (see Bucket remotes). +## Working-tree writes (merge, switch, abort, sparse) — ADR-0034 refinements 6-8, ADR-0044 + +This is data safety for the user's own files, not confidentiality. The checkout +is user-owned disk that `sc` rewrites during merge, switch, undo, rebase, abort +and `sparse set`; a stopped operation also writes conflict markers and, for +protected paths, plaintext. + +- **Enforced:** + - A merge, switch, cherry-pick, rebase, undo or sparse-spec change refuses, + before any write, to lay a file or directory onto an untracked or ignored + one (`refuse_untracked_overwrite`), including a path outside the sparse + view that the result adds (#248, #251). A refusal changes nothing; move the + file away and retry. + - A stop's `materialize` deletes or overwrites a path only if the tree it + replaces tracks it and the disk copy is a regular file with exactly the + tracked bytes, or the stop itself recorded the path (`.sc/CONFLICT_PLAINTEXT`, + `.sc/CONFLICT_SIDECARS`, `.sc/sparse-widen`) and it still matches what was + laid. An edit, a replacement or a symlink survives (#255, #256, #259). + - `merge --abort`, `cherry-pick --abort` and `rebase --abort` finish rather + than refuse, so a kept resolution edit cannot trap the user in the stop: + they restore the tip, drop the stop's state and return an `AbortReport` + whose `kept` paths (changed since the stop) and `skipped` protected paths + `sc` prints. Recorded plaintext and sidecars are removed whatever their + bytes, since an edit of them is the resolution the abort discards. + - A file/directory change at one path crosses switch, undo, abort and + `--continue` (#247, #250). Emptied directories are cleared, but every + directory is checked before the first removal, so a directory that holds a + user file refuses and an abort stays retryable. + - A sparse view change never counts a case alias of a laid-out path as tracked + (#241); where the guard cannot tell, it refuses. + - A snapshot never holds one path as both a file and a directory (#278, #280, + ADR-0034 refinement 9). An absent protected or out-of-view file is carried + back verbatim, so a directory made at its path would otherwise yield a tree + with a blob and a tree under one name; both tree builders (public and + private-branch) refuse it with `FileDirCollision` before any tree is written, + for commit, merge completion, `sc ws harvest` and `sc ws refresh`. Remove the + directory and retry. +- **Not enforced (known limits):** + - A user file whose bytes equal the tracked or laid ones cannot be told from + the stop's own write, so it is removed. + - The private-branch stop keeps no decided root and no record of the bytes it + laid, so its `merge --abort` (which needs `--identity`) restores the tip over + every path the tip tracks: a post-stop edit of one, or a user file at a path + the stop removed, is lost, and a conflicted path counts as the stop's. Private + forward operations remove in-view paths on the dirty check alone, without a + byte comparison. + - Protected old paths are plaintext on disk and cannot be compared without a + key: a forward operation still removes a user's plaintext copy, and an abort + restores one only if the stop recorded it. A stop recorded by a build before + #259 lacks the decrypted entries, so its abort keeps such a file. + - A crash after the sidecar record is retired and before the sidecar is + unlinked leaves the `.theirs` file unrecorded, and a retried `sc commit` + would commit it as ordinary content (ROADMAP.md, Deferred). + - On a partial clone the out-of-view check sees only the subtrees the store + holds; an addition inside an absent subtree fails the later commit with + `GappedPathContent` instead. + ## Untrusted-input hardening (DoS) — ADR-0039 - A single max object size caps every untrusted length: wire frames, @@ -687,8 +818,9 @@ too, not just the src-control-side metadata. log entry, idx) is capped at a fixed 256 MiB after the bucket GET has buffered it, so it bounds decode work, not the download. It is not tied to the operator's cap, so lowering the cap cannot make a bucket unreadable. - - By-id reply spools (server ceiling 2 GiB, client read cap 4 GiB) are - constants independent of the cap: they bound disk, not RAM. + - By-id reply spools (server ceiling `min(--max-pack-size, 2 GiB)`, client + read cap 4 GiB) do not depend on the object cap: the 2 GiB and 4 GiB are + constants, and the spools bound disk, not RAM. - The ceiling (1 GiB) is compile-time, so no configuration can make a length claim unbounded. diff --git a/docs/adr/README.md b/docs/adr/README.md index 1e1da46f..c346529a 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -50,15 +50,15 @@ supersedes the old one and update the **Status** line of the superseded record. | [0037](0037-partial-clone.md) | Partial clone (promisor store + prefix-scoped fetch) | Accepted; refusals lifted by 0048, 0049, 0050 | 27 | | [0038](0038-agent-session-transcripts.md) | Agent session transcripts as CAS objects | Accepted | 30 | | [0039](0039-security-hardening-sweep.md) | Security hardening sweep (audit fix-now items) | Accepted | 28 | -| [0040](0040-sc-http-access-control.md) | sc+http access control (read-only, fail-closed bind, bearer tokens) | Accepted; bind gate narrowed by 0042 | 29 | +| [0040](0040-sc-http-access-control.md) | sc+http access control (read-only, fail-closed bind, bearer tokens) | Accepted; bind gate narrowed by 0042, token expiry and rotation in its addendum | 29 | | [0041](0041-listener-resource-limits.md) | Listener resource limits (connection cap, session timeout, spool cap, accept backoff) | Accepted | 31 | | [0042](0042-in-binary-tls-sc-https.md) | In-binary TLS — `sc+https://` via rustls | Accepted; CA validation 0054, pin management 0053 | 32 | | [0043](0043-randomized-protected-encryption.md) | Randomized protected-path encryption (dual-read, randomized-write) | Accepted; rotate-for-paths built as 0051 | 33 | | [0044](0044-per-branch-access-control.md) | Per-branch access control with opaque private branches | Accepted; sparse merge-in section (#240) | 34 | | [0045](0045-native-desktop-read-model.md) | Native desktop adapter and keyless read model | Accepted | 35 | -| [0046](0046-wal-bucket-remotes.md) | Bucket remotes via an immutable-pack WAL + one CAS'd manifest | Accepted; compaction 0057 | 36 | +| [0046](0046-wal-bucket-remotes.md) | Bucket remotes via an immutable-pack WAL + one CAS'd manifest | Accepted; compaction built as 0057 | 36 | | [0047](0047-want-by-id-fetch-and-gap-fill.md) | Want-by-id fetch and explicit gap fill for partial clones | Accepted (built) | 37 | -| [0048](0048-presence-based-partial-clones.md) | Presence-based walks for partial clones | Accepted (built) | 37b | +| [0048](0048-presence-based-partial-clones.md) | Presence-based walks for partial clones | Accepted (built); 0055 adds a second kind of gap | 37b | | [0049](0049-merge-on-partial-clones.md) | Merge on partial clones | Accepted (built) | 37c | | [0050](0050-sessions-on-partial-clones.md) | Agent sessions on partial clones | Accepted (built) | 37d | | [0051](0051-rotate-protected-paths.md) | Rotating protected paths (`sc rewrap --rotate`) | Accepted (built) | — | @@ -66,8 +66,8 @@ supersedes the old one and update the **Status** line of the superseded record. | [0053](0053-tls-pin-management.md) | TLS pin management (`sc tls`) | Accepted (built) | — | | [0054](0054-tls-ca-validation.md) | CA-path validation for `sc+https://` | Accepted (built) | — | | [0055](0055-blob-size-clone-filter.md) | Blob-size clone filter (`--filter blob:limit=`) | Accepted (built) | 27 | -| [0056](0056-named-concurrent-ws-sessions.md) | Named and concurrent `sc ws` sessions | Accepted (built; refresh designed, deferred) | 20 | -| [0057](0057-bucket-compaction.md) | Bucket remote compaction and gc (`sc remote compact`) | Accepted (built) | 36 | +| [0056](0056-named-concurrent-ws-sessions.md) | Named and concurrent `sc ws` sessions and `sc ws refresh` | Accepted (built, including `sc ws refresh`) | 20 | +| [0057](0057-bucket-compaction.md) | Bucket remote compaction and gc (`sc remote compact`) | Accepted | 36 | See [`ROADMAP.md`](../../ROADMAP.md) for the phase sequence. ADR numbers are assigned in creation order, so 0017 — the secret scanner — sequences as phase P5, From a45da0f5bc03410508845589ad1b9549b861e229 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Wed, 21 Oct 2026 10:21:51 +0300 Subject: [PATCH 2/3] docs: address review findings (reindex recovery, abort wording, CAS attempts, dry-run cost, author, identity default) --- .claude/skills/sc-cli-reference/SKILL.md | 9 ++++++--- ARCHITECTURE.md | 10 ++++++---- CLAUDE.md | 5 +++-- ROADMAP.md | 6 +++--- SECURITY.md | 5 +++-- docs/THREAT-MODEL.md | 12 +++++++----- 6 files changed, 28 insertions(+), 19 deletions(-) diff --git a/.claude/skills/sc-cli-reference/SKILL.md b/.claude/skills/sc-cli-reference/SKILL.md index 72e376d1..24c6f3fc 100644 --- a/.claude/skills/sc-cli-reference/SKILL.md +++ b/.claude/skills/sc-cli-reference/SKILL.md @@ -26,6 +26,8 @@ cargo run --bin sc -- keygen [--out ] # (default ~/.sc/identity; 0600 on cargo run --bin sc -- secret-demo # committed-secrets authorize/deny/grant proof # Identity default everywhere a command takes --identity: the flag, then $SC_IDENTITY, # then ~/.sc/identity (a missing file just skips protected paths where decryption is optional). +# Exception: `transcript drop` signs only with an explicit --identity (otherwise the tombstone +# is unsigned and local-only). # Persistent repo commands (sc init creates .sc/ in the working directory): cargo run --bin sc -- init # initialize a new persistent repo @@ -164,8 +166,9 @@ cargo run --bin sc -- work --agents 3 [--name ] [--budget-mb ] -- # decrypted secrets into each agent env) bash demo/run_work_demo.sh # parallel-agents round-trip proof cargo run --bin sc -- ws fork --agents N [--name ] [--identity ] [--author ] # durable multi-invocation - # (--agents default 2; --author is recorded on any - # commit a later harvest makes) + # (--agents default 2; --author is stored in the + # manifest, but harvest takes its own --author, + # then $SC_AUTHOR, then the OS user) # session: materializes N checkouts under # .sc/ws// and persists a manifest (P20). # --name opens a further named session in @@ -519,7 +522,7 @@ cargo run --bin sc -- clone sc+https://host[:port]/repo # TLS clone (P32 # pinning (no pin file read/written; ADR-0054); # remote add/fetch/push accept the same URL form cargo run --bin sc -- tls list [--json] # pins in the known_hosts file (ADR-0053) -cargo run --bin sc -- tls forget # drop every pin for a host (port defaults to 8730; +cargo run --bin sc -- tls forget # drop the pins for a host and port (port defaults to 8730; # changed server key); a later # accept-new connect (no SC_HTTPS_STRICT, no # SC_HTTPS_FINGERPRINT) re-pins; errors if nothing diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index e301a77e..da76a81d 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -392,10 +392,12 @@ A forward operation (merge, switch, cherry-pick, rebase, undo, sparse-spec chang refuses, before any write, to lay a file or directory onto an untracked or ignored one. A tracked path is deleted or overwritten only when the disk copy is a regular file with exactly the tracked bytes, so an edit, a replacement or a symlink -survives; an emptied directory is cleared so a file and a directory can swap at one +survives (where bytes can be compared: a protected path without a usable identity +and a private-branch forward operation cannot, see THREAT-MODEL); an emptied directory is cleared so a file and a directory can swap at one path, and every such directory is checked before the first removal, so a refusal changes nothing. An abort (`merge_abort`, `cherry_pick_abort`, `rebase_abort`) -always completes and returns an `AbortReport`: it removes what the stop laid, +completes past a kept edit (a directory holding untracked content or a blocked +parent still refuses) and returns an `AbortReport`: it removes what the stop laid, overwrites a file only if it is still the stopped tree's bytes or the stop owns it (recorded plaintext, a conflicted path), and lists what it left in `kept` (`sc` prints `left in place (changed since the stop)`). See ADR-0034 refinement 8. @@ -1225,8 +1227,8 @@ as a write-ahead log of **immutable, content-addressed packs** plus **one compare-and-swapped manifest**. A push uploads a pack, appends a parent-linked `log/` entry carrying its ref updates, then CASes the manifest's `head_seq` forward (`Bucket::put_if_tag`). A lost CAS refreshes and retries (up -to 16 times, with jittered exponential backoff between attempts, 2 ms doubling to -a 250 ms cap), so pushes to different branches both land; a push whose target +to 16 attempts, with jittered exponential backoff between them, a ceiling that +starts at 4 ms and doubles to a 250 ms cap), so pushes to different branches both land; a push whose target branch no longer matches its expected old tip fails with `NonFastForward`, exactly as on any other transport, and a loser's orphaned log entry is invisible to readers because they walk `parent_seq` back from the manifest. There is no diff --git a/CLAUDE.md b/CLAUDE.md index 53156b57..fb85d5e7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -230,7 +230,8 @@ transport-adjacent. The rest, imperatively: 24h): a writer stalled longer than that between uploading a pack and its commit can still lose the pack (the leases follow-on closes it). `--dry-run` writes and deletes nothing but still downloads and decodes every - candidate pack, so on S3 it costs the same egress and requests as a real run. + candidate pack, so on S3 its download egress and read requests match a real run (the uploads, + CAS and deletes it skips are the rest). - **`sc serve --store` still requires a local serve home with `.sc/`** — tokens, TLS identity, and pack spills live there; the bucket holds all served content. @@ -374,7 +375,7 @@ transport-adjacent. The rest, imperatively: - **`sc ws run` releases the repo lock while the agent command runs (ADR-0056).** It resolves the workspace and secret environment under the lock, then spawns the command without it, so other sessions and `sc` commands stay - usable; an `sc` the command runs from inside the workspace finds the host repo + usable; an `sc` command run from inside the workspace finds the host repo and can change it. `sc work` keeps the lock for its whole session. - **Downgrade hazards (an older binary on a repo or peer this build has touched):** (1) a binary from before ADR-0056 does not root a named session's diff --git a/ROADMAP.md b/ROADMAP.md index ba17dd40..20148ee9 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -609,7 +609,7 @@ across every phase. entries, checkpoints and unreferenced packs behind an aged cutoff; readers restart on a moved manifest; no format change; `--dry-run` writes nothing but downloads and decodes the same candidate packs as a real run. A lost manifest - CAS is retried up to 16 times with jittered backoff (#272). The `DirBucket` + CAS is attempted up to 16 times with jittered backoff (#272). The `DirBucket` CAS lock is an OS advisory file lock on `/.cas-lock` (#264): it queues waiters and is dropped when its holder dies, but `lock()` waits without a timeout, the file is never removed, binaries from before and after #264 do not @@ -837,8 +837,8 @@ P28 review follow-ons, the port de-flake, the clippy allow-list) are recorded in serve key (#281, follow-on to #214).** `.sc/local-key` gets an owner-only DACL; the identity file (`sc keygen --out`, `0600` on unix only) and the serve-home TLS key still get default permissions on non-unix. The helper lives in - `scl-repo`, while `cli` and `tlsio` must not depend on it (leaf rule), so it - moves to `core` or arrives as a callback. + `scl-repo` (`pub(crate)`), and `tlsio` is a leaf that must not depend on it, so + it moves to `core` or reaches `tlsio` as a callback. - **Remaining history-editing depth:** operation objects in the CAS (Jujutsu-deep upgrade to the file oplog), oplog entries for remote-tracking refs. diff --git a/SECURITY.md b/SECURITY.md index ceab0113..90cd504d 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -30,8 +30,9 @@ it are consolidated in [`docs/THREAT-MODEL.md`](docs/THREAT-MODEL.md). In brief: perimeter.** Public content sits in the bucket as plaintext at rest and a writer with raw bucket access can move refs outside the `sc` client; sealed content stays ciphertext. `sc remote compact` deletes bucket objects, so it - needs list and delete rights that ordinary pushers and `sc serve --store` - instances should not hold. A bucket cannot judge a transcript drop tombstone, so + needs list and delete rights, which belong to a dedicated compaction identity; + ordinary pushers and `sc serve --store` instances should not hold delete + rights. A bucket cannot judge a transcript drop tombstone, so it serves a dropped transcript's body next to the tombstone; the receiver honors the drop only for a signer in its `[signers] trusted` list. - Committed secrets injected by `sc run` live in an **authorized local process diff --git a/docs/THREAT-MODEL.md b/docs/THREAT-MODEL.md index a5621e2c..93c7405a 100644 --- a/docs/THREAT-MODEL.md +++ b/docs/THREAT-MODEL.md @@ -254,8 +254,9 @@ too, not just the src-control-side metadata. (resolved, #260). The lock is an advisory `flock` on a file that is never removed, held only for single index updates, and waited for at most 30 seconds; past that the ingest fails with `IndexBusy` after its objects are - stored, and they stay unindexed on that remote until a reindex (`sc clone` of - it). Advisory locking is only as reliable as the filesystem's `flock` (not + stored, and the signatures stay unindexed on that remote: they are not served + and its gc sweeps them after the grace window (no reindex of a remote is built, + and a re-push skips objects the remote already holds). Advisory locking is only as reliable as the filesystem's `flock` (not guaranteed on NFS). Writers can also evict pending entries from a push-only remote (trust the signer before the push). Trust is checked at evaluation and is not retroactive. A tombstone a remote already held before @@ -487,7 +488,7 @@ too, not just the src-control-side metadata. unless `--allow-short-grace` is given, which is only safe while nothing pushes to or fetches from the bucket. - **Manifest CAS retries are bounded, and contention is a denial of service, - not a bypass.** `update_ref` retries a lost compare-and-swap up to 16 times + not a bypass.** `update_ref` retries a lost compare-and-swap up to 16 attempts with a jittered, capped exponential backoff (about 2.5 s in total at worst), then fails with "manifest cas contention". A busy fleet or a hostile writer can starve a pusher this way and force a rerun; it cannot move a ref the @@ -703,8 +704,9 @@ protected paths, plaintext. tracked bytes, or the stop itself recorded the path (`.sc/CONFLICT_PLAINTEXT`, `.sc/CONFLICT_SIDECARS`, `.sc/sparse-widen`) and it still matches what was laid. An edit, a replacement or a symlink survives (#255, #256, #259). - - `merge --abort`, `cherry-pick --abort` and `rebase --abort` finish rather - than refuse, so a kept resolution edit cannot trap the user in the stop: + - `merge --abort`, `cherry-pick --abort` and `rebase --abort` finish past a + kept resolution edit instead of refusing, so it cannot trap the user in the + stop (a directory holding untracked content, or a blocked parent, still refuses): they restore the tip, drop the stop's state and return an `AbortReport` whose `kept` paths (changed since the stop) and `skipped` protected paths `sc` prints. Recorded plaintext and sidecars are removed whatever their From bf9b383e0747bbbf7fa074bd7dce600b082fc48a Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Wed, 21 Oct 2026 10:35:03 +0300 Subject: [PATCH 3/3] docs: scope the working-tree guarantees to public branches (review) --- ARCHITECTURE.md | 4 +++- docs/THREAT-MODEL.md | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index da76a81d..551c55f9 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -387,7 +387,9 @@ plain file, and setuid/setgid/sticky are always cleared (#242, #243). `sc resolve` applies the chosen side's mode after rewriting a deleted conflicted file. A chmod the filesystem refuses is tolerated with a one-time warning. -`materialize` treats the disk as the user's wherever it cannot prove otherwise. +`materialize` treats the disk as the user's wherever it cannot prove otherwise +(on public branches; private-branch merges, forward operations and aborts have +weaker guards, listed in THREAT-MODEL). A forward operation (merge, switch, cherry-pick, rebase, undo, sparse-spec change) refuses, before any write, to lay a file or directory onto an untracked or ignored one. A tracked path is deleted or overwritten only when the disk copy is a regular diff --git a/docs/THREAT-MODEL.md b/docs/THREAT-MODEL.md index 93c7405a..e826c60a 100644 --- a/docs/THREAT-MODEL.md +++ b/docs/THREAT-MODEL.md @@ -698,7 +698,9 @@ protected paths, plaintext. before any write, to lay a file or directory onto an untracked or ignored one (`refuse_untracked_overwrite`), including a path outside the sparse view that the result adds (#248, #251). A refusal changes nothing; move the - file away and retry. + file away and retry. Not guarded: merging into an unborn branch, and the + conflict stop of a merge into a private branch (it checks only out-of-view + paths, and refuses a `.theirs` sidecar onto an existing file). - A stop's `materialize` deletes or overwrites a path only if the tree it replaces tracks it and the disk copy is a regular file with exactly the tracked bytes, or the stop itself recorded the path (`.sc/CONFLICT_PLAINTEXT`,