From 4386631d334a89d86516f71502414a368e172226 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Tue, 20 Oct 2026 10:22:20 +0300 Subject: [PATCH 01/11] docs: sync README, ARCHITECTURE and contributor docs with the implementation README: 36 phases plus P37a-d, add objio and apps/desktop to the layout, correct the dependency direction (repo does not link gitio), 27 demos (the file said 25 and 26), current status (CA validation, rewrap --rotate, bucket remotes, partial-clone merge and sessions, desktop), point at `sc --help`. ARCHITECTURE: workspace diagram gains tlsio, objio, demo and fuzz; add sections for P36 (bucket WAL), P37b, P37c and P37d; refresh the .sc layout (serve-tls, promisor-lazy, sparse-prev, local-key, protected-cache, CONFLICT_*); note token expiry/rotate, sc tls, CA validation, --secrets-fd; fix the P29 `--read-only` wording and mark P8/P27 follow-on lists as shipped. CONTRIBUTING: dependency rule with objio, desktop prerequisites, full CI and security workflow list. SECURITY: bucket remotes, CA validation, sc tls pin, token expiry, --secrets-fd. docs/agents: ADR count, crate list, ACCESS.md rows for the fuzz, sbom and vendor-provenance workflows. Co-Authored-By: Claude Sonnet 5.5 --- ARCHITECTURE.md | 185 +++++++++++++++++++++++++++++++++--------- CONTRIBUTING.md | 34 ++++++-- README.md | 77 +++++++++++------- SECURITY.md | 23 ++++-- docs/agents/ACCESS.md | 1 + docs/agents/domain.md | 9 +- 6 files changed, 242 insertions(+), 87 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 22b4e23f..57fcec98 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -3,8 +3,8 @@ A next-generation version control system built around a snapshot-and-tag model (Jujutsu-inspired) with per-file permissions, native committed secrets, and in-memory clones. This document covers the architecture as built — the two -founding wedges and the full collaborative VCS grown around them across 30 -vertical-slice phases. The rationale behind each major decision is recorded as an +founding wedges and the full collaborative VCS grown around them across 36 +vertical-slice phases plus the P37 partial-clone series (37a-37d). The rationale behind each major decision is recorded as an ADR in [`docs/adr/`](docs/adr/), and a compact per-phase appendix closes this doc. ## Thesis and MVP scope @@ -33,10 +33,12 @@ Everything in Phase 1 was designed so Phase 2 dropped in without re-architecting secrets are just a special object kind in the same content-addressed store. That composability held across every phase since. **All three long-term-thesis pillars now ship** — in-RAM worktrees (P1), committed secrets (P2), and per-file -permissions (P7, encrypted paths) — atop a full collaborative VCS: three-way -merge and history editing, packfiles + GC, ssh:// / sc+http:// / Git network -transports, sparse and partial clone, signed commits, and intent-deep provenance -via sealed agent-session transcripts. The phase-by-phase build log is in +permissions (P7, encrypted paths; per-branch in P34) — atop a full collaborative +VCS: three-way merge and history editing, packfiles + GC, ssh:// / sc+http(s):// / +bucket (sc+wal://, sc+s3://) / Git network transports, sparse and partial clone +(with lazy fetch, merge and agent sessions on a partial clone), signed commits, +intent-deep provenance via sealed agent-session transcripts, and a read-only +native desktop browser. The phase-by-phase build log is in [`ROADMAP.md`](ROADMAP.md) and [`CLAUDE.md`](CLAUDE.md); the sections below cover the architecture, with a compact per-phase appendix at the end. @@ -61,9 +63,12 @@ to reason about — which is exactly the property Phase 1 is meant to demonstrat ## System overview -The codebase is a Cargo workspace of eight Rust crates with a strict dependency -direction (top-level adapters `{cli, desktop} → repo → {vfs, gitio, crypto} -→ core`, with the separate leaf edge `repo → tlsio`): +The codebase is a Cargo workspace of eight library/binary crates plus the +desktop app, with a strict dependency direction: the top-level adapters +`{cli, desktop}` sit on `repo`; `repo → {vfs, crypto} → core`, plus the separate +leaf edges `repo → tlsio` and `repo → objio` (`tlsio` and `objio` depend on no +other workspace crate, not even `core`). `gitio → core` is linked by `cli`, never +by `repo`: ``` src-control/ @@ -75,20 +80,28 @@ src-control/ │ │ plus a system-git mirror bridge for hosted Git (the only crate linking gix) │ ├── crypto/ envelope encryption (secrets + protected paths) AND Ed25519 signing + │ │ v2 dual-key identities (scl-crypto; the only crate linking RustCrypto) +│ ├── tlsio/ rustls/rcgen wrappers for sc+https:// (P32); a leaf, the only crate +│ │ linking rustls +│ ├── objio/ `Bucket` trait + directory and S3 backends for sc+wal:// / sc+s3:// (P36); +│ │ a leaf, the only crate linking the S3 SDK │ ├── repo/ durable .sc/ repo: refs/branches/working tree, three-way merge, history │ │ editing (cherry-pick/rebase/amend/undo), remotes + wire transport (ssh://, -│ │ sc+http://), protection + secret lifecycle, GC, sparse & partial clone, -│ │ signatures & session transcripts, agent workspaces -│ └── cli/ `sc` binary — the full ~42-command surface (see CLAUDE.md) -├── apps/desktop/ Tauri v2 top-level adapter + bundled TypeScript renderer; -│ native read model only, with no UI dependency in domain crates +│ │ sc+http(s)://, bucket WAL), protection + secret lifecycle, GC, sparse & +│ │ partial clone, signatures & session transcripts, agent workspaces +│ └── cli/ `sc` binary — the full ~43-command surface (`sc --help`; the annotated +│ reference is the sc-cli-reference skill) +├── apps/desktop/ Tauri v2 top-level adapter (`scl-desktop`) + bundled React/TypeScript +│ renderer; native read model only, with no UI dependency in domain crates +├── demo/ 27 end-to-end proof scripts (`run_*.sh`) +├── fuzz/ cargo-fuzz target for the wire decoder (`wire_decode`) └── ARCHITECTURE.md ``` `core` knows nothing about Git, worktrees, or cryptography. `gitio` is the only crate that links `gix`, keeping the Git dependency quarantined behind one boundary. `crypto` is the only crate that links the RustCrypto stack, keeping -the cryptographic dependency quarantined behind another. `repo` owns the `.sc/` +the cryptographic dependency quarantined behind another; `tlsio` does the same +for rustls and `objio` for the S3 SDK. `repo` owns the `.sc/` on-disk layout and all repo orchestration; it is deliberately **Git-agnostic** (it never links `gitio`) — `cli` links both `repo` and `gitio` and passes imported snapshots down. This matters because the long-term plan is to own the @@ -205,8 +218,9 @@ first-class remote, and Phase 18 reaches *hosted* Git (GitHub over https/ssh) through a spawned-system-`git` mirror bridge (`gix` can fetch but not push, so a lazily-built bare `mirror.git` beside the marks map is the transport). All of it keeps `gix` quarantined in `gitio`. This is distinct from sc's **own** network -transport: `repo` speaks a framed wire protocol over ssh:// (P12) and sc+http:// -(P26) that mirrors its `Transport` trait — no `gix` involved (see the appendix). +transport: `repo` speaks a framed wire protocol over ssh:// (P12), sc+http:// +(P26) and sc+https:// (P32) that mirrors its `Transport` trait, and treats a +bucket as a remote (P36) — no `gix` involved (see the appendix). ## Cryptography and key management @@ -228,7 +242,10 @@ a `BTreeMap` so canonical encoding is insertion-order-independent), kept separat from the file tree so `checkout` never materializes them. An authorized context decrypts in memory and injects the value into a child process environment; `sc secret-demo` proves the authorize/deny/grant flow end-to-end with the same -zero-residue teardown as Phase 1. See ADR-0010. +zero-residue teardown as Phase 1. `sc run` injects through the child's +environment by default; `sc run --secrets-fd N` (ADR-0052, unix) hands the +values over a pipe on descriptor N instead, which keeps them out of env dumps +but is still not isolation. See ADR-0010. The consequence is that a clone in an **unauthorized** context — one whose private key is not among the recipients — receives the secret object intact but @@ -320,14 +337,26 @@ phases but the shape is stable: snapshot→transcript-object index, one-to-many (P30). Both gc-rooted, local-only. - `sparse` — the local, uncommitted sparse-checkout prefix spec (P24); `promisor` — a partial clone's fetch-filter + origin (P27). -- `serve-tokens.toml` — bearer-token access control for `sc serve --http` (P29). +- `serve-tokens.toml` — bearer-token access control for `sc serve --http` (P29); + tokens may carry an expiry and be rotated in place (`sc serve token add + --expires` / `rotate`). `serve-tls/` — the auto-minted `sc+https://` server + identity (P32). +- `promisor-lazy` — empty marker: this partial clone may fill gaps by object id + (P37a); `sparse-prev` — the previous sparse spec while a `sparse set` is in + flight (P37b). +- `local-key`, `protected-cache` (and `ws/cache-`) — the P33 keyed stat cache + for protected paths; never committed, never transferred. - `tmp/` — guard-cleaned scratch for streaming pack transfer (P25). - `ws/` — durable agent-session workspaces: `session.toml` manifest + per-agent checkouts (P20). `git-remotes//` — the `git_oid ↔ sc_id` marks map and a lazily-built bare `mirror.git` for network Git (P10/P18). - Transient in-progress state (removed on completion/abort): `MERGE_HEAD`, - `PICK_HEAD`, `REBASE_STATE`, and their `*_DECIDED_ROOT` records — an - interrupted merge/pick/rebase resumes or aborts cleanly from these. + `PICK_HEAD`, `REBASE_STATE`, and their `*_DECIDED_ROOT` records, plus + `CONFLICT_PLAINTEXT` / `CONFLICT_SIDECARS` listing what a stopped operation + wrote into the working tree — an interrupted merge/pick/rebase resumes or + aborts cleanly from these. +- The `sc+https://` client's pin file is user-level, not part of `.sc/` + (`~/.config/sc/known_hosts`, managed with `sc tls`). ### Git-like working tree @@ -398,11 +427,10 @@ Phase 8 adds three tightly coupled capabilities to the persistent store: transport read path resolves packed, sharded, and compressed objects from the remote store. This replaces the prior object-at-a-time transfer. -Remaining follow-ons: network Git remotes, HTTP transport, streaming (>4 GiB) -frames, bulk re-wrap, multiple escrow keys, interactive workspace sessions -and auto-merge of clean workspace results (merge shipped as Phase 4; -break-glass escrow shipped as Phase 11; ssh-native network transport shipped -as Phase 12; agent workspaces shipped as Phase 13). +Follow-ons named when this phase shipped have since landed as their own phases: +network Git remotes (P18), the HTTP transport (P26), streaming frames (P25), +bulk re-wrap and multiple escrow keys (P17), and interactive workspace sessions +with auto-merge (P20). ## Phase 9 — Git export (built) @@ -881,9 +909,9 @@ their content, so push's filtered reachability walk still sends only the client's new in-filter objects. `commit` refuses (`Error::GappedPathContent`) rather than silently drop content under an unfetched subtree. `sc backfill --all` is the genuine full-clone escape hatch: fetch everything, verify the -closure, then remove `.sc/promisor`. Merge, replay, `sc ws fork/harvest`, and -`sc work` are all refused outright on a partial clone as a deliberate MVP -coarsening rather than per-case gap tolerance. See ADR-0037. +closure, then remove `.sc/promisor`. P27 refused merge, replay, `sc ws +fork/harvest`, and `sc work` outright on a partial clone as a deliberate MVP +coarsening; P37b-P37d lifted those refusals (see below). See ADR-0037. ## Phase 28 — security hardening sweep (built) @@ -904,12 +932,15 @@ ADR-0039. connection opening, closing P26's stated "unauthenticated" boundary: a read-only mode that rejects any write-shaped request before it reaches `wire::serve`, a fail-closed bind check, and bearer-token authentication -against a `.sc/serve-tokens.toml` allowlist. `EC_READONLY` (or equivalent -server flag) forces the read-only gate independent of tokens, so a -public-read mirror needs no token management at all. Each gate runs before +against a `.sc/serve-tokens.toml` allowlist. `--read-only` forces the +read-only gate independent of tokens (a rejected write gets the typed +`EC_READONLY` wire error), so a public-read mirror needs no token management at +all. Each gate runs before the P25/P26 wire handshake begins, matching P26's own status-line-before- handshake discipline, so a rejected connection never reaches -`LocalTransport`. See ADR-0040. +`LocalTransport`. Tokens later gained an optional expiry (`sc serve token add +--expires`; an expired token gets the same 401 as an unknown one) and `sc serve +token rotate` (ADR-0040 addendum). See ADR-0040. ## Phase 30 — agent session transcripts (built) @@ -957,7 +988,7 @@ Proven by `demo/run_limits_demo.sh`. See ADR-0041. A new leaf crate, `crates/tlsio` (`scl-tlsio`), is the only crate linking rustls/rcgen/`ring` — extending the dependency rule with the separate leaf edge -`repo → tlsio` while leaving `repo → {vfs, gitio, crypto} → core` and the RustCrypto quarantine in +`repo → tlsio` while leaving `repo → {vfs, crypto} → core` and the RustCrypto quarantine in `crates/crypto` untouched. Two seam functions grow a TLS wrap and nothing else changes: the client wraps the `TcpStream` before the opening write when the URL is `sc+https://`; the server wraps before the opening read when @@ -970,7 +1001,13 @@ Trust is accept-new TOFU, the SSH `known_hosts` shape: the client pins `SC_HTTPS_STRICT=1` refuses unknown hosts). A pin mismatch always hard-fails. The server auto-mints a self-signed identity into `.sc/serve-tls/` (key `0600`) unless PEM is supplied; `sc serve fingerprint` -prints the pin an operator distributes out-of-band. **Gate change:** a +prints the pin an operator distributes out-of-band. Follow-ons: `sc tls +list/forget/pin/path` manages the pin file (ADR-0053), and a client can opt into +CA validation instead of pinning with `SC_HTTPS_CA_FILE=` or +`SC_HTTPS_CA=system` (ADR-0054; fail-closed, no revocation checking, the pin +file is neither read nor written). Client connect, TLS handshake and opening +are bounded to 30 s, and bracketed IPv6 hosts parse in `sc+http(s)://` URLs. +**Gate change:** a non-loopback bind now requires `--read-only`, `--allow-public`, or (`--tls` AND ≥1 token) — P29's "tokens alone justify a public bind" is narrowed, since a token protecting only a plaintext channel was never the guarantee @@ -1063,6 +1100,36 @@ messages, paths, timestamps, and inner topology never cross IPC. The WebView has no generic dialog, filesystem, URL, process, or object-store command. See ADR-0045. +## Phase 36 — bucket WAL remotes: sc+wal:// and sc+s3:// (built) + +A remote that needs no server process. `BucketTransport` +(`crates/repo/src/bucket_transport.rs`) implements the same `Transport` trait as +the local, ssh and http remotes on top of a flat key/value bucket, structured +as a write-ahead log of **immutable, content-addressed packs** plus **one +compare-and-swapped manifest**. A push uploads a pack, appends a parent-linked +`log/` entry carrying its ref updates, then CASes the manifest's +`head_seq` forward (`Bucket::put_if_tag`); losing the CAS is `NonFastForward`, +exactly as on any other transport, and a loser's orphaned log entry is invisible +to readers because they walk `parent_seq` back from the manifest. There is no +leader, lock file or lease. The new leaf crate `objio` holds the `Bucket` trait +and its two backends (`DirBucket` for `sc+wal://`, `S3Bucket` for +`sc+s3:///`), quarantining the S3 SDK the way `gix` and +RustCrypto are quarantined. + +Reads treat the bucket as untrusted input: manifest, log entries and pack +records are capped at `MAX_OBJECT_SIZE`, and every WAL record decodes strictly +and fails closed. P36b adds **checkpoint folding**: once the log tail passes 64 +entries a push opportunistically writes `checkpoints/` (branch tips plus a +cumulative pack list) and points the manifest at it, so a cold reader seeds from +the checkpoint and walks only the tail. P36c adds `sc serve --store `: an +`sc serve --http`/`--stdio` instance whose `.sc/` home supplies tokens, the TLS +identity and scratch space while all served content lives in the bucket, so +instances are disposable. Public content sits in the bucket as plaintext at +rest (the bucket ACL is the perimeter; sealed content stays ciphertext), and +`sc clone --filter` is refused against a bucket remote. Bucket compaction and gc +are not built (open issue #216), and bucket remotes do not carry indexed +transcripts (#229). See ADR-0046 and `docs/research/walgit-evaluation.md`. + ## Phase 37a — want-by-id fetch (built) A partial clone marked lazy (`.sc/promisor-lazy`) can read a path outside its @@ -1074,6 +1141,50 @@ no reachability walk. `Store::get` stays network-free; the only code that dials the promisor after clone is `Repo::fetch_by_id_from` in `crates/repo/src/ fill.rs`, which verifies the reply against the request before writing. A fill only adds objects, so push, gc and verify (filter-scoped walks) are unchanged. -Sparse widening over a filled path and the lifted P27 refusals need -presence-based walks and are phase 37b. See ADR-0047 and +Sparse widening over a filled path and the lifted P27 refusals needed +presence-based walks and are P37b-P37d (below). See ADR-0047 and `docs/research/scion-evaluation.md`. + +## Phase 37b — presence-based partial clones (built) + +The filter records what a clone *asked for*, not what is present: an ordinary +`sc fetch` on a partial clone sends no filter and advertises its local tips as +haves, so an object new at an in-filter path can be omitted when it equals an +out-of-filter object reachable from a have. 37b-i therefore makes push, verify +and gc walk what the store **holds** (`reachable_objects_partial`) instead of +the fetch filter; push checks every gap id against the destination before +sending, a view gap fails with `GapInView` before any write (a preflight at the +top of `materialize`, plus `undo` and `sparse set`), and a partial clone fetches +from its origin only. 37b-ii lets a lazy, online clone fill the view's closure +by id (`ensure_view`) before switch, fast-forward, undo and `sparse set`; +`sparse set` may widen past the filter; and a by-id reply is capped at +`min(--max-pack-size, 512 MiB)` and split when refused. `sc export` and `sparse +disable` stay refused on a partial clone. See ADR-0048. + +## Phase 37c — merge, cherry-pick and rebase on a partial clone (built) + +`merge_trees` is a tree-recursive three-way merge that carries a subtree **by +id** when the sides agree and no protection rule applies, reads only where the +sides differ or a rule intersects, and on a lazy, online clone fills what it +asks for by id outside the store lock (a non-lazy clone gets the gap error with +the `sc backfill ` hint). A carried protected subtree keeps the complete +input wrap union. A conflict inside the view is marked and resolved like on a +full clone (`sc conflicts`, `sc resolve`, `merge --abort`, `sc commit`; the +out-of-view part is grafted from the decided root); a conflict outside the view, +or a result that must be sealed outside it, is refused before any ref, +merge-state or working-tree change. Cherry-pick and rebase share the engine +through `replay_commit`, and a stopped pick or rebase keeps its carried subtrees +in the decided root. Content and mode now merge as separate dimensions, so a +mode-only edit merges and a file/directory collision fails before any write on +full clones too. See ADR-0049. + +## Phase 37d — agent sessions on a partial clone (built) + +`sc ws fork/run/harvest/abandon` and `sc work` work on a partial clone, with the +host's sparse view as the agent's whole world (`sc work` no longer hands agents +a complete tree there). The view closure is filled (lazy, online) or refused with +the gap error before anything is written; a workspace commit grafts the +out-of-view subtrees from the base without reading them; and harvest lands +through the ADR-0049 merge, with any conflict falling back to a `work-` +branch as on a full clone. `sc backfill --all` remains the way to a genuine full +clone. See ADR-0050. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 8dec37a9..4270df19 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -25,24 +25,40 @@ cargo fmt --all # format (CI enforces --check) cargo clippy --workspace --all-targets -- -D warnings # lint (CI enforces this) ``` +The workspace includes the Tauri desktop crate (`apps/desktop/src-tauri`), so on +Linux `cargo test --workspace` needs the Tauri v2 system packages (WebKitGTK +and friends; the exact list is in the `Install Tauri Linux prerequisites` step +of [`.github/workflows/ci.yml`](.github/workflows/ci.yml)). If you touch the +desktop frontend, also run `npm ci`, `npm run typecheck` and `npm test` in +`apps/desktop` (see [`apps/desktop/README.md`](apps/desktop/README.md)). + The demos under `demo/` are independent end-to-end proofs and double as integration tests — if you touch a subsystem, run the relevant `demo/run_*.sh`. ## The bar for a change -CI runs `fmt --check`, `clippy -D warnings`, `cargo test --workspace`, and -`cargo doc`. Your PR must be green on all of them. In addition: +CI (`ci.yml`) runs `fmt --check`, `clippy -D warnings`, `cargo test --workspace +--all-targets` (Linux and macOS) and `cargo doc` for the Rust workspace, plus a +`desktop` job (`npm ci`, `npm audit --audit-level=high`, typecheck, test). +Your PR must be green on all of them. Dependency changes also run the RustSec +`audit` workflow, and CodeQL, Scorecard, the SBOM generation, the +`vendor-provenance` check and a nightly `wire_decode` fuzz run live in their own +workflows. In addition: 1. **Every new behavior ships with a test.** Tests live in `#[cfg(test)] mod tests` next to the code. A test that materializes to disk must clean up after itself and assert the path is gone. 2. **Respect the dependency rule and quarantines** (enforced socially, not just by - the compiler): `cli → repo → {vfs, gitio, crypto, tlsio} → core` (`tlsio` is a - leaf with no workspace deps). `core` never depends on Git, worktrees, or - crypto. **`gix` stays in `gitio` only**; **RustCrypto stays in `crypto` - only**; **rustls/rcgen stay in `tlsio` only**. If you find yourself reaching - for `gix`, a RustCrypto type, or TLS elsewhere, add a function to - `gitio`/`crypto`/`tlsio` instead. + the compiler): the adapters `cli` and `desktop` sit on top of `repo`, and + `repo → {vfs, crypto} → core` plus the leaf edges `repo → tlsio` and + `repo → objio` (`tlsio` and `objio` have no workspace deps). `gitio` depends + only on `core`; `repo` never links it (`cli` links both and passes imported + snapshots down). `core` never depends on Git, worktrees, or crypto. **`gix` + stays in `gitio` only**; **RustCrypto stays in `crypto` only**; + **rustls/rcgen stay in `tlsio` only**; **the S3 SDK stays in `objio` only**. + If you find yourself reaching for `gix`, a RustCrypto type, TLS, or an + object-store SDK elsewhere, add a function to `gitio`/`crypto`/`tlsio`/`objio` + instead. 3. **Do not break the [core invariants in `CLAUDE.md`](CLAUDE.md#core-invariants-do-not-break)** — content addressing (`BLAKE3(canonical_encoding)`), `Arc<[u8]>`-shared blobs, the mode-scoped disk invariant (ephemeral = zero residue), and "never silently drop @@ -57,7 +73,7 @@ Some changes get a security pass before merge as standing policy, not ad-hoc judgment. If your PR does any of the following, say so in the PR description and walk the checklist: -- **Touches `crates/crypto`, `crates/tlsio`, or any transport/wire code** → +- **Touches `crates/crypto`, `crates/tlsio`, `crates/objio`, or any transport/wire code** → re-read the relevant [`docs/THREAT-MODEL.md`](docs/THREAT-MODEL.md) section; state in the PR whether any documented boundary moves. If one does, the PR must also update `THREAT-MODEL.md` (and add an ADR if the decision changes). diff --git a/README.md b/README.md index eb41d1aa..fa4da513 100644 --- a/README.md +++ b/README.md @@ -2,8 +2,9 @@ A next-generation version control system built around a snapshot-and-tag model (Jujutsu-inspired), with per-file permissions, native committed secrets, and -in-memory clones as the long-term thesis. Across 34 shipped phases, this -repository now proves that thesis end to end: +in-memory clones as the long-term thesis. Across 36 shipped phases plus the +P37 partial-clone series (37a-37d), this repository now proves that thesis end +to end: 1. **In-memory virtual worktrees** (the agent wedge) — fork N parallel worktrees of a repo entirely in RAM, run and check out against each, and tear them down @@ -22,8 +23,9 @@ repository now proves that thesis end to end: durable `.sc/` repos, branches, three-way merge, history editing (cherry-pick/rebase/amend/undo), durable multi-invocation agent sessions, signed commits with session-transcript provenance, sparse and partial - checkouts, and local/ssh/HTTP+TLS/Git network transports. **Implemented - (Phases 3-34.)** + checkouts (including merge and agent sessions on a partial clone), local/ + ssh/HTTP+TLS/bucket/Git network transports, and a read-only native desktop + browser. **Implemented (Phases 3-37.)** The system builds on / interoperates with Git rather than replacing it: it imports an existing Git repo's `HEAD` in-process (via `gix`), exports history @@ -31,9 +33,10 @@ back to Git commits, and reaches hosted Git (e.g. GitHub) through a spawned system-git mirror bridge. See [ARCHITECTURE.md](ARCHITECTURE.md) for the full design, -[docs/adr/](docs/adr/) for the architecture decision records covering every -phase, and [CLAUDE.md](CLAUDE.md) for project conventions and the per-phase -capability map. +[docs/adr/](docs/adr/) for the architecture decision records (0001-0054) behind +every phase, and [CLAUDE.md](CLAUDE.md) for project conventions and the +per-phase capability map. [CONTRIBUTING.md](CONTRIBUTING.md) covers the +development workflow. > ⚠️ **Pre-1.0, not independently audited — don't trust production secrets to it > yet.** src-control implements real cryptography (committed-secret envelope @@ -41,9 +44,10 @@ capability map. > TLS), but these are MVP implementations that have **not had an independent > security audit**. There are deliberate boundaries you should understand first — > e.g. pre-Phase-33 convergent ciphertext in history stays equality-confirmable -> forever, `sc+https://` trust is TOFU pinning (the first connection to a host is -> unverified), and rotation cuts off future reads but cannot erase ciphertext -> already in history. They are consolidated in +> forever, `sc+https://` trust defaults to TOFU pinning (the first connection to +> a host is unverified unless you pre-pin it or opt into CA validation), and +> rotation cuts off future reads but cannot erase ciphertext already in +> history. They are consolidated in > [docs/THREAT-MODEL.md](docs/THREAT-MODEL.md). Report vulnerabilities privately > per [SECURITY.md](SECURITY.md). @@ -60,12 +64,17 @@ crates/ Ed25519 signing (the ONLY crate that links RustCrypto) tlsio/ TLS for the sc+https:// transport (the ONLY crate that links rustls/rcgen; a leaf — no workspace deps) + objio/ object-store buckets for sc+wal:// and sc+s3:// remotes (the ONLY crate + that links the S3 SDK; a leaf — no workspace deps) repo/ durable .sc repo — refs, branches, working tree, merge, history editing, remotes, protection, GC, sparse/partial clone, transcripts, - transport + transport (ssh, sc+http(s), bucket WAL) cli/ `sc` binary +apps/ + desktop/ read-only Tauri v2 + React/TypeScript repository browser (see + apps/desktop/README.md); calls `repo` directly, never the `sc` binary demo/ - 25 end-to-end proof scripts covering everything below, e.g.: + 27 end-to-end proof scripts covering everything below, e.g.: run_demo.sh independent before/after filesystem diff, zero residue run_repo_demo.sh persistent repo end-to-end demo run_ssh_remote_demo.sh ssh-native transport round trip @@ -73,10 +82,13 @@ demo/ run_transcript_demo.sh sealed agent-session transcript proof ``` -Dependency direction is strict: `cli → repo → {vfs, gitio, crypto, tlsio} → core` -(`tlsio` is a leaf). Only `gitio` links `gix`, only `crypto` links RustCrypto, -and only `tlsio` links rustls, keeping the rest of the system Git-, crypto-, -and TLS-library-agnostic. +Dependency direction is strict: the adapters `cli` and `desktop` sit on top of +`repo`; `repo → {vfs, crypto} → core`, plus the leaf edges `repo → tlsio` and +`repo → objio` (`tlsio` and `objio` depend on no other workspace crate). `gitio` +depends only on `core`, and `repo` never links it: `cli` links both and passes +imported snapshots down. Only `gitio` links `gix`, only `crypto` links +RustCrypto, only `tlsio` links rustls, and only `objio` links the S3 SDK, +keeping the rest of the system Git-, crypto-, TLS- and object-store-agnostic. ## Quick start @@ -113,9 +125,10 @@ cargo run --bin sc -- export --to /path/to/git/repo bash demo/run_demo.sh ``` -See `CLAUDE.md`'s Commands section for the full command surface (merge, -history editing, secrets lifecycle, sparse/partial clone, ssh/HTTP remotes, -session transcripts, and more). +Run `sc --help` (and `sc --help`) for the full command surface: +merge, history editing, secrets lifecycle, sparse/partial clone, ssh/HTTP(S)/ +bucket remotes, session transcripts, and more. The annotated reference lives in +the `sc-cli-reference` skill (`.claude/skills/sc-cli-reference/SKILL.md`). ## How Phase 1 works @@ -142,13 +155,19 @@ permissions via encrypted paths (Phase 7, randomized sealing since Phase 33) all ship alongside a full persistent, branchable, content-addressed VCS — merge and history editing, durable multi-invocation agent sessions with auto-merge, a commit-time secret scanner, secret/permission lifecycle -(rotation, escrow, revocation tombstones, bulk rewrap), signed commits with -sealed session-transcript provenance, sparse and partial checkouts, and local, -ssh-native, sc-native HTTP/TLS (with bearer-token auth and listener resource -limits), and Git (including hosted GitHub) network transports, plus private -branches sealed to a recipient set until publish. All 34 phases are -implemented and tested; `demo/` holds 26 independent end-to-end proofs, one -per major capability. Remaining follow-ons (transparent lazy-fetch, -per-case gap-tolerant merge on partial clones, CA-path validation alongside -TOFU pinning, rotate-for-paths, and similar hardening items) are tracked in -[ROADMAP.md](ROADMAP.md)'s Deferred section rather than being open MVP scope. +(rotation, escrow, revocation tombstones, bulk rewrap, `sc rewrap --rotate` +for protected paths), signed commits with sealed session-transcript +provenance, sparse and partial checkouts (lazy want-by-id fetch, merge, +cherry-pick, rebase and agent sessions on a partial clone), and local, +ssh-native, sc-native HTTP/TLS (bearer tokens with expiry and rotation, +listener resource limits, TOFU pinning managed by `sc tls`, or CA validation +via `SC_HTTPS_CA_FILE`/`SC_HTTPS_CA=system`), bucket (`sc+wal://`, `sc+s3://`), +and Git (including hosted GitHub) network transports, plus private branches +sealed to a recipient set until publish and a read-only Tauri desktop +browser. All 36 phases and P37a-37d are implemented and tested; `demo/` holds +27 independent end-to-end proofs, one per major capability. Remaining +follow-ons (transcript delivery gaps on drop and bucket remotes, bucket +compaction and gc, named concurrent `sc ws` sessions, size/count clone +filters, an operator-configurable object-size cap, and similar hardening +items) are tracked in [ROADMAP.md](ROADMAP.md)'s Deferred section and as open +GitHub issues rather than being open MVP scope. diff --git a/SECURITY.md b/SECURITY.md index 85546793..b3a71aef 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -17,16 +17,23 @@ it are consolidated in [`docs/THREAT-MODEL.md`](docs/THREAT-MODEL.md). In brief: nonce per seal). Content sealed *before* P33 used convergent encryption and remains **equality-confirmable forever** — rotation cannot erase ciphertext already in history. -- **Two confidential transports ship:** `sc+https://` (in-binary TLS with - accept-new TOFU pinning; the *first* connection to a host is unverified by - construction, and `SC_HTTPS_FINGERPRINT` / `SC_HTTPS_STRICT=1` close that - window) and `ssh://`. Plain `sc serve --http` remains **plaintext** and its - bearer tokens cross the wire in the clear — use it only on loopback or behind - the documented stream-mode proxy setup; prefer `sc+https://` for any public - bind. +- **Two confidential sc-native transports ship:** `sc+https://` (in-binary + TLS; by default accept-new TOFU pinning, so the *first* connection to a host + is unverified by construction, and `SC_HTTPS_FINGERPRINT`, `sc tls pin` or + `SC_HTTPS_STRICT=1` close that window; `SC_HTTPS_CA_FILE` / `SC_HTTPS_CA=system` + switch a client to CA validation instead, with no revocation checking) and + `ssh://`. Plain `sc serve --http` remains **plaintext** and its bearer + tokens (which can carry an expiry and be rotated) cross the wire in the + clear — use it only on loopback or behind the documented stream-mode proxy + setup; prefer `sc+https://` for any public bind. +- **Bucket remotes (`sc+wal://`, `sc+s3://`) put the bucket's ACL on the + perimeter.** Public content sits in the bucket as plaintext at rest and a + writer with raw bucket access can move refs outside the `sc` client; sealed + content stays ciphertext. - Committed secrets injected by `sc run` live in an **authorized local process context, not strong isolation** — same-user processes can observe the child - environment. + environment. `sc run --secrets-fd N` keeps them out of the environment but + is still not isolation. - **Rotation/revocation ≠ erasure.** They cut off *future* reads through the current registry but cannot erase ciphertext already in history and copied to other clones; real cutover of a leaked credential always means rotating the diff --git a/docs/agents/ACCESS.md b/docs/agents/ACCESS.md index d9d3bf26..85f014fa 100644 --- a/docs/agents/ACCESS.md +++ b/docs/agents/ACCESS.md @@ -29,6 +29,7 @@ it as the revocation checklist. Written as OSTIF-audit follow-up T-14 (G-012). | `GITHUB_TOKEN` — `ci.yml` | `contents: read` | Ephemeral per-run. | | `GITHUB_TOKEN` — `codeql.yml` | `contents: read` top-level; analyze job adds `security-events: write`, `actions: read` | Ephemeral per-run. | | `GITHUB_TOKEN` — `audit.yml` | `contents: read`, `checks: write`, `issues: write` (job-scoped as of audit T-20) | Files RustSec findings as public issues. | +| `GITHUB_TOKEN` — `fuzz.yml`, `sbom.yml`, `vendor-provenance.yml` | `contents: read` | Ephemeral per-run. Nightly `wire_decode` fuzz, weekly CycloneDX SBOMs, and the vendored-glib provenance check; none holds a write scope. | | `GITHUB_TOKEN` — `scorecard_analysis.yml` | `read-all` top-level; job adds `security-events: write`, `id-token: write` | Publishes Scorecard results. | | AI agents (Claude Code et al.) | `gh` CLI under the maintainer's auth: issue/PR create, comment, label, close; local commit + push | Operate the public triage pipeline (`issue-tracker.md`). **Excluded** from draft security advisories and private reports; security fixes need distinct human review (see vulnerability-response.md). Compromise playbook: incident-response class 2. | | Dependabot | Version + security update PRs (cargo, npm, github-actions) | PRs merge only through the standing required checks. | diff --git a/docs/agents/domain.md b/docs/agents/domain.md index 90d4de6b..afa6de45 100644 --- a/docs/agents/domain.md +++ b/docs/agents/domain.md @@ -14,7 +14,7 @@ currently live in two root files — treat them as the glossary/context source: pointer to it. The full per-phase narrative log is archived at `docs/archive/claude-md-phase-log-2026-07.md`. - **`ARCHITECTURE.md`** at the repo root — the full design and rationale. -- **`docs/adr/`** — 43+ ADRs (0001…) recording every significant decision; +- **`docs/adr/`** — 54 ADRs (0001…0054) recording every significant decision; read the ADRs that touch the area you're about to work in. This is a single-context repo, so there are no per-crate `crates//docs/adr/` dirs — all decisions live in the one root `docs/adr/`. @@ -33,11 +33,12 @@ Single-context repo: / ├── CLAUDE.md ← working guide + capability map (AGENTS.md points here) ├── ARCHITECTURE.md ← full design + rationale -├── docs/adr/ ← 0001…0043+ architectural decisions (one root collection) -└── crates/ ← core, vfs, gitio, crypto, tlsio, repo, cli (one cohesive VCS domain) +├── docs/adr/ ← 0001…0054 architectural decisions (one root collection) +├── crates/ ← core, vfs, gitio, crypto, tlsio, objio, repo, cli (one cohesive VCS domain) +└── apps/desktop/ ← Tauri v2 read-only browser (a top-level adapter over repo) ``` -(This is a seven-crate Rust workspace but **one** domain — a next-gen VCS — so a +(This is an eight-crate Rust workspace plus a desktop app but **one** domain — a next-gen VCS — so a single context, not a multi-context `CONTEXT-MAP.md` layout.) ## Use the project's vocabulary From f7a6c649ae730ad79b70d7aec2eaedfb9436622f Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Tue, 20 Oct 2026 10:30:12 +0300 Subject: [PATCH 02/11] docs: describe the bucket CAS retry before NonFastForward Co-Authored-By: Claude Sonnet 5.5 --- ARCHITECTURE.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 57fcec98..2a7f39ca 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1108,7 +1108,9 @@ the local, ssh and http remotes on top of a flat key/value bucket, structured as a write-ahead log of **immutable, content-addressed packs** plus **one compare-and-swapped manifest**. A push uploads a pack, appends a parent-linked `log/` entry carrying its ref updates, then CASes the manifest's -`head_seq` forward (`Bucket::put_if_tag`); losing the CAS is `NonFastForward`, +`head_seq` forward (`Bucket::put_if_tag`). A lost CAS refreshes and retries (up +to 16 times), so pushes to different branches both land; a push whose target +branch no longer matches its expected old tip fails with `NonFastForward`, exactly as on any other transport, and a loser's orphaned log entry is invisible to readers because they walk `parent_seq` back from the manifest. There is no leader, lock file or lease. The new leaf crate `objio` holds the `Bucket` trait From 5fcf39f6db4f63fa32bebd1fc2a3c88ec1db34aa Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Tue, 20 Oct 2026 10:35:02 +0300 Subject: [PATCH 03/11] docs: fix stale read-only, pin-path and partial-clone walk statements Co-Authored-By: Claude Sonnet 5.5 --- ARCHITECTURE.md | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 2a7f39ca..4257b1bc 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -356,7 +356,8 @@ phases but the shape is stable: wrote into the working tree — an interrupted merge/pick/rebase resumes or aborts cleanly from these. - The `sc+https://` client's pin file is user-level, not part of `.sc/` - (`~/.config/sc/known_hosts`, managed with `sc tls`). + (`$SC_HTTPS_KNOWN_HOSTS`, else `$XDG_CONFIG_HOME/sc/known_hosts`, else + `~/.config/sc/known_hosts`; managed with `sc tls`). ### Git-like working tree @@ -905,8 +906,8 @@ explicitly and offline — no lazy-fetch from inside a read path. Building a *new* commit on a partial clone needed real machinery beyond carry-by-id: `worktree::graft_out_of_sparse` splices the tip's untouched out-of-filter subtrees back into a freshly built root by id, never reading -their content, so push's filtered reachability walk still sends only the -client's new in-filter objects. `commit` refuses (`Error::GappedPathContent`) +their content, so push's filtered reachability walk (until P37b-i made it +presence-based) still sent only the client's new in-filter objects. `commit` refuses (`Error::GappedPathContent`) rather than silently drop content under an unfetched subtree. `sc backfill --all` is the genuine full-clone escape hatch: fetch everything, verify the closure, then remove `.sc/promisor`. P27 refused merge, replay, `sc ws @@ -930,15 +931,16 @@ ADR-0039. `sc serve --http` gains three composed access-control gates at the connection opening, closing P26's stated "unauthenticated" boundary: a -read-only mode that rejects any write-shaped request before it reaches -`wire::serve`, a fail-closed bind check, and bearer-token authentication +read-only mode that rejects each mutating request inside +`wire::serve_with_policy` before any store write, a fail-closed bind check, and bearer-token authentication against a `.sc/serve-tokens.toml` allowlist. `--read-only` forces the read-only gate independent of tokens (a rejected write gets the typed `EC_READONLY` wire error), so a public-read mirror needs no token management at -all. Each gate runs before -the P25/P26 wire handshake begins, matching P26's own status-line-before- -handshake discipline, so a rejected connection never reaches -`LocalTransport`. Tokens later gained an optional expiry (`sc serve token add +all. The bind and token gates +run before the P25/P26 wire handshake begins, matching P26's own +status-line-before-handshake discipline, so a connection they reject never +reaches `LocalTransport`; the read-only gate is enforced per request after the +handshake. Tokens later gained an optional expiry (`sc serve token add --expires`; an expired token gets the same 401 as an unknown one) and `sc serve token rotate` (ADR-0040 addendum). See ADR-0040. @@ -996,8 +998,9 @@ the URL is `sc+https://`; the server wraps before the opening read when byte-for-byte unchanged; `PROTOCOL_VERSION` stays 3. Trust is accept-new TOFU, the SSH `known_hosts` shape: the client pins -`SHA-256(SPKI DER)` per host in `~/.config/sc/known_hosts` -(`SC_HTTPS_KNOWN_HOSTS` overrides; `SC_HTTPS_FINGERPRINT` pre-pins for CI; +`SHA-256(SPKI DER)` per host in the pin file (`$SC_HTTPS_KNOWN_HOSTS`, else +`$XDG_CONFIG_HOME/sc/known_hosts`, else `~/.config/sc/known_hosts`; +`SC_HTTPS_FINGERPRINT` pre-pins for CI; `SC_HTTPS_STRICT=1` refuses unknown hosts). A pin mismatch always hard-fails. The server auto-mints a self-signed identity into `.sc/serve-tls/` (key `0600`) unless PEM is supplied; `sc serve fingerprint` @@ -1006,7 +1009,7 @@ list/forget/pin/path` manages the pin file (ADR-0053), and a client can opt into CA validation instead of pinning with `SC_HTTPS_CA_FILE=` or `SC_HTTPS_CA=system` (ADR-0054; fail-closed, no revocation checking, the pin file is neither read nor written). Client connect, TLS handshake and opening -are bounded to 30 s, and bracketed IPv6 hosts parse in `sc+http(s)://` URLs. +are each held to a 30 s inactivity bound, and bracketed IPv6 hosts parse in `sc+http(s)://` URLs. **Gate change:** a non-loopback bind now requires `--read-only`, `--allow-public`, or (`--tls` AND ≥1 token) — P29's "tokens alone justify a public bind" is narrowed, @@ -1142,7 +1145,8 @@ with the objects it can serve plus the ids it will not send, with no haves and no reachability walk. `Store::get` stays network-free; the only code that dials the promisor after clone is `Repo::fetch_by_id_from` in `crates/repo/src/ fill.rs`, which verifies the reply against the request before writing. A fill -only adds objects, so push, gc and verify (filter-scoped walks) are unchanged. +only adds objects; push, gc and verify were filter-scoped walks when P37a +shipped and became presence-based in P37b-i (below). Sparse widening over a filled path and the lifted P27 refusals needed presence-based walks and are P37b-P37d (below). See ADR-0047 and `docs/research/scion-evaluation.md`. From 8f5a64df31a75f5699080ac10bfc3a1f72ec0697 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Tue, 20 Oct 2026 10:40:14 +0300 Subject: [PATCH 04/11] docs: separate opening gates from per-request read-only enforcement Co-Authored-By: Claude Sonnet 5.5 --- ARCHITECTURE.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 4257b1bc..3dbc2e9e 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -929,11 +929,11 @@ ADR-0039. ## Phase 29 — sc+http access control (built) -`sc serve --http` gains three composed access-control gates at the -connection opening, closing P26's stated "unauthenticated" boundary: a -read-only mode that rejects each mutating request inside -`wire::serve_with_policy` before any store write, a fail-closed bind check, and bearer-token authentication -against a `.sc/serve-tokens.toml` allowlist. `--read-only` forces the +`sc serve --http` gains three composed access-control gates, closing P26's +stated "unauthenticated" boundary: a fail-closed bind check and bearer-token +authentication against a `.sc/serve-tokens.toml` allowlist, both at the +connection opening, and a read-only mode that rejects each mutating request +inside `wire::serve_with_policy` before any store write. `--read-only` forces the read-only gate independent of tokens (a rejected write gets the typed `EC_READONLY` wire error), so a public-read mirror needs no token management at all. The bind and token gates From ec645466eaedf428f5ea4301d150e73d2be869f3 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Tue, 20 Oct 2026 10:53:29 +0300 Subject: [PATCH 05/11] docs: note bucket signature/transcript gap (#229), fix CLAUDE.md dependency chain Co-Authored-By: Claude Sonnet 5.5 --- ARCHITECTURE.md | 8 ++++++-- CLAUDE.md | 2 +- README.md | 3 ++- SECURITY.md | 4 +++- 4 files changed, 12 insertions(+), 5 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 3dbc2e9e..9cb45fc8 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1132,8 +1132,12 @@ identity and scratch space while all served content lives in the bucket, so instances are disposable. Public content sits in the bucket as plaintext at rest (the bucket ACL is the perimeter; sealed content stays ciphertext), and `sc clone --filter` is refused against a bucket remote. Bucket compaction and gc -are not built (open issue #216), and bucket remotes do not carry indexed -transcripts (#229). See ADR-0046 and `docs/research/walgit-evaluation.md`. +are not built (open issue #216). A bucket's `get_pack` is reachability-only, so +signatures and transcripts do not travel through it (open issue #229): a signed +commit pushed to a bucket remote shows as unsigned after a later fetch or clone +from that bucket (including through `sc serve --store`), and attached +transcripts are stored but never served back. Verify signed history from a +non-bucket remote until that is fixed. See ADR-0046 and `docs/research/walgit-evaluation.md`. ## Phase 37a — want-by-id fetch (built) diff --git a/CLAUDE.md b/CLAUDE.md index 1ae33c22..de7a2844 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -36,7 +36,7 @@ recorded as an ADR in `docs/adr/`. Keep them in sync when the design changes. Crate roles are what `ls crates/` + each `Cargo.toml` say; the rules are what matters. Strict dependency direction: top-level adapters `{cli, desktop} → repo → -{vfs, gitio, crypto} → core`, with the separate leaf edges `repo → tlsio` +{vfs, crypto} → core` (`gitio → core` is linked by `cli`, never by `repo`), with the separate leaf edges `repo → tlsio` **and `repo → objio`** (`tlsio` and `objio` are leaves — each depends on no other workspace crate, not even `core`). **`core` must never depend on Git, worktrees, or crypto.** **`gix` must stay quarantined in `gitio`** — if you diff --git a/README.md b/README.md index fa4da513..da15ac27 100644 --- a/README.md +++ b/README.md @@ -166,7 +166,8 @@ and Git (including hosted GitHub) network transports, plus private branches sealed to a recipient set until publish and a read-only Tauri desktop browser. All 36 phases and P37a-37d are implemented and tested; `demo/` holds 27 independent end-to-end proofs, one per major capability. Remaining -follow-ons (transcript delivery gaps on drop and bucket remotes, bucket +follow-ons (signatures and transcripts not travelling through bucket remotes +(#229), transcript drop delivery gaps, bucket compaction and gc, named concurrent `sc ws` sessions, size/count clone filters, an operator-configurable object-size cap, and similar hardening items) are tracked in [ROADMAP.md](ROADMAP.md)'s Deferred section and as open diff --git a/SECURITY.md b/SECURITY.md index b3a71aef..ce4f5c84 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -29,7 +29,9 @@ it are consolidated in [`docs/THREAT-MODEL.md`](docs/THREAT-MODEL.md). In brief: - **Bucket remotes (`sc+wal://`, `sc+s3://`) put the bucket's ACL on the perimeter.** Public content sits in the bucket as plaintext at rest and a writer with raw bucket access can move refs outside the `sc` client; sealed - content stays ciphertext. + content stays ciphertext. Signatures and transcripts do not travel through a + bucket (#229): a signed commit pushed there reads as unsigned after a fetch or + clone from the bucket, so `sc verify --require` cannot vouch for it. - Committed secrets injected by `sc run` live in an **authorized local process context, not strong isolation** — same-user processes can observe the child environment. `sc run --secrets-fd N` keeps them out of the environment but From 7fcb8ed2d9d6dc8f20d3459ffe12e22f085f6e44 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Tue, 20 Oct 2026 11:00:35 +0300 Subject: [PATCH 06/11] docs: describe sc transcript drop and signed tombstones Co-Authored-By: Claude Sonnet 5.5 --- ARCHITECTURE.md | 17 ++++++++++++++++- README.md | 2 +- SECURITY.md | 5 +++++ 3 files changed, 22 insertions(+), 2 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 9cb45fc8..bec8dfd0 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -335,6 +335,8 @@ phases but the shape is stable: so `sc undo`/redo can invert it (P14). Local-only, like a reflog. - `signatures` — snapshot→signature-object index (P22); `transcripts` — snapshot→transcript-object index, one-to-many (P30). Both gc-rooted, local-only. + `transcript-drops` — the derived set of dropped transcripts (`sc transcript + drop`), rebuilt from the store on clone. - `sparse` — the local, uncommitted sparse-checkout prefix spec (P24); `promisor` — a partial clone's fetch-filter + origin (P27). - `serve-tokens.toml` — bearer-token access control for `sc serve --http` (P29); @@ -964,7 +966,20 @@ a transfer-relevant snapshot, and the receiver reindexes idempotently. `sc gc` roots live transcript ids into reachability before signature pruning runs, so a signed transcript survives exactly as long as the transcript itself does. `sc export --to ` drops transcripts (no Git-native -form exists) and reports a count. See ADR-0038. +form exists) and reports a count. + +`sc transcript drop [--identity ]` removes a transcript from view +without a new object tag or wire change: a drop is a **tombstone**, a +`Transcript` object with an empty ciphertext and wrap list that names the +dropped id, recorded in a local `.sc/transcript-drops` index and treated as a +grow-only set (no undrop; re-attach to restore). Reads, `sc log` markers and +the over-send filter through the drop set, and `sc gc` reclaims the body. A +tombstone travels with fetch and push only when it is signed, and a receiving +peer honors it only if the signature comes from a key in its `[signers] +trusted` list; an unsigned drop is local-only, and an untrusted tombstone stays +inert. Drop is not erasure: a peer that never receives an honored tombstone +keeps the sealed body, and a remote that received a tombstone before trusting +its signer keeps it inert (#230). See ADR-0038 and its addendum. ## Phase 31 — listener resource limits (built) diff --git a/README.md b/README.md index da15ac27..23d13fde 100644 --- a/README.md +++ b/README.md @@ -157,7 +157,7 @@ merge and history editing, durable multi-invocation agent sessions with auto-merge, a commit-time secret scanner, secret/permission lifecycle (rotation, escrow, revocation tombstones, bulk rewrap, `sc rewrap --rotate` for protected paths), signed commits with sealed session-transcript -provenance, sparse and partial checkouts (lazy want-by-id fetch, merge, +provenance (droppable through signed tombstones), sparse and partial checkouts (lazy want-by-id fetch, merge, cherry-pick, rebase and agent sessions on a partial clone), and local, ssh-native, sc-native HTTP/TLS (bearer tokens with expiry and rotation, listener resource limits, TOFU pinning managed by `sc tls`, or CA validation diff --git a/SECURITY.md b/SECURITY.md index ce4f5c84..53cee26d 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -36,6 +36,11 @@ it are consolidated in [`docs/THREAT-MODEL.md`](docs/THREAT-MODEL.md). In brief: context, not strong isolation** — same-user processes can observe the child environment. `sc run --secrets-fd N` keeps them out of the environment but is still not isolation. +- **`sc transcript drop` is not erasure either.** A drop travels as a signed + tombstone that a peer honors only when the signature comes from a key in its + `[signers] trusted` list; an unsigned drop is local-only, and any trusted + signer can hide transcripts on every peer that trusts them. Peers without + the tombstone keep the sealed body. - **Rotation/revocation ≠ erasure.** They cut off *future* reads through the current registry but cannot erase ciphertext already in history and copied to other clones; real cutover of a leaked credential always means rotating the From a1cf3471d236182375ab1c24be0e61afb67323f6 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Tue, 20 Oct 2026 11:05:05 +0300 Subject: [PATCH 07/11] docs: give bucket remote syntax in README until the CLI skill covers it Co-Authored-By: Claude Sonnet 5.5 --- README.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 23d13fde..ef42571a 100644 --- a/README.md +++ b/README.md @@ -128,7 +128,10 @@ bash demo/run_demo.sh Run `sc --help` (and `sc --help`) for the full command surface: merge, history editing, secrets lifecycle, sparse/partial clone, ssh/HTTP(S)/ bucket remotes, session transcripts, and more. The annotated reference lives in -the `sc-cli-reference` skill (`.claude/skills/sc-cli-reference/SKILL.md`). +the `sc-cli-reference` skill (`.claude/skills/sc-cli-reference/SKILL.md`), which +does not yet cover bucket remotes. Those take `sc remote add +sc+wal://` or `sc+s3:///` (see ADR-0046), and `sc serve +--store ` serves a bucket from a disposable instance (`sc serve --help`). ## How Phase 1 works From b2b32b9e61dd9e30f00349fd61f27135c8469e11 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Tue, 20 Oct 2026 12:18:39 +0300 Subject: [PATCH 08/11] docs: update object-size, by-id ceiling, bucket cap and tombstone wording Co-Authored-By: Claude Sonnet 5.5 --- ARCHITECTURE.md | 20 +++++++++++++------- README.md | 2 +- 2 files changed, 14 insertions(+), 8 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index d0e8a216..eb80e7e4 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -974,12 +974,16 @@ without a new object tag or wire change: a drop is a **tombstone**, a dropped id, recorded in a local `.sc/transcript-drops` index and treated as a grow-only set (no undrop; re-attach to restore). Reads, `sc log` markers and the over-send filter through the drop set, and `sc gc` reclaims the body. A -tombstone travels with fetch and push only when it is signed, and a receiving -peer honors it only if the signature comes from a key in its `[signers] +tombstone travels with fetch and push only when it is signed (a push that finds +the remote already at the same branch tip sends nothing, so a drop made +afterwards propagates once a push moves the branch or the remote fetches), and +a receiving peer honors it only if the signature comes from a key in its `[signers] trusted` list; an unsigned drop is local-only, and an untrusted tombstone stays inert. Drop is not erasure: a peer that never receives an honored tombstone keeps the sealed body, and a remote that received a tombstone before trusting -its signer keeps it inert (#230). See ADR-0038 and its addendum. +its signer keeps it inert (#230). `sc clone|fetch|push --no-transcripts` leaves +transcript objects out of one transfer (per command, not remembered; no effect +on git or bucket remotes). See ADR-0038 and its addendum. ## Phase 31 — listener resource limits (built) @@ -1136,9 +1140,10 @@ and its two backends (`DirBucket` for `sc+wal://`, `S3Bucket` for `sc+s3:///`), quarantining the S3 SDK the way `gix` and RustCrypto are quarantined. -Reads treat the bucket as untrusted input: manifest, log entries and pack -records are capped at `MAX_OBJECT_SIZE`, and every WAL record decodes strictly -and fails closed. P36b adds **checkpoint folding**: once the log tail passes 64 +Reads treat the bucket as untrusted input: WAL metadata (manifest, log +entries, checkpoints, idx) is capped at a fixed 256 MiB, pack records follow the +live max object size (`scl_core::max_object_size()`), and every WAL record +decodes strictly and fails closed. P36b adds **checkpoint folding**: once the log tail passes 64 entries a push opportunistically writes `checkpoints/` (branch tips plus a cumulative pack list) and points the manifest at it, so a cold reader seeds from the checkpoint and walks only the tail. P36c adds `sc serve --store `: an @@ -1183,7 +1188,8 @@ top of `materialize`, plus `undo` and `sparse set`), and a partial clone fetches from its origin only. 37b-ii lets a lazy, online clone fill the view's closure by id (`ensure_view`) before switch, fast-forward, undo and `sparse set`; `sparse set` may widen past the filter; and a by-id reply is capped at -`min(--max-pack-size, 512 MiB)` and split when refused. `sc export` and `sparse +`min(--max-pack-size, 2 GiB)` (the 2 GiB ceiling is twice the largest settable +object cap; the client reads at most 4 GiB) and split when refused. `sc export` and `sparse disable` stay refused on a partial clone. See ADR-0048. ## Phase 37c — merge, cherry-pick and rebase on a partial clone (built) diff --git a/README.md b/README.md index ef42571a..cc38666e 100644 --- a/README.md +++ b/README.md @@ -172,6 +172,6 @@ browser. All 36 phases and P37a-37d are implemented and tested; `demo/` holds follow-ons (signatures and transcripts not travelling through bucket remotes (#229), transcript drop delivery gaps, bucket compaction and gc, named concurrent `sc ws` sessions, size/count clone -filters, an operator-configurable object-size cap, and similar hardening +filters, and similar hardening items) are tracked in [ROADMAP.md](ROADMAP.md)'s Deferred section and as open GitHub issues rather than being open MVP scope. From e6ead9df1d2e14d6d8970614522a9bb13f38648a Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Tue, 20 Oct 2026 12:18:54 +0300 Subject: [PATCH 09/11] docs: include a serving mode in the bucket serve example Co-Authored-By: Claude Sonnet 5.5 --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index cc38666e..8102fc57 100644 --- a/README.md +++ b/README.md @@ -131,7 +131,8 @@ bucket remotes, session transcripts, and more. The annotated reference lives in the `sc-cli-reference` skill (`.claude/skills/sc-cli-reference/SKILL.md`), which does not yet cover bucket remotes. Those take `sc remote add sc+wal://` or `sc+s3:///` (see ADR-0046), and `sc serve ---store ` serves a bucket from a disposable instance (`sc serve --help`). +--http 127.0.0.1:8730 --store ` serves a bucket from a +disposable instance (`--stdio` also works; `sc serve --help`). ## How Phase 1 works From 96810deeb9e78bbd6fa7829e14bb1ce8b98ff365 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Tue, 20 Oct 2026 12:43:51 +0300 Subject: [PATCH 10/11] docs: distinguish receive-side from send-side --no-transcripts Co-Authored-By: Claude Sonnet 5.5 --- ARCHITECTURE.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 9edb93e1..c56325c7 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -984,9 +984,12 @@ a receiving peer honors it only if the signature comes from a key in its `[signe trusted` list; an unsigned drop is local-only, and an untrusted tombstone stays inert. Drop is not erasure: a peer that never receives an honored tombstone keeps the sealed body, and a remote that received a tombstone before trusting -its signer keeps it inert (#230). `sc clone|fetch|push --no-transcripts` leaves -transcript objects out of one transfer (per command, not remembered; no effect -on git remotes). See ADR-0038 and its addendum. +its signer keeps it inert (#230). `sc push --no-transcripts` leaves transcripts and their signatures out of the +pack (saving bandwidth); `sc clone|fetch --no-transcripts` cannot, because the +server still sends the full pack, so the receiver hash-checks every record and +then discards transcripts and their signatures, writing and indexing none +(saving disk and exposure, not bandwidth). It is per command, not remembered, +and has no effect on git remotes, which carry none. See ADR-0038 and its addendum. ## Phase 31 — listener resource limits (built) From 3e100980d50694f7cb5194b688e11988ef9f3a46 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Tue, 20 Oct 2026 12:56:27 +0300 Subject: [PATCH 11/11] docs: qualify --secrets-fd as unix-only and the identical-subtree merge carry Co-Authored-By: Claude Sonnet 5.5 --- ARCHITECTURE.md | 3 ++- SECURITY.md | 4 ++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index c56325c7..7d1cd135 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1204,7 +1204,8 @@ disable` stay refused on a partial clone. See ADR-0048. `merge_trees` is a tree-recursive three-way merge that carries a subtree **by id** when the sides agree and no protection rule applies, reads only where the -sides differ or a rule intersects, and on a lazy, online clone fills what it +sides differ or a rule intersects (a subtree with the same id on all three sides +is carried unread even under a protection rule), and on a lazy, online clone fills what it asks for by id outside the store lock (a non-lazy clone gets the gap error with the `sc backfill ` hint). A carried protected subtree keeps the complete input wrap union. A conflict inside the view is marked and resolved like on a diff --git a/SECURITY.md b/SECURITY.md index 40e7dd0e..7a0b9932 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -34,8 +34,8 @@ it are consolidated in [`docs/THREAT-MODEL.md`](docs/THREAT-MODEL.md). In brief: honors the drop only for a signer in its `[signers] trusted` list. - Committed secrets injected by `sc run` live in an **authorized local process context, not strong isolation** — same-user processes can observe the child - environment. `sc run --secrets-fd N` keeps them out of the environment but - is still not isolation. + environment. `sc run --secrets-fd N` (unix only; other platforms refuse it) keeps them + out of the environment but is still not isolation. - **`sc transcript drop` is not erasure either.** A drop travels as a signed tombstone that a peer honors only when the signature comes from a key in its `[signers] trusted` list; an unsigned drop is local-only, and any trusted