Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
7206376
feat(repo): deliver drop tombstones after gc and after late signer tr…
tonibergholm-codento Oct 20, 2026
fd9494c
fix(repo): keep a pending tombstone on an unreadable object; record u…
tonibergholm-codento Oct 20, 2026
4fdc63c
docs,test: sync the drop addendum with #230; assert test roots are go…
tonibergholm-codento Oct 20, 2026
20eeae7
fix(repo): address #230 review: serve-path trigger, late signatures, …
tonibergholm-codento Oct 20, 2026
722c475
fix(repo): cap signature work inside tombstone_honored after selectin…
tonibergholm-codento Oct 20, 2026
f83412f
Merge origin/main into feat/230-tombstone-delivery
tonibergholm-codento Oct 20, 2026
6fc997b
fix(repo): skip recorded drops on ingest, dedupe pending batches, fai…
tonibergholm-codento Oct 20, 2026
364c74c
fix(repo): do not read the pending list for an empty batch (ref #230)
tonibergholm-codento Oct 20, 2026
ccdf3a8
Merge remote-tracking branch 'origin/main' into feat/230-tombstone-de…
tonibergholm-codento Oct 20, 2026
acb618d
fix(repo): verify signatures before indexing, bound the serve-path tr…
tonibergholm-codento Oct 20, 2026
d066dbb
fix(repo): keep the trust fingerprint when a pending entry cannot be …
tonibergholm-codento Oct 20, 2026
05b5114
fix(repo): reevaluate_pending is test-only now (ref #230)
tonibergholm-codento Oct 20, 2026
9019ff4
fix(repo): fsync the .sc dir when the drop index is created; record o…
tonibergholm-codento Oct 20, 2026
ea45995
Merge remote-tracking branch 'origin/main' into feat/230-tombstone-de…
tonibergholm-codento Oct 20, 2026
e9aae44
fix(repo): one trust snapshot per pass, bounded retries, torn-tail-sa…
tonibergholm-codento Oct 20, 2026
b19cabf
docs: forged signatures stay in the CAS but are never indexed (ref #230)
tonibergholm-codento Oct 20, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .claude/skills/sc-cli-reference/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,11 @@ cargo run --bin sc -- transcript drop <id> [--identity <key>] # drop a transcr
# body (loose: after the grace window; already packed: next gc). Not erasure. --identity
# signs the tombstone; peers honor it only if that key
# is under [signers] trusted (unsigned = local-only).
# Propagates on the peer's next fetch (ADR-0038 addendum)
# Propagates on the peer's next fetch (ADR-0038 addendum);
# a signed tombstone also rides when the dropper's gc
# collected its snapshot, and a remote holding one from
# before it trusted the signer honors it once its
# trusted-signer set changes (#230)
cargo run --bin sc -- ws harvest --transcript <path> [--sign] [--identity <key>] # attach the
# file to each harvested workspace's landed snapshot
# (P30; landing status prints before the attach)
Expand Down
17 changes: 11 additions & 6 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -809,7 +809,9 @@ key and an Ed25519 signing key in a single `scl-id-<hex>` file (a v1
`scl-sk-` identity still parses and encrypts, but can't sign). Verification
(`sig_status`) is strict four-state — `Invalid` beats `Trusted` beats
`Untrusted` beats `Unsigned` — and `sc verify [<ref>] [--require]` walks
every parent, not just the mainline.
every parent, not just the mainline. A received signature is verified before
it is indexed and a forged one is dropped (ADR-0038 addendum, #230), so a
transfer cannot make a snapshot `Invalid`.

Transfer needed zero wire changes: signatures ride the existing pack as
ordinary objects, reindexed by id on receipt. `recipients.toml` gains
Expand Down Expand Up @@ -988,14 +990,17 @@ without a new object tag or wire change: a drop is a **tombstone**, a
dropped id, recorded in a local `.sc/transcript-drops` index and treated as a
grow-only set (no undrop; re-attach to restore). Reads, `sc log` markers and
the over-send filter through the drop set, and `sc gc` reclaims the body. A
tombstone travels with fetch and push only when it is signed (a push that finds
the remote already at the same branch tip sends nothing, so a drop made
afterwards propagates once a push moves the branch or the remote fetches), and
tombstone travels with fetch and push only when it is signed, and a
retained signed tombstone rides a transfer when its snapshot is in it or was
collected by the sender's gc, so a stale peer is told (a push that finds the remote already at
the same branch tip sends nothing, so a drop made afterwards propagates once a
push moves the branch or the remote fetches), and
a receiving peer honors it only if the signature comes from a key in its `[signers]
trusted` list; an unsigned drop is local-only, and an untrusted tombstone stays
inert. Drop is not erasure: a peer that never receives an honored tombstone
keeps the sealed body, and a remote that received a tombstone before trusting
its signer keeps it inert (#230). `sc push --no-transcripts` leaves transcripts and their signatures out of the
keeps the sealed body. A tombstone held before its signer was trusted is
remembered in `.sc/transcript-pending` (capped at 1024, newest kept) and
honored on the next open or ingest once the signer is trusted (#230). `sc push --no-transcripts` leaves transcripts and their signatures out of the
pack (saving bandwidth); `sc clone|fetch --no-transcripts` cannot, because the
server still sends the full pack, so the receiver hash-checks every record and
then discards transcripts and their signatures, writing and indexing none
Expand Down
8 changes: 5 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ the code, those win.
| P27 | Partial clone: `sc clone --filter <prefix…>`, `.sc/promisor`, `sc backfill [--all]`; wire `PROTOCOL_VERSION` 3 | [0037](docs/adr/0037-partial-clone.md) |
| P28 | Security hardening: a max object size (default 256 MiB; `--max-object-size` / `SC_MAX_OBJECT_SIZE`, 2 MiB to 1 GiB) caps every untrusted transfer length; strict ref-name validation | [0039](docs/adr/0039-security-hardening-sweep.md) |
| P29 | `sc serve --http` access control: bearer tokens (`sc serve token add [--expires]`/`rotate`/`remove`/`list`; an expired token gets the same 401 as an unknown one), `--read-only`, fail-closed non-loopback bind gate | [0040](docs/adr/0040-sc-http-access-control.md) |
| P30 | Sealed agent-session transcripts: `sc transcript attach/show/list/sign/drop`, always-encrypted body, gc-coupled index; `--no-transcripts` on fetch/clone/push (client-side, per command); drop = grow-only tombstone (a bare `Transcript` object, no wire change), kept by gc, hides the body from fetch/reindex | [0038](docs/adr/0038-agent-session-transcripts.md) |
| P30 | Sealed agent-session transcripts: `sc transcript attach/show/list/sign/drop`, always-encrypted body, gc-coupled index; `--no-transcripts` on fetch/clone/push (client-side, per command); drop = grow-only tombstone (a bare `Transcript` object, no wire change), kept by gc, hides the body from fetch/reindex; retained signed tombstones ride a transfer when their snapshot is in it or was collected by the sender, only verifying signatures are indexed, and held-but-untrusted tombstones are re-judged from `.sc/transcript-pending` when the trusted-signer set changes (#230) | [0038](docs/adr/0038-agent-session-transcripts.md) |
| P31 | Listener resource limits: `--max-connections`, `--timeout`, `--max-pack-size` (both transports), capped read-only drain, accept backoff; later `--max-conn-bytes` per-connection write budget (#233) | [0041](docs/adr/0041-listener-resource-limits.md) |
| P32 | In-binary TLS: `sc+https://` via leaf crate `tlsio`; accept-new TOFU pinning (`sc tls list/forget/pin/path` manages the pin file); `SC_HTTPS_CA_FILE`/`SC_HTTPS_CA=system` validate against a CA instead; client connect/handshake bounded; public plaintext bind no longer justified by tokens alone | [0042](docs/adr/0042-in-binary-tls-sc-https.md), [0053](docs/adr/0053-tls-pin-management.md), [0054](docs/adr/0054-tls-ca-validation.md) |
| P33 | Randomized protected sealing (fresh DEK + nonce; `RANDOMIZED` perms bit); dual-read of pre-P33 convergent ciphertext; per-checkout keyed stat cache; `sc rewrap` upgrades convergent blobs at the tip | [0043](docs/adr/0043-randomized-protected-encryption.md) |
Expand Down Expand Up @@ -181,7 +181,9 @@ transport-adjacent. The rest, imperatively:
- **The P35 desktop app is a keyless, read-only top-level adapter.** Its
renderer has five typed repository commands, no general filesystem/shell
capability, and no DTO field capable of carrying protected ciphertext or
identity material.
identity material. Opening a repository can write the derived transcript-drop
indexes (`.sc/transcript-drops`, `.sc/transcript-pending`) when the trusted
signer set changed; no key material is involved (#230).
- **Partial clones refuse `sc export`, `sparse disable`, an empty `sparse set` (and, on a non-lazy clone, one outside the filter), `sc rewrap`,
`sc grant`/`revoke`, creating, switching to or leaving a private branch, a merge, cherry-pick or rebase whose conflict lies outside the view (a `.theirs` sidecar sits beside its conflicted path, so it is in view whenever that path is; ADR-0049) (no ref, state or working-tree change; on every clone, a conflict stop that needs sealing outside the view is refused the same way), `sc push` to a git remote, and
`sc fetch` from any native remote but the promisor origin.** `sc merge`, `sc cherry-pick` and `sc rebase` work, including conflicts inside the view (P37c). `sc ws` and `sc work` work too (ADR-0050): workspaces and agents see the clone's view only, and harvest lands through the same merge. `sc backfill --all`
Expand Down Expand Up @@ -306,7 +308,7 @@ Deferred work is tracked in **`ROADMAP.md` → Deferred** (single source of
truth — do not re-list items here); entries that name an issue mirror the open
GitHub issues. Standing entries: named/concurrent `sc ws` sessions (#215),
blob-size clone filters (#217), bucket compaction and gc (#216), the non-unix
`.sc/local-key` ACL (#214), transcript-drop delivery gaps (#230), and the open
`.sc/local-key` ACL (#214), and the open
sparse, private-branch and abort edge cases (#241, #246, #247, #248).

## Agent skills
Expand Down
13 changes: 7 additions & 6 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -480,8 +480,13 @@ across every phase.
grow-only drop tombstone (a bare `Transcript` object, so no wire change)
hides the body from `list`/`show`, travels with fetch/clone, blocks
resurrection in either arrival order, and is kept by gc while the dropped
body is swept at gc (after the grace window if loose). (ADR-0038 addendum;
delivery gaps tracked in #230.)
body is swept at gc (after the grace window if loose). (ADR-0038 addendum.)
Issue #230 closed the delivery gaps: a retained signed tombstone rides
a transfer when its snapshot is in it or was collected by the sender's gc (so
a stale peer is told), only verifying signatures enter the signature index,
and a remote that held a tombstone before trusting the signer re-judges it
from a capped `.sc/transcript-pending` index when the trusted-signer set
changes. Crowding by valid signatures from attacker keys is tracked in #257.

- **Phase 32 — In-binary TLS (`sc+https://`).** Closes the audit's High #1
(plaintext bearer tokens/traffic on `sc+http://`) and ADR-0036's "no TLS"
Expand Down Expand Up @@ -753,10 +758,6 @@ P28 review follow-ons, the port de-flake, the clippy allow-list) are recorded in
- **Named/multiple concurrent ws sessions (#215, P20).** P20 ships one unnamed
session per repo; named or concurrent sessions and workspace re-fork/refresh
from a newer tip need a design decision first.
- **Transcript drop delivery gaps (#230, ADR-0038 addendum).** A tombstone is
not sent after the dropper's gc removed its snapshot, and a tombstone a
push-only remote received before trusting the signer is not re-evaluated
later; both are documented limits, not regressions.
- **Sparse untracked-file guard edge cases.** #241: on a case-insensitive
filesystem the untracked-file guard can alias an old-view path with a newly
in-view one. #248: an untracked file at an out-of-view path that the other side adds wins
Expand Down
13 changes: 13 additions & 0 deletions crates/repo/src/layout.rs
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,19 @@ impl Layout {
pub fn transcript_drops_path(&self) -> PathBuf {
self.dot_sc.join("transcript-drops")
}
/// `.sc/transcript-pending` — tombstone object ids this repo holds, has a
/// signature for, and has not honored yet (untrusted signer, or snapshot
/// not held), one hex id per line, newest last, capped. Re-evaluated on
/// every open and ingest; gc roots them (ADR-0038, #230).
pub fn transcript_pending_path(&self) -> PathBuf {
self.dot_sc.join("transcript-pending")
}
/// `.sc/transcript-pending-trust` — fingerprint of the trusted-signer set
/// the pending tombstones were last judged under; a different set is the
/// trigger to judge them again.
pub fn transcript_pending_trust_path(&self) -> PathBuf {
self.dot_sc.join("transcript-pending-trust")
}
/// `.sc/tmp` — scratch space for transient files (P25 streaming pack
/// transfer: spilled sender/receiver pack bodies). Never durable state —
/// safe to delete at any time; callers that write here are responsible
Expand Down
5 changes: 4 additions & 1 deletion crates/repo/src/repo.rs
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,10 @@ impl Repo {

fn open_layout(layout: Layout, budget_bytes: usize) -> Result<Repo> {
let lock = RepoLock::acquire(&layout)?;
let store = Store::open_persistent(layout.objects_dir(), budget_bytes)?;
let mut store = Store::open_persistent(layout.objects_dir(), budget_bytes)?;
// Honor held tombstones once the trusted-signer set changed (#230). Best
// effort: a failure must not stop an open, and the next open retries.
let _ = crate::transcripts::reevaluate_if_trust_changed(&layout, &mut store);
Ok(Repo {
layout,
vfs: VfsRepo::new(store),
Expand Down
142 changes: 139 additions & 3 deletions crates/repo/src/signatures.rs
Original file line number Diff line number Diff line change
Expand Up @@ -145,13 +145,45 @@ pub(crate) fn index_incoming(
let mut count = 0;
for id in ids {
if let Object::Signature(sig) = store.get(id)? {
append_index(layout, sig.snapshot, *id)?;
count += 1;
// Only a signature that checks out for a target this store holds
// is indexed (and so forwarded and kept by gc): nothing forged or
// dangling can crowd out a real one (#230, ADR-0038).
if acceptable(store, &sig)? {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Bound verification work for incoming signatures

On any writable transport, a peer can submit a large pack of distinct invalid Signature objects targeting one held snapshot; this calls acceptable and performs an Ed25519 verification for every record with no batch-level work cap. Because the default accepted pack size is 16 GiB, a comparatively modest signature-only pack can monopolize a server handler with hundreds of thousands of serial cryptographic checks, and concurrent connections can exhaust all handlers. Apply a total signature-verification budget per ingest rather than verifying every attacker-supplied signature.

Useful? React with 👍 / 👎.

append_index(layout, sig.snapshot, *id)?;
count += 1;
}
}
}
Ok(count)
}

/// Whether `sig` may enter the index: its target is held, and the signature
/// verifies under the key it names in the domain its target kind uses. A
/// snapshot takes the snapshot domain; a transcript the transcript domain; a
/// drop tombstone only the domain-separated drop digest (a plain signature
/// over a tombstone id never authorizes a drop). Any other target kind is
/// refused. Trust is not judged here: that is the reader's call.
pub(crate) fn acceptable(store: &mut scl_core::Store, sig: &SignatureObj) -> Result<bool> {
if !store.contains(&sig.snapshot) {
return Ok(false);
}
Comment on lines +167 to +169

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reconsider signatures rejected before their targets arrive

When a valid signature reaches a push-only remote before its target, this early return leaves the signature object in the CAS but out of the index. A later normal push sends the target but skips the signature because has_object already reports it present, and indexing the target does not scan previously rejected signatures; the snapshot therefore remains Unsigned, or a signed tombstone remains ineffective, indefinitely. Either avoid retaining rejected dangling signatures, remember them for reevaluation, or reconsider them when their target arrives.

Useful? React with 👍 / 👎.

let target = sig.snapshot;
Ok(match store.get(&target)? {
Object::Snapshot(_) => {
scl_crypto::verify_snapshot_sig(&sig.signer, target.as_bytes(), &sig.sig)
}
Object::Transcript(t) if t.dropped_id().is_some() => scl_crypto::verify_transcript_sig(
&sig.signer,
&crate::transcripts::drop_digest(&target),
&sig.sig,
),
Object::Transcript(_) => {
scl_crypto::verify_transcript_sig(&sig.signer, target.as_bytes(), &sig.sig)
}
_ => false,
})
}

/// Full-store rebuild of the signature index (Task 3): scan every object the
/// store can resolve (loose + packed — `Store::all_ids`, not a reachability
/// walk, since a `SignatureObj` is a leaf no tree/snapshot references), keep
Expand All @@ -171,7 +203,7 @@ pub(crate) fn reindex(layout: &Layout, store: &mut scl_core::Store) -> Result<us
let mut entries: Vec<(ObjectId, ObjectId)> = Vec::new();
for id in store.all_ids()? {
if let Object::Signature(sig) = store.get(&id)? {
if store.contains(&sig.snapshot) {
if acceptable(store, &sig)? {
entries.push((sig.snapshot, id));
}
}
Expand All @@ -195,6 +227,26 @@ pub(crate) fn index_by_target(
Ok(out)
}

/// [`index_by_target`] restricted to `targets`, keeping at most `per_target`
/// signature ids each (index order). Only the wanted entries are retained, so
/// a large index costs a scan, not memory.
pub(crate) fn index_for_targets(
layout: &Layout,
targets: &BTreeSet<ObjectId>,
per_target: usize,
) -> Result<std::collections::HashMap<ObjectId, Vec<ObjectId>>> {
let mut out: std::collections::HashMap<ObjectId, Vec<ObjectId>> = Default::default();
for (target, sig_id) in read_index(layout)? {
if targets.contains(&target) {
let v = out.entry(target).or_default();
if v.len() < per_target {
v.push(sig_id);
}
}
}
Ok(out)
}

/// Sender-side seam (Task 3): the indexed signature object ids covering any
/// of `snapshots`, deduped and sorted — the set a push/export needs to
/// carry alongside the snapshots themselves so the receiving repo's index
Expand Down Expand Up @@ -671,4 +723,88 @@ mod tests {
drop(repo);
std::fs::remove_dir_all(&root).unwrap();
}

/// Write `objs` as a pack under `repo` and ingest it like a transfer.
fn ingest(repo: &Repo, objs: &[Object]) {
let enc: Vec<(ObjectId, Vec<u8>)> = objs.iter().map(|o| (o.id(), o.encode())).collect();
let (pack, _idx) = scl_core::pack::build_pack(&enc).unwrap();
let path = repo.layout().dot_sc.join("incoming.pack");
std::fs::write(&path, pack).unwrap();
let arc = repo.store_arc();
let mut store = arc.lock().unwrap();
crate::transport::ingest_pack_file(repo.layout(), &mut store, &path).unwrap();
std::fs::remove_file(&path).unwrap();
}

#[test]
fn a_signature_sorting_below_its_snapshot_in_the_pack_is_still_indexed() {
// The pack is ordered by id, so the signature can come first; every
// object is written before indexing, so its target is held anyway.
let a_root = tmp_root("order-a");
let b_root = tmp_root("order-b");
let a = Repo::init(&a_root).unwrap();
let (_s, identity) = scl_crypto::generate_identity_v2();
let key = identity.signing.as_ref().unwrap();
let (snap, snap_obj, sig_obj) = loop {
std::fs::write(a_root.join("f"), format!("{:?}", std::time::Instant::now())).unwrap();
let snap = a.commit("t", "c").unwrap();
let sig_obj = Object::Signature(SignatureObj {
snapshot: snap,
signer: key.public().to_bytes(),
sig: scl_crypto::sign_snapshot_id(key, snap.as_bytes()),
});
if sig_obj.id() < snap {
let snap_obj = a.store_arc().lock().unwrap().get(&snap).unwrap();
break (snap, snap_obj, sig_obj);
}
};
let b = Repo::init(&b_root).unwrap();
let mut objs = [snap_obj, sig_obj.clone()];
objs.sort_by_key(Object::id);
assert_eq!(objs[0].id(), sig_obj.id(), "the signature leads the pack");
ingest(&b, &objs);
let mut trust = HashMap::new();
trust.insert(key.public().to_bytes(), "alice".to_string());
assert_eq!(
b.sig_status(&snap, &trust).unwrap(),
SigStatus::Trusted("alice".to_string())
);
drop((a, b));
std::fs::remove_dir_all(&a_root).unwrap();
std::fs::remove_dir_all(&b_root).unwrap();
assert!(!a_root.exists() && !b_root.exists());
}

#[test]
fn a_forged_snapshot_signature_over_the_wire_leaves_the_snapshot_unsigned() {
// Pins the ingest filter: a received forgery is not indexed, so it
// cannot turn a snapshot Invalid (ADR-0038, #230).
let a_root = tmp_root("forge-a");
let b_root = tmp_root("forge-b");
let a = Repo::init(&a_root).unwrap();
std::fs::write(a_root.join("f"), b"x").unwrap();
let snap = a.commit("t", "c").unwrap();
let snap_obj = a.store_arc().lock().unwrap().get(&snap).unwrap();
let (_s, identity) = scl_crypto::generate_identity_v2();
let signer = identity.signing.as_ref().unwrap().public().to_bytes();
let forged = Object::Signature(SignatureObj {
snapshot: snap,
signer,
sig: [9u8; 64],
});
let b = Repo::init(&b_root).unwrap();
ingest(&b, &[snap_obj, forged.clone()]);
let mut trust = HashMap::new();
trust.insert(signer, "alice".to_string());
assert_eq!(b.sig_status(&snap, &trust).unwrap(), SigStatus::Unsigned);
assert!(b.signatures_for(&snap).unwrap().is_empty());
assert!(
b.store_arc().lock().unwrap().contains(&forged.id()),
"the object is stored, only the index refuses it"
);
drop((a, b));
std::fs::remove_dir_all(&a_root).unwrap();
std::fs::remove_dir_all(&b_root).unwrap();
assert!(!a_root.exists() && !b_root.exists());
}
}
Loading
Loading