From e0806415b9bf8aeb09c4e047c3e5b73f55ec5b27 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 11:07:56 +0300 Subject: [PATCH 01/14] docs(spec): P31 listener resource limits design (#38) Brainstormed design for the four serve bounds: ServeLimits CLI struct, atomic-slot connection cap with RAII guard + 503, persistent session read/write timeouts, WirePolicy with counted spool abort (EC_TOO_LARGE), 8 MiB RO drain cap, Go-style accept backoff. Spool cap applies to both --http and --stdio (brainstorm decision). Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- .../2026-07-10-p31-listener-limits-design.md | 173 ++++++++++++++++++ 1 file changed, 173 insertions(+) create mode 100644 docs/superpowers/specs/2026-07-10-p31-listener-limits-design.md diff --git a/docs/superpowers/specs/2026-07-10-p31-listener-limits-design.md b/docs/superpowers/specs/2026-07-10-p31-listener-limits-design.md new file mode 100644 index 00000000..24324f50 --- /dev/null +++ b/docs/superpowers/specs/2026-07-10-p31-listener-limits-design.md @@ -0,0 +1,173 @@ +# P31: HTTP listener resource limits — design + +- **Date:** 2026-07-10 +- **Spec issue:** [#38 — P31: HTTP listener resource limits (sc serve --http)](https://github.com/git-agentic/src-control/issues/38) +- **Decision trail:** wayfinder map [#24](https://github.com/git-agentic/src-control/issues/24) → + decision [#28](https://github.com/git-agentic/src-control/issues/28), research + [#27](https://github.com/git-agentic/src-control/issues/27) + (`docs/research/bounded-server-patterns.md`) +- **Closes:** audit High #2 — ADR-0036's three named operational boundaries + (unbounded thread-per-connection, no idle-transfer watchdog, no accept-loop + backoff) **plus** the unbounded aggregate pack spool the research surfaced + (reachable by read-only clients via the drain-for-sync arm, `wire.rs`). + +Zero new dependencies. `PROTOCOL_VERSION` stays 3. + +## Goal + +`sc serve` survives hostile or broken clients with bounded threads, bounded +disk, and a live accept loop — with defaults that protect out of the box and +per-invocation flags for operators who need different numbers. + +## CLI surface + +`sc serve` gains three flags: + +| Flag | Default | 0 means | Scope | +|---|---|---|---| +| `--max-connections ` | 32 | unlimited | `--http` only (refused with `--stdio`) | +| `--timeout ` | 300 | disabled | `--http` only (refused with `--stdio`) | +| `--max-pack-size ` | 16 GiB | unlimited | **both** `--http` and `--stdio` | + +- `--max-pack-size` values below `MAX_OBJECT_SIZE` (256 MiB) are rejected at + parse time with a clear error (a cap that can't fit one maximal object is a + misconfiguration, not a policy). +- `--max-connections`/`--timeout` are socket/accept-loop mechanics, so they are + meaningless for `--stdio` (ssh owns that transport's sockets) and refused + there rather than silently ignored. +- `--max-pack-size` applies to both transports because the spool mechanism is + the shared wire layer and disk exhaustion is transport-agnostic (decision: + brainstorm 2026-07-10; same default everywhere). + +The flags build a `ServeLimits` struct in the CLI; no config file, no env vars +(both rejected on decision ticket #28 / brainstorm). + +## Listener changes (`crates/repo/src/http_transport.rs`) + +**Connection limit — atomic slots + RAII guard.** `serve_http_listener` holds +an `Arc`. Per accepted socket, before spawning: increment; if the +new count exceeds the limit, write HTTP status **503**, close, decrement — no +wire handshake, no thread. Otherwise the connection thread owns a `SlotGuard` +whose `Drop` decrements, so panics and every error path free the slot +(the `TempPackGuard` discipline). No queuing: for long-lived wire sessions, +fail-fast-with-status beats invisible waits (decision #28). + +**Idle/progress deadline.** `handle_http_connection` currently clears the 30s +opening read timeout before the wire handoff (`set_read_timeout(None)`); it +instead sets the configured session timeout on **both** read and write sides +(`set_read_timeout` + `set_write_timeout`) — a write timeout covers a client +that stops *reading* mid-`GetPack`, which today parks the thread in a blocking +`write`. Under P25 chunking (≤ `SC_PACK_CHUNK` frames) a per-syscall timeout is +progress-based: slow-but-flowing transfers never trip it; only true zero-byte +stalls do. On timeout the error is **connection-fatal** (frame desync makes +recovery impossible): log to stderr, thread exits, guard frees the slot, +`TempPackGuard` removes any partial spool. Both `io::ErrorKind::WouldBlock` +(Unix) and `TimedOut` (Windows) are treated as timeout. The opening keeps its +existing stricter 30s. + +**Accept-loop backoff.** On `listener.incoming()` errors: sleep 5ms, doubling +per consecutive error to a 1s cap, reset on the next successful accept — Go's +`net/http` shape verbatim, hardcoded, no knob, no reserved-fd trick +(decision #28). This turns fd exhaustion from a busy-loop into a paced retry. + +## Wire-layer changes (`crates/repo/src/wire.rs`) + +**`WirePolicy` replaces the bare bool.** `serve_with_policy(root, r, w, +read_only: bool)` becomes `serve_with_policy(root, r, w, policy: WirePolicy)` +where + +```rust +pub struct WirePolicy { + pub read_only: bool, + pub max_pack_size: u64, // 0 = unlimited + pub ro_drain_cap: u64, // constant: 8 MiB +} +``` + +The existing `serve` wrapper maps to `WirePolicy { read_only: false, +max_pack_size: , ro_drain_cap: RO_DRAIN_CAP }`; `--stdio` and `--http` +both construct it from `ServeLimits`. In-crate callers update; this is a +pre-1.0 in-workspace signature change, not a wire change. + +**Spool cap — counted abort mid-stream.** `read_pack_stream` and +`spill_pack_stream` gain a byte budget. The running total is checked as chunk +frames arrive (git's `receive.maxInputSize` shape: abort *while counting*, +never after materializing); exceeding it raises a new typed +`Error::PackTooLarge { limit }`, the partial spool is removed by the guard, +and the server replies with a new wire error code **`EC_TOO_LARGE = 6`**. +Old clients degrade gracefully — unknown codes already fall back to +`Error::Remote(msg)` — so no protocol version bump. The cap composes *above* +ADR-0039's per-frame/per-record/decompression caps; it never replaces them. + +**Read-only drain cap.** The RO `PutPack` arm's drain-for-sync +(`spill_pack_stream` then drop) is capped at **8 MiB** (`RO_DRAIN_CAP`): a pack +that ends within the cap gets today's clean typed `EC_READONLY` reply; a larger +one gets a best-effort `EC_READONLY` write and the connection is closed +(sync is unrecoverable past the cap by design — decision #28: honest +misconfigurations keep the clear error, hostile bulk spools almost nothing). + +## Client-side mapping (`http_transport.rs`, `wire.rs`) + +- Opening status `503` → a typed, clearly retryable error + ("server busy (connection limit reached) — retry later"). +- `EC_TOO_LARGE` → `Error::PackTooLarge` with the server's limit in the + message, so `sc push` prints an actionable failure. + +## Error handling summary + +| Event | Server behavior | Client sees | +|---|---|---| +| Accept while full | 503 + close, no handshake | "server busy, retry" | +| Zero-byte stall > timeout | log + drop connection, spool cleaned | connection error | +| Pack exceeds `--max-pack-size` | abort mid-stream, `EC_TOO_LARGE`, spool cleaned, connection continues | "pack exceeds server limit (N)" | +| RO push ≤ 8 MiB | drain, `EC_READONLY` (unchanged) | "read-only" | +| RO push > 8 MiB | best-effort `EC_READONLY`, close | read-only error or connection error | +| Accept error (EMFILE etc.) | 5ms→1s backoff, loop stays alive | connect delay/refusal | + +## Testing + +Pinned regression tests per bound: + +1. Slot exhaustion: fill `--max-connections 1` with a held session; second + connect gets the busy error; after the first closes, a third succeeds + (slot actually freed). +2. Guard leak check: a connection whose handler errors (bad opening) frees its + slot. +3. Stall reaping: a client that connects, handshakes, then goes silent is + dropped after the (test-shortened) timeout; `.sc/tmp` empty afterwards. +4. Write-side stall: a client that stops reading mid-`GetPack` is dropped. +5. Spool cap: a pack exceeding a small `--max-pack-size` fails mid-stream with + the typed error, `.sc/tmp` is empty, and the same connection can issue a + subsequent successful request. +6. Config floor: `--max-pack-size` below `MAX_OBJECT_SIZE` refused at startup. +7. RO drain: small RO push still gets `EC_READONLY`; oversized RO push gets the + connection closed, ≤ 8 MiB spooled (assert via temp-dir observation), zero + residue after. +8. Backoff: consecutive accept-error handling doesn't busy-loop (shape test via + injected error source or timing bound). +9. `--stdio` spool cap: same as (5) over the wire-harness stdio path. + +`demo/run_limits_demo.sh`: real loopback TCP; proves busy-status at the cap, +stall reaping, spool-cap rejection with zero `.sc/tmp` residue on both ends — +run twice (the repo's demo discipline). + +## Documentation + +- **New ADR** (next number): the four bounds; the two recorded + divergences-from-prior-art (timeout on-by-default vs git-daemon's off; + finite 16 GiB spool default vs git's unlimited `receive.maxInputSize`); + the 8 MiB drain posture; the both-transports scope of the spool cap. +- **`docs/THREAT-MODEL.md`:** mark ADR-0036's three boundaries closed; name and + close the aggregate-spool gap; carry the two map re-affirmation wordings — + env-var secret exposure (#31: same-user adversary defeats the identity key; + fd/stdin stays an additive deferred mode) and signature replay (#32: + ancestry is already id-bound; the residue is ref binding only — + "replayable to a ref tip within its own history"). +- **CLAUDE.md:** the three new flags on the serve command block; ADR-0036's + "accepted design consequences" paragraph updated to point here. + +## Non-goals + +Pre-auth connection sub-caps (sshd MaxStartups shape), reserved-fd shedding, +client-side spool caps, any config-file surface — recorded follow-ons, not +this phase. From a5951437d0754d0af7ec4ac7c942d46833320d07 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 11:15:12 +0300 Subject: [PATCH 02/14] docs(plan): P31 listener limits implementation plan (#38) Eight TDD tasks: typed errors + EC_TOO_LARGE, read_pack_stream byte budget, WirePolicy + capped spool/RO drain, ServeLimits + slots/503 + session timeouts + accept backoff, CLI flags, demo, ADR-0041 + doc sweep, final verification. Also corrects the spec: a mid-stream spool abort is connection-fatal (stream desync), not connection-reusable. Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- .../plans/2026-07-10-p31-listener-limits.md | 789 ++++++++++++++++++ .../2026-07-10-p31-listener-limits-design.md | 14 +- 2 files changed, 798 insertions(+), 5 deletions(-) create mode 100644 docs/superpowers/plans/2026-07-10-p31-listener-limits.md diff --git a/docs/superpowers/plans/2026-07-10-p31-listener-limits.md b/docs/superpowers/plans/2026-07-10-p31-listener-limits.md new file mode 100644 index 00000000..b312b7da --- /dev/null +++ b/docs/superpowers/plans/2026-07-10-p31-listener-limits.md @@ -0,0 +1,789 @@ +# P31: HTTP Listener Resource Limits Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Bound `sc serve` against hostile/broken clients: connection cap with busy status, session read+write timeouts, aggregate pack-spool cap (both transports), read-only drain cap, accept-loop backoff. + +**Architecture:** A `ServeLimits` struct (CLI-built) threads into the HTTP listener; the wire layer's bare `read_only: bool` grows into `WirePolicy { read_only, max_pack_size, ro_drain_cap }`. Connection slots are an `Arc` with an RAII guard; the spool cap is a counted mid-stream abort in `read_pack_stream` surfacing as new wire code `EC_TOO_LARGE = 6` (old clients degrade to message text — no version bump). + +**Tech Stack:** Rust (std only — no new dependencies). Spec: `docs/superpowers/specs/2026-07-10-p31-listener-limits-design.md`. Issue: #38. + +## Global Constraints + +- **Zero new dependencies.** `PROTOCOL_VERSION` stays **3**. +- Defaults, copied from the spec: `--max-connections` **32**, `--timeout` **300** s, `--max-pack-size` **16 GiB** (`17_179_869_184`); `0` = unlimited/disabled for all three. RO drain cap: **8 MiB** constant, no knob. +- `--max-pack-size` values `> 0` but `< MAX_OBJECT_SIZE` (256 MiB, `scl_core::MAX_OBJECT_SIZE`) are rejected. +- `--max-connections`/`--timeout` are `--http`-only (refused with `--stdio`); `--max-pack-size` applies to both transports. +- Mid-stream spool abort is **connection-fatal** (stream desync): best-effort typed error, then close. +- Repo conventions: `RUSTFLAGS="-D warnings"` (CI), per-crate `thiserror`, doc comments state intent, tests clean up temp dirs and assert zero residue, `cargo fmt` before every commit. +- Commit messages end with: `Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB` + +## File Structure + +- `crates/repo/src/error.rs` — two new variants: `PackTooLarge(String)`, `ServerBusy`. +- `crates/repo/src/wire.rs` — `EC_TOO_LARGE`, `WirePolicy`, `DEFAULT_MAX_PACK_SIZE`, `RO_DRAIN_CAP`, `validate_max_pack_size`, budgeted `read_pack_stream`/`spill_pack_stream`, both `PutPack` arms. +- `crates/repo/src/http_transport.rs` — `ServeLimits`, `SlotGuard`, `AcceptBackoff`, 503 status, session timeouts, client `ServerBusy` mapping. +- `crates/cli/src/main.rs` — three flags + validation + plumbing (`run_serve`). +- `demo/run_limits_demo.sh` — end-to-end proof. +- `docs/adr/0041-listener-resource-limits.md`, `docs/THREAT-MODEL.md`, `CLAUDE.md` — docs. + +--- + +### Task 1: Typed errors + wire code round trip + +**Files:** +- Modify: `crates/repo/src/error.rs` (append inside `pub enum Error`, near `ReadOnly`) +- Modify: `crates/repo/src/wire.rs` (EC constants block ~line 75; `err_to_wire` ~line 485; `wire_to_err` ~line 499; tests at bottom) + +**Interfaces:** +- Produces: `Error::PackTooLarge(String)` (payload = human limit text, e.g. `"16 GiB (17179869184 bytes)"`), `Error::ServerBusy`, `pub(crate) const EC_TOO_LARGE: u8 = 6`. Round trip: `err_to_wire(PackTooLarge) == (EC_TOO_LARGE, msg)`; `wire_to_err(EC_TOO_LARGE, msg) == PackTooLarge(msg)`. + +- [ ] **Step 1: Write the failing test** (in `wire.rs` `mod tests`) + +```rust +#[test] +fn pack_too_large_round_trips_through_wire_codes() { + let e = Error::PackTooLarge("16 GiB (17179869184 bytes)".to_string()); + let (code, msg) = err_to_wire(&e); + assert_eq!(code, EC_TOO_LARGE); + match wire_to_err(code, msg) { + Error::PackTooLarge(m) => assert!(m.contains("17179869184")), + other => panic!("expected PackTooLarge, got {other:?}"), + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cargo test -p scl-repo pack_too_large_round_trips -- --nocapture` +Expected: compile error — `PackTooLarge` not defined. + +- [ ] **Step 3: Implement** + +In `error.rs`, after the `ReadOnly` variant: + +```rust +/// The server aborted an incoming pack mid-stream because it exceeded the +/// operator's `--max-pack-size` cap (P31). Payload is the server's own +/// human-readable limit text, carried verbatim across the wire. +#[error("pack exceeds the server's --max-pack-size limit: {0}")] +PackTooLarge(String), +/// The server refused the connection at accept time because +/// `--max-connections` was reached (P31). Retryable. +#[error("server busy (connection limit reached); retry later")] +ServerBusy, +``` + +In `wire.rs`, EC block: + +```rust +const EC_TOO_LARGE: u8 = 6; +``` + +`err_to_wire`: add arm `Error::PackTooLarge(_) => EC_TOO_LARGE,` (message flows via the existing `e.to_string()` path). +`wire_to_err`: add arm `EC_TOO_LARGE => Error::PackTooLarge(msg),`. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cargo test -p scl-repo pack_too_large_round_trips` +Expected: PASS + +- [ ] **Step 5: Commit** + +```bash +cargo fmt --all && git add -A crates/repo && git commit -m "feat(wire): PackTooLarge/ServerBusy errors + EC_TOO_LARGE wire code (P31)" +``` + +--- + +### Task 2: Byte budget in `read_pack_stream` + +**Files:** +- Modify: `crates/repo/src/wire.rs:430` (`read_pack_stream`), `crates/repo/src/wire.rs:796` (`spill_pack_stream` call site), test call sites `wire.rs:1155,1169,1181,1302,1313` +- Modify: `crates/repo/src/stdio_transport.rs:120,566` (client + harness call sites) + +**Interfaces:** +- Consumes: `Error::PackTooLarge(String)` (Task 1). +- Produces: `pub fn read_pack_stream(r: &mut impl Read, sink: &mut (impl Write + ?Sized), max_bytes: u64) -> Result` — `max_bytes == 0` means unlimited; exceeding aborts with `Error::PackTooLarge` *before* writing the over-budget chunk to `sink`. + +- [ ] **Step 1: Write the failing tests** (in `wire.rs` `mod tests`; `write_pack_stream(w, r, chunk_size)` already exists) + +```rust +#[test] +fn pack_stream_over_budget_aborts_typed() { + let payload = vec![7u8; 4096]; + let mut buf = Vec::new(); + write_pack_stream(&mut buf, &mut std::io::Cursor::new(&payload), 1024).unwrap(); + let mut sink = Vec::new(); + let err = read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink, 2048).unwrap_err(); + assert!(matches!(err, Error::PackTooLarge(_)), "got {err:?}"); + assert!(sink.len() <= 2048, "sink got over-budget bytes: {}", sink.len()); +} + +#[test] +fn pack_stream_under_budget_passes() { + let payload = vec![7u8; 4096]; + let mut buf = Vec::new(); + write_pack_stream(&mut buf, &mut std::io::Cursor::new(&payload), 1024).unwrap(); + let mut sink = Vec::new(); + assert_eq!( + read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink, 4096).unwrap(), + 4096 + ); +} +``` + +- [ ] **Step 2: Run to verify failure** + +Run: `cargo test -p scl-repo pack_stream_over_budget` +Expected: compile error — wrong arity. + +- [ ] **Step 3: Implement** + +Change the signature and add the check inside the chunk arm, before `sink.write_all`: + +```rust +pub fn read_pack_stream( + r: &mut impl Read, + sink: &mut (impl Write + ?Sized), + max_bytes: u64, +) -> Result { + let mut total: u64 = 0; + loop { + let frame = + read_frame_opt(r)?.ok_or_else(|| Error::Protocol("EOF before ST_PACK_END".into()))?; + match frame.split_first() { + Some((&ST_PACK_CHUNK, rest)) => { + let next = total + rest.len() as u64; + if max_bytes != 0 && next > max_bytes { + return Err(Error::PackTooLarge(format!("{max_bytes} bytes"))); + } + sink.write_all(rest)?; + total = next; + } + Some((&ST_PACK_END, [])) => return Ok(total), + _ => return Err(Error::Protocol("unexpected frame in pack stream".into())), + } + } +} +``` + +Update every existing call site to pass `0` (unlimited) for now: `wire.rs:796` (`spill_pack_stream` — Task 3 threads the real cap), `stdio_transport.rs:120` (client destream — client-side caps are a spec non-goal) and `:566` (test harness), and the five `wire.rs` tests. Update the doc comment: budget semantics + "0 = unlimited". + +- [ ] **Step 4: Run to verify pass** + +Run: `cargo test -p scl-repo pack_stream` +Expected: new tests PASS, existing pack-stream tests PASS. + +- [ ] **Step 5: Commit** + +```bash +cargo fmt --all && git add -A crates/repo && git commit -m "feat(wire): byte budget with typed mid-stream abort in read_pack_stream (P31)" +``` + +--- + +### Task 3: `WirePolicy` + capped spool in both `PutPack` arms + +**Files:** +- Modify: `crates/repo/src/wire.rs` — `serve_with_policy` (~line 604), `serve` wrapper (~line 780), `spill_pack_stream` (~line 790), RO drain arm (~line 669), normal `PutPack` arm (~line 724), the read-only test at ~line 1200 +- Modify: `crates/repo/src/http_transport.rs:602` (call site; real limits threaded in Task 5 — pass `WirePolicy { read_only, ..Default::default() }` for now) + +**Interfaces:** +- Consumes: budgeted `read_pack_stream` (Task 2), `EC_TOO_LARGE` (Task 1). +- Produces: + +```rust +pub const DEFAULT_MAX_PACK_SIZE: u64 = 16 * 1024 * 1024 * 1024; // 16 GiB +pub const RO_DRAIN_CAP: u64 = 8 * 1024 * 1024; // 8 MiB + +#[derive(Debug, Clone, Copy)] +pub struct WirePolicy { + pub read_only: bool, + pub max_pack_size: u64, // 0 = unlimited + pub ro_drain_cap: u64, +} +impl Default for WirePolicy { + fn default() -> Self { + Self { read_only: false, max_pack_size: DEFAULT_MAX_PACK_SIZE, ro_drain_cap: RO_DRAIN_CAP } + } +} + +pub fn validate_max_pack_size(max: u64) -> Result<()>; // 0 ok; else must be >= MAX_OBJECT_SIZE +pub fn serve_with_policy(root: &Path, r: &mut impl Read, w: &mut impl Write, policy: WirePolicy) -> Result<()>; +``` + +- [ ] **Step 1: Write the failing tests** (model on the existing `serve_with_policy` read-only test at ~line 1200; `some_id`, `tmp_repo`, `write_frame`, `parse_response` helpers exist) + +```rust +/// A PutPack whose chunk stream exceeds policy.max_pack_size gets a +/// best-effort EC_TOO_LARGE reply and the connection closes (desync). +#[test] +fn putpack_over_cap_replies_too_large_and_closes() { + let root = tmp_repo("cap"); + let mut input = Vec::new(); + write_frame(&mut input, &Request::Hello { version: PROTOCOL_VERSION }.encode()).unwrap(); + write_frame(&mut input, &Request::PutPack.encode()).unwrap(); + let payload = vec![9u8; 8192]; + write_pack_stream(&mut input, &mut std::io::Cursor::new(&payload), 1024).unwrap(); + // A trailing request that must never be answered (connection closed): + write_frame(&mut input, &Request::HeadBranch.encode()).unwrap(); + + let mut reader = std::io::Cursor::new(input); + let mut output = Vec::new(); + let policy = WirePolicy { read_only: false, max_pack_size: 4096, ro_drain_cap: RO_DRAIN_CAP }; + serve_with_policy(&root, &mut reader, &mut output, policy).unwrap(); + + let mut frames = Vec::new(); + let mut r = std::io::Cursor::new(output); + while let Some(f) = read_frame_opt(&mut r).unwrap() { + frames.push(parse_response(f)); + } + assert_eq!(frames.len(), 2, "hello-ok + too-large error only, got {frames:?}"); + assert!(frames[0].is_ok()); + assert!(matches!(frames[1].as_ref().unwrap_err(), Error::PackTooLarge(_))); + // Zero spool residue: + let tmp = root.join(".sc").join("tmp"); + assert!(!tmp.exists() || std::fs::read_dir(&tmp).unwrap().next().is_none()); + let _ = std::fs::remove_dir_all(&root); +} + +#[test] +fn validate_max_pack_size_floor() { + assert!(validate_max_pack_size(0).is_ok()); + assert!(validate_max_pack_size(scl_core::MAX_OBJECT_SIZE as u64).is_ok()); + assert!(validate_max_pack_size(1024).is_err()); +} +``` + +- [ ] **Step 2: Run to verify failure** + +Run: `cargo test -p scl-repo putpack_over_cap` +Expected: compile error — `WirePolicy` not defined. + +- [ ] **Step 3: Implement** + +1. Add the constants, `WirePolicy`, and `validate_max_pack_size` (error: `Error::InvalidArgument(format!("--max-pack-size {max} is below MAX_OBJECT_SIZE ({}); a cap that cannot fit one object is a misconfiguration", scl_core::MAX_OBJECT_SIZE))`). +2. `serve_with_policy` takes `policy: WirePolicy`; replace internal `read_only` reads with `policy.read_only`. `serve` wrapper: `serve_with_policy(root, r, w, WirePolicy::default())`. +3. `spill_pack_stream(r, layout, max_bytes: u64)` — pass `max_bytes` through to `read_pack_stream`. +4. Normal `PutPack` arm: `spill_pack_stream(r, transport.layout(), policy.max_pack_size)`; on `Err(e @ Error::PackTooLarge(_))` → `let (code, msg) = err_to_wire(&e); let _ = write_err(w, code, &msg); return Ok(());` (best-effort reply, then close — desync). Other errors keep the current reply-and-continue behavior. +5. RO drain arm: `spill_pack_stream(r, transport.layout(), policy.ro_drain_cap)`; on `Ok(guard)` → drop + `EC_READONLY` reply + `continue` (unchanged); on `Err(Error::PackTooLarge(_))` → best-effort `write_err(w, EC_READONLY-mapped ReadOnly error)` then `return Ok(())` (close); other errors propagate as today. +6. Update `http_transport.rs:602`: `crate::wire::serve_with_policy(root, &mut reader, &mut stream, crate::wire::WirePolicy { read_only, ..Default::default() })`. +7. Update the existing read-only test at ~line 1200 to construct a `WirePolicy`. + +- [ ] **Step 4: Run to verify pass** + +Run: `cargo test -p scl-repo --lib wire` +Expected: new tests PASS; all existing wire tests (including RO drain + streaming) PASS. + +- [ ] **Step 5: Write the RO-drain-cap test** + +```rust +/// An oversized push on a read-only connection drains at most ro_drain_cap +/// bytes, then the connection closes; zero spool residue. +#[test] +fn readonly_oversized_push_is_dropped_after_drain_cap() { + let root = tmp_repo("rocap"); + let mut input = Vec::new(); + write_frame(&mut input, &Request::Hello { version: PROTOCOL_VERSION }.encode()).unwrap(); + write_frame(&mut input, &Request::PutPack.encode()).unwrap(); + let payload = vec![9u8; 8192]; + write_pack_stream(&mut input, &mut std::io::Cursor::new(&payload), 1024).unwrap(); + write_frame(&mut input, &Request::HeadBranch.encode()).unwrap(); // must never be answered + + let mut reader = std::io::Cursor::new(input); + let mut output = Vec::new(); + let policy = WirePolicy { read_only: true, max_pack_size: 0, ro_drain_cap: 4096 }; + serve_with_policy(&root, &mut reader, &mut output, policy).unwrap(); + + let mut frames = Vec::new(); + let mut r = std::io::Cursor::new(output); + while let Some(f) = read_frame_opt(&mut r).unwrap() { + frames.push(parse_response(f)); + } + assert_eq!(frames.len(), 2); + assert!(matches!(frames[1].as_ref().unwrap_err(), Error::ReadOnly | Error::Remote(_))); + let tmp = root.join(".sc").join("tmp"); + assert!(!tmp.exists() || std::fs::read_dir(&tmp).unwrap().next().is_none()); + let _ = std::fs::remove_dir_all(&root); +} +``` + +- [ ] **Step 6: Run to verify pass** — `cargo test -p scl-repo readonly_oversized` +Expected: PASS (implementation from Step 3 item 5 already covers it; if it fails, fix the RO arm, not the test). + +- [ ] **Step 7: Full-crate check + commit** + +Run: `cargo test -p scl-repo && cargo clippy -p scl-repo --all-targets -- -D warnings` + +```bash +cargo fmt --all && git add -A crates/repo && git commit -m "feat(wire): WirePolicy with spool cap + capped read-only drain (P31)" +``` + +--- + +### Task 4: `ServeLimits`, connection slots + 503, session timeouts, accept backoff + +**Files:** +- Modify: `crates/repo/src/http_transport.rs` — `write_status` (~line 141), client status match (~line 315), `serve_http` (~line 449), `serve_http_listener` (~line 492), `handle_http_connection` (~lines 524–602), new types near `OPENING_READ_TIMEOUT`; tests in the same file's `mod tests` + +**Interfaces:** +- Consumes: `WirePolicy` (Task 3), `Error::ServerBusy` (Task 1). +- Produces: + +```rust +#[derive(Debug, Clone, Copy)] +pub struct ServeLimits { + pub max_connections: u32, // 0 = unlimited + pub timeout_secs: u64, // 0 = disabled + pub max_pack_size: u64, // 0 = unlimited +} +impl Default for ServeLimits { + fn default() -> Self { + Self { max_connections: 32, timeout_secs: 300, max_pack_size: crate::wire::DEFAULT_MAX_PACK_SIZE } + } +} + +pub fn serve_http(addr: &str, root: &Path, read_only: bool, allow_public: bool, limits: ServeLimits) -> Result<()>; +pub fn serve_http_listener(listener: TcpListener, root: &Path, read_only: bool, mandatory_auth: bool, limits: ServeLimits) -> Result<()>; +struct AcceptBackoff; // fn on_error(&mut self) -> Duration (5ms, ×2, cap 1s); fn on_success(&mut self) +``` + +- [ ] **Step 1: Write the failing unit tests** (pure parts first) + +```rust +#[test] +fn accept_backoff_doubles_and_resets() { + let mut b = AcceptBackoff::new(); + assert_eq!(b.on_error(), Duration::from_millis(5)); + assert_eq!(b.on_error(), Duration::from_millis(10)); + assert_eq!(b.on_error(), Duration::from_millis(20)); + for _ in 0..20 { b.on_error(); } + assert_eq!(b.on_error(), Duration::from_secs(1)); // capped + b.on_success(); + assert_eq!(b.on_error(), Duration::from_millis(5)); // reset +} + +#[test] +fn status_503_round_trips() { + let mut buf = Vec::new(); + write_status(&mut buf, 503).unwrap(); + assert_eq!(read_status(&mut &buf[..]).unwrap(), 503); +} +``` + +- [ ] **Step 2: Run to verify failure** — `cargo test -p scl-repo accept_backoff` → compile error. + +- [ ] **Step 3: Implement the pure parts** + +```rust +/// Exponential accept-error backoff (P31): Go net/http's shape — 5ms +/// doubling to a 1s cap, reset on the next successful accept. Turns fd +/// exhaustion (EMFILE) from a busy-loop into a paced retry. Hardcoded: no +/// operator tuning story exists (Go ships it knobless too). +struct AcceptBackoff { + cur: Duration, +} +impl AcceptBackoff { + const START: Duration = Duration::from_millis(5); + const CAP: Duration = Duration::from_secs(1); + fn new() -> Self { + Self { cur: Self::START } + } + fn on_error(&mut self) -> Duration { + let d = self.cur; + self.cur = (self.cur * 2).min(Self::CAP); + d + } + fn on_success(&mut self) { + self.cur = Self::START; + } +} +``` + +`write_status`: add `503 => "Service Unavailable",`. +Client status match (~line 315): add before `other`: + +```rust +503 => return Err(Error::ServerBusy), +``` + +- [ ] **Step 4: Run** — `cargo test -p scl-repo accept_backoff status_503` → PASS. Commit: + +```bash +cargo fmt --all && git add -A crates/repo && git commit -m "feat(http): AcceptBackoff + 503 busy status plumbing (P31)" +``` + +- [ ] **Step 5: Write the failing socket tests** (pattern: bind `127.0.0.1:0`, spawn `serve_http_listener` on a thread, connect with raw `TcpStream` + `write_client_opening`/`read_status`; repo root = temp dir containing an empty `.sc/` subdir) + +```rust +fn tmp_served_root(tag: &str) -> std::path::PathBuf { + let root = std::env::temp_dir().join(format!("scl-http-{tag}-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&root); + std::fs::create_dir_all(root.join(".sc")).unwrap(); + root +} + +fn spawn_listener(root: &std::path::Path, limits: ServeLimits) -> std::net::SocketAddr { + let listener = TcpListener::bind("127.0.0.1:0").unwrap(); + let addr = listener.local_addr().unwrap(); + let root = root.to_path_buf(); + std::thread::spawn(move || { + let _ = serve_http_listener(listener, &root, false, false, limits); + }); + addr +} + +fn open_ok(addr: std::net::SocketAddr) -> TcpStream { + let mut s = TcpStream::connect(addr).unwrap(); + write_client_opening(&mut s, "127.0.0.1", "/", None).unwrap(); + assert_eq!(read_status(&mut s).unwrap(), 200); + s +} + +#[test] +fn connection_limit_shed_and_recover() { + let root = tmp_served_root("slots"); + let limits = ServeLimits { max_connections: 1, timeout_secs: 0, ..Default::default() }; + let addr = spawn_listener(&root, limits); + + let held = open_ok(addr); // occupies the single slot + + // Second connection: shed with 503 before any opening is read. + let mut second = TcpStream::connect(addr).unwrap(); + assert_eq!(read_status(&mut second).unwrap(), 503); + + // Free the slot; a fresh connection now succeeds (poll: the server + // notices the disconnect on its next read). + drop(held); + let ok = (0..50).any(|_| { + std::thread::sleep(Duration::from_millis(100)); + let mut s = match TcpStream::connect(addr) { Ok(s) => s, Err(_) => return false }; + write_client_opening(&mut s, "127.0.0.1", "/", None).is_ok() + && read_status(&mut s) == Ok(200) + }); + assert!(ok, "slot was never freed"); + let _ = std::fs::remove_dir_all(&root); +} + +#[test] +fn errored_connection_frees_its_slot() { + let root = tmp_served_root("errslot"); + let limits = ServeLimits { max_connections: 1, timeout_secs: 0, ..Default::default() }; + let addr = spawn_listener(&root, limits); + + // A connection whose handler errors (garbage opening → 400) must free + // its slot via the guard, not leak it. + let mut bad = TcpStream::connect(addr).unwrap(); + bad.write_all(b"garbage\r\n\r\n").unwrap(); + let _ = read_status(&mut bad); // 400 (best-effort) + drop(bad); + + let ok = (0..50).any(|_| { + std::thread::sleep(Duration::from_millis(100)); + let mut s = match TcpStream::connect(addr) { Ok(s) => s, Err(_) => return false }; + write_client_opening(&mut s, "127.0.0.1", "/", None).is_ok() + && read_status(&mut s) == Ok(200) + }); + assert!(ok, "errored connection leaked its slot"); + let _ = std::fs::remove_dir_all(&root); +} + +/// A client that stops READING mid-GetPack must be reaped by the write +/// timeout (today it parks the server thread in a blocking write forever). +/// Server sends a pack big enough to overflow socket buffers; the client +/// never reads; after the timeout the server closes. +#[test] +fn write_side_stall_is_reaped_by_timeout() { + let root = tmp_served_root("wstall"); + // Put a few MiB of committed content in the repo first (init + commit a + // large random file via the Repo API — see repo.rs tests for the + // init/commit helper pattern), so GetPack has real bytes to send. + // ... then: + let limits = ServeLimits { max_connections: 0, timeout_secs: 1, ..Default::default() }; + let addr = spawn_listener(&root, limits); + + let mut s = open_ok(addr); + // Speak just enough wire protocol to trigger a GetPack, then stop reading: + // handshake HELLO, then a GetPack for the branch tip with empty haves. + // (Use wire::Request::{Hello, GetPack}.encode() + write_frame directly.) + // After sending the GetPack request, sleep instead of reading. + std::thread::sleep(Duration::from_secs(5)); + // The server must have dropped the connection rather than blocking + // forever: our next write eventually errors (reset), or read sees EOF. + s.set_read_timeout(Some(Duration::from_secs(5))).unwrap(); + let mut sink = [0u8; 4096]; + loop { + match s.read(&mut sink) { + Ok(0) | Err(_) => break, // EOF or reset — server gave up: pass + Ok(_) => continue, // drain what was already buffered + } + } + let _ = std::fs::remove_dir_all(&root); +} + +#[test] +fn silent_session_is_reaped_by_timeout() { + let root = tmp_served_root("reap"); + let limits = ServeLimits { max_connections: 0, timeout_secs: 1, ..Default::default() }; + let addr = spawn_listener(&root, limits); + + let mut s = open_ok(addr); // handshake never sent; go silent + std::thread::sleep(Duration::from_secs(3)); + // Server must have dropped us: the read sees EOF/reset, not a hang. + s.set_read_timeout(Some(Duration::from_secs(5))).unwrap(); + let mut buf = [0u8; 1]; + match s.read(&mut buf) { + Ok(0) => {} // clean EOF — dropped + Err(_) => {} // reset — also dropped + Ok(_) => panic!("server sent data to a silent client"), + } + // Spool dir is clean. + let tmp = root.join(".sc").join("tmp"); + assert!(!tmp.exists() || std::fs::read_dir(&tmp).unwrap().next().is_none()); + let _ = std::fs::remove_dir_all(&root); +} +``` + +- [ ] **Step 6: Run to verify failure** — `cargo test -p scl-repo connection_limit_shed` → compile error (`ServeLimits`, new arities). + +- [ ] **Step 7: Implement the listener changes** + +1. Add `ServeLimits` (+ `Default`) with the doc comment stating each default and its provenance (git-daemon 32; on-by-default 300s divergence; 16 GiB divergence — cite the spec). +2. RAII slot guard: + +```rust +/// Decrements the live-connection count on drop, so every exit path — clean +/// return, error, panic unwind — frees its slot (the TempPackGuard +/// discipline applied to connection slots). +struct SlotGuard(std::sync::Arc); +impl Drop for SlotGuard { + fn drop(&mut self) { + self.0.fetch_sub(1, std::sync::atomic::Ordering::SeqCst); + } +} +``` + +3. `serve_http_listener(listener, root, read_only, mandatory_auth, limits)`: + +```rust +let live = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0)); +let mut backoff = AcceptBackoff::new(); +for incoming in listener.incoming() { + let mut stream = match incoming { + Ok(s) => { + backoff.on_success(); + s + } + Err(e) => { + eprintln!("sc serve --http: accept error: {e}"); + std::thread::sleep(backoff.on_error()); + continue; + } + }; + // Connection cap: acquire a slot BEFORE spawning; shed with 503 when full. + let guard = { + let prev = live.fetch_add(1, std::sync::atomic::Ordering::SeqCst); + let g = SlotGuard(live.clone()); + if limits.max_connections != 0 && prev >= limits.max_connections as usize { + let _ = write_status(&mut stream, 503); + continue; // g drops here → count restored; no thread, no handshake + } + g + }; + let root = root.to_path_buf(); + std::thread::spawn(move || { + let _guard = guard; // slot held for the connection's lifetime + if let Err(e) = handle_http_connection(stream, &root, read_only, mandatory_auth, limits) { + eprintln!("sc serve --http: connection error: {e}"); + } + }); +} +Ok(()) +``` + +4. `handle_http_connection(..., limits: ServeLimits)`: replace the timeout-clear block (~line 594) with: + +```rust +// P31: the opening timeout is REPLACED (not cleared) by the session +// timeout, on BOTH read and write sides — a write timeout covers a client +// that stops reading mid-GetPack. Under P25 chunking a per-syscall timeout +// is progress-based: only true zero-byte stalls trip it. 0 disables. +let session = if limits.timeout_secs == 0 { + None +} else { + Some(Duration::from_secs(limits.timeout_secs)) +}; +stream + .set_read_timeout(session) + .map_err(|e| Error::ConnectionLost(format!("sc+http set session read_timeout: {e}")))?; +stream + .set_write_timeout(session) + .map_err(|e| Error::ConnectionLost(format!("sc+http set session write_timeout: {e}")))?; +``` + +and end with `crate::wire::serve_with_policy(root, &mut reader, &mut stream, crate::wire::WirePolicy { read_only, max_pack_size: limits.max_pack_size, ro_drain_cap: crate::wire::RO_DRAIN_CAP })`. A timeout surfaces as `Error::Io` (`WouldBlock` on Unix, `TimedOut` on Windows) from `serve_with_policy` → the handler returns `Err` → logged, thread exits, guard frees the slot. No special-casing needed beyond the log line; do NOT retry on `WouldBlock`. +5. `serve_http(addr, root, read_only, allow_public, limits)`: pass `limits` through; call `crate::wire::validate_max_pack_size(limits.max_pack_size)?` before binding. + +- [ ] **Step 8: Run to verify pass** + +Run: `cargo test -p scl-repo connection_limit_shed silent_session` +Expected: PASS. Also run the full crate: `cargo test -p scl-repo` — the existing http tests (auth, bind gate) must stay green with the added `limits` parameter (update their call sites to `ServeLimits::default()` or `timeout_secs: 0` where a test holds idle connections). + +- [ ] **Step 9: Commit** + +```bash +cargo fmt --all && git add -A crates/repo && git commit -m "feat(http): connection cap + 503, session timeouts, accept backoff (P31)" +``` + +--- + +### Task 5: CLI flags + stdio plumbing + +**Files:** +- Modify: `crates/cli/src/main.rs` — `Cmd::Serve` variant (~line 280), dispatch (~line 813), `run_serve` (~line 3122) + +**Interfaces:** +- Consumes: `ServeLimits`, `serve_http` (Task 4), `wire::{serve_with_policy, WirePolicy, RO_DRAIN_CAP, DEFAULT_MAX_PACK_SIZE, validate_max_pack_size}` (Task 3). + +- [ ] **Step 1: Add the flags** to `Cmd::Serve`: + +```rust +/// Maximum concurrent connections (`--http` only); 0 = unlimited. +/// Default 32 (git-daemon parity). At the limit new connections get an +/// immediate busy status and close. (P31) +#[arg(long)] +max_connections: Option, +/// Session idle timeout in seconds (`--http` only); 0 = disabled. +/// Default 300. A connection making zero-byte progress for this long is +/// dropped. (P31) +#[arg(long)] +timeout: Option, +/// Maximum incoming pack size in bytes (both --http and --stdio); +/// 0 = unlimited. Default 16 GiB. Must be ≥ 256 MiB (MAX_OBJECT_SIZE) +/// when non-zero. (P31) +#[arg(long)] +max_pack_size: Option, +``` + +- [ ] **Step 2: Update `run_serve`** + +```rust +fn run_serve( + stdio: bool, + http: Option, + read_only: bool, + allow_public: bool, + max_connections: Option, + timeout: Option, + max_pack_size: Option, + path: PathBuf, +) -> Result<()> { + match (stdio, http) { + (true, None) => { + if read_only || allow_public { + anyhow::bail!("--read-only/--allow-public apply only to --http, not --stdio"); + } + if max_connections.is_some() || timeout.is_some() { + anyhow::bail!("--max-connections/--timeout apply only to --http, not --stdio"); + } + let max_pack = max_pack_size.unwrap_or(scl_repo::wire::DEFAULT_MAX_PACK_SIZE); + scl_repo::wire::validate_max_pack_size(max_pack)?; + let policy = scl_repo::wire::WirePolicy { + read_only: false, + max_pack_size: max_pack, + ro_drain_cap: scl_repo::wire::RO_DRAIN_CAP, + }; + let mut stdin = std::io::stdin().lock(); + let mut stdout = std::io::stdout().lock(); + scl_repo::wire::serve_with_policy(&path, &mut stdin, &mut stdout, policy)?; + Ok(()) + } + (false, Some(addr)) => { + let mut limits = scl_repo::http_transport::ServeLimits::default(); + if let Some(n) = max_connections { limits.max_connections = n; } + if let Some(t) = timeout { limits.timeout_secs = t; } + if let Some(m) = max_pack_size { limits.max_pack_size = m; } + scl_repo::http_transport::serve_http(&addr, &path, read_only, allow_public, limits)?; + Ok(()) + } + // ... existing arms unchanged + } +} +``` + +Update the dispatch site to pass the three new fields. Note `WirePolicy`/`RO_DRAIN_CAP`/`DEFAULT_MAX_PACK_SIZE`/`validate_max_pack_size`/`ServeLimits` must be `pub` exports (they are, from Tasks 3–4). + +- [ ] **Step 3: Verify behavior manually** + +```bash +cargo run -q --bin sc -- serve --stdio --timeout 5 . 2>&1 | head -2 # expect: the --http-only bail +cargo run -q --bin sc -- serve --http 127.0.0.1:0 --max-pack-size 1024 . 2>&1 | head -2 # expect: floor error naming MAX_OBJECT_SIZE +``` + +Expected: both print clear errors and exit non-zero. + +- [ ] **Step 4: Full workspace test + commit** + +Run: `cargo test --workspace && cargo clippy --workspace --all-targets -- -D warnings` + +```bash +cargo fmt --all && git add -A crates/cli && git commit -m "feat(cli): sc serve --max-connections/--timeout/--max-pack-size (P31)" +``` + +--- + +### Task 6: Demo script + +**Files:** +- Create: `demo/run_limits_demo.sh` (pattern: `demo/run_http_auth_demo.sh` — read it first for the port-pick/cleanup/run-twice conventions, including the `listening on` stdout parse for the OS-assigned port) + +**Interfaces:** +- Consumes: the CLI flags (Task 5), 503 busy behavior, timeout reaping, `EC_TOO_LARGE`. + +- [ ] **Step 1: Write the demo.** It must (following the sibling script's structure — `set -euo pipefail`, temp workdir, `sc` built once, server started with `--http 127.0.0.1:0` and the port parsed from `listening on`): + 1. Init a repo, commit a file, start `sc serve --http 127.0.0.1:0 --max-connections 1 --timeout 2 --max-pack-size 268435456 --allow-public`… (use loopback — no `--allow-public` needed). + 2. **Busy:** hold one connection open with a raw client (a `sleep`-piped `nc` if available, else a second `sc clone` in a loop is unreliable — prefer opening a TCP connection via `python3`… **no**: python3 is not guaranteed; use a background `sc clone` of a large-enough repo? Simplest reliable primitive: `exec 3<>/dev/tcp/127.0.0.1/$PORT` in bash holds a connection without sending an opening — the slot is taken at accept time, before the opening read). Then a second `sc clone sc+http://…` must fail with the busy error (assert on stderr text), close fd 3, and the same clone must succeed. + 3. **Timeout:** open `exec 3<>/dev/tcp/...` again, sleep 4 (> `--timeout 2` + opening 30s does not apply — the opening timeout drops it… NOTE: an opening-silent connection is dropped by the **existing 30s opening timeout**, not the session timeout; a demo can't cheaply hold a post-opening session silent without a wire client. So the demo proves timeout indirectly: assert the busy slot from step 2's silent holder is auto-freed — a clone succeeds after the drop **without** closing fd 3 manually. Print what's being proven.) + 4. **Spool cap:** restart the server with `--max-pack-size 268435456` (the floor) and push a > 256 MiB… too slow for a demo. Instead restart with the floor value and demonstrate the *refusal path* differently: a push of a small repo succeeds (cap present, harmless); then restart with `--max-pack-size 1024` and assert startup fails with the floor error. The mid-stream abort itself is covered by unit tests; the demo proves the knobs and the busy/reap behavior end-to-end. + 5. Assert zero `.sc/tmp` residue on the served repo; run the whole script twice (the repo's demo discipline). + +- [ ] **Step 2: Run it twice** + +Run: `bash demo/run_limits_demo.sh && bash demo/run_limits_demo.sh` +Expected: both runs end with the script's success line; no leftover temp dirs. + +- [ ] **Step 3: Commit** + +```bash +git add demo/run_limits_demo.sh && git commit -m "demo: listener limits proof — busy shed, slot reap, cap knobs (P31)" +``` + +--- + +### Task 7: Docs — ADR-0041, THREAT-MODEL, CLAUDE.md + +**Files:** +- Create: `docs/adr/0041-listener-resource-limits.md` +- Modify: `docs/THREAT-MODEL.md` (the ADR-0036 operational-hardening items, ~lines 91-93, plus the env-var and signature sections) +- Modify: `CLAUDE.md` (serve command block; the P26 "accepted design consequences" paragraph) + +**Interfaces:** none (docs only). Read `docs/adr/0040-sc-http-access-control.md` first and match its structure (Status/Date/Phase/Context/Decision/Consequences). + +- [ ] **Step 1: Write ADR-0041.** Must contain: the four bounds with defaults and mechanisms; the two recorded divergences (timeout **on**-by-default vs git-daemon's off — defaults-off knobs don't close audit Highs; finite 16 GiB spool default vs git's unlimited `receive.maxInputSize` — sc servers often share the working tree's volume); the 8 MiB RO drain posture (honest misconfigurations keep the typed error, hostile bulk spools almost nothing); mid-stream abort = connection-fatal (desync); spool cap applies to both transports (shared wire layer); rejected alternatives (block-at-accept semaphore, git-daemon grace-then-shed, Rust-Book queue pool, reserved-fd shed, env-var/config-file knobs); links to decision tickets #28/#27 and `docs/research/bounded-server-patterns.md`. +- [ ] **Step 2: Update THREAT-MODEL.md.** (a) The three ADR-0036 boundaries → closed by P31, naming the mechanisms; (b) name + close the aggregate-spool gap (read-only drain path included); (c) the env-var re-affirmation wording from map ticket #31 (same-user adversary defeats the identity key; fd/stdin stays an additive deferred mode with recorded revisit triggers); (d) the signature-replay re-affirmation from map ticket #32 (ancestry already id-bound; residue is ref binding only — "replayable to a ref tip within its own history"; gittuf-shaped attestation deferred with revisit triggers). +- [ ] **Step 3: Update CLAUDE.md.** Add the three flags to the serve command block with one-line semantics; amend the P26 paragraph's "accepted design consequences, not yet closed" list to point at P31; add a short "Phase 31 is built." section following the existing phase-section pattern. +- [ ] **Step 4: Commit** + +```bash +git add docs CLAUDE.md && git commit -m "docs: ADR-0041 listener limits + THREAT-MODEL closures and re-affirmations (P31)" +``` + +--- + +### Task 8: Final verification + +- [ ] **Step 1:** `cargo test --workspace --all-targets` — all green. +- [ ] **Step 2:** `cargo clippy --workspace --all-targets -- -D warnings && cargo fmt --all -- --check` — clean. +- [ ] **Step 3:** Run the neighboring demos that exercise the touched paths: `bash demo/run_http_remote_demo.sh && bash demo/run_http_auth_demo.sh && bash demo/run_ssh_remote_demo.sh && bash demo/run_limits_demo.sh` — all end with their success lines (the first three prove no regression from the `WirePolicy`/signature changes; defaults must keep every existing flow working). +- [ ] **Step 4:** Confirm zero residue: `git status --short` shows only intended files. diff --git a/docs/superpowers/specs/2026-07-10-p31-listener-limits-design.md b/docs/superpowers/specs/2026-07-10-p31-listener-limits-design.md index 24324f50..2f562b7d 100644 --- a/docs/superpowers/specs/2026-07-10-p31-listener-limits-design.md +++ b/docs/superpowers/specs/2026-07-10-p31-listener-limits-design.md @@ -93,8 +93,12 @@ pre-1.0 in-workspace signature change, not a wire change. `spill_pack_stream` gain a byte budget. The running total is checked as chunk frames arrive (git's `receive.maxInputSize` shape: abort *while counting*, never after materializing); exceeding it raises a new typed -`Error::PackTooLarge { limit }`, the partial spool is removed by the guard, -and the server replies with a new wire error code **`EC_TOO_LARGE = 6`**. +`Error::PackTooLarge`, the partial spool is removed by the guard, +and the server sends a best-effort reply with a new wire error code +**`EC_TOO_LARGE = 6`**, then closes the connection — aborting mid-stream +leaves unread chunk frames in flight, so the stream is desynced by +construction and the connection cannot be reused (same posture as the RO +drain overflow below). Old clients degrade gracefully — unknown codes already fall back to `Error::Remote(msg)` — so no protocol version bump. The cap composes *above* ADR-0039's per-frame/per-record/decompression caps; it never replaces them. @@ -119,7 +123,7 @@ misconfigurations keep the clear error, hostile bulk spools almost nothing). |---|---|---| | Accept while full | 503 + close, no handshake | "server busy, retry" | | Zero-byte stall > timeout | log + drop connection, spool cleaned | connection error | -| Pack exceeds `--max-pack-size` | abort mid-stream, `EC_TOO_LARGE`, spool cleaned, connection continues | "pack exceeds server limit (N)" | +| Pack exceeds `--max-pack-size` | abort mid-stream, best-effort `EC_TOO_LARGE`, spool cleaned, close | "pack exceeds server limit (N)" or connection error | | RO push ≤ 8 MiB | drain, `EC_READONLY` (unchanged) | "read-only" | | RO push > 8 MiB | best-effort `EC_READONLY`, close | read-only error or connection error | | Accept error (EMFILE etc.) | 5ms→1s backoff, loop stays alive | connect delay/refusal | @@ -137,8 +141,8 @@ Pinned regression tests per bound: dropped after the (test-shortened) timeout; `.sc/tmp` empty afterwards. 4. Write-side stall: a client that stops reading mid-`GetPack` is dropped. 5. Spool cap: a pack exceeding a small `--max-pack-size` fails mid-stream with - the typed error, `.sc/tmp` is empty, and the same connection can issue a - subsequent successful request. + the typed error, `.sc/tmp` is empty, the connection is closed, and a fresh + connection succeeds. 6. Config floor: `--max-pack-size` below `MAX_OBJECT_SIZE` refused at startup. 7. RO drain: small RO push still gets `EC_READONLY`; oversized RO push gets the connection closed, ≤ 8 MiB spooled (assert via temp-dir observation), zero From 9c2fbd3f35b9dd95f417c4307f3abe79f904c425 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 11:20:34 +0300 Subject: [PATCH 03/14] feat(wire): PackTooLarge/ServerBusy errors + EC_TOO_LARGE wire code (P31) Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/repo/src/error.rs | 9 +++++++++ crates/repo/src/wire.rs | 21 ++++++++++++++++++++- 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/crates/repo/src/error.rs b/crates/repo/src/error.rs index 0d388ef5..f64e8ddf 100644 --- a/crates/repo/src/error.rs +++ b/crates/repo/src/error.rs @@ -102,6 +102,15 @@ pub enum Error { /// (`--read-only` or an `ro`-scope token). P29. #[error("server is read-only")] ReadOnly, + /// The server aborted an incoming pack mid-stream because it exceeded the + /// operator's `--max-pack-size` cap (P31). Payload is the server's own + /// human-readable limit text, carried verbatim across the wire. + #[error("pack exceeds the server's --max-pack-size limit: {0}")] + PackTooLarge(String), + /// The server refused the connection at accept time because + /// `--max-connections` was reached (P31). Retryable. + #[error("server busy (connection limit reached); retry later")] + ServerBusy, } pub type Result = std::result::Result; diff --git a/crates/repo/src/wire.rs b/crates/repo/src/wire.rs index e81d15ad..8ab46892 100644 --- a/crates/repo/src/wire.rs +++ b/crates/repo/src/wire.rs @@ -79,6 +79,7 @@ const EC_PROTOCOL: u8 = 4; /// P29: the server is enforcing a read-only serve-token policy and refused a /// mutating verb before any store write. See [`serve_with_policy`]. const EC_READONLY: u8 = 5; +pub(crate) const EC_TOO_LARGE: u8 = 6; const EC_OTHER: u8 = 255; /// A client request — the wire mirror of one [`crate::transport::Transport`] @@ -487,6 +488,7 @@ pub fn err_to_wire(e: &Error) -> (u8, String) { Error::Core(scl_core::Error::NotFound(_)) => EC_NOT_FOUND, Error::Protocol(_) => EC_PROTOCOL, Error::ReadOnly => EC_READONLY, + Error::PackTooLarge(_) => EC_TOO_LARGE, _ => EC_OTHER, }; (code, e.to_string()) @@ -494,16 +496,22 @@ pub fn err_to_wire(e: &Error) -> (u8, String) { /// Reconstruct a typed error from its wire code; unknown/untyped codes become /// `Error::Remote(msg)` so the message is never lost. -pub fn err_from_wire(code: u8, msg: String) -> Error { +pub fn wire_to_err(code: u8, msg: String) -> Error { match code { EC_NOT_A_REPO => Error::NotARepo, EC_NON_FAST_FORWARD => Error::NonFastForward, EC_PROTOCOL => Error::Protocol(msg), EC_READONLY => Error::ReadOnly, + EC_TOO_LARGE => Error::PackTooLarge(msg), _ => Error::Remote(msg), // EC_NOT_FOUND + EC_OTHER: typed enough as text } } +/// Alias for backward compatibility with existing call sites. +fn err_from_wire(code: u8, msg: String) -> Error { + wire_to_err(code, msg) +} + // --- response body builders/decoders (one symmetric pair per verb shape) --- /// Response body for Hello: the server's protocol version. @@ -1316,4 +1324,15 @@ mod tests { "expected Protocol error, got {err:?}" ); } + + #[test] + fn pack_too_large_round_trips_through_wire_codes() { + let e = Error::PackTooLarge("16 GiB (17179869184 bytes)".to_string()); + let (code, msg) = err_to_wire(&e); + assert_eq!(code, EC_TOO_LARGE); + match wire_to_err(code, msg) { + Error::PackTooLarge(m) => assert!(m.contains("17179869184")), + other => panic!("expected PackTooLarge, got {other:?}"), + } + } } From 92bb3f8743dfa8f3225145dd1ec891b83645228b Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 11:27:35 +0300 Subject: [PATCH 04/14] feat(wire): byte budget with typed mid-stream abort in read_pack_stream (P31) Add max_bytes parameter to read_pack_stream with P31 Task 2 semantics: - max_bytes == 0 means unlimited - max_bytes > 0 caps the total bytes written to sink - Over-budget chunk abort with Error::PackTooLarge before writing to sink - TDD: two new tests verify budget enforcement and pass-through - All existing call sites pass 0 (unlimited) for now; Task 3 threads real cap Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/repo/src/stdio_transport.rs | 4 +-- crates/repo/src/wire.rs | 55 +++++++++++++++++++++++++----- 2 files changed, 49 insertions(+), 10 deletions(-) diff --git a/crates/repo/src/stdio_transport.rs b/crates/repo/src/stdio_transport.rs index 0ccc3d5f..a705e4f9 100644 --- a/crates/repo/src/stdio_transport.rs +++ b/crates/repo/src/stdio_transport.rs @@ -117,7 +117,7 @@ impl Transport for WireClient { )?; let frame = wire::read_frame(&mut rw.0)?; wire::parse_response(frame)?; // Ok(empty body) means "stream follows"; Err propagates typed - wire::read_pack_stream(&mut rw.0, out)?; + wire::read_pack_stream(&mut rw.0, out, 0)?; Ok(()) } @@ -563,7 +563,7 @@ mod tests { wire::Request::UpdateRef { .. } => return, // die without replying wire::Request::PutPack => { let mut buf = Vec::new(); - wire::read_pack_stream(&mut server_read, &mut buf).unwrap(); + wire::read_pack_stream(&mut server_read, &mut buf, 0).unwrap(); let ids = t.put_pack(&mut &buf[..]).unwrap(); wire::write_ok(&mut server_write, &wire::ids_body(&ids)).unwrap(); } diff --git a/crates/repo/src/wire.rs b/crates/repo/src/wire.rs index 8ab46892..d4e808e1 100644 --- a/crates/repo/src/wire.rs +++ b/crates/repo/src/wire.rs @@ -428,15 +428,27 @@ fn read_up_to(src: &mut (impl Read + ?Sized), buf: &mut [u8]) -> Result { /// Returns the total number of bytes streamed. Any frame that is neither a /// well-formed chunk nor the end marker — including EOF before `ST_PACK_END` /// — is `Err(Error::Protocol(_))`. -pub fn read_pack_stream(r: &mut impl Read, sink: &mut (impl Write + ?Sized)) -> Result { +/// +/// `max_bytes` bounds the total bytes written to `sink`: if `max_bytes != 0` +/// and a chunk would exceed the budget, aborts with `Error::PackTooLarge` +/// *before* writing the chunk. `max_bytes == 0` means unlimited. +pub fn read_pack_stream( + r: &mut impl Read, + sink: &mut (impl Write + ?Sized), + max_bytes: u64, +) -> Result { let mut total: u64 = 0; loop { let frame = read_frame_opt(r)?.ok_or_else(|| Error::Protocol("EOF before ST_PACK_END".into()))?; match frame.split_first() { Some((&ST_PACK_CHUNK, rest)) => { + let next = total + rest.len() as u64; + if max_bytes != 0 && next > max_bytes { + return Err(Error::PackTooLarge(format!("{max_bytes} bytes"))); + } sink.write_all(rest)?; - total += rest.len() as u64; + total = next; } Some((&ST_PACK_END, [])) => return Ok(total), _ => return Err(Error::Protocol("unexpected frame in pack stream".into())), @@ -801,7 +813,7 @@ fn spill_pack_stream( ) -> Result { let guard = crate::transport::TempPackGuard::new(layout)?; let mut f = std::fs::File::create(guard.path())?; - read_pack_stream(r, &mut f)?; + read_pack_stream(r, &mut f, 0)?; Ok(guard) } @@ -1160,7 +1172,7 @@ mod tests { let mut sink = Vec::new(); let mut r = std::io::Cursor::new(buf); - let total = read_pack_stream(&mut r, &mut sink).unwrap(); + let total = read_pack_stream(&mut r, &mut sink, 0).unwrap(); assert_eq!(total, data.len() as u64); assert_eq!(sink, data); } @@ -1174,7 +1186,7 @@ mod tests { let mut buf = Vec::new(); write_pack_stream(&mut buf, &mut std::io::Cursor::new(&data), chunk_size).unwrap(); let mut sink = Vec::new(); - let total = read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink).unwrap(); + let total = read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink, 0).unwrap(); assert_eq!(total, data.len() as u64); assert_eq!(sink, data, "mismatch for len={len}"); } @@ -1186,7 +1198,7 @@ mod tests { let mut buf = Vec::new(); write_pack_stream(&mut buf, &mut std::io::Cursor::new(&data), 7).unwrap(); let mut sink = Vec::new(); - let total = read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink).unwrap(); + let total = read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink, 0).unwrap(); assert_eq!(total, 0); assert!(sink.is_empty()); } @@ -1307,7 +1319,7 @@ mod tests { let mut buf = Vec::new(); write_frame(&mut buf, &[ST_PACK_CHUNK, b'a', b'b']).unwrap(); let mut sink = Vec::new(); - let err = read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink).unwrap_err(); + let err = read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink, 0).unwrap_err(); assert!( matches!(err, Error::Protocol(_)), "expected Protocol error, got {err:?}" @@ -1318,13 +1330,40 @@ mod tests { write_frame(&mut buf, &[ST_PACK_CHUNK, b'a']).unwrap(); write_frame(&mut buf, &[ST_OK]).unwrap(); // not a valid pack-stream marker let mut sink = Vec::new(); - let err = read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink).unwrap_err(); + let err = read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink, 0).unwrap_err(); assert!( matches!(err, Error::Protocol(_)), "expected Protocol error, got {err:?}" ); } + #[test] + fn pack_stream_over_budget_aborts_typed() { + let payload = vec![7u8; 4096]; + let mut buf = Vec::new(); + write_pack_stream(&mut buf, &mut std::io::Cursor::new(&payload), 1024).unwrap(); + let mut sink = Vec::new(); + let err = read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink, 2048).unwrap_err(); + assert!(matches!(err, Error::PackTooLarge(_)), "got {err:?}"); + assert!( + sink.len() <= 2048, + "sink got over-budget bytes: {}", + sink.len() + ); + } + + #[test] + fn pack_stream_under_budget_passes() { + let payload = vec![7u8; 4096]; + let mut buf = Vec::new(); + write_pack_stream(&mut buf, &mut std::io::Cursor::new(&payload), 1024).unwrap(); + let mut sink = Vec::new(); + assert_eq!( + read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink, 4096).unwrap(), + 4096 + ); + } + #[test] fn pack_too_large_round_trips_through_wire_codes() { let e = Error::PackTooLarge("16 GiB (17179869184 bytes)".to_string()); From 7159b86487a0702e0a43046dc13063258280f980 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 11:33:51 +0300 Subject: [PATCH 05/14] feat(wire): WirePolicy with spool cap + capped read-only drain (P31) Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/repo/src/http_transport.rs | 10 +- crates/repo/src/wire.rs | 225 +++++++++++++++++++++++++++--- 2 files changed, 216 insertions(+), 19 deletions(-) diff --git a/crates/repo/src/http_transport.rs b/crates/repo/src/http_transport.rs index d2c93f62..4c86e701 100644 --- a/crates/repo/src/http_transport.rs +++ b/crates/repo/src/http_transport.rs @@ -599,7 +599,15 @@ fn handle_http_connection( .map_err(|e| Error::ConnectionLost(format!("sc+http clear read_timeout: {e}")))?; let read_only = server_read_only || token_read_only; - crate::wire::serve_with_policy(root, &mut reader, &mut stream, read_only) + crate::wire::serve_with_policy( + root, + &mut reader, + &mut stream, + crate::wire::WirePolicy { + read_only, + ..Default::default() + }, + ) } #[cfg(test)] diff --git a/crates/repo/src/wire.rs b/crates/repo/src/wire.rs index d4e808e1..0d49cb14 100644 --- a/crates/repo/src/wire.rs +++ b/crates/repo/src/wire.rs @@ -610,6 +610,59 @@ pub fn decode_refs_body(b: &[u8]) -> Result> { use crate::transport::{LocalTransport, Transport}; +/// Default cap on an incoming `PutPack` spool when no operator override is +/// configured (P31): 16 GiB. Threaded through [`WirePolicy::max_pack_size`]; +/// `0` means unlimited. +pub const DEFAULT_MAX_PACK_SIZE: u64 = 16 * 1024 * 1024 * 1024; + +/// Cap on how much of a read-only connection's rejected `PutPack` stream is +/// drained to disk before the connection is closed rather than kept in sync +/// (P31): 8 MiB. A read-only push is going to be thrown away regardless, so +/// there is no reason to spool an attacker-sized pack just to reject it — +/// unlike the normal `PutPack` arm, which must accept up to +/// [`WirePolicy::max_pack_size`] because that pack might actually be used. +pub const RO_DRAIN_CAP: u64 = 8 * 1024 * 1024; + +/// Per-connection policy for [`serve_with_policy`] (P31): read-only gating +/// (P29) plus the two spool caps above. `Default` matches every pre-P31 +/// caller's prior unbounded behavior for `read_only`/`ro_drain_cap` sizing +/// intent, but now bounds `max_pack_size` to [`DEFAULT_MAX_PACK_SIZE`] rather +/// than leaving `PutPack` spool growth unbounded. +#[derive(Debug, Clone, Copy)] +pub struct WirePolicy { + pub read_only: bool, + /// Cap on a normal (non-read-only) `PutPack` spool, in bytes. `0` means + /// unlimited. + pub max_pack_size: u64, + /// Cap on how much of a read-only connection's rejected `PutPack` stream + /// is drained before closing the connection, in bytes. + pub ro_drain_cap: u64, +} + +impl Default for WirePolicy { + fn default() -> Self { + Self { + read_only: false, + max_pack_size: DEFAULT_MAX_PACK_SIZE, + ro_drain_cap: RO_DRAIN_CAP, + } + } +} + +/// Validate an operator-supplied `--max-pack-size` value. `0` (unlimited) is +/// always fine; a nonzero cap below [`scl_core::MAX_OBJECT_SIZE`] can never +/// fit even one object and is rejected as a misconfiguration rather than +/// silently failing every push at runtime. +pub fn validate_max_pack_size(max: u64) -> Result<()> { + if max != 0 && max < scl_core::MAX_OBJECT_SIZE as u64 { + return Err(Error::InvalidArgument(format!( + "--max-pack-size {max} is below MAX_OBJECT_SIZE ({}); a cap that cannot fit one object is a misconfiguration", + scl_core::MAX_OBJECT_SIZE + ))); + } + Ok(()) +} + /// Serve the repo at `root` to one wire-protocol peer until `Bye`/EOF. /// /// This is the whole server: every verb dispatches onto [`LocalTransport`], @@ -617,15 +670,17 @@ use crate::transport::{LocalTransport, Transport}; /// exactly as for a local remote. Verb failures are *replies*; only protocol /// violations end the session. /// -/// `read_only` (P29): when true, the three mutating verbs +/// `policy.read_only` (P29): when true, the three mutating verbs /// (`PutObject`/`PutPack`/`UpdateRef`) are rejected with [`Error::ReadOnly`] -/// BEFORE any store write; read verbs are always allowed. [`serve`] is the -/// `read_only = false` wrapper every pre-P29 caller uses unchanged. +/// BEFORE any store write; read verbs are always allowed. `policy.max_pack_size` +/// / `policy.ro_drain_cap` (P31) bound the `PutPack` spool in both the normal +/// and read-only-drain arms. [`serve`] is the `WirePolicy::default()` wrapper +/// every pre-P31 caller uses unchanged. pub fn serve_with_policy( root: &std::path::Path, r: &mut impl Read, w: &mut impl Write, - read_only: bool, + policy: WirePolicy, ) -> Result<()> { // Handshake: HELLO must come first, and versions must match, before any // repo access happens. @@ -677,7 +732,7 @@ pub fn serve_with_policy( }; // P29 read-only gate: reject mutating verbs before any store write. - if read_only { + if policy.read_only { match &req { Request::PutObject { .. } | Request::UpdateRef { .. } => { // Single request frames — nothing trails them; reject and @@ -694,12 +749,27 @@ pub fn serve_with_policy( // so no object reaches the store: "rejected before any // store write" holds. `spill_pack_stream` + guard drop is // the same bounded-RAM machinery the normal PutPack arm - // uses. - let guard = spill_pack_stream(r, transport.layout())?; - drop(guard); - let (code, msg) = err_to_wire(&Error::ReadOnly); - write_err(w, code, &msg)?; - continue; + // uses, bounded to `policy.ro_drain_cap` (P31) rather than + // the normal arm's larger `max_pack_size` — a read-only + // push is discarded regardless, so there's no reason to + // spool an attacker-sized pack just to reject it. + match spill_pack_stream(r, transport.layout(), policy.ro_drain_cap) { + Ok(guard) => { + drop(guard); + let (code, msg) = err_to_wire(&Error::ReadOnly); + write_err(w, code, &msg)?; + continue; + } + Err(Error::PackTooLarge(_)) => { + // Over the drain cap: the stream is desynced (we + // stopped reading mid-pack), so a best-effort + // reply is all we can do — then close. + let (code, msg) = err_to_wire(&Error::ReadOnly); + let _ = write_err(w, code, &msg); + return Ok(()); + } + Err(e) => return Err(e), + } } _ => {} } @@ -742,7 +812,7 @@ pub fn serve_with_policy( } } Request::PutPack => { - match spill_pack_stream(r, transport.layout()) { + match spill_pack_stream(r, transport.layout(), policy.max_pack_size) { Ok(guard) => { match transport.ingest_from(guard.path()) { Ok(ids) => write_ok(w, &ids_body(&ids))?, @@ -754,6 +824,17 @@ pub fn serve_with_policy( // `guard` drops here — temp pack file removed, // whether ingestion succeeded or failed. } + Err(e @ Error::PackTooLarge(_)) => { + // Over the cap: the stream is desynced (we stopped + // reading mid-pack, so any bytes still on the wire + // are neither read nor a subsequent request), so a + // best-effort reply is the most we can do — then + // close the connection rather than try to serve + // further requests off a desynced stream. + let (code, msg) = err_to_wire(&e); + let _ = write_err(w, code, &msg); + return Ok(()); + } Err(e) => { let (code, msg) = err_to_wire(&e); write_err(w, code, &msg)?; @@ -795,10 +876,12 @@ pub fn serve_with_policy( } } -/// Serve with full read/write access — the pre-P29 behavior. Every existing -/// caller (stdio transport, sync, tests) uses this unchanged. +/// Serve with full read/write access and default spool caps — the pre-P29 +/// behavior for read-only, now with `WirePolicy::default()`'s bounded +/// `max_pack_size`/`ro_drain_cap` (P31). Every existing caller (stdio +/// transport, sync, tests) uses this unchanged. pub fn serve(root: &std::path::Path, r: &mut impl Read, w: &mut impl Write) -> Result<()> { - serve_with_policy(root, r, w, false) + serve_with_policy(root, r, w, WirePolicy::default()) } /// Destream an incoming pack chunk stream (`ST_PACK_CHUNK`/`ST_PACK_END` @@ -806,14 +889,16 @@ pub fn serve(root: &std::path::Path, r: &mut impl Read, w: &mut impl Write) -> R /// fresh temp pack file, bounded to one chunk in RAM at a time. The guard is /// created before any read, so a stream that errors partway (a malformed /// frame, a dropped connection) still leaves nothing behind — `Drop` removes -/// whatever was written so far. +/// whatever was written so far. `max_bytes` bounds the spool (0 = unlimited, +/// P31) — see [`read_pack_stream`]. fn spill_pack_stream( r: &mut impl Read, layout: &crate::layout::Layout, + max_bytes: u64, ) -> Result { let guard = crate::transport::TempPackGuard::new(layout)?; let mut f = std::fs::File::create(guard.path())?; - read_pack_stream(r, &mut f, 0)?; + read_pack_stream(r, &mut f, max_bytes)?; Ok(guard) } @@ -1217,7 +1302,11 @@ mod tests { let (client_read, mut server_write) = std::io::pipe().unwrap(); let (mut server_read, client_write) = std::io::pipe().unwrap(); let handle = std::thread::spawn(move || { - serve_with_policy(&root, &mut server_read, &mut server_write, read_only) + let policy = WirePolicy { + read_only, + ..Default::default() + }; + serve_with_policy(&root, &mut server_read, &mut server_write, policy) }); let client = crate::stdio_transport::WireClient::handshake(client_read, client_write).unwrap(); @@ -1374,4 +1463,104 @@ mod tests { other => panic!("expected PackTooLarge, got {other:?}"), } } + + /// A PutPack whose chunk stream exceeds policy.max_pack_size gets a + /// best-effort EC_TOO_LARGE reply and the connection closes (desync). + #[test] + fn putpack_over_cap_replies_too_large_and_closes() { + let root = tmp_repo("cap"); + let mut input = Vec::new(); + write_frame( + &mut input, + &Request::Hello { + version: PROTOCOL_VERSION, + } + .encode(), + ) + .unwrap(); + write_frame(&mut input, &Request::PutPack.encode()).unwrap(); + let payload = vec![9u8; 8192]; + write_pack_stream(&mut input, &mut std::io::Cursor::new(&payload), 1024).unwrap(); + // A trailing request that must never be answered (connection closed): + write_frame(&mut input, &Request::HeadBranch.encode()).unwrap(); + + let mut reader = std::io::Cursor::new(input); + let mut output = Vec::new(); + let policy = WirePolicy { + read_only: false, + max_pack_size: 4096, + ro_drain_cap: RO_DRAIN_CAP, + }; + serve_with_policy(&root, &mut reader, &mut output, policy).unwrap(); + + let mut frames = Vec::new(); + let mut r = std::io::Cursor::new(output); + while let Some(f) = read_frame_opt(&mut r).unwrap() { + frames.push(parse_response(f)); + } + assert_eq!( + frames.len(), + 2, + "hello-ok + too-large error only, got {frames:?}" + ); + assert!(frames[0].is_ok()); + assert!(matches!( + frames[1].as_ref().unwrap_err(), + Error::PackTooLarge(_) + )); + // Zero spool residue: + let tmp = root.join(".sc").join("tmp"); + assert!(!tmp.exists() || std::fs::read_dir(&tmp).unwrap().next().is_none()); + let _ = std::fs::remove_dir_all(&root); + } + + #[test] + fn validate_max_pack_size_floor() { + assert!(validate_max_pack_size(0).is_ok()); + assert!(validate_max_pack_size(scl_core::MAX_OBJECT_SIZE as u64).is_ok()); + assert!(validate_max_pack_size(1024).is_err()); + } + + /// An oversized push on a read-only connection drains at most ro_drain_cap + /// bytes, then the connection closes; zero spool residue. + #[test] + fn readonly_oversized_push_is_dropped_after_drain_cap() { + let root = tmp_repo("rocap"); + let mut input = Vec::new(); + write_frame( + &mut input, + &Request::Hello { + version: PROTOCOL_VERSION, + } + .encode(), + ) + .unwrap(); + write_frame(&mut input, &Request::PutPack.encode()).unwrap(); + let payload = vec![9u8; 8192]; + write_pack_stream(&mut input, &mut std::io::Cursor::new(&payload), 1024).unwrap(); + write_frame(&mut input, &Request::HeadBranch.encode()).unwrap(); // must never be answered + + let mut reader = std::io::Cursor::new(input); + let mut output = Vec::new(); + let policy = WirePolicy { + read_only: true, + max_pack_size: 0, + ro_drain_cap: 4096, + }; + serve_with_policy(&root, &mut reader, &mut output, policy).unwrap(); + + let mut frames = Vec::new(); + let mut r = std::io::Cursor::new(output); + while let Some(f) = read_frame_opt(&mut r).unwrap() { + frames.push(parse_response(f)); + } + assert_eq!(frames.len(), 2); + assert!(matches!( + frames[1].as_ref().unwrap_err(), + Error::ReadOnly | Error::Remote(_) + )); + let tmp = root.join(".sc").join("tmp"); + assert!(!tmp.exists() || std::fs::read_dir(&tmp).unwrap().next().is_none()); + let _ = std::fs::remove_dir_all(&root); + } } From 8cc5b7cff752b5ae1424cd247704e2d5c38d449e Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 11:39:50 +0300 Subject: [PATCH 06/14] feat(http): AcceptBackoff + 503 busy status plumbing (P31) Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/repo/src/http_transport.rs | 53 +++++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) diff --git a/crates/repo/src/http_transport.rs b/crates/repo/src/http_transport.rs index 4c86e701..12f824f7 100644 --- a/crates/repo/src/http_transport.rs +++ b/crates/repo/src/http_transport.rs @@ -144,6 +144,7 @@ pub(crate) fn write_status(w: &mut impl Write, code: u16) -> Result<()> { 404 => "Not Found", 400 => "Bad Request", 401 => "Unauthorized", + 503 => "Service Unavailable", _ => { return Err(Error::InvalidArgument(format!( "unsupported HTTP status code: {code}" @@ -323,6 +324,7 @@ impl HttpTransport { )) } 404 => return Err(Error::NotARepo), + 503 => return Err(Error::ServerBusy), other => { return Err(Error::Protocol(format!( "sc+http server returned unexpected status {other}" @@ -383,6 +385,34 @@ impl Transport for HttpTransport { /// where it's cleared before a legitimate large pack transfer begins. const OPENING_READ_TIMEOUT: Duration = Duration::from_secs(30); +/// Exponential accept-error backoff (P31): Go net/http's shape — 5ms +/// doubling to a 1s cap, reset on the next successful accept. Turns fd +/// exhaustion (EMFILE) from a busy-loop into a paced retry. Hardcoded: no +/// operator tuning story exists (Go ships it knobless too). +// TODO(P31 Task 4 step 7): wired into `serve_http_listener`'s accept loop in +// the same task's second commit — `allow(dead_code)` covers the gap between +// the two commits, not a permanent suppression. +#[allow(dead_code)] +struct AcceptBackoff { + cur: Duration, +} +#[allow(dead_code)] +impl AcceptBackoff { + const START: Duration = Duration::from_millis(5); + const CAP: Duration = Duration::from_secs(1); + fn new() -> Self { + Self { cur: Self::START } + } + fn on_error(&mut self) -> Duration { + let d = self.cur; + self.cur = (self.cur * 2).min(Self::CAP); + d + } + fn on_success(&mut self) { + self.cur = Self::START; + } +} + /// Is `host` a loopback address (always safe to bind)? IPv4 `127.0.0.0/8`, /// IPv6 `::1`, or the literal `localhost`. Everything else (`0.0.0.0`, a LAN /// IP, `::`) is non-loopback and subject to the fail-closed bind gate in @@ -1285,4 +1315,27 @@ mod tests { let _ = std::fs::remove_dir_all(&root); } + + // ── Task 4: AcceptBackoff + 503 busy status (pure parts). ── + + #[test] + fn accept_backoff_doubles_and_resets() { + let mut b = AcceptBackoff::new(); + assert_eq!(b.on_error(), Duration::from_millis(5)); + assert_eq!(b.on_error(), Duration::from_millis(10)); + assert_eq!(b.on_error(), Duration::from_millis(20)); + for _ in 0..20 { + b.on_error(); + } + assert_eq!(b.on_error(), Duration::from_secs(1)); // capped + b.on_success(); + assert_eq!(b.on_error(), Duration::from_millis(5)); // reset + } + + #[test] + fn status_503_round_trips() { + let mut buf = Vec::new(); + write_status(&mut buf, 503).unwrap(); + assert_eq!(read_status(&mut &buf[..]).unwrap(), 503); + } } From 0e64fbc78e32cc2be039388ad2635af692169447 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 11:46:22 +0300 Subject: [PATCH 07/14] feat(http): connection cap + 503, session timeouts, accept backoff (P31) Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/cli/src/main.rs | 8 +- crates/repo/src/http_transport.rs | 327 ++++++++++++++++++++++++++++-- 2 files changed, 314 insertions(+), 21 deletions(-) diff --git a/crates/cli/src/main.rs b/crates/cli/src/main.rs index 393d1a97..24822964 100644 --- a/crates/cli/src/main.rs +++ b/crates/cli/src/main.rs @@ -3137,7 +3137,13 @@ fn run_serve( Ok(()) } (false, Some(addr)) => { - scl_repo::http_transport::serve_http(&addr, &path, read_only, allow_public)?; + scl_repo::http_transport::serve_http( + &addr, + &path, + read_only, + allow_public, + scl_repo::http_transport::ServeLimits::default(), + )?; Ok(()) } (true, Some(_)) => anyhow::bail!("sc serve accepts only one of --stdio or --http"), diff --git a/crates/repo/src/http_transport.rs b/crates/repo/src/http_transport.rs index 12f824f7..775c1709 100644 --- a/crates/repo/src/http_transport.rs +++ b/crates/repo/src/http_transport.rs @@ -389,14 +389,9 @@ const OPENING_READ_TIMEOUT: Duration = Duration::from_secs(30); /// doubling to a 1s cap, reset on the next successful accept. Turns fd /// exhaustion (EMFILE) from a busy-loop into a paced retry. Hardcoded: no /// operator tuning story exists (Go ships it knobless too). -// TODO(P31 Task 4 step 7): wired into `serve_http_listener`'s accept loop in -// the same task's second commit — `allow(dead_code)` covers the gap between -// the two commits, not a permanent suppression. -#[allow(dead_code)] struct AcceptBackoff { cur: Duration, } -#[allow(dead_code)] impl AcceptBackoff { const START: Duration = Duration::from_millis(5); const CAP: Duration = Duration::from_secs(1); @@ -413,6 +408,57 @@ impl AcceptBackoff { } } +/// Server-side resource bounds for `sc serve --http` (P31): a hostile or +/// merely slow-and-many-clients peer population must not be able to exhaust +/// server memory, fds, or threads. `0` disables the corresponding limit in +/// every field, matching the rest of this file's `0 = unbounded` convention +/// (e.g. `max_connections`/`ro_drain_cap`). +/// +/// Defaults, each with its own provenance/divergence noted (see the P31 +/// design doc for the full survey): +/// - `max_connections: 32` — matches `git-daemon`'s `--max-connections` +/// default, the closest prior-art analog for a bare TCP git-style server. +/// - `timeout_secs: 300` — on by default, unlike `git-daemon`'s `--timeout` +/// (which defaults to 0/disabled). A silent or write-stalled peer parking +/// a thread forever is exactly the DoS shape this phase closes, so the +/// safer default is chosen deliberately over matching git-daemon here. +/// - `max_pack_size: DEFAULT_MAX_PACK_SIZE` (16 GiB) — also a deliberate +/// divergence: no comparable git-daemon knob exists, so this default is set +/// generously (see Task 3) rather than borrowed from prior art. +#[derive(Debug, Clone, Copy)] +pub struct ServeLimits { + /// Max concurrent connections; beyond this, a new connection is shed with + /// `503 Service Unavailable` before its opening is even read. `0` = + /// unlimited. + pub max_connections: u32, + /// Per-connection idle/session timeout in seconds, applied to both the + /// read and write halves once the session begins (see + /// [`handle_http_connection`]). `0` = disabled. + pub timeout_secs: u64, + /// Forwarded to [`crate::wire::WirePolicy::max_pack_size`]. `0` = + /// unlimited. + pub max_pack_size: u64, +} +impl Default for ServeLimits { + fn default() -> Self { + Self { + max_connections: 32, + timeout_secs: 300, + max_pack_size: crate::wire::DEFAULT_MAX_PACK_SIZE, + } + } +} + +/// Decrements the live-connection count on drop, so every exit path — clean +/// return, error, panic unwind — frees its slot (the TempPackGuard +/// discipline applied to connection slots). +struct SlotGuard(std::sync::Arc); +impl Drop for SlotGuard { + fn drop(&mut self) { + self.0.fetch_sub(1, std::sync::atomic::Ordering::SeqCst); + } +} + /// Is `host` a loopback address (always safe to bind)? IPv4 `127.0.0.0/8`, /// IPv6 `::1`, or the literal `localhost`. Everything else (`0.0.0.0`, a LAN /// IP, `::`) is non-loopback and subject to the fail-closed bind gate in @@ -480,7 +526,9 @@ pub fn serve_http( root: &std::path::Path, read_only: bool, allow_public: bool, + limits: ServeLimits, ) -> Result<()> { + crate::wire::validate_max_pack_size(limits.max_pack_size)?; if !bind_is_allowed(addr, root, read_only, allow_public)? { return Err(Error::InvalidArgument(format!( "refusing to bind non-loopback address {addr} without --read-only, \ @@ -502,7 +550,7 @@ pub fn serve_http( .map_err(|e| Error::ConnectionLost(format!("sc+http local_addr: {e}")))?; println!("listening on {bound}"); std::io::Write::flush(&mut std::io::stdout()).ok(); - serve_http_listener(listener, root, read_only, mandatory_auth) + serve_http_listener(listener, root, read_only, mandatory_auth, limits) } /// Accept-loop core, factored out of [`serve_http`] so tests can bind @@ -519,23 +567,47 @@ pub fn serve_http( /// or a fatal accept-level error occurs; a per-connection error (bad /// opening, a `wire::serve` failure, a dropped socket) is logged to stderr /// and the loop continues — it must never take down the whole server. +/// +/// P31 additions: a live-connection counter enforces `limits.max_connections` +/// (shedding with `503` BEFORE spawning a thread or reading an opening, so a +/// shed connection costs no fd beyond the accept itself); a per-`accept()` +/// error is paced by [`AcceptBackoff`] instead of busy-looping. pub fn serve_http_listener( listener: TcpListener, root: &std::path::Path, read_only: bool, mandatory_auth: bool, + limits: ServeLimits, ) -> Result<()> { + let live = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0)); + let mut backoff = AcceptBackoff::new(); for incoming in listener.incoming() { - let stream = match incoming { - Ok(s) => s, + let mut stream = match incoming { + Ok(s) => { + backoff.on_success(); + s + } Err(e) => { eprintln!("sc serve --http: accept error: {e}"); + std::thread::sleep(backoff.on_error()); continue; } }; + // Connection cap: acquire a slot BEFORE spawning; shed with 503 when full. + let guard = { + let prev = live.fetch_add(1, std::sync::atomic::Ordering::SeqCst); + let g = SlotGuard(live.clone()); + if limits.max_connections != 0 && prev >= limits.max_connections as usize { + let _ = write_status(&mut stream, 503); + continue; // g drops here → count restored; no thread, no handshake + } + g + }; let root = root.to_path_buf(); std::thread::spawn(move || { - if let Err(e) = handle_http_connection(stream, &root, read_only, mandatory_auth) { + let _guard = guard; // slot held for the connection's lifetime + if let Err(e) = handle_http_connection(stream, &root, read_only, mandatory_auth, limits) + { eprintln!("sc serve --http: connection error: {e}"); } }); @@ -548,14 +620,19 @@ pub fn serve_http_listener( /// /// The read timeout is set BEFORE `read_client_opening` (closing the /// slow-loris gap: the opening is bounded in bytes but not time) and -/// cleared again AFTER the 200 status is written and BEFORE handing off to -/// `wire::serve` — a legitimate large pack transfer must not be cut off -/// mid-stream by the same timeout that guards the opening. +/// REPLACED (P31, not simply cleared) by `limits.timeout_secs`'s session +/// timeout — on both the read and write sides — AFTER the 200 status is +/// written and BEFORE handing off to `wire::serve`: a legitimate large pack +/// transfer must not be cut off mid-transfer by the short opening timeout, +/// but a session that goes silent (or, on the write side, stalls reading a +/// streamed response) must still be reaped rather than parking the thread +/// forever. fn handle_http_connection( mut stream: TcpStream, root: &std::path::Path, server_read_only: bool, mandatory_auth: bool, + limits: ServeLimits, ) -> Result<()> { stream .set_read_timeout(Some(OPENING_READ_TIMEOUT)) @@ -621,12 +698,21 @@ fn handle_http_connection( write_status(&mut stream, 200)?; - // Clear the opening's read timeout before the wire protocol begins: a - // real streamed pack transfer can legitimately take longer than - // `OPENING_READ_TIMEOUT` and must not be timed out mid-transfer. + // P31: the opening timeout is REPLACED (not cleared) by the session + // timeout, on BOTH read and write sides — a write timeout covers a client + // that stops reading mid-GetPack. Under P25 chunking a per-syscall timeout + // is progress-based: only true zero-byte stalls trip it. 0 disables. + let session = if limits.timeout_secs == 0 { + None + } else { + Some(Duration::from_secs(limits.timeout_secs)) + }; + stream + .set_read_timeout(session) + .map_err(|e| Error::ConnectionLost(format!("sc+http set session read_timeout: {e}")))?; stream - .set_read_timeout(None) - .map_err(|e| Error::ConnectionLost(format!("sc+http clear read_timeout: {e}")))?; + .set_write_timeout(session) + .map_err(|e| Error::ConnectionLost(format!("sc+http set session write_timeout: {e}")))?; let read_only = server_read_only || token_read_only; crate::wire::serve_with_policy( @@ -635,7 +721,8 @@ fn handle_http_connection( &mut stream, crate::wire::WirePolicy { read_only, - ..Default::default() + max_pack_size: limits.max_pack_size, + ro_drain_cap: crate::wire::RO_DRAIN_CAP, }, ) } @@ -937,7 +1024,14 @@ mod tests { let listener = TcpListener::bind("127.0.0.1:0").unwrap(); let port = listener.local_addr().unwrap().port(); std::thread::spawn(move || { - serve_http_listener(listener, &root, read_only, mandatory_auth).unwrap(); + serve_http_listener( + listener, + &root, + read_only, + mandatory_auth, + ServeLimits::default(), + ) + .unwrap(); }); port } @@ -1096,7 +1190,7 @@ mod tests { // Non-loopback, no --read-only / --allow-public / tokens → refused, // and refused *before* any bind is attempted (port 0 would always // succeed to bind if we got that far). - let err = serve_http("0.0.0.0:0", &root, false, false).unwrap_err(); + let err = serve_http("0.0.0.0:0", &root, false, false, ServeLimits::default()).unwrap_err(); assert!( matches!(err, Error::InvalidArgument(_)), "public bind refused: {err:?}" @@ -1338,4 +1432,197 @@ mod tests { write_status(&mut buf, 503).unwrap(); assert_eq!(read_status(&mut &buf[..]).unwrap(), 503); } + + // ── Task 4: connection slots + 503, session timeouts (socket-level). ── + + fn tmp_served_root(tag: &str) -> std::path::PathBuf { + let root = std::env::temp_dir().join(format!("scl-http-{tag}-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&root); + std::fs::create_dir_all(root.join(".sc")).unwrap(); + root + } + + fn spawn_listener(root: &std::path::Path, limits: ServeLimits) -> std::net::SocketAddr { + let listener = TcpListener::bind("127.0.0.1:0").unwrap(); + let addr = listener.local_addr().unwrap(); + let root = root.to_path_buf(); + std::thread::spawn(move || { + let _ = serve_http_listener(listener, &root, false, false, limits); + }); + addr + } + + fn open_ok(addr: std::net::SocketAddr) -> TcpStream { + let mut s = TcpStream::connect(addr).unwrap(); + write_client_opening(&mut s, "127.0.0.1", "/", None).unwrap(); + assert_eq!(read_status(&mut s).unwrap(), 200); + s + } + + #[test] + fn connection_limit_shed_and_recover() { + let root = tmp_served_root("slots"); + let limits = ServeLimits { + max_connections: 1, + timeout_secs: 0, + ..Default::default() + }; + let addr = spawn_listener(&root, limits); + + let held = open_ok(addr); // occupies the single slot + + // Second connection: shed with 503 before any opening is read. + let mut second = TcpStream::connect(addr).unwrap(); + assert_eq!(read_status(&mut second).unwrap(), 503); + + // Free the slot; a fresh connection now succeeds (poll: the server + // notices the disconnect on its next read). + drop(held); + let ok = (0..50).any(|_| { + std::thread::sleep(Duration::from_millis(100)); + let mut s = match TcpStream::connect(addr) { + Ok(s) => s, + Err(_) => return false, + }; + write_client_opening(&mut s, "127.0.0.1", "/", None).is_ok() + && read_status(&mut s).ok() == Some(200) + }); + assert!(ok, "slot was never freed"); + let _ = std::fs::remove_dir_all(&root); + } + + #[test] + fn errored_connection_frees_its_slot() { + let root = tmp_served_root("errslot"); + let limits = ServeLimits { + max_connections: 1, + timeout_secs: 0, + ..Default::default() + }; + let addr = spawn_listener(&root, limits); + + // A connection whose handler errors (garbage opening → 400) must free + // its slot via the guard, not leak it. + let mut bad = TcpStream::connect(addr).unwrap(); + bad.write_all(b"garbage\r\n\r\n").unwrap(); + let _ = read_status(&mut bad); // 400 (best-effort) + drop(bad); + + let ok = (0..50).any(|_| { + std::thread::sleep(Duration::from_millis(100)); + let mut s = match TcpStream::connect(addr) { + Ok(s) => s, + Err(_) => return false, + }; + write_client_opening(&mut s, "127.0.0.1", "/", None).is_ok() + && read_status(&mut s).ok() == Some(200) + }); + assert!(ok, "errored connection leaked its slot"); + let _ = std::fs::remove_dir_all(&root); + } + + /// A client that stops READING mid-GetPack must be reaped by the write + /// timeout (today it parks the server thread in a blocking write forever). + /// Server sends a pack big enough to overflow socket buffers; the client + /// never reads; after the timeout the server closes. + #[test] + fn write_side_stall_is_reaped_by_timeout() { + let root = std::env::temp_dir().join(format!("scl-http-wstall-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&root); + std::fs::create_dir_all(&root).unwrap(); + crate::repo::Repo::init(&root).unwrap(); + // A few MiB of committed content, so GetPack has real bytes to send — + // enough (and high-entropy enough to resist the pack's zstd + // compression) to overflow socket send/receive buffers and force the + // server's write to actually block once the client stops reading. + // A forced newline every 8 bytes keeps every "line" well under the + // scanner's B64_MIN_RUN (20 chars), so this pseudo-random content + // never trips the P5 commit-time secret scan. + let mut big = Vec::with_capacity(4 * 1024 * 1024); + let mut state: u64 = 0x243F_6A88_85A3_08D3; + while big.len() < 4 * 1024 * 1024 { + state ^= state << 13; + state ^= state >> 7; + state ^= state << 17; + big.extend_from_slice(&state.to_le_bytes()); + big.push(b'\n'); + } + std::fs::write(root.join("big.bin"), &big).unwrap(); + let repo = crate::repo::Repo::open(&root).unwrap(); + let tip = repo.commit("t", "big file").unwrap(); + drop(repo); + + let limits = ServeLimits { + max_connections: 0, + timeout_secs: 1, + ..Default::default() + }; + let addr = spawn_listener(&root, limits); + + let mut s = open_ok(addr); + // Speak just enough wire protocol to trigger a GetPack, then stop + // reading: HELLO handshake, then a GetPack for the branch tip with + // empty haves. + crate::wire::write_frame( + &mut s, + &crate::wire::Request::Hello { + version: crate::wire::PROTOCOL_VERSION, + } + .encode(), + ) + .unwrap(); + // Read (and discard) the HELLO reply frame so the request stream + // stays in sync before the GetPack request. + let _ = crate::wire::read_frame(&mut s).unwrap(); + crate::wire::write_frame( + &mut s, + &crate::wire::Request::GetPack { + wants: vec![tip], + haves: vec![], + filter: vec![], + } + .encode(), + ) + .unwrap(); + // After sending the GetPack request, sleep instead of reading — + // never drain the response, so the server's write eventually blocks. + std::thread::sleep(Duration::from_secs(5)); + // The server must have dropped the connection rather than blocking + // forever: our next read eventually sees EOF or a reset. + s.set_read_timeout(Some(Duration::from_secs(5))).unwrap(); + let mut sink = [0u8; 4096]; + loop { + match s.read(&mut sink) { + Ok(0) | Err(_) => break, // EOF or reset — server gave up: pass + Ok(_) => continue, // drain what was already buffered + } + } + let _ = std::fs::remove_dir_all(&root); + } + + #[test] + fn silent_session_is_reaped_by_timeout() { + let root = tmp_served_root("reap"); + let limits = ServeLimits { + max_connections: 0, + timeout_secs: 1, + ..Default::default() + }; + let addr = spawn_listener(&root, limits); + + let mut s = open_ok(addr); // handshake never sent; go silent + std::thread::sleep(Duration::from_secs(3)); + // Server must have dropped us: the read sees EOF/reset, not a hang. + s.set_read_timeout(Some(Duration::from_secs(5))).unwrap(); + let mut buf = [0u8; 1]; + match s.read(&mut buf) { + Ok(0) => {} // clean EOF — dropped + Err(_) => {} // reset — also dropped + Ok(_) => panic!("server sent data to a silent client"), + } + // Spool dir is clean. + let tmp = root.join(".sc").join("tmp"); + assert!(!tmp.exists() || std::fs::read_dir(&tmp).unwrap().next().is_none()); + let _ = std::fs::remove_dir_all(&root); + } } From 77d5f78ec06edcc18d34fc23939faa8524728567 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 11:56:03 +0300 Subject: [PATCH 08/14] fix(http): document write-stall test's two reap paths + residue assert; fix stale timeout doc (P31 review) Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/repo/src/http_transport.rs | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/crates/repo/src/http_transport.rs b/crates/repo/src/http_transport.rs index 775c1709..3d9cd77e 100644 --- a/crates/repo/src/http_transport.rs +++ b/crates/repo/src/http_transport.rs @@ -381,8 +381,10 @@ impl Transport for HttpTransport { /// `read_client_opening` bounds the opening in BYTES (`MAX_OPENING_BYTES`) /// but not in TIME, so a peer that trickles in under the byte cap and then /// stalls would otherwise hold a server thread (and its socket) forever. -/// Applied only around the opening read — see [`serve_http_listener`] for -/// where it's cleared before a legitimate large pack transfer begins. +/// Applied only around the opening read — the opening timeout is REPLACED +/// (not cleared) by `limits.timeout_secs`'s session read+write timeouts +/// after the 200 status is written, before handing off to `wire::serve` — +/// see [`handle_http_connection`] for the timeout sequencing. const OPENING_READ_TIMEOUT: Duration = Duration::from_secs(30); /// Exponential accept-error backoff (P31): Go net/http's shape — 5ms @@ -1525,6 +1527,15 @@ mod tests { /// timeout (today it parks the server thread in a blocking write forever). /// Server sends a pack big enough to overflow socket buffers; the client /// never reads; after the timeout the server closes. + /// + /// Why it cannot flake: the server must reap the connection within ~1s + /// through ONE OF TWO PATHS. (a) If the ~4 MiB GetPack response overflows + /// the loopback socket buffers, the server blocks in `write` and the 1s + /// WRITE timeout fires. (b) If the kernel buffers absorb the whole + /// response, the server finishes writing, returns to its request loop, + /// blocks in `read_frame_opt` on the silent client, and the 1s READ + /// timeout fires. Either way the client's drain loop observes EOF/reset + /// well inside its 5s margins. #[test] fn write_side_stall_is_reaped_by_timeout() { let root = std::env::temp_dir().join(format!("scl-http-wstall-{}", std::process::id())); @@ -1597,6 +1608,9 @@ mod tests { Ok(_) => continue, // drain what was already buffered } } + // Spool dir is clean. + let tmp = root.join(".sc").join("tmp"); + assert!(!tmp.exists() || std::fs::read_dir(&tmp).unwrap().next().is_none()); let _ = std::fs::remove_dir_all(&root); } From a8026a3eedbcca69fa1ccc72fe71d54e3af0910d Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 12:02:51 +0300 Subject: [PATCH 09/14] feat(cli): sc serve --max-connections/--timeout/--max-pack-size (P31) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add three new flags to Cmd::Serve: --max-connections (--http only), --timeout (--http only), --max-pack-size (both transports). - CLI dispatch routes flags to run_serve; stdio rejects http-only flags with clear errors; http builds ServeLimits from flags, defaults from ServeLimits::default(); stdio validates and applies --max-pack-size via wire::WirePolicy with serve_with_policy (not legacy serve). - Flags resolve per brief §Step 2: max_pack_size defaults to DEFAULT_MAX_PACK_SIZE, validated through validate_max_pack_size, wired into WirePolicy.read_only=false for stdio and custom limits for --http. Both manual verification commands pass (--timeout rejects on stdio, --max-pack-size=1024 fails floor check). Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- crates/cli/src/main.rs | 65 +++++++++++++++++++++++++++++++++++------- 1 file changed, 55 insertions(+), 10 deletions(-) diff --git a/crates/cli/src/main.rs b/crates/cli/src/main.rs index 24822964..159f97f6 100644 --- a/crates/cli/src/main.rs +++ b/crates/cli/src/main.rs @@ -298,6 +298,21 @@ enum Cmd { /// configured token, binding a non-loopback address is refused. #[arg(long)] allow_public: bool, + /// Maximum concurrent connections (`--http` only); 0 = unlimited. + /// Default 32 (git-daemon parity). At the limit new connections get an + /// immediate busy status and close. (P31) + #[arg(long)] + max_connections: Option, + /// Session idle timeout in seconds (`--http` only); 0 = disabled. + /// Default 300. A connection making zero-byte progress for this long is + /// dropped. (P31) + #[arg(long)] + timeout: Option, + /// Maximum incoming pack size in bytes (both --http and --stdio); + /// 0 = unlimited. Default 16 GiB. Must be ≥ 256 MiB (MAX_OBJECT_SIZE) + /// when non-zero. (P31) + #[arg(long)] + max_pack_size: Option, /// Repo root to serve (the directory containing `.sc/`). Required /// unless a `token` subcommand is used. path: Option, @@ -816,12 +831,24 @@ fn main() -> Result<()> { http, read_only, allow_public, + max_connections, + timeout, + max_pack_size, path, } => match sub { Some(ServeSub::Token { op }) => run_serve_token(op), None => { let path = path.ok_or_else(|| anyhow::anyhow!("sc serve requires a "))?; - run_serve(stdio, http, read_only, allow_public, path) + run_serve( + stdio, + http, + read_only, + allow_public, + max_connections, + timeout, + max_pack_size, + path, + ) } }, Cmd::Remote { op } => run_remote(op), @@ -3118,12 +3145,16 @@ fn run_clone_git(url: &str, dst: &std::path::Path) -> Result<()> { /// dropped, e.g. by process termination). Exactly one mode is required. /// `--read-only`/`--allow-public` are `--http`-only (P29): `--stdio` /// delegates auth/access entirely to ssh, so combining them is refused -/// rather than silently ignored. +/// rather than silently ignored. `--max-connections`/`--timeout` are also +/// `--http`-only (P31). `--max-pack-size` applies to both (P31). fn run_serve( stdio: bool, http: Option, read_only: bool, allow_public: bool, + max_connections: Option, + timeout: Option, + max_pack_size: Option, path: PathBuf, ) -> Result<()> { match (stdio, http) { @@ -3131,19 +3162,33 @@ fn run_serve( if read_only || allow_public { anyhow::bail!("--read-only/--allow-public apply only to --http, not --stdio"); } + if max_connections.is_some() || timeout.is_some() { + anyhow::bail!("--max-connections/--timeout apply only to --http, not --stdio"); + } + let max_pack = max_pack_size.unwrap_or(scl_repo::wire::DEFAULT_MAX_PACK_SIZE); + scl_repo::wire::validate_max_pack_size(max_pack)?; + let policy = scl_repo::wire::WirePolicy { + read_only: false, + max_pack_size: max_pack, + ro_drain_cap: scl_repo::wire::RO_DRAIN_CAP, + }; let mut stdin = std::io::stdin().lock(); let mut stdout = std::io::stdout().lock(); - scl_repo::wire::serve(&path, &mut stdin, &mut stdout)?; + scl_repo::wire::serve_with_policy(&path, &mut stdin, &mut stdout, policy)?; Ok(()) } (false, Some(addr)) => { - scl_repo::http_transport::serve_http( - &addr, - &path, - read_only, - allow_public, - scl_repo::http_transport::ServeLimits::default(), - )?; + let mut limits = scl_repo::http_transport::ServeLimits::default(); + if let Some(n) = max_connections { + limits.max_connections = n; + } + if let Some(t) = timeout { + limits.timeout_secs = t; + } + if let Some(m) = max_pack_size { + limits.max_pack_size = m; + } + scl_repo::http_transport::serve_http(&addr, &path, read_only, allow_public, limits)?; Ok(()) } (true, Some(_)) => anyhow::bail!("sc serve accepts only one of --stdio or --http"), From 724f190d0c83ad656e300291e937c5d4f5549a1e Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 12:12:22 +0300 Subject: [PATCH 10/14] =?UTF-8?q?demo:=20listener=20limits=20proof=20?= =?UTF-8?q?=E2=80=94=20busy=20shed,=20slot=20reap,=20cap=20knobs=20(P31)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Proves sc serve --http's --max-connections/--timeout/--max-pack-size knobs end to end: a raw held TCP connection (exec 3<>/dev/tcp) occupies the single slot and a second connection is shed with 503/"server busy" (a deterministic raw-socket probe plus a best-effort real `sc clone`, which occasionally observes a benign connection-reset instead of a clean 503 due to a TCP quirk of the shed path — both outcomes are accepted and explained inline); closing the holder frees the slot for a fresh clone. --timeout's post-opening session-idle reap can't be cheaply driven without a wire client, so the script says so and points at the existing unit tests instead of faking coverage. --max-pack-size at the 256 MiB floor serves normally while a value below the floor refuses to start. Zero .sc/tmp residue asserted; runs clean twice in a row. Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- demo/run_limits_demo.sh | 187 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 187 insertions(+) create mode 100755 demo/run_limits_demo.sh diff --git a/demo/run_limits_demo.sh b/demo/run_limits_demo.sh new file mode 100755 index 00000000..1cc16137 --- /dev/null +++ b/demo/run_limits_demo.sh @@ -0,0 +1,187 @@ +#!/usr/bin/env bash +# P31 listener-limits demo: `sc serve --http`'s three new operator knobs — +# `--max-connections` (503-style busy shed at the limit), `--timeout` +# (session idle read+write timeout), and `--max-pack-size` (floor: +# MAX_OBJECT_SIZE, 256 MiB) — layered on P26/P29's `sc serve --http`. +# +# Honesty note on scope: a connection that never sends its opening is +# reaped by the FIXED 30s opening timeout, not by `--timeout` (which +# governs the post-opening session). This script cannot cheaply hold a +# post-opening session silent without a wire client, so it proves +# `--timeout` end-to-end indirectly: it frees a busy slot by closing the +# holder's fd rather than waiting out any timer, and says so explicitly. +# The 30s opening-reap path and the mid-stream pack-size abort are both +# covered by unit tests in crates/repo/src/http_transport.rs — this script +# proves the CLI knobs parse, take effect, and the busy/free lifecycle +# behaves end to end. +# +# Self-checking: every claim is an assertion; any failure exits non-zero +# before the RESULT line. +set -euo pipefail + +cd "$(dirname "$0")/.." +cargo build --quiet --bin sc +SC="$PWD/target/debug/sc" + +fail() { echo "FAIL: $1"; exit 1; } + +W="$(mktemp -d)" +A="$W/A" +SERVER_PID="" + +cleanup() { + if [ -n "$SERVER_PID" ] && kill -0 "$SERVER_PID" 2>/dev/null; then + kill "$SERVER_PID" 2>/dev/null || true + wait "$SERVER_PID" 2>/dev/null || true + fi + # Close the held raw connection if it's still open. + exec 3<&- 2>/dev/null || true + rm -rf "$W" +} +trap cleanup EXIT + +pick_port() { + local candidate + for candidate in 8751 8752 8753 8911 18751 18752; do + if ! nc -z 127.0.0.1 "$candidate" 2>/dev/null; then + echo "$candidate" + return 0 + fi + done + fail "no free port found among the candidates" +} + +wait_for_port() { + local port="$1" tries=0 + while ! nc -z 127.0.0.1 "$port" 2>/dev/null; do + tries=$((tries + 1)) + [ "$tries" -lt 100 ] || fail "server on port $port never became ready" + sleep 0.05 + done +} + +check_no_tmp_residue() { + local dir="$1" label="$2" + [ -d "$dir/.sc/tmp" ] && [ -n "$(ls -A "$dir/.sc/tmp" 2>/dev/null)" ] \ + && fail "$label: .sc/tmp is not empty" + return 0 +} + +echo "=== 1: init origin A, one commit ===" +mkdir -p "$A"; cd "$A" +"$SC" init >/dev/null +printf 'v1\n' > tracked.txt +"$SC" commit -m "initial" --author origin >/dev/null +echo "A: initial commit ✔" + +echo +echo "=== 2: sc serve --http 127.0.0.1:PORT --max-connections 1 --timeout 2 --max-pack-size 268435456 (background) ===" +cd "$W" +PORT="$(pick_port)" +"$SC" serve --http "127.0.0.1:$PORT" --max-connections 1 --timeout 2 \ + --max-pack-size 268435456 "$A" & +SERVER_PID=$! +wait_for_port "$PORT" +echo "server: listening on 127.0.0.1:$PORT (pid $SERVER_PID), max-connections=1 ✔" + +URL="sc+http://127.0.0.1:$PORT/repo" + +echo +echo "=== 3: busy shed — hold the one connection slot, a second clone is rejected ===" +# exec opens a raw TCP connection and holds it; the slot is taken at +# accept() time, before the server ever reads an opening from it. +exec 3<>"/dev/tcp/127.0.0.1/$PORT" +# The shed path writes the 503 status and closes without ever reading the +# probing client's opening bytes. +# +# Primary, deterministic proof: a bare second TCP connection that (like the +# held one) sends nothing. The shed path never reads from it — it writes the +# 503 status and closes cleanly — so this probe is not subject to any +# reset race and always observes a clean "503 Service Unavailable". +exec 4<>"/dev/tcp/127.0.0.1/$PORT" +PROBE_RESP="$(cat <&4 2>/dev/null || true)" +exec 4<&- +echo "$PROBE_RESP" | grep -q "503 Service Unavailable" \ + || fail "expected a 503 Service Unavailable status on the second connection while the slot is held, got: $PROBE_RESP" +echo "busy: raw second connection shed with '503 Service Unavailable' while the slot is held ✔" + +# Secondary, best-effort proof with the real client: `sc clone` writes its +# opening before reading the status, so the server's close-with-unread-data +# on the shed path occasionally races that clean 503 read into a raw +# "connection reset" instead (a benign TCP quirk of the shed path, not a +# wire-protocol bug — either outcome still proves the connection was +# rejected while the slot was held, since a free slot always completes a +# clean clone, as step 4 demonstrates). Accept either textual outcome. +if "$SC" clone "$URL" "$W/c-busy" 2>"$W/err-busy.txt"; then + fail "clone should have been rejected while the one slot is held" +fi +[ ! -e "$W/c-busy" ] || fail "rejected clone should not have created a destination" +grep -qi "server busy\|connection reset\|connection lost" "$W/err-busy.txt" \ + || fail "expected 'server busy' or a benign reset while the slot is held, got: $(cat "$W/err-busy.txt")" +echo "busy: sc clone also rejected while the slot is held ($(cat "$W/err-busy.txt")) ✔" + +echo +echo "=== 4: slot auto-freed — closing the held connection frees it for a fresh clone ===" +# --timeout governs a SILENT connection AFTER its opening; a raw held fd +# that never sends an opening is instead reaped by the fixed 30s opening +# timeout (not this flag) -- too slow to wait out in a demo. We prove the +# busy/free lifecycle end to end by closing the holder ourselves: the slot +# frees immediately and the next clone succeeds. (The 30s opening-timeout +# reap path itself is proven by +# crates/repo/src/http_transport.rs::silent_session_is_reaped_by_timeout +# and its opening-phase sibling, not by this script.) +exec 3<&- +"$SC" clone "$URL" "$W/c-free" >/dev/null +[ "$(cat "$W/c-free/tracked.txt")" = "v1" ] || fail "c-free: clone content mismatch" +echo "freed: slot released once the holder disconnects; clone now succeeds ✔" +echo "(--timeout's post-opening session-idle reap is covered by the http_transport unit tests, not this script)" + +echo +echo "=== 5: stop the limited server ===" +kill "$SERVER_PID" +wait "$SERVER_PID" 2>/dev/null || true +SERVER_PID="" +! nc -z 127.0.0.1 "$PORT" 2>/dev/null || fail "server still accepting connections after kill" +echo "server: stopped ✔" + +echo +echo "=== 6: --max-pack-size at the floor is accepted; below the floor refuses to start ===" +PORT2="$(pick_port)" +"$SC" serve --http "127.0.0.1:$PORT2" --max-pack-size 268435456 "$A" & +SERVER_PID=$! +wait_for_port "$PORT2" +echo "server: started with --max-pack-size at the floor (268435456) ✔" + +URL2="sc+http://127.0.0.1:$PORT2/repo" +"$SC" clone "$URL2" "$W/c-floor" >/dev/null +[ "$(cat "$W/c-floor/tracked.txt")" = "v1" ] || fail "c-floor: clone content mismatch" +echo "clone through a floor-capped server: succeeds (cap present, harmless for a small repo) ✔" + +kill "$SERVER_PID" +wait "$SERVER_PID" 2>/dev/null || true +SERVER_PID="" +! nc -z 127.0.0.1 "$PORT2" 2>/dev/null || fail "second server still accepting connections after kill" + +if "$SC" serve --http "127.0.0.1:$PORT2" --max-pack-size 1024 "$A" 2>"$W/err-cap.txt"; then + fail "server should have refused to start with --max-pack-size below MAX_OBJECT_SIZE" +fi +grep -qi "max-pack-size" "$W/err-cap.txt" \ + || fail "expected a --max-pack-size floor error, got: $(cat "$W/err-cap.txt")" +! nc -z 127.0.0.1 "$PORT2" 2>/dev/null || fail "server bound despite refusing to start" +echo "server: --max-pack-size 1024 (below the 256 MiB floor) refused to start ✔" +echo "(the mid-stream pack-size abort itself is covered by unit tests, not this script)" + +echo +echo "=== 7: zero .sc/tmp residue on the served repo and every clone ===" +check_no_tmp_residue "$A" "origin A" +check_no_tmp_residue "$W/c-free" "c-free" +check_no_tmp_residue "$W/c-floor" "c-floor" +echo "origin + clones: zero .sc/tmp residue ✔" + +echo +echo "RESULT: sc serve --http listener limits — a connection at --max-connections 1" +echo "is shed with a 'server busy' error while a slot is held, the slot frees once" +echo "the holder disconnects and a fresh clone succeeds (the post-opening" +echo "--timeout reap itself is unit-tested, not demoed here), --max-pack-size at" +echo "the 256 MiB floor serves normally while a value below the floor refuses to" +echo "start the server, and zero .sc/tmp residue is left anywhere. OK" From f60ed505cd46c40b7b0e9c7c5b10640ae3600b06 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 12:19:37 +0300 Subject: [PATCH 11/14] fix(demo): prove the floor cap on the gated push path (P31 review) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review Important: --max-pack-size gates only the PutPack spool (the server's receive path); a clone's GetPack never consults it, so the earlier clone-through-the-capped-server step passed even if the cap's enforcement broke. Step 6 now commits in the capped-server clone and pushes back through sc+http:// — the small pack exercises the gated spill path end to end, verified by a fresh clone seeing the pushed content — and the RESULT line claims exactly that ("a small push lands through the PutPack spool under the 256 MiB floor cap") instead of "serves normally". Also (review Minor) the raw 503 probe read is now bounded (read -r -t 10) so a regressed accept-but-stall server fails the demo within 10s instead of hanging until the 30s opening reap. Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- demo/run_limits_demo.sh | 34 ++++++++++++++++++++++++++++------ 1 file changed, 28 insertions(+), 6 deletions(-) diff --git a/demo/run_limits_demo.sh b/demo/run_limits_demo.sh index 1cc16137..bde0259e 100755 --- a/demo/run_limits_demo.sh +++ b/demo/run_limits_demo.sh @@ -99,7 +99,11 @@ exec 3<>"/dev/tcp/127.0.0.1/$PORT" # 503 status and closes cleanly — so this probe is not subject to any # reset race and always observes a clean "503 Service Unavailable". exec 4<>"/dev/tcp/127.0.0.1/$PORT" -PROBE_RESP="$(cat <&4 2>/dev/null || true)" +# Bounded read (read -t), not a bare `cat`: a regressed server that accepts +# but stalls instead of writing the 503 must fail this step within 10s, not +# hang until the 30s opening reap kicks it loose. +PROBE_RESP="" +IFS= read -r -t 10 PROBE_RESP <&4 || true exec 4<&- echo "$PROBE_RESP" | grep -q "503 Service Unavailable" \ || fail "expected a 503 Service Unavailable status on the second connection while the slot is held, got: $PROBE_RESP" @@ -145,7 +149,7 @@ SERVER_PID="" echo "server: stopped ✔" echo -echo "=== 6: --max-pack-size at the floor is accepted; below the floor refuses to start ===" +echo "=== 6: --max-pack-size floor — a small push lands under the cap; below the floor refuses to start ===" PORT2="$(pick_port)" "$SC" serve --http "127.0.0.1:$PORT2" --max-pack-size 268435456 "$A" & SERVER_PID=$! @@ -155,7 +159,23 @@ echo "server: started with --max-pack-size at the floor (268435456) ✔" URL2="sc+http://127.0.0.1:$PORT2/repo" "$SC" clone "$URL2" "$W/c-floor" >/dev/null [ "$(cat "$W/c-floor/tracked.txt")" = "v1" ] || fail "c-floor: clone content mismatch" -echo "clone through a floor-capped server: succeeds (cap present, harmless for a small repo) ✔" +echo "clone through a floor-capped server: succeeds ✔" + +# The cap gates the PutPack spool (the server's receive path) — a clone's +# GetPack never consults it. Exercise the actually-gated path: a small +# commit pushed THROUGH the capped server must land (its pack is far under +# the 256 MiB floor cap). If the cap's enforcement broke open-endedly or +# regressed to rejecting everything, this push is the step that notices. +cd "$W/c-floor" +printf 'v2 via capped push\n' > pushed.txt +"$SC" commit -m "push through the floor-capped server" --author capclient >/dev/null +"$SC" push origin >/dev/null \ + || fail "small push through the floor-capped server should have landed" +cd "$W" +"$SC" clone "$URL2" "$W/c-cap-verify" >/dev/null +[ "$(cat "$W/c-cap-verify/pushed.txt")" = "v2 via capped push" ] \ + || fail "c-cap-verify: pushed commit did not land through the capped server" +echo "small push through the floor-capped server: lands (PutPack spool path, under the cap) ✔" kill "$SERVER_PID" wait "$SERVER_PID" 2>/dev/null || true @@ -176,12 +196,14 @@ echo "=== 7: zero .sc/tmp residue on the served repo and every clone ===" check_no_tmp_residue "$A" "origin A" check_no_tmp_residue "$W/c-free" "c-free" check_no_tmp_residue "$W/c-floor" "c-floor" +check_no_tmp_residue "$W/c-cap-verify" "c-cap-verify" echo "origin + clones: zero .sc/tmp residue ✔" echo echo "RESULT: sc serve --http listener limits — a connection at --max-connections 1" echo "is shed with a 'server busy' error while a slot is held, the slot frees once" echo "the holder disconnects and a fresh clone succeeds (the post-opening" -echo "--timeout reap itself is unit-tested, not demoed here), --max-pack-size at" -echo "the 256 MiB floor serves normally while a value below the floor refuses to" -echo "start the server, and zero .sc/tmp residue is left anywhere. OK" +echo "--timeout reap itself is unit-tested, not demoed here), a small push lands" +echo "through the PutPack spool under the 256 MiB floor cap while a --max-pack-size" +echo "below the floor refuses to start the server, and zero .sc/tmp residue is" +echo "left anywhere. OK" From 34724fcd94e54e8db432f5ee8602d27ea2024d74 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 12:25:46 +0300 Subject: [PATCH 12/14] docs: ADR-0041 listener limits + THREAT-MODEL closures and re-affirmations (P31) Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- CLAUDE.md | 68 +++++++-- docs/THREAT-MODEL.md | 71 ++++++++- docs/adr/0041-listener-resource-limits.md | 169 ++++++++++++++++++++++ docs/adr/README.md | 1 + 4 files changed, 294 insertions(+), 15 deletions(-) create mode 100644 docs/adr/0041-listener-resource-limits.md diff --git a/CLAUDE.md b/CLAUDE.md index 42b52cdf..4cbcca44 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -280,12 +280,26 @@ bash demo/run_streaming_demo.sh # streaming pack transfer proof (P # in 250+ chunk frames, byte-for-byte clone, zero # .sc/tmp residue on either end cargo run --bin sc -- serve --http [--read-only] [--allow-public] + [--max-connections ] [--timeout ] [--max-pack-size ] # sc-native wire protocol over TCP (P26); e.g. # `sc serve --http 127.0.0.1:8730 .`; exactly one # of --stdio/--http is required; a non-loopback # is refused unless --read-only, # --allow-public, or a configured serve token # justifies it (P29) + # --max-connections : concurrent-connection cap, + # --http only, default 32, 0=unlimited; at the limit + # a new connection gets busy-status-and-close, no + # queuing (P31) + # --timeout : session idle/progress timeout, + # --http only, default 300, 0=disabled; read+write, + # persists the whole session, connection-fatal on + # trip; opening keeps its own 30s (P31) + # --max-pack-size : incoming-pack aggregate + # cap, both --http and --stdio, default 16 GiB, + # 0=unlimited, floor 256 MiB (MAX_OBJECT_SIZE); + # counted mid-stream abort -> EC_TOO_LARGE (P31) + # justifies it (P29) cargo run --bin sc -- serve token add --label --scope ro|rw # mint an # sct- bearer token in .sc/serve-tokens.toml; # raw value prints once on stdout (P29) @@ -996,13 +1010,13 @@ serve --http` is unauthenticated, as `--stdio` delegates auth to ssh — production auth + TLS means fronting with a reverse proxy; this is the reach primitive, not a hosted-git competitor); **not HTTP-proxy/CDN safe** (a strict proxy won't tunnel the post-opening raw protocol, the accepted -cost of the persistent-connection model). **Accepted design consequences, -not yet closed (deferred, see ROADMAP):** `serve_http_listener` is -unbounded thread-per-connection with no pool/backpressure; the opening -timeout is cleared once `wire::serve` takes over, so a client that stalls -mid-transfer holds its thread indefinitely (no idle-transfer watchdog -yet); the accept loop has no backoff on sustained fd exhaustion. See -ADR-0036. +cost of the persistent-connection model). **Accepted design consequences at +the time — closed by P31 (ADR-0041):** `serve_http_listener`'s unbounded +thread-per-connection (no pool/backpressure), the opening timeout being +cleared once `wire::serve` takes over (no idle-transfer watchdog), and the +accept loop's lack of backoff on sustained fd exhaustion are all closed by +P31's `--max-connections`, `--timeout`, and hardcoded accept backoff, +respectively — see the P31 section below. See ADR-0036. **Phase 27 is built.** The P25–P27 scale-&-reach horizon's capstone: partial clone. `.sc/promisor` (`crates/repo/src/promisor.rs`, local, @@ -1242,12 +1256,46 @@ bytes, the `sc log` marker, and `sc gc` pruning a deleted branch's transcript while the still-reachable one from earlier survives — run twice, zero residue. No new dependency. See ADR-0038. +**Phase 31 is built.** Listener resource limits close ADR-0036's three +named-but-open accepted consequences plus a fourth this phase's own +research pass (`docs/research/bounded-server-patterns.md`, ticket #27) +found and named for the first time: the aggregate incoming-pack spool was +unbounded on both transports, including via the read-only rejection path. +`sc serve --http` gains `--max-connections ` (default 32, matching +`git-daemon`, 0 = unlimited; at the limit a new connection gets an +immediate busy status at the pre-handshake opening seam and closes — no +queuing), `--timeout ` (default 300, 0 = disables; read+write +timeouts now persist for the whole session rather than being cleared after +the opening, and a trip is connection-fatal since mid-stream abort desyncs +the frame stream; the opening keeps its own unchanged 30s), and +`--max-pack-size ` (default 16 GiB, 0 = unlimited, floor 256 MiB = +`MAX_OBJECT_SIZE`; applies to **both** `--http` and `--stdio` since it +lives in the shared `WirePolicy`, not the `--http`-only `ServeLimits`; a +counted mid-stream abort replies `EC_TOO_LARGE` best-effort before closing). +The read-only pre-drain (`wire.rs`'s `EC_READONLY` arm, which drains an +incoming pack before replying so the connection stays in sync) is now +capped at 8 MiB (`RO_DRAIN_CAP`): an honest small misconfigured push still +gets the clean typed error, a larger bulk spool is dropped mid-send instead +of fully written to disk. The accept loop gained a hardcoded Go-`net/http`- +shaped exponential backoff (5ms doubling to a 1s cap, reset on success) — +no operator knob, matching Go's own precedent. Two recorded divergences +from prior art: the timeout ships **on** by default, unlike `git-daemon`'s +off-by-default `--timeout` (a defaults-off knob doesn't close an +audit-tracked High); the pack-spool default is **finite** (16 GiB), unlike +git's unlimited `receive.maxInputSize` (sc servers often share the working +tree's volume). `PROTOCOL_VERSION` stays 3 — none of the four bounds touch +the wire protocol, and `EC_TOO_LARGE` degrades to opaque text on an old +client, same as `EC_READONLY`'s precedent. Zero new dependencies. Proven by +`demo/run_limits_demo.sh` (the floor-cap enforcement on the push path and +the busy-status shed under `--max-connections`); the session-timeout reaper +and accept-backoff pacing are unit-test-proven rather than demo-proven, +since forcing a real stalled socket or fd exhaustion reliably in a demo +script is impractical. See ADR-0041. + Remaining follow-ons: operation objects in the CAS, oplog entries for remote-tracking refs, extending the sparse gate to `materialize_conflict_state`'s `to_encrypt`/sidecar write loops (see the -P24 boundary note above), the three P26 `sc serve --http` hardening items -named above (connection pool/backpressure, idle-transfer watchdog, -accept-loop backoff), the three P27 items named above (transparent +P24 boundary note above), the three P27 items named above (transparent lazy-fetch as a deferred alternative to explicit `sc backfill`, per-case gap-tolerant merge/rebase/`ws harvest`/`sc work` instead of blanket refusal, and blob-size/object-count clone filters alongside the diff --git a/docs/THREAT-MODEL.md b/docs/THREAT-MODEL.md index d802bdc4..e2165cc4 100644 --- a/docs/THREAT-MODEL.md +++ b/docs/THREAT-MODEL.md @@ -28,6 +28,18 @@ too, not just the src-control-side metadata. wrappers can observe the plaintext through the child environment; this is fundamental to env-var injection. The parent's intermediate buffer is zeroized (best-effort), the child copy is not. +- **Re-affirmed (map ticket #31): fd/stdin injection stays deferred.** The + security claim for a non-env injection mode is mostly illusory against the + named adversary — a same-user attacker who can read the identity key file + and `.sc/` directly decrypts everything regardless of injection mechanism; + what fd/stdin would actually remove is *accidental* propagation (env dumps, + inheritance by grandchildren), which is hygiene, not attack surface. Env + stays the default contract for `sc run`, `sc work --with-secrets`, and `sc + ws run --with-secrets` alike. Revisit triggers: real committed-secret use in + agent sessions, or an observed env-leak incident class (agents dumping env + into logs or committed files) — at which point an additive fd-pipe opt-in + mode is the presumptive shape, since deferring an *additional* mode carries + no lock-in cost. - **Rotation ≠ erasure:** `sc secret rotate` re-seals under a fresh DEK and cuts off future reads through the current registry, but the old ciphertext object remains reachable and decryptable by anyone who kept the old DEK. `sc secret @@ -63,6 +75,19 @@ too, not just the src-control-side metadata. into a different branch position (a signature binds identity to a snapshot *id*, not to a branch position). `amend`/`rebase`/`merge` results start **unsigned by design** — a new snapshot id is a new claim that must be re-signed. +- **Re-affirmed and sharpened (map ticket #32): replay is ref binding only.** A + snapshot's parents are hashed into its id, so a signature over the id already + binds identity to the snapshot **and its entire ancestry** — a signed snapshot + cannot be grafted under different parents without voiding the signature. + "Replayed elsewhere in history" overstates the residue; the accurate claim is + narrower: a legitimately-signed snapshot is **replayable to a ref tip within + its own history** — nothing stops a hostile remote from pointing a branch name + at an older or side-branch signed snapshot (a rollback/ref-swap), the same gap + git addresses with signed pushes or gittuf-class ref metadata. Deferred: a + gittuf-shaped signed ref attestation (freshness/monotonicity semantics, + verified at fetch/clone) is its own trust-model effort, not a corner of an + unrelated map. Revisit triggers: the first real multi-writer or hosted `sc + serve` deployment, or hosting sc repos on untrusted infrastructure. ## Agent session transcripts (`sc transcript`) — ADR-0038 @@ -76,7 +101,7 @@ too, not just the src-control-side metadata. scanner warns at attach time but never blocks; rotation of the underlying secret remedies). -## Network transport (`sc serve --http` / `sc+http://`) — ADR-0036/0040 +## Network transport (`sc serve --http` / `sc+http://`) — ADR-0036/0040/0041 - **No TLS.** `sc serve --http` is **plaintext**. When bearer-token auth is configured, the token crosses the wire in the clear — a public deployment MUST @@ -88,10 +113,46 @@ too, not just the src-control-side metadata. - **Minor pre-auth information leak:** a `404` (no repo) is written before the auth gate, so an unauthenticated client can distinguish "a repo is served here" (`401`) from "no repo" (`404`). No content is exposed. -- **Not proxy/CDN-safe** (raw post-opening protocol), and the server is unbounded - thread-per-connection with no idle-transfer watchdog — operational hardening - items tracked in [`ROADMAP.md`](../ROADMAP.md)'s Deferred section. -- The ssh:// transport delegates authentication entirely to ssh (ADR-0022). +- **Not proxy/CDN-safe** (raw post-opening protocol) — this remains open, deferred + in [`ROADMAP.md`](../ROADMAP.md). +- **Listener resource bounds — closed by P31 (ADR-0041).** The three operational- + hardening items this section used to track as open are now closed, plus a + fourth gap P31's own research pass first named: + - **Connection exhaustion:** an atomic connection counter enforces + `--max-connections` (default 32, matching `git-daemon`); at the limit a new + connection is accepted, immediately given a busy status at the pre-handshake + opening seam, and closed — no queuing, no unbounded thread spawn. + - **Idle/stalled-peer exhaustion:** `--timeout` (default 300s, 0 disables) sets + read *and* write timeouts on the session socket for its whole lifetime — not + just the opening's original 30s, which used to be cleared once the wire + handoff began. A trip is connection-fatal (frame desync precludes recovery); + the spooled temp pack is guard-cleaned on the resulting unwind. + - **Accept-loop hot-spin under fd exhaustion:** a hardcoded exponential backoff + (5ms doubling to a 1s cap, reset on the next successful accept — Go + `net/http`'s shape) paces retries around `EMFILE`/`ENFILE` instead of + busy-looping. + - **Aggregate pack-spool exhaustion (named and closed together, P31):** this + item was never listed here or in ADR-0036 before P31's research pass found + it. The incoming-pack spool used no aggregate size bound — only ADR-0039's + per-frame/per-record caps applied — so a single oversized or endless pack + could exhaust server disk on **either** transport. `--max-pack-size` + (default 16 GiB, 0 = unlimited, floor 256 MiB = `MAX_OBJECT_SIZE`) now + counts the running total and aborts mid-stream past the cap. This includes + the **read-only drain path**: the pre-`EC_READONLY` drain that keeps a + connection in sync when a read-only-scoped client streams a pack the server + is about to reject was itself unbounded — an `ro`-token client could still + write arbitrary bytes to `.sc/tmp` before being told no. It is now capped at + ~8 MiB (`RO_DRAIN_CAP`): an honest small misconfigured push still gets the + clean typed `EC_READONLY` error, while a larger bulk spool is dropped + mid-send instead of fully drained to disk. + - **Not closed by P31:** per-IP/per-token sub-limiting (a distributed + many-connections-from-many-sources attack below the per-listener cap is + unaddressed), and a legitimate transfer slower than a true 300s zero-byte + stall is unaffected by `--timeout`, but an operator on an unusually slow + network must raise it explicitly. See ADR-0041 for the full decision, + including rejected alternatives. +- The ssh:// transport delegates authentication entirely to ssh (ADR-0022); + `--max-pack-size` is the one P31 bound that also applies to `--stdio`. ## Untrusted-input hardening (DoS) — ADR-0039 diff --git a/docs/adr/0041-listener-resource-limits.md b/docs/adr/0041-listener-resource-limits.md new file mode 100644 index 00000000..5464096c --- /dev/null +++ b/docs/adr/0041-listener-resource-limits.md @@ -0,0 +1,169 @@ +# ADR-0041: Listener resource limits + +- **Status:** Accepted +- **Date:** 2026-07-10 +- **Phase:** 31 +- **Builds on:** ADR-0036 (P26 sc-native HTTP transport — names three of the + four gaps closed here), ADR-0040 (P29 access control — the seam where a + busy/error status is already written before the wire handshake), ADR-0039 + (P28 DoS hardening — `MAX_OBJECT_SIZE`, composed with, not replaced by, + this phase's spool cap) +- **Research:** `docs/research/bounded-server-patterns.md` (ticket #27, + std-only bounded-server survey pinned to file:line). Decided at the + decision level via the wayfinder map, ticket #28. + +## Context + +ADR-0036 shipped `sc serve --http` with three accepted-but-open consequences: +unbounded thread-per-connection (no pool, cap, or backpressure), no +idle-transfer watchdog (the opening's 30s read timeout is cleared once +`wire::serve` takes over), and no accept-loop backoff (a sustained +`EMFILE`/`ENFILE` hot-spins `listener.incoming()`). The #27 research pass +found a fourth gap neither ADR-0036 nor `docs/THREAT-MODEL.md` had named: the +aggregate incoming-pack spool is unbounded, and the read-only rejection path +drains it too (`wire.rs`'s `EC_READONLY` arm spools the whole pack before +replying, so an `ro`-token client can still write arbitrary bytes to disk). +This phase closes all four, dep-free, with no wire-protocol change. + +## Decision + +`sc serve` gains three new flags plus one hardcoded internal behavior, all +scoped to the listener (`--stdio` is unaffected except where noted): + +1. **`--max-connections ` (default 32, 0 = unlimited).** An atomic + live-connection counter (`SlotGuard`, RAII-decremented on every exit path + — clean return, error, panic unwind). At the limit, a new connection is + accepted, immediately written a busy status at the HTTP opening — the + same pre-handshake seam ADR-0040's `401`/`404` already use — and closed. + No queuing: for long-lived multi-verb wire sessions, fail-fast beats an + invisible wait. Default 32 matches `git-daemon --max-connections`, the + closest prior-art analog for a bare TCP git-style server. + +2. **`--timeout ` (default 300, 0 = disabled).** Read *and* write + timeouts are set on the `TcpStream` once the session begins and persist + for the whole session, replacing (not merely preceding) the opening's + `OPENING_READ_TIMEOUT` — previously that 30s timeout was cleared outright + after the `200` status. A timeout trip is connection-fatal: mid-stream + abort desyncs the frame stream (no partial-frame recovery is possible), + so the connection is dropped rather than resumed; `TempPackGuard` cleans + any spooled pack file on the resulting unwind. Under P25 chunking, a + per-syscall timeout is inherently progress-based — only a true zero-byte + stall trips it, not a slow-but-moving transfer. The opening keeps its + existing, unchanged 30s. + +3. **`--max-pack-size ` (default 16 GiB, 0 = unlimited, floor 256 MiB + = `MAX_OBJECT_SIZE`).** Applies to **both** `--http` and `--stdio` — it + lives in `WirePolicy` (`crates/repo/src/wire.rs`), the shared layer both + transports call into, not in `ServeLimits` (the `--http`-only listener + config). A configured non-zero value below `MAX_OBJECT_SIZE` is rejected + as invalid config at startup. Enforcement is a counted mid-stream abort in + the pack-stream readers (mirroring git's `receive.maxInputSize`): once the + running total crosses the cap, the reader stops, a best-effort + `EC_TOO_LARGE` reply is written (best-effort because the abort already + desynced the stream — the reply is a courtesy, not a guarantee the client + parses it before the socket closes), and the connection is torn down. + This composes above, and never replaces, ADR-0039's existing per-frame + wire-length cap, per-record pack cap, and zstd decompression-bomb guard — + this is an *aggregate* bound across an entire incoming pack, those are + *per-unit* bounds within it. + +4. **Read-only drain cap — `RO_DRAIN_CAP` (8 MiB, hardcoded).** The + deliberate pre-`EC_READONLY` drain (draining keeps a wire connection in + sync when the client has already started streaming a pack the server is + about to reject) now stops at ~8 MiB. An honest misconfiguration — a + read-only-scoped client attempting a small legitimate push — still gets + the clean typed `EC_READONLY` error. A hostile or careless bulk spool + past that size is dropped mid-send instead of fully drained to disk, so + an `ro`-token connection can no longer be used to write unbounded bytes + to `.sc/tmp` regardless of the eventual rejection. This gap was never + named in ADR-0036 or the threat model before this phase's research pass + found it — P31 both names and closes it. + +5. **Accept-loop backoff (hardcoded, no flag).** Go `net/http`'s exact + shape: on an accept error, sleep starting at 5ms, doubling on each + consecutive error, capped at 1s, reset to 5ms on the next successful + accept. Turns fd exhaustion (`EMFILE`/`ENFILE`) from a busy-loop into a + paced retry. No operator knob — Go ships this knobless too, and no + tuning story exists yet to justify one. + +## Recorded divergences from prior art + +- **Timeout on by default, unlike `git-daemon`'s `--timeout` (default 0 / + disabled).** A defaults-off knob does not close an audit-tracked High — a + silent or write-stalled peer parking a server thread forever is exactly + the DoS shape this phase exists to close, and without a reaper the + connection cap (#1) is hostage to any single stalled client holding a + slot forever. 300s, not a shorter value, because the one legitimate + casualty of a shorter deadline is a client computing between verbs (e.g. + `sc ws run` doing real work mid-session). +- **Finite 16 GiB spool default, unlike git's unlimited + `receive.maxInputSize` (default 0).** `sc serve` deployments often share + the same volume as the working tree/`.sc/tmp` spool, and this phase's + posture is fail-closed defaults throughout rather than borrowing git's + unbounded default. + +## Rejected alternatives + +- **Block-at-accept semaphore (queue rather than shed at the connection + cap).** Rejected: a wire connection is a long-lived, many-verb session, + not a one-shot HTTP request — an invisible queued wait is worse UX than a + clear, immediately retryable busy status. +- **git-daemon's grace-then-shed pattern.** Rejected as unnecessary + complexity for this MVP; the immediate busy-status-and-close is simpler + and gives the client the same actionable signal sooner. +- **The Rust Book's fixed-size thread-pool-with-queue pattern.** Rejected + for the same reason as the semaphore: it trades a clear rejection for an + invisible queue, wrong shape for long-lived sessions. +- **A reserved-fd accept-loop shed (hold back N fds for cleanup work).** + Rejected: the connection cap (#1) already demotes fd exhaustion to + external pressure (e.g. other processes on the host) rather than a + self-inflicted condition; the refinement can ride a later hardening pass + if real-world churn ever proves the need. +- **Env-var or `.sc/config`-file knobs for these limits.** Rejected for + this phase: no `.sc/config` surface exists yet, and CLI flags are + consistent with every other `sc serve` option shipped so far + (`--read-only`, `--allow-public`, ADR-0040's token flags). A config-file + surface is a separate, larger decision not scoped here. +- **An sshd-`MaxStartups`-style pre-auth sub-cap distinct from the total + connection cap.** Deliberately deferred until real-world churn patterns + justify a second knob; one total cap is the simpler starting shape. + +## Consequences + +- ADR-0036's three named accepted-but-open consequences (unbounded + thread-per-connection, no idle-transfer watchdog, no accept-loop backoff) + are closed by this phase. +- The aggregate-spool gap — never named in ADR-0036 or the threat model, + surfaced only by the #27 research pass — is both named and closed, + including its read-only-drain-path variant. +- `PROTOCOL_VERSION` stays 3: none of the four bounds touch the wire + protocol. `EC_TOO_LARGE` degrades gracefully to opaque message text on an + old client that doesn't recognize the code, matching `EC_READONLY`'s + precedent from ADR-0040. +- Zero new dependencies. +- `--stdio` gains only the pack-size cap (shared `WirePolicy`); connection + count, session timeout, and accept backoff are `--http`-listener-specific + concepts that don't apply to a single ssh-spawned stdio session. +- Proven by `demo/run_limits_demo.sh` (the floor-cap enforcement on the push + path and the busy-status shed under `--max-connections`); the timeout + reaper and accept-backoff pacing are unit-test-proven rather than + demo-proven, since reliably forcing a real stalled-socket or fd-exhaustion + condition in a demo script is impractical. + +## Threat model honesty + +- **Defends:** thread/fd exhaustion from unbounded connection churn (cap + + shed); a stalled or hostile-slow peer holding a server thread forever + (session timeout, both directions); unbounded disk consumption from an + oversized or endless incoming pack, on both transports and including the + read-only-drain path (spool cap + drain cap); accept-loop hot-spin under + sustained fd exhaustion (backoff). +- **Does NOT defend:** a client operating within all four bounds but still + malicious in content (these are resource bounds, not content/auth + controls — see ADR-0040 for the access-control layer); distributed + multi-connection exhaustion below the per-listener cap (e.g. many + distinct source IPs each opening a few connections) — no per-IP or + per-token sub-limiting exists; a legitimate transfer that is simply + slower than 300s of true zero-byte stall is unaffected, but an operator + with a much slower network than anticipated must raise `--timeout` + explicitly. diff --git a/docs/adr/README.md b/docs/adr/README.md index ed1842e7..018bd416 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -51,6 +51,7 @@ supersedes the old one and update the **Status** line of the superseded record. | [0038](0038-agent-session-transcripts.md) | Agent session transcripts as CAS objects | Accepted | 30 | | [0039](0039-security-hardening-sweep.md) | Security hardening sweep (audit fix-now items) | Accepted | 28 | | [0040](0040-sc-http-access-control.md) | sc+http access control (read-only, fail-closed bind, bearer tokens) | Accepted | 29 | +| [0041](0041-listener-resource-limits.md) | Listener resource limits (connection cap, session timeout, spool cap, accept backoff) | Accepted | 31 | See [`ROADMAP.md`](../../ROADMAP.md) for the phase sequence. ADR numbers are assigned in creation order, so 0017 — the secret scanner — sequences as phase P5, From 7072abdbc04fc98bf141f7b46eda81fb161cfa03 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 12:30:08 +0300 Subject: [PATCH 13/14] =?UTF-8?q?docs(adr):=20fix=20count=20=E2=80=94=20tw?= =?UTF-8?q?o=20hardcoded=20behaviors,=20not=20one=20(P31=20review)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- docs/adr/0041-listener-resource-limits.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adr/0041-listener-resource-limits.md b/docs/adr/0041-listener-resource-limits.md index 5464096c..5fc33ebd 100644 --- a/docs/adr/0041-listener-resource-limits.md +++ b/docs/adr/0041-listener-resource-limits.md @@ -27,7 +27,7 @@ This phase closes all four, dep-free, with no wire-protocol change. ## Decision -`sc serve` gains three new flags plus one hardcoded internal behavior, all +`sc serve` gains three new flags plus two hardcoded internal behaviors, all scoped to the listener (`--stdio` is unaffected except where noted): 1. **`--max-connections ` (default 32, 0 = unlimited).** An atomic From cf8bffb34972839b3150332b6e551a11e6b55241 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 10 Jul 2026 12:45:29 +0300 Subject: [PATCH 14/14] =?UTF-8?q?fix(p31):=20final-review=20wave=20?= =?UTF-8?q?=E2=80=94=20spawn-failure=20shed,=20PackTooLarge=20message,=20d?= =?UTF-8?q?oc=20fixes?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Six final-review findings against the P31 listener-resource-limits work, all small and isolated: 1. serve_http_listener now uses std::thread::Builder::spawn instead of the panicking std::thread::spawn, so OS thread-creation failure (EAGAIN) is logged and paced with the existing accept backoff instead of killing the accept loop. 2. err_to_wire no longer double-prefixes Error::PackTooLarge's Display text on round trip; the read_pack_stream abort payload now names the cap and points at --max-pack-size / splitting the push. 3. Removed a stray duplicated `# justifies it (P29)` line in CLAUDE.md. 4. ADR-0041 now states precisely how the 503 pre-handshake seam differs from ADR-0040's 401/404 (before vs. after reading the opening), and records two known boundaries: the RST race on a 503 write, and --max-connections 0 leaving thread exhaustion reachable. 5. Fixed the ServeLimits doc comment's stray reference to ro_drain_cap, which actually lives on wire::WirePolicy. 6. read_frame_inner's EOF diagnostics now include the source io::Error instead of swallowing it, so a session-timeout trip is distinguishable from a generic EOF. 7. Dropped the err_from_wire delegating alias, updating its two call sites to call wire_to_err directly. Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB --- CLAUDE.md | 1 - crates/repo/src/http_transport.rs | 21 +++++++++-- crates/repo/src/wire.rs | 43 +++++++++++++++++------ docs/adr/0041-listener-resource-limits.md | 41 ++++++++++++++++++--- 4 files changed, 86 insertions(+), 20 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 4cbcca44..57009658 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -299,7 +299,6 @@ cargo run --bin sc -- serve --http [--read-only] [--allow-public] # cap, both --http and --stdio, default 16 GiB, # 0=unlimited, floor 256 MiB (MAX_OBJECT_SIZE); # counted mid-stream abort -> EC_TOO_LARGE (P31) - # justifies it (P29) cargo run --bin sc -- serve token add --label --scope ro|rw # mint an # sct- bearer token in .sc/serve-tokens.toml; # raw value prints once on stdout (P29) diff --git a/crates/repo/src/http_transport.rs b/crates/repo/src/http_transport.rs index 3d9cd77e..781779e2 100644 --- a/crates/repo/src/http_transport.rs +++ b/crates/repo/src/http_transport.rs @@ -413,8 +413,9 @@ impl AcceptBackoff { /// Server-side resource bounds for `sc serve --http` (P31): a hostile or /// merely slow-and-many-clients peer population must not be able to exhaust /// server memory, fds, or threads. `0` disables the corresponding limit in -/// every field, matching the rest of this file's `0 = unbounded` convention -/// (e.g. `max_connections`/`ro_drain_cap`). +/// every field of this struct, matching the `0 = unbounded` convention also +/// used by [`crate::wire::WirePolicy::ro_drain_cap`] (wire.rs — a different +/// struct, not a field of this one). /// /// Defaults, each with its own provenance/divergence noted (see the P31 /// design doc for the full survey): @@ -606,13 +607,27 @@ pub fn serve_http_listener( g }; let root = root.to_path_buf(); - std::thread::spawn(move || { + let spawn_result = std::thread::Builder::new().spawn(move || { let _guard = guard; // slot held for the connection's lifetime if let Err(e) = handle_http_connection(stream, &root, read_only, mandatory_auth, limits) { eprintln!("sc serve --http: connection error: {e}"); } }); + // OS thread creation can fail (e.g. EAGAIN under fd/thread exhaustion). + // `std::thread::spawn` PANICS in that case, which would kill this whole + // accept loop — exactly the "accept loop stays alive" property this + // phase exists to guarantee. `Builder::spawn` instead returns an + // `io::Result`; on `Err`, the closure we tried to spawn is dropped + // right here, which drops the moved-in `stream` and `SlotGuard` for us + // (closing the socket and freeing the slot) — the client just sees a + // plain close, not a 503, since writing a response would require + // restructuring the guard/stream ownership above. We shed with the + // same backoff used for accept() errors and keep looping. + if let Err(e) = spawn_result { + eprintln!("sc serve --http: thread spawn failed: {e}"); + std::thread::sleep(backoff.on_error()); + } } Ok(()) } diff --git a/crates/repo/src/wire.rs b/crates/repo/src/wire.rs index 0d49cb14..831e79a1 100644 --- a/crates/repo/src/wire.rs +++ b/crates/repo/src/wire.rs @@ -368,7 +368,7 @@ fn read_frame_inner(r: &mut impl Read) -> Result>> { } let mut buf = vec![0u8; len]; r.read_exact(&mut buf) - .map_err(|_| Error::Protocol(format!("EOF inside {len}-byte frame body")))?; + .map_err(|e| Error::Protocol(format!("read inside {len}-byte frame body failed: {e}")))?; Ok(Some(buf)) } @@ -445,7 +445,10 @@ pub fn read_pack_stream( Some((&ST_PACK_CHUNK, rest)) => { let next = total + rest.len() as u64; if max_bytes != 0 && next > max_bytes { - return Err(Error::PackTooLarge(format!("{max_bytes} bytes"))); + return Err(Error::PackTooLarge(format!( + "server cap is {max_bytes} bytes; raise --max-pack-size on the \ + server or split the push" + ))); } sink.write_all(rest)?; total = next; @@ -485,7 +488,7 @@ pub fn parse_response(frame: Vec) -> Result> { let code = c.u8()?; let msg = c.str()?; c.done()?; - Err(err_from_wire(code, msg)) + Err(wire_to_err(code, msg)) } s => Err(Error::Protocol(format!("bad response status {s}"))), } @@ -493,6 +496,14 @@ pub fn parse_response(frame: Vec) -> Result> { /// Map a repo error onto its wire code + message. Errors sync logic matches on /// keep their identity; the rest carry only their message. +/// +/// `Error::PackTooLarge(inner)` is special-cased: its `Display` impl already +/// prepends the "pack exceeds the server's --max-pack-size limit: " prefix +/// around `inner`, and [`wire_to_err`] reconstructs a fresh +/// `Error::PackTooLarge(msg)` from whatever we send here — sending +/// `e.to_string()` would carry that prefix as part of `msg`, so the +/// round-tripped error's `Display` would show the prefix twice. Sending only +/// `inner` keeps the prefix appearing exactly once, on display. pub fn err_to_wire(e: &Error) -> (u8, String) { let code = match e { Error::NotARepo => EC_NOT_A_REPO, @@ -503,7 +514,11 @@ pub fn err_to_wire(e: &Error) -> (u8, String) { Error::PackTooLarge(_) => EC_TOO_LARGE, _ => EC_OTHER, }; - (code, e.to_string()) + let msg = match e { + Error::PackTooLarge(inner) => inner.clone(), + _ => e.to_string(), + }; + (code, msg) } /// Reconstruct a typed error from its wire code; unknown/untyped codes become @@ -519,11 +534,6 @@ pub fn wire_to_err(code: u8, msg: String) -> Error { } } -/// Alias for backward compatibility with existing call sites. -fn err_from_wire(code: u8, msg: String) -> Error { - wire_to_err(code, msg) -} - // --- response body builders/decoders (one symmetric pair per verb shape) --- /// Response body for Hello: the server's protocol version. @@ -1086,7 +1096,7 @@ mod tests { // Untyped errors become Remote(msg) and keep their message. let (code, msg) = err_to_wire(&crate::error::Error::Unborn); - let back = err_from_wire(code, msg); + let back = wire_to_err(code, msg); match back { crate::error::Error::Remote(m) => assert!(m.contains("unborn"), "message lost: {m}"), other => panic!("expected Remote, got {other:?}"), @@ -1459,7 +1469,18 @@ mod tests { let (code, msg) = err_to_wire(&e); assert_eq!(code, EC_TOO_LARGE); match wire_to_err(code, msg) { - Error::PackTooLarge(m) => assert!(m.contains("17179869184")), + Error::PackTooLarge(m) => { + assert!(m.contains("17179869184")); + // The Display prefix ("pack exceeds the server's + // --max-pack-size limit: ") must appear exactly once, not + // doubled by a round trip through the wire codes. + let display = Error::PackTooLarge(m).to_string(); + assert_eq!( + display.matches("pack exceeds").count(), + 1, + "expected exactly one occurrence of the Display prefix, got: {display}" + ); + } other => panic!("expected PackTooLarge, got {other:?}"), } } diff --git a/docs/adr/0041-listener-resource-limits.md b/docs/adr/0041-listener-resource-limits.md index 5fc33ebd..1e83297d 100644 --- a/docs/adr/0041-listener-resource-limits.md +++ b/docs/adr/0041-listener-resource-limits.md @@ -33,11 +33,16 @@ scoped to the listener (`--stdio` is unaffected except where noted): 1. **`--max-connections ` (default 32, 0 = unlimited).** An atomic live-connection counter (`SlotGuard`, RAII-decremented on every exit path — clean return, error, panic unwind). At the limit, a new connection is - accepted, immediately written a busy status at the HTTP opening — the - same pre-handshake seam ADR-0040's `401`/`404` already use — and closed. - No queuing: for long-lived multi-verb wire sessions, fail-fast beats an - invisible wait. Default 32 matches `git-daemon --max-connections`, the - closest prior-art analog for a bare TCP git-style server. + accepted, immediately written a busy status, and closed — at the same + pre-handshake HTTP-opening seam ADR-0040's `401`/`404` write to, but + *earlier* in it: ADR-0040's `401`/`404` are written only after reading + and parsing the client's opening (they respond to what the opening said), + while the `503` here is written before any read of the opening at all — + the connection is shed purely on the live-connection count, with the + opening never consulted. No queuing: for long-lived multi-verb wire + sessions, fail-fast beats an invisible wait. Default 32 matches + `git-daemon --max-connections`, the closest prior-art analog for a bare + TCP git-style server. 2. **`--timeout ` (default 300, 0 = disabled).** Read *and* write timeouts are set on the `TcpStream` once the session begins and persist @@ -167,3 +172,29 @@ scoped to the listener (`--stdio` is unaffected except where noted): slower than 300s of true zero-byte stall is unaffected, but an operator with a much slower network than anticipated must raise `--timeout` explicitly. +- **Known boundary (a): the `503` write races the client's own send.** + Because the busy status is written *before* the opening is read at all + (see Decision #1's seam-timing note above), a client that is mid-send on + its opening when the slot check fails can observe the server's write + colliding with its own outstanding write, and the kernel/TCP stack may + surface that as a connection reset instead of a clean `503` response — + observed empirically at roughly 1-in-3 on macOS loopback in this phase's + own testing. Draining the client's opening before closing (so the `503` + always lands cleanly) was considered and rejected: it would reintroduce a + per-shed-connection read cost on precisely the path `--max-connections` + exists to keep cheap — an attacker driving the server into the shed state + could then force it to spend read cycles per rejected connection instead + of none. The `503` is best-effort, not guaranteed-delivered; a client must + already treat a bare reset as retryable, same as a real busy response. +- **Known boundary (b): `--max-connections 0` (unlimited) still exposes + thread exhaustion.** With the cap disabled, nothing prevents the accept + loop from spawning threads until the OS refuses (`EAGAIN` from + `pthread_create` or similar). Finding 1 of this phase's final-review wave + closed the crash mode this used to cause — `serve_http_listener` now uses + `std::thread::Builder::spawn` instead of the panicking `std::thread::spawn`, + so a failed spawn is logged, paced by the same `AcceptBackoff` used for + `accept()` errors, and the loop continues — but it does not close the + underlying exposure: an operator who explicitly disables the connection + cap is still trusting the OS's own thread/fd limits as the only backstop. + `--max-connections 0` is an explicit opt-out of this phase's shedding + behavior, not a claim that unlimited connections are safe.