diff --git a/CLAUDE.md b/CLAUDE.md index 42b52cdf..57009658 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -280,12 +280,25 @@ bash demo/run_streaming_demo.sh # streaming pack transfer proof (P # in 250+ chunk frames, byte-for-byte clone, zero # .sc/tmp residue on either end cargo run --bin sc -- serve --http [--read-only] [--allow-public] + [--max-connections ] [--timeout ] [--max-pack-size ] # sc-native wire protocol over TCP (P26); e.g. # `sc serve --http 127.0.0.1:8730 .`; exactly one # of --stdio/--http is required; a non-loopback # is refused unless --read-only, # --allow-public, or a configured serve token # justifies it (P29) + # --max-connections : concurrent-connection cap, + # --http only, default 32, 0=unlimited; at the limit + # a new connection gets busy-status-and-close, no + # queuing (P31) + # --timeout : session idle/progress timeout, + # --http only, default 300, 0=disabled; read+write, + # persists the whole session, connection-fatal on + # trip; opening keeps its own 30s (P31) + # --max-pack-size : incoming-pack aggregate + # cap, both --http and --stdio, default 16 GiB, + # 0=unlimited, floor 256 MiB (MAX_OBJECT_SIZE); + # counted mid-stream abort -> EC_TOO_LARGE (P31) cargo run --bin sc -- serve token add --label --scope ro|rw # mint an # sct- bearer token in .sc/serve-tokens.toml; # raw value prints once on stdout (P29) @@ -996,13 +1009,13 @@ serve --http` is unauthenticated, as `--stdio` delegates auth to ssh — production auth + TLS means fronting with a reverse proxy; this is the reach primitive, not a hosted-git competitor); **not HTTP-proxy/CDN safe** (a strict proxy won't tunnel the post-opening raw protocol, the accepted -cost of the persistent-connection model). **Accepted design consequences, -not yet closed (deferred, see ROADMAP):** `serve_http_listener` is -unbounded thread-per-connection with no pool/backpressure; the opening -timeout is cleared once `wire::serve` takes over, so a client that stalls -mid-transfer holds its thread indefinitely (no idle-transfer watchdog -yet); the accept loop has no backoff on sustained fd exhaustion. See -ADR-0036. +cost of the persistent-connection model). **Accepted design consequences at +the time — closed by P31 (ADR-0041):** `serve_http_listener`'s unbounded +thread-per-connection (no pool/backpressure), the opening timeout being +cleared once `wire::serve` takes over (no idle-transfer watchdog), and the +accept loop's lack of backoff on sustained fd exhaustion are all closed by +P31's `--max-connections`, `--timeout`, and hardcoded accept backoff, +respectively — see the P31 section below. See ADR-0036. **Phase 27 is built.** The P25–P27 scale-&-reach horizon's capstone: partial clone. `.sc/promisor` (`crates/repo/src/promisor.rs`, local, @@ -1242,12 +1255,46 @@ bytes, the `sc log` marker, and `sc gc` pruning a deleted branch's transcript while the still-reachable one from earlier survives — run twice, zero residue. No new dependency. See ADR-0038. +**Phase 31 is built.** Listener resource limits close ADR-0036's three +named-but-open accepted consequences plus a fourth this phase's own +research pass (`docs/research/bounded-server-patterns.md`, ticket #27) +found and named for the first time: the aggregate incoming-pack spool was +unbounded on both transports, including via the read-only rejection path. +`sc serve --http` gains `--max-connections ` (default 32, matching +`git-daemon`, 0 = unlimited; at the limit a new connection gets an +immediate busy status at the pre-handshake opening seam and closes — no +queuing), `--timeout ` (default 300, 0 = disables; read+write +timeouts now persist for the whole session rather than being cleared after +the opening, and a trip is connection-fatal since mid-stream abort desyncs +the frame stream; the opening keeps its own unchanged 30s), and +`--max-pack-size ` (default 16 GiB, 0 = unlimited, floor 256 MiB = +`MAX_OBJECT_SIZE`; applies to **both** `--http` and `--stdio` since it +lives in the shared `WirePolicy`, not the `--http`-only `ServeLimits`; a +counted mid-stream abort replies `EC_TOO_LARGE` best-effort before closing). +The read-only pre-drain (`wire.rs`'s `EC_READONLY` arm, which drains an +incoming pack before replying so the connection stays in sync) is now +capped at 8 MiB (`RO_DRAIN_CAP`): an honest small misconfigured push still +gets the clean typed error, a larger bulk spool is dropped mid-send instead +of fully written to disk. The accept loop gained a hardcoded Go-`net/http`- +shaped exponential backoff (5ms doubling to a 1s cap, reset on success) — +no operator knob, matching Go's own precedent. Two recorded divergences +from prior art: the timeout ships **on** by default, unlike `git-daemon`'s +off-by-default `--timeout` (a defaults-off knob doesn't close an +audit-tracked High); the pack-spool default is **finite** (16 GiB), unlike +git's unlimited `receive.maxInputSize` (sc servers often share the working +tree's volume). `PROTOCOL_VERSION` stays 3 — none of the four bounds touch +the wire protocol, and `EC_TOO_LARGE` degrades to opaque text on an old +client, same as `EC_READONLY`'s precedent. Zero new dependencies. Proven by +`demo/run_limits_demo.sh` (the floor-cap enforcement on the push path and +the busy-status shed under `--max-connections`); the session-timeout reaper +and accept-backoff pacing are unit-test-proven rather than demo-proven, +since forcing a real stalled socket or fd exhaustion reliably in a demo +script is impractical. See ADR-0041. + Remaining follow-ons: operation objects in the CAS, oplog entries for remote-tracking refs, extending the sparse gate to `materialize_conflict_state`'s `to_encrypt`/sidecar write loops (see the -P24 boundary note above), the three P26 `sc serve --http` hardening items -named above (connection pool/backpressure, idle-transfer watchdog, -accept-loop backoff), the three P27 items named above (transparent +P24 boundary note above), the three P27 items named above (transparent lazy-fetch as a deferred alternative to explicit `sc backfill`, per-case gap-tolerant merge/rebase/`ws harvest`/`sc work` instead of blanket refusal, and blob-size/object-count clone filters alongside the diff --git a/crates/cli/src/main.rs b/crates/cli/src/main.rs index 393d1a97..159f97f6 100644 --- a/crates/cli/src/main.rs +++ b/crates/cli/src/main.rs @@ -298,6 +298,21 @@ enum Cmd { /// configured token, binding a non-loopback address is refused. #[arg(long)] allow_public: bool, + /// Maximum concurrent connections (`--http` only); 0 = unlimited. + /// Default 32 (git-daemon parity). At the limit new connections get an + /// immediate busy status and close. (P31) + #[arg(long)] + max_connections: Option, + /// Session idle timeout in seconds (`--http` only); 0 = disabled. + /// Default 300. A connection making zero-byte progress for this long is + /// dropped. (P31) + #[arg(long)] + timeout: Option, + /// Maximum incoming pack size in bytes (both --http and --stdio); + /// 0 = unlimited. Default 16 GiB. Must be ≥ 256 MiB (MAX_OBJECT_SIZE) + /// when non-zero. (P31) + #[arg(long)] + max_pack_size: Option, /// Repo root to serve (the directory containing `.sc/`). Required /// unless a `token` subcommand is used. path: Option, @@ -816,12 +831,24 @@ fn main() -> Result<()> { http, read_only, allow_public, + max_connections, + timeout, + max_pack_size, path, } => match sub { Some(ServeSub::Token { op }) => run_serve_token(op), None => { let path = path.ok_or_else(|| anyhow::anyhow!("sc serve requires a "))?; - run_serve(stdio, http, read_only, allow_public, path) + run_serve( + stdio, + http, + read_only, + allow_public, + max_connections, + timeout, + max_pack_size, + path, + ) } }, Cmd::Remote { op } => run_remote(op), @@ -3118,12 +3145,16 @@ fn run_clone_git(url: &str, dst: &std::path::Path) -> Result<()> { /// dropped, e.g. by process termination). Exactly one mode is required. /// `--read-only`/`--allow-public` are `--http`-only (P29): `--stdio` /// delegates auth/access entirely to ssh, so combining them is refused -/// rather than silently ignored. +/// rather than silently ignored. `--max-connections`/`--timeout` are also +/// `--http`-only (P31). `--max-pack-size` applies to both (P31). fn run_serve( stdio: bool, http: Option, read_only: bool, allow_public: bool, + max_connections: Option, + timeout: Option, + max_pack_size: Option, path: PathBuf, ) -> Result<()> { match (stdio, http) { @@ -3131,13 +3162,33 @@ fn run_serve( if read_only || allow_public { anyhow::bail!("--read-only/--allow-public apply only to --http, not --stdio"); } + if max_connections.is_some() || timeout.is_some() { + anyhow::bail!("--max-connections/--timeout apply only to --http, not --stdio"); + } + let max_pack = max_pack_size.unwrap_or(scl_repo::wire::DEFAULT_MAX_PACK_SIZE); + scl_repo::wire::validate_max_pack_size(max_pack)?; + let policy = scl_repo::wire::WirePolicy { + read_only: false, + max_pack_size: max_pack, + ro_drain_cap: scl_repo::wire::RO_DRAIN_CAP, + }; let mut stdin = std::io::stdin().lock(); let mut stdout = std::io::stdout().lock(); - scl_repo::wire::serve(&path, &mut stdin, &mut stdout)?; + scl_repo::wire::serve_with_policy(&path, &mut stdin, &mut stdout, policy)?; Ok(()) } (false, Some(addr)) => { - scl_repo::http_transport::serve_http(&addr, &path, read_only, allow_public)?; + let mut limits = scl_repo::http_transport::ServeLimits::default(); + if let Some(n) = max_connections { + limits.max_connections = n; + } + if let Some(t) = timeout { + limits.timeout_secs = t; + } + if let Some(m) = max_pack_size { + limits.max_pack_size = m; + } + scl_repo::http_transport::serve_http(&addr, &path, read_only, allow_public, limits)?; Ok(()) } (true, Some(_)) => anyhow::bail!("sc serve accepts only one of --stdio or --http"), diff --git a/crates/repo/src/error.rs b/crates/repo/src/error.rs index 0d388ef5..f64e8ddf 100644 --- a/crates/repo/src/error.rs +++ b/crates/repo/src/error.rs @@ -102,6 +102,15 @@ pub enum Error { /// (`--read-only` or an `ro`-scope token). P29. #[error("server is read-only")] ReadOnly, + /// The server aborted an incoming pack mid-stream because it exceeded the + /// operator's `--max-pack-size` cap (P31). Payload is the server's own + /// human-readable limit text, carried verbatim across the wire. + #[error("pack exceeds the server's --max-pack-size limit: {0}")] + PackTooLarge(String), + /// The server refused the connection at accept time because + /// `--max-connections` was reached (P31). Retryable. + #[error("server busy (connection limit reached); retry later")] + ServerBusy, } pub type Result = std::result::Result; diff --git a/crates/repo/src/http_transport.rs b/crates/repo/src/http_transport.rs index d2c93f62..781779e2 100644 --- a/crates/repo/src/http_transport.rs +++ b/crates/repo/src/http_transport.rs @@ -144,6 +144,7 @@ pub(crate) fn write_status(w: &mut impl Write, code: u16) -> Result<()> { 404 => "Not Found", 400 => "Bad Request", 401 => "Unauthorized", + 503 => "Service Unavailable", _ => { return Err(Error::InvalidArgument(format!( "unsupported HTTP status code: {code}" @@ -323,6 +324,7 @@ impl HttpTransport { )) } 404 => return Err(Error::NotARepo), + 503 => return Err(Error::ServerBusy), other => { return Err(Error::Protocol(format!( "sc+http server returned unexpected status {other}" @@ -379,10 +381,87 @@ impl Transport for HttpTransport { /// `read_client_opening` bounds the opening in BYTES (`MAX_OPENING_BYTES`) /// but not in TIME, so a peer that trickles in under the byte cap and then /// stalls would otherwise hold a server thread (and its socket) forever. -/// Applied only around the opening read — see [`serve_http_listener`] for -/// where it's cleared before a legitimate large pack transfer begins. +/// Applied only around the opening read — the opening timeout is REPLACED +/// (not cleared) by `limits.timeout_secs`'s session read+write timeouts +/// after the 200 status is written, before handing off to `wire::serve` — +/// see [`handle_http_connection`] for the timeout sequencing. const OPENING_READ_TIMEOUT: Duration = Duration::from_secs(30); +/// Exponential accept-error backoff (P31): Go net/http's shape — 5ms +/// doubling to a 1s cap, reset on the next successful accept. Turns fd +/// exhaustion (EMFILE) from a busy-loop into a paced retry. Hardcoded: no +/// operator tuning story exists (Go ships it knobless too). +struct AcceptBackoff { + cur: Duration, +} +impl AcceptBackoff { + const START: Duration = Duration::from_millis(5); + const CAP: Duration = Duration::from_secs(1); + fn new() -> Self { + Self { cur: Self::START } + } + fn on_error(&mut self) -> Duration { + let d = self.cur; + self.cur = (self.cur * 2).min(Self::CAP); + d + } + fn on_success(&mut self) { + self.cur = Self::START; + } +} + +/// Server-side resource bounds for `sc serve --http` (P31): a hostile or +/// merely slow-and-many-clients peer population must not be able to exhaust +/// server memory, fds, or threads. `0` disables the corresponding limit in +/// every field of this struct, matching the `0 = unbounded` convention also +/// used by [`crate::wire::WirePolicy::ro_drain_cap`] (wire.rs — a different +/// struct, not a field of this one). +/// +/// Defaults, each with its own provenance/divergence noted (see the P31 +/// design doc for the full survey): +/// - `max_connections: 32` — matches `git-daemon`'s `--max-connections` +/// default, the closest prior-art analog for a bare TCP git-style server. +/// - `timeout_secs: 300` — on by default, unlike `git-daemon`'s `--timeout` +/// (which defaults to 0/disabled). A silent or write-stalled peer parking +/// a thread forever is exactly the DoS shape this phase closes, so the +/// safer default is chosen deliberately over matching git-daemon here. +/// - `max_pack_size: DEFAULT_MAX_PACK_SIZE` (16 GiB) — also a deliberate +/// divergence: no comparable git-daemon knob exists, so this default is set +/// generously (see Task 3) rather than borrowed from prior art. +#[derive(Debug, Clone, Copy)] +pub struct ServeLimits { + /// Max concurrent connections; beyond this, a new connection is shed with + /// `503 Service Unavailable` before its opening is even read. `0` = + /// unlimited. + pub max_connections: u32, + /// Per-connection idle/session timeout in seconds, applied to both the + /// read and write halves once the session begins (see + /// [`handle_http_connection`]). `0` = disabled. + pub timeout_secs: u64, + /// Forwarded to [`crate::wire::WirePolicy::max_pack_size`]. `0` = + /// unlimited. + pub max_pack_size: u64, +} +impl Default for ServeLimits { + fn default() -> Self { + Self { + max_connections: 32, + timeout_secs: 300, + max_pack_size: crate::wire::DEFAULT_MAX_PACK_SIZE, + } + } +} + +/// Decrements the live-connection count on drop, so every exit path — clean +/// return, error, panic unwind — frees its slot (the TempPackGuard +/// discipline applied to connection slots). +struct SlotGuard(std::sync::Arc); +impl Drop for SlotGuard { + fn drop(&mut self) { + self.0.fetch_sub(1, std::sync::atomic::Ordering::SeqCst); + } +} + /// Is `host` a loopback address (always safe to bind)? IPv4 `127.0.0.0/8`, /// IPv6 `::1`, or the literal `localhost`. Everything else (`0.0.0.0`, a LAN /// IP, `::`) is non-loopback and subject to the fail-closed bind gate in @@ -450,7 +529,9 @@ pub fn serve_http( root: &std::path::Path, read_only: bool, allow_public: bool, + limits: ServeLimits, ) -> Result<()> { + crate::wire::validate_max_pack_size(limits.max_pack_size)?; if !bind_is_allowed(addr, root, read_only, allow_public)? { return Err(Error::InvalidArgument(format!( "refusing to bind non-loopback address {addr} without --read-only, \ @@ -472,7 +553,7 @@ pub fn serve_http( .map_err(|e| Error::ConnectionLost(format!("sc+http local_addr: {e}")))?; println!("listening on {bound}"); std::io::Write::flush(&mut std::io::stdout()).ok(); - serve_http_listener(listener, root, read_only, mandatory_auth) + serve_http_listener(listener, root, read_only, mandatory_auth, limits) } /// Accept-loop core, factored out of [`serve_http`] so tests can bind @@ -489,26 +570,64 @@ pub fn serve_http( /// or a fatal accept-level error occurs; a per-connection error (bad /// opening, a `wire::serve` failure, a dropped socket) is logged to stderr /// and the loop continues — it must never take down the whole server. +/// +/// P31 additions: a live-connection counter enforces `limits.max_connections` +/// (shedding with `503` BEFORE spawning a thread or reading an opening, so a +/// shed connection costs no fd beyond the accept itself); a per-`accept()` +/// error is paced by [`AcceptBackoff`] instead of busy-looping. pub fn serve_http_listener( listener: TcpListener, root: &std::path::Path, read_only: bool, mandatory_auth: bool, + limits: ServeLimits, ) -> Result<()> { + let live = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0)); + let mut backoff = AcceptBackoff::new(); for incoming in listener.incoming() { - let stream = match incoming { - Ok(s) => s, + let mut stream = match incoming { + Ok(s) => { + backoff.on_success(); + s + } Err(e) => { eprintln!("sc serve --http: accept error: {e}"); + std::thread::sleep(backoff.on_error()); continue; } }; + // Connection cap: acquire a slot BEFORE spawning; shed with 503 when full. + let guard = { + let prev = live.fetch_add(1, std::sync::atomic::Ordering::SeqCst); + let g = SlotGuard(live.clone()); + if limits.max_connections != 0 && prev >= limits.max_connections as usize { + let _ = write_status(&mut stream, 503); + continue; // g drops here → count restored; no thread, no handshake + } + g + }; let root = root.to_path_buf(); - std::thread::spawn(move || { - if let Err(e) = handle_http_connection(stream, &root, read_only, mandatory_auth) { + let spawn_result = std::thread::Builder::new().spawn(move || { + let _guard = guard; // slot held for the connection's lifetime + if let Err(e) = handle_http_connection(stream, &root, read_only, mandatory_auth, limits) + { eprintln!("sc serve --http: connection error: {e}"); } }); + // OS thread creation can fail (e.g. EAGAIN under fd/thread exhaustion). + // `std::thread::spawn` PANICS in that case, which would kill this whole + // accept loop — exactly the "accept loop stays alive" property this + // phase exists to guarantee. `Builder::spawn` instead returns an + // `io::Result`; on `Err`, the closure we tried to spawn is dropped + // right here, which drops the moved-in `stream` and `SlotGuard` for us + // (closing the socket and freeing the slot) — the client just sees a + // plain close, not a 503, since writing a response would require + // restructuring the guard/stream ownership above. We shed with the + // same backoff used for accept() errors and keep looping. + if let Err(e) = spawn_result { + eprintln!("sc serve --http: thread spawn failed: {e}"); + std::thread::sleep(backoff.on_error()); + } } Ok(()) } @@ -518,14 +637,19 @@ pub fn serve_http_listener( /// /// The read timeout is set BEFORE `read_client_opening` (closing the /// slow-loris gap: the opening is bounded in bytes but not time) and -/// cleared again AFTER the 200 status is written and BEFORE handing off to -/// `wire::serve` — a legitimate large pack transfer must not be cut off -/// mid-stream by the same timeout that guards the opening. +/// REPLACED (P31, not simply cleared) by `limits.timeout_secs`'s session +/// timeout — on both the read and write sides — AFTER the 200 status is +/// written and BEFORE handing off to `wire::serve`: a legitimate large pack +/// transfer must not be cut off mid-transfer by the short opening timeout, +/// but a session that goes silent (or, on the write side, stalls reading a +/// streamed response) must still be reaped rather than parking the thread +/// forever. fn handle_http_connection( mut stream: TcpStream, root: &std::path::Path, server_read_only: bool, mandatory_auth: bool, + limits: ServeLimits, ) -> Result<()> { stream .set_read_timeout(Some(OPENING_READ_TIMEOUT)) @@ -591,15 +715,33 @@ fn handle_http_connection( write_status(&mut stream, 200)?; - // Clear the opening's read timeout before the wire protocol begins: a - // real streamed pack transfer can legitimately take longer than - // `OPENING_READ_TIMEOUT` and must not be timed out mid-transfer. + // P31: the opening timeout is REPLACED (not cleared) by the session + // timeout, on BOTH read and write sides — a write timeout covers a client + // that stops reading mid-GetPack. Under P25 chunking a per-syscall timeout + // is progress-based: only true zero-byte stalls trip it. 0 disables. + let session = if limits.timeout_secs == 0 { + None + } else { + Some(Duration::from_secs(limits.timeout_secs)) + }; + stream + .set_read_timeout(session) + .map_err(|e| Error::ConnectionLost(format!("sc+http set session read_timeout: {e}")))?; stream - .set_read_timeout(None) - .map_err(|e| Error::ConnectionLost(format!("sc+http clear read_timeout: {e}")))?; + .set_write_timeout(session) + .map_err(|e| Error::ConnectionLost(format!("sc+http set session write_timeout: {e}")))?; let read_only = server_read_only || token_read_only; - crate::wire::serve_with_policy(root, &mut reader, &mut stream, read_only) + crate::wire::serve_with_policy( + root, + &mut reader, + &mut stream, + crate::wire::WirePolicy { + read_only, + max_pack_size: limits.max_pack_size, + ro_drain_cap: crate::wire::RO_DRAIN_CAP, + }, + ) } #[cfg(test)] @@ -899,7 +1041,14 @@ mod tests { let listener = TcpListener::bind("127.0.0.1:0").unwrap(); let port = listener.local_addr().unwrap().port(); std::thread::spawn(move || { - serve_http_listener(listener, &root, read_only, mandatory_auth).unwrap(); + serve_http_listener( + listener, + &root, + read_only, + mandatory_auth, + ServeLimits::default(), + ) + .unwrap(); }); port } @@ -1058,7 +1207,7 @@ mod tests { // Non-loopback, no --read-only / --allow-public / tokens → refused, // and refused *before* any bind is attempted (port 0 would always // succeed to bind if we got that far). - let err = serve_http("0.0.0.0:0", &root, false, false).unwrap_err(); + let err = serve_http("0.0.0.0:0", &root, false, false, ServeLimits::default()).unwrap_err(); assert!( matches!(err, Error::InvalidArgument(_)), "public bind refused: {err:?}" @@ -1277,4 +1426,232 @@ mod tests { let _ = std::fs::remove_dir_all(&root); } + + // ── Task 4: AcceptBackoff + 503 busy status (pure parts). ── + + #[test] + fn accept_backoff_doubles_and_resets() { + let mut b = AcceptBackoff::new(); + assert_eq!(b.on_error(), Duration::from_millis(5)); + assert_eq!(b.on_error(), Duration::from_millis(10)); + assert_eq!(b.on_error(), Duration::from_millis(20)); + for _ in 0..20 { + b.on_error(); + } + assert_eq!(b.on_error(), Duration::from_secs(1)); // capped + b.on_success(); + assert_eq!(b.on_error(), Duration::from_millis(5)); // reset + } + + #[test] + fn status_503_round_trips() { + let mut buf = Vec::new(); + write_status(&mut buf, 503).unwrap(); + assert_eq!(read_status(&mut &buf[..]).unwrap(), 503); + } + + // ── Task 4: connection slots + 503, session timeouts (socket-level). ── + + fn tmp_served_root(tag: &str) -> std::path::PathBuf { + let root = std::env::temp_dir().join(format!("scl-http-{tag}-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&root); + std::fs::create_dir_all(root.join(".sc")).unwrap(); + root + } + + fn spawn_listener(root: &std::path::Path, limits: ServeLimits) -> std::net::SocketAddr { + let listener = TcpListener::bind("127.0.0.1:0").unwrap(); + let addr = listener.local_addr().unwrap(); + let root = root.to_path_buf(); + std::thread::spawn(move || { + let _ = serve_http_listener(listener, &root, false, false, limits); + }); + addr + } + + fn open_ok(addr: std::net::SocketAddr) -> TcpStream { + let mut s = TcpStream::connect(addr).unwrap(); + write_client_opening(&mut s, "127.0.0.1", "/", None).unwrap(); + assert_eq!(read_status(&mut s).unwrap(), 200); + s + } + + #[test] + fn connection_limit_shed_and_recover() { + let root = tmp_served_root("slots"); + let limits = ServeLimits { + max_connections: 1, + timeout_secs: 0, + ..Default::default() + }; + let addr = spawn_listener(&root, limits); + + let held = open_ok(addr); // occupies the single slot + + // Second connection: shed with 503 before any opening is read. + let mut second = TcpStream::connect(addr).unwrap(); + assert_eq!(read_status(&mut second).unwrap(), 503); + + // Free the slot; a fresh connection now succeeds (poll: the server + // notices the disconnect on its next read). + drop(held); + let ok = (0..50).any(|_| { + std::thread::sleep(Duration::from_millis(100)); + let mut s = match TcpStream::connect(addr) { + Ok(s) => s, + Err(_) => return false, + }; + write_client_opening(&mut s, "127.0.0.1", "/", None).is_ok() + && read_status(&mut s).ok() == Some(200) + }); + assert!(ok, "slot was never freed"); + let _ = std::fs::remove_dir_all(&root); + } + + #[test] + fn errored_connection_frees_its_slot() { + let root = tmp_served_root("errslot"); + let limits = ServeLimits { + max_connections: 1, + timeout_secs: 0, + ..Default::default() + }; + let addr = spawn_listener(&root, limits); + + // A connection whose handler errors (garbage opening → 400) must free + // its slot via the guard, not leak it. + let mut bad = TcpStream::connect(addr).unwrap(); + bad.write_all(b"garbage\r\n\r\n").unwrap(); + let _ = read_status(&mut bad); // 400 (best-effort) + drop(bad); + + let ok = (0..50).any(|_| { + std::thread::sleep(Duration::from_millis(100)); + let mut s = match TcpStream::connect(addr) { + Ok(s) => s, + Err(_) => return false, + }; + write_client_opening(&mut s, "127.0.0.1", "/", None).is_ok() + && read_status(&mut s).ok() == Some(200) + }); + assert!(ok, "errored connection leaked its slot"); + let _ = std::fs::remove_dir_all(&root); + } + + /// A client that stops READING mid-GetPack must be reaped by the write + /// timeout (today it parks the server thread in a blocking write forever). + /// Server sends a pack big enough to overflow socket buffers; the client + /// never reads; after the timeout the server closes. + /// + /// Why it cannot flake: the server must reap the connection within ~1s + /// through ONE OF TWO PATHS. (a) If the ~4 MiB GetPack response overflows + /// the loopback socket buffers, the server blocks in `write` and the 1s + /// WRITE timeout fires. (b) If the kernel buffers absorb the whole + /// response, the server finishes writing, returns to its request loop, + /// blocks in `read_frame_opt` on the silent client, and the 1s READ + /// timeout fires. Either way the client's drain loop observes EOF/reset + /// well inside its 5s margins. + #[test] + fn write_side_stall_is_reaped_by_timeout() { + let root = std::env::temp_dir().join(format!("scl-http-wstall-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&root); + std::fs::create_dir_all(&root).unwrap(); + crate::repo::Repo::init(&root).unwrap(); + // A few MiB of committed content, so GetPack has real bytes to send — + // enough (and high-entropy enough to resist the pack's zstd + // compression) to overflow socket send/receive buffers and force the + // server's write to actually block once the client stops reading. + // A forced newline every 8 bytes keeps every "line" well under the + // scanner's B64_MIN_RUN (20 chars), so this pseudo-random content + // never trips the P5 commit-time secret scan. + let mut big = Vec::with_capacity(4 * 1024 * 1024); + let mut state: u64 = 0x243F_6A88_85A3_08D3; + while big.len() < 4 * 1024 * 1024 { + state ^= state << 13; + state ^= state >> 7; + state ^= state << 17; + big.extend_from_slice(&state.to_le_bytes()); + big.push(b'\n'); + } + std::fs::write(root.join("big.bin"), &big).unwrap(); + let repo = crate::repo::Repo::open(&root).unwrap(); + let tip = repo.commit("t", "big file").unwrap(); + drop(repo); + + let limits = ServeLimits { + max_connections: 0, + timeout_secs: 1, + ..Default::default() + }; + let addr = spawn_listener(&root, limits); + + let mut s = open_ok(addr); + // Speak just enough wire protocol to trigger a GetPack, then stop + // reading: HELLO handshake, then a GetPack for the branch tip with + // empty haves. + crate::wire::write_frame( + &mut s, + &crate::wire::Request::Hello { + version: crate::wire::PROTOCOL_VERSION, + } + .encode(), + ) + .unwrap(); + // Read (and discard) the HELLO reply frame so the request stream + // stays in sync before the GetPack request. + let _ = crate::wire::read_frame(&mut s).unwrap(); + crate::wire::write_frame( + &mut s, + &crate::wire::Request::GetPack { + wants: vec![tip], + haves: vec![], + filter: vec![], + } + .encode(), + ) + .unwrap(); + // After sending the GetPack request, sleep instead of reading — + // never drain the response, so the server's write eventually blocks. + std::thread::sleep(Duration::from_secs(5)); + // The server must have dropped the connection rather than blocking + // forever: our next read eventually sees EOF or a reset. + s.set_read_timeout(Some(Duration::from_secs(5))).unwrap(); + let mut sink = [0u8; 4096]; + loop { + match s.read(&mut sink) { + Ok(0) | Err(_) => break, // EOF or reset — server gave up: pass + Ok(_) => continue, // drain what was already buffered + } + } + // Spool dir is clean. + let tmp = root.join(".sc").join("tmp"); + assert!(!tmp.exists() || std::fs::read_dir(&tmp).unwrap().next().is_none()); + let _ = std::fs::remove_dir_all(&root); + } + + #[test] + fn silent_session_is_reaped_by_timeout() { + let root = tmp_served_root("reap"); + let limits = ServeLimits { + max_connections: 0, + timeout_secs: 1, + ..Default::default() + }; + let addr = spawn_listener(&root, limits); + + let mut s = open_ok(addr); // handshake never sent; go silent + std::thread::sleep(Duration::from_secs(3)); + // Server must have dropped us: the read sees EOF/reset, not a hang. + s.set_read_timeout(Some(Duration::from_secs(5))).unwrap(); + let mut buf = [0u8; 1]; + match s.read(&mut buf) { + Ok(0) => {} // clean EOF — dropped + Err(_) => {} // reset — also dropped + Ok(_) => panic!("server sent data to a silent client"), + } + // Spool dir is clean. + let tmp = root.join(".sc").join("tmp"); + assert!(!tmp.exists() || std::fs::read_dir(&tmp).unwrap().next().is_none()); + let _ = std::fs::remove_dir_all(&root); + } } diff --git a/crates/repo/src/stdio_transport.rs b/crates/repo/src/stdio_transport.rs index 0ccc3d5f..a705e4f9 100644 --- a/crates/repo/src/stdio_transport.rs +++ b/crates/repo/src/stdio_transport.rs @@ -117,7 +117,7 @@ impl Transport for WireClient { )?; let frame = wire::read_frame(&mut rw.0)?; wire::parse_response(frame)?; // Ok(empty body) means "stream follows"; Err propagates typed - wire::read_pack_stream(&mut rw.0, out)?; + wire::read_pack_stream(&mut rw.0, out, 0)?; Ok(()) } @@ -563,7 +563,7 @@ mod tests { wire::Request::UpdateRef { .. } => return, // die without replying wire::Request::PutPack => { let mut buf = Vec::new(); - wire::read_pack_stream(&mut server_read, &mut buf).unwrap(); + wire::read_pack_stream(&mut server_read, &mut buf, 0).unwrap(); let ids = t.put_pack(&mut &buf[..]).unwrap(); wire::write_ok(&mut server_write, &wire::ids_body(&ids)).unwrap(); } diff --git a/crates/repo/src/wire.rs b/crates/repo/src/wire.rs index e81d15ad..831e79a1 100644 --- a/crates/repo/src/wire.rs +++ b/crates/repo/src/wire.rs @@ -79,6 +79,7 @@ const EC_PROTOCOL: u8 = 4; /// P29: the server is enforcing a read-only serve-token policy and refused a /// mutating verb before any store write. See [`serve_with_policy`]. const EC_READONLY: u8 = 5; +pub(crate) const EC_TOO_LARGE: u8 = 6; const EC_OTHER: u8 = 255; /// A client request — the wire mirror of one [`crate::transport::Transport`] @@ -367,7 +368,7 @@ fn read_frame_inner(r: &mut impl Read) -> Result>> { } let mut buf = vec![0u8; len]; r.read_exact(&mut buf) - .map_err(|_| Error::Protocol(format!("EOF inside {len}-byte frame body")))?; + .map_err(|e| Error::Protocol(format!("read inside {len}-byte frame body failed: {e}")))?; Ok(Some(buf)) } @@ -427,15 +428,30 @@ fn read_up_to(src: &mut (impl Read + ?Sized), buf: &mut [u8]) -> Result { /// Returns the total number of bytes streamed. Any frame that is neither a /// well-formed chunk nor the end marker — including EOF before `ST_PACK_END` /// — is `Err(Error::Protocol(_))`. -pub fn read_pack_stream(r: &mut impl Read, sink: &mut (impl Write + ?Sized)) -> Result { +/// +/// `max_bytes` bounds the total bytes written to `sink`: if `max_bytes != 0` +/// and a chunk would exceed the budget, aborts with `Error::PackTooLarge` +/// *before* writing the chunk. `max_bytes == 0` means unlimited. +pub fn read_pack_stream( + r: &mut impl Read, + sink: &mut (impl Write + ?Sized), + max_bytes: u64, +) -> Result { let mut total: u64 = 0; loop { let frame = read_frame_opt(r)?.ok_or_else(|| Error::Protocol("EOF before ST_PACK_END".into()))?; match frame.split_first() { Some((&ST_PACK_CHUNK, rest)) => { + let next = total + rest.len() as u64; + if max_bytes != 0 && next > max_bytes { + return Err(Error::PackTooLarge(format!( + "server cap is {max_bytes} bytes; raise --max-pack-size on the \ + server or split the push" + ))); + } sink.write_all(rest)?; - total += rest.len() as u64; + total = next; } Some((&ST_PACK_END, [])) => return Ok(total), _ => return Err(Error::Protocol("unexpected frame in pack stream".into())), @@ -472,7 +488,7 @@ pub fn parse_response(frame: Vec) -> Result> { let code = c.u8()?; let msg = c.str()?; c.done()?; - Err(err_from_wire(code, msg)) + Err(wire_to_err(code, msg)) } s => Err(Error::Protocol(format!("bad response status {s}"))), } @@ -480,6 +496,14 @@ pub fn parse_response(frame: Vec) -> Result> { /// Map a repo error onto its wire code + message. Errors sync logic matches on /// keep their identity; the rest carry only their message. +/// +/// `Error::PackTooLarge(inner)` is special-cased: its `Display` impl already +/// prepends the "pack exceeds the server's --max-pack-size limit: " prefix +/// around `inner`, and [`wire_to_err`] reconstructs a fresh +/// `Error::PackTooLarge(msg)` from whatever we send here — sending +/// `e.to_string()` would carry that prefix as part of `msg`, so the +/// round-tripped error's `Display` would show the prefix twice. Sending only +/// `inner` keeps the prefix appearing exactly once, on display. pub fn err_to_wire(e: &Error) -> (u8, String) { let code = match e { Error::NotARepo => EC_NOT_A_REPO, @@ -487,19 +511,25 @@ pub fn err_to_wire(e: &Error) -> (u8, String) { Error::Core(scl_core::Error::NotFound(_)) => EC_NOT_FOUND, Error::Protocol(_) => EC_PROTOCOL, Error::ReadOnly => EC_READONLY, + Error::PackTooLarge(_) => EC_TOO_LARGE, _ => EC_OTHER, }; - (code, e.to_string()) + let msg = match e { + Error::PackTooLarge(inner) => inner.clone(), + _ => e.to_string(), + }; + (code, msg) } /// Reconstruct a typed error from its wire code; unknown/untyped codes become /// `Error::Remote(msg)` so the message is never lost. -pub fn err_from_wire(code: u8, msg: String) -> Error { +pub fn wire_to_err(code: u8, msg: String) -> Error { match code { EC_NOT_A_REPO => Error::NotARepo, EC_NON_FAST_FORWARD => Error::NonFastForward, EC_PROTOCOL => Error::Protocol(msg), EC_READONLY => Error::ReadOnly, + EC_TOO_LARGE => Error::PackTooLarge(msg), _ => Error::Remote(msg), // EC_NOT_FOUND + EC_OTHER: typed enough as text } } @@ -590,6 +620,59 @@ pub fn decode_refs_body(b: &[u8]) -> Result> { use crate::transport::{LocalTransport, Transport}; +/// Default cap on an incoming `PutPack` spool when no operator override is +/// configured (P31): 16 GiB. Threaded through [`WirePolicy::max_pack_size`]; +/// `0` means unlimited. +pub const DEFAULT_MAX_PACK_SIZE: u64 = 16 * 1024 * 1024 * 1024; + +/// Cap on how much of a read-only connection's rejected `PutPack` stream is +/// drained to disk before the connection is closed rather than kept in sync +/// (P31): 8 MiB. A read-only push is going to be thrown away regardless, so +/// there is no reason to spool an attacker-sized pack just to reject it — +/// unlike the normal `PutPack` arm, which must accept up to +/// [`WirePolicy::max_pack_size`] because that pack might actually be used. +pub const RO_DRAIN_CAP: u64 = 8 * 1024 * 1024; + +/// Per-connection policy for [`serve_with_policy`] (P31): read-only gating +/// (P29) plus the two spool caps above. `Default` matches every pre-P31 +/// caller's prior unbounded behavior for `read_only`/`ro_drain_cap` sizing +/// intent, but now bounds `max_pack_size` to [`DEFAULT_MAX_PACK_SIZE`] rather +/// than leaving `PutPack` spool growth unbounded. +#[derive(Debug, Clone, Copy)] +pub struct WirePolicy { + pub read_only: bool, + /// Cap on a normal (non-read-only) `PutPack` spool, in bytes. `0` means + /// unlimited. + pub max_pack_size: u64, + /// Cap on how much of a read-only connection's rejected `PutPack` stream + /// is drained before closing the connection, in bytes. + pub ro_drain_cap: u64, +} + +impl Default for WirePolicy { + fn default() -> Self { + Self { + read_only: false, + max_pack_size: DEFAULT_MAX_PACK_SIZE, + ro_drain_cap: RO_DRAIN_CAP, + } + } +} + +/// Validate an operator-supplied `--max-pack-size` value. `0` (unlimited) is +/// always fine; a nonzero cap below [`scl_core::MAX_OBJECT_SIZE`] can never +/// fit even one object and is rejected as a misconfiguration rather than +/// silently failing every push at runtime. +pub fn validate_max_pack_size(max: u64) -> Result<()> { + if max != 0 && max < scl_core::MAX_OBJECT_SIZE as u64 { + return Err(Error::InvalidArgument(format!( + "--max-pack-size {max} is below MAX_OBJECT_SIZE ({}); a cap that cannot fit one object is a misconfiguration", + scl_core::MAX_OBJECT_SIZE + ))); + } + Ok(()) +} + /// Serve the repo at `root` to one wire-protocol peer until `Bye`/EOF. /// /// This is the whole server: every verb dispatches onto [`LocalTransport`], @@ -597,15 +680,17 @@ use crate::transport::{LocalTransport, Transport}; /// exactly as for a local remote. Verb failures are *replies*; only protocol /// violations end the session. /// -/// `read_only` (P29): when true, the three mutating verbs +/// `policy.read_only` (P29): when true, the three mutating verbs /// (`PutObject`/`PutPack`/`UpdateRef`) are rejected with [`Error::ReadOnly`] -/// BEFORE any store write; read verbs are always allowed. [`serve`] is the -/// `read_only = false` wrapper every pre-P29 caller uses unchanged. +/// BEFORE any store write; read verbs are always allowed. `policy.max_pack_size` +/// / `policy.ro_drain_cap` (P31) bound the `PutPack` spool in both the normal +/// and read-only-drain arms. [`serve`] is the `WirePolicy::default()` wrapper +/// every pre-P31 caller uses unchanged. pub fn serve_with_policy( root: &std::path::Path, r: &mut impl Read, w: &mut impl Write, - read_only: bool, + policy: WirePolicy, ) -> Result<()> { // Handshake: HELLO must come first, and versions must match, before any // repo access happens. @@ -657,7 +742,7 @@ pub fn serve_with_policy( }; // P29 read-only gate: reject mutating verbs before any store write. - if read_only { + if policy.read_only { match &req { Request::PutObject { .. } | Request::UpdateRef { .. } => { // Single request frames — nothing trails them; reject and @@ -674,12 +759,27 @@ pub fn serve_with_policy( // so no object reaches the store: "rejected before any // store write" holds. `spill_pack_stream` + guard drop is // the same bounded-RAM machinery the normal PutPack arm - // uses. - let guard = spill_pack_stream(r, transport.layout())?; - drop(guard); - let (code, msg) = err_to_wire(&Error::ReadOnly); - write_err(w, code, &msg)?; - continue; + // uses, bounded to `policy.ro_drain_cap` (P31) rather than + // the normal arm's larger `max_pack_size` — a read-only + // push is discarded regardless, so there's no reason to + // spool an attacker-sized pack just to reject it. + match spill_pack_stream(r, transport.layout(), policy.ro_drain_cap) { + Ok(guard) => { + drop(guard); + let (code, msg) = err_to_wire(&Error::ReadOnly); + write_err(w, code, &msg)?; + continue; + } + Err(Error::PackTooLarge(_)) => { + // Over the drain cap: the stream is desynced (we + // stopped reading mid-pack), so a best-effort + // reply is all we can do — then close. + let (code, msg) = err_to_wire(&Error::ReadOnly); + let _ = write_err(w, code, &msg); + return Ok(()); + } + Err(e) => return Err(e), + } } _ => {} } @@ -722,7 +822,7 @@ pub fn serve_with_policy( } } Request::PutPack => { - match spill_pack_stream(r, transport.layout()) { + match spill_pack_stream(r, transport.layout(), policy.max_pack_size) { Ok(guard) => { match transport.ingest_from(guard.path()) { Ok(ids) => write_ok(w, &ids_body(&ids))?, @@ -734,6 +834,17 @@ pub fn serve_with_policy( // `guard` drops here — temp pack file removed, // whether ingestion succeeded or failed. } + Err(e @ Error::PackTooLarge(_)) => { + // Over the cap: the stream is desynced (we stopped + // reading mid-pack, so any bytes still on the wire + // are neither read nor a subsequent request), so a + // best-effort reply is the most we can do — then + // close the connection rather than try to serve + // further requests off a desynced stream. + let (code, msg) = err_to_wire(&e); + let _ = write_err(w, code, &msg); + return Ok(()); + } Err(e) => { let (code, msg) = err_to_wire(&e); write_err(w, code, &msg)?; @@ -775,10 +886,12 @@ pub fn serve_with_policy( } } -/// Serve with full read/write access — the pre-P29 behavior. Every existing -/// caller (stdio transport, sync, tests) uses this unchanged. +/// Serve with full read/write access and default spool caps — the pre-P29 +/// behavior for read-only, now with `WirePolicy::default()`'s bounded +/// `max_pack_size`/`ro_drain_cap` (P31). Every existing caller (stdio +/// transport, sync, tests) uses this unchanged. pub fn serve(root: &std::path::Path, r: &mut impl Read, w: &mut impl Write) -> Result<()> { - serve_with_policy(root, r, w, false) + serve_with_policy(root, r, w, WirePolicy::default()) } /// Destream an incoming pack chunk stream (`ST_PACK_CHUNK`/`ST_PACK_END` @@ -786,14 +899,16 @@ pub fn serve(root: &std::path::Path, r: &mut impl Read, w: &mut impl Write) -> R /// fresh temp pack file, bounded to one chunk in RAM at a time. The guard is /// created before any read, so a stream that errors partway (a malformed /// frame, a dropped connection) still leaves nothing behind — `Drop` removes -/// whatever was written so far. +/// whatever was written so far. `max_bytes` bounds the spool (0 = unlimited, +/// P31) — see [`read_pack_stream`]. fn spill_pack_stream( r: &mut impl Read, layout: &crate::layout::Layout, + max_bytes: u64, ) -> Result { let guard = crate::transport::TempPackGuard::new(layout)?; let mut f = std::fs::File::create(guard.path())?; - read_pack_stream(r, &mut f)?; + read_pack_stream(r, &mut f, max_bytes)?; Ok(guard) } @@ -981,7 +1096,7 @@ mod tests { // Untyped errors become Remote(msg) and keep their message. let (code, msg) = err_to_wire(&crate::error::Error::Unborn); - let back = err_from_wire(code, msg); + let back = wire_to_err(code, msg); match back { crate::error::Error::Remote(m) => assert!(m.contains("unborn"), "message lost: {m}"), other => panic!("expected Remote, got {other:?}"), @@ -1152,7 +1267,7 @@ mod tests { let mut sink = Vec::new(); let mut r = std::io::Cursor::new(buf); - let total = read_pack_stream(&mut r, &mut sink).unwrap(); + let total = read_pack_stream(&mut r, &mut sink, 0).unwrap(); assert_eq!(total, data.len() as u64); assert_eq!(sink, data); } @@ -1166,7 +1281,7 @@ mod tests { let mut buf = Vec::new(); write_pack_stream(&mut buf, &mut std::io::Cursor::new(&data), chunk_size).unwrap(); let mut sink = Vec::new(); - let total = read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink).unwrap(); + let total = read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink, 0).unwrap(); assert_eq!(total, data.len() as u64); assert_eq!(sink, data, "mismatch for len={len}"); } @@ -1178,7 +1293,7 @@ mod tests { let mut buf = Vec::new(); write_pack_stream(&mut buf, &mut std::io::Cursor::new(&data), 7).unwrap(); let mut sink = Vec::new(); - let total = read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink).unwrap(); + let total = read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink, 0).unwrap(); assert_eq!(total, 0); assert!(sink.is_empty()); } @@ -1197,7 +1312,11 @@ mod tests { let (client_read, mut server_write) = std::io::pipe().unwrap(); let (mut server_read, client_write) = std::io::pipe().unwrap(); let handle = std::thread::spawn(move || { - serve_with_policy(&root, &mut server_read, &mut server_write, read_only) + let policy = WirePolicy { + read_only, + ..Default::default() + }; + serve_with_policy(&root, &mut server_read, &mut server_write, policy) }); let client = crate::stdio_transport::WireClient::handshake(client_read, client_write).unwrap(); @@ -1299,7 +1418,7 @@ mod tests { let mut buf = Vec::new(); write_frame(&mut buf, &[ST_PACK_CHUNK, b'a', b'b']).unwrap(); let mut sink = Vec::new(); - let err = read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink).unwrap_err(); + let err = read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink, 0).unwrap_err(); assert!( matches!(err, Error::Protocol(_)), "expected Protocol error, got {err:?}" @@ -1310,10 +1429,159 @@ mod tests { write_frame(&mut buf, &[ST_PACK_CHUNK, b'a']).unwrap(); write_frame(&mut buf, &[ST_OK]).unwrap(); // not a valid pack-stream marker let mut sink = Vec::new(); - let err = read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink).unwrap_err(); + let err = read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink, 0).unwrap_err(); assert!( matches!(err, Error::Protocol(_)), "expected Protocol error, got {err:?}" ); } + + #[test] + fn pack_stream_over_budget_aborts_typed() { + let payload = vec![7u8; 4096]; + let mut buf = Vec::new(); + write_pack_stream(&mut buf, &mut std::io::Cursor::new(&payload), 1024).unwrap(); + let mut sink = Vec::new(); + let err = read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink, 2048).unwrap_err(); + assert!(matches!(err, Error::PackTooLarge(_)), "got {err:?}"); + assert!( + sink.len() <= 2048, + "sink got over-budget bytes: {}", + sink.len() + ); + } + + #[test] + fn pack_stream_under_budget_passes() { + let payload = vec![7u8; 4096]; + let mut buf = Vec::new(); + write_pack_stream(&mut buf, &mut std::io::Cursor::new(&payload), 1024).unwrap(); + let mut sink = Vec::new(); + assert_eq!( + read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink, 4096).unwrap(), + 4096 + ); + } + + #[test] + fn pack_too_large_round_trips_through_wire_codes() { + let e = Error::PackTooLarge("16 GiB (17179869184 bytes)".to_string()); + let (code, msg) = err_to_wire(&e); + assert_eq!(code, EC_TOO_LARGE); + match wire_to_err(code, msg) { + Error::PackTooLarge(m) => { + assert!(m.contains("17179869184")); + // The Display prefix ("pack exceeds the server's + // --max-pack-size limit: ") must appear exactly once, not + // doubled by a round trip through the wire codes. + let display = Error::PackTooLarge(m).to_string(); + assert_eq!( + display.matches("pack exceeds").count(), + 1, + "expected exactly one occurrence of the Display prefix, got: {display}" + ); + } + other => panic!("expected PackTooLarge, got {other:?}"), + } + } + + /// A PutPack whose chunk stream exceeds policy.max_pack_size gets a + /// best-effort EC_TOO_LARGE reply and the connection closes (desync). + #[test] + fn putpack_over_cap_replies_too_large_and_closes() { + let root = tmp_repo("cap"); + let mut input = Vec::new(); + write_frame( + &mut input, + &Request::Hello { + version: PROTOCOL_VERSION, + } + .encode(), + ) + .unwrap(); + write_frame(&mut input, &Request::PutPack.encode()).unwrap(); + let payload = vec![9u8; 8192]; + write_pack_stream(&mut input, &mut std::io::Cursor::new(&payload), 1024).unwrap(); + // A trailing request that must never be answered (connection closed): + write_frame(&mut input, &Request::HeadBranch.encode()).unwrap(); + + let mut reader = std::io::Cursor::new(input); + let mut output = Vec::new(); + let policy = WirePolicy { + read_only: false, + max_pack_size: 4096, + ro_drain_cap: RO_DRAIN_CAP, + }; + serve_with_policy(&root, &mut reader, &mut output, policy).unwrap(); + + let mut frames = Vec::new(); + let mut r = std::io::Cursor::new(output); + while let Some(f) = read_frame_opt(&mut r).unwrap() { + frames.push(parse_response(f)); + } + assert_eq!( + frames.len(), + 2, + "hello-ok + too-large error only, got {frames:?}" + ); + assert!(frames[0].is_ok()); + assert!(matches!( + frames[1].as_ref().unwrap_err(), + Error::PackTooLarge(_) + )); + // Zero spool residue: + let tmp = root.join(".sc").join("tmp"); + assert!(!tmp.exists() || std::fs::read_dir(&tmp).unwrap().next().is_none()); + let _ = std::fs::remove_dir_all(&root); + } + + #[test] + fn validate_max_pack_size_floor() { + assert!(validate_max_pack_size(0).is_ok()); + assert!(validate_max_pack_size(scl_core::MAX_OBJECT_SIZE as u64).is_ok()); + assert!(validate_max_pack_size(1024).is_err()); + } + + /// An oversized push on a read-only connection drains at most ro_drain_cap + /// bytes, then the connection closes; zero spool residue. + #[test] + fn readonly_oversized_push_is_dropped_after_drain_cap() { + let root = tmp_repo("rocap"); + let mut input = Vec::new(); + write_frame( + &mut input, + &Request::Hello { + version: PROTOCOL_VERSION, + } + .encode(), + ) + .unwrap(); + write_frame(&mut input, &Request::PutPack.encode()).unwrap(); + let payload = vec![9u8; 8192]; + write_pack_stream(&mut input, &mut std::io::Cursor::new(&payload), 1024).unwrap(); + write_frame(&mut input, &Request::HeadBranch.encode()).unwrap(); // must never be answered + + let mut reader = std::io::Cursor::new(input); + let mut output = Vec::new(); + let policy = WirePolicy { + read_only: true, + max_pack_size: 0, + ro_drain_cap: 4096, + }; + serve_with_policy(&root, &mut reader, &mut output, policy).unwrap(); + + let mut frames = Vec::new(); + let mut r = std::io::Cursor::new(output); + while let Some(f) = read_frame_opt(&mut r).unwrap() { + frames.push(parse_response(f)); + } + assert_eq!(frames.len(), 2); + assert!(matches!( + frames[1].as_ref().unwrap_err(), + Error::ReadOnly | Error::Remote(_) + )); + let tmp = root.join(".sc").join("tmp"); + assert!(!tmp.exists() || std::fs::read_dir(&tmp).unwrap().next().is_none()); + let _ = std::fs::remove_dir_all(&root); + } } diff --git a/demo/run_limits_demo.sh b/demo/run_limits_demo.sh new file mode 100755 index 00000000..bde0259e --- /dev/null +++ b/demo/run_limits_demo.sh @@ -0,0 +1,209 @@ +#!/usr/bin/env bash +# P31 listener-limits demo: `sc serve --http`'s three new operator knobs — +# `--max-connections` (503-style busy shed at the limit), `--timeout` +# (session idle read+write timeout), and `--max-pack-size` (floor: +# MAX_OBJECT_SIZE, 256 MiB) — layered on P26/P29's `sc serve --http`. +# +# Honesty note on scope: a connection that never sends its opening is +# reaped by the FIXED 30s opening timeout, not by `--timeout` (which +# governs the post-opening session). This script cannot cheaply hold a +# post-opening session silent without a wire client, so it proves +# `--timeout` end-to-end indirectly: it frees a busy slot by closing the +# holder's fd rather than waiting out any timer, and says so explicitly. +# The 30s opening-reap path and the mid-stream pack-size abort are both +# covered by unit tests in crates/repo/src/http_transport.rs — this script +# proves the CLI knobs parse, take effect, and the busy/free lifecycle +# behaves end to end. +# +# Self-checking: every claim is an assertion; any failure exits non-zero +# before the RESULT line. +set -euo pipefail + +cd "$(dirname "$0")/.." +cargo build --quiet --bin sc +SC="$PWD/target/debug/sc" + +fail() { echo "FAIL: $1"; exit 1; } + +W="$(mktemp -d)" +A="$W/A" +SERVER_PID="" + +cleanup() { + if [ -n "$SERVER_PID" ] && kill -0 "$SERVER_PID" 2>/dev/null; then + kill "$SERVER_PID" 2>/dev/null || true + wait "$SERVER_PID" 2>/dev/null || true + fi + # Close the held raw connection if it's still open. + exec 3<&- 2>/dev/null || true + rm -rf "$W" +} +trap cleanup EXIT + +pick_port() { + local candidate + for candidate in 8751 8752 8753 8911 18751 18752; do + if ! nc -z 127.0.0.1 "$candidate" 2>/dev/null; then + echo "$candidate" + return 0 + fi + done + fail "no free port found among the candidates" +} + +wait_for_port() { + local port="$1" tries=0 + while ! nc -z 127.0.0.1 "$port" 2>/dev/null; do + tries=$((tries + 1)) + [ "$tries" -lt 100 ] || fail "server on port $port never became ready" + sleep 0.05 + done +} + +check_no_tmp_residue() { + local dir="$1" label="$2" + [ -d "$dir/.sc/tmp" ] && [ -n "$(ls -A "$dir/.sc/tmp" 2>/dev/null)" ] \ + && fail "$label: .sc/tmp is not empty" + return 0 +} + +echo "=== 1: init origin A, one commit ===" +mkdir -p "$A"; cd "$A" +"$SC" init >/dev/null +printf 'v1\n' > tracked.txt +"$SC" commit -m "initial" --author origin >/dev/null +echo "A: initial commit ✔" + +echo +echo "=== 2: sc serve --http 127.0.0.1:PORT --max-connections 1 --timeout 2 --max-pack-size 268435456 (background) ===" +cd "$W" +PORT="$(pick_port)" +"$SC" serve --http "127.0.0.1:$PORT" --max-connections 1 --timeout 2 \ + --max-pack-size 268435456 "$A" & +SERVER_PID=$! +wait_for_port "$PORT" +echo "server: listening on 127.0.0.1:$PORT (pid $SERVER_PID), max-connections=1 ✔" + +URL="sc+http://127.0.0.1:$PORT/repo" + +echo +echo "=== 3: busy shed — hold the one connection slot, a second clone is rejected ===" +# exec opens a raw TCP connection and holds it; the slot is taken at +# accept() time, before the server ever reads an opening from it. +exec 3<>"/dev/tcp/127.0.0.1/$PORT" +# The shed path writes the 503 status and closes without ever reading the +# probing client's opening bytes. +# +# Primary, deterministic proof: a bare second TCP connection that (like the +# held one) sends nothing. The shed path never reads from it — it writes the +# 503 status and closes cleanly — so this probe is not subject to any +# reset race and always observes a clean "503 Service Unavailable". +exec 4<>"/dev/tcp/127.0.0.1/$PORT" +# Bounded read (read -t), not a bare `cat`: a regressed server that accepts +# but stalls instead of writing the 503 must fail this step within 10s, not +# hang until the 30s opening reap kicks it loose. +PROBE_RESP="" +IFS= read -r -t 10 PROBE_RESP <&4 || true +exec 4<&- +echo "$PROBE_RESP" | grep -q "503 Service Unavailable" \ + || fail "expected a 503 Service Unavailable status on the second connection while the slot is held, got: $PROBE_RESP" +echo "busy: raw second connection shed with '503 Service Unavailable' while the slot is held ✔" + +# Secondary, best-effort proof with the real client: `sc clone` writes its +# opening before reading the status, so the server's close-with-unread-data +# on the shed path occasionally races that clean 503 read into a raw +# "connection reset" instead (a benign TCP quirk of the shed path, not a +# wire-protocol bug — either outcome still proves the connection was +# rejected while the slot was held, since a free slot always completes a +# clean clone, as step 4 demonstrates). Accept either textual outcome. +if "$SC" clone "$URL" "$W/c-busy" 2>"$W/err-busy.txt"; then + fail "clone should have been rejected while the one slot is held" +fi +[ ! -e "$W/c-busy" ] || fail "rejected clone should not have created a destination" +grep -qi "server busy\|connection reset\|connection lost" "$W/err-busy.txt" \ + || fail "expected 'server busy' or a benign reset while the slot is held, got: $(cat "$W/err-busy.txt")" +echo "busy: sc clone also rejected while the slot is held ($(cat "$W/err-busy.txt")) ✔" + +echo +echo "=== 4: slot auto-freed — closing the held connection frees it for a fresh clone ===" +# --timeout governs a SILENT connection AFTER its opening; a raw held fd +# that never sends an opening is instead reaped by the fixed 30s opening +# timeout (not this flag) -- too slow to wait out in a demo. We prove the +# busy/free lifecycle end to end by closing the holder ourselves: the slot +# frees immediately and the next clone succeeds. (The 30s opening-timeout +# reap path itself is proven by +# crates/repo/src/http_transport.rs::silent_session_is_reaped_by_timeout +# and its opening-phase sibling, not by this script.) +exec 3<&- +"$SC" clone "$URL" "$W/c-free" >/dev/null +[ "$(cat "$W/c-free/tracked.txt")" = "v1" ] || fail "c-free: clone content mismatch" +echo "freed: slot released once the holder disconnects; clone now succeeds ✔" +echo "(--timeout's post-opening session-idle reap is covered by the http_transport unit tests, not this script)" + +echo +echo "=== 5: stop the limited server ===" +kill "$SERVER_PID" +wait "$SERVER_PID" 2>/dev/null || true +SERVER_PID="" +! nc -z 127.0.0.1 "$PORT" 2>/dev/null || fail "server still accepting connections after kill" +echo "server: stopped ✔" + +echo +echo "=== 6: --max-pack-size floor — a small push lands under the cap; below the floor refuses to start ===" +PORT2="$(pick_port)" +"$SC" serve --http "127.0.0.1:$PORT2" --max-pack-size 268435456 "$A" & +SERVER_PID=$! +wait_for_port "$PORT2" +echo "server: started with --max-pack-size at the floor (268435456) ✔" + +URL2="sc+http://127.0.0.1:$PORT2/repo" +"$SC" clone "$URL2" "$W/c-floor" >/dev/null +[ "$(cat "$W/c-floor/tracked.txt")" = "v1" ] || fail "c-floor: clone content mismatch" +echo "clone through a floor-capped server: succeeds ✔" + +# The cap gates the PutPack spool (the server's receive path) — a clone's +# GetPack never consults it. Exercise the actually-gated path: a small +# commit pushed THROUGH the capped server must land (its pack is far under +# the 256 MiB floor cap). If the cap's enforcement broke open-endedly or +# regressed to rejecting everything, this push is the step that notices. +cd "$W/c-floor" +printf 'v2 via capped push\n' > pushed.txt +"$SC" commit -m "push through the floor-capped server" --author capclient >/dev/null +"$SC" push origin >/dev/null \ + || fail "small push through the floor-capped server should have landed" +cd "$W" +"$SC" clone "$URL2" "$W/c-cap-verify" >/dev/null +[ "$(cat "$W/c-cap-verify/pushed.txt")" = "v2 via capped push" ] \ + || fail "c-cap-verify: pushed commit did not land through the capped server" +echo "small push through the floor-capped server: lands (PutPack spool path, under the cap) ✔" + +kill "$SERVER_PID" +wait "$SERVER_PID" 2>/dev/null || true +SERVER_PID="" +! nc -z 127.0.0.1 "$PORT2" 2>/dev/null || fail "second server still accepting connections after kill" + +if "$SC" serve --http "127.0.0.1:$PORT2" --max-pack-size 1024 "$A" 2>"$W/err-cap.txt"; then + fail "server should have refused to start with --max-pack-size below MAX_OBJECT_SIZE" +fi +grep -qi "max-pack-size" "$W/err-cap.txt" \ + || fail "expected a --max-pack-size floor error, got: $(cat "$W/err-cap.txt")" +! nc -z 127.0.0.1 "$PORT2" 2>/dev/null || fail "server bound despite refusing to start" +echo "server: --max-pack-size 1024 (below the 256 MiB floor) refused to start ✔" +echo "(the mid-stream pack-size abort itself is covered by unit tests, not this script)" + +echo +echo "=== 7: zero .sc/tmp residue on the served repo and every clone ===" +check_no_tmp_residue "$A" "origin A" +check_no_tmp_residue "$W/c-free" "c-free" +check_no_tmp_residue "$W/c-floor" "c-floor" +check_no_tmp_residue "$W/c-cap-verify" "c-cap-verify" +echo "origin + clones: zero .sc/tmp residue ✔" + +echo +echo "RESULT: sc serve --http listener limits — a connection at --max-connections 1" +echo "is shed with a 'server busy' error while a slot is held, the slot frees once" +echo "the holder disconnects and a fresh clone succeeds (the post-opening" +echo "--timeout reap itself is unit-tested, not demoed here), a small push lands" +echo "through the PutPack spool under the 256 MiB floor cap while a --max-pack-size" +echo "below the floor refuses to start the server, and zero .sc/tmp residue is" +echo "left anywhere. OK" diff --git a/docs/THREAT-MODEL.md b/docs/THREAT-MODEL.md index d802bdc4..e2165cc4 100644 --- a/docs/THREAT-MODEL.md +++ b/docs/THREAT-MODEL.md @@ -28,6 +28,18 @@ too, not just the src-control-side metadata. wrappers can observe the plaintext through the child environment; this is fundamental to env-var injection. The parent's intermediate buffer is zeroized (best-effort), the child copy is not. +- **Re-affirmed (map ticket #31): fd/stdin injection stays deferred.** The + security claim for a non-env injection mode is mostly illusory against the + named adversary — a same-user attacker who can read the identity key file + and `.sc/` directly decrypts everything regardless of injection mechanism; + what fd/stdin would actually remove is *accidental* propagation (env dumps, + inheritance by grandchildren), which is hygiene, not attack surface. Env + stays the default contract for `sc run`, `sc work --with-secrets`, and `sc + ws run --with-secrets` alike. Revisit triggers: real committed-secret use in + agent sessions, or an observed env-leak incident class (agents dumping env + into logs or committed files) — at which point an additive fd-pipe opt-in + mode is the presumptive shape, since deferring an *additional* mode carries + no lock-in cost. - **Rotation ≠ erasure:** `sc secret rotate` re-seals under a fresh DEK and cuts off future reads through the current registry, but the old ciphertext object remains reachable and decryptable by anyone who kept the old DEK. `sc secret @@ -63,6 +75,19 @@ too, not just the src-control-side metadata. into a different branch position (a signature binds identity to a snapshot *id*, not to a branch position). `amend`/`rebase`/`merge` results start **unsigned by design** — a new snapshot id is a new claim that must be re-signed. +- **Re-affirmed and sharpened (map ticket #32): replay is ref binding only.** A + snapshot's parents are hashed into its id, so a signature over the id already + binds identity to the snapshot **and its entire ancestry** — a signed snapshot + cannot be grafted under different parents without voiding the signature. + "Replayed elsewhere in history" overstates the residue; the accurate claim is + narrower: a legitimately-signed snapshot is **replayable to a ref tip within + its own history** — nothing stops a hostile remote from pointing a branch name + at an older or side-branch signed snapshot (a rollback/ref-swap), the same gap + git addresses with signed pushes or gittuf-class ref metadata. Deferred: a + gittuf-shaped signed ref attestation (freshness/monotonicity semantics, + verified at fetch/clone) is its own trust-model effort, not a corner of an + unrelated map. Revisit triggers: the first real multi-writer or hosted `sc + serve` deployment, or hosting sc repos on untrusted infrastructure. ## Agent session transcripts (`sc transcript`) — ADR-0038 @@ -76,7 +101,7 @@ too, not just the src-control-side metadata. scanner warns at attach time but never blocks; rotation of the underlying secret remedies). -## Network transport (`sc serve --http` / `sc+http://`) — ADR-0036/0040 +## Network transport (`sc serve --http` / `sc+http://`) — ADR-0036/0040/0041 - **No TLS.** `sc serve --http` is **plaintext**. When bearer-token auth is configured, the token crosses the wire in the clear — a public deployment MUST @@ -88,10 +113,46 @@ too, not just the src-control-side metadata. - **Minor pre-auth information leak:** a `404` (no repo) is written before the auth gate, so an unauthenticated client can distinguish "a repo is served here" (`401`) from "no repo" (`404`). No content is exposed. -- **Not proxy/CDN-safe** (raw post-opening protocol), and the server is unbounded - thread-per-connection with no idle-transfer watchdog — operational hardening - items tracked in [`ROADMAP.md`](../ROADMAP.md)'s Deferred section. -- The ssh:// transport delegates authentication entirely to ssh (ADR-0022). +- **Not proxy/CDN-safe** (raw post-opening protocol) — this remains open, deferred + in [`ROADMAP.md`](../ROADMAP.md). +- **Listener resource bounds — closed by P31 (ADR-0041).** The three operational- + hardening items this section used to track as open are now closed, plus a + fourth gap P31's own research pass first named: + - **Connection exhaustion:** an atomic connection counter enforces + `--max-connections` (default 32, matching `git-daemon`); at the limit a new + connection is accepted, immediately given a busy status at the pre-handshake + opening seam, and closed — no queuing, no unbounded thread spawn. + - **Idle/stalled-peer exhaustion:** `--timeout` (default 300s, 0 disables) sets + read *and* write timeouts on the session socket for its whole lifetime — not + just the opening's original 30s, which used to be cleared once the wire + handoff began. A trip is connection-fatal (frame desync precludes recovery); + the spooled temp pack is guard-cleaned on the resulting unwind. + - **Accept-loop hot-spin under fd exhaustion:** a hardcoded exponential backoff + (5ms doubling to a 1s cap, reset on the next successful accept — Go + `net/http`'s shape) paces retries around `EMFILE`/`ENFILE` instead of + busy-looping. + - **Aggregate pack-spool exhaustion (named and closed together, P31):** this + item was never listed here or in ADR-0036 before P31's research pass found + it. The incoming-pack spool used no aggregate size bound — only ADR-0039's + per-frame/per-record caps applied — so a single oversized or endless pack + could exhaust server disk on **either** transport. `--max-pack-size` + (default 16 GiB, 0 = unlimited, floor 256 MiB = `MAX_OBJECT_SIZE`) now + counts the running total and aborts mid-stream past the cap. This includes + the **read-only drain path**: the pre-`EC_READONLY` drain that keeps a + connection in sync when a read-only-scoped client streams a pack the server + is about to reject was itself unbounded — an `ro`-token client could still + write arbitrary bytes to `.sc/tmp` before being told no. It is now capped at + ~8 MiB (`RO_DRAIN_CAP`): an honest small misconfigured push still gets the + clean typed `EC_READONLY` error, while a larger bulk spool is dropped + mid-send instead of fully drained to disk. + - **Not closed by P31:** per-IP/per-token sub-limiting (a distributed + many-connections-from-many-sources attack below the per-listener cap is + unaddressed), and a legitimate transfer slower than a true 300s zero-byte + stall is unaffected by `--timeout`, but an operator on an unusually slow + network must raise it explicitly. See ADR-0041 for the full decision, + including rejected alternatives. +- The ssh:// transport delegates authentication entirely to ssh (ADR-0022); + `--max-pack-size` is the one P31 bound that also applies to `--stdio`. ## Untrusted-input hardening (DoS) — ADR-0039 diff --git a/docs/adr/0041-listener-resource-limits.md b/docs/adr/0041-listener-resource-limits.md new file mode 100644 index 00000000..1e83297d --- /dev/null +++ b/docs/adr/0041-listener-resource-limits.md @@ -0,0 +1,200 @@ +# ADR-0041: Listener resource limits + +- **Status:** Accepted +- **Date:** 2026-07-10 +- **Phase:** 31 +- **Builds on:** ADR-0036 (P26 sc-native HTTP transport — names three of the + four gaps closed here), ADR-0040 (P29 access control — the seam where a + busy/error status is already written before the wire handshake), ADR-0039 + (P28 DoS hardening — `MAX_OBJECT_SIZE`, composed with, not replaced by, + this phase's spool cap) +- **Research:** `docs/research/bounded-server-patterns.md` (ticket #27, + std-only bounded-server survey pinned to file:line). Decided at the + decision level via the wayfinder map, ticket #28. + +## Context + +ADR-0036 shipped `sc serve --http` with three accepted-but-open consequences: +unbounded thread-per-connection (no pool, cap, or backpressure), no +idle-transfer watchdog (the opening's 30s read timeout is cleared once +`wire::serve` takes over), and no accept-loop backoff (a sustained +`EMFILE`/`ENFILE` hot-spins `listener.incoming()`). The #27 research pass +found a fourth gap neither ADR-0036 nor `docs/THREAT-MODEL.md` had named: the +aggregate incoming-pack spool is unbounded, and the read-only rejection path +drains it too (`wire.rs`'s `EC_READONLY` arm spools the whole pack before +replying, so an `ro`-token client can still write arbitrary bytes to disk). +This phase closes all four, dep-free, with no wire-protocol change. + +## Decision + +`sc serve` gains three new flags plus two hardcoded internal behaviors, all +scoped to the listener (`--stdio` is unaffected except where noted): + +1. **`--max-connections ` (default 32, 0 = unlimited).** An atomic + live-connection counter (`SlotGuard`, RAII-decremented on every exit path + — clean return, error, panic unwind). At the limit, a new connection is + accepted, immediately written a busy status, and closed — at the same + pre-handshake HTTP-opening seam ADR-0040's `401`/`404` write to, but + *earlier* in it: ADR-0040's `401`/`404` are written only after reading + and parsing the client's opening (they respond to what the opening said), + while the `503` here is written before any read of the opening at all — + the connection is shed purely on the live-connection count, with the + opening never consulted. No queuing: for long-lived multi-verb wire + sessions, fail-fast beats an invisible wait. Default 32 matches + `git-daemon --max-connections`, the closest prior-art analog for a bare + TCP git-style server. + +2. **`--timeout ` (default 300, 0 = disabled).** Read *and* write + timeouts are set on the `TcpStream` once the session begins and persist + for the whole session, replacing (not merely preceding) the opening's + `OPENING_READ_TIMEOUT` — previously that 30s timeout was cleared outright + after the `200` status. A timeout trip is connection-fatal: mid-stream + abort desyncs the frame stream (no partial-frame recovery is possible), + so the connection is dropped rather than resumed; `TempPackGuard` cleans + any spooled pack file on the resulting unwind. Under P25 chunking, a + per-syscall timeout is inherently progress-based — only a true zero-byte + stall trips it, not a slow-but-moving transfer. The opening keeps its + existing, unchanged 30s. + +3. **`--max-pack-size ` (default 16 GiB, 0 = unlimited, floor 256 MiB + = `MAX_OBJECT_SIZE`).** Applies to **both** `--http` and `--stdio` — it + lives in `WirePolicy` (`crates/repo/src/wire.rs`), the shared layer both + transports call into, not in `ServeLimits` (the `--http`-only listener + config). A configured non-zero value below `MAX_OBJECT_SIZE` is rejected + as invalid config at startup. Enforcement is a counted mid-stream abort in + the pack-stream readers (mirroring git's `receive.maxInputSize`): once the + running total crosses the cap, the reader stops, a best-effort + `EC_TOO_LARGE` reply is written (best-effort because the abort already + desynced the stream — the reply is a courtesy, not a guarantee the client + parses it before the socket closes), and the connection is torn down. + This composes above, and never replaces, ADR-0039's existing per-frame + wire-length cap, per-record pack cap, and zstd decompression-bomb guard — + this is an *aggregate* bound across an entire incoming pack, those are + *per-unit* bounds within it. + +4. **Read-only drain cap — `RO_DRAIN_CAP` (8 MiB, hardcoded).** The + deliberate pre-`EC_READONLY` drain (draining keeps a wire connection in + sync when the client has already started streaming a pack the server is + about to reject) now stops at ~8 MiB. An honest misconfiguration — a + read-only-scoped client attempting a small legitimate push — still gets + the clean typed `EC_READONLY` error. A hostile or careless bulk spool + past that size is dropped mid-send instead of fully drained to disk, so + an `ro`-token connection can no longer be used to write unbounded bytes + to `.sc/tmp` regardless of the eventual rejection. This gap was never + named in ADR-0036 or the threat model before this phase's research pass + found it — P31 both names and closes it. + +5. **Accept-loop backoff (hardcoded, no flag).** Go `net/http`'s exact + shape: on an accept error, sleep starting at 5ms, doubling on each + consecutive error, capped at 1s, reset to 5ms on the next successful + accept. Turns fd exhaustion (`EMFILE`/`ENFILE`) from a busy-loop into a + paced retry. No operator knob — Go ships this knobless too, and no + tuning story exists yet to justify one. + +## Recorded divergences from prior art + +- **Timeout on by default, unlike `git-daemon`'s `--timeout` (default 0 / + disabled).** A defaults-off knob does not close an audit-tracked High — a + silent or write-stalled peer parking a server thread forever is exactly + the DoS shape this phase exists to close, and without a reaper the + connection cap (#1) is hostage to any single stalled client holding a + slot forever. 300s, not a shorter value, because the one legitimate + casualty of a shorter deadline is a client computing between verbs (e.g. + `sc ws run` doing real work mid-session). +- **Finite 16 GiB spool default, unlike git's unlimited + `receive.maxInputSize` (default 0).** `sc serve` deployments often share + the same volume as the working tree/`.sc/tmp` spool, and this phase's + posture is fail-closed defaults throughout rather than borrowing git's + unbounded default. + +## Rejected alternatives + +- **Block-at-accept semaphore (queue rather than shed at the connection + cap).** Rejected: a wire connection is a long-lived, many-verb session, + not a one-shot HTTP request — an invisible queued wait is worse UX than a + clear, immediately retryable busy status. +- **git-daemon's grace-then-shed pattern.** Rejected as unnecessary + complexity for this MVP; the immediate busy-status-and-close is simpler + and gives the client the same actionable signal sooner. +- **The Rust Book's fixed-size thread-pool-with-queue pattern.** Rejected + for the same reason as the semaphore: it trades a clear rejection for an + invisible queue, wrong shape for long-lived sessions. +- **A reserved-fd accept-loop shed (hold back N fds for cleanup work).** + Rejected: the connection cap (#1) already demotes fd exhaustion to + external pressure (e.g. other processes on the host) rather than a + self-inflicted condition; the refinement can ride a later hardening pass + if real-world churn ever proves the need. +- **Env-var or `.sc/config`-file knobs for these limits.** Rejected for + this phase: no `.sc/config` surface exists yet, and CLI flags are + consistent with every other `sc serve` option shipped so far + (`--read-only`, `--allow-public`, ADR-0040's token flags). A config-file + surface is a separate, larger decision not scoped here. +- **An sshd-`MaxStartups`-style pre-auth sub-cap distinct from the total + connection cap.** Deliberately deferred until real-world churn patterns + justify a second knob; one total cap is the simpler starting shape. + +## Consequences + +- ADR-0036's three named accepted-but-open consequences (unbounded + thread-per-connection, no idle-transfer watchdog, no accept-loop backoff) + are closed by this phase. +- The aggregate-spool gap — never named in ADR-0036 or the threat model, + surfaced only by the #27 research pass — is both named and closed, + including its read-only-drain-path variant. +- `PROTOCOL_VERSION` stays 3: none of the four bounds touch the wire + protocol. `EC_TOO_LARGE` degrades gracefully to opaque message text on an + old client that doesn't recognize the code, matching `EC_READONLY`'s + precedent from ADR-0040. +- Zero new dependencies. +- `--stdio` gains only the pack-size cap (shared `WirePolicy`); connection + count, session timeout, and accept backoff are `--http`-listener-specific + concepts that don't apply to a single ssh-spawned stdio session. +- Proven by `demo/run_limits_demo.sh` (the floor-cap enforcement on the push + path and the busy-status shed under `--max-connections`); the timeout + reaper and accept-backoff pacing are unit-test-proven rather than + demo-proven, since reliably forcing a real stalled-socket or fd-exhaustion + condition in a demo script is impractical. + +## Threat model honesty + +- **Defends:** thread/fd exhaustion from unbounded connection churn (cap + + shed); a stalled or hostile-slow peer holding a server thread forever + (session timeout, both directions); unbounded disk consumption from an + oversized or endless incoming pack, on both transports and including the + read-only-drain path (spool cap + drain cap); accept-loop hot-spin under + sustained fd exhaustion (backoff). +- **Does NOT defend:** a client operating within all four bounds but still + malicious in content (these are resource bounds, not content/auth + controls — see ADR-0040 for the access-control layer); distributed + multi-connection exhaustion below the per-listener cap (e.g. many + distinct source IPs each opening a few connections) — no per-IP or + per-token sub-limiting exists; a legitimate transfer that is simply + slower than 300s of true zero-byte stall is unaffected, but an operator + with a much slower network than anticipated must raise `--timeout` + explicitly. +- **Known boundary (a): the `503` write races the client's own send.** + Because the busy status is written *before* the opening is read at all + (see Decision #1's seam-timing note above), a client that is mid-send on + its opening when the slot check fails can observe the server's write + colliding with its own outstanding write, and the kernel/TCP stack may + surface that as a connection reset instead of a clean `503` response — + observed empirically at roughly 1-in-3 on macOS loopback in this phase's + own testing. Draining the client's opening before closing (so the `503` + always lands cleanly) was considered and rejected: it would reintroduce a + per-shed-connection read cost on precisely the path `--max-connections` + exists to keep cheap — an attacker driving the server into the shed state + could then force it to spend read cycles per rejected connection instead + of none. The `503` is best-effort, not guaranteed-delivered; a client must + already treat a bare reset as retryable, same as a real busy response. +- **Known boundary (b): `--max-connections 0` (unlimited) still exposes + thread exhaustion.** With the cap disabled, nothing prevents the accept + loop from spawning threads until the OS refuses (`EAGAIN` from + `pthread_create` or similar). Finding 1 of this phase's final-review wave + closed the crash mode this used to cause — `serve_http_listener` now uses + `std::thread::Builder::spawn` instead of the panicking `std::thread::spawn`, + so a failed spawn is logged, paced by the same `AcceptBackoff` used for + `accept()` errors, and the loop continues — but it does not close the + underlying exposure: an operator who explicitly disables the connection + cap is still trusting the OS's own thread/fd limits as the only backstop. + `--max-connections 0` is an explicit opt-out of this phase's shedding + behavior, not a claim that unlimited connections are safe. diff --git a/docs/adr/README.md b/docs/adr/README.md index ed1842e7..018bd416 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -51,6 +51,7 @@ supersedes the old one and update the **Status** line of the superseded record. | [0038](0038-agent-session-transcripts.md) | Agent session transcripts as CAS objects | Accepted | 30 | | [0039](0039-security-hardening-sweep.md) | Security hardening sweep (audit fix-now items) | Accepted | 28 | | [0040](0040-sc-http-access-control.md) | sc+http access control (read-only, fail-closed bind, bearer tokens) | Accepted | 29 | +| [0041](0041-listener-resource-limits.md) | Listener resource limits (connection cap, session timeout, spool cap, accept backoff) | Accepted | 31 | See [`ROADMAP.md`](../../ROADMAP.md) for the phase sequence. ADR numbers are assigned in creation order, so 0017 — the secret scanner — sequences as phase P5, diff --git a/docs/superpowers/plans/2026-07-10-p31-listener-limits.md b/docs/superpowers/plans/2026-07-10-p31-listener-limits.md new file mode 100644 index 00000000..b312b7da --- /dev/null +++ b/docs/superpowers/plans/2026-07-10-p31-listener-limits.md @@ -0,0 +1,789 @@ +# P31: HTTP Listener Resource Limits Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Bound `sc serve` against hostile/broken clients: connection cap with busy status, session read+write timeouts, aggregate pack-spool cap (both transports), read-only drain cap, accept-loop backoff. + +**Architecture:** A `ServeLimits` struct (CLI-built) threads into the HTTP listener; the wire layer's bare `read_only: bool` grows into `WirePolicy { read_only, max_pack_size, ro_drain_cap }`. Connection slots are an `Arc` with an RAII guard; the spool cap is a counted mid-stream abort in `read_pack_stream` surfacing as new wire code `EC_TOO_LARGE = 6` (old clients degrade to message text — no version bump). + +**Tech Stack:** Rust (std only — no new dependencies). Spec: `docs/superpowers/specs/2026-07-10-p31-listener-limits-design.md`. Issue: #38. + +## Global Constraints + +- **Zero new dependencies.** `PROTOCOL_VERSION` stays **3**. +- Defaults, copied from the spec: `--max-connections` **32**, `--timeout` **300** s, `--max-pack-size` **16 GiB** (`17_179_869_184`); `0` = unlimited/disabled for all three. RO drain cap: **8 MiB** constant, no knob. +- `--max-pack-size` values `> 0` but `< MAX_OBJECT_SIZE` (256 MiB, `scl_core::MAX_OBJECT_SIZE`) are rejected. +- `--max-connections`/`--timeout` are `--http`-only (refused with `--stdio`); `--max-pack-size` applies to both transports. +- Mid-stream spool abort is **connection-fatal** (stream desync): best-effort typed error, then close. +- Repo conventions: `RUSTFLAGS="-D warnings"` (CI), per-crate `thiserror`, doc comments state intent, tests clean up temp dirs and assert zero residue, `cargo fmt` before every commit. +- Commit messages end with: `Claude-Session: https://claude.ai/code/session_01LhadyW9scQL95h3ag9yySB` + +## File Structure + +- `crates/repo/src/error.rs` — two new variants: `PackTooLarge(String)`, `ServerBusy`. +- `crates/repo/src/wire.rs` — `EC_TOO_LARGE`, `WirePolicy`, `DEFAULT_MAX_PACK_SIZE`, `RO_DRAIN_CAP`, `validate_max_pack_size`, budgeted `read_pack_stream`/`spill_pack_stream`, both `PutPack` arms. +- `crates/repo/src/http_transport.rs` — `ServeLimits`, `SlotGuard`, `AcceptBackoff`, 503 status, session timeouts, client `ServerBusy` mapping. +- `crates/cli/src/main.rs` — three flags + validation + plumbing (`run_serve`). +- `demo/run_limits_demo.sh` — end-to-end proof. +- `docs/adr/0041-listener-resource-limits.md`, `docs/THREAT-MODEL.md`, `CLAUDE.md` — docs. + +--- + +### Task 1: Typed errors + wire code round trip + +**Files:** +- Modify: `crates/repo/src/error.rs` (append inside `pub enum Error`, near `ReadOnly`) +- Modify: `crates/repo/src/wire.rs` (EC constants block ~line 75; `err_to_wire` ~line 485; `wire_to_err` ~line 499; tests at bottom) + +**Interfaces:** +- Produces: `Error::PackTooLarge(String)` (payload = human limit text, e.g. `"16 GiB (17179869184 bytes)"`), `Error::ServerBusy`, `pub(crate) const EC_TOO_LARGE: u8 = 6`. Round trip: `err_to_wire(PackTooLarge) == (EC_TOO_LARGE, msg)`; `wire_to_err(EC_TOO_LARGE, msg) == PackTooLarge(msg)`. + +- [ ] **Step 1: Write the failing test** (in `wire.rs` `mod tests`) + +```rust +#[test] +fn pack_too_large_round_trips_through_wire_codes() { + let e = Error::PackTooLarge("16 GiB (17179869184 bytes)".to_string()); + let (code, msg) = err_to_wire(&e); + assert_eq!(code, EC_TOO_LARGE); + match wire_to_err(code, msg) { + Error::PackTooLarge(m) => assert!(m.contains("17179869184")), + other => panic!("expected PackTooLarge, got {other:?}"), + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cargo test -p scl-repo pack_too_large_round_trips -- --nocapture` +Expected: compile error — `PackTooLarge` not defined. + +- [ ] **Step 3: Implement** + +In `error.rs`, after the `ReadOnly` variant: + +```rust +/// The server aborted an incoming pack mid-stream because it exceeded the +/// operator's `--max-pack-size` cap (P31). Payload is the server's own +/// human-readable limit text, carried verbatim across the wire. +#[error("pack exceeds the server's --max-pack-size limit: {0}")] +PackTooLarge(String), +/// The server refused the connection at accept time because +/// `--max-connections` was reached (P31). Retryable. +#[error("server busy (connection limit reached); retry later")] +ServerBusy, +``` + +In `wire.rs`, EC block: + +```rust +const EC_TOO_LARGE: u8 = 6; +``` + +`err_to_wire`: add arm `Error::PackTooLarge(_) => EC_TOO_LARGE,` (message flows via the existing `e.to_string()` path). +`wire_to_err`: add arm `EC_TOO_LARGE => Error::PackTooLarge(msg),`. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cargo test -p scl-repo pack_too_large_round_trips` +Expected: PASS + +- [ ] **Step 5: Commit** + +```bash +cargo fmt --all && git add -A crates/repo && git commit -m "feat(wire): PackTooLarge/ServerBusy errors + EC_TOO_LARGE wire code (P31)" +``` + +--- + +### Task 2: Byte budget in `read_pack_stream` + +**Files:** +- Modify: `crates/repo/src/wire.rs:430` (`read_pack_stream`), `crates/repo/src/wire.rs:796` (`spill_pack_stream` call site), test call sites `wire.rs:1155,1169,1181,1302,1313` +- Modify: `crates/repo/src/stdio_transport.rs:120,566` (client + harness call sites) + +**Interfaces:** +- Consumes: `Error::PackTooLarge(String)` (Task 1). +- Produces: `pub fn read_pack_stream(r: &mut impl Read, sink: &mut (impl Write + ?Sized), max_bytes: u64) -> Result` — `max_bytes == 0` means unlimited; exceeding aborts with `Error::PackTooLarge` *before* writing the over-budget chunk to `sink`. + +- [ ] **Step 1: Write the failing tests** (in `wire.rs` `mod tests`; `write_pack_stream(w, r, chunk_size)` already exists) + +```rust +#[test] +fn pack_stream_over_budget_aborts_typed() { + let payload = vec![7u8; 4096]; + let mut buf = Vec::new(); + write_pack_stream(&mut buf, &mut std::io::Cursor::new(&payload), 1024).unwrap(); + let mut sink = Vec::new(); + let err = read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink, 2048).unwrap_err(); + assert!(matches!(err, Error::PackTooLarge(_)), "got {err:?}"); + assert!(sink.len() <= 2048, "sink got over-budget bytes: {}", sink.len()); +} + +#[test] +fn pack_stream_under_budget_passes() { + let payload = vec![7u8; 4096]; + let mut buf = Vec::new(); + write_pack_stream(&mut buf, &mut std::io::Cursor::new(&payload), 1024).unwrap(); + let mut sink = Vec::new(); + assert_eq!( + read_pack_stream(&mut std::io::Cursor::new(buf), &mut sink, 4096).unwrap(), + 4096 + ); +} +``` + +- [ ] **Step 2: Run to verify failure** + +Run: `cargo test -p scl-repo pack_stream_over_budget` +Expected: compile error — wrong arity. + +- [ ] **Step 3: Implement** + +Change the signature and add the check inside the chunk arm, before `sink.write_all`: + +```rust +pub fn read_pack_stream( + r: &mut impl Read, + sink: &mut (impl Write + ?Sized), + max_bytes: u64, +) -> Result { + let mut total: u64 = 0; + loop { + let frame = + read_frame_opt(r)?.ok_or_else(|| Error::Protocol("EOF before ST_PACK_END".into()))?; + match frame.split_first() { + Some((&ST_PACK_CHUNK, rest)) => { + let next = total + rest.len() as u64; + if max_bytes != 0 && next > max_bytes { + return Err(Error::PackTooLarge(format!("{max_bytes} bytes"))); + } + sink.write_all(rest)?; + total = next; + } + Some((&ST_PACK_END, [])) => return Ok(total), + _ => return Err(Error::Protocol("unexpected frame in pack stream".into())), + } + } +} +``` + +Update every existing call site to pass `0` (unlimited) for now: `wire.rs:796` (`spill_pack_stream` — Task 3 threads the real cap), `stdio_transport.rs:120` (client destream — client-side caps are a spec non-goal) and `:566` (test harness), and the five `wire.rs` tests. Update the doc comment: budget semantics + "0 = unlimited". + +- [ ] **Step 4: Run to verify pass** + +Run: `cargo test -p scl-repo pack_stream` +Expected: new tests PASS, existing pack-stream tests PASS. + +- [ ] **Step 5: Commit** + +```bash +cargo fmt --all && git add -A crates/repo && git commit -m "feat(wire): byte budget with typed mid-stream abort in read_pack_stream (P31)" +``` + +--- + +### Task 3: `WirePolicy` + capped spool in both `PutPack` arms + +**Files:** +- Modify: `crates/repo/src/wire.rs` — `serve_with_policy` (~line 604), `serve` wrapper (~line 780), `spill_pack_stream` (~line 790), RO drain arm (~line 669), normal `PutPack` arm (~line 724), the read-only test at ~line 1200 +- Modify: `crates/repo/src/http_transport.rs:602` (call site; real limits threaded in Task 5 — pass `WirePolicy { read_only, ..Default::default() }` for now) + +**Interfaces:** +- Consumes: budgeted `read_pack_stream` (Task 2), `EC_TOO_LARGE` (Task 1). +- Produces: + +```rust +pub const DEFAULT_MAX_PACK_SIZE: u64 = 16 * 1024 * 1024 * 1024; // 16 GiB +pub const RO_DRAIN_CAP: u64 = 8 * 1024 * 1024; // 8 MiB + +#[derive(Debug, Clone, Copy)] +pub struct WirePolicy { + pub read_only: bool, + pub max_pack_size: u64, // 0 = unlimited + pub ro_drain_cap: u64, +} +impl Default for WirePolicy { + fn default() -> Self { + Self { read_only: false, max_pack_size: DEFAULT_MAX_PACK_SIZE, ro_drain_cap: RO_DRAIN_CAP } + } +} + +pub fn validate_max_pack_size(max: u64) -> Result<()>; // 0 ok; else must be >= MAX_OBJECT_SIZE +pub fn serve_with_policy(root: &Path, r: &mut impl Read, w: &mut impl Write, policy: WirePolicy) -> Result<()>; +``` + +- [ ] **Step 1: Write the failing tests** (model on the existing `serve_with_policy` read-only test at ~line 1200; `some_id`, `tmp_repo`, `write_frame`, `parse_response` helpers exist) + +```rust +/// A PutPack whose chunk stream exceeds policy.max_pack_size gets a +/// best-effort EC_TOO_LARGE reply and the connection closes (desync). +#[test] +fn putpack_over_cap_replies_too_large_and_closes() { + let root = tmp_repo("cap"); + let mut input = Vec::new(); + write_frame(&mut input, &Request::Hello { version: PROTOCOL_VERSION }.encode()).unwrap(); + write_frame(&mut input, &Request::PutPack.encode()).unwrap(); + let payload = vec![9u8; 8192]; + write_pack_stream(&mut input, &mut std::io::Cursor::new(&payload), 1024).unwrap(); + // A trailing request that must never be answered (connection closed): + write_frame(&mut input, &Request::HeadBranch.encode()).unwrap(); + + let mut reader = std::io::Cursor::new(input); + let mut output = Vec::new(); + let policy = WirePolicy { read_only: false, max_pack_size: 4096, ro_drain_cap: RO_DRAIN_CAP }; + serve_with_policy(&root, &mut reader, &mut output, policy).unwrap(); + + let mut frames = Vec::new(); + let mut r = std::io::Cursor::new(output); + while let Some(f) = read_frame_opt(&mut r).unwrap() { + frames.push(parse_response(f)); + } + assert_eq!(frames.len(), 2, "hello-ok + too-large error only, got {frames:?}"); + assert!(frames[0].is_ok()); + assert!(matches!(frames[1].as_ref().unwrap_err(), Error::PackTooLarge(_))); + // Zero spool residue: + let tmp = root.join(".sc").join("tmp"); + assert!(!tmp.exists() || std::fs::read_dir(&tmp).unwrap().next().is_none()); + let _ = std::fs::remove_dir_all(&root); +} + +#[test] +fn validate_max_pack_size_floor() { + assert!(validate_max_pack_size(0).is_ok()); + assert!(validate_max_pack_size(scl_core::MAX_OBJECT_SIZE as u64).is_ok()); + assert!(validate_max_pack_size(1024).is_err()); +} +``` + +- [ ] **Step 2: Run to verify failure** + +Run: `cargo test -p scl-repo putpack_over_cap` +Expected: compile error — `WirePolicy` not defined. + +- [ ] **Step 3: Implement** + +1. Add the constants, `WirePolicy`, and `validate_max_pack_size` (error: `Error::InvalidArgument(format!("--max-pack-size {max} is below MAX_OBJECT_SIZE ({}); a cap that cannot fit one object is a misconfiguration", scl_core::MAX_OBJECT_SIZE))`). +2. `serve_with_policy` takes `policy: WirePolicy`; replace internal `read_only` reads with `policy.read_only`. `serve` wrapper: `serve_with_policy(root, r, w, WirePolicy::default())`. +3. `spill_pack_stream(r, layout, max_bytes: u64)` — pass `max_bytes` through to `read_pack_stream`. +4. Normal `PutPack` arm: `spill_pack_stream(r, transport.layout(), policy.max_pack_size)`; on `Err(e @ Error::PackTooLarge(_))` → `let (code, msg) = err_to_wire(&e); let _ = write_err(w, code, &msg); return Ok(());` (best-effort reply, then close — desync). Other errors keep the current reply-and-continue behavior. +5. RO drain arm: `spill_pack_stream(r, transport.layout(), policy.ro_drain_cap)`; on `Ok(guard)` → drop + `EC_READONLY` reply + `continue` (unchanged); on `Err(Error::PackTooLarge(_))` → best-effort `write_err(w, EC_READONLY-mapped ReadOnly error)` then `return Ok(())` (close); other errors propagate as today. +6. Update `http_transport.rs:602`: `crate::wire::serve_with_policy(root, &mut reader, &mut stream, crate::wire::WirePolicy { read_only, ..Default::default() })`. +7. Update the existing read-only test at ~line 1200 to construct a `WirePolicy`. + +- [ ] **Step 4: Run to verify pass** + +Run: `cargo test -p scl-repo --lib wire` +Expected: new tests PASS; all existing wire tests (including RO drain + streaming) PASS. + +- [ ] **Step 5: Write the RO-drain-cap test** + +```rust +/// An oversized push on a read-only connection drains at most ro_drain_cap +/// bytes, then the connection closes; zero spool residue. +#[test] +fn readonly_oversized_push_is_dropped_after_drain_cap() { + let root = tmp_repo("rocap"); + let mut input = Vec::new(); + write_frame(&mut input, &Request::Hello { version: PROTOCOL_VERSION }.encode()).unwrap(); + write_frame(&mut input, &Request::PutPack.encode()).unwrap(); + let payload = vec![9u8; 8192]; + write_pack_stream(&mut input, &mut std::io::Cursor::new(&payload), 1024).unwrap(); + write_frame(&mut input, &Request::HeadBranch.encode()).unwrap(); // must never be answered + + let mut reader = std::io::Cursor::new(input); + let mut output = Vec::new(); + let policy = WirePolicy { read_only: true, max_pack_size: 0, ro_drain_cap: 4096 }; + serve_with_policy(&root, &mut reader, &mut output, policy).unwrap(); + + let mut frames = Vec::new(); + let mut r = std::io::Cursor::new(output); + while let Some(f) = read_frame_opt(&mut r).unwrap() { + frames.push(parse_response(f)); + } + assert_eq!(frames.len(), 2); + assert!(matches!(frames[1].as_ref().unwrap_err(), Error::ReadOnly | Error::Remote(_))); + let tmp = root.join(".sc").join("tmp"); + assert!(!tmp.exists() || std::fs::read_dir(&tmp).unwrap().next().is_none()); + let _ = std::fs::remove_dir_all(&root); +} +``` + +- [ ] **Step 6: Run to verify pass** — `cargo test -p scl-repo readonly_oversized` +Expected: PASS (implementation from Step 3 item 5 already covers it; if it fails, fix the RO arm, not the test). + +- [ ] **Step 7: Full-crate check + commit** + +Run: `cargo test -p scl-repo && cargo clippy -p scl-repo --all-targets -- -D warnings` + +```bash +cargo fmt --all && git add -A crates/repo && git commit -m "feat(wire): WirePolicy with spool cap + capped read-only drain (P31)" +``` + +--- + +### Task 4: `ServeLimits`, connection slots + 503, session timeouts, accept backoff + +**Files:** +- Modify: `crates/repo/src/http_transport.rs` — `write_status` (~line 141), client status match (~line 315), `serve_http` (~line 449), `serve_http_listener` (~line 492), `handle_http_connection` (~lines 524–602), new types near `OPENING_READ_TIMEOUT`; tests in the same file's `mod tests` + +**Interfaces:** +- Consumes: `WirePolicy` (Task 3), `Error::ServerBusy` (Task 1). +- Produces: + +```rust +#[derive(Debug, Clone, Copy)] +pub struct ServeLimits { + pub max_connections: u32, // 0 = unlimited + pub timeout_secs: u64, // 0 = disabled + pub max_pack_size: u64, // 0 = unlimited +} +impl Default for ServeLimits { + fn default() -> Self { + Self { max_connections: 32, timeout_secs: 300, max_pack_size: crate::wire::DEFAULT_MAX_PACK_SIZE } + } +} + +pub fn serve_http(addr: &str, root: &Path, read_only: bool, allow_public: bool, limits: ServeLimits) -> Result<()>; +pub fn serve_http_listener(listener: TcpListener, root: &Path, read_only: bool, mandatory_auth: bool, limits: ServeLimits) -> Result<()>; +struct AcceptBackoff; // fn on_error(&mut self) -> Duration (5ms, ×2, cap 1s); fn on_success(&mut self) +``` + +- [ ] **Step 1: Write the failing unit tests** (pure parts first) + +```rust +#[test] +fn accept_backoff_doubles_and_resets() { + let mut b = AcceptBackoff::new(); + assert_eq!(b.on_error(), Duration::from_millis(5)); + assert_eq!(b.on_error(), Duration::from_millis(10)); + assert_eq!(b.on_error(), Duration::from_millis(20)); + for _ in 0..20 { b.on_error(); } + assert_eq!(b.on_error(), Duration::from_secs(1)); // capped + b.on_success(); + assert_eq!(b.on_error(), Duration::from_millis(5)); // reset +} + +#[test] +fn status_503_round_trips() { + let mut buf = Vec::new(); + write_status(&mut buf, 503).unwrap(); + assert_eq!(read_status(&mut &buf[..]).unwrap(), 503); +} +``` + +- [ ] **Step 2: Run to verify failure** — `cargo test -p scl-repo accept_backoff` → compile error. + +- [ ] **Step 3: Implement the pure parts** + +```rust +/// Exponential accept-error backoff (P31): Go net/http's shape — 5ms +/// doubling to a 1s cap, reset on the next successful accept. Turns fd +/// exhaustion (EMFILE) from a busy-loop into a paced retry. Hardcoded: no +/// operator tuning story exists (Go ships it knobless too). +struct AcceptBackoff { + cur: Duration, +} +impl AcceptBackoff { + const START: Duration = Duration::from_millis(5); + const CAP: Duration = Duration::from_secs(1); + fn new() -> Self { + Self { cur: Self::START } + } + fn on_error(&mut self) -> Duration { + let d = self.cur; + self.cur = (self.cur * 2).min(Self::CAP); + d + } + fn on_success(&mut self) { + self.cur = Self::START; + } +} +``` + +`write_status`: add `503 => "Service Unavailable",`. +Client status match (~line 315): add before `other`: + +```rust +503 => return Err(Error::ServerBusy), +``` + +- [ ] **Step 4: Run** — `cargo test -p scl-repo accept_backoff status_503` → PASS. Commit: + +```bash +cargo fmt --all && git add -A crates/repo && git commit -m "feat(http): AcceptBackoff + 503 busy status plumbing (P31)" +``` + +- [ ] **Step 5: Write the failing socket tests** (pattern: bind `127.0.0.1:0`, spawn `serve_http_listener` on a thread, connect with raw `TcpStream` + `write_client_opening`/`read_status`; repo root = temp dir containing an empty `.sc/` subdir) + +```rust +fn tmp_served_root(tag: &str) -> std::path::PathBuf { + let root = std::env::temp_dir().join(format!("scl-http-{tag}-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&root); + std::fs::create_dir_all(root.join(".sc")).unwrap(); + root +} + +fn spawn_listener(root: &std::path::Path, limits: ServeLimits) -> std::net::SocketAddr { + let listener = TcpListener::bind("127.0.0.1:0").unwrap(); + let addr = listener.local_addr().unwrap(); + let root = root.to_path_buf(); + std::thread::spawn(move || { + let _ = serve_http_listener(listener, &root, false, false, limits); + }); + addr +} + +fn open_ok(addr: std::net::SocketAddr) -> TcpStream { + let mut s = TcpStream::connect(addr).unwrap(); + write_client_opening(&mut s, "127.0.0.1", "/", None).unwrap(); + assert_eq!(read_status(&mut s).unwrap(), 200); + s +} + +#[test] +fn connection_limit_shed_and_recover() { + let root = tmp_served_root("slots"); + let limits = ServeLimits { max_connections: 1, timeout_secs: 0, ..Default::default() }; + let addr = spawn_listener(&root, limits); + + let held = open_ok(addr); // occupies the single slot + + // Second connection: shed with 503 before any opening is read. + let mut second = TcpStream::connect(addr).unwrap(); + assert_eq!(read_status(&mut second).unwrap(), 503); + + // Free the slot; a fresh connection now succeeds (poll: the server + // notices the disconnect on its next read). + drop(held); + let ok = (0..50).any(|_| { + std::thread::sleep(Duration::from_millis(100)); + let mut s = match TcpStream::connect(addr) { Ok(s) => s, Err(_) => return false }; + write_client_opening(&mut s, "127.0.0.1", "/", None).is_ok() + && read_status(&mut s) == Ok(200) + }); + assert!(ok, "slot was never freed"); + let _ = std::fs::remove_dir_all(&root); +} + +#[test] +fn errored_connection_frees_its_slot() { + let root = tmp_served_root("errslot"); + let limits = ServeLimits { max_connections: 1, timeout_secs: 0, ..Default::default() }; + let addr = spawn_listener(&root, limits); + + // A connection whose handler errors (garbage opening → 400) must free + // its slot via the guard, not leak it. + let mut bad = TcpStream::connect(addr).unwrap(); + bad.write_all(b"garbage\r\n\r\n").unwrap(); + let _ = read_status(&mut bad); // 400 (best-effort) + drop(bad); + + let ok = (0..50).any(|_| { + std::thread::sleep(Duration::from_millis(100)); + let mut s = match TcpStream::connect(addr) { Ok(s) => s, Err(_) => return false }; + write_client_opening(&mut s, "127.0.0.1", "/", None).is_ok() + && read_status(&mut s) == Ok(200) + }); + assert!(ok, "errored connection leaked its slot"); + let _ = std::fs::remove_dir_all(&root); +} + +/// A client that stops READING mid-GetPack must be reaped by the write +/// timeout (today it parks the server thread in a blocking write forever). +/// Server sends a pack big enough to overflow socket buffers; the client +/// never reads; after the timeout the server closes. +#[test] +fn write_side_stall_is_reaped_by_timeout() { + let root = tmp_served_root("wstall"); + // Put a few MiB of committed content in the repo first (init + commit a + // large random file via the Repo API — see repo.rs tests for the + // init/commit helper pattern), so GetPack has real bytes to send. + // ... then: + let limits = ServeLimits { max_connections: 0, timeout_secs: 1, ..Default::default() }; + let addr = spawn_listener(&root, limits); + + let mut s = open_ok(addr); + // Speak just enough wire protocol to trigger a GetPack, then stop reading: + // handshake HELLO, then a GetPack for the branch tip with empty haves. + // (Use wire::Request::{Hello, GetPack}.encode() + write_frame directly.) + // After sending the GetPack request, sleep instead of reading. + std::thread::sleep(Duration::from_secs(5)); + // The server must have dropped the connection rather than blocking + // forever: our next write eventually errors (reset), or read sees EOF. + s.set_read_timeout(Some(Duration::from_secs(5))).unwrap(); + let mut sink = [0u8; 4096]; + loop { + match s.read(&mut sink) { + Ok(0) | Err(_) => break, // EOF or reset — server gave up: pass + Ok(_) => continue, // drain what was already buffered + } + } + let _ = std::fs::remove_dir_all(&root); +} + +#[test] +fn silent_session_is_reaped_by_timeout() { + let root = tmp_served_root("reap"); + let limits = ServeLimits { max_connections: 0, timeout_secs: 1, ..Default::default() }; + let addr = spawn_listener(&root, limits); + + let mut s = open_ok(addr); // handshake never sent; go silent + std::thread::sleep(Duration::from_secs(3)); + // Server must have dropped us: the read sees EOF/reset, not a hang. + s.set_read_timeout(Some(Duration::from_secs(5))).unwrap(); + let mut buf = [0u8; 1]; + match s.read(&mut buf) { + Ok(0) => {} // clean EOF — dropped + Err(_) => {} // reset — also dropped + Ok(_) => panic!("server sent data to a silent client"), + } + // Spool dir is clean. + let tmp = root.join(".sc").join("tmp"); + assert!(!tmp.exists() || std::fs::read_dir(&tmp).unwrap().next().is_none()); + let _ = std::fs::remove_dir_all(&root); +} +``` + +- [ ] **Step 6: Run to verify failure** — `cargo test -p scl-repo connection_limit_shed` → compile error (`ServeLimits`, new arities). + +- [ ] **Step 7: Implement the listener changes** + +1. Add `ServeLimits` (+ `Default`) with the doc comment stating each default and its provenance (git-daemon 32; on-by-default 300s divergence; 16 GiB divergence — cite the spec). +2. RAII slot guard: + +```rust +/// Decrements the live-connection count on drop, so every exit path — clean +/// return, error, panic unwind — frees its slot (the TempPackGuard +/// discipline applied to connection slots). +struct SlotGuard(std::sync::Arc); +impl Drop for SlotGuard { + fn drop(&mut self) { + self.0.fetch_sub(1, std::sync::atomic::Ordering::SeqCst); + } +} +``` + +3. `serve_http_listener(listener, root, read_only, mandatory_auth, limits)`: + +```rust +let live = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0)); +let mut backoff = AcceptBackoff::new(); +for incoming in listener.incoming() { + let mut stream = match incoming { + Ok(s) => { + backoff.on_success(); + s + } + Err(e) => { + eprintln!("sc serve --http: accept error: {e}"); + std::thread::sleep(backoff.on_error()); + continue; + } + }; + // Connection cap: acquire a slot BEFORE spawning; shed with 503 when full. + let guard = { + let prev = live.fetch_add(1, std::sync::atomic::Ordering::SeqCst); + let g = SlotGuard(live.clone()); + if limits.max_connections != 0 && prev >= limits.max_connections as usize { + let _ = write_status(&mut stream, 503); + continue; // g drops here → count restored; no thread, no handshake + } + g + }; + let root = root.to_path_buf(); + std::thread::spawn(move || { + let _guard = guard; // slot held for the connection's lifetime + if let Err(e) = handle_http_connection(stream, &root, read_only, mandatory_auth, limits) { + eprintln!("sc serve --http: connection error: {e}"); + } + }); +} +Ok(()) +``` + +4. `handle_http_connection(..., limits: ServeLimits)`: replace the timeout-clear block (~line 594) with: + +```rust +// P31: the opening timeout is REPLACED (not cleared) by the session +// timeout, on BOTH read and write sides — a write timeout covers a client +// that stops reading mid-GetPack. Under P25 chunking a per-syscall timeout +// is progress-based: only true zero-byte stalls trip it. 0 disables. +let session = if limits.timeout_secs == 0 { + None +} else { + Some(Duration::from_secs(limits.timeout_secs)) +}; +stream + .set_read_timeout(session) + .map_err(|e| Error::ConnectionLost(format!("sc+http set session read_timeout: {e}")))?; +stream + .set_write_timeout(session) + .map_err(|e| Error::ConnectionLost(format!("sc+http set session write_timeout: {e}")))?; +``` + +and end with `crate::wire::serve_with_policy(root, &mut reader, &mut stream, crate::wire::WirePolicy { read_only, max_pack_size: limits.max_pack_size, ro_drain_cap: crate::wire::RO_DRAIN_CAP })`. A timeout surfaces as `Error::Io` (`WouldBlock` on Unix, `TimedOut` on Windows) from `serve_with_policy` → the handler returns `Err` → logged, thread exits, guard frees the slot. No special-casing needed beyond the log line; do NOT retry on `WouldBlock`. +5. `serve_http(addr, root, read_only, allow_public, limits)`: pass `limits` through; call `crate::wire::validate_max_pack_size(limits.max_pack_size)?` before binding. + +- [ ] **Step 8: Run to verify pass** + +Run: `cargo test -p scl-repo connection_limit_shed silent_session` +Expected: PASS. Also run the full crate: `cargo test -p scl-repo` — the existing http tests (auth, bind gate) must stay green with the added `limits` parameter (update their call sites to `ServeLimits::default()` or `timeout_secs: 0` where a test holds idle connections). + +- [ ] **Step 9: Commit** + +```bash +cargo fmt --all && git add -A crates/repo && git commit -m "feat(http): connection cap + 503, session timeouts, accept backoff (P31)" +``` + +--- + +### Task 5: CLI flags + stdio plumbing + +**Files:** +- Modify: `crates/cli/src/main.rs` — `Cmd::Serve` variant (~line 280), dispatch (~line 813), `run_serve` (~line 3122) + +**Interfaces:** +- Consumes: `ServeLimits`, `serve_http` (Task 4), `wire::{serve_with_policy, WirePolicy, RO_DRAIN_CAP, DEFAULT_MAX_PACK_SIZE, validate_max_pack_size}` (Task 3). + +- [ ] **Step 1: Add the flags** to `Cmd::Serve`: + +```rust +/// Maximum concurrent connections (`--http` only); 0 = unlimited. +/// Default 32 (git-daemon parity). At the limit new connections get an +/// immediate busy status and close. (P31) +#[arg(long)] +max_connections: Option, +/// Session idle timeout in seconds (`--http` only); 0 = disabled. +/// Default 300. A connection making zero-byte progress for this long is +/// dropped. (P31) +#[arg(long)] +timeout: Option, +/// Maximum incoming pack size in bytes (both --http and --stdio); +/// 0 = unlimited. Default 16 GiB. Must be ≥ 256 MiB (MAX_OBJECT_SIZE) +/// when non-zero. (P31) +#[arg(long)] +max_pack_size: Option, +``` + +- [ ] **Step 2: Update `run_serve`** + +```rust +fn run_serve( + stdio: bool, + http: Option, + read_only: bool, + allow_public: bool, + max_connections: Option, + timeout: Option, + max_pack_size: Option, + path: PathBuf, +) -> Result<()> { + match (stdio, http) { + (true, None) => { + if read_only || allow_public { + anyhow::bail!("--read-only/--allow-public apply only to --http, not --stdio"); + } + if max_connections.is_some() || timeout.is_some() { + anyhow::bail!("--max-connections/--timeout apply only to --http, not --stdio"); + } + let max_pack = max_pack_size.unwrap_or(scl_repo::wire::DEFAULT_MAX_PACK_SIZE); + scl_repo::wire::validate_max_pack_size(max_pack)?; + let policy = scl_repo::wire::WirePolicy { + read_only: false, + max_pack_size: max_pack, + ro_drain_cap: scl_repo::wire::RO_DRAIN_CAP, + }; + let mut stdin = std::io::stdin().lock(); + let mut stdout = std::io::stdout().lock(); + scl_repo::wire::serve_with_policy(&path, &mut stdin, &mut stdout, policy)?; + Ok(()) + } + (false, Some(addr)) => { + let mut limits = scl_repo::http_transport::ServeLimits::default(); + if let Some(n) = max_connections { limits.max_connections = n; } + if let Some(t) = timeout { limits.timeout_secs = t; } + if let Some(m) = max_pack_size { limits.max_pack_size = m; } + scl_repo::http_transport::serve_http(&addr, &path, read_only, allow_public, limits)?; + Ok(()) + } + // ... existing arms unchanged + } +} +``` + +Update the dispatch site to pass the three new fields. Note `WirePolicy`/`RO_DRAIN_CAP`/`DEFAULT_MAX_PACK_SIZE`/`validate_max_pack_size`/`ServeLimits` must be `pub` exports (they are, from Tasks 3–4). + +- [ ] **Step 3: Verify behavior manually** + +```bash +cargo run -q --bin sc -- serve --stdio --timeout 5 . 2>&1 | head -2 # expect: the --http-only bail +cargo run -q --bin sc -- serve --http 127.0.0.1:0 --max-pack-size 1024 . 2>&1 | head -2 # expect: floor error naming MAX_OBJECT_SIZE +``` + +Expected: both print clear errors and exit non-zero. + +- [ ] **Step 4: Full workspace test + commit** + +Run: `cargo test --workspace && cargo clippy --workspace --all-targets -- -D warnings` + +```bash +cargo fmt --all && git add -A crates/cli && git commit -m "feat(cli): sc serve --max-connections/--timeout/--max-pack-size (P31)" +``` + +--- + +### Task 6: Demo script + +**Files:** +- Create: `demo/run_limits_demo.sh` (pattern: `demo/run_http_auth_demo.sh` — read it first for the port-pick/cleanup/run-twice conventions, including the `listening on` stdout parse for the OS-assigned port) + +**Interfaces:** +- Consumes: the CLI flags (Task 5), 503 busy behavior, timeout reaping, `EC_TOO_LARGE`. + +- [ ] **Step 1: Write the demo.** It must (following the sibling script's structure — `set -euo pipefail`, temp workdir, `sc` built once, server started with `--http 127.0.0.1:0` and the port parsed from `listening on`): + 1. Init a repo, commit a file, start `sc serve --http 127.0.0.1:0 --max-connections 1 --timeout 2 --max-pack-size 268435456 --allow-public`… (use loopback — no `--allow-public` needed). + 2. **Busy:** hold one connection open with a raw client (a `sleep`-piped `nc` if available, else a second `sc clone` in a loop is unreliable — prefer opening a TCP connection via `python3`… **no**: python3 is not guaranteed; use a background `sc clone` of a large-enough repo? Simplest reliable primitive: `exec 3<>/dev/tcp/127.0.0.1/$PORT` in bash holds a connection without sending an opening — the slot is taken at accept time, before the opening read). Then a second `sc clone sc+http://…` must fail with the busy error (assert on stderr text), close fd 3, and the same clone must succeed. + 3. **Timeout:** open `exec 3<>/dev/tcp/...` again, sleep 4 (> `--timeout 2` + opening 30s does not apply — the opening timeout drops it… NOTE: an opening-silent connection is dropped by the **existing 30s opening timeout**, not the session timeout; a demo can't cheaply hold a post-opening session silent without a wire client. So the demo proves timeout indirectly: assert the busy slot from step 2's silent holder is auto-freed — a clone succeeds after the drop **without** closing fd 3 manually. Print what's being proven.) + 4. **Spool cap:** restart the server with `--max-pack-size 268435456` (the floor) and push a > 256 MiB… too slow for a demo. Instead restart with the floor value and demonstrate the *refusal path* differently: a push of a small repo succeeds (cap present, harmless); then restart with `--max-pack-size 1024` and assert startup fails with the floor error. The mid-stream abort itself is covered by unit tests; the demo proves the knobs and the busy/reap behavior end-to-end. + 5. Assert zero `.sc/tmp` residue on the served repo; run the whole script twice (the repo's demo discipline). + +- [ ] **Step 2: Run it twice** + +Run: `bash demo/run_limits_demo.sh && bash demo/run_limits_demo.sh` +Expected: both runs end with the script's success line; no leftover temp dirs. + +- [ ] **Step 3: Commit** + +```bash +git add demo/run_limits_demo.sh && git commit -m "demo: listener limits proof — busy shed, slot reap, cap knobs (P31)" +``` + +--- + +### Task 7: Docs — ADR-0041, THREAT-MODEL, CLAUDE.md + +**Files:** +- Create: `docs/adr/0041-listener-resource-limits.md` +- Modify: `docs/THREAT-MODEL.md` (the ADR-0036 operational-hardening items, ~lines 91-93, plus the env-var and signature sections) +- Modify: `CLAUDE.md` (serve command block; the P26 "accepted design consequences" paragraph) + +**Interfaces:** none (docs only). Read `docs/adr/0040-sc-http-access-control.md` first and match its structure (Status/Date/Phase/Context/Decision/Consequences). + +- [ ] **Step 1: Write ADR-0041.** Must contain: the four bounds with defaults and mechanisms; the two recorded divergences (timeout **on**-by-default vs git-daemon's off — defaults-off knobs don't close audit Highs; finite 16 GiB spool default vs git's unlimited `receive.maxInputSize` — sc servers often share the working tree's volume); the 8 MiB RO drain posture (honest misconfigurations keep the typed error, hostile bulk spools almost nothing); mid-stream abort = connection-fatal (desync); spool cap applies to both transports (shared wire layer); rejected alternatives (block-at-accept semaphore, git-daemon grace-then-shed, Rust-Book queue pool, reserved-fd shed, env-var/config-file knobs); links to decision tickets #28/#27 and `docs/research/bounded-server-patterns.md`. +- [ ] **Step 2: Update THREAT-MODEL.md.** (a) The three ADR-0036 boundaries → closed by P31, naming the mechanisms; (b) name + close the aggregate-spool gap (read-only drain path included); (c) the env-var re-affirmation wording from map ticket #31 (same-user adversary defeats the identity key; fd/stdin stays an additive deferred mode with recorded revisit triggers); (d) the signature-replay re-affirmation from map ticket #32 (ancestry already id-bound; residue is ref binding only — "replayable to a ref tip within its own history"; gittuf-shaped attestation deferred with revisit triggers). +- [ ] **Step 3: Update CLAUDE.md.** Add the three flags to the serve command block with one-line semantics; amend the P26 paragraph's "accepted design consequences, not yet closed" list to point at P31; add a short "Phase 31 is built." section following the existing phase-section pattern. +- [ ] **Step 4: Commit** + +```bash +git add docs CLAUDE.md && git commit -m "docs: ADR-0041 listener limits + THREAT-MODEL closures and re-affirmations (P31)" +``` + +--- + +### Task 8: Final verification + +- [ ] **Step 1:** `cargo test --workspace --all-targets` — all green. +- [ ] **Step 2:** `cargo clippy --workspace --all-targets -- -D warnings && cargo fmt --all -- --check` — clean. +- [ ] **Step 3:** Run the neighboring demos that exercise the touched paths: `bash demo/run_http_remote_demo.sh && bash demo/run_http_auth_demo.sh && bash demo/run_ssh_remote_demo.sh && bash demo/run_limits_demo.sh` — all end with their success lines (the first three prove no regression from the `WirePolicy`/signature changes; defaults must keep every existing flow working). +- [ ] **Step 4:** Confirm zero residue: `git status --short` shows only intended files. diff --git a/docs/superpowers/specs/2026-07-10-p31-listener-limits-design.md b/docs/superpowers/specs/2026-07-10-p31-listener-limits-design.md new file mode 100644 index 00000000..2f562b7d --- /dev/null +++ b/docs/superpowers/specs/2026-07-10-p31-listener-limits-design.md @@ -0,0 +1,177 @@ +# P31: HTTP listener resource limits — design + +- **Date:** 2026-07-10 +- **Spec issue:** [#38 — P31: HTTP listener resource limits (sc serve --http)](https://github.com/git-agentic/src-control/issues/38) +- **Decision trail:** wayfinder map [#24](https://github.com/git-agentic/src-control/issues/24) → + decision [#28](https://github.com/git-agentic/src-control/issues/28), research + [#27](https://github.com/git-agentic/src-control/issues/27) + (`docs/research/bounded-server-patterns.md`) +- **Closes:** audit High #2 — ADR-0036's three named operational boundaries + (unbounded thread-per-connection, no idle-transfer watchdog, no accept-loop + backoff) **plus** the unbounded aggregate pack spool the research surfaced + (reachable by read-only clients via the drain-for-sync arm, `wire.rs`). + +Zero new dependencies. `PROTOCOL_VERSION` stays 3. + +## Goal + +`sc serve` survives hostile or broken clients with bounded threads, bounded +disk, and a live accept loop — with defaults that protect out of the box and +per-invocation flags for operators who need different numbers. + +## CLI surface + +`sc serve` gains three flags: + +| Flag | Default | 0 means | Scope | +|---|---|---|---| +| `--max-connections ` | 32 | unlimited | `--http` only (refused with `--stdio`) | +| `--timeout ` | 300 | disabled | `--http` only (refused with `--stdio`) | +| `--max-pack-size ` | 16 GiB | unlimited | **both** `--http` and `--stdio` | + +- `--max-pack-size` values below `MAX_OBJECT_SIZE` (256 MiB) are rejected at + parse time with a clear error (a cap that can't fit one maximal object is a + misconfiguration, not a policy). +- `--max-connections`/`--timeout` are socket/accept-loop mechanics, so they are + meaningless for `--stdio` (ssh owns that transport's sockets) and refused + there rather than silently ignored. +- `--max-pack-size` applies to both transports because the spool mechanism is + the shared wire layer and disk exhaustion is transport-agnostic (decision: + brainstorm 2026-07-10; same default everywhere). + +The flags build a `ServeLimits` struct in the CLI; no config file, no env vars +(both rejected on decision ticket #28 / brainstorm). + +## Listener changes (`crates/repo/src/http_transport.rs`) + +**Connection limit — atomic slots + RAII guard.** `serve_http_listener` holds +an `Arc`. Per accepted socket, before spawning: increment; if the +new count exceeds the limit, write HTTP status **503**, close, decrement — no +wire handshake, no thread. Otherwise the connection thread owns a `SlotGuard` +whose `Drop` decrements, so panics and every error path free the slot +(the `TempPackGuard` discipline). No queuing: for long-lived wire sessions, +fail-fast-with-status beats invisible waits (decision #28). + +**Idle/progress deadline.** `handle_http_connection` currently clears the 30s +opening read timeout before the wire handoff (`set_read_timeout(None)`); it +instead sets the configured session timeout on **both** read and write sides +(`set_read_timeout` + `set_write_timeout`) — a write timeout covers a client +that stops *reading* mid-`GetPack`, which today parks the thread in a blocking +`write`. Under P25 chunking (≤ `SC_PACK_CHUNK` frames) a per-syscall timeout is +progress-based: slow-but-flowing transfers never trip it; only true zero-byte +stalls do. On timeout the error is **connection-fatal** (frame desync makes +recovery impossible): log to stderr, thread exits, guard frees the slot, +`TempPackGuard` removes any partial spool. Both `io::ErrorKind::WouldBlock` +(Unix) and `TimedOut` (Windows) are treated as timeout. The opening keeps its +existing stricter 30s. + +**Accept-loop backoff.** On `listener.incoming()` errors: sleep 5ms, doubling +per consecutive error to a 1s cap, reset on the next successful accept — Go's +`net/http` shape verbatim, hardcoded, no knob, no reserved-fd trick +(decision #28). This turns fd exhaustion from a busy-loop into a paced retry. + +## Wire-layer changes (`crates/repo/src/wire.rs`) + +**`WirePolicy` replaces the bare bool.** `serve_with_policy(root, r, w, +read_only: bool)` becomes `serve_with_policy(root, r, w, policy: WirePolicy)` +where + +```rust +pub struct WirePolicy { + pub read_only: bool, + pub max_pack_size: u64, // 0 = unlimited + pub ro_drain_cap: u64, // constant: 8 MiB +} +``` + +The existing `serve` wrapper maps to `WirePolicy { read_only: false, +max_pack_size: , ro_drain_cap: RO_DRAIN_CAP }`; `--stdio` and `--http` +both construct it from `ServeLimits`. In-crate callers update; this is a +pre-1.0 in-workspace signature change, not a wire change. + +**Spool cap — counted abort mid-stream.** `read_pack_stream` and +`spill_pack_stream` gain a byte budget. The running total is checked as chunk +frames arrive (git's `receive.maxInputSize` shape: abort *while counting*, +never after materializing); exceeding it raises a new typed +`Error::PackTooLarge`, the partial spool is removed by the guard, +and the server sends a best-effort reply with a new wire error code +**`EC_TOO_LARGE = 6`**, then closes the connection — aborting mid-stream +leaves unread chunk frames in flight, so the stream is desynced by +construction and the connection cannot be reused (same posture as the RO +drain overflow below). +Old clients degrade gracefully — unknown codes already fall back to +`Error::Remote(msg)` — so no protocol version bump. The cap composes *above* +ADR-0039's per-frame/per-record/decompression caps; it never replaces them. + +**Read-only drain cap.** The RO `PutPack` arm's drain-for-sync +(`spill_pack_stream` then drop) is capped at **8 MiB** (`RO_DRAIN_CAP`): a pack +that ends within the cap gets today's clean typed `EC_READONLY` reply; a larger +one gets a best-effort `EC_READONLY` write and the connection is closed +(sync is unrecoverable past the cap by design — decision #28: honest +misconfigurations keep the clear error, hostile bulk spools almost nothing). + +## Client-side mapping (`http_transport.rs`, `wire.rs`) + +- Opening status `503` → a typed, clearly retryable error + ("server busy (connection limit reached) — retry later"). +- `EC_TOO_LARGE` → `Error::PackTooLarge` with the server's limit in the + message, so `sc push` prints an actionable failure. + +## Error handling summary + +| Event | Server behavior | Client sees | +|---|---|---| +| Accept while full | 503 + close, no handshake | "server busy, retry" | +| Zero-byte stall > timeout | log + drop connection, spool cleaned | connection error | +| Pack exceeds `--max-pack-size` | abort mid-stream, best-effort `EC_TOO_LARGE`, spool cleaned, close | "pack exceeds server limit (N)" or connection error | +| RO push ≤ 8 MiB | drain, `EC_READONLY` (unchanged) | "read-only" | +| RO push > 8 MiB | best-effort `EC_READONLY`, close | read-only error or connection error | +| Accept error (EMFILE etc.) | 5ms→1s backoff, loop stays alive | connect delay/refusal | + +## Testing + +Pinned regression tests per bound: + +1. Slot exhaustion: fill `--max-connections 1` with a held session; second + connect gets the busy error; after the first closes, a third succeeds + (slot actually freed). +2. Guard leak check: a connection whose handler errors (bad opening) frees its + slot. +3. Stall reaping: a client that connects, handshakes, then goes silent is + dropped after the (test-shortened) timeout; `.sc/tmp` empty afterwards. +4. Write-side stall: a client that stops reading mid-`GetPack` is dropped. +5. Spool cap: a pack exceeding a small `--max-pack-size` fails mid-stream with + the typed error, `.sc/tmp` is empty, the connection is closed, and a fresh + connection succeeds. +6. Config floor: `--max-pack-size` below `MAX_OBJECT_SIZE` refused at startup. +7. RO drain: small RO push still gets `EC_READONLY`; oversized RO push gets the + connection closed, ≤ 8 MiB spooled (assert via temp-dir observation), zero + residue after. +8. Backoff: consecutive accept-error handling doesn't busy-loop (shape test via + injected error source or timing bound). +9. `--stdio` spool cap: same as (5) over the wire-harness stdio path. + +`demo/run_limits_demo.sh`: real loopback TCP; proves busy-status at the cap, +stall reaping, spool-cap rejection with zero `.sc/tmp` residue on both ends — +run twice (the repo's demo discipline). + +## Documentation + +- **New ADR** (next number): the four bounds; the two recorded + divergences-from-prior-art (timeout on-by-default vs git-daemon's off; + finite 16 GiB spool default vs git's unlimited `receive.maxInputSize`); + the 8 MiB drain posture; the both-transports scope of the spool cap. +- **`docs/THREAT-MODEL.md`:** mark ADR-0036's three boundaries closed; name and + close the aggregate-spool gap; carry the two map re-affirmation wordings — + env-var secret exposure (#31: same-user adversary defeats the identity key; + fd/stdin stays an additive deferred mode) and signature replay (#32: + ancestry is already id-bound; the residue is ref binding only — + "replayable to a ref tip within its own history"). +- **CLAUDE.md:** the three new flags on the serve command block; ADR-0036's + "accepted design consequences" paragraph updated to point here. + +## Non-goals + +Pre-auth connection sub-caps (sshd MaxStartups shape), reserved-fd shedding, +client-side spool caps, any config-file surface — recorded follow-ons, not +this phase.