Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 57 additions & 10 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -280,12 +280,25 @@ bash demo/run_streaming_demo.sh # streaming pack transfer proof (P
# in 250+ chunk frames, byte-for-byte clone, zero
# .sc/tmp residue on either end
cargo run --bin sc -- serve --http <addr> <path> [--read-only] [--allow-public]
[--max-connections <n>] [--timeout <secs>] [--max-pack-size <bytes>]
# sc-native wire protocol over TCP (P26); e.g.
# `sc serve --http 127.0.0.1:8730 .`; exactly one
# of --stdio/--http is required; a non-loopback
# <addr> is refused unless --read-only,
# --allow-public, or a configured serve token
# justifies it (P29)
# --max-connections <n>: concurrent-connection cap,
# --http only, default 32, 0=unlimited; at the limit
# a new connection gets busy-status-and-close, no
# queuing (P31)
# --timeout <secs>: session idle/progress timeout,
# --http only, default 300, 0=disabled; read+write,
# persists the whole session, connection-fatal on
# trip; opening keeps its own 30s (P31)
# --max-pack-size <bytes>: incoming-pack aggregate
# cap, both --http and --stdio, default 16 GiB,
# 0=unlimited, floor 256 MiB (MAX_OBJECT_SIZE);
# counted mid-stream abort -> EC_TOO_LARGE (P31)
cargo run --bin sc -- serve token add --label <name> --scope ro|rw # mint an
# sct-<hex> bearer token in .sc/serve-tokens.toml;
# raw value prints once on stdout (P29)
Expand Down Expand Up @@ -996,13 +1009,13 @@ serve --http` is unauthenticated, as `--stdio` delegates auth to ssh —
production auth + TLS means fronting with a reverse proxy; this is the
reach primitive, not a hosted-git competitor); **not HTTP-proxy/CDN safe**
(a strict proxy won't tunnel the post-opening raw protocol, the accepted
cost of the persistent-connection model). **Accepted design consequences,
not yet closed (deferred, see ROADMAP):** `serve_http_listener` is
unbounded thread-per-connection with no pool/backpressure; the opening
timeout is cleared once `wire::serve` takes over, so a client that stalls
mid-transfer holds its thread indefinitely (no idle-transfer watchdog
yet); the accept loop has no backoff on sustained fd exhaustion. See
ADR-0036.
cost of the persistent-connection model). **Accepted design consequences at
the time — closed by P31 (ADR-0041):** `serve_http_listener`'s unbounded
thread-per-connection (no pool/backpressure), the opening timeout being
cleared once `wire::serve` takes over (no idle-transfer watchdog), and the
accept loop's lack of backoff on sustained fd exhaustion are all closed by
P31's `--max-connections`, `--timeout`, and hardcoded accept backoff,
respectively — see the P31 section below. See ADR-0036.

**Phase 27 is built.** The P25–P27 scale-&-reach horizon's capstone:
partial clone. `.sc/promisor` (`crates/repo/src/promisor.rs`, local,
Expand Down Expand Up @@ -1242,12 +1255,46 @@ bytes, the `sc log` marker, and `sc gc` pruning a deleted branch's
transcript while the still-reachable one from earlier survives — run
twice, zero residue. No new dependency. See ADR-0038.

**Phase 31 is built.** Listener resource limits close ADR-0036's three
named-but-open accepted consequences plus a fourth this phase's own
research pass (`docs/research/bounded-server-patterns.md`, ticket #27)
found and named for the first time: the aggregate incoming-pack spool was
unbounded on both transports, including via the read-only rejection path.
`sc serve --http` gains `--max-connections <n>` (default 32, matching
`git-daemon`, 0 = unlimited; at the limit a new connection gets an
immediate busy status at the pre-handshake opening seam and closes — no
queuing), `--timeout <secs>` (default 300, 0 = disables; read+write
timeouts now persist for the whole session rather than being cleared after
the opening, and a trip is connection-fatal since mid-stream abort desyncs
the frame stream; the opening keeps its own unchanged 30s), and
`--max-pack-size <bytes>` (default 16 GiB, 0 = unlimited, floor 256 MiB =
`MAX_OBJECT_SIZE`; applies to **both** `--http` and `--stdio` since it
lives in the shared `WirePolicy`, not the `--http`-only `ServeLimits`; a
counted mid-stream abort replies `EC_TOO_LARGE` best-effort before closing).
The read-only pre-drain (`wire.rs`'s `EC_READONLY` arm, which drains an
incoming pack before replying so the connection stays in sync) is now
capped at 8 MiB (`RO_DRAIN_CAP`): an honest small misconfigured push still
gets the clean typed error, a larger bulk spool is dropped mid-send instead
of fully written to disk. The accept loop gained a hardcoded Go-`net/http`-
shaped exponential backoff (5ms doubling to a 1s cap, reset on success) —
no operator knob, matching Go's own precedent. Two recorded divergences
from prior art: the timeout ships **on** by default, unlike `git-daemon`'s
off-by-default `--timeout` (a defaults-off knob doesn't close an
audit-tracked High); the pack-spool default is **finite** (16 GiB), unlike
git's unlimited `receive.maxInputSize` (sc servers often share the working
tree's volume). `PROTOCOL_VERSION` stays 3 — none of the four bounds touch
the wire protocol, and `EC_TOO_LARGE` degrades to opaque text on an old
client, same as `EC_READONLY`'s precedent. Zero new dependencies. Proven by
`demo/run_limits_demo.sh` (the floor-cap enforcement on the push path and
the busy-status shed under `--max-connections`); the session-timeout reaper
and accept-backoff pacing are unit-test-proven rather than demo-proven,
since forcing a real stalled socket or fd exhaustion reliably in a demo
script is impractical. See ADR-0041.

Remaining follow-ons: operation objects in the CAS, oplog entries for
remote-tracking refs, extending the sparse gate to
`materialize_conflict_state`'s `to_encrypt`/sidecar write loops (see the
P24 boundary note above), the three P26 `sc serve --http` hardening items
named above (connection pool/backpressure, idle-transfer watchdog,
accept-loop backoff), the three P27 items named above (transparent
P24 boundary note above), the three P27 items named above (transparent
lazy-fetch as a deferred alternative to explicit `sc backfill`, per-case
gap-tolerant merge/rebase/`ws harvest`/`sc work` instead of blanket
refusal, and blob-size/object-count clone filters alongside the
Expand Down
59 changes: 55 additions & 4 deletions crates/cli/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -298,6 +298,21 @@ enum Cmd {
/// configured token, binding a non-loopback address is refused.
#[arg(long)]
allow_public: bool,
/// Maximum concurrent connections (`--http` only); 0 = unlimited.
/// Default 32 (git-daemon parity). At the limit new connections get an
/// immediate busy status and close. (P31)
#[arg(long)]
max_connections: Option<u32>,
/// Session idle timeout in seconds (`--http` only); 0 = disabled.
/// Default 300. A connection making zero-byte progress for this long is
/// dropped. (P31)
#[arg(long)]
timeout: Option<u64>,
/// Maximum incoming pack size in bytes (both --http and --stdio);
/// 0 = unlimited. Default 16 GiB. Must be ≥ 256 MiB (MAX_OBJECT_SIZE)
/// when non-zero. (P31)
#[arg(long)]
max_pack_size: Option<u64>,
/// Repo root to serve (the directory containing `.sc/`). Required
/// unless a `token` subcommand is used.
path: Option<PathBuf>,
Expand Down Expand Up @@ -816,12 +831,24 @@ fn main() -> Result<()> {
http,
read_only,
allow_public,
max_connections,
timeout,
max_pack_size,
path,
} => match sub {
Some(ServeSub::Token { op }) => run_serve_token(op),
None => {
let path = path.ok_or_else(|| anyhow::anyhow!("sc serve requires a <path>"))?;
run_serve(stdio, http, read_only, allow_public, path)
run_serve(
stdio,
http,
read_only,
allow_public,
max_connections,
timeout,
max_pack_size,
path,
)
}
},
Cmd::Remote { op } => run_remote(op),
Expand Down Expand Up @@ -3118,26 +3145,50 @@ fn run_clone_git(url: &str, dst: &std::path::Path) -> Result<()> {
/// dropped, e.g. by process termination). Exactly one mode is required.
/// `--read-only`/`--allow-public` are `--http`-only (P29): `--stdio`
/// delegates auth/access entirely to ssh, so combining them is refused
/// rather than silently ignored.
/// rather than silently ignored. `--max-connections`/`--timeout` are also
/// `--http`-only (P31). `--max-pack-size` applies to both (P31).
fn run_serve(
stdio: bool,
http: Option<String>,
read_only: bool,
allow_public: bool,
max_connections: Option<u32>,
timeout: Option<u64>,
max_pack_size: Option<u64>,
path: PathBuf,
) -> Result<()> {
match (stdio, http) {
(true, None) => {
if read_only || allow_public {
anyhow::bail!("--read-only/--allow-public apply only to --http, not --stdio");
}
if max_connections.is_some() || timeout.is_some() {
anyhow::bail!("--max-connections/--timeout apply only to --http, not --stdio");
}
let max_pack = max_pack_size.unwrap_or(scl_repo::wire::DEFAULT_MAX_PACK_SIZE);
scl_repo::wire::validate_max_pack_size(max_pack)?;
let policy = scl_repo::wire::WirePolicy {
read_only: false,
max_pack_size: max_pack,
ro_drain_cap: scl_repo::wire::RO_DRAIN_CAP,
};
let mut stdin = std::io::stdin().lock();
let mut stdout = std::io::stdout().lock();
scl_repo::wire::serve(&path, &mut stdin, &mut stdout)?;
scl_repo::wire::serve_with_policy(&path, &mut stdin, &mut stdout, policy)?;
Ok(())
}
(false, Some(addr)) => {
scl_repo::http_transport::serve_http(&addr, &path, read_only, allow_public)?;
let mut limits = scl_repo::http_transport::ServeLimits::default();
if let Some(n) = max_connections {
limits.max_connections = n;
}
if let Some(t) = timeout {
limits.timeout_secs = t;
}
if let Some(m) = max_pack_size {
limits.max_pack_size = m;
}
scl_repo::http_transport::serve_http(&addr, &path, read_only, allow_public, limits)?;
Ok(())
}
(true, Some(_)) => anyhow::bail!("sc serve accepts only one of --stdio or --http"),
Expand Down
9 changes: 9 additions & 0 deletions crates/repo/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,15 @@ pub enum Error {
/// (`--read-only` or an `ro`-scope token). P29.
#[error("server is read-only")]
ReadOnly,
/// The server aborted an incoming pack mid-stream because it exceeded the
/// operator's `--max-pack-size` cap (P31). Payload is the server's own
/// human-readable limit text, carried verbatim across the wire.
#[error("pack exceeds the server's --max-pack-size limit: {0}")]
PackTooLarge(String),
/// The server refused the connection at accept time because
/// `--max-connections` was reached (P31). Retryable.
#[error("server busy (connection limit reached); retry later")]
ServerBusy,
}

pub type Result<T> = std::result::Result<T, Error>;
Loading
Loading