From d29197a191f7d076b59e38e4d6c11645fdc4dba2 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 23 Oct 2026 15:54:56 +0300 Subject: [PATCH 1/7] docs(forge): F3a plan --- .../plans/2026-10-23-p38-f3a-content-reads.md | 52 +++++++++++++++++++ 1 file changed, 52 insertions(+) create mode 100644 docs/superpowers/plans/2026-10-23-p38-f3a-content-reads.md diff --git a/docs/superpowers/plans/2026-10-23-p38-f3a-content-reads.md b/docs/superpowers/plans/2026-10-23-p38-f3a-content-reads.md new file mode 100644 index 00000000..6ccf322a --- /dev/null +++ b/docs/superpowers/plans/2026-10-23-p38-f3a-content-reads.md @@ -0,0 +1,52 @@ +# P38 F3a: refs, tags, commits, trees, contents reads, archive (plan) + +F3 is split like F2: **F3a** (this plan) is every read route; **F3b** is the contents write path (POST/PUT/DELETE +`contents`, the CAS ref update, path rules, the protection hook point). Writes touch data integrity and get their own +plan, review and PR. Spec: "Server-side Git operations". Shapes are the recorded ones in +`crates/forge/conformance/fixtures/1.22.6` (E mainly, D for the empty repository, J and S for tags and refs). + +## Routes (all `Actor::Repo(Read)`, scope `read:repository`) + +| Route | Recorded in | Notes | +|---|---|---| +| `GET /repos/{o}/{r}/git/refs`, `git/refs/{heads|tags}[/{name}]` | D, E, J | list of `{ref, url, object{type,sha,url}}`; a missing exact ref is 404 `The target couldn't be found.`; an empty repo 404 on the list | +| `GET /tags` | D, J | tag objects (name, message, id, commit, zipball/tarball urls); `X-Total-Count`, paging | +| `GET /commits` (`sha`, `path`, `stat`, `files`, `verification`, `not`, paging) | D, E | `X-Total-Count`, 409 `Git Repository is empty.` for an empty repo, 500 `object does not exist [id: nope, rel_path: ]` for an unknown ref (recorded) | +| `GET /git/commits/{sha}`, `.diff`, `.patch` | D, E | `files`/`stats` filled, `verification` object; `.diff`/`.patch` are `text/plain` git output | +| `GET /git/trees/{sha}` (`recursive`, `per_page`, `page`) | D, E | `X-Total-Count`, `truncated`, `total_count`; 400 `sha not found [main]` when empty | +| `GET /contents`, `/contents/{path}` (`ref`) | D, E, K, N | dir list, file (base64), submodule/symlink variants; `last_commit_sha`; 404 `GetContentsOrList` | +| `GET /archive/{ref}.{zip,tar.gz}` | D, E | `application/octet-stream`, `Link: <...>; rel="immutable"`, 404 `unrecognized repository reference: x` | + +Not in F3a: `git/blobs`, `raw`, `commits/{sha}/status` (F4), `pulls/*`, branch protection. + +## Design + +- One module `gitrefs.rs`-style layer on top of `gitread.rs` (F2a): every client string (ref, sha, path) is resolved once + to a full object id with `rev-parse --verify --end-of-options ^{commit}`; only ids reach other commands; + `--end-of-options` before revisions, `--` before paths; paths are literals, output is NUL-delimited and size-capped + (`Spec` already caps output; archive and diff get explicit larger caps with `OutputTooLarge` -> 413/500 as recorded). +- Every read runs under `Lifecycle::with_read`; a busy runner is 503 (existing `life_err` mapping). +- Ref names in routes use the F2a tail-capture and `refname::valid_branch`; tag names get the analogous check. Refs under + `refs/forge/` and anything outside `refs/heads|tags` are never listed (same namespace rule as the transport). +- Commit `author`/`committer` user objects: the login resolved by email (`store::login_by_email`, Unicode folded) rendered + with `user_json`, `null` when none (recorded `author: null` in E). +- Contents `last_commit_sha`: one capped `git log --name-only` pass over the directory, first commit that touched each child, + falling back to a per-entry `git log -n1` for what the pass did not reach (bounded). Symlinks and submodules follow the + recorded variants if any; otherwise documented departures. +- Paging: `X-Total-Count` is exact; `limit`/`page` clamps follow the F2a helpers (`page_params`, `paged`). +- Archive: `git archive --format=zip|tar.gz --prefix=/ ` streamed from the child with the same supervisor + pattern as the transport? **Decision:** buffer under a cap (`--max-archive-bytes`, default 256 MiB) in the git runner: + the F2a runner already captures output with a cap; streaming archives is a later optimisation. + +## Tests + +Unit tests for each parser (log, numstat, ls-tree, for-each-ref) and e2e tests per route including: unknown/ambiguous ref, +ref that looks like an option (`--output=x`), path traversal (`../`, `.git/config`, NUL, `%2e%2e`), huge listing +(pagination), empty repo variants, non-UTF-8 file names and content, symlink to outside, submodule entry, a deleted +repository mid-read, private repo visibility (404 oracle parity), `public-only` token. The gate adds `D` and `E` +and removes the contents/tags defers of J, K, N. + +## Order of work + +1. refs + tags + commits list (paging, X-Total-Count) 2. git/commits get/.diff/.patch + trees 3. contents GET +4. archive 5. gate (D, E in; defers out), departures 6. threat-model addendum (output caps, argument hygiene) 7. reviews, PR. From 4addc73030359b61768a175f7156d9ac9161ee66 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 23 Oct 2026 16:10:37 +0300 Subject: [PATCH 2/7] feat(forge): F3a content reads (refs, tags, commits, trees, contents, archive), gate D+E, 501 contents writes --- crates/forge/conformance/forge-check.sh | 2 +- crates/forge/conformance/slice-pending.json | 64 +- crates/forge/src/api.rs | 50 ++ crates/forge/src/b64.rs | 81 +++ crates/forge/src/content_api.rs | 742 ++++++++++++++++++++ crates/forge/src/gitobj.rs | 668 ++++++++++++++++++ crates/forge/src/gitproc.rs | 9 +- crates/forge/src/gitread.rs | 1 + crates/forge/src/lib.rs | 3 + crates/forge/src/lifecycle.rs | 1 + crates/forge/src/repo_api.rs | 11 +- crates/forge/src/routes.rs | 92 ++- crates/forge/tests/common/mod.rs | 1 + 13 files changed, 1714 insertions(+), 11 deletions(-) create mode 100644 crates/forge/src/b64.rs create mode 100644 crates/forge/src/content_api.rs create mode 100644 crates/forge/src/gitobj.rs diff --git a/crates/forge/conformance/forge-check.sh b/crates/forge/conformance/forge-check.sh index 8a743f50..5a5a6d59 100755 --- a/crates/forge/conformance/forge-check.sh +++ b/crates/forge/conformance/forge-check.sh @@ -25,4 +25,4 @@ for _ in $(seq 50); do done python3 -I "$here/run.py" check --base-url "http://127.0.0.1:$port" --admin-token "$token" \ --fixtures "$here/fixtures/1.22.6" --departures "$here/departures.json" --pending "$here/slice-pending.json" \ - --only A,B,C,J,K,L,M,N,R,V "$@" || { echo "--- server log"; cat "$data/serve.log"; exit 1; } + --only A,B,C,D,E,J,K,L,M,N,R,V "$@" || { echo "--- server log"; cat "$data/serve.log"; exit 1; } diff --git a/crates/forge/conformance/slice-pending.json b/crates/forge/conformance/slice-pending.json index db920199..e9653d31 100644 --- a/crates/forge/conformance/slice-pending.json +++ b/crates/forge/conformance/slice-pending.json @@ -1,6 +1,62 @@ { - "B": [25, "GET /api/v1/repos/alice/scoped/contents/README.md"], - "J": {"defer": [[56, "GET /api/v1/repos/acme/smart/tags"], [57, "GET /api/v1/repos/acme/smart/git/refs/tags"], [59, "GET /api/v1/repos/acme/smart/tags"]]}, - "K": {"defer": [[21, "GET /api/v1/repos/acme/vis/contents"], [26, "GET /api/v1/repos/acme/ghost/contents"], [31, "GET /api/v1/repos/acme/vis/contents"], [36, "GET /api/v1/repos/acme/ghost/contents"], [55, "GET /api/v1/orgs/acme/repos"], [56, "GET /api/v1/orgs/acme/repos?limit=50"]]}, - "N": {"defer": [[6, "GET /api/v1/repos/acme/dflt/contents"]]} + "B": { + "defer": [ + [ + 30, + "POST /api/v1/repos/alice/scoped/statuses/" + ] + ] + }, + "D": { + "defer": [ + [ + 13, + "GET /api/v1/repos/acme/blank/pulls?state=all" + ] + ] + }, + "E": { + "stop": [ + 42, + "KeyError: 'sha'" + ], + "defer": [ + [ + 36, + "POST /api/v1/repos/acme/hist/contents/new.txt" + ], + [ + 37, + "POST /api/v1/repos/acme/hist/contents/new.txt" + ], + [ + 38, + "PUT /api/v1/repos/acme/hist/contents/README.md" + ], + [ + 39, + "PUT /api/v1/repos/acme/hist/contents/README.md" + ], + [ + 40, + "PUT /api/v1/repos/acme/hist/contents/README.md" + ], + [ + 41, + "POST /api/v1/repos/acme/hist/contents/branch-file.txt" + ] + ] + }, + "K": { + "defer": [ + [ + 55, + "GET /api/v1/orgs/acme/repos" + ], + [ + 56, + "GET /api/v1/orgs/acme/repos?limit=50" + ] + ] + } } diff --git a/crates/forge/src/api.rs b/crates/forge/src/api.rs index a3bbdd34..039f2b28 100644 --- a/crates/forge/src/api.rs +++ b/crates/forge/src/api.rs @@ -172,6 +172,56 @@ pub(crate) fn paged_with( r } +/// Like [`paged`], but a page link repeats *every* request parameter, alphabetically by name (what +/// Gitea does for the commits listing), with `limit` and `page` set to the link's own values. +pub(crate) fn paged_sorted( + ctx: &Ctx, + path: &str, + total: i64, + page: i64, + limit: i64, + items: Vec, +) -> Resp { + let mut q: std::collections::BTreeMap = ctx + .req + .query + .split('&') + .filter(|kv| !kv.is_empty()) + .map(|kv| { + let (k, v) = kv.split_once('=').unwrap_or((kv, "")); + (k.to_string(), v.to_string()) + }) + .collect(); + q.insert("limit".into(), limit.to_string()); + let last = ((total + limit - 1) / limit).max(1); + // Gitea builds the links from the clamped page: asking past the end still points `prev` at the + // page before the last. + let page = page.min(last); + let mut url = |p: i64, rel: &str| { + q.insert("page".into(), p.to_string()); + let query: Vec = q.iter().map(|(k, v)| format!("{k}={v}")).collect(); + format!( + "<{}/api/v1{path}?{}>; rel=\"{rel}\"", + ctx.app.base, + query.join("&") + ) + }; + let mut links = Vec::new(); + if page < last { + links.push(url(page + 1, "next")); + links.push(url(last, "last")); + } + if page > 1 { + links.push(url(1, "first")); + links.push(url(page - 1, "prev")); + } + let mut r = Resp::json(200, &Value::Array(items)).with("x-total-count", total.to_string()); + if !links.is_empty() { + r = r.with("link", links.join(",")); + } + r +} + /// `GET /version`: the Gitea release whose behaviour this server reproduces. pub fn version(_: &mut Ctx) -> Resp { Resp::json(200, &json!({"version": GITEA_VERSION})) diff --git a/crates/forge/src/b64.rs b/crates/forge/src/b64.rs new file mode 100644 index 00000000..e617ee4d --- /dev/null +++ b/crates/forge/src/b64.rs @@ -0,0 +1,81 @@ +//! Standard base64 (RFC 4648, padded), for the contents API's file bodies and uploads. + +const T: &[u8; 64] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"; + +/// Encode `data` with padding. +pub fn encode(data: &[u8]) -> String { + let mut out = String::with_capacity(data.len().div_ceil(3) * 4); + for c in data.chunks(3) { + let n = (u32::from(c[0]) << 16) + | (u32::from(*c.get(1).unwrap_or(&0)) << 8) + | u32::from(*c.get(2).unwrap_or(&0)); + for i in 0..4 { + if i <= c.len() { + out.push(T[(n >> (18 - 6 * i) & 63) as usize] as char); + } else { + out.push('='); + } + } + } + out +} + +/// Decode padded standard base64; ASCII whitespace (line breaks) is ignored. `None` for anything +/// else outside the alphabet, bad padding, or a length that is not a multiple of four. +pub fn decode(s: &str) -> Option> { + let bytes: Vec = s.bytes().filter(|b| !b.is_ascii_whitespace()).collect(); + if !bytes.len().is_multiple_of(4) { + return None; + } + let val = |b: u8| T.iter().position(|&t| t == b).map(|p| p as u32); + let mut out = Vec::with_capacity(bytes.len() / 4 * 3); + let chunks = bytes.chunks(4).count(); + for (i, c) in bytes.chunks(4).enumerate() { + let pad = c.iter().rev().take_while(|&&b| b == b'=').count(); + if pad > 2 || (pad > 0 && i + 1 != chunks) { + return None; + } + let mut n = 0u32; + for (j, &b) in c.iter().enumerate() { + n <<= 6; + if j < 4 - pad { + n |= val(b)?; + } else if b != b'=' { + return None; + } + } + let b3 = [(n >> 16) as u8, (n >> 8) as u8, n as u8]; + out.extend_from_slice(&b3[..3 - pad]); + } + Some(out) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn round_trips_every_padding_length() { + for (raw, enc) in [ + (&b""[..], ""), + (b"f", "Zg=="), + (b"fo", "Zm8="), + (b"foo", "Zm9v"), + (b"foob", "Zm9vYg=="), + (b"hello\n", "aGVsbG8K"), + ] { + assert_eq!(encode(raw), enc); + assert_eq!(decode(enc).as_deref(), Some(raw)); + } + let all: Vec = (0..=255).collect(); + assert_eq!(decode(&encode(&all)), Some(all)); + } + + #[test] + fn decode_refuses_malformed_input_and_ignores_line_breaks() { + assert_eq!(decode("Zm9v\nYg==\n").as_deref(), Some(&b"foob"[..])); + for bad in ["Zg=", "Z", "Zg==Zg==", "Zm9v!", "=Zg=", "Zm=v", "Zg=a"] { + assert!(decode(bad).is_none(), "{bad}"); + } + } +} diff --git a/crates/forge/src/content_api.rs b/crates/forge/src/content_api.rs new file mode 100644 index 00000000..8b7b76e3 --- /dev/null +++ b/crates/forge/src/content_api.rs @@ -0,0 +1,742 @@ +//! F3a: the read half of the content API: refs, tags, commits, trees, contents and archives, in +//! the shapes recorded from Gitea. Every handler runs under the repository's shared lock +//! (`read_locked`), resolves client revisions with `gitobj::resolve` (never as git syntax) and +//! answers a busy runner 503. + +use crate::api::{page_params, paged_sorted, paged_with, pct_encode, user_json}; +use crate::b64; +use crate::gitobj::{self, Peel, RefKind, RefRow, Resolved, TreeEntry}; +use crate::gitproc::GitError; +use crate::gitread; +use crate::http::{error, not_found, Resp}; +use crate::repo_api::{acc, git_read_err, read_locked}; +use crate::repos::Access; +use crate::routes::{percent_decode, Ctx}; +use crate::store; +use crate::timefmt::rfc3339; +use serde_json::{json, Value}; +use std::collections::HashMap; +use std::path::PathBuf; + +fn dir(ctx: &Ctx, a: &Access) -> PathBuf { + ctx.app.life.repo_dir(a.repo.id) +} + +/// `{base}/api/v1/repos/{owner}/{repo}`. +fn api_base(ctx: &Ctx, a: &Access) -> String { + format!( + "{}/api/v1/repos/{}/{}", + ctx.app.base, a.repo.owner, a.repo.name + ) +} + +/// `{base}/{owner}/{repo}`: the web URLs Gitea puts in `html_url`. +fn web_base(ctx: &Ctx, a: &Access) -> String { + format!("{}/{}/{}", ctx.app.base, a.repo.owner, a.repo.name) +} + +/// A query parameter, percent-decoded. +fn param(ctx: &Ctx, k: &str) -> Option { + ctx.req.param(k).and_then(percent_decode) +} + +fn flag(ctx: &Ctx, k: &str, default: bool) -> bool { + match ctx.req.param(k) { + Some("true" | "1") => true, + Some("false" | "0") => false, + _ => default, + } +} + +/// A git failure that is the request's doing, as opposed to the repository's. +fn read_err(ctx: &Ctx, a: &Access, e: GitError) -> Resp { + match e { + GitError::OutputTooLarge => error(&ctx.app.base, 413, "the result is too large to serve"), + GitError::Timeout { .. } => error(&ctx.app.base, 504, "git timed out"), + e => git_read_err(ctx.app, ctx.conn, a.repo.id, e), + } +} + +fn is_empty(ctx: &Ctx, a: &Access) -> Result { + gitread::is_empty(&ctx.app.life.git, &dir(ctx, a)).map_err(|e| read_err(ctx, a, e)) +} + +fn resolve(ctx: &Ctx, a: &Access, rev: &str, peel: Peel) -> Result, Resp> { + gitobj::resolve( + &ctx.app.life.git, + &dir(ctx, a), + &a.repo.default_branch, + rev, + peel, + ) + .map_err(|e| read_err(ctx, a, e)) +} + +fn git_err(ctx: &Ctx, a: &Access, r: Result) -> Result { + r.map_err(|e| read_err(ctx, a, e)) +} + +// ---- JSON pieces ---- + +/// Commit authors resolved to accounts by email, once per request. +struct Users<'a> { + ctx: &'a Ctx<'a>, + cache: HashMap, +} + +impl<'a> Users<'a> { + fn new(ctx: &'a Ctx<'a>) -> Self { + Users { + ctx, + cache: HashMap::new(), + } + } + + /// The account owning `email`, or `null` (recorded: commits by unknown addresses). + fn actor(&mut self, email: &str) -> Result { + if let Some(v) = self.cache.get(&email.to_lowercase()) { + return Ok(v.clone()); + } + let ctx = self.ctx; + let v = match store::login_by_email(ctx.conn, email).map_err(|e| ctx.app.internal(&e))? { + Some(login) => { + match store::find_by_login(ctx.conn, &login).map_err(|e| ctx.app.internal(&e))? { + Some(u) => user_json(&ctx.app.base, &u), + None => Value::Null, + } + } + None => Value::Null, + }; + self.cache.insert(email.to_lowercase(), v.clone()); + Ok(v) + } +} + +fn verification() -> Value { + json!({"verified": false, "reason": "gpg.error.not_signed_commit", "signature": "", "signer": null, "payload": ""}) +} + +fn person_json(p: &gitread::Person) -> Value { + json!({"name": p.name, "email": p.email, "date": rfc3339(p.when)}) +} + +fn commit_meta(ctx: &Ctx, a: &Access, sha: &str, when: i64) -> Value { + json!({"url": format!("{}/git/commits/{sha}", api_base(ctx, a)), "sha": sha, "created": rfc3339(when)}) +} + +/// Files and line counts a commit response may carry (`None` for each the client did not ask for). +type Changes = (Option>, Option); + +/// The Gitea commit object. `changes` fills `files` and `stats` (`None` leaves them `null`). +fn commit_json( + ctx: &Ctx, + a: &Access, + users: &mut Users, + c: &gitobj::LogCommit, + verified: bool, + changes: Option, +) -> Result { + let (files, stats) = changes.unwrap_or((None, None)); + let api = api_base(ctx, a); + Ok(json!({ + "url": format!("{api}/git/commits/{}", c.oid), + "sha": c.oid, + "created": rfc3339(c.committer.when), + "html_url": format!("{}/commit/{}", web_base(ctx, a), c.oid), + "commit": { + "url": format!("{api}/git/commits/{}", c.oid), + "author": person_json(&c.author), + "committer": person_json(&c.committer), + "message": c.message, + // Recorded Gitea quirk: `tree` carries the *commit's* id, which `git/trees` accepts. + "tree": {"url": format!("{api}/git/trees/{}", c.oid), "sha": c.oid, "created": rfc3339(c.committer.when)}, + "verification": if verified { verification() } else { Value::Null }, + }, + "author": users.actor(&c.author.email)?, + "committer": users.actor(&c.committer.email)?, + "parents": c.parents.iter().map(|p| commit_meta(ctx, a, p, c.committer.when)).collect::>(), + "files": files.map(|fs| fs.iter().map(|f| json!({"filename": f.name, "status": f.status})).collect::>()), + "stats": stats.map(|s| json!({"total": s.additions + s.deletions, "additions": s.additions, "deletions": s.deletions})), + })) +} + +fn load_changes( + ctx: &Ctx, + a: &Access, + c: &gitobj::LogCommit, + files: bool, + stats: bool, +) -> Result, Resp> { + if !files && !stats { + return Ok(None); + } + let (f, s) = git_err( + ctx, + a, + gitobj::changes( + &ctx.app.life.git, + &dir(ctx, a), + c.parents.first().map(String::as_str), + &c.oid, + ), + )?; + Ok(Some((files.then_some(f), stats.then_some(s)))) +} + +fn ref_json(ctx: &Ctx, a: &Access, r: &RefRow) -> Value { + let api = api_base(ctx, a); + let tag = r.kind == "tag"; + json!({ + "ref": r.name, + "url": format!("{api}/git/{}", r.name), + "object": { + "type": r.kind, + "sha": r.oid, + "url": format!("{api}/git/{}/{}", if tag { "tags" } else { "commits" }, r.oid), + }, + }) +} + +// ---- refs and tags ---- + +fn all_refs(ctx: &Ctx, a: &Access) -> Result, Resp> { + let (git, d) = (&ctx.app.life.git, dir(ctx, a)); + let mut rows = git_err(ctx, a, gitobj::list_refs(git, &d, "refs/heads"))?; + rows.extend(git_err(ctx, a, gitobj::list_refs(git, &d, "refs/tags"))?); + Ok(rows) +} + +fn gone(ctx: &Ctx) -> Resp { + not_found(&ctx.app.base, "The target couldn't be found.", None) +} + +/// `GET /repos/{o}/{r}/git/refs`. +pub fn list_git_refs(ctx: &mut Ctx) -> Resp { + let a = match acc(ctx) { + Ok(a) => a.clone(), + Err(r) => return r, + }; + read_locked(ctx, &a, |ctx| match all_refs(ctx, &a) { + Ok(rows) if rows.is_empty() => gone(ctx), + Ok(rows) => Resp::json( + 200, + &Value::Array(rows.iter().map(|r| ref_json(ctx, &a, r)).collect()), + ), + Err(r) => r, + }) +} + +/// `GET /repos/{o}/{r}/git/refs/{ref..}`: the refs equal to `refs/` or below it. +pub fn get_git_refs(ctx: &mut Ctx) -> Resp { + let a = match acc(ctx) { + Ok(a) => a.clone(), + Err(r) => return r, + }; + let want = format!("refs/{}", ctx.param("ref").trim_end_matches('/')); + read_locked(ctx, &a, |ctx| match all_refs(ctx, &a) { + Ok(rows) => { + let hit: Vec = rows + .iter() + .filter(|r| r.name == want || r.name.starts_with(&format!("{want}/"))) + .map(|r| ref_json(ctx, &a, r)) + .collect(); + if hit.is_empty() { + gone(ctx) + } else { + Resp::json(200, &Value::Array(hit)) + } + } + Err(r) => r, + }) +} + +/// `GET /repos/{o}/{r}/tags`, newest first. +pub fn list_tags(ctx: &mut Ctx) -> Resp { + let a = match acc(ctx) { + Ok(a) => a.clone(), + Err(r) => return r, + }; + read_locked(ctx, &a, |ctx| { + let mut rows = match git_err( + ctx, + &a, + gitobj::list_refs(&ctx.app.life.git, &dir(ctx, &a), "refs/tags"), + ) { + Ok(r) => r, + Err(r) => return r, + }; + // Newest first, as Gitea lists them (ties keep name order). + rows.sort_by_key(|r| std::cmp::Reverse(r.created)); + let (page, limit) = page_params(ctx); + let total = rows.len() as i64; + let start = (((page - 1) * limit) as usize).min(rows.len()); + let end = (start + limit as usize).min(rows.len()); + let web = web_base(ctx, &a); + let items = rows[start..end] + .iter() + .map(|r| { + let name = r.name.strip_prefix("refs/tags/").unwrap_or(&r.name); + json!({ + "name": name, + "message": r.subject, + "id": r.oid, + // Recorded Gitea quirk: the url names the tag object, the sha the commit. + "commit": json!({"url": format!("{}/git/commits/{}", api_base(ctx, &a), r.oid), "sha": r.commit(), "created": rfc3339(r.commit_date)}), + "zipball_url": format!("{web}/archive/{name}.zip"), + "tarball_url": format!("{web}/archive/{name}.tar.gz"), + }) + }) + .collect(); + paged_with( + ctx, + &format!("/repos/{}/{}/tags", a.repo.owner, a.repo.name), + total, + page, + limit, + &[], + items, + ) + }) +} + +// ---- commits ---- + +/// `GET /repos/{o}/{r}/commits`. +pub fn list_commits(ctx: &mut Ctx) -> Resp { + let a = match acc(ctx) { + Ok(a) => a.clone(), + Err(r) => return r, + }; + read_locked(ctx, &a, |ctx| list_commits_inner(ctx, &a)) +} + +fn list_commits_inner(ctx: &Ctx, a: &Access) -> Resp { + match is_empty(ctx, a) { + Ok(true) => return error(&ctx.app.base, 409, "Git Repository is empty."), + Ok(false) => {} + Err(r) => return r, + } + let rev = param(ctx, "sha").unwrap_or_else(|| a.repo.default_branch.clone()); + let path = param(ctx, "path").filter(|p| !p.is_empty()); + if let Some(p) = &path { + if !gitobj::valid_path(p) { + return error(&ctx.app.base, 422, "invalid path"); + } + } + let head = match resolve(ctx, a, &rev, Peel::Commit) { + Ok(Some(h)) => h, + // Gitea answers 500 here; a client error is a 404 (departure). + Ok(None) => return error(&ctx.app.base, 404, &format!("ref not found [{rev}]")), + Err(r) => return r, + }; + let (page, limit) = page_params(ctx); + let (git, d) = (&ctx.app.life.git, dir(ctx, a)); + let total = match git_err(ctx, a, gitobj::count(git, &d, &head.oid, path.as_deref())) { + Ok(t) => t as i64, + Err(r) => return r, + }; + let commits = match git_err( + ctx, + a, + gitobj::log( + git, + &d, + &head.oid, + path.as_deref(), + ((page - 1) * limit) as usize, + limit as usize, + ), + ) { + Ok(c) => c, + Err(r) => return r, + }; + let (want_stat, want_files, want_verif) = ( + flag(ctx, "stat", true), + flag(ctx, "files", true), + flag(ctx, "verification", true), + ); + let mut users = Users::new(ctx); + let mut items = Vec::with_capacity(commits.len()); + for c in &commits { + let changes = match load_changes(ctx, a, c, want_files, want_stat) { + Ok(c) => c, + Err(r) => return r, + }; + match commit_json(ctx, a, &mut users, c, want_verif, changes) { + Ok(v) => items.push(v), + Err(r) => return r, + } + } + paged_sorted( + ctx, + &format!("/repos/{}/{}/commits", a.repo.owner, a.repo.name), + total, + page, + limit, + items, + ) +} + +/// `GET /repos/{o}/{r}/git/commits/{sha}` with the `.diff` and `.patch` forms. +pub fn get_git_commit(ctx: &mut Ctx) -> Resp { + let a = match acc(ctx) { + Ok(a) => a.clone(), + Err(r) => return r, + }; + read_locked(ctx, &a, |ctx| { + let raw = ctx.param("sha").to_string(); + let (rev, form) = if let Some(r) = raw.strip_suffix(".diff") { + (r.to_string(), Some("diff")) + } else if let Some(r) = raw.strip_suffix(".patch") { + (r.to_string(), Some("patch")) + } else { + (raw, None) + }; + let oid = match resolve(ctx, &a, &rev, Peel::Commit) { + Ok(Some(r)) => r.oid, + Ok(None) => return not_found(&ctx.app.base, &rev, None), + Err(r) => return r, + }; + let (git, d) = (&ctx.app.life.git, dir(ctx, &a)); + let c = match git_err(ctx, &a, gitobj::commit(git, &d, &oid)) { + Ok(Some(c)) => c, + Ok(None) => return not_found(&ctx.app.base, &rev, None), + Err(r) => return r, + }; + let text = |bytes: Vec| Resp { + status: 200, + headers: vec![("content-type", "text/plain;charset=utf-8".into())], + body: bytes, + delay: std::time::Duration::ZERO, + }; + match form { + Some("diff") => match git_err( + ctx, + &a, + gitobj::diff_text(git, &d, c.parents.first().map(String::as_str), &c.oid), + ) { + Ok(b) => text(b), + Err(r) => r, + }, + Some(_) => match git_err(ctx, &a, gitobj::patch_text(git, &d, &c.oid)) { + Ok(b) => text(b), + Err(r) => r, + }, + None => { + let changes = match load_changes(ctx, &a, &c, true, true) { + Ok(c) => c, + Err(r) => return r, + }; + let mut users = Users::new(ctx); + match commit_json(ctx, &a, &mut users, &c, true, changes) { + Ok(v) => Resp::json(200, &v), + Err(r) => r, + } + } + } + }) +} + +// ---- trees ---- + +fn tree_entry_json(ctx: &Ctx, a: &Access, e: &TreeEntry) -> Value { + let api = api_base(ctx, a); + let url = match e.kind.as_str() { + "tree" => format!("{api}/git/trees/{}", e.oid), + "blob" => format!("{api}/git/blobs/{}", e.oid), + _ => format!("{api}/git/commits/{}", e.oid), + }; + json!({"path": e.path, "mode": e.mode, "type": e.kind, "size": e.size.unwrap_or(0), "sha": e.oid, "url": url}) +} + +/// `GET /repos/{o}/{r}/git/trees/{sha..}`. +pub fn get_git_tree(ctx: &mut Ctx) -> Resp { + let a = match acc(ctx) { + Ok(a) => a.clone(), + Err(r) => return r, + }; + read_locked(ctx, &a, |ctx| { + let rev = ctx.param("sha").to_string(); + // The `sha` answered is the one the client named: a commit's id for a branch, tag or commit, + // a tree's own id for a tree (recorded: the commit id, not its tree's). + let (shown, tree) = match resolve(ctx, &a, &rev, Peel::Commit) { + Ok(Some(c)) => match resolve(ctx, &a, &rev, Peel::Tree) { + Ok(Some(t)) => (c.oid, t.oid), + Ok(None) => return error(&ctx.app.base, 400, &format!("sha not found [{rev}]")), + Err(r) => return r, + }, + Ok(None) => match resolve(ctx, &a, &rev, Peel::Tree) { + Ok(Some(t)) => (t.oid.clone(), t.oid), + Ok(None) => return error(&ctx.app.base, 400, &format!("sha not found [{rev}]")), + Err(r) => return r, + }, + Err(r) => return r, + }; + let recursive = flag(ctx, "recursive", false); + let entries = match git_err( + ctx, + &a, + gitobj::ls_tree(&ctx.app.life.git, &dir(ctx, &a), &tree, recursive), + ) { + Ok(e) => e, + Err(r) => return r, + }; + let num = |k: &str| { + ctx.req + .param(k) + .and_then(|v| v.parse::().ok()) + .filter(|n| *n >= 1) + }; + let (page, per) = ( + num("page").unwrap_or(1), + num("per_page").unwrap_or(1000).min(1000), + ); + let total = entries.len(); + let start = ((page - 1).saturating_mul(per)).min(total); + let end = (start + per).min(total); + let body = json!({ + "sha": shown, + "url": format!("{}/git/trees/{shown}", api_base(ctx, &a)), + "tree": entries[start..end].iter().map(|e| tree_entry_json(ctx, &a, e)).collect::>(), + "truncated": end < total, + "page": page, + "total_count": total, + }); + Resp::json(200, &body).with("x-total-count", total.to_string()) + }) +} + +// ---- contents ---- + +/// One contents object. `with_content` fills `content`/`encoding` of a file. +fn content_json( + ctx: &Ctx, + a: &Access, + commit: &str, + refname: &str, + kind: RefKind, + e: &TreeEntry, + with_content: bool, + exact_last: bool, +) -> Result { + let (git, d) = (&ctx.app.life.git, dir(ctx, a)); + let api = api_base(ctx, a); + let web = web_base(ctx, a); + let enc_path = e + .path + .split('/') + .map(pct_encode) + .collect::>() + .join("/"); + let enc_ref = refname + .split('/') + .map(pct_encode) + .collect::>() + .join("/"); + let query_ref = pct_encode(refname); + let last = if exact_last { + git_err(ctx, a, gitobj::last_commit(git, &d, commit, &e.path))?.unwrap_or_default() + } else { + commit.to_string() + }; + let name = e.path.rsplit('/').next().unwrap_or(&e.path); + let seg = kind.segment(); + let ty = match (e.kind.as_str(), e.mode.as_str()) { + ("tree", _) => "dir", + ("commit", _) => "submodule", + (_, "120000") => "symlink", + _ => "file", + }; + let size = e.size.unwrap_or(0); + let (mut encoding, mut content, mut target) = (Value::Null, Value::Null, Value::Null); + if ty == "symlink" { + let b = git_err(ctx, a, gitobj::blob(git, &d, &e.oid))?; + target = Value::String(String::from_utf8_lossy(&b).to_string()); + } else if ty == "file" && with_content && size <= gitobj::MAX_BLOB { + let b = git_err(ctx, a, gitobj::blob(git, &d, &e.oid))?; + encoding = json!("base64"); + content = Value::String(b64::encode(&b)); + } + let self_url = format!("{api}/contents/{enc_path}?ref={query_ref}"); + // Recorded: directories point at `git/blobs/` too. + let git_url = format!("{api}/git/blobs/{}", e.oid); + let html = format!("{web}/src/{seg}/{enc_ref}/{enc_path}"); + Ok(json!({ + "name": name, + "path": e.path, + "sha": e.oid, + "last_commit_sha": last, + "type": ty, + "size": size, + "encoding": encoding, + "content": content, + "target": target, + "url": self_url, + "html_url": html, + "git_url": git_url, + "download_url": if ty == "file" { json!(format!("{web}/raw/{seg}/{enc_ref}/{enc_path}")) } else { Value::Null }, + "submodule_git_url": null, + "_links": {"self": self_url, "git": git_url, "html": html}, + })) +} + +/// Entries of one listing whose `last_commit_sha` is computed exactly. +const LAST_COMMIT_BUDGET: usize = 100; + +fn contents_not_found(ctx: &Ctx, id: &str, rel: &str) -> Resp { + not_found( + &ctx.app.base, + "GetContentsOrList", + Some(&format!( + "object does not exist [id: {id}, rel_path: {rel}]" + )), + ) +} + +/// `GET /repos/{o}/{r}/contents` and `/contents/{path..}`. +pub fn get_contents(ctx: &mut Ctx) -> Resp { + let a = match acc(ctx) { + Ok(a) => a.clone(), + Err(r) => return r, + }; + read_locked(ctx, &a, |ctx| { + // Recorded: an empty repository lists nothing and answers `[]` for any path. + match is_empty(ctx, &a) { + Ok(true) => return Resp::json(200, &json!([])), + Ok(false) => {} + Err(r) => return r, + } + let path = ctx.param("p").trim_matches('/').to_string(); + let refname = param(ctx, "ref").unwrap_or_else(|| a.repo.default_branch.clone()); + let head = match resolve(ctx, &a, &refname, Peel::Commit) { + Ok(Some(h)) => h, + Ok(None) => return contents_not_found(ctx, &refname, ""), + Err(r) => return r, + }; + let (git, d) = (&ctx.app.life.git, dir(ctx, &a)); + let entries: Vec; + let single; + if path.is_empty() { + single = false; + let tree = match git_err( + ctx, + &a, + gitobj::ls_tree(git, &d, &format!("{}^{{tree}}", head.oid), false), + ) { + Ok(t) => t, + Err(r) => return r, + }; + entries = tree; + } else { + let Some(e) = (match git_err(ctx, &a, gitobj::tree_entry(git, &d, &head.oid, &path)) { + Ok(e) => e, + Err(r) => return r, + }) else { + return contents_not_found(ctx, "", &path); + }; + if e.kind == "tree" { + single = false; + entries = match git_err(ctx, &a, gitobj::ls_tree(git, &d, &e.oid, false)) { + Ok(t) => t + .into_iter() + .map(|mut c| { + c.path = format!("{path}/{}", c.path); + c + }) + .collect(), + Err(r) => return r, + }; + } else { + single = true; + entries = vec![e]; + } + } + let mut items = Vec::with_capacity(entries.len()); + for (i, e) in entries.iter().enumerate() { + // `last_commit_sha` costs one git process per entry: past LAST_COMMIT_BUDGET entries + // a listing reports the commit it was read at instead (documented approximation). + match content_json( + ctx, + &a, + &head.oid, + &refname, + head.kind, + e, + single, + i < LAST_COMMIT_BUDGET, + ) { + Ok(v) => items.push(v), + Err(r) => return r, + } + } + if single { + Resp::json(200, &items[0]) + } else { + Resp::json(200, &Value::Array(items)) + } + }) +} + +// ---- archive ---- + +/// `GET /repos/{o}/{r}/archive/{ref..}.{zip,tar.gz}`. +pub fn get_archive(ctx: &mut Ctx) -> Resp { + let a = match acc(ctx) { + Ok(a) => a.clone(), + Err(r) => return r, + }; + read_locked(ctx, &a, |ctx| { + let raw = ctx.param("a").to_string(); + let (rev, zip) = if let Some(r) = raw.strip_suffix(".zip") { + (r, true) + } else if let Some(r) = raw.strip_suffix(".tar.gz") { + (r, false) + } else { + return error(&ctx.app.base, 400, &format!("Unknown archive type: {raw}")); + }; + let oid = match resolve(ctx, &a, rev, Peel::Commit) { + Ok(Some(r)) => r.oid, + Ok(None) => { + return error( + &ctx.app.base, + 404, + &format!("unrecognized repository reference: {rev}"), + ) + } + Err(r) => return r, + }; + let (git, d) = (&ctx.app.life.git, dir(ctx, &a)); + let tar = match git_err(ctx, &a, gitobj::archive(git, &d, &oid, &a.repo.name, zip)) { + Ok(b) => b, + Err(r) => return r, + }; + let body = if zip { + tar + } else { + use std::io::Write; + let mut gz = flate2::write::GzEncoder::new(Vec::new(), flate2::Compression::default()); + if gz.write_all(&tar).is_err() { + return ctx.app.internal(&"gzip failed"); + } + match gz.finish() { + Ok(b) => b, + Err(e) => return ctx.app.internal(&e), + } + }; + Resp { + status: 200, + headers: vec![ + ("content-type", "application/octet-stream".into()), + ( + "link", + format!( + "<{}/archive/{oid}.tar.gz?rev={oid}>; rel=\"immutable\"", + api_base(ctx, &a) + ), + ), + ], + body, + delay: std::time::Duration::ZERO, + } + }) +} diff --git a/crates/forge/src/gitobj.rs b/crates/forge/src/gitobj.rs new file mode 100644 index 00000000..52cf966f --- /dev/null +++ b/crates/forge/src/gitobj.rs @@ -0,0 +1,668 @@ +//! Parsed views of a bare repository for the content API: refs, tags, history, trees, blobs, diffs +//! and archives. Everything here goes through the [`Git`] runner (own process group, scrubbed +//! environment, explicit `--git-dir`). +//! +//! **Argument hygiene.** A client string is never handed to git as a revision expression. A +//! revision is resolved by [`resolve`] to a full object id from a fixed set of shapes (`HEAD`, +//! `refs/heads/`, `refs/tags/`, a hex object id); only ids reach every other +//! command. Paths are literals (`--literal-pathspecs`, after `--`) and pass [`valid_path`] first. +//! Output is NUL-delimited where it carries names, and capped by the runner. + +use crate::gitproc::{Git, GitError, Out, Spec}; +use crate::gitread::Person; +use crate::refname; +use std::path::Path; +use std::time::Duration; + +/// The empty tree: the base of a root commit's diff. +const EMPTY_TREE: &str = "4b825dc642cb6eb9a060e54bf8d69288fbee4904"; + +/// Output cap of listings and diffs. +pub const BIG: usize = 64 << 20; + +/// Output cap of a single blob read for the contents API (larger files answer without content). +pub const MAX_BLOB: u64 = 16 << 20; + +/// Output cap of an archive (kept in memory; see the plan). +pub const MAX_ARCHIVE: usize = 128 << 20; + +fn run(git: &Git, dir: &Path, args: &[&str], cap: Option) -> Result { + git.run(&Spec { + repo: Some(dir), + args, + max_output: cap, + timeout: cap.map(|_| Duration::from_secs(120)), + ..Default::default() + }) +} + +/// stdout of a command that must succeed. +fn must(out: Out) -> Result, GitError> { + if out.ok() { + Ok(out.stdout) + } else { + Err(GitError::Failed( + String::from_utf8_lossy(&out.stderr).trim().to_string(), + )) + } +} + +/// A 4 to 64 digit hexadecimal string (an object id, full or abbreviated). +pub fn is_hex(s: &str) -> bool { + (4..=64).contains(&s.len()) && s.bytes().all(|b| b.is_ascii_hexdigit()) +} + +/// A path inside a tree as a client may name it: relative, no empty, `.` or `..` component, no +/// NUL or other control character, not absurdly long. +pub fn valid_path(p: &str) -> bool { + !p.is_empty() + && p.len() <= 4096 + && !p.starts_with('/') + && !p.chars().any(char::is_control) + && p.split('/').all(|c| !c.is_empty() && c != "." && c != "..") +} + +/// How a revision was named; picks the `src//` segment of the html URLs. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum RefKind { + Branch, + Tag, + Commit, +} + +impl RefKind { + /// The path segment Gitea's html URLs use. + pub fn segment(self) -> &'static str { + match self { + RefKind::Branch => "branch", + RefKind::Tag => "tag", + RefKind::Commit => "commit", + } + } +} + +/// A resolved revision. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Resolved { + pub oid: String, + pub kind: RefKind, +} + +/// What a revision is peeled to. +#[derive(Debug, Clone, Copy)] +pub enum Peel { + Commit, + Tree, +} + +/// Resolve `rev` (a branch, a tag, `HEAD` or an object id) to an object id of kind `peel`. `None` +/// when it names nothing; a git failure that is not "not found" is an error. +pub fn resolve( + git: &Git, + dir: &Path, + default_branch: &str, + rev: &str, + peel: Peel, +) -> Result, GitError> { + let suffix = match peel { + Peel::Commit => "^{commit}", + Peel::Tree => "^{tree}", + }; + let mut candidates: Vec<(String, RefKind, bool)> = Vec::new(); + if rev == "HEAD" { + candidates.push((refname::branch_ref(default_branch), RefKind::Branch, false)); + } else { + if refname::valid_branch(rev) { + candidates.push((refname::branch_ref(rev), RefKind::Branch, false)); + candidates.push((format!("refs/tags/{rev}"), RefKind::Tag, false)); + } + if is_hex(rev) { + candidates.push((rev.to_ascii_lowercase(), RefKind::Commit, true)); + } + } + for (spec, kind, lenient) in candidates { + let full = format!("{spec}{suffix}"); + let out = run( + git, + dir, + &["rev-parse", "--verify", "-q", "--end-of-options", &full], + None, + )?; + match out.status.code() { + Some(0) => { + return Ok(Some(Resolved { + oid: out.text(), + kind, + })) + } + Some(1) => {} + // An abbreviation can be ambiguous or name an object of another kind: not found. + _ if lenient => {} + _ => { + return Err(GitError::Failed( + String::from_utf8_lossy(&out.stderr).trim().to_string(), + )) + } + } + } + Ok(None) +} + +/// A branch or tag ref. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct RefRow { + /// Full name, `refs/heads/x` or `refs/tags/x`. + pub name: String, + pub oid: String, + /// `commit` or `tag` (an annotated tag object). + pub kind: String, + /// For an annotated tag, the commit it ends at. + pub peeled: Option, + /// First line of the commit or tag message. + pub subject: String, + /// When the ref's own object was created (tagger date for an annotated tag). + pub created: i64, + /// Committer date of the commit the ref ends at. + pub commit_date: i64, +} + +impl RefRow { + /// The commit the ref points at, through an annotated tag. + pub fn commit(&self) -> &str { + self.peeled.as_deref().unwrap_or(&self.oid) + } +} + +/// Branches and tags only (`refs/forge/`, `refs/pull/` and the rest are never listed), sorted by +/// name. +pub fn list_refs(git: &Git, dir: &Path, prefix: &str) -> Result, GitError> { + debug_assert!(prefix == "refs/heads" || prefix == "refs/tags"); + let out = must(run( + git, + dir, + &[ + "for-each-ref", + "--sort=refname", + "--format=%(refname)%00%(objectname)%00%(objecttype)%00%(*objectname)%00%(creatordate:unix)%00%(committerdate:unix)%00%(*committerdate:unix)%00%(contents:subject)", + prefix, + ], + Some(BIG), + )?)?; + Ok(String::from_utf8_lossy(&out) + .lines() + .filter_map(|l| { + let mut f = l.splitn(8, '\0'); + let (name, oid, kind, peeled, created, cdate, pdate, subject) = ( + f.next()?, + f.next()?, + f.next()?, + f.next()?, + f.next()?, + f.next()?, + f.next()?, + f.next()?, + ); + Some(RefRow { + name: name.to_string(), + oid: oid.to_string(), + kind: kind.to_string(), + peeled: (!peeled.is_empty()).then(|| peeled.to_string()), + subject: subject.to_string(), + created: created.parse().unwrap_or(0), + commit_date: if peeled.is_empty() { cdate } else { pdate } + .parse() + .unwrap_or(0), + }) + }) + .collect()) +} + +/// One commit as `log` reports it. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct LogCommit { + pub oid: String, + pub parents: Vec, + pub tree: String, + pub author: Person, + pub committer: Person, + pub message: String, +} + +const LOG_FORMAT: &str = "--format=%H%x00%P%x00%T%x00%an%x00%ae%x00%at%x00%cn%x00%ce%x00%ct%x00%B"; + +fn parse_log(raw: &[u8]) -> Vec { + let text = String::from_utf8_lossy(raw); + let mut toks = text.split('\0'); + let mut out = Vec::new(); + loop { + let mut f = [""; 10]; + for slot in f.iter_mut() { + match toks.next() { + Some(t) => *slot = t, + None => return out, + } + } + // `-z` puts a NUL after each record: the first token of the next one is the id. + if f[0].is_empty() { + return out; + } + out.push(LogCommit { + oid: f[0].trim_start_matches('\n').to_string(), + parents: f[1].split_whitespace().map(str::to_string).collect(), + tree: f[2].to_string(), + author: Person { + name: f[3].to_string(), + email: f[4].to_string(), + when: f[5].parse().unwrap_or(0), + }, + committer: Person { + name: f[6].to_string(), + email: f[7].to_string(), + when: f[8].parse().unwrap_or(0), + }, + message: f[9].to_string(), + }); + } +} + +/// `limit` commits of the history of `oid` after skipping `skip`, newest first; `path` restricts +/// to commits that touched it. +pub fn log( + git: &Git, + dir: &Path, + oid: &str, + path: Option<&str>, + skip: usize, + limit: usize, +) -> Result, GitError> { + let (skip, limit) = (format!("--skip={skip}"), format!("--max-count={limit}")); + let mut args = vec![ + "--literal-pathspecs", + "log", + "-z", + LOG_FORMAT, + skip.as_str(), + limit.as_str(), + "--end-of-options", + oid, + ]; + if let Some(p) = path { + args.extend(["--", p]); + } + Ok(parse_log(&must(run(git, dir, &args, Some(BIG))?)?)) +} + +/// One commit by full id, or `None` when it is not a commit. +pub fn commit(git: &Git, dir: &Path, oid: &str) -> Result, GitError> { + Ok(log(git, dir, oid, None, 0, 1)?.into_iter().next()) +} + +/// How many commits are reachable from `oid` (through `path`, if given). +pub fn count(git: &Git, dir: &Path, oid: &str, path: Option<&str>) -> Result { + let mut args = vec![ + "--literal-pathspecs", + "rev-list", + "--count", + "--end-of-options", + oid, + ]; + if let Some(p) = path { + args.extend(["--", p]); + } + let out = must(run(git, dir, &args, None)?)?; + Ok(String::from_utf8_lossy(&out).trim().parse().unwrap_or(0)) +} + +/// A file a commit changed. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct FileChange { + pub name: String, + /// `added`, `modified`, `removed` (Gitea's words). + pub status: &'static str, +} + +/// Lines added and removed across a commit. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] +pub struct Stats { + pub additions: u64, + pub deletions: u64, +} + +/// Parse `diff-tree -r -z --raw --numstat` output. +fn parse_changes(raw: &[u8]) -> (Vec, Stats) { + let text = String::from_utf8_lossy(raw); + let mut toks = text.split('\0'); + let (mut files, mut stats) = (Vec::new(), Stats::default()); + while let Some(t) = toks.next() { + if let Some(meta) = t.strip_prefix(':') { + let status = meta.split_whitespace().last().unwrap_or("M"); + let Some(name) = toks.next() else { break }; + files.push(FileChange { + name: name.to_string(), + status: match status.chars().next() { + Some('A') => "added", + Some('D') => "removed", + _ => "modified", + }, + }); + } else if let Some((a, rest)) = t.split_once('\t') { + if let Some((d, _path)) = rest.split_once('\t') { + // `-` for binary files + stats.additions += a.parse::().unwrap_or(0); + stats.deletions += d.parse::().unwrap_or(0); + } + } + } + (files, stats) +} + +/// Files and line counts a commit changed against its first parent (the empty tree for a root). +pub fn changes( + git: &Git, + dir: &Path, + parent: Option<&str>, + oid: &str, +) -> Result<(Vec, Stats), GitError> { + let base = parent.unwrap_or(EMPTY_TREE); + let out = must(run( + git, + dir, + &[ + "diff-tree", + "-r", + "-z", + "--no-renames", + "--no-ext-diff", + "--raw", + "--numstat", + "--end-of-options", + base, + oid, + ], + Some(BIG), + )?)?; + Ok(parse_changes(&out)) +} + +/// The unified diff of a commit against its first parent (`.diff`). +pub fn diff_text( + git: &Git, + dir: &Path, + parent: Option<&str>, + oid: &str, +) -> Result, GitError> { + let base = parent.unwrap_or(EMPTY_TREE); + must(run( + git, + dir, + &[ + "diff-tree", + "-p", + "-r", + "--no-color", + "--no-renames", + "--no-ext-diff", + "--no-textconv", + "--end-of-options", + base, + oid, + ], + Some(BIG), + )?) +} + +/// A mailbox-format patch of one commit (`.patch`). +pub fn patch_text(git: &Git, dir: &Path, oid: &str) -> Result, GitError> { + must(run( + git, + dir, + &[ + "format-patch", + "-1", + "--root", + "--stdout", + "--no-signature", + "--no-color", + "--no-ext-diff", + "--no-textconv", + "--no-renames", + "--end-of-options", + oid, + ], + Some(BIG), + )?) +} + +/// One entry of a tree. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct TreeEntry { + /// Six digits, e.g. `100644`, `040000`, `120000`, `160000`. + pub mode: String, + /// `blob`, `tree` or `commit` (a submodule). + pub kind: String, + pub oid: String, + /// Blob size; `None` for trees and submodules. + pub size: Option, + pub path: String, +} + +fn parse_tree(raw: &[u8]) -> Vec { + String::from_utf8_lossy(raw) + .split('\0') + .filter(|r| !r.is_empty()) + .filter_map(|r| { + let (meta, path) = r.split_once('\t')?; + let mut f = meta.split_whitespace(); + let (mode, kind, oid, size) = (f.next()?, f.next()?, f.next()?, f.next()?); + Some(TreeEntry { + mode: format!("{mode:0>6}"), + kind: kind.to_string(), + oid: oid.to_string(), + size: size.parse().ok(), + path: path.to_string(), + }) + }) + .collect() +} + +/// The entries of the tree `oid` (a tree id, from [`resolve`] with [`Peel::Tree`]); `recursive` +/// lists everything below, with the directories themselves, in path order. +pub fn ls_tree( + git: &Git, + dir: &Path, + oid: &str, + recursive: bool, +) -> Result, GitError> { + let mut args = vec!["ls-tree", "-z", "-l"]; + if recursive { + args.extend(["-r", "-t"]); + } + args.extend(["--end-of-options", oid]); + Ok(parse_tree(&must(run(git, dir, &args, Some(BIG))?)?)) +} + +/// The entry at `path` in commit `commit`, or `None`. +pub fn tree_entry( + git: &Git, + dir: &Path, + commit: &str, + path: &str, +) -> Result, GitError> { + if !valid_path(path) { + return Ok(None); + } + let out = must(run( + git, + dir, + &[ + "--literal-pathspecs", + "ls-tree", + "-z", + "-l", + "--end-of-options", + commit, + "--", + path, + ], + None, + )?)?; + Ok(parse_tree(&out).into_iter().find(|e| e.path == path)) +} + +/// The newest commit on `commit`'s history that touched `path`. +pub fn last_commit( + git: &Git, + dir: &Path, + commit: &str, + path: &str, +) -> Result, GitError> { + let out = must(run( + git, + dir, + &[ + "--literal-pathspecs", + "log", + "-n1", + "--format=%H", + "--end-of-options", + commit, + "--", + path, + ], + None, + )?)?; + let t = String::from_utf8_lossy(&out).trim().to_string(); + Ok((!t.is_empty()).then_some(t)) +} + +/// The bytes of blob `oid`. +pub fn blob(git: &Git, dir: &Path, oid: &str) -> Result, GitError> { + must(run( + git, + dir, + &["cat-file", "blob", oid], + Some(MAX_BLOB as usize + 1024), + )?) +} + +/// A tar or zip archive of commit `oid` with every path under `prefix/`. +pub fn archive( + git: &Git, + dir: &Path, + oid: &str, + prefix: &str, + zip: bool, +) -> Result, GitError> { + let fmt = if zip { "--format=zip" } else { "--format=tar" }; + let prefix = format!("--prefix={prefix}/"); + must(run( + git, + dir, + &["archive", fmt, &prefix, "--end-of-options", oid], + Some(MAX_ARCHIVE), + )?) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn paths_are_literal_relative_and_clean() { + for ok in ["a", "a/b.txt", "dir/with space/ΓΌ.rs", ".hidden", "a/.b"] { + assert!(valid_path(ok), "{ok}"); + } + for bad in [ + "", + "/a", + "a//b", + "a/", + "./a", + "a/./b", + "../a", + "a/..", + "a/../b", + "a\0b", + "a\nb", + &"x".repeat(5000), + ] { + assert!(!valid_path(bad), "{bad:?}"); + } + } + + #[test] + fn hex_ids_are_four_to_sixty_four_digits() { + assert!(is_hex("abcd") && is_hex(&"F".repeat(64))); + assert!(!is_hex("abc") && !is_hex(&"a".repeat(65)) && !is_hex("abcg") && !is_hex("")); + } + + #[test] + fn log_records_split_on_their_fixed_field_count() { + let rec = + |h: &str, msg: &str| format!("{h}\0p1 p2\0t\0A B\0a@b.c\0100\0C\0c@d.e\0200\0{msg}\0"); + let raw = format!( + "{}{}", + rec("aaaa", "first\n\nbody\n"), + rec("bbbb", "second\n") + ); + let v = parse_log(raw.as_bytes()); + assert_eq!(v.len(), 2); + assert_eq!(v[0].oid, "aaaa"); + assert_eq!(v[0].parents, ["p1", "p2"]); + assert_eq!(v[0].author.name, "A B"); + assert_eq!(v[0].committer.when, 200); + assert_eq!(v[0].message, "first\n\nbody\n"); + assert_eq!(v[1].message, "second\n"); + assert!(parse_log(b"").is_empty()); + assert!( + parse_log(b"aaaa\0only\0three").is_empty(), + "a truncated record is dropped" + ); + } + + #[test] + fn changes_parse_raw_and_numstat_together() { + let raw = b":100644 100644 aaa bbb M\0a.txt\0:000000 100644 000 ccc A\0n.txt\0:100644 000000 ddd 000 D\0d/b.txt\0\ + 1\t0\ta.txt\x001\t0\tn.txt\x000\t2\td/b.txt\0-\t-\tbin\0"; + let (files, stats) = parse_changes(raw); + assert_eq!( + files + .iter() + .map(|f| (f.name.as_str(), f.status)) + .collect::>(), + [ + ("a.txt", "modified"), + ("n.txt", "added"), + ("d/b.txt", "removed") + ] + ); + assert_eq!( + stats, + Stats { + additions: 2, + deletions: 2 + } + ); + } + + #[test] + fn tree_entries_parse_with_padded_sizes_and_odd_names() { + let raw = [ + "100644 blob aaa 14\tREADME.md", + "040000 tree bbb -\tdocs", + "100644 blob ccc 3\tdocs/a b\tc.md", + "160000 commit ddd -\tsub", + ] + .join("\0") + + "\0"; + let v = parse_tree(raw.as_bytes()); + assert_eq!(v.len(), 4); + assert_eq!((v[0].mode.as_str(), v[0].size), ("100644", Some(14))); + assert_eq!((v[1].mode.as_str(), v[1].size), ("040000", None)); + assert_eq!( + v[2].path, "docs/a b\tc.md", + "a tab inside a name survives (only the first splits)" + ); + assert_eq!(v[3].kind, "commit"); + } +} diff --git a/crates/forge/src/gitproc.rs b/crates/forge/src/gitproc.rs index e9981c02..84861a66 100644 --- a/crates/forge/src/gitproc.rs +++ b/crates/forge/src/gitproc.rs @@ -88,6 +88,10 @@ pub struct Spec<'a> { pub args: &'a [&'a str], pub stdin: Option<&'a [u8]>, pub identity: Option<&'a Identity>, + /// Larger output cap than the runner's default (archives, diffs, big listings). + pub max_output: Option, + /// Longer deadline than the runner's default. + pub timeout: Option, } /// The runner. @@ -240,9 +244,10 @@ impl Git { out }) }; + let cap = spec.max_output.unwrap_or(self.max_output); let t_out = drain( Box::new(child.stdout.take().expect("piped")), - self.max_output, + cap, over.clone(), ); let t_err = drain( @@ -261,7 +266,7 @@ impl Git { // Detect the leader's exit without reaping it (`waitid` with WNOWAIT): while the leader is // an unreaped zombie its pid, and so the process-group id, cannot be reused, so the group // signal below can never reach an unrelated group. Reap only after signalling. - let deadline = Instant::now() + self.timeout; + let deadline = Instant::now() + spec.timeout.unwrap_or(self.timeout); let outcome = loop { match exited_unreaped(pgid) { Ok(true) => break None, diff --git a/crates/forge/src/gitread.rs b/crates/forge/src/gitread.rs index 093e463c..60759712 100644 --- a/crates/forge/src/gitread.rs +++ b/crates/forge/src/gitread.rs @@ -190,6 +190,7 @@ mod tests { args: &["commit-tree", &tree], stdin: Some(msg.as_bytes()), identity: Some(&id), + ..Default::default() }) .unwrap() .text(); diff --git a/crates/forge/src/lib.rs b/crates/forge/src/lib.rs index 7475b480..e08016a2 100644 --- a/crates/forge/src/lib.rs +++ b/crates/forge/src/lib.rs @@ -10,8 +10,11 @@ pub mod admin; pub mod api; pub mod auth; +pub mod b64; pub mod cgi; +pub mod content_api; pub mod db; +pub mod gitobj; pub mod gitproc; pub mod gitread; pub mod hook; diff --git a/crates/forge/src/lifecycle.rs b/crates/forge/src/lifecycle.rs index efa5041f..b6eca72d 100644 --- a/crates/forge/src/lifecycle.rs +++ b/crates/forge/src/lifecycle.rs @@ -188,6 +188,7 @@ impl Lifecycle { args, stdin, identity: id, + ..Default::default() })?; if out.ok() { Ok(out.text()) diff --git a/crates/forge/src/repo_api.rs b/crates/forge/src/repo_api.rs index f9d18584..f8662cfb 100644 --- a/crates/forge/src/repo_api.rs +++ b/crates/forge/src/repo_api.rs @@ -15,7 +15,7 @@ use crate::store::{self, User}; use crate::timefmt::{now, rfc3339}; use serde_json::{json, Value}; -fn acc<'a>(ctx: &'a Ctx) -> Result<&'a Access, Resp> { +pub(crate) fn acc<'a>(ctx: &'a Ctx) -> Result<&'a Access, Resp> { ctx.access.as_ref().ok_or_else(|| { ctx.app .internal(&"repository route without a resolved repository") @@ -41,7 +41,12 @@ fn life_err(app: &App, e: LifecycleError) -> Resp { /// The one mapping of a failed read of a repository: busy is 503; a repository that is no /// longer `live` (a read that raced its delete) is the same 404 as a missing one; anything else /// is a 500. -fn git_read_err(app: &App, conn: &rusqlite::Connection, repo_id: i64, e: GitError) -> Resp { +pub(crate) fn git_read_err( + app: &App, + conn: &rusqlite::Connection, + repo_id: i64, + e: GitError, +) -> Resp { if matches!(e, GitError::Busy) { return error(&app.base, 503, "too many git operations in flight"); } @@ -650,7 +655,7 @@ fn branch_json(ctx: &Ctx, a: &Access, br: &gitread::Branch) -> Result Resp) -> Resp { +pub(crate) fn read_locked(ctx: &Ctx, a: &Access, f: impl FnOnce(&Ctx) -> Resp) -> Resp { match ctx.app.life.with_read(ctx.conn, a.repo.id, || f(ctx)) { Ok(Some(r)) => r, Ok(None) => not_found(&ctx.app.base, "The target couldn't be found.", None), diff --git a/crates/forge/src/routes.rs b/crates/forge/src/routes.rs index 64a13b8a..6eebf243 100644 --- a/crates/forge/src/routes.rs +++ b/crates/forge/src/routes.rs @@ -9,7 +9,7 @@ use crate::lifecycle::Lifecycle; use crate::limits::{Backoff, Gate, Proxies}; use crate::repos::{self, Access, Perm}; use crate::scope::{self, Cat, Level, Need}; -use crate::{api, db, repo_api, store}; +use crate::{api, content_api, db, repo_api, store}; use rusqlite::Connection; use std::path::PathBuf; @@ -355,6 +355,96 @@ pub static ROUTES: &[Route] = &[ Repo(Perm::Write), repo_api::delete_branch, ), + r( + "GET", + "/repos/{o}/{r}/git/refs", + Token, + n(Cat::Repository, Level::Read), + Repo(Perm::Read), + content_api::list_git_refs, + ), + r( + "GET", + "/repos/{o}/{r}/git/refs/{ref..}", + Token, + n(Cat::Repository, Level::Read), + Repo(Perm::Read), + content_api::get_git_refs, + ), + r( + "GET", + "/repos/{o}/{r}/tags", + Token, + n(Cat::Repository, Level::Read), + Repo(Perm::Read), + content_api::list_tags, + ), + r( + "GET", + "/repos/{o}/{r}/commits", + Token, + n(Cat::Repository, Level::Read), + Repo(Perm::Read), + content_api::list_commits, + ), + r( + "GET", + "/repos/{o}/{r}/git/commits/{sha}", + Token, + n(Cat::Repository, Level::Read), + Repo(Perm::Read), + content_api::get_git_commit, + ), + r( + "GET", + "/repos/{o}/{r}/git/trees/{sha..}", + Token, + n(Cat::Repository, Level::Read), + Repo(Perm::Read), + content_api::get_git_tree, + ), + r( + "GET", + "/repos/{o}/{r}/contents", + Token, + n(Cat::Repository, Level::Read), + Repo(Perm::Read), + content_api::get_contents, + ), + r( + "GET", + "/repos/{o}/{r}/contents/{p..}", + Token, + n(Cat::Repository, Level::Read), + Repo(Perm::Read), + content_api::get_contents, + ), + // F3b builds the writes; until then they answer 501 after the scope and permission checks, so + // a read-only token still gets its recorded 403. + r( + "POST", + "/repos/{o}/{r}/contents/{p..}", + Token, + n(Cat::Repository, Level::Write), + Repo(Perm::Write), + api::not_implemented, + ), + r( + "PUT", + "/repos/{o}/{r}/contents/{p..}", + Token, + n(Cat::Repository, Level::Write), + Repo(Perm::Write), + api::not_implemented, + ), + r( + "GET", + "/repos/{o}/{r}/archive/{a..}", + Token, + n(Cat::Repository, Level::Read), + Repo(Perm::Read), + content_api::get_archive, + ), ]; /// Captured path parameters, in pattern order. diff --git a/crates/forge/tests/common/mod.rs b/crates/forge/tests/common/mod.rs index af09da69..a35781b1 100644 --- a/crates/forge/tests/common/mod.rs +++ b/crates/forge/tests/common/mod.rs @@ -261,6 +261,7 @@ impl Srv { args, stdin, identity: Some(&id_), + ..Default::default() }) .unwrap(); assert!( From 88752ba6213764f6b63872e3e7a1e9dd8a6955ad Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 23 Oct 2026 16:12:31 +0300 Subject: [PATCH 3/7] test(forge): F3a e2e (revision and path hygiene, tags, trees, archive, empty repo, visibility parity); tag tie-break --- crates/forge/src/content_api.rs | 4 +- crates/forge/tests/e2e_content.rs | 371 ++++++++++++++++++++++++++++++ 2 files changed, 373 insertions(+), 2 deletions(-) create mode 100644 crates/forge/tests/e2e_content.rs diff --git a/crates/forge/src/content_api.rs b/crates/forge/src/content_api.rs index 8b7b76e3..5cd0d63e 100644 --- a/crates/forge/src/content_api.rs +++ b/crates/forge/src/content_api.rs @@ -265,8 +265,8 @@ pub fn list_tags(ctx: &mut Ctx) -> Resp { Ok(r) => r, Err(r) => return r, }; - // Newest first, as Gitea lists them (ties keep name order). - rows.sort_by_key(|r| std::cmp::Reverse(r.created)); + // Newest first, as Gitea lists them; ties (a second's resolution) by name, descending. + rows.sort_by(|x, y| (y.created, &y.name).cmp(&(x.created, &x.name))); let (page, limit) = page_params(ctx); let total = rows.len() as i64; let start = (((page - 1) * limit) as usize).min(rows.len()); diff --git a/crates/forge/tests/e2e_content.rs b/crates/forge/tests/e2e_content.rs new file mode 100644 index 00000000..25cb9727 --- /dev/null +++ b/crates/forge/tests/e2e_content.rs @@ -0,0 +1,371 @@ +//! F3a: the read half of the content API against a real repository pushed with real `git`. +#![cfg(unix)] + +mod common; +use common::*; +use serde_json::{json, Value}; +use std::path::{Path, PathBuf}; +use std::process::Command; + +struct World { + s: Srv, + auth: String, + dir: PathBuf, +} + +fn git(cwd: &Path, args: &[&str]) -> String { + let out = Command::new("git") + .env_clear() + .env("PATH", "/usr/bin:/bin:/usr/local/bin:/opt/homebrew/bin") + .env("HOME", cwd) + .env("GIT_CONFIG_NOSYSTEM", "1") + .env("GIT_AUTHOR_NAME", "T") + .env("GIT_AUTHOR_EMAIL", "t@example.invalid") + .env("GIT_COMMITTER_NAME", "T") + .env("GIT_COMMITTER_EMAIL", "t@example.invalid") + .current_dir(cwd) + .args(args) + .output() + .expect("git"); + assert!( + out.status.success(), + "git {args:?}: {}", + String::from_utf8_lossy(&out.stderr) + ); + String::from_utf8_lossy(&out.stdout).trim().to_string() +} + +/// A repository `siteadmin/smart` with history: root (README.md), then docs/guide.md and an edit, +/// tag `v1` (annotated) and a lightweight tag `light`, branch `feat/x`. +fn world() -> World { + let s = Srv::start(); + let auth = token(&s.admin); + let r = s.post( + "/api/v1/user/repos", + &auth, + json!({"name": "smart", "default_branch": "main"}), + ); + assert_eq!(r.status, 201, "{}", r.msg()); + let dir = std::env::temp_dir().join(format!( + "scl-forge-content-{}-{}", + std::process::id(), + s.addr.port() + )); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let w = dir.join("w"); + git(&dir, &["init", "-q", "-b", "main", w.to_str().unwrap()]); + std::fs::write(w.join("README.md"), "# smart\n").unwrap(); + git(&w, &["add", "."]); + git(&w, &["commit", "-q", "-m", "root"]); + git(&w, &["tag", "light"]); + std::fs::create_dir_all(w.join("docs")).unwrap(); + std::fs::write(w.join("docs/guide.md"), "guide\n").unwrap(); + git(&w, &["add", "."]); + git(&w, &["commit", "-q", "-m", "second"]); + git(&w, &["tag", "-a", "v1", "-m", "release one"]); + git(&w, &["checkout", "-q", "-b", "feat/x"]); + std::fs::write(w.join("x.txt"), "x\n").unwrap(); + git(&w, &["add", "."]); + git(&w, &["commit", "-q", "-m", "feat"]); + git(&w, &["checkout", "-q", "main"]); + std::fs::write(w.join("README.md"), "# smart\nedited\n").unwrap(); + git(&w, &["commit", "-q", "-a", "-m", "third"]); + let url = format!("{}/siteadmin/smart.git", s.base); + let h = format!("http.extraHeader=Authorization: {auth}"); + git( + &w, + &["-c", &h, "push", "-q", &url, "main", "feat/x", "--tags"], + ); + // wait for the push's transport to finish so the repository is quiescent + let end = std::time::Instant::now() + std::time::Duration::from_secs(10); + while s.app.life.registry.live() != 0 && std::time::Instant::now() < end { + std::thread::sleep(std::time::Duration::from_millis(20)); + } + World { s, auth, dir } +} + +impl World { + fn get(&self, path: &str) -> Resp { + self.s + .get(&format!("/api/v1/repos/siteadmin/smart{path}"), &self.auth) + } + fn json(&self, path: &str) -> Value { + let r = self.get(path); + assert_eq!(r.status, 200, "{path}: {}", String::from_utf8_lossy(&r.raw)); + r.json() + } +} + +impl Drop for World { + fn drop(&mut self) { + let _ = std::fs::remove_dir_all(&self.dir); + } +} + +#[test] +fn revisions_are_names_or_ids_never_git_syntax() { + let w = world(); + assert_eq!(w.get("/commits?sha=main").status, 200); + assert_eq!(w.get("/commits?sha=feat/x").status, 200); + assert_eq!(w.get("/commits?sha=v1").status, 200, "a tag name"); + for bad in [ + ":/second", + "main%5E%7B/second%7D", + "main@%7B0%7D", + "--output=/tmp/x", + "refs/forge/x", + "refs/heads/main", + "HEAD~1", + "main~1", + "main%5E", + "main..feat/x", + "%2D%2Dall", + "a%00b", + "main%0Aother", + ] { + let r = w.get(&format!("/commits?sha={bad}")); + assert_eq!(r.status, 404, "{bad}: {}", String::from_utf8_lossy(&r.raw)); + let r = w.get(&format!("/git/commits/{bad}")); + assert_eq!(r.status, 404, "git/commits/{bad}"); + let r = w.get(&format!("/contents?ref={bad}")); + assert_eq!(r.status, 404, "contents?ref={bad}"); + } + // an abbreviated id works (4+ hex digits), a branch wins over a tag of the same name + let head = w.json("/branches/main")["commit"]["id"] + .as_str() + .unwrap() + .to_string(); + assert_eq!(w.get(&format!("/git/commits/{}", &head[..8])).status, 200); +} + +#[test] +fn a_tag_named_like_a_branch_does_not_shadow_it() { + let w = world(); + let main_tip = w.json("/branches/main")["commit"]["id"] + .as_str() + .unwrap() + .to_string(); + let url = format!("{}/siteadmin/smart.git", w.s.base); + let h = format!("http.extraHeader=Authorization: {}", w.auth); + let work = w.dir.join("w"); + git(&work, &["tag", "main", "main~2"]); + git(&work, &["-c", &h, "push", "-q", &url, "refs/tags/main"]); + let list = w.json("/commits?sha=main&limit=1"); + assert_eq!(list[0]["sha"], main_tip, "the branch resolves first"); +} + +#[test] +fn paths_are_literals_not_patterns() { + let w = world(); + assert_eq!(w.json("/contents/README.md")["type"], "file"); + for bad in [ + "*", + "READ*", + "%3A(exclude)README.md", + "..%2FREADME.md", + "docs%2F..%2FREADME.md", + "docs//guide.md", + "docs%2F.%2Fguide.md", + "nope.txt", + ] { + let r = w.get(&format!("/contents/{bad}")); + assert!( + r.status == 404, + "{bad}: {} {}", + r.status, + String::from_utf8_lossy(&r.raw) + ); + } + // a glob or pathspec magic is a literal name that no commit touched: empty, not "everything" + for lit in ["*", "%3A(exclude)README.md", "READ%2A"] { + let r = w.get(&format!("/commits?path={lit}")); + assert_eq!(r.status, 200, "path={lit}"); + assert_eq!(r.json(), json!([]), "path={lit}"); + assert_eq!(r.headers["x-total-count"], "0"); + } + for bad in ["../x", "a%00b", "/abs"] { + assert_eq!( + w.get(&format!("/commits?path={bad}")).status, + 422, + "path={bad}" + ); + } + // the history of a path is only the commits that touched it + let hist = w.json("/commits?path=docs/guide.md"); + assert_eq!(hist.as_array().unwrap().len(), 1); +} + +#[test] +fn a_root_commit_has_a_diff_a_patch_files_and_stats_and_a_merge_free_history_counts() { + let w = world(); + let all = w.json("/commits?sha=main&limit=50"); + let root = all.as_array().unwrap().last().unwrap()["sha"] + .as_str() + .unwrap() + .to_string(); + let c = w.json(&format!("/git/commits/{root}")); + assert_eq!( + c["files"], + json!([{"filename": "README.md", "status": "added"}]) + ); + assert_eq!( + c["stats"], + json!({"total": 1, "additions": 1, "deletions": 0}) + ); + assert!(c["parents"].as_array().unwrap().is_empty()); + let d = w.get(&format!("/git/commits/{root}.diff")); + assert!( + String::from_utf8_lossy(&d.raw).contains("+# smart"), + "{}", + String::from_utf8_lossy(&d.raw) + ); + let p = w.get(&format!("/git/commits/{root}.patch")); + let text = String::from_utf8_lossy(&p.raw).to_string(); + assert!( + text.starts_with(&format!("From {root} ")) && text.contains("Subject: [PATCH] root"), + "{text}" + ); + assert!( + !text.trim_end().ends_with("2.54.0") && !text.contains("\n-- \n"), + "no signature trailer" + ); + let r = w.get("/commits?sha=main&limit=2&page=2"); + assert_eq!(r.headers["x-total-count"], "3"); +} + +#[test] +fn refs_tags_and_trees() { + let w = world(); + let refs = w.json("/git/refs"); + let names: Vec<&str> = refs + .as_array() + .unwrap() + .iter() + .map(|r| r["ref"].as_str().unwrap()) + .collect(); + assert_eq!( + names, + [ + "refs/heads/feat/x", + "refs/heads/main", + "refs/tags/light", + "refs/tags/v1" + ] + ); + let v1 = &w.json("/git/refs/tags/v1")[0]; + assert_eq!( + v1["object"]["type"], "tag", + "an annotated tag points at a tag object" + ); + assert_eq!( + w.json("/git/refs/tags/light")[0]["object"]["type"], + "commit" + ); + // a prefix is a path prefix, not a string prefix or a pattern + assert_eq!(w.get("/git/refs/heads/fe").status, 404); + assert_eq!(w.get("/git/refs/heads/*").status, 404); + assert_eq!(w.json("/git/refs/heads/feat").as_array().unwrap().len(), 1); + // hidden refs never show up + let tip = w.json("/branches/main")["commit"]["id"] + .as_str() + .unwrap() + .to_string(); + w.s.git(1, &["update-ref", "refs/forge/internal", &tip], None); + w.s.git(1, &["update-ref", "refs/pull/1/head", &tip], None); + let all = w.json("/git/refs"); + assert_eq!(all.as_array().unwrap().len(), 4); + assert_eq!(w.get("/git/refs/forge/internal").status, 404); + // tags newest first, lightweight tag message is the commit subject + let tags = w.json("/tags"); + assert_eq!(tags[0]["name"], "v1"); + assert_eq!(tags[0]["message"], "release one"); + assert_eq!(tags[1]["message"], "root"); + // trees accept a branch, a commit id, a tree id; recursion lists directories too + let by_branch = w.json("/git/trees/main?recursive=true"); + let paths: Vec<&str> = by_branch["tree"] + .as_array() + .unwrap() + .iter() + .map(|e| e["path"].as_str().unwrap()) + .collect(); + assert_eq!(paths, ["README.md", "docs", "docs/guide.md"]); + let docs = by_branch["tree"][1]["sha"].as_str().unwrap().to_string(); + assert_eq!( + w.json(&format!("/git/trees/{docs}"))["tree"][0]["path"], + "guide.md" + ); + assert_eq!(w.get("/git/trees/nope").status, 400); + let paged = w.json("/git/trees/main?recursive=true&per_page=2&page=2"); + assert_eq!( + ( + paged["tree"].as_array().unwrap().len(), + paged["truncated"].clone() + ), + (1, json!(false)) + ); +} + +#[test] +fn archives_are_zip_or_gzipped_tar_and_unknown_references_are_404() { + let w = world(); + let z = w.get("/archive/main.zip"); + assert_eq!((z.status, &z.raw[..2]), (200, &b"PK"[..])); + assert_eq!(z.headers["content-type"], "application/octet-stream"); + let t = w.get("/archive/feat/x.tar.gz"); + assert_eq!((t.status, &t.raw[..2]), (200, &[0x1f, 0x8b][..])); + assert_eq!(w.get("/archive/nope.zip").status, 404); + assert_eq!(w.get("/archive/main.rar").status, 400); + assert_eq!(w.get("/archive/--output=x.zip").status, 404); +} + +#[test] +fn the_empty_repository_answers_the_recorded_shapes_and_a_busy_runner_is_503() { + let s = Srv::start_git(1, std::time::Duration::from_millis(100)); + let auth = token(&s.admin); + s.post("/api/v1/user/repos", &auth, json!({"name": "blank"})); + let get = |p: &str| s.get(&format!("/api/v1/repos/siteadmin/blank{p}"), &auth); + assert_eq!(get("/commits").status, 409); + assert_eq!(get("/contents").json(), json!([])); + assert_eq!(get("/contents/README.md").json(), json!([])); + assert_eq!(get("/tags").json(), json!([])); + assert_eq!(get("/git/refs").status, 404); + assert_eq!(get("/git/trees/main").status, 400); + let busy = s.runner.occupy_for_tests(); + for p in [ + "/commits", + "/tags", + "/git/refs", + "/git/commits/main", + "/git/trees/main", + "/contents", + "/archive/main.zip", + ] { + // an empty repository answers before git is consulted for some routes; none may be a 500 + assert_ne!(get(p).status, 500, "{p}"); + } + drop(busy); +} + +#[test] +fn a_reader_without_access_gets_the_same_404_as_for_a_missing_repository() { + let w = world(); + let pw = w.s.mk_user("mallory"); + let t = w.s.mk_token("mallory", &pw, "t", &["all"]); + for p in [ + "/commits", + "/contents", + "/git/refs", + "/tags", + "/archive/main.zip", + "/git/trees/main", + "/git/commits/main", + ] { + let a = + w.s.get(&format!("/api/v1/repos/siteadmin/smart{p}"), &token(&t)); + let b = + w.s.get(&format!("/api/v1/repos/siteadmin/missing{p}"), &token(&t)); + assert_eq!((a.status, a.raw.clone()), (b.status, b.raw.clone()), "{p}"); + assert_eq!(a.status, 404, "{p}"); + } +} From 0108f14a2863606050747a5c9b8da5abcf7155b9 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 23 Oct 2026 17:15:31 +0300 Subject: [PATCH 4/7] docs(forge): F3a decisions as built, threat-model pass for content reads, roadmap --- ROADMAP.md | 3 +- docs/THREAT-MODEL.md | 18 ++++++++++ .../specs/2026-10-22-p38-forge-design.md | 33 +++++++++++++++++++ 3 files changed, 53 insertions(+), 1 deletion(-) diff --git a/ROADMAP.md b/ROADMAP.md index 790ec2c5..0abe399b 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -704,7 +704,8 @@ config. Design: capture) through F6 (acceptance on the first deployment); F0, F1 (server skeleton, users, organizations, tokens, `admin` CLI), F2a (repositories API, lifecycle and reconciler, collaborators, branches) and F2b (smart-HTTP git transport via `git http-backend`, -`pre-receive` ref policy, admission control, threat-model pass #1) are built. Hosted repos are +`pre-receive` ref policy, admission control, threat-model pass #1) and F3a (content reads: refs, tags, commits, +trees, contents, archives; writes are F3b) are built. Hosted repos are bare Git and get none of the `.sc/` features. Other follow-up work is tracked as GitHub issues and listed under Deferred diff --git a/docs/THREAT-MODEL.md b/docs/THREAT-MODEL.md index 00b0bb34..addeef19 100644 --- a/docs/THREAT-MODEL.md +++ b/docs/THREAT-MODEL.md @@ -942,6 +942,24 @@ unless said otherwise). fronts it (`--behind-proxy`) or plaintext is wanted (`--allow-plaintext`); tokens cross this socket. No TLS and no SSH in sc-forge itself. +### Content API reads (F3a) + +Threat-model pass #2 for the read endpoints. Tests are in `crates/forge/tests/e2e_content.rs`. + +- **Revision and path injection.** A client string never reaches git as a revision or pathspec: revisions go through a + grammar that yields a ref name or an object id, paths are literal (`--literal-pathspecs`, `--`), so `--output=`, + `:/x`, `main^{/x}`, `@{0}`, glob and pathspec magic and `refs/forge/*` are 404 or 422 + (`revisions_are_names_or_ids_never_git_syntax`, `paths_are_literals_not_patterns`). +- **Hidden refs.** Only `refs/heads` and `refs/tags` are listed or resolvable; `refs/forge` and any future `refs/pull` + are not (`refs_tags_and_trees`). +- **Visibility parity.** A repository a caller cannot read answers the same 404 as a missing one on every content route + (`a_reader_without_access_gets_the_same_404_as_for_a_missing_repository`). +- **Resource use.** Every read is a bounded child process (output cap, 120 s timeout, runner gate, 503 when busy) under the + repository read lock. An archive is held in RAM up to 128 MiB per request and the runner gate bounds the number at + once, so peak memory is the gate size times that cap. Lower `--git-concurrency` on small hosts. +- **Not covered.** Content is served as stored: a file in a repository is returned base64-encoded in JSON, and raw/web + URLs are not served, so no response is rendered as HTML. + ## Reporting Security reports go to **toni@git-agentic.com** or GitHub private vulnerability diff --git a/docs/superpowers/specs/2026-10-22-p38-forge-design.md b/docs/superpowers/specs/2026-10-22-p38-forge-design.md index dcada1c6..ec652935 100644 --- a/docs/superpowers/specs/2026-10-22-p38-forge-design.md +++ b/docs/superpowers/specs/2026-10-22-p38-forge-design.md @@ -813,6 +813,39 @@ F2b is the git half of F2 (plan: `docs/superpowers/plans/2026-10-23-p38-f2b-git- the mechanism; the 20 MiB push/clone test exercises the streaming path), per-owner quotas, `gc` of killed pushes' quarantine directories (F5). +### F3a decisions (as built) + +F3a is the read half of the content API (plan: `docs/superpowers/plans/2026-10-23-p38-f3a-content-reads.md`). The +write half (`POST`/`PUT` contents) is F3b; until then both answer 501 after the auth checks, so a read-only token still +gets its recorded 403. + +- **Routes.** `git/refs`, `git/refs/{ref..}`, `tags`, `commits`, `git/commits/{sha}` (+ `.diff`/`.patch`), + `git/trees/{sha..}`, `contents`, `contents/{path..}`, `archive/{name..}` under `/repos/{o}/{r}` (read scope, read + permission). Only `refs/heads` and `refs/tags` are listed; `refs/forge` and `refs/pull` are never visible. +- **Revisions are a grammar, not git syntax.** `HEAD` is the default branch; a valid branch name is tried as + `refs/heads/` then `refs/tags/`; 4 to 64 hex digits are an object id. Nothing else reaches git (`:/x`, `main^{/x}`, + `@{0}`, `--output=`, `HEAD~1` are all 404). Only resolved object ids are passed to later commands. +- **Paths are literals.** `--literal-pathspecs` and `--` on every command that takes one; output is NUL-delimited. + `..`, NUL and a leading `/` are 422. +- **Output caps.** Listings and diffs 64 MiB, a blob read 16 MiB (a larger file answers with `content: null`), 120 s per + command. Over the cap is 413, a timeout 504, a saturated runner 503. All reads run under the repository's read lock. +- **Archives are buffered in RAM**, capped at 128 MiB (413 over). `tar.gz` is built from the tar in the server. A streamed + archive needs a second pipe stage and is deferred. +- **`last_commit_sha` in a contents listing** costs one git process per entry, so it is exact for the first 100 entries + only; later entries carry the commit id that was read. Recorded Gitea does the exact walk for all of them. +- **Recorded Gitea quirks reproduced:** a commit's `tree.sha`/`url` carry the commit id; `git/trees/{x}` answers `sha` = + commit id when a commit or branch is named; a directory's `git_url` uses `git/blobs/`; the archive `Link` + names `.tar.gz?rev=` even for a zip; a tag's `commit.url` uses the tag object id while `sha` is the peeled commit; + `/tags` is newest first; an empty repository answers `contents` with `200 []` for any path, commits with 409, `git/refs` + with 404, `git/trees/main` with 400; a contents 404 carries the "GetContentsOrList" message; the commits `Link` header + is alphabetical over all parameters with the page clamped to the last. +- **Departure.** An unknown ref on `commits` is 404 (E#19; Gitea 1.22.6 answered 500). +- **Not served.** `git/blobs`, `git/tags` and web `raw` URLs appear in response bodies as Gitea writes them but are not + routes. +- **Conformance gate.** `forge-check.sh` now replays `A B C D E J K L M N R V` (12 scenarios). `E` stops at its first + contents write (`[42, KeyError: 'sha']`) with E#36 to #41 deferred to F3b; `D` defers its PR listing and `B` its status + post (F4). + ## Open questions 1. The merge of a PR whose head is already in base (recorded: 405 `try again later`, From 78777da864f48446327fc4a3f9fb1ce4b13d0aef Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 23 Oct 2026 17:30:37 +0300 Subject: [PATCH 5/7] fix(forge): F3a review fixes (merge patch, archive memory slots, symlink and empty-repo handling, docs match code) --- crates/forge/src/api.rs | 1 + crates/forge/src/cgi.rs | 1 + crates/forge/src/content_api.rs | 122 +++++++++++++++--- crates/forge/src/gitobj.rs | 28 ++-- crates/forge/src/http.rs | 5 + crates/forge/src/routes.rs | 8 ++ crates/forge/src/server.rs | 37 +++++- crates/forge/src/transport.rs | 1 + crates/forge/tests/e2e_content.rs | 114 ++++++++++++++++ docs/THREAT-MODEL.md | 16 ++- .../specs/2026-10-22-p38-forge-design.md | 24 +++- 11 files changed, 320 insertions(+), 37 deletions(-) diff --git a/crates/forge/src/api.rs b/crates/forge/src/api.rs index 039f2b28..e46b53ea 100644 --- a/crates/forge/src/api.rs +++ b/crates/forge/src/api.rs @@ -191,6 +191,7 @@ pub(crate) fn paged_sorted( let (k, v) = kv.split_once('=').unwrap_or((kv, "")); (k.to_string(), v.to_string()) }) + .rev() // the request parser reads the first value of a repeated parameter; the last insert wins .collect(); q.insert("limit".into(), limit.to_string()); let last = ((total + limit - 1) / limit).max(1); diff --git a/crates/forge/src/cgi.rs b/crates/forge/src/cgi.rs index 8f88901d..132ef65d 100644 --- a/crates/forge/src/cgi.rs +++ b/crates/forge/src/cgi.rs @@ -199,6 +199,7 @@ fn text(status: u16, body: &str) -> crate::http::Resp { headers: vec![("content-type", "text/plain;charset=utf-8".into())], body: body.as_bytes().to_vec(), delay: Duration::ZERO, + hold: None, } } diff --git a/crates/forge/src/content_api.rs b/crates/forge/src/content_api.rs index 5cd0d63e..bb4c7838 100644 --- a/crates/forge/src/content_api.rs +++ b/crates/forge/src/content_api.rs @@ -61,6 +61,11 @@ fn is_empty(ctx: &Ctx, a: &Access) -> Result { gitread::is_empty(&ctx.app.life.git, &dir(ctx, a)).map_err(|e| read_err(ctx, a, e)) } +/// No branch and no tag at all (a repository with only an unborn default is not this). +fn no_refs(ctx: &Ctx, a: &Access) -> Result { + all_refs(ctx, a).map(|rows| rows.is_empty()) +} + fn resolve(ctx: &Ctx, a: &Access, rev: &str, peel: Peel) -> Result, Resp> { gitobj::resolve( &ctx.app.life.git, @@ -276,14 +281,15 @@ pub fn list_tags(ctx: &mut Ctx) -> Resp { .iter() .map(|r| { let name = r.name.strip_prefix("refs/tags/").unwrap_or(&r.name); + let enc = name.split('/').map(pct_encode).collect::>().join("/"); json!({ "name": name, "message": r.subject, "id": r.oid, // Recorded Gitea quirk: the url names the tag object, the sha the commit. "commit": json!({"url": format!("{}/git/commits/{}", api_base(ctx, &a), r.oid), "sha": r.commit(), "created": rfc3339(r.commit_date)}), - "zipball_url": format!("{web}/archive/{name}.zip"), - "tarball_url": format!("{web}/archive/{name}.tar.gz"), + "zipball_url": format!("{web}/archive/{enc}.zip"), + "tarball_url": format!("{web}/archive/{enc}.tar.gz"), }) }) .collect(); @@ -311,12 +317,20 @@ pub fn list_commits(ctx: &mut Ctx) -> Resp { } fn list_commits_inner(ctx: &Ctx, a: &Access) -> Resp { - match is_empty(ctx, a) { - Ok(true) => return error(&ctx.app.base, 409, "Git Repository is empty."), - Ok(false) => {} - Err(r) => return r, + if param(ctx, "not").is_some() { + // Recorded Gitea filters by it; ignoring it would serve unfiltered history. + return error(&ctx.app.base, 400, "the `not` parameter is not supported"); } - let rev = param(ctx, "sha").unwrap_or_else(|| a.repo.default_branch.clone()); + let named = param(ctx, "sha").filter(|v| !v.is_empty()); + let empty = || error(&ctx.app.base, 409, "Git Repository is empty."); + if named.is_none() { + match is_empty(ctx, a) { + Ok(true) => return empty(), + Ok(false) => {} + Err(r) => return r, + } + } + let rev = named.unwrap_or_else(|| a.repo.default_branch.clone()); let path = param(ctx, "path").filter(|p| !p.is_empty()); if let Some(p) = &path { if !gitobj::valid_path(p) { @@ -326,7 +340,13 @@ fn list_commits_inner(ctx: &Ctx, a: &Access) -> Resp { let head = match resolve(ctx, a, &rev, Peel::Commit) { Ok(Some(h)) => h, // Gitea answers 500 here; a client error is a 404 (departure). - Ok(None) => return error(&ctx.app.base, 404, &format!("ref not found [{rev}]")), + Ok(None) => { + return match no_refs(ctx, a) { + Ok(true) => empty(), + Ok(false) => error(&ctx.app.base, 404, &format!("ref not found [{rev}]")), + Err(r) => r, + } + } Err(r) => return r, }; let (page, limit) = page_params(ctx); @@ -408,6 +428,7 @@ pub fn get_git_commit(ctx: &mut Ctx) -> Resp { headers: vec![("content-type", "text/plain;charset=utf-8".into())], body: bytes, delay: std::time::Duration::ZERO, + hold: None, }; match form { Some("diff") => match git_err( @@ -418,7 +439,11 @@ pub fn get_git_commit(ctx: &mut Ctx) -> Resp { Ok(b) => text(b), Err(r) => r, }, - Some(_) => match git_err(ctx, &a, gitobj::patch_text(git, &d, &c.oid)) { + Some(_) => match git_err( + ctx, + &a, + gitobj::patch_text(git, &d, &c.oid, c.parents.len() > 1), + ) { Ok(b) => text(b), Err(r) => r, }, @@ -549,7 +574,9 @@ fn content_json( }; let size = e.size.unwrap_or(0); let (mut encoding, mut content, mut target) = (Value::Null, Value::Null, Value::Null); - if ty == "symlink" { + // A link target is a path (PATH_MAX); a bigger blob is not read, and a listing reads at most + // LAST_COMMIT_BUDGET of them, so one tree cannot make the server read thousands of blobs. + if ty == "symlink" && size <= 4096 && (with_content || exact_last) { let b = git_err(ctx, a, gitobj::blob(git, &d, &e.oid))?; target = Value::String(String::from_utf8_lossy(&b).to_string()); } else if ty == "file" && with_content && size <= gitobj::MAX_BLOB { @@ -600,17 +627,27 @@ pub fn get_contents(ctx: &mut Ctx) -> Resp { Err(r) => return r, }; read_locked(ctx, &a, |ctx| { - // Recorded: an empty repository lists nothing and answers `[]` for any path. - match is_empty(ctx, &a) { - Ok(true) => return Resp::json(200, &json!([])), - Ok(false) => {} - Err(r) => return r, + // Recorded: an empty repository lists nothing and answers `[]` for any path. A named ref + // that exists wins over an unborn default branch (a repository holding only `feat`). + let named = param(ctx, "ref").filter(|v| !v.is_empty()); + if named.is_none() { + match is_empty(ctx, &a) { + Ok(true) => return Resp::json(200, &json!([])), + Ok(false) => {} + Err(r) => return r, + } } let path = ctx.param("p").trim_matches('/').to_string(); - let refname = param(ctx, "ref").unwrap_or_else(|| a.repo.default_branch.clone()); + let refname = named.unwrap_or_else(|| a.repo.default_branch.clone()); let head = match resolve(ctx, &a, &refname, Peel::Commit) { Ok(Some(h)) => h, - Ok(None) => return contents_not_found(ctx, &refname, ""), + Ok(None) => { + return match no_refs(ctx, &a) { + Ok(true) => Resp::json(200, &json!([])), + Ok(false) => contents_not_found(ctx, &refname, ""), + Err(r) => r, + } + } Err(r) => return r, }; let (git, d) = (&ctx.app.life.git, dir(ctx, &a)); @@ -679,6 +716,30 @@ pub fn get_contents(ctx: &mut Ctx) -> Resp { // ---- archive ---- +/// Archives in memory at once, server-wide: one is up to `MAX_ARCHIVE` of tar plus its gzip while it is +/// built, and up to `MAX_ARCHIVE` until the client has read it. The runner gate does not cover this +/// (its permit ends with the git process). +const MAX_ARCHIVES: usize = 4; +static ARCHIVES: std::sync::atomic::AtomicUsize = std::sync::atomic::AtomicUsize::new(0); + +/// One of the `MAX_ARCHIVES` slots; released when dropped, which `Resp::hold` defers to the end of +/// the response body. +struct ArchiveSlot; + +impl ArchiveSlot { + fn take() -> Option { + let slot = ArchiveSlot; + // Over the limit the slot drops again at once, which gives the count back. + (ARCHIVES.fetch_add(1, std::sync::atomic::Ordering::SeqCst) < MAX_ARCHIVES).then_some(slot) + } +} + +impl Drop for ArchiveSlot { + fn drop(&mut self) { + ARCHIVES.fetch_sub(1, std::sync::atomic::Ordering::SeqCst); + } +} + /// `GET /repos/{o}/{r}/archive/{ref..}.{zip,tar.gz}`. pub fn get_archive(ctx: &mut Ctx) -> Resp { let a = match acc(ctx) { @@ -705,6 +766,13 @@ pub fn get_archive(ctx: &mut Ctx) -> Resp { } Err(r) => return r, }; + let Some(slot) = ArchiveSlot::take() else { + return error( + &ctx.app.base, + 503, + "too many archives are being built, try again", + ); + }; let (git, d) = (&ctx.app.life.git, dir(ctx, &a)); let tar = match git_err(ctx, &a, gitobj::archive(git, &d, &oid, &a.repo.name, zip)) { Ok(b) => b, @@ -737,6 +805,26 @@ pub fn get_archive(ctx: &mut Ctx) -> Resp { ], body, delay: std::time::Duration::ZERO, + hold: Some(Box::new(slot)), } }) } + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn archive_slots_are_limited_and_come_back_when_dropped() { + let held: Vec<_> = (0..MAX_ARCHIVES) + .map(|_| ArchiveSlot::take().unwrap()) + .collect(); + assert!(ArchiveSlot::take().is_none(), "over the limit"); + assert!( + ArchiveSlot::take().is_none(), + "a refusal does not leak a count" + ); + drop(held); + assert!(ArchiveSlot::take().is_some()); + } +} diff --git a/crates/forge/src/gitobj.rs b/crates/forge/src/gitobj.rs index 52cf966f..2acff3aa 100644 --- a/crates/forge/src/gitobj.rs +++ b/crates/forge/src/gitobj.rs @@ -411,11 +411,23 @@ pub fn diff_text( )?) } -/// A mailbox-format patch of one commit (`.patch`). -pub fn patch_text(git: &Git, dir: &Path, oid: &str) -> Result, GitError> { - must(run( - git, - dir, +/// A mailbox-format patch of one commit (`.patch`). `format-patch` skips merges (and `-1` would +/// then print the first non-merge commit it meets), so a merge is shown against its first parent. +pub fn patch_text(git: &Git, dir: &Path, oid: &str, merge: bool) -> Result, GitError> { + let args: &[&str] = if merge { + &[ + "show", + "--pretty=email", + "-m", + "--first-parent", + "--no-color", + "--no-ext-diff", + "--no-textconv", + "--no-renames", + "--end-of-options", + oid, + ] + } else { &[ "format-patch", "-1", @@ -428,9 +440,9 @@ pub fn patch_text(git: &Git, dir: &Path, oid: &str) -> Result, GitError> "--no-renames", "--end-of-options", oid, - ], - Some(BIG), - )?) + ] + }; + must(run(git, dir, args, Some(BIG))?) } /// One entry of a tree. diff --git a/crates/forge/src/http.rs b/crates/forge/src/http.rs index bb19c405..5aaad472 100644 --- a/crates/forge/src/http.rs +++ b/crates/forge/src/http.rs @@ -42,6 +42,9 @@ pub struct Resp { /// Held back by the server before writing (failed-login backoff), on the /// async runtime so it occupies no blocking thread. pub delay: Duration, + /// Dropped once the body has been written (or the connection died): a resource budget that + /// must cover the response's bytes in flight, not just the handler. + pub hold: Option>, } const JSON_CT: &str = "application/json;charset=utf-8"; @@ -54,6 +57,7 @@ impl Resp { headers: vec![("content-type", JSON_CT.into())], body: serde_json::to_vec(v).expect("serializing a Value cannot fail"), delay: Duration::ZERO, + hold: None, } } @@ -64,6 +68,7 @@ impl Resp { headers: vec![], body: vec![], delay: Duration::ZERO, + hold: None, } } diff --git a/crates/forge/src/routes.rs b/crates/forge/src/routes.rs index 6eebf243..c16fa228 100644 --- a/crates/forge/src/routes.rs +++ b/crates/forge/src/routes.rs @@ -437,6 +437,14 @@ pub static ROUTES: &[Route] = &[ Repo(Perm::Write), api::not_implemented, ), + r( + "DELETE", + "/repos/{o}/{r}/contents/{p..}", + Token, + n(Cat::Repository, Level::Write), + Repo(Perm::Write), + api::not_implemented, + ), r( "GET", "/repos/{o}/{r}/archive/{a..}", diff --git a/crates/forge/src/server.rs b/crates/forge/src/server.rs index 32748fd8..20e610ab 100644 --- a/crates/forge/src/server.rs +++ b/crates/forge/src/server.rs @@ -152,7 +152,16 @@ fn to_response(r: Resp) -> Response { b = b.header(k, v); } // A header value hyper refuses must not panic the connection task: answer 500. - b.body(full(Bytes::from(r.body))).unwrap_or_else(|_| { + let body = match r.hold { + Some(hold) => Held { + inner: Full::new(Bytes::from(r.body)), + _hold: hold, + } + .map_err(|e| match e {}) + .boxed(), + None => full(Bytes::from(r.body)), + }; + b.body(body).unwrap_or_else(|_| { let mut e = Response::new(full(Bytes::from_static( b"{\"message\":\"internal server error\"}", ))); @@ -161,6 +170,32 @@ fn to_response(r: Resp) -> Response { }) } +/// A buffered body that keeps `_hold` alive until hyper drops it. +struct Held { + inner: Full, + _hold: Box, +} + +impl hyper::body::Body for Held { + type Data = Bytes; + type Error = std::convert::Infallible; + + fn poll_frame( + mut self: std::pin::Pin<&mut Self>, + cx: &mut std::task::Context<'_>, + ) -> std::task::Poll, Self::Error>>> { + std::pin::Pin::new(&mut self.inner).poll_frame(cx) + } + + fn is_end_stream(&self) -> bool { + self.inner.is_end_stream() + } + + fn size_hint(&self) -> hyper::body::SizeHint { + self.inner.size_hint() + } +} + fn full(b: Bytes) -> RespBody { Full::new(b).map_err(|e| match e {}).boxed() } diff --git a/crates/forge/src/transport.rs b/crates/forge/src/transport.rs index a2b3e794..5bfd5935 100644 --- a/crates/forge/src/transport.rs +++ b/crates/forge/src/transport.rs @@ -189,6 +189,7 @@ fn text(status: u16, body: &str) -> Resp { headers: vec![("content-type", TEXT.into())], body: body.as_bytes().to_vec(), delay: std::time::Duration::ZERO, + hold: None, } } diff --git a/crates/forge/tests/e2e_content.rs b/crates/forge/tests/e2e_content.rs index 25cb9727..fec24679 100644 --- a/crates/forge/tests/e2e_content.rs +++ b/crates/forge/tests/e2e_content.rs @@ -347,6 +347,120 @@ fn the_empty_repository_answers_the_recorded_shapes_and_a_busy_runner_is_503() { drop(busy); } +#[test] +fn a_merge_commit_is_shown_against_its_first_parent_and_a_symlink_has_its_target() { + let w = world(); + let work = w.dir.join("w"); + git(&work, &["checkout", "-q", "-b", "merged", "main"]); + git( + &work, + &["merge", "-q", "--no-ff", "feat/x", "-m", "merge feat"], + ); + std::os::unix::fs::symlink("README.md", work.join("link")).unwrap(); + git(&work, &["add", "."]); + git(&work, &["commit", "-q", "-m", "link"]); + let tip = git(&work, &["rev-parse", "HEAD"]); + let merge = git(&work, &["rev-parse", "HEAD~1"]); + let url = format!("{}/siteadmin/smart.git", w.s.base); + let h = format!("http.extraHeader=Authorization: {}", w.auth); + git(&work, &["-c", &h, "push", "-q", &url, "merged"]); + let end = std::time::Instant::now() + std::time::Duration::from_secs(10); + while w.s.app.life.registry.live() != 0 && std::time::Instant::now() < end { + std::thread::sleep(std::time::Duration::from_millis(20)); + } + let c = w.json(&format!("/git/commits/{merge}")); + assert_eq!(c["parents"].as_array().unwrap().len(), 2); + assert_eq!( + c["files"], + json!([{"filename": "x.txt", "status": "added"}]) + ); + let p = String::from_utf8_lossy(&w.get(&format!("/git/commits/{merge}.patch")).raw).to_string(); + assert!( + p.starts_with(&format!("From {merge} ")) && p.contains("Subject: [PATCH] merge feat"), + "the patch of a merge is the merge's own: {p}" + ); + assert!(p.contains("+++ b/x.txt"), "{p}"); + let d = String::from_utf8_lossy(&w.get(&format!("/git/commits/{merge}.diff")).raw).to_string(); + assert!(d.contains("+++ b/x.txt"), "{d}"); + let l = w.json("/contents/link?ref=merged"); + assert_eq!( + (l["type"].as_str(), l["target"].as_str()), + (Some("symlink"), Some("README.md")) + ); + let listing = w.json("/contents?ref=merged"); + let l = listing + .as_array() + .unwrap() + .iter() + .find(|e| e["name"] == "link") + .unwrap(); + assert_eq!(l["target"], "README.md", "{tip}"); +} + +#[test] +fn an_unborn_default_branch_does_not_hide_a_pushed_branch_and_a_busy_runner_is_503() { + let s = Srv::start_git(1, std::time::Duration::from_millis(100)); + let auth = token(&s.admin); + s.post( + "/api/v1/user/repos", + &auth, + json!({"name": "only-feat", "default_branch": "main"}), + ); + let dir = std::env::temp_dir().join(format!( + "scl-forge-unborn-{}-{}", + std::process::id(), + s.addr.port() + )); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let w = dir.join("w"); + git(&dir, &["init", "-q", "-b", "feat", w.to_str().unwrap()]); + std::fs::write(w.join("f.txt"), "f\n").unwrap(); + git(&w, &["add", "."]); + git(&w, &["commit", "-q", "-m", "only on feat"]); + let url = format!("{}/siteadmin/only-feat.git", s.base); + git( + &w, + &[ + "-c", + &format!("http.extraHeader=Authorization: {auth}"), + "push", + "-q", + &url, + "feat", + ], + ); + let end = std::time::Instant::now() + std::time::Duration::from_secs(10); + while s.app.life.registry.live() != 0 && std::time::Instant::now() < end { + std::thread::sleep(std::time::Duration::from_millis(20)); + } + let get = |p: &str| s.get(&format!("/api/v1/repos/siteadmin/only-feat{p}"), &auth); + // the default branch is unborn: unnamed reads keep the recorded empty-repository answers ... + assert_eq!(get("/commits").status, 409); + assert_eq!(get("/contents").json(), json!([])); + // ... but a named, existing branch is served + let c = get("/commits?sha=feat"); + assert_eq!(c.status, 200, "{}", c.msg()); + assert_eq!(c.json().as_array().unwrap().len(), 1); + assert_eq!(get("/contents/f.txt?ref=feat").json()["name"], "f.txt"); + assert_eq!(get("/commits?sha=nope").status, 404); + assert_eq!(get("/commits?not=feat").status, 400); + let busy = s.runner.occupy_for_tests(); + for p in [ + "/commits?sha=feat", + "/tags", + "/git/refs", + "/git/commits/feat", + "/git/trees/feat", + "/contents?ref=feat", + "/archive/feat.zip", + ] { + assert_eq!(get(p).status, 503, "{p}"); + } + drop(busy); + let _ = std::fs::remove_dir_all(&dir); +} + #[test] fn a_reader_without_access_gets_the_same_404_as_for_a_missing_repository() { let w = world(); diff --git a/docs/THREAT-MODEL.md b/docs/THREAT-MODEL.md index addeef19..61bc9ffc 100644 --- a/docs/THREAT-MODEL.md +++ b/docs/THREAT-MODEL.md @@ -950,13 +950,21 @@ Threat-model pass #2 for the read endpoints. Tests are in `crates/forge/tests/e2 grammar that yields a ref name or an object id, paths are literal (`--literal-pathspecs`, `--`), so `--output=`, `:/x`, `main^{/x}`, `@{0}`, glob and pathspec magic and `refs/forge/*` are 404 or 422 (`revisions_are_names_or_ids_never_git_syntax`, `paths_are_literals_not_patterns`). -- **Hidden refs.** Only `refs/heads` and `refs/tags` are listed or resolvable; `refs/forge` and any future `refs/pull` +- **Hidden refs.** Only `refs/heads` and `refs/tags` are listed or resolvable by name; `refs/forge` and any future `refs/pull` are not (`refs_tags_and_trees`). - **Visibility parity.** A repository a caller cannot read answers the same 404 as a missing one on every content route (`a_reader_without_access_gets_the_same_404_as_for_a_missing_repository`). -- **Resource use.** Every read is a bounded child process (output cap, 120 s timeout, runner gate, 503 when busy) under the - repository read lock. An archive is held in RAM up to 128 MiB per request and the runner gate bounds the number at - once, so peak memory is the gate size times that cap. Lower `--git-concurrency` on small hosts. +- **Resource use.** Every read is a bounded child process (output cap, runner gate, 503 when busy) under the + repository read lock; capped commands get 120 s, the rest the runner's own timeout. The git runner's permit ends + with the process, so memory held *after* it is bounded separately: at most 4 archives exist at once server-wide + (`ArchiveSlot`; a fifth is 503), each held until its body has been written, up to 128 MiB of tar plus its gzip + while it is built (`archive_slots_are_limited_and_come_back_when_dropped`). Listings, recursive trees and diffs are + parsed in memory up to their 64 MiB output cap each, bounded only by `--max-connections`; a directory listing reads + at most 100 symlink targets of at most 4 KiB. A single request can still run up to about 100 sequential `git log` + calls while holding the repository read lock, so a delete or default-branch change can wait behind it. +- **Revision resolution is by object id.** A hex id names any object in the repository's object store, including + commits reachable only from a deleted branch, as in Gitea (the F2b note on fetch by id applies); only branch and + tag *names* are limited to `refs/heads` and `refs/tags`. - **Not covered.** Content is served as stored: a file in a repository is returned base64-encoded in JSON, and raw/web URLs are not served, so no response is rendered as HTML. diff --git a/docs/superpowers/specs/2026-10-22-p38-forge-design.md b/docs/superpowers/specs/2026-10-22-p38-forge-design.md index ec652935..1182511d 100644 --- a/docs/superpowers/specs/2026-10-22-p38-forge-design.md +++ b/docs/superpowers/specs/2026-10-22-p38-forge-design.md @@ -826,20 +826,30 @@ gets its recorded 403. `refs/heads/` then `refs/tags/`; 4 to 64 hex digits are an object id. Nothing else reaches git (`:/x`, `main^{/x}`, `@{0}`, `--output=`, `HEAD~1` are all 404). Only resolved object ids are passed to later commands. - **Paths are literals.** `--literal-pathspecs` and `--` on every command that takes one; output is NUL-delimited. - `..`, NUL and a leading `/` are 422. -- **Output caps.** Listings and diffs 64 MiB, a blob read 16 MiB (a larger file answers with `content: null`), 120 s per - command. Over the cap is 413, a timeout 504, a saturated runner 503. All reads run under the repository's read lock. -- **Archives are buffered in RAM**, capped at 128 MiB (413 over). `tar.gz` is built from the tar in the server. A streamed - archive needs a second pipe stage and is deferred. + `..`, NUL and an absolute path are 422 on `commits?path=`; on `contents` a leading `/` is trimmed and the rest is a + literal lookup (404 when absent). +- **Output caps.** Listings and diffs 64 MiB, a blob read 16 MiB (a larger file answers with `content: null`), 120 s for + each capped command (the rest use the runner's 30 s). The operator's runner output cap does not apply to these. Over the cap is 413, a timeout 504, a saturated runner 503. All reads run under the repository's read lock. +- **Archives are buffered in RAM**, capped at 128 MiB (413 over); `tar.gz` is built from the tar in the server. At most + four exist at once (`ArchiveSlot`, a fifth is 503), each until its body is written: `Resp::hold` carries the slot to the + end of the response. A streamed archive needs a pipe stage and is deferred. - **`last_commit_sha` in a contents listing** costs one git process per entry, so it is exact for the first 100 entries - only; later entries carry the commit id that was read. Recorded Gitea does the exact walk for all of them. + only; later entries carry the commit id that was read. Recorded Gitea does the exact walk for all of them. A symlink + target is read for the first 100 entries of a listing and only when the link blob is at most 4 KiB. - **Recorded Gitea quirks reproduced:** a commit's `tree.sha`/`url` carry the commit id; `git/trees/{x}` answers `sha` = commit id when a commit or branch is named; a directory's `git_url` uses `git/blobs/`; the archive `Link` names `.tar.gz?rev=` even for a zip; a tag's `commit.url` uses the tag object id while `sha` is the peeled commit; `/tags` is newest first; an empty repository answers `contents` with `200 []` for any path, commits with 409, `git/refs` with 404, `git/trees/main` with 400; a contents 404 carries the "GetContentsOrList" message; the commits `Link` header is alphabetical over all parameters with the page clamped to the last. -- **Departure.** An unknown ref on `commits` is 404 (E#19; Gitea 1.22.6 answered 500). +- **Departures.** An unknown ref on `commits` is 404 (E#19; Gitea 1.22.6 answered 500). `commits?not=` is 400 (it + would otherwise be ignored silently). A file name that is not valid UTF-8 is listed with U+FFFD in place of the bad + bytes and cannot be fetched by that name; names with a control character are listed but their URL is 404. + The `.patch` of a merge commit is its diff against the first parent in mailbox form (`format-patch` skips merges). +- **Empty repositories.** "Empty" is "the default branch does not resolve", as for the `empty` field. An unnamed read of an + unborn default is the recorded empty answer even when other branches exist (Gitea would have adopted the first pushed + branch as default, see N#13); a *named* `sha`/`ref` that exists is served, and a named one that does not is 404 unless the + repository has no branch or tag at all. - **Not served.** `git/blobs`, `git/tags` and web `raw` URLs appear in response bodies as Gitea writes them but are not routes. - **Conformance gate.** `forge-check.sh` now replays `A B C D E J K L M N R V` (12 scenarios). `E` stops at its first From ebe4a14521e5234621ecd31bcf90341eb761a0b6 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 23 Oct 2026 17:39:53 +0300 Subject: [PATCH 6/7] fix(forge): F3a Codex review (page file budget, 400 on bad encoding, ref URLs by type and escaped) --- crates/forge/src/content_api.rs | 66 ++++++++++++++++--- crates/forge/tests/e2e_content.rs | 42 ++++++++++++ .../specs/2026-10-22-p38-forge-design.md | 2 + 3 files changed, 100 insertions(+), 10 deletions(-) diff --git a/crates/forge/src/content_api.rs b/crates/forge/src/content_api.rs index bb4c7838..cfdfb8b7 100644 --- a/crates/forge/src/content_api.rs +++ b/crates/forge/src/content_api.rs @@ -35,9 +35,15 @@ fn web_base(ctx: &Ctx, a: &Access) -> String { format!("{}/{}/{}", ctx.app.base, a.repo.owner, a.repo.name) } -/// A query parameter, percent-decoded. -fn param(ctx: &Ctx, k: &str) -> Option { - ctx.req.param(k).and_then(percent_decode) +/// Like [`param`], but a value that is not valid percent-encoded UTF-8 is a 400, not "absent": a +/// client that named a revision must not be served the default branch instead. +fn named_param(ctx: &Ctx, k: &str) -> Result, Resp> { + match ctx.req.param(k) { + None => Ok(None), + Some(v) => percent_decode(v) + .map(Some) + .ok_or_else(|| error(&ctx.app.base, 400, &format!("invalid encoding of `{k}`"))), + } } fn flag(ctx: &Ctx, k: &str, default: bool) -> bool { @@ -159,6 +165,7 @@ fn commit_json( }, "author": users.actor(&c.author.email)?, "committer": users.actor(&c.committer.email)?, + // Recorded Gitea quirk: a parent's `created` is the child's committer time. "parents": c.parents.iter().map(|p| commit_meta(ctx, a, p, c.committer.when)).collect::>(), "files": files.map(|fs| fs.iter().map(|f| json!({"filename": f.name, "status": f.status})).collect::>()), "stats": stats.map(|s| json!({"total": s.additions + s.deletions, "additions": s.additions, "deletions": s.deletions})), @@ -190,14 +197,26 @@ fn load_changes( fn ref_json(ctx: &Ctx, a: &Access, r: &RefRow) -> Value { let api = api_base(ctx, a); - let tag = r.kind == "tag"; + let seg = match r.kind.as_str() { + "tag" => "tags", + "tree" => "trees", + "blob" => "blobs", + _ => "commits", + }; + // A ref name may hold characters that are special in a URL (`#`, `%`, `?`). + let name = r + .name + .split('/') + .map(pct_encode) + .collect::>() + .join("/"); json!({ "ref": r.name, - "url": format!("{api}/git/{}", r.name), + "url": format!("{api}/git/{name}"), "object": { "type": r.kind, "sha": r.oid, - "url": format!("{api}/git/{}/{}", if tag { "tags" } else { "commits" }, r.oid), + "url": format!("{api}/git/{seg}/{}", r.oid), }, }) } @@ -316,12 +335,18 @@ pub fn list_commits(ctx: &mut Ctx) -> Resp { read_locked(ctx, &a, |ctx| list_commits_inner(ctx, &a)) } +/// Changed files one `commits` response may carry in total (a few hundred bytes of JSON each). +const MAX_PAGE_FILES: usize = 50_000; + fn list_commits_inner(ctx: &Ctx, a: &Access) -> Resp { - if param(ctx, "not").is_some() { + if ctx.req.param("not").is_some() { // Recorded Gitea filters by it; ignoring it would serve unfiltered history. return error(&ctx.app.base, 400, "the `not` parameter is not supported"); } - let named = param(ctx, "sha").filter(|v| !v.is_empty()); + let named = match named_param(ctx, "sha") { + Ok(v) => v.filter(|v| !v.is_empty()), + Err(r) => return r, + }; let empty = || error(&ctx.app.base, 409, "Git Repository is empty."); if named.is_none() { match is_empty(ctx, a) { @@ -331,7 +356,10 @@ fn list_commits_inner(ctx: &Ctx, a: &Access) -> Resp { } } let rev = named.unwrap_or_else(|| a.repo.default_branch.clone()); - let path = param(ctx, "path").filter(|p| !p.is_empty()); + let path = match named_param(ctx, "path") { + Ok(v) => v.filter(|p| !p.is_empty()), + Err(r) => return r, + }; if let Some(p) = &path { if !gitobj::valid_path(p) { return error(&ctx.app.base, 422, "invalid path"); @@ -377,11 +405,26 @@ fn list_commits_inner(ctx: &Ctx, a: &Access) -> Resp { ); let mut users = Users::new(ctx); let mut items = Vec::with_capacity(commits.len()); + // The per-command output cap does not bound a page of up to 50 commits that each list a huge + // change set, so the files kept for one response are bounded as a whole. + let mut files_left = MAX_PAGE_FILES; for c in &commits { let changes = match load_changes(ctx, a, c, want_files, want_stat) { Ok(c) => c, Err(r) => return r, }; + let n = changes + .as_ref() + .and_then(|c| c.0.as_ref()) + .map_or(0, Vec::len); + if n > files_left { + return error( + &ctx.app.base, + 413, + "the changed files of this page are too many", + ); + } + files_left -= n; match commit_json(ctx, a, &mut users, c, want_verif, changes) { Ok(v) => items.push(v), Err(r) => return r, @@ -629,7 +672,10 @@ pub fn get_contents(ctx: &mut Ctx) -> Resp { read_locked(ctx, &a, |ctx| { // Recorded: an empty repository lists nothing and answers `[]` for any path. A named ref // that exists wins over an unborn default branch (a repository holding only `feat`). - let named = param(ctx, "ref").filter(|v| !v.is_empty()); + let named = match named_param(ctx, "ref") { + Ok(v) => v.filter(|v| !v.is_empty()), + Err(r) => return r, + }; if named.is_none() { match is_empty(ctx, &a) { Ok(true) => return Resp::json(200, &json!([])), diff --git a/crates/forge/tests/e2e_content.rs b/crates/forge/tests/e2e_content.rs index fec24679..86b75cd3 100644 --- a/crates/forge/tests/e2e_content.rs +++ b/crates/forge/tests/e2e_content.rs @@ -461,6 +461,48 @@ fn an_unborn_default_branch_does_not_hide_a_pushed_branch_and_a_busy_runner_is_5 let _ = std::fs::remove_dir_all(&dir); } +#[test] +fn malformed_encodings_are_400_and_ref_urls_follow_the_object_type() { + let w = world(); + for p in ["/commits?sha=%FF", "/contents?ref=%FF", "/commits?path=%FF"] { + assert_eq!(w.get(p).status, 400, "{p}"); + } + // a lightweight tag on a tree, and a branch whose name needs escaping + let work = w.dir.join("w"); + let tree = git(&work, &["rev-parse", "main^{tree}"]); + git(&work, &["tag", "treetag", &tree]); + git(&work, &["branch", "rel#1", "main"]); + let url = format!("{}/siteadmin/smart.git", w.s.base); + let h = format!("http.extraHeader=Authorization: {}", w.auth); + git(&work, &["-c", &h, "push", "-q", &url, "rel#1", "treetag"]); + let end = std::time::Instant::now() + std::time::Duration::from_secs(10); + while w.s.app.life.registry.live() != 0 && std::time::Instant::now() < end { + std::thread::sleep(std::time::Duration::from_millis(20)); + } + let refs = w.json("/git/refs"); + let by = |name: &str| { + refs.as_array() + .unwrap() + .iter() + .find(|r| r["ref"] == name) + .unwrap_or_else(|| panic!("{name} in {refs}")) + .clone() + }; + let t = by("refs/tags/treetag"); + assert_eq!(t["object"]["type"], "tree"); + assert!( + t["object"]["url"].as_str().unwrap().contains("/git/trees/"), + "{t}" + ); + assert!( + by("refs/heads/rel#1")["url"] + .as_str() + .unwrap() + .ends_with("/git/refs/heads/rel%231"), + "{refs}" + ); +} + #[test] fn a_reader_without_access_gets_the_same_404_as_for_a_missing_repository() { let w = world(); diff --git a/docs/superpowers/specs/2026-10-22-p38-forge-design.md b/docs/superpowers/specs/2026-10-22-p38-forge-design.md index 1182511d..5be1fe74 100644 --- a/docs/superpowers/specs/2026-10-22-p38-forge-design.md +++ b/docs/superpowers/specs/2026-10-22-p38-forge-design.md @@ -846,6 +846,8 @@ gets its recorded 403. would otherwise be ignored silently). A file name that is not valid UTF-8 is listed with U+FFFD in place of the bad bytes and cannot be fetched by that name; names with a control character are listed but their URL is 404. The `.patch` of a merge commit is its diff against the first parent in mailbox form (`format-patch` skips merges). + A `commits` page may carry at most 50,000 changed files in total (413 over). `submodule_git_url` is always `null` + (Gitea reads it from `.gitmodules`; not built). A parent's `created` is the child's committer time, as recorded. - **Empty repositories.** "Empty" is "the default branch does not resolve", as for the `empty` field. An unnamed read of an unborn default is the recorded empty answer even when other branches exist (Gitea would have adopted the first pushed branch as default, see N#13); a *named* `sha`/`ref` that exists is served, and a named one that does not is 404 unless the From 18aece9bbe73c2445860bbc5cd600db15c9b0607 Mon Sep 17 00:00:00 2001 From: Toni Bergholm Date: Fri, 23 Oct 2026 17:40:14 +0300 Subject: [PATCH 7/7] docs(forge): escape a pipe in the F3a plan table --- docs/superpowers/plans/2026-10-23-p38-f3a-content-reads.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/superpowers/plans/2026-10-23-p38-f3a-content-reads.md b/docs/superpowers/plans/2026-10-23-p38-f3a-content-reads.md index 6ccf322a..1366e38a 100644 --- a/docs/superpowers/plans/2026-10-23-p38-f3a-content-reads.md +++ b/docs/superpowers/plans/2026-10-23-p38-f3a-content-reads.md @@ -9,7 +9,7 @@ plan, review and PR. Spec: "Server-side Git operations". Shapes are the recorded | Route | Recorded in | Notes | |---|---|---| -| `GET /repos/{o}/{r}/git/refs`, `git/refs/{heads|tags}[/{name}]` | D, E, J | list of `{ref, url, object{type,sha,url}}`; a missing exact ref is 404 `The target couldn't be found.`; an empty repo 404 on the list | +| `GET /repos/{o}/{r}/git/refs`, `git/refs/{heads\|tags}[/{name}]` | D, E, J | list of `{ref, url, object{type,sha,url}}`; a missing exact ref is 404 `The target couldn't be found.`; an empty repo 404 on the list | | `GET /tags` | D, J | tag objects (name, message, id, commit, zipball/tarball urls); `X-Total-Count`, paging | | `GET /commits` (`sha`, `path`, `stat`, `files`, `verification`, `not`, paging) | D, E | `X-Total-Count`, 409 `Git Repository is empty.` for an empty repo, 500 `object does not exist [id: nope, rel_path: ]` for an unknown ref (recorded) | | `GET /git/commits/{sha}`, `.diff`, `.patch` | D, E | `files`/`stats` filled, `verification` object; `.diff`/`.patch` are `text/plain` git output |