Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -848,6 +848,10 @@ smaller ones (protected-cache hygiene, theirs-side resolve carry, sparse and
P28 review follow-ons, the port de-flake, the clippy allow-list) are recorded in
`CLAUDE.md`, `ARCHITECTURE.md` and the ADRs.

- **Forge lifecycle and resource follow-ups (#316):** default-branch PATCH versus receive-pack
deletion, cancellation and shutdown evidence with real Git transfers, Linux memory-limit
execution, authentication-budget limits, scope recordings and quota budgets. These remain
separate from the F0 conformance checklist (#297) and F6 deployment acceptance.
- **Remaining history-editing depth:** operation objects in the CAS
(Jujutsu-deep upgrade to the file oplog), oplog entries for
remote-tracking refs.
Expand Down
5 changes: 3 additions & 2 deletions crates/forge/conformance/FINDINGS.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,8 +149,9 @@ every failure is a difference listed in the last table.
## Not recorded or partial

- SSH, LFS, Actions, webhooks, releases, labels, milestones and issues other than comments.
- `mergeable` is compared by type only, since it flips while a PR is checked; the conflict state is
kept as the note `conflicting_pr_mergeable_after_8s`.
- `mergeable` can flip while Gitea checks a PR. The conflicting PRs in G and T settle before recording
and compare `false` by value. The sc-forge gate additionally pins the clean PR lifecycle in F and
the clean merge in G through departures, because its mergeability is computed synchronously.
- A non-fast-forward push to a protected branch is refused by the git client before it reaches the
server, so only the forced variants exercise the server.
- Which of two same-second rows lists first (tokens, statuses, branches) is arbitrary on some versions;
Expand Down
31 changes: 20 additions & 11 deletions crates/forge/conformance/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ serves, **recorded from real Gitea** rather than written by hand. A scenario run
- `check` runs the same scenarios against any server and compares the result with the fixtures.
- `diff` compares two fixture directories offline, for example `1.22.6` against `1.26.2`.

`show` prints a readable digest of a fixture directory. `selftest` (offline, run by `forge-check.sh`) checks that the compact fixture writer round-trips and that
every stored fixture is exactly what it would write. Everything is Python 3 standard library plus
`show` prints a readable digest of a fixture directory. `selftest` (offline, run by `forge-check.sh`) checks the comparator's negative controls,
that the compact fixture writer round-trips and that every stored fixture is exactly what it would write. Everything is Python 3 standard library plus
`git` and (for recording) Docker. Always run Python as `python3 -I`.

## Record against real Gitea
Expand Down Expand Up @@ -46,9 +46,12 @@ python3 -I run.py diff fixtures/1.22.6 fixtures/1.26.2 # offline version diff
python3 -I run.py show fixtures/1.26.2 --only G --body # what was recorded
```

Against the in-tree server use `./forge-check.sh` (it replays scenarios `A,B,C,L,N,R`; the others wait for their routes, see `slice-pending.json` and the spec's "F2a decisions"; it bootstraps an empty data directory with the `admin` CLI,
Against the in-tree server use `./forge-check.sh` (it replays all 24 scenarios; it bootstraps an empty data directory with the `admin` CLI,
starts `sc-forge serve` on a free loopback port, runs `check` with the departures and `slice-pending.json`, and
stops the server; CI runs it). `check --pending FILE` takes a JSON map `{scenario: [N, "METHOD /path"]}` (a bare `N` is also accepted): the
stops the server; CI runs it). `slice-pending.json` is empty. K#55-56 compare their complete access-filtered
bodies; departures exclude Gitea's `X-Total-Count`, which leaks invisible repositories. sc-forge's visible
counts are pinned by `e2e_repos::listings_are_filtered_by_access_with_visible_counts`.
`check --pending FILE` takes a JSON map `{scenario: [N, "METHOD /path"]}` (a bare `N` is also accepted): the
server is expected to answer the first N exchanges of that scenario, the scenario must then stop at the named setup
call (a route from a later slice), and the rest are reported as pending. A server that answers more than N, or a scenario that
completes, fails the run, so the file shrinks as slices land and is empty when the contract is whole. It is for the
Expand Down Expand Up @@ -82,9 +85,14 @@ observations called notes). Exchange N of the run is compared with exchange N of
normalized to placeholders, and the SSH authority a server derives from its own config
(`ssh://git@localhost:2222/acme/alpha.git`, `git@host:acme/alpha.git`) becomes `<ssh-authority>`, so
`ssh_url` is compared by repo path only.
- The one place `mergeable` is compared by value: the conflicting PR in scenarios G and T. The call polls
- The conflicting PR in scenarios G and T compares `mergeable` by value. The call polls
until 8 consecutive reads agree (`settle_keys`), the settled value must be `false` when recording, and
the exchange lists `strict_keys: ["mergeable"]` so `check` compares that value too.
For sc-forge, departure `fields` also pin every PR response in F to `mergeable: true` (independent,
conflict-free branches, including close/reopen and lists), G#1 to `true` before the clean merge and
G#6 to `false` after it. This checks synchronous answers without freezing Gitea's transient values.
`html_url` is always compared by value after URL normalization. `selftest` mutates a recorded PR's
boolean and repository URL and requires the real gate to reject both.
- Error text: for every recorded exchange with status >= 400 whose body has `message` (or a non-empty
`errors`), the actual text must equal the recorded text after normalization: lower case, and only
id-shaped trailing details dropped (`[uid: 0, name: ]`, `[id: x, rel_path: ]`, numeric `id:`/`issue_id:`
Expand Down Expand Up @@ -176,14 +184,15 @@ success, ...), as well as every deliberate sc-forge difference.
- Teams (scenario B: create team, add member, carol's org permission) are tagged `team` and covered by
departures because sc-forge has no teams. Basic-**password** auth on an ordinary API route (scenarios B,
R, X) is covered by departures too: sc-forge accepts a password only on `/users/{u}/tokens*`.
- The `message_contains` words for sc-forge's own error texts (merge style, state, rename, branch) are
placeholders: reword them when sc-forge's messages are fixed.
- The `message_contains` words pin sc-forge's own refusal texts, including merge style, state,
rename, branch and ref-namespace refusals (#315).

Not covered by a departure: the merge of a PR whose head is already in base. sc-forge reproduces Gitea's
`405 "Please try again later"` with `changed_files: 0`, so the recorded exchanges apply unchanged. The
combined status for a sha with no statuses (1.22.6 `state:""`, 1.26.2 `pending`) and for an unknown sha
(1.22.6 `200` with empty fields, 1.26.2 `404`) are not 500/empty-message/malformed cases, so the rule gives
no departure; add one when sc-forge's answer is decided.
`405 "Please try again later"` with `changed_files: 0`, so the recorded exchanges apply unchanged.
The combined status for a commit with no statuses is `pending` with the real sha (H#0), and an
unknown sha is 404 (H#48). These follow 1.26.2 because 1.22.6's bodies contain empty answers.
After a refused `bogus` status, the existing commit still has no statuses (H#49: `pending`, count 0).
The departures and spec's "Statuses" section record these decisions.

### Normalization

Expand Down
183 changes: 183 additions & 0 deletions crates/forge/conformance/departures.json
Original file line number Diff line number Diff line change
Expand Up @@ -1582,5 +1582,188 @@
]
},
"reason": "sc-forge refuses a read-only collaborator's tag push at the advertisement with 403 Forbidden; Gitea advertises and then declines in pre-receive"
},
{
"scenario": "F",
"exchange": 0,
"kind": "http",
"match": "POST /api/v1/repos/acme/prs/pulls",
"expect": {
"fields": {
"mergeable": true
}
},
"reason": "Clean independent branches remain mergeable across create, list, head update, close, reopen and edit; sc-forge computes this synchronously."
},
{
"scenario": "F",
"exchange": 2,
"kind": "http",
"match": "POST /api/v1/repos/acme/prs/pulls",
"expect": {
"fields": {
"mergeable": true
}
},
"reason": "Clean independent branches remain mergeable across create, list, head update, close, reopen and edit; sc-forge computes this synchronously."
},
{
"scenario": "F",
"exchange": 11,
"kind": "http",
"match": "GET /api/v1/repos/acme/prs/pulls/<id:path:pulls#1>",
"expect": {
"fields": {
"mergeable": true
}
},
"reason": "Clean independent branches remain mergeable across create, list, head update, close, reopen and edit; sc-forge computes this synchronously."
},
{
"scenario": "F",
"exchange": 15,
"kind": "http",
"match": "GET /api/v1/repos/acme/prs/pulls/<id:path:pulls#1>",
"expect": {
"fields": {
"mergeable": true
}
},
"reason": "Clean independent branches remain mergeable across create, list, head update, close, reopen and edit; sc-forge computes this synchronously."
},
{
"scenario": "F",
"exchange": 22,
"kind": "http",
"match": "PATCH /api/v1/repos/acme/prs/pulls/<id:path:pulls#3>",
"expect": {
"fields": {
"mergeable": true
}
},
"reason": "Clean independent branches remain mergeable across create, list, head update, close, reopen and edit; sc-forge computes this synchronously."
},
{
"scenario": "F",
"exchange": 24,
"kind": "http",
"match": "PATCH /api/v1/repos/acme/prs/pulls/<id:path:pulls#3>",
"expect": {
"fields": {
"mergeable": true
}
},
"reason": "Clean independent branches remain mergeable across create, list, head update, close, reopen and edit; sc-forge computes this synchronously."
},
{
"scenario": "F",
"exchange": 25,
"kind": "http",
"match": "PATCH /api/v1/repos/acme/prs/pulls/<id:path:pulls#3>",
"expect": {
"fields": {
"mergeable": true
}
},
"reason": "Clean independent branches remain mergeable across create, list, head update, close, reopen and edit; sc-forge computes this synchronously."
},
{
"scenario": "F",
"exchange": 7,
"kind": "http",
"match": "GET /api/v1/repos/acme/prs/pulls?state=all&head=acme%3Af2&limit=50",
"expect": {
"fields": {
"0.mergeable": true,
"1.mergeable": true
}
},
"reason": "Clean independent branches remain mergeable across create, list, head update, close, reopen and edit; sc-forge computes this synchronously."
},
{
"scenario": "F",
"exchange": 9,
"kind": "http",
"match": "GET /api/v1/repos/acme/prs/pulls?state=open&limit=50",
"expect": {
"fields": {
"0.mergeable": true,
"1.mergeable": true
}
},
"reason": "Clean independent branches remain mergeable across create, list, head update, close, reopen and edit; sc-forge computes this synchronously."
},
{
"scenario": "F",
"exchange": 17,
"kind": "http",
"match": "GET /api/v1/repos/acme/prs/pulls?state=open&limit=50",
"expect": {
"fields": {
"0.mergeable": true,
"1.mergeable": true
}
},
"reason": "Clean independent branches remain mergeable across create, list, head update, close, reopen and edit; sc-forge computes this synchronously."
},
{
"scenario": "F",
"exchange": 23,
"kind": "http",
"match": "GET /api/v1/repos/acme/prs/pulls?state=closed&limit=50",
"expect": {
"fields": {
"0.mergeable": true
}
},
"reason": "Clean independent branches remain mergeable across create, list, head update, close, reopen and edit; sc-forge computes this synchronously."
},
{
"scenario": "G",
"exchange": 1,
"kind": "http",
"match": "GET /api/v1/repos/acme/merges/pulls/<id:path:pulls#1>",
"expect": {
"fields": {
"mergeable": true
}
},
"reason": "The clean PR is mergeable immediately; Gitea recorded its asynchronous check before completion."
},
{
"scenario": "G",
"exchange": 6,
"kind": "http",
"match": "GET /api/v1/repos/acme/merges/pulls/<id:path:pulls#1>",
"expect": {
"fields": {
"mergeable": false
}
},
"reason": "A merged PR is frozen and no longer mergeable; sc-forge reports this synchronously."
},
{
"scenario": "K",
"exchange": 55,
"kind": "http",
"match": "GET /api/v1/orgs/acme/repos",
"expect": {
"ignore_headers": [
"x-total-count"
]
},
"reason": "Gitea leaks all 11 organization repositories in X-Total-Count even when the body is access-filtered. The complete body stays compared; e2e_repos::listings_are_filtered_by_access_with_visible_counts checks visible-count filtering for a read collaborator and an outsider."
},
{
"scenario": "K",
"exchange": 56,
"kind": "http",
"match": "GET /api/v1/orgs/acme/repos?limit=50",
"expect": {
"ignore_headers": [
"x-total-count"
]
},
"reason": "Gitea leaks all 11 organization repositories in X-Total-Count even when the body is access-filtered. The complete body stays compared; e2e_repos::listings_are_filtered_by_access_with_visible_counts checks visible-count filtering for a read collaborator and an outsider."
}
]
11 changes: 11 additions & 0 deletions crates/forge/conformance/run.py
Original file line number Diff line number Diff line change
Expand Up @@ -1194,6 +1194,17 @@ def cmd_selftest(a):
assert not cmp_exchange(step, dict(step), Bij()), "an identical git step matches"
other = dict(step, cmd="git clone --depth=1 http://<creds>@<host>/acme/a.git <tmp>/w")
assert any("git command" in t for _, t in cmp_exchange(step, other, Bij())), "changed arguments must fail"
# The real gate must reject a wrong mergeability value and a URL naming another repository.
here = os.path.dirname(os.path.abspath(__file__))
pr = J.load(open(os.path.join(here, "fixtures", "1.22.6", "F-pulls.json")))["exchanges"][0]
deps = load_departures(os.path.join(here, "departures.json"), {s[0] for s in SCENARIOS})
dep = next((d for d in deps if d["scenario"] == "F" and 0 in dep_indexes(d["exchange"], [pr])), None)
def compare_pr(act):
return apply_override(pr, act, dep, Bij()) if dep else cmp_exchange(pr, act, Bij())
assert not any(lv == "fail" for lv, _ in compare_pr(pr)), "the recorded clean PR matches"
for key, value in (("mergeable", False), ("html_url", "<base>/acme/wrong/pulls/1")):
wrong = dict(pr, body=dict(pr["body"], **{key: value}))
assert any(lv == "fail" and key in text for lv, text in compare_pr(wrong)), f"wrong {key} must fail"
bad = []
for d in sorted(os.listdir(a.fixtures_root)):
full = os.path.join(a.fixtures_root, d)
Expand Down
15 changes: 1 addition & 14 deletions crates/forge/conformance/slice-pending.json
Original file line number Diff line number Diff line change
@@ -1,14 +1 @@
{
"K": {
"defer": [
[
55,
"GET /api/v1/orgs/acme/repos"
],
[
56,
"GET /api/v1/orgs/acme/repos?limit=50"
]
]
}
}
{}
Loading
Loading