From 286cc3c63970539da806ec6b64dd6a2e9b653c03 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Mon, 16 Nov 2026 14:27:19 +0000 Subject: [PATCH 1/5] fix(api): refuse a sync link or credential that git cannot take when it is stored MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A sync run gives git the upstream URL as one argument and each credential value as one environment entry. Since #748 a run fails as configuration when the URL is over 131,070 bytes or a username or password is over 131,042 bytes, but POST /repos and the git credential routes stored such values, so the client learned it only from a failed run. They now answer 400 with invalid_params, using the worker's constants, which are now exported (sync.MaxUpstreamURLBytes, sync.MaxCredentialBytes). The run check stays for values stored before. Signed-off-by: Thomas Kosiewski --- _Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `xhigh`_ Change-Id: Ia9cceeab87af3193f31a3171b65a578d881778ac --- docs/explanation/wire-compatibility.md | 1 + docs/reference/http-api.md | 17 ++- internal/api/handlers_credentials.go | 9 +- internal/api/handlers_sync.go | 84 +++++++++++-- internal/api/sync_limits_test.go | 162 +++++++++++++++++++++++++ internal/sync/README.md | 2 +- internal/sync/upstream_args_test.go | 18 +-- internal/sync/worker.go | 28 +++-- 8 files changed, 285 insertions(+), 36 deletions(-) create mode 100644 internal/api/sync_limits_test.go diff --git a/docs/explanation/wire-compatibility.md b/docs/explanation/wire-compatibility.md index d601a868..74c7d349 100644 --- a/docs/explanation/wire-compatibility.md +++ b/docs/explanation/wire-compatibility.md @@ -235,6 +235,7 @@ the [archived phase notes](../archive/phases/)): | A `crit` member in the JWT header: the vendored docs and spec say nothing about it. They list `alg` and `typ` as the header ([authentication guide](../upstream/getting-started/authentication.md)) | A token whose header has a `crit` member of any value fails verification, even when the signature is valid. The status is the one for any token that fails verification (the row above) | RFC 7515 §4.1.11: a recipient that does not understand a listed extension must refuse the token. open.storage understands none, so it refuses any `crit` | | Every path field and `path` parameter is a plain `type: string`. The spec says nothing about tree entry names whose bytes are not valid UTF-8 | A response path that is not valid UTF-8 still lists with one U+FFFD per invalid byte, and gets an extra sibling `_base64` with the raw bytes in standard base64 (for example `path_base64`, `paths_base64`, `old_path_base64`, `original_path_base64`). The sibling is absent for valid UTF-8 paths, so those responses are unchanged. This covers `files`, `files/metadata`, the diff routes, `blame` and `grep`. The merge conflict paths do not have it yet, and JSON request bodies cannot name such a file yet. The `path` query parameter takes the raw bytes percent-encoded ([path encoding](../reference/http-api.md#conventions)) | JSON strings hold Unicode text, so such a byte has no exact JSON form. A sibling field (like GitHub's `pathRaw`) is additive: no SDK reads it, and the SDKs ignore response keys they do not know. If upstream documents another form, we carry both ([#587](https://github.com/coder/openstorage/issues/587)) | | Text the request asks the server to store (a webhook `secret`, a git credential `username` and `password`, the `owner`, `name` and `auth.token` of a sync link, a `pull-upstream` `ref`) is plain `type: string`. The spec documents no `400` for its content | Such a value that holds a NUL byte answers `400` `bad_request` ("invalid secret (want UTF-8 text without NUL)", with `invalid_params` naming the field where the route has them), on every backend. A `GET /repos` `cursor` whose repo ID holds a NUL byte or invalid UTF-8 answers the `400` of an undecodable cursor. A path or body ID with such text (a webhook or git credential `id`) answers the `404` of an unknown ID | PostgreSQL refuses a NUL byte or invalid UTF-8 in a text parameter (SQLSTATE 22021) and answered `500` for most of these values, while SQLite and S3-WAL stored them. With `OPENSTORAGE_SECRETS_KEY` set in the default `encrypt` mode, the `secret`, `password` and `auth.token` reach PostgreSQL encrypted, so PostgreSQL stored them too, and such a value now answers `400` there as well. A `ref` is stored only by PostgreSQL's job queue, as `jsonb`, which cannot hold a NUL (SQLSTATE 22P05). Every backend now gives the same answer, as for `q` and `prefix` below. The PostgreSQL store answers `ErrNotFound` for such a key before it runs the query | +| The `owner` and `name` of a sync link (`upstream`, `base_repo`), its host (`upstream.host`, `base_repo.upstream_host`) and `base_repo.auth.token`, and a git credential `username` and `password` are plain `type: string` with no `maxLength`. The spec documents no `400` for their length | A sync link whose upstream URL (`https://{host}/{owner}/{name}.git`) is over 131,070 bytes, and a token, `username` or `password` over 131,042 bytes, answer `400` before anything is stored, with `invalid_params` naming each field ("invalid upstream URL (want at most 131070 bytes)": the `owner`, the `name`, and the host when the body sent one) ([HTTP API](../reference/http-api.md#repository-lifecycle-repowrite--orgread)) | The sync run gives git the URL as one argument and each credential value as one environment entry, and Linux refuses one of 128 KiB or more, counting its terminating NUL and the `/` that git adds to the URL or the name of the variable. Such a link failed every run as `repository configuration error`, and the client learned it only from the failed run. A link or credential stored before still fails its runs that way | | `commit_message` and the `name` and `email` of an `author` or `committer` (`commit-pack`, `diff-commit`, `restore-commit` and `POST /repos/merge`), the `author` of `POST /repos/notes` (also the deprecated body of `DELETE /repos/notes`), and the `query.pattern` of `POST /repos/grep` are plain `type: string`. The spec documents no `400` for their content | Such a value that holds a NUL byte answers `400` ("invalid commit_message (want UTF-8 text without NUL)"). `merge`, `notes` and `grep` name the field in `invalid_params`. `commit-pack` and `diff-commit` answer the result envelope with `status` `invalid`, and `restore-commit` the plain problem body. A merge request refuses the NUL even when no commit is made, as it refuses a newline in an ident | git refuses a NUL in a message, takes a pattern as a process argument and an ident as an environment value, and none of them can hold a NUL. These requests answered `500` | | Every `path` parameter, grep `paths` entry and `query.pattern`, blame `range`, and the `name` and `email` of an `author` or `committer` (commit writes, `merge`, `notes`) is a plain `type: string` with no `maxLength`. The spec documents no `400` for their length | A `path` or `paths` entry over 131,060 bytes, a `range` or `query.pattern` over 131,071 bytes, and a `name` or `email` over 131,051 bytes answer `400` before git starts, with the field and the limit in the message ("invalid path (want at most 131060 bytes)"). `merge`, `notes` and `grep` name the field in `invalid_params`, and `commit-pack` and `diff-commit` answer the result envelope with `status` `invalid`. A `commit_message` has no such limit: git reads it on standard input ([HTTP API](../reference/http-api.md#conventions)) | git takes each of these values as one argument or one environment entry, and Linux refuses one of 128 KiB or more, counting its terminating NUL. These requests answered `500` from 128 KiB on. The limit holds on every platform, so a client gets the same answer from every server | | `GET /repos` `q` and `prefix` are plain `type: string`. The spec documents a `400` only for a pagination cursor that cannot be decoded | A `q` or `prefix` that holds a NUL byte or invalid UTF-8 answers `400` `bad_request` ("invalid q (want UTF-8 text without NUL)", or the same text for `prefix`) on `/api/repos` and `/api/v1/repos`, on every backend ([HTTP API](../reference/http-api.md#repository-lifecycle-repowrite--orgread)). The only other list route with such a filter, `GET .../notes/refs` (`prefix`), has no NUL or UTF-8 check: it compares the raw bytes with the ref names in memory | PostgreSQL refuses a NUL byte or invalid UTF-8 in a text parameter (SQLSTATE 22021). The API refuses such a value before the store runs, so every backend gives the same answer ([#783](https://github.com/coder/openstorage/pull/783)). An empty `200` page would be correct only if no repository name can hold such bytes, and nothing proves that ([#788](https://github.com/coder/openstorage/issues/788)) | diff --git a/docs/reference/http-api.md b/docs/reference/http-api.md index f1b031c2..4a9905c7 100644 --- a/docs/reference/http-api.md +++ b/docs/reference/http-api.md @@ -374,6 +374,14 @@ the same scopes, query parameters and responses. `base_repo.auth.token`, `upstream.owner`, `upstream.name`), and nothing is stored. The stored text must be UTF-8 without NUL, as a PostgreSQL text value must; a request body decodes to valid UTF-8 already. +- A sync create whose values git cannot take → 400 `bad_request`, and nothing is + stored. An upstream URL (`https://{host}/{owner}/{name}.git`) over 131,070 bytes + names the `owner`, the `name`, and the host field when the body sent one + (`upstream.host`, `base_repo.upstream_host`), each with the reason `must keep the + upstream URL at most 131070 bytes` (message `invalid upstream URL (want at most + 131070 bytes)`). A `base_repo.auth.token` over 131,042 bytes names that field (`must + be at most 131042 bytes`). The limits are those of the sync run + ([Upstream sync](#upstream-sync), [Git credentials](#git-credentials-generic-https-upstreams)). - Providers: `code, github, gitlab, bitbucket, gitea, forgejo, codeberg, sr.ht`, plus generic with `upstream_host`. - A body with several invalid fields, in any create form, gets an `invalid_params` item @@ -475,7 +483,8 @@ the same scopes, query parameters and responses. stores the username exactly as sent. An update that omits it clears the stored one. A `username` or `password` that holds a NUL byte → 400 `bad_request` with `invalid_params` -naming the field, on every credential route, and nothing is stored. A credential `id` +naming the field, on every credential route, and nothing is stored. So does one over +131,042 bytes (`must be at most 131042 bytes`), which the sync run could not hand to git. A credential `id` that holds a NUL byte, in the body or in the path (also invalid UTF-8 in the path), names no credential → 404 `credential_not_found`, as an unknown `id` does. @@ -507,6 +516,8 @@ environment. A stored username or password that contains a carriage return, line or NUL fails the sync as a configuration error. So does one over 131,042 bytes, before git gets it: git takes it as one environment entry, such as `OPENSTORAGE_ASKPASS_PASSWORD=`, which must fit 128 KiB with its terminating NUL. +The credential routes and `POST /repos` refuse such a long value, so only a value +stored before they did reaches this check. ### Files & trees (`git:read`) @@ -830,7 +841,9 @@ Before [#647](https://github.com/coder/openstorage/issues/647), a failed guard a A pull or push run whose upstream URL (`https://{host}/{owner}/{name}.git`) is over 131,070 bytes fails as `repository configuration error` before git starts. git gets the URL as one argument, and its HTTP transport hands it on with a `/` added, which must fit -128 KiB with its terminating NUL. A stored credential that git cannot take fails the same +128 KiB with its terminating NUL. `POST /repos` refuses such a link with `400` +([Repository lifecycle](#repository-lifecycle-repowrite--orgread)), so only a link +registered before it did reaches this check. A stored credential that git cannot take fails the same way ([Git credentials](#git-credentials-generic-https-upstreams)). Values that each fit can still pass the total limit together: one git start can carry the URL twice and both credential entries, about 512 KiB at the limits, so a server with a stack limit of 2 MiB diff --git a/internal/api/handlers_credentials.go b/internal/api/handlers_credentials.go index d87436a6..94a3ab73 100644 --- a/internal/api/handlers_credentials.go +++ b/internal/api/handlers_credentials.go @@ -138,12 +138,17 @@ func gcredValidate(withID bool, id, username, password string) error { if withID && id == "" { idErr = newFieldError("id", "is required", "id is required") } + // The sync worker hands each value to git in an environment entry, + // which must fit upsync.MaxCredentialBytes. if password == "" { passwordErr = newFieldError("password", "is required", "password is required") - } else { - passwordErr = invalidTextFields([2]string{"password", password}) + } else if passwordErr = invalidTextFields([2]string{"password", password}); passwordErr == nil { + passwordErr = tooLongField("password", password, upsync.MaxCredentialBytes) } usernameErr := invalidTextFields([2]string{"username", username}) + if usernameErr == nil { + usernameErr = tooLongField("username", username, upsync.MaxCredentialBytes) + } if passwordErr == nil && (idErr != nil || usernameErr != nil) && meta.ReservedSecret(password) { passwordErr = gcredReservedPassword() } diff --git a/internal/api/handlers_sync.go b/internal/api/handlers_sync.go index 4ada64bd..b93121e1 100644 --- a/internal/api/handlers_sync.go +++ b/internal/api/handlers_sync.go @@ -23,6 +23,7 @@ import ( "context" "encoding/json" "errors" + "fmt" "io" "net/http" neturl "net/url" @@ -140,21 +141,77 @@ func validateSyncBaseRepo(spec *usyncBaseRepo, branchErr error, errs *bodyErrors if spec.Auth != nil { token = spec.Auth.Token } - errs.add(invalidTextFields([2]string{"base_repo.auth.token", token})) - errs.add(branchErr) + // The token is stored as the link's git credential. + tokenErr := invalidTextFields([2]string{"base_repo.auth.token", token}) + if tokenErr == nil { + tokenErr = tooLongField("base_repo.auth.token", token, upsync.MaxCredentialBytes) + } + var hostErr error host = spec.UpstreamHost switch { case host != "" && !validUpstreamHost(host): - errs.add(newFieldError("base_repo.upstream_host", "is not a valid host", "invalid base_repo upstream_host "+strconv.Quote(spec.UpstreamHost))) + hostErr = newFieldError("base_repo.upstream_host", "is not a valid host", "invalid base_repo upstream_host "+strconv.Quote(spec.UpstreamHost)) case host == "" && known: if host = defaultHost; host == "" { - errs.add(newFieldError("base_repo.upstream_host", "is required for this provider", - "base_repo upstream_host is required for provider "+strconv.Quote(spec.Provider))) + hostErr = newFieldError("base_repo.upstream_host", "is required for this provider", + "base_repo upstream_host is required for provider "+strconv.Quote(spec.Provider)) } } + urlErr := upstreamURLProblem("base_repo", spec.Owner, spec.Name, host, hostErr, sentHost(spec.UpstreamHost, "base_repo.upstream_host"), tokenErr, branchErr) + if urlErr != nil { + errs.add(urlErr) + return host + } + errs.add(tokenErr) + errs.add(branchErr) + errs.add(hostErr) return host } +// upstreamURLProblem returns the problem of a sync link whose upstream URL +// (upsync.Upstream.CloneURL) is longer than git takes +// (upsync.MaxUpstreamURLBytes), or nil. prefix is the body object +// (upstream or base_repo). The URL is made of the owner, the name and the +// host, so the problem names prefix.owner, prefix.name, and hostPath, the +// host field, when the body sent the host (sentHost). The URL needs all of +// them: an owner or name with a problem of its own, or a host with a +// problem (hostErr), leaves it unchecked. between holds the problems of the +// fields that lie between the name and the host in the docs. They join the +// problem, so that its items keep the order of the fields, as addFields +// does. The message never holds the URL. +func upstreamURLProblem(prefix, owner, name, host string, hostErr error, hostPath string, between ...error) error { + if owner == "" || name == "" || host == "" || hostErr != nil || + textField(prefix+".owner", owner) != nil || textField(prefix+".name", name) != nil { + return nil + } + url := upsync.Upstream{Owner: owner, Name: name, UpstreamHost: host}.CloneURL() + if len(url) <= upsync.MaxUpstreamURLBytes { + return nil + } + reason := fmt.Sprintf("must keep the upstream URL at most %d bytes", upsync.MaxUpstreamURLBytes) + params := []invalidParam{{Name: prefix + ".owner", Reason: reason}, {Name: prefix + ".name", Reason: reason}} + msgs := []string{tooLongMsg("upstream URL", upsync.MaxUpstreamURLBytes)} + for _, e := range between { + if e != nil { + params = append(params, invalidParamsFor(e)...) + msgs = append(msgs, e.Error()) + } + } + if hostPath != "" { + params = append(params, invalidParam{Name: hostPath, Reason: reason}) + } + return newFieldErrors(strings.Join(msgs, "; "), params...) +} + +// sentHost is path, the wire path of a host field, when the body sent the +// host (value), and "" when the host is the provider's default. +func sentHost(value, path string) string { + if value == "" { + return "" + } + return path +} + // addFields adds the problem of each of fields (dotted wire path -> value), // in order. The empty fields are named by one error that holds msg, as // requiredFields does. When another field with a problem of its own lies @@ -251,23 +308,30 @@ func (cfg upstreamConfig) validate(errs *bodyErrors) (host string) { } return textField(name, value) }) + var accessErr, hostErr error switch { case cfg.Access != "" && cfg.Access != "app" && cfg.Access != "public": - errs.add(newFieldError("upstream.access", "must be app or public", "invalid upstream access "+strconv.Quote(cfg.Access))) + accessErr = newFieldError("upstream.access", "must be app or public", "invalid upstream access "+strconv.Quote(cfg.Access)) case known && cfg.Provider != "github" && cfg.Access != "": - errs.add(newFieldError("upstream.access", "applies to provider github only", "upstream access applies to provider github only")) + accessErr = newFieldError("upstream.access", "applies to provider github only", "upstream access applies to provider github only") case known && cfg.Provider == "github" && cfg.Access != "public": - errs.add(newFieldError("upstream.access", "must be public", upstreamAppAccessMsg)) + accessErr = newFieldError("upstream.access", "must be public", upstreamAppAccessMsg) } host = cfg.Host switch { case host != "" && !validUpstreamHost(host): - errs.add(newFieldError("upstream.host", "is not a valid host", "invalid upstream host "+strconv.Quote(cfg.Host))) + hostErr = newFieldError("upstream.host", "is not a valid host", "invalid upstream host "+strconv.Quote(cfg.Host)) case host == "" && known: if host = defaultHost; host == "" { - errs.add(newFieldError("upstream.host", "is required for this provider", "upstream host is required for provider "+strconv.Quote(cfg.Provider))) + hostErr = newFieldError("upstream.host", "is required for this provider", "upstream host is required for provider "+strconv.Quote(cfg.Provider)) } } + if urlErr := upstreamURLProblem("upstream", cfg.Owner, cfg.Name, host, hostErr, sentHost(cfg.Host, "upstream.host"), accessErr); urlErr != nil { + errs.add(urlErr) + return host + } + errs.add(accessErr) + errs.add(hostErr) return host } diff --git a/internal/api/sync_limits_test.go b/internal/api/sync_limits_test.go new file mode 100644 index 00000000..04ca599b --- /dev/null +++ b/internal/api/sync_limits_test.go @@ -0,0 +1,162 @@ +package api + +import ( + "context" + "encoding/json" + "net/http" + "net/url" + "strings" + "testing" + + "github.com/coder/openstorage/internal/auth" + "github.com/coder/openstorage/internal/meta" + upsync "github.com/coder/openstorage/internal/sync" +) + +// TestSyncValuesGitCannotTakeAtRegistration: POST /repos refuses a sync +// link whose upstream URL is over 131,070 bytes or whose token is over +// 131,042 bytes, and the git credential routes refuse a username or a +// password over 131,042 bytes: the limits of the sync run, which would +// fail such a link on every run. A value at the limit is stored. The +// numbers are literals, so a limit that moves fails here. +func TestSyncValuesGitCannotTakeAtRegistration(t *testing.T) { + const ( + urlMax = 131070 + credMax = 131042 + host = "forge.example.invalid" + urlMsg = "invalid upstream URL (want at most 131070 bytes)" + urlWhy = "must keep the upstream URL at most 131070 bytes" + credWhy = "must be at most 131042 bytes" + ) + if upsync.MaxUpstreamURLBytes != urlMax || upsync.MaxCredentialBytes != credMax { + t.Fatalf("limits %d and %d, want %d and %d", upsync.MaxUpstreamURLBytes, upsync.MaxCredentialBytes, urlMax, credMax) + } + // owner returns an owner that makes the URL of name at base n bytes. + owner := func(base, name string, n int) string { + return strings.Repeat("o", n-len(base)-len("/"+name+".git")) + } + hostBase := "https://" + host + "/" + githubBase := "https://github.com/" + body := func(v any) string { + raw, err := json.Marshal(v) + if err != nil { + t.Fatal(err) + } + return string(raw) + } + upstream := func(o string, extra map[string]any) string { + cfg := map[string]any{"provider": "gitea", "owner": o, "name": "n", "host": host} + for k, v := range extra { + cfg[k] = v + } + return body(map[string]any{"upstream": cfg}) + } + baseRepo := func(o, token string) string { + return body(map[string]any{"base_repo": map[string]any{ + "provider": "gitea", "owner": o, "name": "n", "upstream_host": host, "auth": map[string]any{"token": token}}}) + } + if got := len(upsync.Upstream{Owner: owner(hostBase, "n", urlMax), Name: "n", UpstreamHost: host}.CloneURL()); got != urlMax { + t.Fatalf("the URL at the limit is %d bytes, want %d", got, urlMax) + } + + runBackends(t, func(t *testing.T, e *testEnv) { + org := e.newOrg(t) + tok := e.mint(t, tokenSpec{iss: org, repo: "lim-" + randomHex(t, 4), scopes: []string{auth.ScopeRepoWrite}}) + post := func(name, body, wantError string, want ...ipParam) ipCase { + return ipCase{name: name, method: http.MethodPost, path: "/api/v1/repos", token: tok, body: body, wantError: wantError, want: want} + } + over := owner(hostBase, "n", urlMax+1) + ipRun(t, e, []ipCase{ + post("upstream URL one byte over", upstream(over, nil), urlMsg, + ipParam{"upstream.owner", urlWhy}, ipParam{"upstream.name", urlWhy}, ipParam{"upstream.host", urlWhy}), + // The host of the provider is no field of the body. + post("upstream URL over the limit with the default host", + body(map[string]any{"upstream": map[string]any{"provider": "github", "owner": owner(githubBase, "n", urlMax+1), "name": "n", "access": "public"}}), + urlMsg, ipParam{"upstream.owner", urlWhy}, ipParam{"upstream.name", urlWhy}), + // access lies between name and host in the docs: its item + // stays there. + post("upstream URL over the limit and a bad access", upstream(over, map[string]any{"access": "x"}), + urlMsg+`; invalid upstream access "x"`, + ipParam{"upstream.owner", urlWhy}, ipParam{"upstream.name", urlWhy}, + ipParam{"upstream.access", "must be app or public"}, ipParam{"upstream.host", urlWhy}), + // A host with a problem of its own makes no URL. + post("upstream bad host is not a URL problem", upstream(over, map[string]any{"host": "bad host"}), + `invalid upstream host "bad host"`, ipParam{"upstream.host", "is not a valid host"}), + post("base_repo URL one byte over", baseRepo(over, "t"), urlMsg, + ipParam{"base_repo.owner", urlWhy}, ipParam{"base_repo.name", urlWhy}, ipParam{"base_repo.upstream_host", urlWhy}), + post("base_repo token one byte over", baseRepo("o", strings.Repeat("t", credMax+1)), + "invalid base_repo.auth.token (want at most 131042 bytes)", ipParam{"base_repo.auth.token", credWhy}), + post("base_repo URL and token over", baseRepo(over, strings.Repeat("t", credMax+1)), + urlMsg+"; invalid base_repo.auth.token (want at most 131042 bytes)", + ipParam{"base_repo.owner", urlWhy}, ipParam{"base_repo.name", urlWhy}, + ipParam{"base_repo.auth.token", credWhy}, ipParam{"base_repo.upstream_host", urlWhy}), + }) + + // At the limits the link and its credential are stored. + atURL := createRepo(t, e, org, "lim-url-"+randomHex(t, 4), []byte(upstream(owner(hostBase, "n", urlMax), nil)), http.StatusCreated) + atBase := createRepo(t, e, org, "lim-base-"+randomHex(t, 4), + []byte(baseRepo(owner(hostBase, "n", urlMax), strings.Repeat("t", credMax))), http.StatusCreated) + for _, created := range []createRepoResponse{atURL, atBase} { + link, err := e.store.(meta.EventingStore).GetRepoUpstream(context.Background(), created.RepoID) + if err != nil { + t.Fatalf("upstream of %s: %v", created.RepoID, err) + } + if got := len(upsync.Upstream{Owner: link.Owner, Name: link.Name, UpstreamHost: link.UpstreamHost}.CloneURL()); got != urlMax { + t.Fatalf("stored URL is %d bytes, want %d", got, urlMax) + } + } + cred, err := e.store.GitCredentialForRepo(context.Background(), atBase.RepoID) + if err != nil || len(cred.Password) != credMax { + t.Fatalf("stored token: %d bytes, err %v; want %d bytes", len(cred.Password), err, credMax) + } + + // The git credential routes. + repo, err := e.store.CreateRepo(context.Background(), org, "https://example.com/"+org+"/lim-cred-"+randomHex(t, 4), "main") + if err != nil { + t.Fatalf("create repo: %v", err) + } + stored, err := e.store.CreateGitCredential(context.Background(), meta.GitCredential{RepoID: repo.RepoID, Username: "u0", Password: "p0"}) + if err != nil { + t.Fatalf("create credential: %v", err) + } + orgTok := e.mint(t, tokenSpec{iss: org, repo: auth.RepoClaimOrg, scopes: []string{auth.ScopeRepoWrite}}) + repoTok := e.mint(t, tokenSpec{iss: org, repo: repo.URL, scopes: []string{auth.ScopeRepoWrite}}) + const v1 = "/api/v1/repos/git-credentials" + byPath := "/api/repos/" + url.PathEscape(repo.URL) + "/git-credentials/" + stored.ID + long := strings.Repeat("c", credMax+1) + both := `{"username":"` + long + `","password":"` + long + `"}` + const bothMsg = "invalid password (want at most 131042 bytes); invalid username (want at most 131042 bytes)" + bothItems := []ipParam{{"password", credWhy}, {"username", credWhy}} + ipRun(t, e, []ipCase{ + {name: "POST username and password over", method: http.MethodPost, path: v1, token: repoTok, body: both, wantError: bothMsg, want: bothItems}, + {name: "PUT username and password over", method: http.MethodPut, path: v1, token: orgTok, + body: `{"id":"` + stored.ID + `","username":"` + long + `","password":"` + long + `"}`, wantError: bothMsg, want: bothItems}, + {name: "PUT by path username and password over", method: http.MethodPut, path: byPath, token: repoTok, body: both, wantError: bothMsg, want: bothItems}, + {name: "POST password over", method: http.MethodPost, path: v1, token: repoTok, body: `{"password":"` + long + `"}`, + wantError: "invalid password (want at most 131042 bytes)", want: []ipParam{{"password", credWhy}}}, + // A value over the limit that also holds a NUL is one problem. + {name: "POST username over with a NUL", method: http.MethodPost, path: v1, token: repoTok, + body: `{"username":"` + long + `\u0000","password":"p"}`, wantError: "invalid username (want UTF-8 text without NUL)", + want: []ipParam{{"username", ipcNulReason}}}, + }) + fresh, err := e.store.CreateRepo(context.Background(), org, "https://example.com/"+org+"/lim-fresh-"+randomHex(t, 4), "main") + if err != nil { + t.Fatalf("create repo: %v", err) + } + freshTok := e.mint(t, tokenSpec{iss: org, repo: fresh.URL, scopes: []string{auth.ScopeRepoWrite}}) + atLimit := strings.Repeat("c", credMax) + pair := `{"username":"` + atLimit + `","password":"` + atLimit + `"}` + for _, tc := range []struct { + name, method, path, token, body string + }{ + {"POST", http.MethodPost, v1, freshTok, pair}, + {"PUT", http.MethodPut, v1, orgTok, `{"id":"` + stored.ID + `","username":"` + atLimit + `","password":"` + atLimit + `"}`}, + {"PUT by path", http.MethodPut, byPath, repoTok, pair}, + } { + resp, out := e.doReq(t, tc.method, tc.path, tc.token, []byte(tc.body), "") + if resp.StatusCode >= 300 { + t.Fatalf("%s at the limit: status = %d, want 2xx (body %.300s)", tc.name, resp.StatusCode, out) + } + } + }) +} diff --git a/internal/sync/README.md b/internal/sync/README.md index 723006c4..b8adedf0 100644 --- a/internal/sync/README.md +++ b/internal/sync/README.md @@ -27,7 +27,7 @@ Upstream git mirror sync (pull) and outbound push-forward. The package defines t - `Worker.assertReady` panics when a required dependency is unset; `Run` and `RunPush` assert non-empty org and repo ID [worker.go:assertReady] - `Run` returns nil without events when the repo no longer exists [worker.go:Run] - A transfer refuses to push when the holder changed (`errClaimStolen`), the row is gone (`errDetached`), or the claim lease is inside `claimFenceMargin` [worker.go:transfer] -- `hermeticGitEnv` strips inherited `GIT_*` variables and sets `GIT_ALLOW_PROTOCOL` [worker.go:hermeticGitEnv]; stored credentials containing CR, LF, or NUL fail with `FailConfig`, and so does one over `maxCredentialBytes` [worker.go:Worker.upstreamAuth]; an upstream URL over `maxUpstreamURLBytes` fails with `FailConfig` before any git starts [worker.go:upstreamURLFailure] +- `hermeticGitEnv` strips inherited `GIT_*` variables and sets `GIT_ALLOW_PROTOCOL` [worker.go:hermeticGitEnv]; stored credentials containing CR, LF, or NUL fail with `FailConfig`, and so does one over `MaxCredentialBytes` [worker.go:Worker.upstreamAuth]; an upstream URL over `MaxUpstreamURLBytes` fails with `FailConfig` before any git starts [worker.go:upstreamURLFailure]. The API refuses both limits at registration (`POST /repos`, git credential routes) with the same constants; the run checks stay for stored values - Only `FailUnreachable` push failures retry, at most `MaxPushTransientRetries` times; pushes use `--atomic` [worker_push.go:RunPush] - `proxyEnv` removes inherited proxy variables in any case; the egress proxy maps policy denials to 403 and dial failures to 502 [egress.go:proxyEnv] - `MemStore.BeginSync` returns `ErrSyncInProgress` while an unexpired claim exists; legacy claims get a grace period; `RequestPush` never downgrades force; a stolen push claim inherits force [memstore.go:BeginPush] diff --git a/internal/sync/upstream_args_test.go b/internal/sync/upstream_args_test.go index 8fdd4eb0..40539e4b 100644 --- a/internal/sync/upstream_args_test.go +++ b/internal/sync/upstream_args_test.go @@ -39,8 +39,8 @@ func longPathUpstream(t *testing.T, root string) string { // TestSyncArgLimitsAreTheDocumentedNumbers pins the limits that // docs/reference/http-api.md states. func TestSyncArgLimitsAreTheDocumentedNumbers(t *testing.T) { - if maxUpstreamURLBytes != 131070 || maxCredentialBytes != 131042 { - t.Fatalf("maxUpstreamURLBytes = %d, maxCredentialBytes = %d; the docs say 131070 and 131042", maxUpstreamURLBytes, maxCredentialBytes) + if MaxUpstreamURLBytes != 131070 || MaxCredentialBytes != 131042 { + t.Fatalf("MaxUpstreamURLBytes = %d, MaxCredentialBytes = %d; the docs say 131070 and 131042", MaxUpstreamURLBytes, MaxCredentialBytes) } } @@ -57,15 +57,15 @@ func TestUpstreamValuesPastGitArgLimit(t *testing.T) { v values config bool }{ - {"URL at the limit", values{urlBytes: maxUpstreamURLBytes}, false}, - {"URL one byte over", values{urlBytes: maxUpstreamURLBytes + 1}, true}, + {"URL at the limit", values{urlBytes: MaxUpstreamURLBytes}, false}, + {"URL one byte over", values{urlBytes: MaxUpstreamURLBytes + 1}, true}, {"URL of 200,000 bytes", values{urlBytes: 200_000}, true}, - {"password at the limit", values{password: maxCredentialBytes}, false}, - {"password one byte over", values{password: maxCredentialBytes + 1}, true}, + {"password at the limit", values{password: MaxCredentialBytes}, false}, + {"password one byte over", values{password: MaxCredentialBytes + 1}, true}, {"password of 200,000 bytes", values{password: 200_000}, true}, - {"username at the limit", values{username: maxCredentialBytes}, false}, - {"password at the limit next to a username", values{username: 1, password: maxCredentialBytes}, false}, - {"username one byte over", values{username: maxCredentialBytes + 1}, true}, + {"username at the limit", values{username: MaxCredentialBytes}, false}, + {"password at the limit next to a username", values{username: 1, password: MaxCredentialBytes}, false}, + {"username one byte over", values{username: MaxCredentialBytes + 1}, true}, } { for _, lane := range []string{"pull", "push"} { t.Run(lane+"/"+tc.name, func(t *testing.T) { diff --git a/internal/sync/worker.go b/internal/sync/worker.go index 08c8b546..25327ff9 100644 --- a/internal/sync/worker.go +++ b/internal/sync/worker.go @@ -1027,21 +1027,25 @@ func hermeticGitEnv(allowProtocol string) []string { ) } -// maxCredentialBytes bounds a stored username or password: git gets it as +// MaxCredentialBytes bounds a stored username or password: git gets it as // one environment entry, OPENSTORAGE_ASKPASS_PASSWORD= (or the -// username entry, as long), which must fit gitlimit.MaxArgBytes. -const maxCredentialBytes = gitlimit.MaxArgBytes - len("OPENSTORAGE_ASKPASS_PASSWORD=") - -// maxUpstreamURLBytes bounds an upstream URL. git gets it as one argument, -// and its HTTP transport hands it on to send-pack and fetch-pack with a -// "/" added, which must fit gitlimit.MaxArgBytes. -const maxUpstreamURLBytes = gitlimit.MaxArgBytes - len("/") +// username entry, as long), which must fit gitlimit.MaxArgBytes. The API +// refuses a longer value when it is stored; the run checks it again for +// values stored before. +const MaxCredentialBytes = gitlimit.MaxArgBytes - len("OPENSTORAGE_ASKPASS_PASSWORD=") + +// MaxUpstreamURLBytes bounds an upstream URL (Upstream.CloneURL). git gets +// it as one argument, and its HTTP transport hands it on to send-pack and +// fetch-pack with a "/" added, which must fit gitlimit.MaxArgBytes. The +// API refuses a link with a longer URL when it is registered; the run +// checks it again for links registered before. +const MaxUpstreamURLBytes = gitlimit.MaxArgBytes - len("/") // upstreamURLFailure fails a run whose upstream URL is over -// maxUpstreamURLBytes, before any git starts: such a link fails the same +// MaxUpstreamURLBytes, before any git starts: such a link fails the same // way on every run, so it is configuration. The error never names the URL. func upstreamURLFailure(up Upstream) *syncFailure { - if len(up.CloneURL()) <= maxUpstreamURLBytes { + if len(up.CloneURL()) <= MaxUpstreamURLBytes { return nil } return &syncFailure{code: FailConfig, cause: errors.New("upstream URL is longer than git takes as one argument")} @@ -1053,7 +1057,7 @@ func upstreamURLFailure(up Upstream) *syncFailure { // either). The pair travels through git's line-based credential-helper // protocol, so a stored value with CR, LF or NUL fails the run as // configuration instead of injecting helper attributes. So does a value -// over maxCredentialBytes, which git cannot take in its environment. The +// over MaxCredentialBytes, which git cannot take in its environment. The // error never names the value. func (w *Worker) upstreamAuth(ctx context.Context, repoID, provider string) (username, password string, fail *syncFailure) { cred, err := w.Credentials.CredentialForRepo(ctx, repoID) @@ -1066,7 +1070,7 @@ func (w *Worker) upstreamAuth(ctx context.Context, repoID, provider string) (use return "", "", nil case strings.ContainsAny(cred.Username+cred.Password, "\r\n\x00"): return "", "", &syncFailure{code: FailConfig, cause: errors.New("stored git credential contains CR, LF or NUL")} - case max(len(cred.Username), len(cred.Password)) > maxCredentialBytes: + case max(len(cred.Username), len(cred.Password)) > MaxCredentialBytes: return "", "", &syncFailure{code: FailConfig, cause: errors.New("stored git credential is longer than git takes in its environment")} } username, password = credentialAuth(provider, cred.Username, cred.Password) From c047b0367a97dfe89247c77ffcb182b95804999f Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Mon, 16 Nov 2026 14:41:04 +0000 Subject: [PATCH 2/5] chore: re-record the lane-schedule fingerprints MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Thomas Kosiewski --- _Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `xhigh`_ Change-Id: I8c1967a3e7a4c06c2113400a34401752e86ae5e4 --- .agents/skills/verify-lane-schedule/SKILL.md | 2 +- .../verify-lane-schedule/model/fingerprints.txt | 15 +++++++++------ 2 files changed, 10 insertions(+), 7 deletions(-) diff --git a/.agents/skills/verify-lane-schedule/SKILL.md b/.agents/skills/verify-lane-schedule/SKILL.md index ce7cc822..9414b15a 100644 --- a/.agents/skills/verify-lane-schedule/SKILL.md +++ b/.agents/skills/verify-lane-schedule/SKILL.md @@ -226,7 +226,7 @@ With every fact TRUE in the code, setting each fact FALSE in the model alone fai ## Coverage record -Sync commit: `a792629c551f5f5e8065e33e288804e77e38bbd9` (the last commit at which someone confirmed that the model describes this code; only `check.sh --record` moves it). +Sync commit: `286cc3c63970539da806ec6b64dd6a2e9b653c03` (the last commit at which someone confirmed that the model describes this code; only `check.sh --record` moves it). Every declaration in these files is covered: diff --git a/.agents/skills/verify-lane-schedule/model/fingerprints.txt b/.agents/skills/verify-lane-schedule/model/fingerprints.txt index c2418529..deefea19 100644 --- a/.agents/skills/verify-lane-schedule/model/fingerprints.txt +++ b/.agents/skills/verify-lane-schedule/model/fingerprints.txt @@ -74,6 +74,7 @@ internal/api/handlers_sync.go:import_"cmp" 6ee637994d3b529ead32b904453999f4347fa internal/api/handlers_sync.go:import_"context" 35d620b75b6b66f8c228f26b970a741d77137e4e691aa896119dd9d000eaadd2 internal/api/handlers_sync.go:import_"encoding/json" fe3dcc1a8fa119065b01019a2ad5ec1705648b9ce41c6ae8bfbc6b1d438adf42 internal/api/handlers_sync.go:import_"errors" a94ac0ab3a513cbb2bed2f97787a32bf9d423101f67d08dab897af948672c924 +internal/api/handlers_sync.go:import_"fmt" 7b04a3fec895247a2e6af9e527c6e53593a4afd23cf0b2ba70f3d8edb3fbc397 internal/api/handlers_sync.go:import_"github.com/coder/openstorage/internal/auth" a6fc462be510789f5df8f2fd769b94612c4099fecfdd8be88a7a6b4ac6f12b03 internal/api/handlers_sync.go:import_"github.com/coder/openstorage/internal/meta" 4260dc2e40a07d98bb999c941803200e032f17530aab6c21fd272ac3fcd2fdb6 internal/api/handlers_sync.go:import_"github.com/coder/openstorage/internal/sync" 3b3f0175cf3dee88544af1c7af3be7e737e79ae40f672de5cc2c6cc7e6d1f40e @@ -88,6 +89,7 @@ internal/api/handlers_sync.go:isUpstreamDetach 25715f9e18d1bf7bc62e630e8164eab04 internal/api/handlers_sync.go:parseUpstreamConfig dd27982847b44628b2a14a4364e4a23ea414ef90a9e7a05cb5f7edc52e12cc00 internal/api/handlers_sync.go:policyDeniesPullUpstream 098960c7bef537c91ca905ce894432882eef7ef465a3ddb0b0f9e7224be746e7 internal/api/handlers_sync.go:registerSyncRoutes 30a124203d6812c4b8fa939a4866caae6a74ac2b0b71e36a7de2fed3e0c25134 +internal/api/handlers_sync.go:sentHost 06d61ad20c0eda8ee6d85423eba7a886af4d12740a62517242e024cb01e1381d internal/api/handlers_sync.go:syncServer.answerTakenSyncName 22cf55fac3487ceec250be953518f3cd9b54265fa9061a6f94c4dd2d8567b0e5 internal/api/handlers_sync.go:syncServer.completeOwnSyncCreate a301e0bad45ed03f793aa89eca3036f4e652051a33060812075c97ccf1076b16 internal/api/handlers_sync.go:syncServer.createAbandon 07809dcf477a5a8cec0ff8c63825bddf587cc49377865352ec4ce616d5ac9fda @@ -108,10 +110,11 @@ internal/api/handlers_sync.go:type_usyncBaseRepo fa63c2d8bb62db2a56acff9a8de55e5 internal/api/handlers_sync.go:type_usyncBaseRepoAuth cbce39e3f3df0443cfcb499a9e439e361335ee86e6e97c205c21b18b48141cd6 internal/api/handlers_sync.go:type_usyncPullUpstreamRequest b98132ea1ae7690f3f00aece7dd8fb2a00f69d0e9ea5f1f95c198c983cc18b0b internal/api/handlers_sync.go:type_usyncPushUpstreamRequest 3ad57ba2d21ad394741cb6b3ced05a273c72b38ca84d7a73b34d79b4510f4284 -internal/api/handlers_sync.go:upstreamConfig.validate 2f930a6635c9e09c1c0e59a36ebd0a95da941d2158b8ca4bb5a73d7cf558ca17 +internal/api/handlers_sync.go:upstreamConfig.validate 93145c084d06574d887a74f780b331417c9198d38d5bd65a4d46824ff201cfd9 internal/api/handlers_sync.go:upstreamHostname f65b41a625e2606e71c567cbc6d99d991e59b6ef4da9e976294b9ff3895aac34 +internal/api/handlers_sync.go:upstreamURLProblem e66d2c29ef48b6891a62cca5ace9afbd52d579f5f4ae11aeba3cc61b3f3c488d internal/api/handlers_sync.go:validUpstreamHost 577f0a7dca4f2f92aef75121a9ba2f3de54ed91a6a0807f81e4d70193bd58a52 -internal/api/handlers_sync.go:validateSyncBaseRepo 772f61c09307f8b94a4f792fc5595630ae83bec2bf4b82a68bc735398768cc90 +internal/api/handlers_sync.go:validateSyncBaseRepo ff9604dc1950f41a1e0d3c506440c67f51a3b399ee34a83a94cea08459594331 internal/api/handlers_sync.go:withStrictPullBody 449923a2f639f42183416d391cac81b304bcf8b6cda9d207474e66e16178766f internal/api/handlers_sync.go:withUpstreamDetach b16659bb293b5f9eb05b5ade652dfe4fb224e8a7b6c475829007ee1933a164ec internal/events/refstore.go:BaseRepoID d45f84f8f7f8b9e27d21c7b17553527e76db8c3a5fa34718d2768840b351c8ae @@ -755,7 +758,7 @@ internal/sync/worker.go:Worker.restoreLocal 26a4d6afcb497aad19bf44a05a934271739a internal/sync/worker.go:Worker.retryFinish ffaeb07c443ff810394a4163e9bbba358e94167e2a862610fc85ea7cd04bb3ac internal/sync/worker.go:Worker.retryFinishWithin e5d50e3769f6ee0472f6f27eea02cb1f50b666bc32e18f95bc357ee6a270617c internal/sync/worker.go:Worker.transfer 9a7214ab5245a9dff3f77236343915bd12d363e65d0a931d562ddb5aabe7046e -internal/sync/worker.go:Worker.upstreamAuth 75d2138c45b126b08085f60b4ab76dc7c541ba77b8774aaa67c7db58ac0508f4 +internal/sync/worker.go:Worker.upstreamAuth f7c972b140b7e1c99a838e12a0874a06c59d8a1e56b1ad20ca51ad93acfdbac9 internal/sync/worker.go:Worker.waitPublication 5523b2acf8503a7d67b1c7a16ac8193b665e56cf2359c4b2c665712c2387079f internal/sync/worker.go:allLocalWins 2e1ed7ba97088a77c021e2f4b6814a936cd110f583795fddf5b52e55a198452a internal/sync/worker.go:authGitEnv 3f12bf1bdc1c34f599c5eb37ea5b01b61eb5e88bef4850133b8620f1994e717b @@ -763,6 +766,8 @@ internal/sync/worker.go:claimContext c804c041a9696fa2f25121c65031f594017701ce950 internal/sync/worker.go:claimLease c4e9048ce62dc44ec0095330d893478da5b9d84e20caa798125a4e8c3fa64dce internal/sync/worker.go:classifyFetchFailure f70cf722890e1703edf17e5d89762e4a3f32b49102eabbb1314a18c41ce75a42 internal/sync/worker.go:classifyPushFailure 2389057e736b3844b0b69c1171f7a502a5c12219f91d19da17d273ffd26b7977 +internal/sync/worker.go:const_MaxCredentialBytes e953924b849f74132b59fde912c5fc10da98559cfe437ad7f407f670587ceea1 +internal/sync/worker.go:const_MaxUpstreamURLBytes 65714124227545669b488bb3f7700b4963e57550b4a11b2ac82407070da2a102 internal/sync/worker.go:const_claimFenceMargin 6152874979c4576b0964fc7d77f93d9510f0d5ad648137d7650e2ef965ac6cb4 internal/sync/worker.go:const_claimLeaseSlack 75bc413ad4dddefab0fe6b0831b1f754f3526a77e08c2f294249fb7aba5f7bdb internal/sync/worker.go:const_defaultRunTimeout 248c39614bd5f60b5e7b7245d8ddb644c406386aee36a5b5a9b855b5b87e5ba4 @@ -772,8 +777,6 @@ internal/sync/worker.go:const_finishSyncAttempts 90532dd664650956ab4924c95cd15d6 internal/sync/worker.go:const_gitCredentialHelper 3109b7a280092f426cb473da823e3627281580830a7f7c2343ea8a1c4662db9d internal/sync/worker.go:const_localPublicationRetryDelay 963a26c10d88d979392c7d2ea65b948d998c245b8d8e150368f0a2724d57ed35 internal/sync/worker.go:const_localPublicationWaitTimeout 0cb9c18756fc851aedff391a3ab0c18de9efd089592812aa214dab2964985d2e -internal/sync/worker.go:const_maxCredentialBytes 98750963cc70d915fdbc6f65075b3a99b14977ce30cbf7aeadbd8f9656a11daf -internal/sync/worker.go:const_maxUpstreamURLBytes 0a542ae23e38888620342e09c841e9a979ed63bdbb97cb43634eb5aa03b73ca3 internal/sync/worker.go:const_shutdownRestoreDelay 32040a037f16b75ee6bd37c9d600b621277690f9cca1965d6c5b2c705f5a5785 internal/sync/worker.go:const_shutdownRestoreTimeout d6c2c3e701cd3841cbe1687a2b45790e9c5d4a2b48cf15e6e2567922a2555616 internal/sync/worker.go:containsAnyMarker 162157c304b7474a3d545d6ce708cd229788e83acf94c884db594cd56c9a7d12 @@ -811,7 +814,7 @@ internal/sync/worker.go:type_RepoInfo e8a02330a52c30199fa31037b4286d8d23ab173f23 internal/sync/worker.go:type_RepoLookup 819f1bc4b699b113c5c1400e9137b05dd20a8fa12cd1d0d621cca9ac686e29df internal/sync/worker.go:type_Worker 02235720d66907846dcf03ab0427a9eec32fb417af3c0462bbf5a381c76dfdf1 internal/sync/worker.go:type_syncFailure 6f04984a8621d6e7bf3665dd7af1eb8f7634d923899e5e61b18c60e246c3c59c -internal/sync/worker.go:upstreamURLFailure c95da462ad37745269d7f07a9019c3cc127cad41551c2549cb6bbe3d14483194 +internal/sync/worker.go:upstreamURLFailure a4f2c466197974a038dfc2edd51f12da58db9bb332df8709573a6fbd3101cd20 internal/sync/worker.go:var_authFailureMarkers 28df1810afcf9a0f750de4a7b73b0a925f66ba08df738a9c43a062abc68c8d40 internal/sync/worker.go:var_errClaimStolen 5869b81a47a71928975f5577f5b485bef2e6d0d98f3fd0f31a81b191a67f74a6 internal/sync/worker.go:var_errDetached 09503fc22b7999840131138daf5a27e594cb8b6773f3be5b2001b2bbb0d356e2 From 9507c4835c2fe60db21680b6b074add138693e70 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Mon, 16 Nov 2026 15:07:09 +0000 Subject: [PATCH 3/5] test(api): pin each sync limit case that the review found untested MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Ten mutants of the new checks passed the tests: a bad host that would also make the URL too long, the default host of a base_repo, the default_branch problem inside the URL problem and its order, an owner, name, password or token that holds a NUL and is over its limit, and a host with a path and a trailing slash, at and over the limit. Each now has a case, and each of those mutants fails. The docs say how a host with a scheme makes the URL, and that a link or credential stored by an older server still meets the run check. Signed-off-by: Thomas Kosiewski --- _Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `xhigh`_ Change-Id: I7788a7e1b1007beaa98be91924b94cfc18cfbe17 --- docs/explanation/wire-compatibility.md | 2 +- docs/reference/http-api.md | 28 +++++++++------- internal/api/sync_limits_test.go | 46 ++++++++++++++++++++++---- 3 files changed, 57 insertions(+), 19 deletions(-) diff --git a/docs/explanation/wire-compatibility.md b/docs/explanation/wire-compatibility.md index 74c7d349..5c65ffb0 100644 --- a/docs/explanation/wire-compatibility.md +++ b/docs/explanation/wire-compatibility.md @@ -235,7 +235,7 @@ the [archived phase notes](../archive/phases/)): | A `crit` member in the JWT header: the vendored docs and spec say nothing about it. They list `alg` and `typ` as the header ([authentication guide](../upstream/getting-started/authentication.md)) | A token whose header has a `crit` member of any value fails verification, even when the signature is valid. The status is the one for any token that fails verification (the row above) | RFC 7515 §4.1.11: a recipient that does not understand a listed extension must refuse the token. open.storage understands none, so it refuses any `crit` | | Every path field and `path` parameter is a plain `type: string`. The spec says nothing about tree entry names whose bytes are not valid UTF-8 | A response path that is not valid UTF-8 still lists with one U+FFFD per invalid byte, and gets an extra sibling `_base64` with the raw bytes in standard base64 (for example `path_base64`, `paths_base64`, `old_path_base64`, `original_path_base64`). The sibling is absent for valid UTF-8 paths, so those responses are unchanged. This covers `files`, `files/metadata`, the diff routes, `blame` and `grep`. The merge conflict paths do not have it yet, and JSON request bodies cannot name such a file yet. The `path` query parameter takes the raw bytes percent-encoded ([path encoding](../reference/http-api.md#conventions)) | JSON strings hold Unicode text, so such a byte has no exact JSON form. A sibling field (like GitHub's `pathRaw`) is additive: no SDK reads it, and the SDKs ignore response keys they do not know. If upstream documents another form, we carry both ([#587](https://github.com/coder/openstorage/issues/587)) | | Text the request asks the server to store (a webhook `secret`, a git credential `username` and `password`, the `owner`, `name` and `auth.token` of a sync link, a `pull-upstream` `ref`) is plain `type: string`. The spec documents no `400` for its content | Such a value that holds a NUL byte answers `400` `bad_request` ("invalid secret (want UTF-8 text without NUL)", with `invalid_params` naming the field where the route has them), on every backend. A `GET /repos` `cursor` whose repo ID holds a NUL byte or invalid UTF-8 answers the `400` of an undecodable cursor. A path or body ID with such text (a webhook or git credential `id`) answers the `404` of an unknown ID | PostgreSQL refuses a NUL byte or invalid UTF-8 in a text parameter (SQLSTATE 22021) and answered `500` for most of these values, while SQLite and S3-WAL stored them. With `OPENSTORAGE_SECRETS_KEY` set in the default `encrypt` mode, the `secret`, `password` and `auth.token` reach PostgreSQL encrypted, so PostgreSQL stored them too, and such a value now answers `400` there as well. A `ref` is stored only by PostgreSQL's job queue, as `jsonb`, which cannot hold a NUL (SQLSTATE 22P05). Every backend now gives the same answer, as for `q` and `prefix` below. The PostgreSQL store answers `ErrNotFound` for such a key before it runs the query | -| The `owner` and `name` of a sync link (`upstream`, `base_repo`), its host (`upstream.host`, `base_repo.upstream_host`) and `base_repo.auth.token`, and a git credential `username` and `password` are plain `type: string` with no `maxLength`. The spec documents no `400` for their length | A sync link whose upstream URL (`https://{host}/{owner}/{name}.git`) is over 131,070 bytes, and a token, `username` or `password` over 131,042 bytes, answer `400` before anything is stored, with `invalid_params` naming each field ("invalid upstream URL (want at most 131070 bytes)": the `owner`, the `name`, and the host when the body sent one) ([HTTP API](../reference/http-api.md#repository-lifecycle-repowrite--orgread)) | The sync run gives git the URL as one argument and each credential value as one environment entry, and Linux refuses one of 128 KiB or more, counting its terminating NUL and the `/` that git adds to the URL or the name of the variable. Such a link failed every run as `repository configuration error`, and the client learned it only from the failed run. A link or credential stored before still fails its runs that way | +| The `owner` and `name` of a sync link (`upstream`, `base_repo`), its host (`upstream.host`, `base_repo.upstream_host`) and `base_repo.auth.token`, and a git credential `username` and `password` are plain `type: string` with no `maxLength`. The spec documents no `400` for their length | A sync link whose upstream URL (`https://{host}/{owner}/{name}.git`, or the host as given when it has a scheme, without its trailing `/`) is over 131,070 bytes, and a token, `username` or `password` over 131,042 bytes, answer `400` before anything is stored, with `invalid_params` naming each field ("invalid upstream URL (want at most 131070 bytes)": the `owner`, the `name`, and the host when the body sent one) ([HTTP API](../reference/http-api.md#repository-lifecycle-repowrite--orgread)) | The sync run gives git the URL as one argument and each credential value as one environment entry, and Linux refuses one of 128 KiB or more, counting its terminating NUL and the `/` that git adds to the URL or the name of the variable. Such a link failed every run as `repository configuration error`, and the client learned it only from the failed run. A link or credential stored without the check (before it, or by an older server during an upgrade) still fails its runs that way | | `commit_message` and the `name` and `email` of an `author` or `committer` (`commit-pack`, `diff-commit`, `restore-commit` and `POST /repos/merge`), the `author` of `POST /repos/notes` (also the deprecated body of `DELETE /repos/notes`), and the `query.pattern` of `POST /repos/grep` are plain `type: string`. The spec documents no `400` for their content | Such a value that holds a NUL byte answers `400` ("invalid commit_message (want UTF-8 text without NUL)"). `merge`, `notes` and `grep` name the field in `invalid_params`. `commit-pack` and `diff-commit` answer the result envelope with `status` `invalid`, and `restore-commit` the plain problem body. A merge request refuses the NUL even when no commit is made, as it refuses a newline in an ident | git refuses a NUL in a message, takes a pattern as a process argument and an ident as an environment value, and none of them can hold a NUL. These requests answered `500` | | Every `path` parameter, grep `paths` entry and `query.pattern`, blame `range`, and the `name` and `email` of an `author` or `committer` (commit writes, `merge`, `notes`) is a plain `type: string` with no `maxLength`. The spec documents no `400` for their length | A `path` or `paths` entry over 131,060 bytes, a `range` or `query.pattern` over 131,071 bytes, and a `name` or `email` over 131,051 bytes answer `400` before git starts, with the field and the limit in the message ("invalid path (want at most 131060 bytes)"). `merge`, `notes` and `grep` name the field in `invalid_params`, and `commit-pack` and `diff-commit` answer the result envelope with `status` `invalid`. A `commit_message` has no such limit: git reads it on standard input ([HTTP API](../reference/http-api.md#conventions)) | git takes each of these values as one argument or one environment entry, and Linux refuses one of 128 KiB or more, counting its terminating NUL. These requests answered `500` from 128 KiB on. The limit holds on every platform, so a client gets the same answer from every server | | `GET /repos` `q` and `prefix` are plain `type: string`. The spec documents a `400` only for a pagination cursor that cannot be decoded | A `q` or `prefix` that holds a NUL byte or invalid UTF-8 answers `400` `bad_request` ("invalid q (want UTF-8 text without NUL)", or the same text for `prefix`) on `/api/repos` and `/api/v1/repos`, on every backend ([HTTP API](../reference/http-api.md#repository-lifecycle-repowrite--orgread)). The only other list route with such a filter, `GET .../notes/refs` (`prefix`), has no NUL or UTF-8 check: it compares the raw bytes with the ref names in memory | PostgreSQL refuses a NUL byte or invalid UTF-8 in a text parameter (SQLSTATE 22021). The API refuses such a value before the store runs, so every backend gives the same answer ([#783](https://github.com/coder/openstorage/pull/783)). An empty `200` page would be correct only if no repository name can hold such bytes, and nothing proves that ([#788](https://github.com/coder/openstorage/issues/788)) | diff --git a/docs/reference/http-api.md b/docs/reference/http-api.md index 4a9905c7..b602ca07 100644 --- a/docs/reference/http-api.md +++ b/docs/reference/http-api.md @@ -375,7 +375,8 @@ the same scopes, query parameters and responses. stored text must be UTF-8 without NUL, as a PostgreSQL text value must; a request body decodes to valid UTF-8 already. - A sync create whose values git cannot take → 400 `bad_request`, and nothing is - stored. An upstream URL (`https://{host}/{owner}/{name}.git`) over 131,070 bytes + stored. An upstream URL over 131,070 bytes (`https://{host}/{owner}/{name}.git`; a + host given with `http://` or `https://` is used as is, without its trailing `/`) names the `owner`, the `name`, and the host field when the body sent one (`upstream.host`, `base_repo.upstream_host`), each with the reason `must keep the upstream URL at most 131070 bytes` (message `invalid upstream URL (want at most @@ -517,7 +518,8 @@ or NUL fails the sync as a configuration error. So does one over 131,042 bytes, git gets it: git takes it as one environment entry, such as `OPENSTORAGE_ASKPASS_PASSWORD=`, which must fit 128 KiB with its terminating NUL. The credential routes and `POST /repos` refuse such a long value, so only a value -stored before they did reaches this check. +stored without that check (before it, or by an older server during an upgrade) reaches +this one. ### Files & trees (`git:read`) @@ -838,16 +840,18 @@ Before [#647](https://github.com/coder/openstorage/issues/647), a failed guard a | `POST /api/repos/{repo_name}/upstream/pull` | git:write | Path style only (spec 2026-10-05). Same handler, scope, ref-policy check and responses as `POST /repos/pull-upstream`. The optional, deprecated body `{ref?, after?}` is decoded strictly: an unknown field or a `null` → 400. Its fields are ignored | | `POST /repos/push-upstream` | git:write + git:read | `{force?}` → **202 Accepted** (async). Sync-linked repos only. open.storage extension | -A pull or push run whose upstream URL (`https://{host}/{owner}/{name}.git`) is over -131,070 bytes fails as `repository configuration error` before git starts. git gets the URL -as one argument, and its HTTP transport hands it on with a `/` added, which must fit -128 KiB with its terminating NUL. `POST /repos` refuses such a link with `400` -([Repository lifecycle](#repository-lifecycle-repowrite--orgread)), so only a link -registered before it did reaches this check. A stored credential that git cannot take fails the same -way ([Git credentials](#git-credentials-generic-https-upstreams)). Values that each fit -can still pass the total limit together: one git start can carry the URL twice and both -credential entries, about 512 KiB at the limits, so a server with a stack limit of 2 MiB -or less can fail such a run as a failed fetch or push. +A pull or push run whose upstream URL (`https://{host}/{owner}/{name}.git`, or the host +as given when it has a scheme, without its trailing `/`) is over 131,070 bytes fails as +`repository configuration error` before git starts. git gets the URL as one argument, +and its HTTP transport hands it on with a `/` added, which must fit 128 KiB with its +terminating NUL. `POST /repos` refuses such a link with `400` ([Repository +lifecycle](#repository-lifecycle-repowrite--orgread)), so only a link stored without +that check (before it, or by an older server during an upgrade) reaches this one. A +stored credential that git cannot take fails the same way ([Git +credentials](#git-credentials-generic-https-upstreams)). Values that each fit can still +pass the total limit together: one git start can carry the URL twice and both credential +entries, about 512 KiB at the limits, so a server with a stack limit of 2 MiB or less +can fail such a run as a failed fetch or push. #### Deprecation header diff --git a/internal/api/sync_limits_test.go b/internal/api/sync_limits_test.go index 04ca599b..caffd9a2 100644 --- a/internal/api/sync_limits_test.go +++ b/internal/api/sync_limits_test.go @@ -51,10 +51,17 @@ func TestSyncValuesGitCannotTakeAtRegistration(t *testing.T) { } return body(map[string]any{"upstream": cfg}) } - baseRepo := func(o, token string) string { - return body(map[string]any{"base_repo": map[string]any{ - "provider": "gitea", "owner": o, "name": "n", "upstream_host": host, "auth": map[string]any{"token": token}}}) + baseRepo := func(o, token string, extra ...map[string]any) string { + spec := map[string]any{"provider": "gitea", "owner": o, "name": "n", "upstream_host": host, "auth": map[string]any{"token": token}} + for _, e := range extra { + for k, v := range e { + spec[k] = v + } + } + return body(map[string]any{"base_repo": spec}) } + // A host with a scheme is used as is, without its trailing "/". + pathHost := "https://" + host + "/sub/" if got := len(upsync.Upstream{Owner: owner(hostBase, "n", urlMax), Name: "n", UpstreamHost: host}.CloneURL()); got != urlMax { t.Fatalf("the URL at the limit is %d bytes, want %d", got, urlMax) } @@ -79,9 +86,18 @@ func TestSyncValuesGitCannotTakeAtRegistration(t *testing.T) { urlMsg+`; invalid upstream access "x"`, ipParam{"upstream.owner", urlWhy}, ipParam{"upstream.name", urlWhy}, ipParam{"upstream.access", "must be app or public"}, ipParam{"upstream.host", urlWhy}), - // A host with a problem of its own makes no URL. - post("upstream bad host is not a URL problem", upstream(over, map[string]any{"host": "bad host"}), + post("upstream URL over the limit with a host that has a path", upstream(owner(pathHost, "n", urlMax+1), map[string]any{"host": pathHost}), + urlMsg, ipParam{"upstream.owner", urlWhy}, ipParam{"upstream.name", urlWhy}, ipParam{"upstream.host", urlWhy}), + // A field with a problem of its own makes no URL, even when + // the URL would be over the limit. + post("upstream bad host is not a URL problem", + upstream(owner("https://bad host/", "n", urlMax+1), map[string]any{"host": "bad host"}), `invalid upstream host "bad host"`, ipParam{"upstream.host", "is not a valid host"}), + post("upstream owner with a NUL is not a URL problem", upstream(over+"\x00", nil), + "invalid upstream.owner (want UTF-8 text without NUL)", ipParam{"upstream.owner", ipcNulReason}), + post("upstream name with a NUL is not a URL problem", + upstream("o", map[string]any{"name": strings.Repeat("n", urlMax) + "\x00"}), + "invalid upstream.name (want UTF-8 text without NUL)", ipParam{"upstream.name", ipcNulReason}), post("base_repo URL one byte over", baseRepo(over, "t"), urlMsg, ipParam{"base_repo.owner", urlWhy}, ipParam{"base_repo.name", urlWhy}, ipParam{"base_repo.upstream_host", urlWhy}), post("base_repo token one byte over", baseRepo("o", strings.Repeat("t", credMax+1)), @@ -90,13 +106,28 @@ func TestSyncValuesGitCannotTakeAtRegistration(t *testing.T) { urlMsg+"; invalid base_repo.auth.token (want at most 131042 bytes)", ipParam{"base_repo.owner", urlWhy}, ipParam{"base_repo.name", urlWhy}, ipParam{"base_repo.auth.token", credWhy}, ipParam{"base_repo.upstream_host", urlWhy}), + // auth.token and default_branch lie between name and + // upstream_host in the docs: their items stay there. + post("base_repo URL, token and default branch", + baseRepo(over, strings.Repeat("t", credMax+1), map[string]any{"default_branch": "bad..name"}), + urlMsg+"; invalid base_repo.auth.token (want at most 131042 bytes); invalid default_branch", + ipParam{"base_repo.owner", urlWhy}, ipParam{"base_repo.name", urlWhy}, ipParam{"base_repo.auth.token", credWhy}, + ipParam{"base_repo.default_branch", "is not a valid branch name"}, ipParam{"base_repo.upstream_host", urlWhy}), + post("base_repo URL over the limit with the default host", + body(map[string]any{"base_repo": map[string]any{"provider": "github", "owner": owner(githubBase, "n", urlMax+1), "name": "n"}}), + urlMsg, ipParam{"base_repo.owner", urlWhy}, ipParam{"base_repo.name", urlWhy}), + // A value over the limit that also holds a NUL is one problem. + post("base_repo token over with a NUL", baseRepo("o", strings.Repeat("t", credMax+1)+"\x00"), + "invalid base_repo.auth.token (want UTF-8 text without NUL)", ipParam{"base_repo.auth.token", ipcNulReason}), }) // At the limits the link and its credential are stored. atURL := createRepo(t, e, org, "lim-url-"+randomHex(t, 4), []byte(upstream(owner(hostBase, "n", urlMax), nil)), http.StatusCreated) + atPath := createRepo(t, e, org, "lim-path-"+randomHex(t, 4), + []byte(upstream(owner(pathHost, "n", urlMax), map[string]any{"host": pathHost})), http.StatusCreated) atBase := createRepo(t, e, org, "lim-base-"+randomHex(t, 4), []byte(baseRepo(owner(hostBase, "n", urlMax), strings.Repeat("t", credMax))), http.StatusCreated) - for _, created := range []createRepoResponse{atURL, atBase} { + for _, created := range []createRepoResponse{atURL, atPath, atBase} { link, err := e.store.(meta.EventingStore).GetRepoUpstream(context.Background(), created.RepoID) if err != nil { t.Fatalf("upstream of %s: %v", created.RepoID, err) @@ -138,6 +169,9 @@ func TestSyncValuesGitCannotTakeAtRegistration(t *testing.T) { {name: "POST username over with a NUL", method: http.MethodPost, path: v1, token: repoTok, body: `{"username":"` + long + `\u0000","password":"p"}`, wantError: "invalid username (want UTF-8 text without NUL)", want: []ipParam{{"username", ipcNulReason}}}, + {name: "POST password over with a NUL", method: http.MethodPost, path: v1, token: repoTok, + body: `{"password":"` + long + `\u0000"}`, wantError: "invalid password (want UTF-8 text without NUL)", + want: []ipParam{{"password", ipcNulReason}}}, }) fresh, err := e.store.CreateRepo(context.Background(), org, "https://example.com/"+org+"/lim-fresh-"+randomHex(t, 4), "main") if err != nil { From 2fb75df1c7bb6f59b2f61c7cdb6e0f7c635fc9c9 Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Mon, 16 Nov 2026 15:39:32 +0000 Subject: [PATCH 4/5] test(api): pin the base_repo bad host and the unsupported provider next to a long URL MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The second review found two branches without a case: a base_repo whose upstream_host is bad and whose URL would be too long, and an unsupported provider with a host sent, where the URL is still checked. Both mutants now fail. The docs say that a host loses all its trailing slashes. Signed-off-by: Thomas Kosiewski --- _Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `xhigh`_ Change-Id: I244f64c9b45b9ee55c048571a2f4224958435baf --- docs/explanation/wire-compatibility.md | 2 +- docs/reference/http-api.md | 10 +++++----- internal/api/sync_limits_test.go | 9 ++++++++- 3 files changed, 14 insertions(+), 7 deletions(-) diff --git a/docs/explanation/wire-compatibility.md b/docs/explanation/wire-compatibility.md index 5c65ffb0..c9904ca1 100644 --- a/docs/explanation/wire-compatibility.md +++ b/docs/explanation/wire-compatibility.md @@ -235,7 +235,7 @@ the [archived phase notes](../archive/phases/)): | A `crit` member in the JWT header: the vendored docs and spec say nothing about it. They list `alg` and `typ` as the header ([authentication guide](../upstream/getting-started/authentication.md)) | A token whose header has a `crit` member of any value fails verification, even when the signature is valid. The status is the one for any token that fails verification (the row above) | RFC 7515 §4.1.11: a recipient that does not understand a listed extension must refuse the token. open.storage understands none, so it refuses any `crit` | | Every path field and `path` parameter is a plain `type: string`. The spec says nothing about tree entry names whose bytes are not valid UTF-8 | A response path that is not valid UTF-8 still lists with one U+FFFD per invalid byte, and gets an extra sibling `_base64` with the raw bytes in standard base64 (for example `path_base64`, `paths_base64`, `old_path_base64`, `original_path_base64`). The sibling is absent for valid UTF-8 paths, so those responses are unchanged. This covers `files`, `files/metadata`, the diff routes, `blame` and `grep`. The merge conflict paths do not have it yet, and JSON request bodies cannot name such a file yet. The `path` query parameter takes the raw bytes percent-encoded ([path encoding](../reference/http-api.md#conventions)) | JSON strings hold Unicode text, so such a byte has no exact JSON form. A sibling field (like GitHub's `pathRaw`) is additive: no SDK reads it, and the SDKs ignore response keys they do not know. If upstream documents another form, we carry both ([#587](https://github.com/coder/openstorage/issues/587)) | | Text the request asks the server to store (a webhook `secret`, a git credential `username` and `password`, the `owner`, `name` and `auth.token` of a sync link, a `pull-upstream` `ref`) is plain `type: string`. The spec documents no `400` for its content | Such a value that holds a NUL byte answers `400` `bad_request` ("invalid secret (want UTF-8 text without NUL)", with `invalid_params` naming the field where the route has them), on every backend. A `GET /repos` `cursor` whose repo ID holds a NUL byte or invalid UTF-8 answers the `400` of an undecodable cursor. A path or body ID with such text (a webhook or git credential `id`) answers the `404` of an unknown ID | PostgreSQL refuses a NUL byte or invalid UTF-8 in a text parameter (SQLSTATE 22021) and answered `500` for most of these values, while SQLite and S3-WAL stored them. With `OPENSTORAGE_SECRETS_KEY` set in the default `encrypt` mode, the `secret`, `password` and `auth.token` reach PostgreSQL encrypted, so PostgreSQL stored them too, and such a value now answers `400` there as well. A `ref` is stored only by PostgreSQL's job queue, as `jsonb`, which cannot hold a NUL (SQLSTATE 22P05). Every backend now gives the same answer, as for `q` and `prefix` below. The PostgreSQL store answers `ErrNotFound` for such a key before it runs the query | -| The `owner` and `name` of a sync link (`upstream`, `base_repo`), its host (`upstream.host`, `base_repo.upstream_host`) and `base_repo.auth.token`, and a git credential `username` and `password` are plain `type: string` with no `maxLength`. The spec documents no `400` for their length | A sync link whose upstream URL (`https://{host}/{owner}/{name}.git`, or the host as given when it has a scheme, without its trailing `/`) is over 131,070 bytes, and a token, `username` or `password` over 131,042 bytes, answer `400` before anything is stored, with `invalid_params` naming each field ("invalid upstream URL (want at most 131070 bytes)": the `owner`, the `name`, and the host when the body sent one) ([HTTP API](../reference/http-api.md#repository-lifecycle-repowrite--orgread)) | The sync run gives git the URL as one argument and each credential value as one environment entry, and Linux refuses one of 128 KiB or more, counting its terminating NUL and the `/` that git adds to the URL or the name of the variable. Such a link failed every run as `repository configuration error`, and the client learned it only from the failed run. A link or credential stored without the check (before it, or by an older server during an upgrade) still fails its runs that way | +| The `owner` and `name` of a sync link (`upstream`, `base_repo`), its host (`upstream.host`, `base_repo.upstream_host`) and `base_repo.auth.token`, and a git credential `username` and `password` are plain `type: string` with no `maxLength`. The spec documents no `400` for their length | A sync link whose upstream URL (`https://{host}/{owner}/{name}.git`, or the host as given when it has a scheme, without trailing `/`s) is over 131,070 bytes, and a token, `username` or `password` over 131,042 bytes, answer `400` before anything is stored, with `invalid_params` naming each field ("invalid upstream URL (want at most 131070 bytes)": the `owner`, the `name`, and the host when the body sent one) ([HTTP API](../reference/http-api.md#repository-lifecycle-repowrite--orgread)) | The sync run gives git the URL as one argument and each credential value as one environment entry, and Linux refuses one of 128 KiB or more, counting its terminating NUL and the `/` that git adds to the URL or the name of the variable. Such a link failed every run as `repository configuration error`, and the client learned it only from the failed run. A link or credential stored without the check (before it, or by an older server during an upgrade) still fails its runs that way | | `commit_message` and the `name` and `email` of an `author` or `committer` (`commit-pack`, `diff-commit`, `restore-commit` and `POST /repos/merge`), the `author` of `POST /repos/notes` (also the deprecated body of `DELETE /repos/notes`), and the `query.pattern` of `POST /repos/grep` are plain `type: string`. The spec documents no `400` for their content | Such a value that holds a NUL byte answers `400` ("invalid commit_message (want UTF-8 text without NUL)"). `merge`, `notes` and `grep` name the field in `invalid_params`. `commit-pack` and `diff-commit` answer the result envelope with `status` `invalid`, and `restore-commit` the plain problem body. A merge request refuses the NUL even when no commit is made, as it refuses a newline in an ident | git refuses a NUL in a message, takes a pattern as a process argument and an ident as an environment value, and none of them can hold a NUL. These requests answered `500` | | Every `path` parameter, grep `paths` entry and `query.pattern`, blame `range`, and the `name` and `email` of an `author` or `committer` (commit writes, `merge`, `notes`) is a plain `type: string` with no `maxLength`. The spec documents no `400` for their length | A `path` or `paths` entry over 131,060 bytes, a `range` or `query.pattern` over 131,071 bytes, and a `name` or `email` over 131,051 bytes answer `400` before git starts, with the field and the limit in the message ("invalid path (want at most 131060 bytes)"). `merge`, `notes` and `grep` name the field in `invalid_params`, and `commit-pack` and `diff-commit` answer the result envelope with `status` `invalid`. A `commit_message` has no such limit: git reads it on standard input ([HTTP API](../reference/http-api.md#conventions)) | git takes each of these values as one argument or one environment entry, and Linux refuses one of 128 KiB or more, counting its terminating NUL. These requests answered `500` from 128 KiB on. The limit holds on every platform, so a client gets the same answer from every server | | `GET /repos` `q` and `prefix` are plain `type: string`. The spec documents a `400` only for a pagination cursor that cannot be decoded | A `q` or `prefix` that holds a NUL byte or invalid UTF-8 answers `400` `bad_request` ("invalid q (want UTF-8 text without NUL)", or the same text for `prefix`) on `/api/repos` and `/api/v1/repos`, on every backend ([HTTP API](../reference/http-api.md#repository-lifecycle-repowrite--orgread)). The only other list route with such a filter, `GET .../notes/refs` (`prefix`), has no NUL or UTF-8 check: it compares the raw bytes with the ref names in memory | PostgreSQL refuses a NUL byte or invalid UTF-8 in a text parameter (SQLSTATE 22021). The API refuses such a value before the store runs, so every backend gives the same answer ([#783](https://github.com/coder/openstorage/pull/783)). An empty `200` page would be correct only if no repository name can hold such bytes, and nothing proves that ([#788](https://github.com/coder/openstorage/issues/788)) | diff --git a/docs/reference/http-api.md b/docs/reference/http-api.md index b602ca07..fd1a16f1 100644 --- a/docs/reference/http-api.md +++ b/docs/reference/http-api.md @@ -376,7 +376,7 @@ the same scopes, query parameters and responses. decodes to valid UTF-8 already. - A sync create whose values git cannot take → 400 `bad_request`, and nothing is stored. An upstream URL over 131,070 bytes (`https://{host}/{owner}/{name}.git`; a - host given with `http://` or `https://` is used as is, without its trailing `/`) + host given with `http://` or `https://` is used as is, without trailing `/`s) names the `owner`, the `name`, and the host field when the body sent one (`upstream.host`, `base_repo.upstream_host`), each with the reason `must keep the upstream URL at most 131070 bytes` (message `invalid upstream URL (want at most @@ -485,9 +485,9 @@ stores the username exactly as sent. An update that omits it clears the stored o A `username` or `password` that holds a NUL byte → 400 `bad_request` with `invalid_params` naming the field, on every credential route, and nothing is stored. So does one over -131,042 bytes (`must be at most 131042 bytes`), which the sync run could not hand to git. A credential `id` -that holds a NUL byte, in the body or in the path (also invalid UTF-8 in the path), names -no credential → 404 `credential_not_found`, as an unknown `id` does. +131,042 bytes (`must be at most 131042 bytes`), which the sync run could not hand to git. +A credential `id` that holds a NUL byte, in the body or in the path (also invalid UTF-8 in +the path), names no credential → 404 `credential_not_found`, as an unknown `id` does. A stored username is always sent as is. Git sends no credentials when the username is empty, so for a credential with no username the sync worker sends the upstream's @@ -841,7 +841,7 @@ Before [#647](https://github.com/coder/openstorage/issues/647), a failed guard a | `POST /repos/push-upstream` | git:write + git:read | `{force?}` → **202 Accepted** (async). Sync-linked repos only. open.storage extension | A pull or push run whose upstream URL (`https://{host}/{owner}/{name}.git`, or the host -as given when it has a scheme, without its trailing `/`) is over 131,070 bytes fails as +as given when it has a scheme, without trailing `/`s) is over 131,070 bytes fails as `repository configuration error` before git starts. git gets the URL as one argument, and its HTTP transport hands it on with a `/` added, which must fit 128 KiB with its terminating NUL. `POST /repos` refuses such a link with `400` ([Repository diff --git a/internal/api/sync_limits_test.go b/internal/api/sync_limits_test.go index caffd9a2..8d0b0b0f 100644 --- a/internal/api/sync_limits_test.go +++ b/internal/api/sync_limits_test.go @@ -60,7 +60,7 @@ func TestSyncValuesGitCannotTakeAtRegistration(t *testing.T) { } return body(map[string]any{"base_repo": spec}) } - // A host with a scheme is used as is, without its trailing "/". + // A host with a scheme is used as is, without trailing "/"s. pathHost := "https://" + host + "/sub/" if got := len(upsync.Upstream{Owner: owner(hostBase, "n", urlMax), Name: "n", UpstreamHost: host}.CloneURL()); got != urlMax { t.Fatalf("the URL at the limit is %d bytes, want %d", got, urlMax) @@ -95,9 +95,16 @@ func TestSyncValuesGitCannotTakeAtRegistration(t *testing.T) { `invalid upstream host "bad host"`, ipParam{"upstream.host", "is not a valid host"}), post("upstream owner with a NUL is not a URL problem", upstream(over+"\x00", nil), "invalid upstream.owner (want UTF-8 text without NUL)", ipParam{"upstream.owner", ipcNulReason}), + // The URL needs no provider when the body sent the host. + post("upstream URL over the limit with an unsupported provider", upstream(over, map[string]any{"provider": "zz"}), + `unsupported upstream provider "zz"; `+urlMsg, ipParam{"upstream.provider", "is not supported"}, + ipParam{"upstream.owner", urlWhy}, ipParam{"upstream.name", urlWhy}, ipParam{"upstream.host", urlWhy}), post("upstream name with a NUL is not a URL problem", upstream("o", map[string]any{"name": strings.Repeat("n", urlMax) + "\x00"}), "invalid upstream.name (want UTF-8 text without NUL)", ipParam{"upstream.name", ipcNulReason}), + post("base_repo bad host is not a URL problem", + baseRepo(owner("https://bad host/", "n", urlMax+1), "t", map[string]any{"upstream_host": "bad host"}), + `invalid base_repo upstream_host "bad host"`, ipParam{"base_repo.upstream_host", "is not a valid host"}), post("base_repo URL one byte over", baseRepo(over, "t"), urlMsg, ipParam{"base_repo.owner", urlWhy}, ipParam{"base_repo.name", urlWhy}, ipParam{"base_repo.upstream_host", urlWhy}), post("base_repo token one byte over", baseRepo("o", strings.Repeat("t", credMax+1)), From c384208861f9cdeacd186294f8b67beca33c290c Mon Sep 17 00:00:00 2001 From: Thomas Kosiewski Date: Mon, 16 Nov 2026 16:45:17 +0000 Subject: [PATCH 5/5] chore: re-record the lane-schedule fingerprints after merging main MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Thomas Kosiewski --- _Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `xhigh`_ Change-Id: Ia1f99374b8fc08039135fd970a650ad0376ceaa9 --- .agents/skills/verify-lane-schedule/SKILL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.agents/skills/verify-lane-schedule/SKILL.md b/.agents/skills/verify-lane-schedule/SKILL.md index 6551defc..274274da 100644 --- a/.agents/skills/verify-lane-schedule/SKILL.md +++ b/.agents/skills/verify-lane-schedule/SKILL.md @@ -226,7 +226,7 @@ With every fact TRUE in the code, setting each fact FALSE in the model alone fai ## Coverage record -Sync commit: `0f5979dd0589da6224230f74e9bc7d15143ef5d1` (the last commit at which someone confirmed that the model describes this code; only `check.sh --record` moves it). +Sync commit: `33c2441001bb13345d5304a891ca6f16ccf0b24b` (the last commit at which someone confirmed that the model describes this code; only `check.sh --record` moves it). Every declaration in these files is covered: