diff --git a/docs/explanation/wire-compatibility.md b/docs/explanation/wire-compatibility.md index 76c5b3ed..1438bfc9 100644 --- a/docs/explanation/wire-compatibility.md +++ b/docs/explanation/wire-compatibility.md @@ -216,7 +216,7 @@ the [archived phase notes](../archive/phases/)): | `POST /repos` with both `upstream` and `base_repo`: "do not set" both, no documented error | 400 | Ambiguous request; failing loudly beats guessing which form wins | | `POST /repos` without `default_branch`: a sync from GitHub takes the GitHub repository's default branch (spec `40e43e00`, 2026-10-28) | PostgreSQL and SQLite: the repository starts on `main`, and the first successful sync adopts the upstream `HEAD` branch before it sends `repo.sync.succeeded`. S3-WAL: the repository keeps `main` | The create makes no network call ([#599](https://github.com/coder/openstorage/issues/599)). S3-WAL metadata v2 is frozen and cannot store the "branch omitted" flag ([#603](https://github.com/coder/openstorage/issues/603), trigger: the next S3-WAL metadata format change) | | `POST /repos` is idempotent on the name: `200` for any existing name unless its upstream differs (spec 2026-10-07) | A finished repo answers `200` to a matching repeat create: a sync create with the same upstream and `forward_pushes`, or a plain, copy or fork create of a repo without an upstream. A different upstream is `409`, as upstream documents, and so is a create form that does not match the repo (sync onto a plain repo, plain onto a synced repo). Repos created before create tracking (on S3-WAL, every repo created by v0.18.0 or earlier) answer `409` to every repeat. While a copy or fork create is still running, plain, copy and fork creates of the name are `503` with `Retry-After`. A pending sync create is finished by a matching sync create with `git:write` (`200`); other forms are `409` ([Repeat creates](../reference/http-api.md#repository-lifecycle-repowrite--orgread)). A name a deleted repo holds is `404` | A `200` names a repo only after its create finished, so it never names a repo that the failing create then deletes ([#388](https://github.com/coder/openstorage/issues/388)). A form mismatch is `409` because a `200` would hide that the request was not applied. Older repos store no create state (#335 for S3-WAL) | -| RFC 9457 problem-details error bodies (spec). Since upstream's 2026-09-24 spec update, `ProblemDetails` requires a snake_case `code` and tells clients to branch on `code`, not on `detail` | The plain `{"error":""}` envelope carries problem-details fields next to `error`: `type` (`about:blank`), `title`, `status`, `detail` (same string as `error`), `code` ([#349](https://github.com/coder/openstorage/issues/349)), and `instance`, the request path without the query and cut at 16384 bytes ([#352](https://github.com/coder/openstorage/issues/352)). `code` is upstream's generic per-status code, except for eleven of upstream's specific codes: `repository_not_found`, `insufficient_scope`, `repo_name_mismatch`, `branch_not_found`, `ref_not_found`, `credential_not_found`, `credential_exists`, `repository_deleted`, `authentication_unavailable` (403, key store failure) and `route_not_found` (404, with a 405 `method_not_allowed` problem body for a path that has routes for other methods) ([#352](https://github.com/coder/openstorage/issues/352)), and `ref_guard_failed` with its `guard` and `current_sha` members on every failed ref guard ([#568](https://github.com/coder/openstorage/issues/568), [#647](https://github.com/coder/openstorage/issues/647)). A 400 for a request body carries the optional `invalid_params` list (`ValidationProblemDetails`, spec 2026-10-08, [#396](https://github.com/coder/openstorage/issues/396)) when open.storage can name the invalid fields: body decode failures (wrong JSON type, and unknown, duplicate or `null` members of strictly decoded bodies) and field checks (required fields, allowed values, ranges, name formats). A body with several invalid fields gets one item for each of them, as the spec says (`invalid_params` "names each invalid field"), in the order of the route's request body in [http-api.md](../reference/http-api.md), with at most 100 items and a `detail` that joins the messages by `; `. For a field of the wrong JSON type the message is `name reason`, as in the spec's example (`default_branch must be a string`), and no answer of a REST route to a request body that does not decode repeats the text of Go's JSON decoder, which names Go types and structs. A body that is not valid JSON answers `invalid JSON body`. The strictly decoded routes answer `invalid JSON body: body must be a JSON object`, and `restore-commit` answers every body that does not decode with `body must be {"metadata": {...}}` ([#841](https://github.com/coder/openstorage/issues/841)). A strictly decoded body is checked for its members first, and the answer names each member that is unknown, repeated or `null`, or that must be an object and is not (an empty member name names no field), and none of the other problems. When it names more than one member, it cuts each unknown member name at 32 bytes (such an answer has at most 105,203 bytes and the request path), so the `name` of such an item is not the whole wire name. Two unknown names that are the same in their first 32 bytes give one item, and `and N more` counts the others, so that a name appears once in `invalid_params`. The body and `author` each name at most 100 unknown members, and `and N more` counts the others, each name once however often the request repeats it. Once the members are fine, the decoder of a strictly decoded body (`POST .../notes/append` and `POST .../upstream/pull`) names only the first field of the wrong type. On every other route that decodes its body as one JSON object, except `commit-pack`, `diff-commit` and `restore-commit`, when the JSON decoder finds fields of the wrong type, the answer names each of them in the order of the body and none of the other problems: a body with fields of the wrong type and fields that fail a check of their own names the wrong-type fields only. The answer names up to 100 such fields, within a work bound: the server decodes the body once more for each field it names and stops when the decodes add up to 256 KiB, so a body of 2 KiB can name 100 fields, a body of 10 KiB about 25, and a body of more than 128 KiB only the first. The server neither names nor counts the rest, so `detail` does not end with `and N more` for them. `POST /repos/merge`, `POST /repos`, `POST /repos/branches/create`, `POST /repos/tags`, `POST /repos/grep`, `POST /repos/archive`, `POST /repos/notes`, `DELETE /repos/notes` (a problem with a query parameter, or with a missing or empty `object_ref`, adds its message and names no field), the field checks of `POST .../notes/append` and the git-credentials routes `POST` and `PUT` (with their path forms) give a body with several invalid fields one item for each of them, and `PATCH /repo`, the pull-upstream routes, `DELETE /repos/branches` (for a name from the body), `DELETE /repos/tags` and `DELETE /repos/git-credentials` check one body field each, so they name it. The commit write routes (`CommitWriteResponse` or `ProblemDetails` upstream, which have no `invalid_params`), the webhook routes and `push-upstream` (not part of upstream's API) are not converted: a check there can hide the checks after it. `POST /repos` decodes its `upstream` and `base_repo` members on their own, and only when the fields of the body itself have the right types: `{"default_branch":7,"base_repo":{"owner":1}}` names only `default_branch`. Then a body with several fields of the wrong type inside `base_repo` names each of them, and `upstream`, which is decoded strictly, names the first. Every body with the problem members has `Content-Type: application/problem+json`, as upstream documents and serves ([#352](https://github.com/coder/openstorage/issues/352) item 5); result envelopes and success bodies stay `application/json`. Not emitted: upstream's other specific codes and `invalid_params` for failures that cannot name one field. `repo_claim_missing` is not sent because a token without a `repo` claim is org-wide ([#658](https://github.com/coder/openstorage/issues/658)): a per-repo route refuses it with 403, as it refuses `repo: "org"`. `merge_conflict` is listed upstream, but the merge 409 for file conflicts is documented as `MergeConflict`, which has no `code` member; `sync_in_progress` and `upstream_already_configured` are documented on no operation, and no open.storage answer matches them; the hosting, deployment, GitHub App, cold-storage and idempotency codes belong to features open.storage lacks. The [streaming protocols](streaming-commit-protocols.md) `{"result":{"success":false,…}}` envelope carries no problem-details fields, except the 409 body of a failed ref guard on a commit write or a notes write, which keeps the `result` object (and the note members) and adds the problem members as upstream's `CommitWriteRefGuardProblemDetails`, `RestoreCommitRefGuardProblemDetails` and `NotesWriteRefGuardProblemDetails`, sent as `application/problem+json`. Auth failures (401, 403), the gateway 429 and the path-style alias and git-credentials failures use the plain problem body on every route, as upstream documents ([#352](https://github.com/coder/openstorage/issues/352) item 3). `commit-pack`, `diff-commit` and `restore-commit` send the envelope only on the statuses for which upstream documents `CommitWriteResponse` or `ReposCommitsRestoreResponse` (400, 409, 412 and 413 on commit-pack; 400, 409 and 412 on diff-commit; 412 on restore-commit) and the plain problem body on every other failure. Merge answers file conflicts with upstream's `MergeConflict` (`error`, `conflict_type: merge_conflict`, `conflict_paths`, `merge_base_sha`) and every other failure with the plain problem body. A target that moved during the merge is a 409 `conflict` problem body: upstream sends `MergeConflict` with `target_moved` or `repo_changed` only after an automatic stale-target retry, which open.storage does not do. Note writes answer every failure with the plain problem body except the guard body above. Failures use one of these shapes, by route and failure class. Do not assume one shape per route family: branch on the status code, or check both shapes. Per-route contracts are in [HTTP API](../reference/http-api.md) and [streaming protocols](streaming-commit-protocols.md) | Recorded deviations ([phase-3 notes](../archive/phases/12-phase3-implementation-notes.md), [ADR-26](../adr/026-result-envelope-statuses-499-is-not-emittable.md)): all three SDKs branch on status codes. At the 2026-09-25 audit this held for SDK `1.17.1` (stable) and `1.18.0-beta.0` (beta): no SDK reads `code`, `detail`, or any problem-details field other than the legacy `error` string. SDK `0.1.0` (Go, Python, TypeScript, stable since 2026-11-04) reads every error body as JSON whatever its type, and the `1.18.0-beta` clients accept `application/problem+json`; the `1.17.1` Go and TypeScript clients parse error JSON only for `application/json`, so on a problem body they report the raw body text as the message (a breaking change for them, accepted because upstream sends the same type). Upstream says to treat an unknown code as the generic code for its status, so a client that follows that guidance handles every `code` open.storage sends; a client that expects one of the specific codes open.storage does not emit sees the generic one instead. `status` is a JSON number, as in upstream's bodies: a client that decoded the old `{"error"}` body into a string-only map (such as Go's `map[string]string`) must switch to a struct or `map[string]any`. At the 2026-10-08 audit, no SDK (`1.17.1`, `1.18.0-beta.1`) reads `invalid_params`; the member is optional, so a client that ignores it sees no change | +| RFC 9457 problem-details error bodies (spec). Since upstream's 2026-09-24 spec update, `ProblemDetails` requires a snake_case `code` and tells clients to branch on `code`, not on `detail` | The plain `{"error":""}` envelope carries problem-details fields next to `error`: `type` (`about:blank`), `title`, `status`, `detail` (same string as `error`), `code` ([#349](https://github.com/coder/openstorage/issues/349)), and `instance`, the request path without the query and cut at 16384 bytes ([#352](https://github.com/coder/openstorage/issues/352)). `code` is upstream's generic per-status code, except for eleven of upstream's specific codes: `repository_not_found`, `insufficient_scope`, `repo_name_mismatch`, `branch_not_found`, `ref_not_found`, `credential_not_found`, `credential_exists`, `repository_deleted`, `authentication_unavailable` (403, key store failure) and `route_not_found` (404, with a 405 `method_not_allowed` problem body for a path that has routes for other methods) ([#352](https://github.com/coder/openstorage/issues/352)), and `ref_guard_failed` with its `guard` and `current_sha` members on every failed ref guard ([#568](https://github.com/coder/openstorage/issues/568), [#647](https://github.com/coder/openstorage/issues/647)). A 400 for a request body carries the optional `invalid_params` list (`ValidationProblemDetails`, spec 2026-10-08, [#396](https://github.com/coder/openstorage/issues/396)) when open.storage can name the invalid fields: body decode failures (wrong JSON type, and unknown, duplicate or `null` members of strictly decoded bodies) and field checks (required fields, allowed values, ranges, name formats). A body with several invalid fields gets one item for each of them, as the spec says (`invalid_params` "names each invalid field"), in the order of the route's request body in [http-api.md](../reference/http-api.md), with at most 100 items and a `detail` that joins the messages by `; `. For a field of the wrong JSON type the message is `name reason`, as in the spec's example (`default_branch must be a string`), and no answer of a REST route to a request body that does not decode repeats the text of Go's JSON decoder, which names Go types and structs. A body that is not valid JSON answers `invalid JSON body`. The strictly decoded routes answer `invalid JSON body: body must be a JSON object`, and `restore-commit` answers every body that does not decode with `body must be {"metadata": {...}}` ([#841](https://github.com/coder/openstorage/issues/841)). A strictly decoded body is checked for its members first, and the answer names each member that is unknown, repeated or `null`, or that must be an object and is not (an empty member name names no field), and none of the other problems. When it names more than one member, it cuts each unknown member name at 32 bytes (such an answer has at most 105,203 bytes and the request path), so the `name` of such an item is not the whole wire name. When it names one unknown member, it cuts a name of more than 256 bytes to at most 256, ending with `…`, as the other decoders cut the unknown name that they put in `detail`, and `detail` adds the length of the name (such an answer has at most 4,923 bytes and the request path), so the `name` of that item is not the whole wire name either. Two unknown names that are the same in their first 32 bytes give one item, and `and N more` counts the others, so that a name appears once in `invalid_params`. The body and `author` each name at most 100 unknown members, and `and N more` counts the others, each name once however often the request repeats it. Once the members are fine, the decoder of a strictly decoded body (`POST .../notes/append` and `POST .../upstream/pull`) names only the first field of the wrong type. On every other route that decodes its body as one JSON object, except `commit-pack`, `diff-commit` and `restore-commit`, when the JSON decoder finds fields of the wrong type, the answer names each of them in the order of the body and none of the other problems: a body with fields of the wrong type and fields that fail a check of their own names the wrong-type fields only. The answer names up to 100 such fields, within a work bound: the server decodes the body once more for each field it names and stops when the decodes add up to 256 KiB, so a body of 2 KiB can name 100 fields, a body of 10 KiB about 25, and a body of more than 128 KiB only the first. The server neither names nor counts the rest, so `detail` does not end with `and N more` for them. `POST /repos/merge`, `POST /repos`, `POST /repos/branches/create`, `POST /repos/tags`, `POST /repos/grep`, `POST /repos/archive`, `POST /repos/notes`, `DELETE /repos/notes` (a problem with a query parameter, or with a missing or empty `object_ref`, adds its message and names no field), the field checks of `POST .../notes/append` and the git-credentials routes `POST` and `PUT` (with their path forms) give a body with several invalid fields one item for each of them, and `PATCH /repo`, the pull-upstream routes, `DELETE /repos/branches` (for a name from the body), `DELETE /repos/tags` and `DELETE /repos/git-credentials` check one body field each, so they name it. The commit write routes (`CommitWriteResponse` or `ProblemDetails` upstream, which have no `invalid_params`), the webhook routes and `push-upstream` (not part of upstream's API) are not converted: a check there can hide the checks after it. `POST /repos` decodes its `upstream` and `base_repo` members on their own, and only when the fields of the body itself have the right types: `{"default_branch":7,"base_repo":{"owner":1}}` names only `default_branch`. Then a body with several fields of the wrong type inside `base_repo` names each of them, and `upstream`, which is decoded strictly, names the first. Every body with the problem members has `Content-Type: application/problem+json`, as upstream documents and serves ([#352](https://github.com/coder/openstorage/issues/352) item 5); result envelopes and success bodies stay `application/json`. Not emitted: upstream's other specific codes and `invalid_params` for failures that cannot name one field. `repo_claim_missing` is not sent because a token without a `repo` claim is org-wide ([#658](https://github.com/coder/openstorage/issues/658)): a per-repo route refuses it with 403, as it refuses `repo: "org"`. `merge_conflict` is listed upstream, but the merge 409 for file conflicts is documented as `MergeConflict`, which has no `code` member; `sync_in_progress` and `upstream_already_configured` are documented on no operation, and no open.storage answer matches them; the hosting, deployment, GitHub App, cold-storage and idempotency codes belong to features open.storage lacks. The [streaming protocols](streaming-commit-protocols.md) `{"result":{"success":false,…}}` envelope carries no problem-details fields, except the 409 body of a failed ref guard on a commit write or a notes write, which keeps the `result` object (and the note members) and adds the problem members as upstream's `CommitWriteRefGuardProblemDetails`, `RestoreCommitRefGuardProblemDetails` and `NotesWriteRefGuardProblemDetails`, sent as `application/problem+json`. Auth failures (401, 403), the gateway 429 and the path-style alias and git-credentials failures use the plain problem body on every route, as upstream documents ([#352](https://github.com/coder/openstorage/issues/352) item 3). `commit-pack`, `diff-commit` and `restore-commit` send the envelope only on the statuses for which upstream documents `CommitWriteResponse` or `ReposCommitsRestoreResponse` (400, 409, 412 and 413 on commit-pack; 400, 409 and 412 on diff-commit; 412 on restore-commit) and the plain problem body on every other failure. Merge answers file conflicts with upstream's `MergeConflict` (`error`, `conflict_type: merge_conflict`, `conflict_paths`, `merge_base_sha`) and every other failure with the plain problem body. A target that moved during the merge is a 409 `conflict` problem body: upstream sends `MergeConflict` with `target_moved` or `repo_changed` only after an automatic stale-target retry, which open.storage does not do. Note writes answer every failure with the plain problem body except the guard body above. Failures use one of these shapes, by route and failure class. Do not assume one shape per route family: branch on the status code, or check both shapes. Per-route contracts are in [HTTP API](../reference/http-api.md) and [streaming protocols](streaming-commit-protocols.md) | Recorded deviations ([phase-3 notes](../archive/phases/12-phase3-implementation-notes.md), [ADR-26](../adr/026-result-envelope-statuses-499-is-not-emittable.md)): all three SDKs branch on status codes. At the 2026-09-25 audit this held for SDK `1.17.1` (stable) and `1.18.0-beta.0` (beta): no SDK reads `code`, `detail`, or any problem-details field other than the legacy `error` string. SDK `0.1.0` (Go, Python, TypeScript, stable since 2026-11-04) reads every error body as JSON whatever its type, and the `1.18.0-beta` clients accept `application/problem+json`; the `1.17.1` Go and TypeScript clients parse error JSON only for `application/json`, so on a problem body they report the raw body text as the message (a breaking change for them, accepted because upstream sends the same type). Upstream says to treat an unknown code as the generic code for its status, so a client that follows that guidance handles every `code` open.storage sends; a client that expects one of the specific codes open.storage does not emit sees the generic one instead. `status` is a JSON number, as in upstream's bodies: a client that decoded the old `{"error"}` body into a string-only map (such as Go's `map[string]string`) must switch to a struct or `map[string]any`. At the 2026-10-08 audit, no SDK (`1.17.1`, `1.18.0-beta.1`) reads `invalid_params`; the member is optional, so a client that ignores it sees no change | | `201` on write endpoints (spec) | `commit-pack` / `diff-commit` / `restore-commit` and `POST`/`DELETE .../notes` return `200`. The path-style `POST .../notes/append` returns `201` | Recorded deviation ([phase-3 notes](../archive/phases/12-phase3-implementation-notes.md)): the SDKs accept both; schema-validating clients should expect `200` on those legacy routes and `201` on `notes/append` | | `restore-commit`: `412` when the branch already points at the target, the tree would not change, or the target is not an ancestor of the tip (spec) | Enforced for `base_ref`. The deprecated `target_commit_sha` form keeps restoring without these checks | The SDKs send `target_commit_sha`; adding the checks there would change what SDK callers see | | `POST .../branches/create` with a failed `expected_target_sha`: `409` "with code `conflict`" ([branch-create page](https://code.storage/docs/api/repos/branch-create.md), spec `5d8e1801`) | `409` `ref_guard_failed` with `guard` and `current_sha` ([details](../reference/http-api.md#guarded-branch-create-and-move)) | The spec's error-code list gives `ref_guard_failed` to a failed ref guard "such as `expected_target_sha`", and every other guard here answers it. The SDKs return it as a `RefGuardError` that unwraps to their `ConflictError`, and a client that treats an unknown code as the generic code for its status, as the spec says, reads `conflict` | diff --git a/docs/reference/http-api.md b/docs/reference/http-api.md index 10a7e092..10ab12ad 100644 --- a/docs/reference/http-api.md +++ b/docs/reference/http-api.md @@ -136,7 +136,12 @@ are in [`../upstream/openapi.yaml`](../upstream/openapi.yaml). or that has the dotted path of a member named or counted already, adds nothing. Past the 100 names of an object the count compares 64-bit hashes of the names, so two of them count as one in about one body of 1 MiB in a billion. An answer that names one - member keeps its name up to the cut of 16384 bytes above. The other checks run only + unknown member cuts a name of more than 256 bytes as the other bodies above do, to its + first 256 bytes or fewer at a character, and ends it with `…`: `error` and `detail` + add the length in bytes of the member name, without `author.` + (`unknown field "author.xx…" (1000000 bytes)`), and the `invalid_params` `name` ends + with `…` only, as at 32 bytes. Such an answer has at most 4,923 bytes and the request + path in `instance`. The other checks run only when the members are fine, and then the decoder of a strictly decoded body names only the first field of the wrong type (`{"note":1,"object_ref":2}` names `note`). `POST /api/repos/{repo_name}/upstream/pull` takes `{ref?, after?}` and works the same diff --git a/internal/api/decode_body.go b/internal/api/decode_body.go index 3b70911a..fc1c3fe8 100644 --- a/internal/api/decode_body.go +++ b/internal/api/decode_body.go @@ -7,7 +7,8 @@ import ( "net/http" "strconv" "strings" - "unicode/utf8" + + "github.com/coder/openstorage/internal/errtext" ) // maxDecodeWork bounds the bytes that unmarshalBody decodes for one body @@ -117,19 +118,19 @@ func paramsText(params []invalidParam) string { return strings.Join(items, "; ") } -// maxUnknownNameBytes bounds the member name that unknownFieldText repeats. -// The longest field name of a request body has 25 bytes, so a misspelled -// name is repeated whole. A longer name is no field name, and the body -// limit lets it reach 1 MiB: `error` and `detail` repeat it with escapes -// of up to 6 bytes for each byte, which made answers of 12 MB. +// maxUnknownNameBytes bounds the member name that an answer naming one +// unknown member repeats (unknownNameText). The longest field name of a +// request body has 25 bytes, so a misspelled name is repeated whole. A +// longer name is no field name, and the body limit lets it reach 1 MiB: +// `error` and `detail` repeat it with escapes of up to 6 bytes for each +// byte, which made answers of 12 MB. const maxUnknownNameBytes = 256 // unknownFieldText returns `unknown field "x"` for the error that // encoding/json gives a member that DisallowUnknownFields refuses, and "" // for any other error. That error has no type: its text is fixed (pinned by -// TestDecodeError), and it holds the member name only. A name of more than -// maxUnknownNameBytes is cut at a character to at most that many bytes, and -// ends with an ellipsis and its length: `unknown field "xx…" (1000000 bytes)`. +// TestDecodeError), and it holds the member name only, which +// unknownNameText cuts. func unknownFieldText(err error) string { rest, ok := strings.CutPrefix(err.Error(), "json: unknown field ") if !ok || !strings.HasPrefix(rest, `"`) { @@ -142,11 +143,20 @@ func unknownFieldText(err error) string { if len(name) <= maxUnknownNameBytes { return "unknown field " + rest } - cut := maxUnknownNameBytes - for cut > 0 && !utf8.RuneStart(name[cut]) { - cut-- - } - return "unknown field " + strconv.Quote(name[:cut]+"…") + " (" + strconv.Itoa(len(name)) + " bytes)" + return unknownNameText("", name) +} + +// unknownNameText returns `unknown field "x"` for an unknown member name +// of the object at the dotted path path ("" for the body), x being +// path+name. A name of more than maxUnknownNameBytes is cut at a +// character to at most that many bytes, and the text ends with an +// ellipsis and the length of the name: `unknown field "xx…" (1000000 bytes)`. +func unknownNameText(path, name string) string { + text := "unknown field " + strconv.Quote(path+errtext.Cut(name, maxUnknownNameBytes)) + if len(name) > maxUnknownNameBytes { + text += " (" + strconv.Itoa(len(name)) + " bytes)" + } + return text } // nullMembers returns body with every value at the dotted path field diff --git a/internal/api/handlers_write_refs.go b/internal/api/handlers_write_refs.go index f04e92f5..1599ec8c 100644 --- a/internal/api/handlers_write_refs.go +++ b/internal/api/handlers_write_refs.go @@ -1000,7 +1000,8 @@ func strictJSON(body []byte, dst any) error { // that JSON escapes to at most 192: at most 105,203 bytes and the request // path in `instance` (TestStrictBodiesBoundTheNamesTheyRepeat). No field // name in upstream's spec has more than 25 bytes. An answer that names -// one member keeps its whole name, as before (errtext.MaxBytes cuts it). +// one unknown member cuts its name at maxUnknownNameBytes, as the other +// decoders do (unknownNameText). const maxListedNameBytes = 32 // checkStrictObject verifies raw is a JSON object whose member names are @@ -1050,9 +1051,11 @@ type strictProblem struct { kind strictKind } -// err returns the problem as an error. cut cuts an unknown member name at -// maxListedNameBytes: the other kinds name a field of the struct. -func (p strictProblem) err(cut bool) error { +// err returns the problem as an error. An unknown member name, which is +// request text, is cut: at maxListedNameBytes when listed (the answer +// names several problems), else as unknownNameText cuts it. The other +// kinds name a field of the struct. +func (p strictProblem) err(listed bool) error { path := p.path + p.name switch p.kind { case strictRepeated: @@ -1062,10 +1065,12 @@ func (p strictProblem) err(cut bool) error { case strictNotObject: return newFieldError(path, "must be an object", path+" must be a JSON object") } - if cut { + path = p.path + errtext.Cut(p.name, maxUnknownNameBytes) + msg := unknownNameText(p.path, p.name) + if listed { path = p.path + errtext.Cut(p.name, maxListedNameBytes) + msg = fmt.Sprintf("unknown field %q", path) } - msg := fmt.Sprintf("unknown field %q", path) if p.name == "" { // an empty member name has no dotted wire path return errors.New(msg) } diff --git a/internal/api/invalid_params_notes_test.go b/internal/api/invalid_params_notes_test.go index 2f5c03f9..a649447a 100644 --- a/internal/api/invalid_params_notes_test.go +++ b/internal/api/invalid_params_notes_test.go @@ -388,8 +388,8 @@ func TestStrictBodiesNameEveryBadMember(t *testing.T) { // bytes of each, so it has at most 105,203 bytes and the request path: 100 // items, each with two messages in `error` and in `detail` (a name in // author and the same name at the top level as author.). An answer -// that names one member keeps its name up to 16384 bytes -// (TestLongValuesAreCutInAnswers). A name that an earlier item shows after +// that names one unknown member cuts its name at 256 bytes +// (TestInvalidParamsOnBodyFailures). A name that an earlier item shows after // the cut gives no item of its own and counts in `and N more`. Each object // (all copies of author are one) keeps 100 unknown names, and `and N more` // counts each member that the answer leaves out once: a repeat, or a member diff --git a/internal/api/problem_test.go b/internal/api/problem_test.go index 5a24c973..6115ea9a 100644 --- a/internal/api/problem_test.go +++ b/internal/api/problem_test.go @@ -1299,7 +1299,9 @@ func testInvalidParamsOnBodyFailures(t *testing.T, e *testEnv) { type param struct{ name, reason string } // The answer cuts an unknown member name of more than 256 bytes to at // most 256, at a character, and states its length, so it stays small - // whatever escapes the name needs. + // whatever escapes the name needs: under 5 KiB when a strict route + // repeats a name of `<`, six bytes each in JSON, in `error`, `detail` + // and the `invalid_params` name. long := func(n int, s string) string { return strings.Repeat(s, n) } for _, tc := range []struct { name, method, path, token string @@ -1328,14 +1330,22 @@ func testInvalidParamsOnBodyFailures(t *testing.T, e *testEnv) { `invalid JSON body: unknown field "` + long(256, "<") + `…" (1000000 bytes)`, nil}, {"upstream unknown field cut with a quote and a control character", http.MethodPost, "/api/v1/repos", freshTok, `{"upstream":{"\"\u0001` + long(300, "y") + `":1}}`, `invalid upstream: unknown field "\"\x01` + long(254, "y") + `…" (302 bytes)`, nil}, + {"strict unknown field of 256 bytes", http.MethodPost, "/api/repos/" + url + "/notes/append", gitTok, `{"` + long(256, "y") + `":1}`, + `invalid JSON body: unknown field "` + long(256, "y") + `"`, ¶m{long(256, "y"), "is not a known field"}}, + {"strict unknown field of escaped bytes", http.MethodPost, "/api/repos/" + url + "/notes/append", gitTok, `{"` + long(1_000_000, "<") + `":1}`, + `invalid JSON body: unknown field "` + long(256, "<") + `…" (1000000 bytes)`, ¶m{long(256, "<") + "…", "is not a known field"}}, + {"strict unknown field in author cut at a character", http.MethodPost, "/api/repos/" + url + "/notes/append", gitTok, `{"author":{"a` + long(400_000, "é") + `":1}}`, + `invalid JSON body: unknown field "author.a` + long(127, "é") + `…" (800001 bytes)`, ¶m{"author.a" + long(127, "é") + "…", "is not a known field"}}, + {"strict pull unknown field cut with a quote and a control character", http.MethodPost, "/api/repos/" + url + "/upstream/pull", gitTok, `{"\"\u0001` + long(300, "y") + `":1}`, + `invalid JSON body: unknown field "\"\x01` + long(254, "y") + `…" (302 bytes)`, ¶m{"\"\x01" + long(254, "y") + "…", "is not a known field"}}, {"strict empty member name", http.MethodPost, "/api/repos/" + url + "/notes/append", gitTok, `{"":1}`, `invalid JSON body: unknown field ""`, nil}, {"syntax error", http.MethodPatch, "/api/v1/repo", repoTok, `{`, "invalid JSON body", nil}, {"wrong top-level type", http.MethodPatch, "/api/v1/repo", repoTok, `[]`, "invalid JSON body", nil}, } { t.Run(tc.name, func(t *testing.T) { resp, body := e.doReq(t, tc.method, tc.path, tc.token, []byte(tc.body), "") - if resp.StatusCode != http.StatusBadRequest || len(body) > 4<<10 { - t.Fatalf("status = %d, want 400 of at most 4 KiB (body of %d bytes %.300s)", resp.StatusCode, len(body), body) + if resp.StatusCode != http.StatusBadRequest || len(body) > 5<<10 { + t.Fatalf("status = %d, want 400 of at most 5 KiB (body of %d bytes %.300s)", resp.StatusCode, len(body), body) } requireEnvelope(t, resp, body) var p problemBody diff --git a/internal/errtext/README.md b/internal/errtext/README.md index 182fdd23..11706cd1 100644 --- a/internal/errtext/README.md +++ b/internal/errtext/README.md @@ -14,7 +14,7 @@ Bounds the request text that an error answer repeats, so that an answer stays sm ## Callers - `api` cuts `error`, `detail` and each `invalid_params` name of a problem body (`newProblem`), and the `result.message` of a commit write (`wcRefUpdate`) [../api/errors.go:newProblem] [../api/handlers_write_commit.go:wcRefUpdate] -- `api` cuts each unknown member name at 32 bytes in the answer of a strictly decoded body that names several members (`Cut`) [../api/handlers_write_refs.go:strictProblem.err] +- `api` cuts an unknown member name at 256 bytes in an answer that names one unknown member (`unknownNameText`), and each unknown member name at 32 bytes in the answer of a strictly decoded body that names several members (`Cut`) [../api/decode_body.go:unknownNameText] [../api/handlers_write_refs.go:strictProblem.err] - `githttp` cuts the error text of the git and LFS routes [../githttp/server.go:httpError] [../githttp/lfs.go:lfsError] ## Verification