Repository navigation
fix(api): cut the one unknown name that a strict body repeats - #895
Conversation
An error answer must stay small whatever the request holds. Since #886 a strictly decoded body (POST .../notes/append, POST .../upstream/pull) cuts each unknown member name at 32 bytes when its answer names several members, but an answer that names one unknown member still repeated the name in `error`, `detail` and the `invalid_params` name up to the 16384 bytes of #833. JSON escapes `<` to six bytes, so a 1 MiB name of `<` answered 294,797 bytes. The strict decoder now cuts that name as #882 cuts the unknown name of the other decoders: a name of up to 256 bytes is unchanged, a longer one is cut at a character to at most 256 bytes and ends with `…`, and `error` and `detail` add the length of the name in bytes. The `invalid_params` name ends with `…` only, as the 32-byte cut does. One helper, unknownNameText, now holds that rule for both decoders and cuts with errtext.Cut. The worst such answer has 4,923 bytes and the request path. _Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `xhigh`_ Change-Id: Iea7ad210765b91e338a33bcce392577cc1c18c5c Signed-off-by: Thomas Kosiewski <tk@coder.com>
Change-Id: I1dbdf2e109c3de10bb82262c68168a3b573b4969 Signed-off-by: Thomas Kosiewski <tk@coder.com>
|
@codex review |
|
@codex security review |
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex Review: Didn't find any major issues. 🎉 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
🛡️ Codex Security ReviewSecurity review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
Why
open.storage error answers must name each invalid field of a request body (upstream's spec,
docs/upstream/openapi.yaml:invalid_params"names each invalid field") and must never repeat a large piece of request text: an answer must stay small, a few tens of KiB, whatever the request holds.Since #886, an answer from the two strictly decoded bodies (
POST /api/repos/{repo_name}/notes/appendandPOST /api/repos/{repo_name}/upstream/pull) cuts each unknown member name at 32 bytes when it names several members. An answer that names one unknown member still repeated the whole name, up to #833's cut of 16384 bytes, three times: inerror, indetailand in theinvalid_paramsname. JSON escapes<to six bytes, so onmain(dd57090) a 1 MiB name of<answers 294,797 bytes, and 150,000 escaped 0x01 bytes answer 139,473 bytes.What changed
…, anderroranddetailadd the length of the name in bytes:invalid JSON body: unknown field "author.xx…" (1000000 bytes). Theinvalid_paramsname ends with…only (author.xx…).unknownNameText(internal/api/decode_body.go), now holds this rule for both decoders. It cuts witherrtext.Cutand replaces the inline cut loop ofunknownFieldText.strictProblem.err(internal/api/handlers_write_refs.go) calls it for an answer that names one problem and keeps the 32-byte cut for an answer that names several.instance(4,958 bytes with a path of 35 bytes) instead of 294,798. No other answer changes.docs/reference/http-api.mdreplaces "An answer that names one member keeps its name up to the cut of 16384 bytes" with the new rule and its bound. The strict decoding cell ofdocs/explanation/wire-compatibility.mdstates the cut too, because thenameof such an item is no longer the whole wire name. Theinternal/errtextREADME lists the new caller.Scope
instancecut, the 100-item bound and the body limit.Design calls
invalid_paramsname ends with…only, without(N bytes). In every other item,nameis a dotted wire path, and the 32-byte cut of fix(api): name every bad member of a strictly decoded body #886 ends the name with…only, so both cuts give the same shape. The length is inerroranddetail. Option not taken: add(N bytes)to the name too. That would put text that is not a path intonameand would differ from the 32-byte items.main, and a misspelled name stays whole (no field name of upstream's spec has more than 25 bytes). Option not taken: use the 32-byte cut of fix(api): name every bad member of a strictly decoded body #886 for one member too. That would also change the answers for names of 33 to 256 bytes, and it would show less of the name than the other decoders show.Ncounts the member name, without theauthor.prefix. This is the text that the request sent and that the cut measures, so the suffix appears exactly whenNis above 256, as on the other decoders. Option not taken: count the whole dotted path. Then a name of 250 bytes inauthor(257 bytes of path) would show no suffix but would have a path longer than 256.errtext.Cut, not a third copy of the cut.unknownFieldTextkeeps the decoder's own quoting for a name of up to 256 bytes, so its answers do not change. Option not taken: a cut written only for the strict decoder, next toerrtext.Cutand the loop ofunknownFieldText.Verification
Reproduced on
main(dd57090) through the server (SQLite): a 1 MiB name of<answers 294,797 bytes onnotes/appendand 294,798 onupstream/pull. A 16 KiB name of<answers 294,794, and 150,000 escaped 0x01 bytes answer 139,473. On this branch the same requests answer 4,937, 4,938, 4,933 and 4,423 bytes. The worst, a 1 MiB name of<insideauthor, answers 4,958 bytes (294,692 onmain), with a request path of 35 bytes.New rows in
TestInvalidParamsOnBodyFailuresgo through the server. Three are red onmain(an overlay of the two production files):strict unknown field of escaped bytes(294,797 bytes),strict unknown field in author cut at a character(49,377 bytes) andstrict pull unknown field cut with a quote and a control character(wrong message).strict unknown field of 256 bytesis green on both and pins the boundary. The size gate of the table moved from 4 KiB to 5 KiB, because a strict route repeats a 256-byte name three times.Differential (
go test -overlaywith the two production files oforigin/main; the head binary hasapi.unknownNameTextand the main binary does not, md55d67648d…against4a330083…): 3,000 generated requests. They coverednotes/append,upstream/pull,branches/createandPOST /reposwithupstream, with names of 0 to 20,000 bytes of ASCII,<>&, multi-byte characters, escapes and invalid UTF-8, and with null, duplicate and not-object members and syntax errors. 2,359 answers were identical in status, Content-Type and body. All 641 that differ are strict answers that name one unknown member of more than 256 bytes. The largest is 4,046 bytes on this branch and 214,160 bytes onmain. 0 rows broke this rule.Gates:
go build ./...passed.go vet ./internal/api/ ./internal/errtext/passed. gofmt was clean. golangci-lint 2.15.0 found 0 issues. The SQLite race lanego test ./internal/api/... -tags=openstorage_viewcheck -count=1 -race -timeout=40m -skip '/^pg$|//^pg$'passed in 363 s, andgo test ./internal/errtext/...passed. The docs gate passed: 0 broken links, 80 script tests OK. Both fingerprint gates passed (--fingerprints-only). PostgreSQL runs in CI.This head merges main
accb8c62(fix(api): count the 404 for a long /healthz redirect under git #891): the conflict was the problem-details row (line 219) ofwire-compatibility.md, resolved sentence by sentence. On this head the SQLite race lane of./internal/api/...and./internal/errtext/...passes (322.9 s), golangci-lint 2.15.0 finds 0 issues, the docs gate finds 0 broken links, and both fingerprint gates pass.Generated with
xum• Model:anthropic:claude-opus-5-5• Thinking:xhigh