Skip to content

fix(api): cut the one unknown name that a strict body repeats - #895

Merged
ThomasK33 merged 2 commits into
mainfrom
strict-unknown-name-cut
Nov 16, 2026
Merged

ThomasK33 merged 2 commits into
mainfrom
strict-unknown-name-cut

Conversation

@ThomasK33

Copy link
Copy Markdown
Member

Why

open.storage error answers must name each invalid field of a request body (upstream's spec, docs/upstream/openapi.yaml: invalid_params "names each invalid field") and must never repeat a large piece of request text: an answer must stay small, a few tens of KiB, whatever the request holds.

Since #886, an answer from the two strictly decoded bodies (POST /api/repos/{repo_name}/notes/append and POST /api/repos/{repo_name}/upstream/pull) cuts each unknown member name at 32 bytes when it names several members. An answer that names one unknown member still repeated the whole name, up to #833's cut of 16384 bytes, three times: in error, in detail and in the invalid_params name. JSON escapes < to six bytes, so on main (dd57090) a 1 MiB name of < answers 294,797 bytes, and 150,000 escaped 0x01 bytes answer 139,473 bytes.

What changed

  • The strict decoder now cuts the one unknown name in its answer as fix(api): cut a long unknown member name in a decode error answer #882 cuts the unknown name of the other decoders. A name of up to 256 bytes is unchanged. A longer name is cut at a character to at most 256 bytes and ends with …, and error and detail add the length of the name in bytes: invalid JSON body: unknown field "author.xx…" (1000000 bytes). The invalid_params name ends with … only (author.xx…).
  • One helper, unknownNameText (internal/api/decode_body.go), now holds this rule for both decoders. It cuts with errtext.Cut and replaces the inline cut loop of unknownFieldText. strictProblem.err (internal/api/handlers_write_refs.go) calls it for an answer that names one problem and keeps the 32-byte cut for an answer that names several.
  • This is a deliberate change for one case: a strictly decoded body whose answer names one unknown member of more than 256 bytes. The worst such answer has 4,923 bytes and the request path in instance (4,958 bytes with a path of 35 bytes) instead of 294,798. No other answer changes.
  • Docs: docs/reference/http-api.md replaces "An answer that names one member keeps its name up to the cut of 16384 bytes" with the new rule and its bound. The strict decoding cell of docs/explanation/wire-compatibility.md states the cut too, because the name of such an item is no longer the whole wire name. The internal/errtext README lists the new caller.

Scope

Design calls

  • The invalid_params name ends with … only, without (N bytes). In every other item, name is a dotted wire path, and the 32-byte cut of fix(api): name every bad member of a strictly decoded body #886 ends the name with … only, so both cuts give the same shape. The length is in error and detail. Option not taken: add (N bytes) to the name too. That would put text that is not a path into name and would differ from the 32-byte items.
  • Cut at 256 bytes, the rule of fix(api): cut a long unknown member name in a decode error answer #882, not at 32 bytes. A name of up to 256 bytes answers as on main, and a misspelled name stays whole (no field name of upstream's spec has more than 25 bytes). Option not taken: use the 32-byte cut of fix(api): name every bad member of a strictly decoded body #886 for one member too. That would also change the answers for names of 33 to 256 bytes, and it would show less of the name than the other decoders show.
  • N counts the member name, without the author. prefix. This is the text that the request sent and that the cut measures, so the suffix appears exactly when N is above 256, as on the other decoders. Option not taken: count the whole dotted path. Then a name of 250 bytes in author (257 bytes of path) would show no suffix but would have a path longer than 256.
  • One shared helper over errtext.Cut, not a third copy of the cut. unknownFieldText keeps the decoder's own quoting for a name of up to 256 bytes, so its answers do not change. Option not taken: a cut written only for the strict decoder, next to errtext.Cut and the loop of unknownFieldText.

Verification

  • Reproduced on main (dd57090) through the server (SQLite): a 1 MiB name of < answers 294,797 bytes on notes/append and 294,798 on upstream/pull. A 16 KiB name of < answers 294,794, and 150,000 escaped 0x01 bytes answer 139,473. On this branch the same requests answer 4,937, 4,938, 4,933 and 4,423 bytes. The worst, a 1 MiB name of < inside author, answers 4,958 bytes (294,692 on main), with a request path of 35 bytes.

  • New rows in TestInvalidParamsOnBodyFailures go through the server. Three are red on main (an overlay of the two production files): strict unknown field of escaped bytes (294,797 bytes), strict unknown field in author cut at a character (49,377 bytes) and strict pull unknown field cut with a quote and a control character (wrong message). strict unknown field of 256 bytes is green on both and pins the boundary. The size gate of the table moved from 4 KiB to 5 KiB, because a strict route repeats a 256-byte name three times.

  • Differential (go test -overlay with the two production files of origin/main; the head binary has api.unknownNameText and the main binary does not, md5 5d67648d… against 4a330083…): 3,000 generated requests. They covered notes/append, upstream/pull, branches/create and POST /repos with upstream, with names of 0 to 20,000 bytes of ASCII, <>&, multi-byte characters, escapes and invalid UTF-8, and with null, duplicate and not-object members and syntax errors. 2,359 answers were identical in status, Content-Type and body. All 641 that differ are strict answers that name one unknown member of more than 256 bytes. The largest is 4,046 bytes on this branch and 214,160 bytes on main. 0 rows broke this rule.

  • Gates: go build ./... passed. go vet ./internal/api/ ./internal/errtext/ passed. gofmt was clean. golangci-lint 2.15.0 found 0 issues. The SQLite race lane go test ./internal/api/... -tags=openstorage_viewcheck -count=1 -race -timeout=40m -skip '/^pg$|//^pg$' passed in 363 s, and go test ./internal/errtext/... passed. The docs gate passed: 0 broken links, 80 script tests OK. Both fingerprint gates passed (--fingerprints-only). PostgreSQL runs in CI.

  • This head merges main accb8c62 (fix(api): count the 404 for a long /healthz redirect under git #891): the conflict was the problem-details row (line 219) of wire-compatibility.md, resolved sentence by sentence. On this head the SQLite race lane of ./internal/api/... and ./internal/errtext/... passes (322.9 s), golangci-lint 2.15.0 finds 0 issues, the docs gate finds 0 broken links, and both fingerprint gates pass.


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: xhigh

An error answer must stay small whatever the request holds. Since #886
a strictly decoded body (POST .../notes/append, POST .../upstream/pull)
cuts each unknown member name at 32 bytes when its answer names several
members, but an answer that names one unknown member still repeated the
name in `error`, `detail` and the `invalid_params` name up to the 16384
bytes of #833. JSON escapes `<` to six bytes, so a 1 MiB name of `<`
answered 294,797 bytes.

The strict decoder now cuts that name as #882 cuts the unknown name of
the other decoders: a name of up to 256 bytes is unchanged, a longer
one is cut at a character to at most 256 bytes and ends with `…`, and
`error` and `detail` add the length of the name in bytes. The
`invalid_params` name ends with `…` only, as the 32-byte cut does. One
helper, unknownNameText, now holds that rule for both decoders and cuts
with errtext.Cut. The worst such answer has 4,923 bytes and the request
path.

_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `xhigh`_

Change-Id: Iea7ad210765b91e338a33bcce392577cc1c18c5c
Signed-off-by: Thomas Kosiewski <tk@coder.com>
Change-Id: I1dbdf2e109c3de10bb82262c68168a3b573b4969
Signed-off-by: Thomas Kosiewski <tk@coder.com>
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-16T17:23:49.432161Z 42ca0f4 Manual request
🔒 Security Review ✅ Completed 2026-11-16T17:25:16.005106Z 42ca0f4 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🎉

Reviewed commit: 42ca0f428c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: 42ca0f428c

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ThomasK33
ThomasK33 added this pull request to the merge queue Nov 16, 2026
Merged via the queue into main with commit dc6ef0c Nov 16, 2026
15 checks passed
@ThomasK33
ThomasK33 deleted the strict-unknown-name-cut branch November 16, 2026 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant