diff --git a/.changeset/03-review-quality-and-provenance.md b/.changeset/03-review-quality-and-provenance.md index a69c6b69a..ae29b8904 100644 --- a/.changeset/03-review-quality-and-provenance.md +++ b/.changeset/03-review-quality-and-provenance.md @@ -25,3 +25,7 @@ - **Validation pass** — a trusted `prompts/validate.md` pass writes `validation.json` per sig, `rigor_policy` floors each severity tier, and `post --require-rigor` narrows it per invocation: advisory by default, a receipted withhold in gate mode. [Rigor floors](docs/manual.md#rigor-floors-and-the-validation-pass) - **Published key names** — `prompts/review.md` tags the evidence rung as `rigor` and the in-bounds flag as `in-bounds`, and an archtest holds every published key to `review.schema.json`. [Rigor floors](docs/manual.md#rigor-floors-and-the-validation-pass) - **Verdict reconciliation** — binding verdicts reconcile separately from summary and inline content, so a retried approval does not duplicate a GitLab summary note. [Submit per host](docs/manual.md#submit-post-per-host) +- **Plain-body Basis line** — `record` leaves a `review.json` body with no leading quote block byte-identical and reports no `basis-replaced` warning, because there is no confidence lead to insert under. [Review-input problems](docs/manual.md#review-input-problems) +- **Array-valued coverage ranges** — `next --coverage` refuses a `coverage[].ranges` array on plain and chunked bundles with the same `bad-review-input` problem `record` reports, instead of reading it as a silent 0% attestation. [Diff coverage](docs/manual.md#diff-coverage-agent-attestation) +- **Passing gate next step** — a passing `report --check ` names `merge — gate passes` in the envelope's `next` and human footer, leaving `address open threads` for a failing gate. [Machine surface](docs/manual.md#machine-surface) +- **More generated paths** — `*.gen.*` and `zz_generated.*` now count as generated in the shared role heuristic, so `next --coverage` and the risk floor exempt them like the other generated patterns. [Diff coverage](docs/manual.md#diff-coverage-agent-attestation) diff --git a/.changeset/11-fixes.md b/.changeset/11-fixes.md index ae670180d..d96a38968 100644 --- a/.changeset/11-fixes.md +++ b/.changeset/11-fixes.md @@ -15,3 +15,5 @@ - **`pr-wait` inconsistent reads** — a head that is not yet the same on both APIs is retried instead of failing the bounded wait, and `inconsistent_observations` keeps the retry visible. [Bounded foreground waits](docs/manual.md#bounded-foreground-waits) - **Windows watchdog resume** — a command that failed before starting under load retries the thread lookup instead of refusing `resume suspended verify process`. [Bounded foreground waits](docs/manual.md#bounded-foreground-waits) - **`backtest mine` in a partial clone** — a `--filter=blob:none` clone refuses `partial-clone` with the step that fetches the missing objects, instead of stalling. [Backtest](docs/manual.md#backtest-szz-candidates) +- **Quiet local live skip** — a local bundle's live recheck writes one `info:` line to the human stream instead of a warning or a `--json` `warnings[]` entry; failed remote reads still warn. [Machine surface](docs/manual.md#machine-surface) +- **Optional review schema version** — `record` accepts a `review.json` without `schema_version` and reads it as version 1, while a present value other than `"1"` still refuses. [Review-input problems](docs/manual.md#review-input-problems) diff --git a/apps/docs/pages/_home/HowItWorks.astro b/apps/docs/pages/_home/HowItWorks.astro index 729835edc..99131e02c 100644 --- a/apps/docs/pages/_home/HowItWorks.astro +++ b/apps/docs/pages/_home/HowItWorks.astro @@ -24,8 +24,8 @@ const sample = [ { kind: "out", text: " 878a93e0 open true src/retry.ts" }, { kind: "cmd", text: "verdict report acme/app local-work --check important" }, { - kind: "dim", - text: "warn: live recheck skipped: local bundle threads unreadable here (gating on local dimensions only)", + kind: "out", + text: "info: live recheck skipped: local bundle threads unreadable here (gating on local dimensions only)", }, { kind: "out", text: "report: .verdict/state/local-work (1 shown) open=1 head=" }, { kind: "out", text: "1. [open] src/retry.ts:3 (878a93e0)" }, diff --git a/cmd/verdict/stdout_contract_test.go b/cmd/verdict/stdout_contract_test.go new file mode 100644 index 000000000..dc6c70f3c --- /dev/null +++ b/cmd/verdict/stdout_contract_test.go @@ -0,0 +1,91 @@ +package main + +import ( + "encoding/json" + "errors" + "io" + "os" + "os/exec" + "path/filepath" + "strings" + "testing" +) + +func TestJSONVerbsWriteOneEnvelopeToStdout(t *testing.T) { + hermeticEnv(t) + root := t.TempDir() + t.Chdir(root) + base, head := gitRangeFixture(t, root) + if got := runOne(t, "fetch-local", "acme/app", "stdout", "--range", base+".."+head, "--json"); got.code != 0 { + t.Fatalf("fetch-local exit = %d\nstdout: %s\nstderr: %s", got.code, got.out, got.err) + } + writeReviewJSON(t, ".verdict/state/local-stdout/review.json") + for _, args := range [][]string{ + {"next", "acme/app", "local-stdout", "--coverage", "--json"}, + {"record", "acme/app", "local-stdout", "--json"}, + {"report", "acme/app", "local-stdout", "--check", "important", "--max-risk", "medium", "--json"}, + } { + got := runOne(t, args...) + if got.code != 0 { + t.Fatalf("%v exit = %d\nstderr: %s", args, got.code, got.err) + } + assertOneJSONValue(t, args[0], got.out) + } +} + +func assertOneJSONValue(t *testing.T, verb, out string) { + t.Helper() + dec := json.NewDecoder(strings.NewReader(out)) + var first any + if err := dec.Decode(&first); err != nil { + t.Fatalf("%s stdout is not a JSON value: %v\n%s", verb, err, out) + } + var extra any + if err := dec.Decode(&extra); !errors.Is(err, io.EOF) { + t.Fatalf("%s stdout carries more than one JSON value: %v\n%s", verb, err, out) + } +} + +func writeReviewJSON(t *testing.T, path string) { + t.Helper() + body := `{"schema_version":"1","body":"## Summary\nA clean review body with enough text and no quote lead.","approved":false,"comments":[]}` + if err := os.WriteFile(path, []byte(body), 0o644); err != nil { + t.Fatal(err) + } +} + +func gitRangeFixture(t *testing.T, dir string) (base, head string) { + t.Helper() + git := func(args ...string) string { + cmd := exec.CommandContext(t.Context(), "git", args...) + cmd.Dir = dir + cmd.Env = append(os.Environ(), "GIT_CONFIG_NOSYSTEM=1") + out, err := cmd.CombinedOutput() + if err != nil { + t.Fatalf("git %v: %v\n%s", args, err, out) + } + return string(out) + } + git("init", "-q") + notes := filepath.Join(dir, "notes.txt") + if err := os.WriteFile(notes, []byte("one\ntwo\nthree\n"), 0o644); err != nil { + t.Fatal(err) + } + git("add", "notes.txt") + git("-c", "user.name=t", "-c", "user.email=t@example.com", "commit", "-qm", "one") + base = git("rev-parse", "HEAD") + if err := os.WriteFile(notes, []byte("one\nTWO\nthree\n"), 0o644); err != nil { + t.Fatal(err) + } + git("add", "notes.txt") + git("-c", "user.name=t", "-c", "user.email=t@example.com", "commit", "-qm", "two") + head = git("rev-parse", "HEAD") + return trimNewline(base), trimNewline(head) +} + +func trimNewline(s string) string { + for len(s) > 0 && (s[len(s)-1] == '\n' || s[len(s)-1] == '\r') { + s = s[:len(s)-1] + } + return s +} diff --git a/docs/manual.md b/docs/manual.md index fe27e5b47..29ec74efa 100644 --- a/docs/manual.md +++ b/docs/manual.md @@ -239,28 +239,30 @@ an unparsable or non-object document, a comment with no recoverable string coverage still refuses. `record --dry-run` adds `data.normalized` (the exact object the checker read) and `data.repairs`; verdict never rewrites `review.json` on disk. The CLI also owns the body's `> Basis:` line: it -replaces a disagreeing line, or inserts a missing one, with the counts computed -from the review and reports `basis-replaced`. - -| Code | Level | Meaning | -| ---------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------ | -| `invalid-json` | error | The file is not one JSON document; the message carries line:col, and the fix is to emit only the object. | -| `wrong-type` | mixed | Error for a wrong type; warning for a non-numeric line/start_line or a fenced suggestion. | -| `missing-field` | mixed | Error for a required field; warning for missing schema_version, comments, or line/start_line. | -| `bad-enum` | mixed | Error for an unknown closed-set value; warning for a malformed KB id or unknown chunk name. | -| `out-of-range` | error | A length or numeric bound is outside its published range (confidence, brief approach/question, caps). | -| `path-not-in-diff` | mixed | Error for a comment path; warning for a brief site outside the diff. | -| `unknown-field` | warning | A key is not in the published shape; the fix names the nearest known key. | -| `risk-stale-head` | warning | `review.json.risk.assessed_head` disagrees with the bundle's current HEAD SHA; the stored claim is marked `stale: true`. | -| `anchor-outside-hunk` | warning | The line is provably outside every hunk; `record` moves the comment into the body. | -| `rigor-unset` | warning | A comment carries no evidence rung; it records as `said` with `rigor_source: default`. | -| `normalized-fence` | warning | One whole-document code fence was unwrapped. | -| `normalized-extracted` | warning | Prose around exactly one balanced JSON object was dropped. | -| `scalar-to-array` | warning | A scalar where a string array belongs was wrapped in a one-element array. | -| `key-alias` | warning | A known alias (`inBounds`, `in_bounds`, `rung`, `file`) was renamed to its canonical key. | -| `enum-synonym` | warning | A rigor synonym or a case-folded enum value was rewritten to the published value. | -| `normalized-trivia` | warning | A BOM, an explicit null on an optional key, or an exact-duplicate comment was removed. | -| `basis-replaced` | warning | The body's `> Basis:` line disagreed with the computed counts, or was missing from the leading quote block, and the CLI replaced it. | +replaces a disagreeing line, or inserts a missing one under the leading quote +block's confidence lead, with the counts computed from the review and reports +`basis-replaced`. A body with no leading quote block has no lead to insert +under, so it stays byte-identical and reports nothing. + +| Code | Level | Meaning | +| ---------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------- | +| `invalid-json` | error | The file is not one JSON document; the message carries line:col, and the fix is to emit only the object. | +| `wrong-type` | mixed | Error for a wrong type; warning for a non-numeric line/start_line or a fenced suggestion. | +| `missing-field` | mixed | Error for a required field; warning for missing comments or line/start_line. | +| `bad-enum` | mixed | Error for an unknown closed-set value; warning for a malformed KB id or unknown chunk name. | +| `out-of-range` | error | A length or numeric bound is outside its published range (confidence, brief approach/question, caps). | +| `path-not-in-diff` | mixed | Error for a comment path; warning for a brief site outside the diff. | +| `unknown-field` | warning | A key is not in the published shape; the fix names the nearest known key. | +| `risk-stale-head` | warning | `review.json.risk.assessed_head` disagrees with the bundle's current HEAD SHA; the stored claim is marked `stale: true`. | +| `anchor-outside-hunk` | warning | The line is provably outside every hunk; `record` moves the comment into the body. | +| `rigor-unset` | warning | A comment carries no evidence rung; it records as `said` with `rigor_source: default`. | +| `normalized-fence` | warning | One whole-document code fence was unwrapped. | +| `normalized-extracted` | warning | Prose around exactly one balanced JSON object was dropped. | +| `scalar-to-array` | warning | A scalar where a string array belongs was wrapped in a one-element array. | +| `key-alias` | warning | A known alias (`inBounds`, `in_bounds`, `rung`, `file`) was renamed to its canonical key. | +| `enum-synonym` | warning | A rigor synonym or a case-folded enum value was rewritten to the published value. | +| `normalized-trivia` | warning | A BOM, an explicit null on an optional key, or an exact-duplicate comment was removed. | +| `basis-replaced` | warning | The body's `> Basis:` line disagreed with the computed counts and was replaced, or was missing from the leading quote block and was inserted. | ### Sig contract (sig_version 1) @@ -277,8 +279,9 @@ previous merge filled a missing rung with `pointed`, and every other recognized reads as `model`. A review that explicitly wrote `pointed` is indistinguishable from a default-filled one, so those entries read as `default` either way. Changing the normalization orphans history — that ships as a new `sig_version`, never silently. -`review.json` inputs stamp `schema_version: "1"` (`post`/`record` refuse unknown -versions, warn-and-proceed when the stamp is missing); every bundle carries +`review.json` inputs stamp `schema_version: "1"` (`post`/`record` refuse a +present value other than `"1"`; an absent key reads as 1 without a warning); +every bundle carries `.verdict/state/N/store.json` (`{"schema_version":"1"}`) marking the generation of the whole store — plugins read it instead of sniffing shapes. GitHub bundles keep `PR.json`/`reviews.json`/`inline.json` as raw API output (GitHub's schema, @@ -1075,14 +1078,18 @@ the bundle's own hunk snapshot: a comma-separated list of 1-based inclusive source-line intervals, and it is side-agnostic — the evaluator unions the file's `OLD` and `NEW` hunk intervals, so one interval covers a hunk on either side. Duplicate `file` entries union -rather than count twice. A malformed range warns and makes that entry count as -unattested; it never silently attests the file. +rather than count twice. A malformed range string warns and makes that entry +count as unattested; it never silently attests the file. A `ranges` value that +is neither a string nor `null` — an array, number, or object — refuses +`bad-review-input` naming `$.coverage[N].ranges` on `record`, `post`, and +`next --coverage`, so it can never measure as an unattested 0%. Coverage is computed from `hunks.tsv`, the snapshot `fetch` already writes — never from a second diff parse and never from `toc.txt` line counts. Paths the shared role heuristic classifies as generated or vendored are exempt -automatically; no sentinel is required. `percent` is `null`, never `0`, when no -changed line was measured. +automatically — `*.pb.go`, `*.gen.*`, `zz_generated.*`, lockfiles, minified +assets, and `dist/`, `build/`, or `vendor/` trees — so no sentinel is required. +`percent` is `null`, never `0`, when no changed line was measured. The default posture is a warning plus one machine-built body line on a partial attestation. A partial one warns once and adds one machine-built body line; @@ -1344,7 +1351,9 @@ esac Compose it after the expensive step: gate the merge on `--check critical`, keep `post`'s preflight as the fail-closed backstop — the gate advises, -the human still merges. +the human still merges. A passing gate names that merge in the envelope's +`next` (`merge — gate passes`) and in the human footer, so open findings below +the tier's floor never read as the next step. `report --category ` filters the ranked findings to one finding kind (`logic`, `edge-case`, `security`, `performance`, `accidental-commit`, @@ -1380,11 +1389,12 @@ a read that stopped before the host's last page, both leave the surface unread rather than clean: each warns (`live thread recheck unreachable`) and, under the default `--live auto` posture, leaves the dimension neutral: the gate reflects local dimensions only, and the run never exits 3 over the -network. Only local bundles skip the lookup with a warn (`live recheck -skipped: ...`), paying zero network; plain `report` without a gate flag -never calls the network. There is no `--force` on report: fetch + triage + -re-check is the recovery, and advisory `report` without `--check` is the -escape hatch. +network. Only local bundles skip the lookup, paying zero network; they say +so with an `info: live recheck skipped: ...` line on the human stream, never +a warning or a `--json` `warnings[]` entry. Plain `report` without a gate +flag never calls the network. There is no `--force` on report: fetch + +triage + re-check is the recovery, and advisory `report` without `--check` +is the escape hatch. Offline posture (`--live auto|strict|skip`, gate runs only) is always an explicit operator choice — automation never inherits it silently. `auto` diff --git a/evals/cases/live-offline-contract.sh b/evals/cases/live-offline-contract.sh index 8a9fcf06c..5d583189c 100755 --- a/evals/cases/live-offline-contract.sh +++ b/evals/cases/live-offline-contract.sh @@ -138,7 +138,8 @@ if [ "$EVAL_STATUS" -eq 0 ]; then else bad "strict local exits $EVAL_STATUS, want 0" fi -assert_file_grep "local skip warns structurally" "$EVAL_ERR" "live recheck skipped" +assert_file_grep "local skip notes the structural skip on the human stream" "$EVAL_OUT" "info: live recheck skipped" +assert_stream_nogrep "local skip adds no structural warning" "$EVAL_ERR" "live recheck skipped" assert_stream_nogrep "local skip prints no fail-closed line" "$EVAL_ERR" "offline posture is strict" assert_eq "local skip pays zero graphql calls" "0" "$(graphql_since "$m")" diff --git a/evals/cases/live-recheck-report.sh b/evals/cases/live-recheck-report.sh index a65f442df..bcf263d5f 100755 --- a/evals/cases/live-recheck-report.sh +++ b/evals/cases/live-recheck-report.sh @@ -273,7 +273,8 @@ if [ "$EVAL_STATUS" -eq 0 ]; then else bad "local skip exits $EVAL_STATUS, want 0" fi -assert_file_grep "local run warn-skips" "$EVAL_ERR" "live recheck skipped" +assert_file_grep "local run notes the skip on the human stream" "$EVAL_OUT" "info: live recheck skipped" +assert_stream_nogrep "local run adds no skip warning" "$EVAL_ERR" "live recheck skipped" assert_stream_nogrep "local run prints no fetch-local noise" "$EVAL_ERR" "fetch-local" "-i" assert_stream_nogrep "local run prints no live open line" "$EVAL_ERR" "live open" assert_eq "local run makes zero recheck calls" "0" "$(graphql_since "$m")" diff --git a/internal/core/coverage/coverage.go b/internal/core/coverage/coverage.go index ea9080472..8c0b7c1ba 100644 --- a/internal/core/coverage/coverage.go +++ b/internal/core/coverage/coverage.go @@ -54,8 +54,14 @@ const ( // Codes and messages. const ( - // CodeIncomplete is the one refusal this package raises. + // CodeIncomplete is the coverage gate's refusal. CodeIncomplete = "coverage-incomplete" + // CodeBadReviewInput is the review-input refusal record's checker raises. + CodeBadReviewInput = "bad-review-input" + + msgReviewProblem = "review.json has %d problem(s); fix every one, then re-run" + msgRangesWrongType = "value must be a string" + msgRangesWrongTypeFix = `use "1-240,300-320", "all", or omit it` msgHunksUnreadable = "hunks.tsv unreadable: %v" msgEntryNotObject = "coverage: entry %d is not an object — ignored" @@ -148,13 +154,18 @@ type claim struct { // Evaluate reads the bundle's hunk snapshot and the review's attestation, and // returns the summary plus the warnings a caller should surface. A chunked -// bundle is not evaluated: chunks[] is its attestation, and one gate per bundle -// is the contract. +// bundle is not evaluated line by line: chunks[] is its attestation, and one +// gate per bundle is the contract. // // generated is the shared generated-path predicate; this package never owns a // second list. func Evaluate(dir string, review *jq.Obj, generated func(string) bool) (Summary, []string, *core.RefusalError) { if attested, total, ok := chunked(dir, review); ok { + var warns []string + _, _, problems := parseClaims(review, &warns) + if len(problems) > 0 { + return Summary{}, warns, badReviewInput(problems) + } return Summary{Chunked: true, Chunks: total, ChunksAttested: attested}, nil, nil } rows, ref := loadRows(dir) @@ -163,13 +174,24 @@ func Evaluate(dir string, review *jq.Obj, generated func(string) bool) (Summary, } files := intervalsByFile(rows) var warns []string - claims, present := parseClaims(review, &warns) + claims, present, problems := parseClaims(review, &warns) + if len(problems) > 0 { + return Summary{}, warns, badReviewInput(problems) + } warns = append(warns, unknownClaims(claims, files)...) sum := summarize(files, claims, generated) sum.Present = present return sum, warns, nil } +func badReviewInput(problems []core.Problem) *core.RefusalError { + return &core.RefusalError{ + Code: CodeBadReviewInput, + Message: fmt.Sprintf(msgReviewProblem, len(problems)), + Problems: problems, + } +} + // unknownClaims warns about entries naming a path the bundle has no hunks for: // a binary, rename-only, or entirely absent file attests nothing and must not // read as coverage. @@ -233,63 +255,85 @@ func attestation(files map[string][]Range, claims []claim) map[string][]Range { // parseClaims reads coverage[] and reports whether the key was present. Every // malformed entry warns; none of them silently attests a file. -func parseClaims(review *jq.Obj, warns *[]string) ([]claim, bool) { +func parseClaims(review *jq.Obj, warns *[]string) ([]claim, bool, []core.Problem) { if review == nil { - return nil, false + return nil, false, nil } raw, ok := review.Get(Field) if !ok { - return nil, false + return nil, false, nil } items, isArr := raw.([]any) if !isArr { *warns = append(*warns, msgNotArray) - return nil, true + return nil, true, nil } + out, problems := collectClaims(items, warns) + return out, true, problems +} + +func collectClaims(items []any, warns *[]string) ([]claim, []core.Problem) { out := make([]claim, 0, len(items)) + var problems []core.Problem for i, item := range items { - if c, ok := claimAt(item, i, warns); ok { + c, ok, p := claimAt(item, i, warns) + if p != nil { + problems = append(problems, *p) + } + if ok { out = append(out, c) } } - return out, true + return out, problems } // claimAt reads one coverage[] entry, warning and reporting false for an entry // that cannot name a file. A malformed range still yields a claim, marked bad, // so the merge warns about it and never counts it as attested. -func claimAt(item any, i int, warns *[]string) (claim, bool) { +func claimAt(item any, i int, warns *[]string) (claim, bool, *core.Problem) { obj, isObj := item.(*jq.Obj) if !isObj || obj == nil { *warns = append(*warns, fmt.Sprintf(msgEntryNotObject, i)) - return claim{}, false + return claim{}, false, nil } file := strings.TrimSpace(jq.Str(fieldOf(obj, FieldFile))) if file == "" { *warns = append(*warns, fmt.Sprintf(msgEntryNoFile, i)) - return claim{}, false + return claim{}, false, nil } - ranges, whole, bad := claimRanges(obj, file, warns) - return claim{file: file, ranges: ranges, whole: whole, bad: bad}, true + ranges, whole, bad, problem := claimRanges(obj, file, i, warns) + return claim{file: file, ranges: ranges, whole: whole, bad: bad}, true, problem } // claimRanges resolves one entry's ranges field: a whole-file claim, a parsed // range list, or a claim marked bad when the field is present but unreadable. -func claimRanges(obj *jq.Obj, file string, warns *[]string) ([]Range, bool, bool) { - ranges, ok := rangeField(obj) +func claimRanges(obj *jq.Obj, file string, i int, warns *[]string) ([]Range, bool, bool, *core.Problem) { + ranges, present, wrongType := rangeField(obj) switch { - case !ok: + case wrongType: + return nil, false, true, rangesWrongType(i) + case !present: *warns = append(*warns, fmt.Sprintf(msgBadRanges, file, jq.Str(mustField(obj, FieldRanges)))) - return nil, false, true + return nil, false, true, nil case ranges == "" || ranges == RangesAll: - return nil, true, false + return nil, true, false, nil default: parsed, err := ParseRanges(ranges) if err != nil { *warns = append(*warns, fmt.Sprintf(msgBadRanges, file, ranges)) - return nil, false, true + return nil, false, true, nil } - return parsed, false, false + return parsed, false, false, nil + } +} + +func rangesWrongType(i int) *core.Problem { + return &core.Problem{ + Path: fmt.Sprintf("$.coverage[%d].ranges", i), + Code: core.ProblemCodeWrongType, + Level: core.ProblemLevelError, + Message: msgRangesWrongType, + Fix: msgRangesWrongTypeFix, } } @@ -305,16 +349,19 @@ func mustField(o *jq.Obj, key string) any { return fieldOf(o, key) } // string — `null` included — is a malformed attestation, not a whole-file // claim: attesting more than was written is the one error this package must // never make, and a claim only widens by saying `all` or omitting the key. -func rangeField(o *jq.Obj) (string, bool) { +func rangeField(o *jq.Obj) (value string, present, wrongType bool) { raw, ok := o.Get(FieldRanges) if !ok { - return "", true + return "", true, false } s, isStr := raw.(string) if !isStr { - return "", false + if raw == nil { + return "", false, false + } + return "", true, true } - return strings.TrimSpace(s), true + return strings.TrimSpace(s), true, false } // loadRows reads the fetch-time snapshot, rebuilding it in memory from diff --git a/internal/core/coverage/coverage_test.go b/internal/core/coverage/coverage_test.go index 19ba1e03d..3de63ff3b 100644 --- a/internal/core/coverage/coverage_test.go +++ b/internal/core/coverage/coverage_test.go @@ -273,6 +273,25 @@ func TestChunkedSkips(t *testing.T) { assertChunkedSkipped(t, dir) } +func TestChunkedArrayRangesRefuse(t *testing.T) { + dir := writeBundle(t, fixtureRows()) + if err := mkdirAll(dir + "/chunks"); err != nil { + t.Fatal(err) + } + writeFile(t, dir+"/chunks/manifest.json", `{"schema_version":"1","chunk_count":1,"chunks":[{"name":"chunks/01.diff"}]}`) + writeFile(t, dir+"/chunks/01.diff", "diff") + _, _, ref := Evaluate(dir, mustReview(t, `{"coverage":[{"file":"src/db.ts","ranges":["1-2"]}]}`), notGenerated) + if ref == nil { + t.Fatal("Evaluate refused nothing; want bad-review-input for array ranges on a chunked bundle") + } + if ref.Code != "bad-review-input" { + t.Errorf("refusal code = %q, want bad-review-input", ref.Code) + } + if len(ref.Problems) != 1 || ref.Problems[0].Path != "$.coverage[0].ranges" || ref.Problems[0].Code != "wrong-type" { + t.Fatalf("problems = %+v, want one wrong-type at $.coverage[0].ranges", ref.Problems) + } +} + // TestChunkedCountsTheManifest proves the count comes from the manifest and not // from the files beside it: an extra diff and a missing one leave the count // alone, which is the number a receipt reports. diff --git a/internal/core/record/basis_test.go b/internal/core/record/basis_test.go new file mode 100644 index 000000000..9afc926ef --- /dev/null +++ b/internal/core/record/basis_test.go @@ -0,0 +1,25 @@ +package recordcore_test + +import ( + "context" + "strings" + "testing" + + recordcore "github.com/stainless-code/verdict/internal/core/record" +) + +func TestRecordPlainBodySkipsBasisWarning(t *testing.T) { + plain := `{"schema_version":"1","body":"A plain body with enough length to pass the guard, no quote lead.",` + + `"approved":false,"comments":[]}` + bundle(t, "local-plainbody", plain) + c := &collector{} + resp, ref := recordcore.Run(context.Background(), recordRequest(t, c, "local-plainbody")) + if ref != nil { + t.Fatalf("Run refusal: %v", ref) + } + for _, w := range resp.Warnings { + if strings.Contains(w, "Basis line") { + t.Errorf("warnings = %q, want no basis-insertion warning", resp.Warnings) + } + } +} diff --git a/internal/core/record/checkreview.go b/internal/core/record/checkreview.go index 0daa86111..cefae3a6f 100644 --- a/internal/core/record/checkreview.go +++ b/internal/core/record/checkreview.go @@ -248,11 +248,6 @@ func (c *checker) checkReview(obj *jq.Obj) { "write \"comments\": []", []any{})) } c.checkBodyQuality(obj) - if _, ok := obj.Get("schema_version"); !ok { - c.add(c.problem(problemSchemaVersion, core.ProblemCodeMissingField, core.ProblemLevelWarning, - "schema_version is missing; assuming 1", - "write \"schema_version\": \"1\"", schemaVersion1)) - } if c.hasChunks { if _, ok := obj.Get(fieldChunks); !ok { c.unattestedChunks() @@ -261,10 +256,7 @@ func (c *checker) checkReview(obj *jq.Obj) { } func (c *checker) checkSchemaVersion(v any) { - if v == nil || v == false { - c.add(c.problem(problemSchemaVersion, core.ProblemCodeMissingField, core.ProblemLevelWarning, - "schema_version is missing; assuming 1", - "write \"schema_version\": \"1\"", schemaVersion1)) + if v == nil { return } s, ok := v.(string) diff --git a/internal/core/record/checkreview_test.go b/internal/core/record/checkreview_test.go index af6ce32c9..062f6220a 100644 --- a/internal/core/record/checkreview_test.go +++ b/internal/core/record/checkreview_test.go @@ -199,8 +199,14 @@ func TestCheckReviewCoversBundleInputs(t *testing.T) { path string }{ { - name: "schema-missing", json: `{"body":"` + checkBody + `","approved":false}`, - code: core.ProblemCodeMissingField, level: core.ProblemLevelWarning, path: "$.schema_version", + name: "schema-version-wrong-type", + json: `{"schema_version":false,"body":"` + checkBody + `","approved":false,"comments":[]}`, + code: core.ProblemCodeWrongType, level: core.ProblemLevelError, path: "$.schema_version", + }, + { + name: "schema-version-wrong-value", + json: `{"schema_version":"2","body":"` + checkBody + `","approved":false,"comments":[]}`, + code: core.ProblemCodeBadEnum, level: core.ProblemLevelError, path: "$.schema_version", }, { name: "brief-outcome-missing", json: comment(`,"brief":{"sites":[]}`), @@ -320,6 +326,14 @@ func TestCheckReviewCountsTruncatedLevels(t *testing.T) { } } +func TestCheckReviewAcceptsMissingSchemaVersion(t *testing.T) { + t.Parallel() + res := recordcore.CheckReview([]byte(`{"body":"`+checkBody+`","approved":false,"comments":[]}`), checkDiff) + if len(res.Problems) != 0 || res.Warnings != 0 { + t.Fatalf("problems = %+v (warnings %d), want none for an omitted optional key", res.Problems, res.Warnings) + } +} + // TestCheckReviewMissingComments proves the schema-required comments key is // reported instead of silently accepted. func TestCheckReviewMissingComments(t *testing.T) { diff --git a/internal/core/record/coverage_exempt_test.go b/internal/core/record/coverage_exempt_test.go index ba97c0349..cfa2c3e11 100644 --- a/internal/core/record/coverage_exempt_test.go +++ b/internal/core/record/coverage_exempt_test.go @@ -5,6 +5,26 @@ import ( "testing" ) +const generatedTypeScriptDiff = "diff --git a/notes.txt b/notes.txt\nindex 1111111..2222222 100644\n" + + "--- a/notes.txt\n+++ b/notes.txt\n@@ -1,3 +1,3 @@\n top\n-old line\n+new line\n bottom\n" + + "diff --git a/clients/ts/src/refusal-codes.gen.ts b/clients/ts/src/refusal-codes.gen.ts\n" + + "index 3333333..4444444 100644\n--- a/clients/ts/src/refusal-codes.gen.ts\n" + + "+++ b/clients/ts/src/refusal-codes.gen.ts\n@@ -1,3 +1,3 @@\n top\n-old line\n+new line\n bottom\n" + +func TestMergeCoverageExemptsGeneratedTypeScript(t *testing.T) { + coverageBundle(t, generatedTypeScriptDiff, coverageReview("notes.txt", `"coverage":[{"file":"notes.txt","ranges":"all"}],`)) + if ref := runCoverage(t, false, false); ref != nil { + t.Fatalf("record refused: %+v", ref) + } + cov, ok := decodeObject(t, ".verdict/state/"+coverageSlug+"/record.json")["coverage"].(map[string]any) + if !ok { + t.Fatalf("record.json carries no coverage receipt") + } + if cov["percent"] != float64(100) || cov["gaps"] != float64(0) || cov["exempt"] != float64(1) { + t.Errorf("coverage receipt = %+v, want percent 100, gaps 0, exempt 1", cov) + } +} + // A bundle whose every changed file is generated has nothing to attest, so // --require-coverage must pass rather than refuse on a 0/0 denominator: the // exemption is automatic and needs no sentinel. The gate is the one place the diff --git a/internal/core/record/merge.go b/internal/core/record/merge.go index b826a714e..f878cee26 100644 --- a/internal/core/record/merge.go +++ b/internal/core/record/merge.go @@ -122,7 +122,7 @@ func (st *state) validateAndMerge(body string) (string, diagram.Status, error) { st.loadKB() readcommon.WarnContextSources(st.dir, st.review, st.warn) st.kb.Staged = []map[string]any{} - if err := checkReviewSchema(st.warnInput, st.review); err != nil { + if err := checkReviewSchema(st.review); err != nil { return "", diagram.Status{}, err } if err := st.validateInputs(); err != nil { @@ -296,10 +296,12 @@ func validJSON(raw []byte) bool { return err == nil } -func checkReviewSchema(warn func(string), review *jq.Obj) error { +func checkReviewSchema(review *jq.Obj) error { + if review == nil { + return nil + } rv, has := review.Get("schema_version") - if review == nil || !has || rv == nil || rv == false { - warn("review.json carries no schema_version — assuming 1 (agents: write \"schema_version\":\"1\")") + if !has || rv == nil { return nil } if s := jq.R(rv); s == schemaVersion1 { diff --git a/internal/core/record/schemaversion_test.go b/internal/core/record/schemaversion_test.go new file mode 100644 index 000000000..cffda3ac7 --- /dev/null +++ b/internal/core/record/schemaversion_test.go @@ -0,0 +1,24 @@ +package recordcore_test + +import ( + "context" + "strings" + "testing" + + recordcore "github.com/stainless-code/verdict/internal/core/record" +) + +func TestRecordOmitsSchemaVersionWarning(t *testing.T) { + review := `{"body":"> [!IMPORTANT]\n> Record service check with enough length.","approved":false,"comments":[]}` + bundle(t, "local-noschema", review) + c := &collector{} + resp, ref := recordcore.Run(context.Background(), recordRequest(t, c, "local-noschema")) + if ref != nil { + t.Fatalf("Run refusal: %v", ref) + } + for _, w := range resp.Warnings { + if strings.Contains(w, "schema_version") { + t.Errorf("warnings = %q, want none about the optional schema_version", resp.Warnings) + } + } +} diff --git a/internal/core/record/store_contract_test.go b/internal/core/record/store_contract_test.go index f1452196b..4c5fd6cdf 100644 --- a/internal/core/record/store_contract_test.go +++ b/internal/core/record/store_contract_test.go @@ -129,6 +129,7 @@ func TestRunRecordInputShapeRefusals(t *testing.T) { {"triage-not-object", `,"triage":[42]`, "bad-review-input", "triage entry must be an object"}, {"sections-not-array", `,"sections":1`, "bad-review-input", "sections must be an array"}, {"section-not-object", `,"sections":[42]`, "bad-review-input", "section must be an object"}, + {"coverage-ranges-array", `,"coverage":[{"file":"notes.txt","ranges":["1-2"]}]`, "bad-review-input", "value must be a string"}, } for _, tc := range tests { t.Run(tc.name, func(t *testing.T) { diff --git a/internal/core/report/coverage_next_test.go b/internal/core/report/coverage_next_test.go index 2d83ee5c8..8a8fbdfe7 100644 --- a/internal/core/report/coverage_next_test.go +++ b/internal/core/report/coverage_next_test.go @@ -20,6 +20,11 @@ const coverageDiff = "diff --git a/notes.txt b/notes.txt\nindex 1111111..2222222 const generatedDiff = "diff --git a/dist/app.min.js b/dist/app.min.js\nindex 3333333..4444444 100644\n" + "--- a/dist/app.min.js\n+++ b/dist/app.min.js\n@@ -1,3 +1,3 @@\n top\n-old line\n+new line\n bottom\n" +const generatedTypeScriptDiff = coverageDiff + + "diff --git a/clients/ts/src/refusal-codes.gen.ts b/clients/ts/src/refusal-codes.gen.ts\n" + + "index 3333333..4444444 100644\n--- a/clients/ts/src/refusal-codes.gen.ts\n" + + "+++ b/clients/ts/src/refusal-codes.gen.ts\n@@ -1,3 +1,3 @@\n top\n-old line\n+new line\n bottom\n" + // coverageReview builds a review over path whose body clears the degenerate // floor, with extra as the top-level fragment carrying coverage[]. func coverageReview(path, extra string) string { @@ -113,6 +118,44 @@ func TestRunNextCoverageRefusesCorruptReview(t *testing.T) { } } +func TestRunNextCoverageExemptsGeneratedTypeScript(t *testing.T) { + req := coverageNextRequest(t, map[string]string{ + "full.diff": generatedTypeScriptDiff, + "hunks.tsv": hunks.Format(hunks.Parse(generatedTypeScriptDiff)), + "review.json": coverageReview("notes.txt", `"coverage":[{"file":"notes.txt","ranges":"all"}],`), + }) + resp, ref := reportcore.RunNextCoverage(req) + if ref != nil { + t.Fatalf("RunNextCoverage refusal: %v", ref) + } + cov := resp.Coverage + if cov == nil || cov.Percent == nil || *cov.Percent != 100 || cov.Exempt != 1 || len(cov.Unattested) != 0 { + t.Fatalf("coverage = %+v, want 100%% with one generated exempt and no gaps", cov) + } +} + +func TestRunNextCoverageRefusesArrayRanges(t *testing.T) { + req := coverageNextRequest(t, coverageFiles(coverageReview("notes.txt", `"coverage":[{"file":"notes.txt","ranges":["1-2"]}],`))) + resp, ref := reportcore.RunNextCoverage(req) + if ref == nil { + t.Fatalf("RunNextCoverage = %+v; want a bad-review-input refusal", resp.Coverage) + } + if ref.Code != "bad-review-input" { + t.Fatalf("refusal code = %q, want bad-review-input", ref.Code) + } + if want := "review.json has 1 problem(s); fix every one, then re-run"; ref.Message != want { + t.Errorf("refusal message = %q, want %q", ref.Message, want) + } + if len(ref.Problems) != 1 { + t.Fatalf("problems = %+v, want exactly one", ref.Problems) + } + p := ref.Problems[0] + if p.Path != "$.coverage[0].ranges" || p.Code != core.ProblemCodeWrongType || + p.Message != "value must be a string" { + t.Errorf("problem = %+v, want the record checker's wrong-type problem", p) + } +} + // TestRunNextCoverageChunked pins criterion 9's chunked leg: a chunked bundle // reports its chunk count and is not evaluated, because chunks[] is its // attestation. diff --git a/internal/core/report/gate_test.go b/internal/core/report/gate_test.go index 33630c0ea..c95b5515b 100644 --- a/internal/core/report/gate_test.go +++ b/internal/core/report/gate_test.go @@ -91,6 +91,45 @@ func TestRunReportCheckTierGate(t *testing.T) { } } +func TestRunReportGatePassNextNamesMerge(t *testing.T) { + below := "[" + entryJSON("1111111111111111111111111111111111111111", "inline", "open", `"informational"`, "true") + "]" + atFloor := "[" + entryJSON("2222222222222222222222222222222222222222", "inline", "open", `"important"`, "true") + "]" + tests := []struct { + name string + findings string + wantNext string + gateFailed bool + }{ + {name: "pass", findings: below, wantNext: "merge — gate passes", gateFailed: false}, + {name: "fail", findings: atFloor, wantNext: "address open threads", gateFailed: true}, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + resp, ref := reportcore.Run(context.Background(), &core.ReportRequest{ + ReadRequest: fixture(t, tc.findings), + Flags: core.ReportFlags{Check: "important"}, + }) + if ref != nil { + t.Fatalf("Run refusal: %v", ref) + } + assertGateNext(t, resp, tc.gateFailed, tc.wantNext) + }) + } +} + +func assertGateNext(t *testing.T, resp *core.ReportResponse, gateFailed bool, wantNext string) { + t.Helper() + if resp.GateFailed != gateFailed { + t.Errorf("GateFailed = %v, want %v", resp.GateFailed, gateFailed) + } + if resp.Next != wantNext { + t.Errorf("Next = %q, want %q", resp.Next, wantNext) + } + if got, want := resp.Human[len(resp.Human)-1], "next: "+wantNext; got != want { + t.Errorf("human footer = %q, want %q", got, want) + } +} + // TestRunReportCompletenessGate pins the HEAD-reviewed dimension: a matching // post passes, and a missing, head-less, or stale post fails with its typed // line naming the re-fetch command. The combined case proves one run fails on @@ -398,9 +437,15 @@ func TestRunReportLiveGate(t *testing.T) { } } -// TestRunReportLocalBundleLiveSkip pins the structural skip: a local bundle -// has no live threads to verify, so every posture reads neutral with a warn -// and zero network. +type liveProgress struct { + human []string + warnings []string +} + +func (p *liveProgress) Human(line string) { p.human = append(p.human, line) } +func (p *liveProgress) Warn(msg string) { p.warnings = append(p.warnings, msg) } +func (p *liveProgress) Diag(_ string) {} + func TestRunReportLocalBundleLiveSkip(t *testing.T) { dir := t.TempDir() t.Chdir(dir) @@ -409,8 +454,9 @@ func TestRunReportLocalBundleLiveSkip(t *testing.T) { writeFile(t, ".verdict/state/local-gate/findings.json", "["+ entryJSON("1111111111111111111111111111111111111111", "inline", "open", `"important"`, "true")+"]") cmd := &threadCommander{stdout: threadsPage(4, 0)} + progress := &liveProgress{} req := core.ReadRequest{ - Runtime: &core.Runtime{Now: func() time.Time { return fixedNow }, Commander: cmd}, + Runtime: &core.Runtime{Now: func() time.Time { return fixedNow }, Commander: cmd, Progress: progress}, Repo: "acme/app", Pr: "local-gate", } resp, ref := reportcore.Run(context.Background(), &core.ReportRequest{ @@ -423,12 +469,34 @@ func TestRunReportLocalBundleLiveSkip(t *testing.T) { if cmd.calls != 0 { t.Errorf("commander calls = %d; a local bundle pays zero network", cmd.calls) } - if !containsLine(resp.Warnings, "live recheck skipped: local bundle threads unreadable here (gating on local dimensions only)") { - t.Errorf("Warnings = %q, want the local-bundle skip notice", resp.Warnings) + wantInfo := "info: live recheck skipped: local bundle threads unreadable here (gating on local dimensions only)" + if !containsLine(progress.human, wantInfo) { + t.Errorf("human stream = %q, want %q", progress.human, wantInfo) + } + if containsLine(resp.Warnings, "live recheck skipped") { + t.Errorf("Warnings = %q, want no warning for the structural local-bundle skip", resp.Warnings) + } + if containsLine(progress.warnings, "live recheck skipped") { + t.Errorf("warning stream = %q, want no structural skip warning", progress.warnings) } if containsLine(resp.CheckLines, "live open thread(s)") { t.Errorf("CheckLines = %q, want no live-thread verdict line", resp.CheckLines) } + if got := liveDimensionStatus(resp); got != "skip" { + t.Errorf("live dimension status = %q, want skip", got) + } +} + +func liveDimensionStatus(resp *core.ReportResponse) string { + if resp.Data.Gate == nil { + return "" + } + for _, d := range resp.Data.Gate.Dimensions { + if d.ID == "live_threads" { + return d.Status + } + } + return "" } func containsLine(lines []string, want string) bool { diff --git a/internal/core/report/live.go b/internal/core/report/live.go index 66baad152..b6e5d3bec 100644 --- a/internal/core/report/live.go +++ b/internal/core/report/live.go @@ -36,7 +36,7 @@ func runLiveProbe(ctx context.Context, st *state) { st.warn("live recheck skipped (--live skip requested — gating on local dimensions only)") st.live.status = liveSkipExplicit case isLocalDir(st.dir): - st.warn("live recheck skipped: local bundle threads unreadable here (gating on local dimensions only)") + st.req.Human("info: live recheck skipped: local bundle threads unreadable here (gating on local dimensions only)") st.live.status = liveSkip default: probeLiveCount(ctx, st) diff --git a/internal/core/report/render.go b/internal/core/report/render.go index b691540aa..256b63df5 100644 --- a/internal/core/report/render.go +++ b/internal/core/report/render.go @@ -16,6 +16,13 @@ import ( ) func buildResponse(st *state) *core.ReportResponse { + var gateFailed bool + var checkLines []string + var gate *core.ReportGate + if st.gate { + gateFailed, checkLines, gate = gateTail(st) + st.gatePassed = !gateFailed + } resp := &core.ReportResponse{ Repo: st.req.Repo, Pr: st.req.Pr, @@ -32,21 +39,28 @@ func buildResponse(st *state) *core.ReportResponse { resp.Data.ProvenanceTruncated = provenance.MergeTruncated(st.provenance, readcommon.BundleTruncation(st.dir)) } if st.gate { - resp.GateFailed, resp.CheckLines, resp.Data.Gate = gateTail(st) + resp.GateFailed, resp.CheckLines, resp.Data.Gate = gateFailed, checkLines, gate } return resp } -func reportNext(st *state) string { - if st.flags.Output != "" { - return "wrote " + st.flags.Output +func nextStep(st *state, open int) string { + if st.flags.Check != "" && st.gatePassed { + return "merge — gate passes" } - if viewOpen(viewRanked(st)) > 0 { + if open > 0 { return "address open threads" } return "none open — nothing to address" } +func reportNext(st *state) string { + if st.flags.Output != "" { + return "wrote " + st.flags.Output + } + return nextStep(st, viewOpen(viewRanked(st))) +} + func buildData(st *state) *core.ReportData { data := &core.ReportData{ Verification: panelcore.Summarize(st.entries), @@ -289,10 +303,7 @@ func finalizeHuman(st *state, view []readcommon.Ranked, lines []string) []string // The footer follows the view the header reports: with a display floor the // rendered list can be empty while the store still holds open findings, and // "address open threads" beside "(0 shown) open=0" read as a contradiction. - if viewOpen(view) > 0 { - return append(lines, "next: address open threads") - } - return append(lines, "next: none open — nothing to address") + return append(lines, "next: "+nextStep(st, viewOpen(view))) } func appendHumanTail(st *state, lines []string) []string { diff --git a/internal/core/report/service.go b/internal/core/report/service.go index bc1c78b08..f478c1288 100644 --- a/internal/core/report/service.go +++ b/internal/core/report/service.go @@ -92,6 +92,7 @@ type state struct { // field grouping documents the loaded dimensions; alignment nOpen int nAllOpen int gate bool + gatePassed bool liveGate bool compOK bool } diff --git a/internal/provenance/basis.go b/internal/provenance/basis.go index 38a0e2c5a..7815970bf 100644 --- a/internal/provenance/basis.go +++ b/internal/provenance/basis.go @@ -273,10 +273,11 @@ func replaceBasisLine(body string, lines []string, i int, computed, expected str } } -// insertBasisLine adds the computed Basis line under the confidence lead, -// keeping a blank line before a following heading. func insertBasisLine(lines []string, computed string) (string, *core.Problem) { at := leadEnd(lines) + if at == 0 { + return strings.Join(lines, newline), nil + } insert := []string{computed} if at < len(lines) { if trimmed := strings.TrimSpace(lines[at]); trimmed != "" && strings.HasPrefix(trimmed, "#") { diff --git a/internal/provenance/provenance_test.go b/internal/provenance/provenance_test.go index 6f185e953..7f64e2aeb 100644 --- a/internal/provenance/provenance_test.go +++ b/internal/provenance/provenance_test.go @@ -323,6 +323,14 @@ func TestApplyBasisLineInsertsBlankBeforeHeading(t *testing.T) { } } +func TestApplyBasisLineSkipsPlainBody(t *testing.T) { + t.Parallel() + const body = "A plain review body with no quote lead.\n\n## Summary\nReviewed." + if got, p := provenance.ApplyBasisLine("0 0 0 1 0 0 0", body); got != body || p != nil { + t.Fatalf("plain body = %q, %+v; want unchanged without a warning", got, p) + } +} + // TestPlainBasisBelowConfidence accepts matching plain Basis text. func TestPlainBasisBelowConfidence(t *testing.T) { const body = "> Confidence: 3/5\nBasis: 0 0 0 1 0 0 0\n\n## Summary\nReviewed." diff --git a/internal/readcommon/roles.go b/internal/readcommon/roles.go index 31f8f2471..5eec3e9a5 100644 --- a/internal/readcommon/roles.go +++ b/internal/readcommon/roles.go @@ -51,7 +51,7 @@ var ( var generatedSubstrings = []string{ "package-lock.json", "pnpm-lock.yaml", "yarn.lock", "bun.lock", "Cargo.lock", "Gemfile.lock", "poetry.lock", ".min.js", ".min.css", - "__snapshots__/", ".snap", ".pb.go", ".g.dart", ".generated.", + "__snapshots__/", ".snap", ".pb.go", ".g.dart", ".generated.", ".gen.", "zz_generated.", } // isGeneratedPath owns the generated branch, including the start-anchored diff --git a/internal/readcommon/roles_test.go b/internal/readcommon/roles_test.go index 95d2ac78e..120c3222c 100644 --- a/internal/readcommon/roles_test.go +++ b/internal/readcommon/roles_test.go @@ -19,6 +19,9 @@ func TestPathRole(t *testing.T) { {"package-lock.json", "generated"}, {"vendor/github.com/x/y.go", "generated"}, {"web/app.min.js", "generated"}, + {"clients/ts/src/refusal-codes.gen.ts", "generated"}, + {"internal/core/types.gen.go", "generated"}, + {"pkg/apis/zz_generated.deepcopy.go", "generated"}, {"docs/manual.md", "docs"}, {"README.md", "docs"}, {"go.mod", "config"}, diff --git a/internal/readgolden/testdata/report/check-critical-pass.golden b/internal/readgolden/testdata/report/check-critical-pass.golden index be9088463..3779e7b22 100644 --- a/internal/readgolden/testdata/report/check-critical-pass.golden +++ b/internal/readgolden/testdata/report/check-critical-pass.golden @@ -3,7 +3,7 @@ exit: 0 report: .verdict/state/5102 (1 shown) open=1 head=aaaaaaa 1. [open] notes.txt:2 (a3333333) by category: untagged 1 -next: address open threads +next: merge — gate passes --- stderr --- warn: patch identity unavailable — completeness compares heads warn: run bundle has no typed evidence (pre-typed bundle?) — rendering without policy floors diff --git a/internal/readgolden/testdata/report/live-auto-unreachable.golden b/internal/readgolden/testdata/report/live-auto-unreachable.golden index dcaa1350b..6623a000d 100644 --- a/internal/readgolden/testdata/report/live-auto-unreachable.golden +++ b/internal/readgolden/testdata/report/live-auto-unreachable.golden @@ -3,7 +3,7 @@ exit: 0 report: .verdict/state/5102 (1 shown) open=1 head=aaaaaaa 1. [open] notes.txt:2 (a3333333) by category: untagged 1 -next: address open threads +next: merge — gate passes --- stderr --- warn: run bundle has no typed evidence (pre-typed bundle?) — rendering without policy floors warn: live thread recheck unreachable (evals-stub-gh: network disabled (api graphql --paginate -F owner=acme -F name=app -F pr=5102 -f query= query($owne — gating on local dimensions only) diff --git a/internal/readgolden/testdata/report/live-skip.golden b/internal/readgolden/testdata/report/live-skip.golden index e50d087aa..c6493a34f 100644 --- a/internal/readgolden/testdata/report/live-skip.golden +++ b/internal/readgolden/testdata/report/live-skip.golden @@ -3,7 +3,7 @@ exit: 0 report: .verdict/state/5102 (1 shown) open=1 head=aaaaaaa 1. [open] notes.txt:2 (a3333333) by category: untagged 1 -next: address open threads +next: merge — gate passes --- stderr --- warn: run bundle has no typed evidence (pre-typed bundle?) — rendering without policy floors warn: live recheck skipped (--live skip requested — gating on local dimensions only) diff --git a/internal/readgolden/testdata/report/local-check-json.golden b/internal/readgolden/testdata/report/local-check-json.golden index 0b4a6e72a..641ffcb47 100644 --- a/internal/readgolden/testdata/report/local-check-json.golden +++ b/internal/readgolden/testdata/report/local-check-json.golden @@ -6,10 +6,9 @@ exit: 0 "repo": "acme/app", "pr": null, "at": "0000-00-00T00:00:00Z", - "next": "address open threads", + "next": "merge — gate passes", "warnings": [ - "run bundle has no typed evidence (pre-typed bundle?) — rendering without policy floors", - "live recheck skipped: local bundle threads unreadable here (gating on local dimensions only)" + "run bundle has no typed evidence (pre-typed bundle?) — rendering without policy floors" ], "data": { "generated_at": "0000-00-00T00:00:00Z", @@ -105,6 +104,6 @@ exit: 0 } --- stderr --- warn: run bundle has no typed evidence (pre-typed bundle?) — rendering without policy floors -warn: live recheck skipped: local bundle threads unreadable here (gating on local dimensions only) +info: live recheck skipped: local bundle threads unreadable here (gating on local dimensions only) check: no open findings at-or-above critical — gate passes --- files --- diff --git a/internal/record/testdata/degenerate/missing-marker-ok.golden b/internal/record/testdata/degenerate/missing-marker-ok.golden index 729dd6483..8bb4e87ab 100644 --- a/internal/record/testdata/degenerate/missing-marker-ok.golden +++ b/internal/record/testdata/degenerate/missing-marker-ok.golden @@ -8,8 +8,7 @@ exit: 0 "at": "0000-00-00T00:00:00Z", "next": "report or triage the local bundle; post stays refused (no PR target)", "warnings": [ - "no fetch-time hunk snapshot (.verdict/state/local-dg/hunks.tsv missing) — rebuilding from full.diff, validation is best-effort (re-fetch to pin the snapshot)", - "$.body: no Basis line in the leading quote block — inserted the computed counts (basis-replaced)" + "no fetch-time hunk snapshot (.verdict/state/local-dg/hunks.tsv missing) — rebuilding from full.diff, validation is best-effort (re-fetch to pin the snapshot)" ], "data": { "dir": ".verdict/state/local-dg", @@ -32,7 +31,6 @@ exit: 0 --- stderr --- warn: no fetch-time hunk snapshot (.verdict/state/local-dg/hunks.tsv missing) — rebuilding from full.diff, validation is best-effort (re-fetch to pin the snapshot) » anchors: 0 inline, 0 moved to body -warn: $.body: no Basis line in the leading quote block — inserted the computed counts (basis-replaced) » findings: 0 sigs -> .verdict/state/local-dg/findings.json recorded: .verdict/state/local-dg/findings.json (0 open) + .verdict/state/local-dg/record.json — report/triage read it; post stays refused (no PR target) --- files --- diff --git a/internal/schemagen/overlay.json b/internal/schemagen/overlay.json index 898da93c3..946cab64f 100644 --- a/internal/schemagen/overlay.json +++ b/internal/schemagen/overlay.json @@ -453,8 +453,8 @@ }, "review": { "descriptions": { - "": "What the agent writes and post/record validate. One checker reports every problem: error refuses; warning lets record proceed and is promoted by --strict/strict_input. Errors: schema_version other than 1, missing/non-string/degenerate body, non-boolean approved, non-array comments or non-object items, comment path missing/non-string/not in diff, line/start_line wrong type, side wrong type or not RIGHT/LEFT, missing body, non-string suggestion, brief outcome/site/approach/question shape or range failures, rigor/severity wrong type or unknown, non-boolean in-bounds, kb non-array/non-string, resolved/triage/sections shape and enums, confidence non-integer or outside 1-5, non-string security_summary, malformed or partial chunks, and malformed verification claims. Warnings: missing schema_version, missing comments, no line/start_line, anchor outside hunks, unknown fields, unset rigor, an unknown or non-string category (warns once, stores null), brief site outside hunks, fenced suggestion, malformed kb id, and unknown or unattested chunks. Unknown top-level keys are allowed by the schema but still warn.", - "/properties/schema_version": "Agents stamp \"1\"; missing warns, any other value refuses.", + "": "What the agent writes and post/record validate. One checker reports every problem: error refuses; warning lets record proceed and is promoted by --strict/strict_input. An absent schema_version reads as 1; any other value refuses. Errors: schema_version other than 1, missing/non-string/degenerate body, non-boolean approved, non-array comments or non-object items, comment path missing/non-string/not in diff, line/start_line wrong type, side wrong type or not RIGHT/LEFT, missing body, non-string suggestion, brief outcome/site/approach/question shape or range failures, rigor/severity wrong type or unknown, non-boolean in-bounds, kb non-array/non-string, resolved/triage/sections shape and enums, confidence non-integer or outside 1-5, non-string security_summary, malformed or partial chunks, and malformed verification claims. Warnings: missing comments, no line/start_line, anchor outside hunks, unknown fields, unset rigor, an unknown or non-string category (warns once, stores null), brief site outside hunks, fenced suggestion, malformed kb id, and unknown or unattested chunks. Unknown top-level keys are allowed by the schema but still warn.", + "/properties/schema_version": "Agents stamp \"1\"; an absent key reads as 1, any other value refuses.", "/properties/body": "Full review text, verdict marker first line. Missing, non-string, or degenerate text (<20 chars with no verdict marker, or a short placeholder) refuses.", "/properties/approved": "Binding ask; the CLI --bind/--request-changes flags win over this. A non-boolean refuses.", "/properties/comments": "Inline findings; one per anchor. Missing warns as an empty list; a non-array or non-object item refuses.", diff --git a/internal/schemagen/semantics_test.go b/internal/schemagen/semantics_test.go index e154878ea..f8f8dff5a 100644 --- a/internal/schemagen/semantics_test.go +++ b/internal/schemagen/semantics_test.go @@ -421,7 +421,9 @@ var publishedSemantics = map[string]string{ // or type changed. // risk score: additive optional nullable risk claim ($defs/riskClaim). No // existing property, required key, enum, or type changed. - "review.schema.json": "58b8cafc67abfc6fe394f0cd7dff77182def6cd6652851e037ded38dc57defbe", + // schema_version optional: descriptions drop the missing-key warning. No + // property, required key, enum, or type changed. + "review.schema.json": "76232c7a9f815c81f2e2115778ebba6f55cc3bdfb9548b6f734effb0ef0cd768", // additive selected.schema.json (new bundle document: the reaction-scoped // fix selection fetch writes and fix --selected reads). A new document // only: no existing schema property, required key, enum, or type changed. diff --git a/internal/schemagen/testdata/baseline/review.schema.json b/internal/schemagen/testdata/baseline/review.schema.json index 295d61d1c..55a7d5a50 100644 --- a/internal/schemagen/testdata/baseline/review.schema.json +++ b/internal/schemagen/testdata/baseline/review.schema.json @@ -2,11 +2,11 @@ "$schema": "https://json-schema.org/draft/2020-12/schema", "$id": "https://github.com/stainless-code/verdict/blob/main/skills/verdict/schemas/review.schema.json", "title": "review.json — agent-written review input", - "description": "What the agent writes and post/record validate. One checker reports every problem: error refuses; warning lets record proceed and is promoted by --strict/strict_input. Errors: schema_version other than 1, missing/non-string/degenerate body, non-boolean approved, non-array comments or non-object items, comment path missing/non-string/not in diff, line/start_line wrong type, side wrong type or not RIGHT/LEFT, missing body, non-string suggestion, brief outcome/site/approach/question shape or range failures, rigor/severity wrong type or unknown, non-boolean in-bounds, kb non-array/non-string, resolved/triage/sections shape and enums, confidence non-integer or outside 1-5, non-string security_summary, malformed or partial chunks, and malformed verification claims. Warnings: missing schema_version, missing comments, no line/start_line, anchor outside hunks, unknown fields, unset rigor, an unknown or non-string category (warns once, stores null), brief site outside hunks, fenced suggestion, malformed kb id, and unknown or unattested chunks. Unknown top-level keys are allowed by the schema but still warn.", + "description": "What the agent writes and post/record validate. One checker reports every problem: error refuses; warning lets record proceed and is promoted by --strict/strict_input. An absent schema_version reads as 1; any other value refuses. Errors: schema_version other than 1, missing/non-string/degenerate body, non-boolean approved, non-array comments or non-object items, comment path missing/non-string/not in diff, line/start_line wrong type, side wrong type or not RIGHT/LEFT, missing body, non-string suggestion, brief outcome/site/approach/question shape or range failures, rigor/severity wrong type or unknown, non-boolean in-bounds, kb non-array/non-string, resolved/triage/sections shape and enums, confidence non-integer or outside 1-5, non-string security_summary, malformed or partial chunks, and malformed verification claims. Warnings: missing comments, no line/start_line, anchor outside hunks, unknown fields, unset rigor, an unknown or non-string category (warns once, stores null), brief site outside hunks, fenced suggestion, malformed kb id, and unknown or unattested chunks. Unknown top-level keys are allowed by the schema but still warn.", "type": "object", "properties": { "schema_version": { - "description": "Agents stamp \"1\"; missing warns, any other value refuses.", + "description": "Agents stamp \"1\"; an absent key reads as 1, any other value refuses.", "const": "1" }, "body": { diff --git a/pkg/verdict/check_example_test.go b/pkg/verdict/check_example_test.go index 8bbb13deb..1817590d0 100644 --- a/pkg/verdict/check_example_test.go +++ b/pkg/verdict/check_example_test.go @@ -20,5 +20,4 @@ func ExampleCheckReview() { // error missing-field $.comments[0].brief.outcome // error wrong-type $.comments[0].brief.questions // warning rigor-unset $.comments[0].rigor - // warning missing-field $.schema_version } diff --git a/skills/verdict/schemas/review.schema.json b/skills/verdict/schemas/review.schema.json index 295d61d1c..55a7d5a50 100644 --- a/skills/verdict/schemas/review.schema.json +++ b/skills/verdict/schemas/review.schema.json @@ -2,11 +2,11 @@ "$schema": "https://json-schema.org/draft/2020-12/schema", "$id": "https://github.com/stainless-code/verdict/blob/main/skills/verdict/schemas/review.schema.json", "title": "review.json — agent-written review input", - "description": "What the agent writes and post/record validate. One checker reports every problem: error refuses; warning lets record proceed and is promoted by --strict/strict_input. Errors: schema_version other than 1, missing/non-string/degenerate body, non-boolean approved, non-array comments or non-object items, comment path missing/non-string/not in diff, line/start_line wrong type, side wrong type or not RIGHT/LEFT, missing body, non-string suggestion, brief outcome/site/approach/question shape or range failures, rigor/severity wrong type or unknown, non-boolean in-bounds, kb non-array/non-string, resolved/triage/sections shape and enums, confidence non-integer or outside 1-5, non-string security_summary, malformed or partial chunks, and malformed verification claims. Warnings: missing schema_version, missing comments, no line/start_line, anchor outside hunks, unknown fields, unset rigor, an unknown or non-string category (warns once, stores null), brief site outside hunks, fenced suggestion, malformed kb id, and unknown or unattested chunks. Unknown top-level keys are allowed by the schema but still warn.", + "description": "What the agent writes and post/record validate. One checker reports every problem: error refuses; warning lets record proceed and is promoted by --strict/strict_input. An absent schema_version reads as 1; any other value refuses. Errors: schema_version other than 1, missing/non-string/degenerate body, non-boolean approved, non-array comments or non-object items, comment path missing/non-string/not in diff, line/start_line wrong type, side wrong type or not RIGHT/LEFT, missing body, non-string suggestion, brief outcome/site/approach/question shape or range failures, rigor/severity wrong type or unknown, non-boolean in-bounds, kb non-array/non-string, resolved/triage/sections shape and enums, confidence non-integer or outside 1-5, non-string security_summary, malformed or partial chunks, and malformed verification claims. Warnings: missing comments, no line/start_line, anchor outside hunks, unknown fields, unset rigor, an unknown or non-string category (warns once, stores null), brief site outside hunks, fenced suggestion, malformed kb id, and unknown or unattested chunks. Unknown top-level keys are allowed by the schema but still warn.", "type": "object", "properties": { "schema_version": { - "description": "Agents stamp \"1\"; missing warns, any other value refuses.", + "description": "Agents stamp \"1\"; an absent key reads as 1, any other value refuses.", "const": "1" }, "body": {