Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/trigger-filter.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
"@stainless-code/verdict": minor
---

`ready --filter` now gates the review on the closed trigger predicates in the trusted base configuration: matching PRs run and record the decision, non-matching PRs refuse `filtered-out` before any host call, and unusable policy or facts refuse `trigger-failed`.

Fetch now requests and records PR labels in `PR.json` for every host, so the `labels` trigger predicate can read them from the bundle.
2 changes: 2 additions & 0 deletions clients/ts/src/refusal-codes.gen.ts
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ export type RefusalCode =
| "download-failed"
| "duplicate-payload"
| "fetch-failed"
| "filtered-out"
| "fix-ceiling"
| "fix-failed"
| "fix-patch-secret"
Expand Down Expand Up @@ -202,6 +203,7 @@ export type RefusalCode =
| "store-busy"
| "threads-failed"
| "triage-failed"
| "trigger-failed"
| "uninstall-foreign"
| "uninstall-no-receipt"
| "unknown-agent"
Expand Down
7 changes: 5 additions & 2 deletions cmd/verdict/usage.go
Original file line number Diff line number Diff line change
Expand Up @@ -235,14 +235,17 @@ func usageText() string {
" Foreground wait: timeout 1s–30m; interval >=1s. Exit 0 on condition,\n" +
" exit 1 with wait-timeout or condition-unmet data; no local writes.\n" +
"\n" +
" ready <owner/repo> <pr> [--json] [--check] [--wait-for build --timeout D]\n" +
" advisory only (exit 0, never a gate): CI state, open threads, draft/conflict,\n" +
" ready <owner/repo> <pr> [--json] [--check] [--filter] [--wait-for build --timeout D]\n" +
" advisory by default (exit 0): CI state, open threads, draft/conflict,\n" +
" description completeness. Prints blockers with actions + recommendation\n" +
" (review/wait/nudge-author). GitHub and GitLab PR bundles; local bundles refuse\n" +
" (no-pr-target) and other providers refuse typed (provider-unimplemented).\n" +
" --check opts into gating (same output, machine exits): 0 ready, 1 blockers\n" +
" or wait, 2 usage, 3 tool error. Without --check the run stays advisory:\n" +
" exit 0 with blockers listed (typed errors exit 1, usage exits 2).\n" +
" --filter applies the closed trigger in .verdict/config.json at the PR base\n" +
" before any live read: a non-matching PR refuses filtered-out naming the\n" +
" predicate; both outcomes are recorded in the bundle's run.json.\n" +
" gate <owner/repo> <pr> [--max-severity critical|important|informational] [--require-head]\n" +
" [--live auto|strict|skip] [--coverage require|warn] [--json]\n" +
" one composed local gate over the dimensions report already computes: open\n" +
Expand Down
27 changes: 25 additions & 2 deletions docs/manual.md
Original file line number Diff line number Diff line change
Expand Up @@ -579,7 +579,7 @@ the artifact, who writes it, and whether a verb gates on it.

| Artifact | What it is | Who writes it | Gates? |
| ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `run.json` | The typed per-run capture: resolved defaults, per-file resolutions, plan, spend and privacy views, trusted-base inventory hashes, base SHA, the `inputs_sha256` digest, and appended check observations. | `fetch`/`fetch-local`; `post` appends observations | Yes, except `config --preview` (see below). `post`, `record`, `triage`, `report`, `ready`, `config --explain`, and `verify --group` verify it; a run bundle deleted while its controls snapshot remains, or an edited captured input, refuses `controls-stale`; a bare bundle dir reads with a warning and no repository requirements. |
| `run.json` | The typed per-run capture: resolved defaults, per-file resolutions, plan, spend and privacy views, trusted-base inventory hashes, base SHA, the `inputs_sha256` digest, appended check observations, and the `trigger` decision from `ready --filter`. | `fetch`/`fetch-local`; `post` appends observations, `ready --filter` records the trigger decision | Yes, except `config --preview` (see below). `post`, `record`, `triage`, `report`, `ready`, `config --explain`, and `verify --group` verify it; a run bundle deleted while its controls snapshot remains, or an edited captured input, refuses `controls-stale`; a bare bundle dir reads with a warning and no repository requirements. |
| `validation-<group>.json` | The deep-validation receipt for one plan group: run, inputs digest, group, reviewed and execution heads, canonical root, command, exit, timeout and sandbox flags, status, and a redacted output tail. | `verify --group <id> --command <cmd>` | Yes. `post` refuses a binding approval while any `depth: "deep"` group lacks a current receipt; `ready` reports the gap as `profile-work`. |
| `filesource.json` | Provenance for the PR file list: `source` (`paged` or `cli`), `cli_cap` (100), `provider_ceiling` (3000), `count`, and `truncated` or `paging_error`. | GitHub `fetch`, when the CLI list arrives at its 100-file cap | No. It is an advisory coverage note: a paging failure warns and the run proceeds with the CLI's first 100 files. |
| `full.raw.diff` | The raw pre-exclusion diff, kept as the patch-identity input when a filtered diff would otherwise replace it. No shipped verb writes it today: `fetch` writes the host diff to `full.diff` and records the filtered hash in `run.json` without writing filtered bytes. | Nothing today (a future exclusion pass) | Partly. When present it feeds the identity, and `identity.json`'s `raw_diff_sha256` must reproduce it; a present-but-unreadable file refuses instead of falling back to `full.diff`. Absence is normal. |
Expand Down Expand Up @@ -974,6 +974,29 @@ missing or stale receipt as required work without executing anything.
`ready` stays advisory (exit 0); `ready --check` turns the same list into the
machine exits 0 ready, 1 blockers-or-wait, 2 usage, 3 tool error.

### Base-branch trigger gate

`verdict ready owner/repo 42 --filter` gates the readiness read on the `trigger`
predicates in the trusted `.verdict/config.json` snapshot the bundle fetched,
not the worktree's current config. A configured predicate must match: `authors`
(exact login), `paths_include`, `paths_exclude`, `labels`, `base_branches`,
`title_matches` (Go regular expression), or `draft`. Lists OR within a
predicate and AND across predicates. Excluded paths are removed before
`paths_include` is tested, so one generated file cannot hide a protected
change. A truncated `filesource.json` skips the path predicates with a
warning instead of filtering out on partial evidence. The decision lands in
`run.json` as `trigger` (`run`, `reason`, `head`, `at`) and leaves the inputs
digest alone.

The gate runs before any live read or `--wait-for build` poll. A match runs
the ordinary readiness read. A miss refuses `filtered-out` with exit code 1,
naming the failing predicate before any host command runs; without `--filter`
the same PR is unchanged. A head-policy snapshot, damaged snapshot, unreadable
facts, corrupt run receipt, or malformed base policy refuses `trigger-failed`;
an unknown key remains a `bad-config` parse error naming `trigger.<key>`. An
absent base config means no trigger. `--filter` composes with `--check`
(the typed miss keeps exit 1) and `--json`.

On GitLab the same blockers read through `glab`: the merge request carries
`draft`, `has_conflicts` (plus a `conflict` `detailed_merge_status`), and the
`description`, while CI comes from the merge request's pipeline list (a failed
Expand Down Expand Up @@ -3433,7 +3456,7 @@ verdict fetch-local owner/repo slice-b -- <paths...> # narrow to a fil

`fetch --provider gitlab|gitcafe|forgejo|bitbucket|azure` builds the same bundle
shape from another host (`PR.json` normalized to one schema: `provider`,
`number/title/body`, `base_ref/head_ref`, `base_sha/head_sha`, `author`, `url`,
`number/title/body`, `base_ref/head_ref`, `base_sha/head_sha`, `author`, `labels`, `url`,
`state` as open|closed|merged, `draft`, per-file `additions/deletions`).
`write_toc_roles` and the whole validate+merge path run unchanged — anchors
validate against `full.diff` hunks regardless of host. `record`, `report`,
Expand Down
12 changes: 12 additions & 0 deletions docs/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -600,6 +600,12 @@ Fix: for provider failures, run `gh auth status` and confirm the repo/PR spellin
For context failures, hand over regular Markdown, JSON or `.txt` files — at most
64 files, 1 MiB total and eight path levels — then re-run.

## `filtered-out`

Cause: `ready --filter` found a trigger predicate in `.verdict/config.json` at the PR base that the PR does not match; the refusal names the failing predicate.

Fix: review the PR without `--filter`, or change the trigger at the base branch and re-fetch. The decision is recorded in `run.json`.

## `fix-ceiling`

Cause: the fix loop hit its pass ceiling without clearing the sig.
Expand Down Expand Up @@ -1260,6 +1266,12 @@ Cause: `triage` could not write `findings.json`.

Fix: make the bundle directory writable, then re-run.

## `trigger-failed`

Cause: the trigger could not read its trusted-base policy or record the decision in `run.json`.

Fix: re-fetch the bundle so its base snapshot is complete, then re-run `ready --filter`.

## `uninstall-foreign`

Cause: the binary is not installer-managed.
Expand Down
1 change: 1 addition & 0 deletions evals/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,7 @@ the build.
| `threads-resolve-github` | Explicit GitHub thread resolve + receipt | A seeded GitHub bundle: `threads resolve --dry-run` lists the verdict-authored addressed thread and makes zero host calls; a real run makes exactly one literal `resolveReviewThread` call, marks the feed resolved, and writes `threads-resolve.json`; a second run skips `already-resolved` with no call; `--thread` on a human thread refuses `not-verdict-authored` and leaves the receipt untouched; calibrate's follow-up read joins the resolved thread to the addressed finding as `fixed`. |
| `threads-resolve-gitlab` | Explicit GitLab thread resolve + receipt | A seeded GitLab bundle: `threads resolve --dry-run` lists the verdict-authored addressed discussion and makes zero host calls; a real run makes exactly one literal `glab api ... discussions/d1?resolved=true --method PUT` call, marks the feed resolved, and writes `threads-resolve.json`; a second run skips `already-resolved` with no call; `--thread` on a human discussion refuses `not-verdict-authored` and makes no call. |
| `trend-incidents` | `trend --incidents` joins incidents to the stores | Three attached incidents over three bundles: 3 attached, 2 measured, 1 unmeasured (its `findings.json` is gone), 1 hit, 1 miss, `miss_rate: 0.5`, and per-bundle rows with a null `miss_rate` for the unmeasured bundle; the human line reads the literal `incidents: 3 attached, 2 measured, 1 unmeasured — missed 1 of 2 (50.0%)`; the unflagged run never mentions incidents; flipping the caught finding to `false-positive` re-measures 2 misses at `miss_rate: 1`; an absent or empty ledger reads a zero tally and a null `miss_rate`; MCP `trend` with `incidents: true` returns the same object as `--json`; zero host calls |
| `trigger-filter` | `ready --filter`: base-branch trigger gate | A producer-built PR bundle with a base-snapshot trigger: a PR touching `protected/**` runs and records `trigger.reason=matched`; a PR touching only docs refuses `filtered-out` naming `paths_include`, pays zero host calls, and records `run=false` beside that predicate; a typed miss keeps exit 1 under `--check`; without `--filter` the same PR still reads live. |
| `upgrade` | Explicit `verdict upgrade` + `verdict uninstall` | Local loopback release server serves fake archives/checksums/bundle: installer self-replace updates the binary + receipt after SHA-256/signature/`--version` smoke; latest resolves through the redirect; checksum mismatch leaves the old binary; dry-run makes no request or write; npm delegation runs the exact argv, streams output, and propagates exit 7; project-local/skill/unidentifiable refuse typed with the exact command; uninstall removes only managed files (+ `--remove-path` marker) and refuses foreign channels; checksum-only warns/`VERDICT_REQUIRE_SIGNATURE=1` fails closed |
| `usage-adapters` | Optional harness converters outside the binary | `claude-to-usage.sh` and `opencode-to-usage.sh` turn the checked-in transcripts into records that validate against `schemas/usage.schema.json`, preferring a cost-state or session total and deduping repeats; the Stop hook writes a validating usage file; a timestamp-less transcript refuses with nothing written; tokens without a price still convert unpriced; the adapters README documents both as optional; a missing jq is a SKIP, never a pass |
| `usage-errors` | The binary owns usage errors (arity matrix, wins-over-faults) | No verb, missing positionals (one leg per verb; bare verbs never fail), and unknown verbs print the full usage block on stderr with exit 2 and no envelope; corrupt settings plus a missing positional still usage-errors (a complete invocation reads settings and refuses typed `bad-settings`); the block names `verdict` and keeps `--json`, the verify codes, and `token-scopes-excess` |
Expand Down
Loading
Loading