Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/sta-445-bundle-procedure.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@stainless-code/verdict": minor
---

Every fetched bundle now carries `procedure.md`, the review Procedure assembled by the binary from its embedded payload. Drain plans carry `drain.procedure.md` for reply drafting. The `run`, stage, and drain directives name those files as authoritative, so an installed skill that is older can no longer displace them. The fetch `next:` hints and `--show-prompts` output point at the bundle Procedure, and provenance receipts record its sha256 as a bundle input. `PromptsHash` now reads the embedded payload, so an installed user gets the real prompt-set hash without a `procedure/` checkout.
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,8 @@ skip-token ban and test determinism. Read both before writing or reviewing Go.
- **No `/tmp` test dirs.** Use `t.TempDir()` for files and `internal/testenv`
for child processes; a hardcoded path leaks across parallel runs.
- **Exactly six Procedure files are hashed into goldens:** `procedure/PROCEDURE.md`
and `procedure/prompts/{system,review,diagrams,verifier,validate}.md`; a change moves
only `prompts_hash` lines and the literal pin in `internal/provenance/prompts_test.go`.
and `procedure/prompts/{system,review,diagrams,verifier,validate}.md`; a Procedure edit
moves only `prompts_hash`, `internal/provenance/prompts_test.go`'s pin, and `procedure.md` / `drain.procedure.md` receipt `sha256`/`bytes` lines.

## Git and commits

Expand Down
7 changes: 4 additions & 3 deletions apps/docs/content/01-docs/(guides)/local-review.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,10 @@ Use this when there is no pull request yet: a branch you are shaping, a spike, o
</Step>

<Step title="Have an agent review it">
Point any agent at the bundle: it reads `full.diff` through `toc.txt` (plus
the Procedure from the binary: `verdict skill`, then `verdict skill show <path>`)
and writes exactly one
Point any agent at the bundle: it reads
`.verdict/state/local-mywork/procedure.md` first (the binary's authoritative
Procedure for this review — an installed skill may be older), then `full.diff`
through `toc.txt`, and writes exactly one
`.verdict/state/local-mywork/review.json` with `schema_version` 1, `body`,
`approved`, and `comments[]`. This is the agent's turn; there is no command to
run until the file lands.
Expand Down
4 changes: 2 additions & 2 deletions bench/runner/endtoend_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@ func TestRunWithTheShippedBinary(t *testing.T) {
if rep.Samples != 1 || len(rep.Failures) != 0 {
t.Fatalf("samples=%d failures=%v, want one clean sample", rep.Samples, rep.Failures)
}
if want := provenance.PromptsHash(cfg.Root); rep.PromptsHash != want {
t.Errorf("prompts_hash = %q, want the checkout's prompt set (%q): the run dir hash the binary sees must match what the agent read", rep.PromptsHash, want)
if want := provenance.PromptsHash(); rep.PromptsHash != want {
t.Errorf("prompts_hash = %q, want the binary's prompt set (%q): the run dir hash the binary sees must match what the agent read", rep.PromptsHash, want)
}
if rep.Cost.Measured {
t.Error("cost is measured for a fixture agent that reports no spend")
Expand Down
3 changes: 2 additions & 1 deletion cmd/verdict/usage.go
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,8 @@ func usageText() string {
" downstream verbs address it as local-<name>; post and live calibrate refuse it, while\n" +
" calibrate --probe grades it offline.\n" +
" --show-prompts (fetch and fetch-local) renders the assembled agent prompt —\n" +
" PROCEDURE.md + prompts/system.md + prompts/review.md + prompts/diagrams.md (verbatim) + trust-boundary\n" +
" the binary's review Procedure (PROCEDURE.md + prompts/system.md + prompts/review.md +\n" +
" prompts/diagrams.md, verbatim) + trust-boundary\n" +
" prologue + nonce-fenced bundle meta + toc.txt + roles.txt + full.diff +\n" +
" kb.txt/sensitive.txt/panel when present (one fence per file, in that order) —\n" +
" to stdout (human; progress moves to stderr) or data.prompt_dump (--json).\n" +
Expand Down
7 changes: 4 additions & 3 deletions docs/ci-recipe.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,10 @@ into your repo as `.github/workflows/verdict-review.yml`.
agent: it sets `VERDICT_AGENT` in the step's `env:` (one of `claude`, `codex`,
`cursor`, `opencode` — the shipped `run --agent` table) and the `--agent`
argument that reads it. Install that CLI on the runner in an earlier step;
the recipe never downloads it. The agent reads `full.diff` via `toc.txt`
(plus `PR.json` and `roles.txt`; it reads the Procedure from the binary with
`verdict skill`, then `verdict skill show <path>`) and writes exactly
the recipe never downloads it. The agent reads `.verdict/state/<pr>/procedure.md`
first — the binary's authoritative Procedure for this review (an installed
skill may be older) — then `full.diff` via `toc.txt` (plus `PR.json` and
`roles.txt`), and writes exactly
one `.verdict/state/<pr>/review.json` (`schema_version: 1`, `body`,
`approved`, `comments[]`, optional `resolved[]`/`triage[]`). The step fails
unless that file is fresh: `run` refuses `no-review-artifact` when the agent
Expand Down
43 changes: 25 additions & 18 deletions docs/manual.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ verdict init --preset standard --yes
# 1. fetch
verdict fetch owner/repo 123

# 2. review with ANY agent (the installed skill is the prompt; add it with
# `verdict skill install --detect` if it is missing):
# Point the agent at .verdict/state/123/. It reads .verdict/state/123/full.diff fully via
# .verdict/state/123/toc.txt, loads prompts/review.md from its skill dir, and fans out
# 2. review with ANY agent (read .verdict/state/123/procedure.md first; it is
# the binary's authoritative Procedure for this review — an installed skill may be older):
# Point the agent at .verdict/state/123/. It reads .verdict/state/123/procedure.md first,
# then .verdict/state/123/full.diff fully via .verdict/state/123/toc.txt, and fans out
# 0-N lenses per prompts/reviewer-subagent.md for disposition-changing questions only.
# Write exactly one .verdict/state/123/review.json {schema_version: 1, body, approved, comments[], resolved?, triage?, sections?}.

Expand Down Expand Up @@ -319,7 +319,10 @@ refuses typed because its API has no documented resolve endpoint.
[--timeout s] [-- args...] [--post] [--dry-run] [--json]` plans one pass over the threads
waiting on verdict. It refreshes the feed, reads the login once, joins findings,
and writes `drain.json`: `pending[]`, typed `skipped[]` reasons, `other_threads`,
and a `plan_digest` over the pending list. A thread is pending when it is a
and a `plan_digest` over the pending list. The plan also writes
`drain.procedure.md`, the binary's task-scoped Procedure (the index, the
system preamble, and `prompts/address-reviews.md`), which the drain directive
names as authoritative. A thread is pending when it is a
review comment verdict authored, unresolved, and its last comment is a human
question with no ledger entry answering it; the oldest questions come first and
the rest are `deferred`. `--max-threads` defaults to 20. Zero pending is success
Expand All @@ -345,8 +348,10 @@ hosts refuse `provider-unimplemented` before any read. There is no
gates and secret scan, the live thread refresh and self login, the
stale/answered/resolved skips, mention neutralization, and the marker. It holds
the post lock and composes with `--agent` (plan → draft → preview). It calls no
reply or resolve endpoint and leaves `replies.json`, `drain.json`, and
`threads.json` byte-identical. The live read happens in a scratch bundle so
reply or resolve endpoint and leaves `replies.json`, `drain.json`, and `threads.json`
byte-identical, and an existing `drain.procedure.md` too; with `--agent`, a missing
`drain.procedure.md` is created for the drafting agent. The live read happens in a
scratch bundle so
the stored feed is not touched. Instead it writes `<bundle>/drain.dryrun.json`
with `planned[]` entries (`thread_id`, `root_comment_id`, `disposition`, and
the final `body` with its marker) and `skipped[]` reasons; the `--json` envelope
Expand All @@ -360,6 +365,7 @@ writes, and a dry-run preview is never a receipt.
### Bundle namespace (core files vs `x-*`)

A bundle dir holds these core/script artifacts (`PR.json`/`source.json`,
`procedure.md` (the binary's review Procedure; `drain.procedure.md` for a drain bundle),
`full.diff`, `toc.txt`, `roles.txt`, `review.json`, `findings.json`, `findings.dryrun.json`, `record.json`,
`post.json`, `store.json`, `hunks.tsv`, `reviews.json`, `inline.json`, `threads.json`,
`threads-resolve.json`, `drain.json`, `drain.dryrun.json`, `drain.replies.json`, `selected.json`,
Expand Down Expand Up @@ -1340,23 +1346,24 @@ writes — the bundle is byte-identical after the run, and the bare text path
never a warning.

`fetch`/`fetch-local` take `--show-prompts`: after the normal bundle write,
the script renders the assembled agent reading list — `PROCEDURE.md`,
`prompts/system.md`, `prompts/review.md`, `prompts/diagrams.md`, the bundle
the script renders the assembled agent reading list — the binary's embedded review
Procedure (`PROCEDURE.md`, `prompts/system.md`, `prompts/review.md`,
`prompts/diagrams.md`), the bundle
meta (`PR.json` or `source.json`), `toc.txt`, `roles.txt`, `full.diff`, plus
`kb.txt`, `lenses.txt`, `reverts.txt`, `sensitive.txt`, and `panel` when
present, in that order — trusted
prompts verbatim, then the trust-boundary prologue, then one nonce fence
per bundle file — to stdout (human mode; progress moves to stderr) or
`data.prompt_dump` (`--json`). No agent runs, no network beyond the fetch
itself, no store writes beyond the bundle: without the flag both verbs are
byte-identical. It reads the trusted prompts from the current directory, so
run it from a bundle or a checkout (it falls back to the checkout's `procedure/`) carrying `PROCEDURE.md` and `prompts/`. Missing
bundle files skip silently, but missing trusted
prompt files fail loudly: no `prompts/system.md` refuses typed
(`prompt-dump-failed` — the untrusted-data preamble lives there, so a dump
without it is unsafe), while a missing `PROCEDURE.md`, `prompts/review.md`, or
`prompts/diagrams.md` warns (stderr plus envelope `warnings[]`) and the dump
proceeds without it. Trusted prompts print verbatim first, then a
itself, no store writes beyond the bundle: the flag changes only the printed
dump, so the bundle is byte-identical with or without it. The trusted assembly
comes from the binary's embedded payload, never the working directory or a
`procedure/` checkout, so the dump starts with exactly the bytes written to
`<bundle>/procedure.md`. Missing bundle files skip silently, but a missing
embedded Procedure file fails loudly: any of `PROCEDURE.md`,
`prompts/system.md`, `prompts/review.md`, or `prompts/diagrams.md` refuses
typed `prompt-dump-failed`, names the path, and writes no partial
`procedure.md`. Trusted prompts print verbatim first, then a
tool-generated trust-boundary prologue naming the run's nonce, then every
present bundle file inside its own fence — the bundle meta (`PR.json` or
`source.json`), `toc.txt`, `roles.txt`, the diff (`full.diff`, or the
Expand Down
7 changes: 5 additions & 2 deletions docs/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -1088,9 +1088,12 @@ Fix: re-fetch a bundle with file sections (or drop the exclusions), then re-run.

## `prompt-dump-failed`

Cause: the prompt dump needs the skill and prompts sources.
Cause: an embedded review Procedure file is missing from the binary. The
message names the path (`PROCEDURE.md`, `prompts/system.md`,
`prompts/review.md`, or `prompts/diagrams.md`); the working directory cannot
supply a replacement, so no partial Procedure is written or printed.

Fix: run from a checkout carrying `PROCEDURE.md`/`prompts`, then re-run.
Fix: reinstall verdict from a signed release, then re-run.

## `provider-unimplemented`

Expand Down
Loading
Loading