Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .changeset/sta-445-bootstrap-skill.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
"@stainless-code/verdict": minor
---

**Breaking changes**

The installed skill is now a small Bootstrap skill: it only says when to use Verdict and how to reach the binary. The binary serves the full Procedure, so the instructions always match the version that checks your work.

Reinstall the skill on every machine. For harness installs, for each installed agent, run `verdict skill status --agent <x>` and delete the old `verdict` skill directory at the path printed after `->`; then run `verdict skill install --agent <x>`. `verdict skill install --detect` is fine instead of `--agent`. `verdict skill uninstall` removes only the new Bootstrap skill files, so it would leave the old Procedure files beside the stub.

For registry installs, delete the old `verdict` skill directory, then re-run `npx skills add stainless-code/verdict`.

Copies went stale between releases: the Procedure lived inside the installed skill, so an older copy could hand an agent instructions that disagreed with the binary grading its work. Keeping it in the binary removes that skew.
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
- [ ] `make evals` green (or the affected `./evals/one.sh <case>` while iterating, full suite at the end)
- [ ] `make test-race` (race + shuffle) green; `gofmt -l .` empty
- [ ] `golangci-lint run ./...` clean under the repo's `.golangci.yml`
- [ ] `bun fmt --check` on touched md/json/yml (never reflowed `skills/verdict/prompts/` examples)
- [ ] `bun fmt --check` on touched md/json/yml (never reflowed `procedure/prompts/` examples)
- [ ] Changeset added (`.changeset/<name>.md`) — or docs-only / eval-only / comment-only, so none needed
- [ ] Golden diff reviewed (`-update` regenerations explained line-by-line, committed separately from unrelated changes)

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ jobs:
# exempt here and in hooks/pre-commit, enforced by review, not fmt.
# Extensions are the set bun fmt parses (it hard-errors on others);
# bun.lock etc. are skipped the same way `bun fmt --check .` skips them.
git ls-files -z '*.md' '*.mdx' '*.json' '*.ts' '*.js' '*.mjs' '*.cjs' '*.css' '*.html' '*.yml' '*.yaml' '*.webmanifest' ':!skills/verdict/prompts/**' | xargs -r -0 bun fmt --check --
git ls-files -z '*.md' '*.mdx' '*.json' '*.ts' '*.js' '*.mjs' '*.cjs' '*.css' '*.html' '*.yml' '*.yaml' '*.webmanifest' ':!procedure/prompts/**' | xargs -r -0 bun fmt --check --

lint:
name: 🕵 Lint
Expand Down
8 changes: 4 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,9 @@ skip-token ban and test determinism. Read both before writing or reviewing Go.
argv a case did not register (`evals/README.md`, `evals/lib/hoststub.sh`).
- **No `/tmp` test dirs.** Use `t.TempDir()` for files and `internal/testenv`
for child processes; a hardcoded path leaks across parallel runs.
- **Exactly six skill files are hashed into goldens:** `skills/verdict/SKILL.md`
and `prompts/{system,review,diagrams,verifier,validate}.md`; a change moves
only `prompts_hash` lines and the literal pin in `provenance/prompts_test.go`.
- **Exactly six Procedure files are hashed into goldens:** `procedure/PROCEDURE.md`
and `procedure/prompts/{system,review,diagrams,verifier,validate}.md`; a change moves
only `prompts_hash` lines and the literal pin in `internal/provenance/prompts_test.go`.

## Git and commits

Expand Down Expand Up @@ -73,7 +73,7 @@ skip-token ban and test determinism. Read both before writing or reviewing Go.

- Before commit or push, run `bun fmt --check` on changed files when Bun is
available; fix only non-example hunks.
- Never reflow `skills/verdict/prompts/` examples — see `CONTRIBUTING.md` § Formatting.
- Never reflow `procedure/prompts/` examples — see `CONTRIBUTING.md` § Formatting.
- `gofmt -l .` must be empty, and shell files gate on `shellcheck -S warning`
(see `hooks/pre-commit`).
- `golangci-lint run ./...` must report 0 issues under `.golangci.yml`; no new
Expand Down
13 changes: 9 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,15 @@ rename in the tree. The binary installs its own embedded skill copy with

```
verdict/
skills/verdict/ # the skill payload copied by `npx skills add`:
SKILL.md # skill entry point (feed to any agent)
skills/verdict/ # the Bootstrap skill copied by `npx skills add`:
SKILL.md # entry point: where the Procedure lives, how to reach the binary
NOTICE.md # upstream license notice
scripts/verdict # POSIX shim: VERDICT_BIN, skill bin, PATH
procedure/ # the Procedure served only by the binary:
PROCEDURE.md # index of the task-scoped instructions
prompts/ # system, review, incremental, address, fix-ci, build, ci, grade, diagrams, subagent
scripts/verdict # POSIX skill shim: VERDICT_BIN, bundled bin, PATH, then bootstrap
packs/ # built-in lens pack
references/ # shared review references
schemas/ # published JSON Schemas, served by `verdict skill files` at schemas/
install.sh # verified binary installer (macOS/Linux release asset)
install.ps1 # verified binary installer (Windows release asset)
Expand Down Expand Up @@ -110,7 +115,7 @@ Every PR must clear all of these; CI enforces each one:
`.golangci.yml` (CI runs the pinned action; zero issues, no new
`//nolint`).
4. **Formatted.** `bun fmt --check .` before pushing; fix only what it
flags, and never reflow `skills/verdict/prompts/` examples.
flags, and never reflow `procedure/prompts/` examples.
5. **A changeset when the repo's practice requires one.** See
[Versioning](#versioning).

Expand Down
19 changes: 19 additions & 0 deletions GLOSSARY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Verdict

Verdict is a local, agent-driven pull-request review toolkit: it hands a review agent the change, validates what the agent returns, and keeps an auditable trail.

## Guidance

**Procedure**:
The review instructions Verdict gives an agent: what to read, how to judge, how to tag severity and rigor, and how to self-check before recording. Only the Verdict binary serves it, from its own embedded copy, so it always matches the binary in use.
_Avoid_: skill content, prompts, instructions

**Bootstrap skill**:
The small skill file installed into an agent harness. It names when to use Verdict and how to reach the Procedure through the binary, and carries no Procedure itself.
_Avoid_: skill copy, installed skill (when it means the Procedure)

## Review

**Bundle**:
Everything a single review reads, written under `.verdict/state/<target>/`: the change, its metadata, the handed context, the controls, and the Procedure for that review.
_Avoid_: workspace, state dir, snapshot
13 changes: 7 additions & 6 deletions NOTICE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,20 @@

## Adapted review prompts

The review prompts under `skills/verdict/prompts/` are adapted from
The review prompts under `procedure/prompts/` are adapted from
[Pullfrog](https://github.com/pullfrog/pullfrog), an MIT-licensed open-source GitHub Action. The
skill's review discipline additionally borrows from clawpatch and pstack, as described in
`docs/vision.md`. All lineage is credited, not hidden.

- Upstream repository: <https://github.com/pullfrog/pullfrog>
- Pinned upstream commit: `bef4f79cef8c9793608ef50179bae9e925231e38`

The skill directory ships its own copy of this notice, `skills/verdict/NOTICE.md`, because
`npx skills add` copies that directory alone: the adapted prompts have to carry the upstream
license with them, not leave it behind in this repository-root file. The two copies hold the same
license block, and `TestNoticeCoversAdaptedPaths` fails when they drift or when a prompt is
missing from the payload copy.
The Bootstrap skill ships its own copy of this notice, `skills/verdict/NOTICE.md`, because
`npx skills add` copies that directory alone. The adapted Procedure is embedded in the binary and
served from `procedure/`, but the notice travels with the installed copy so the upstream license
is never left behind in this repository-root file. The two copies hold the same license block,
and `TestNoticeCoversAdaptedPaths` fails when they drift or when a prompt is missing from the
payload copy.

<!-- upstream-license:start -->

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,14 +55,14 @@ Check it with `verdict --version`.

### 2. Add the skill

The skill is the review prompt. It ships inside the binary.
The installed skill is a small Bootstrap skill: it names when to use Verdict and how to reach the binary. The binary serves the full Procedure through `verdict skill`, so the instructions always match the version that checks your work.

```bash
verdict skill install --detect
```

`--detect` installs into every agent whose skills directory already exists and
is non-empty. To target one agent, run `verdict skill install --agent claude`. The agent names are in `verdict skill --help`. You can also use the registry: `npx skills add stainless-code/verdict`. Use one route per machine, or the skill loads twice. The registry route installs only the skill, so install the binary first: `install.sh` is not shipped in the npm tarball. After `verdict upgrade`, re-run the install to refresh the copy.
is non-empty. To target one agent, run `verdict skill install --agent claude`. The agent names are in `verdict skill --help`. The shim resolves `VERDICT_BIN`, then the skill's own `bin/verdict`, then `verdict` on `PATH`; it never fetches during a review. You can also use the registry: `npx skills add stainless-code/verdict`. Use one route per machine, or the skill loads twice. The registry route installs only the skill, so install the binary first: `install.sh` is not shipped in the npm tarball. After `verdict upgrade`, re-run the install to refresh the copy.

### 3. Review a PR

Expand Down
5 changes: 3 additions & 2 deletions apps/docs/content/01-docs/(guides)/agent-run-review.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -73,8 +73,9 @@ verdict skill install --dir <skills-dir> --json
verdict skill status --dir <skills-dir> --json
```

Install reports `create` with the payload files (`SKILL.md`, `prompts/`,
`schemas/`, `scripts/verdict`); `status` reads `keep` once in sync.
Install reports `create` with the three Bootstrap skill files (`SKILL.md`,
`NOTICE.md`, `scripts/verdict`); `status` reads `keep` once in sync. The binary
serves the Procedure and the schemas through `verdict skill`.

## Refusals

Expand Down
3 changes: 2 additions & 1 deletion apps/docs/content/01-docs/(guides)/local-review.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ Use this when there is no pull request yet: a branch you are shaping, a spike, o

<Step title="Have an agent review it">
Point any agent at the bundle: it reads `full.diff` through `toc.txt` (plus
`SKILL.md` and `prompts/`) and writes exactly one
the Procedure from the binary: `verdict skill`, then `verdict skill show <path>`)
and writes exactly one
`.verdict/state/local-mywork/review.json` with `schema_version` 1, `body`,
`approved`, and `comments[]`. This is the agent's turn; there is no command to
run until the file lands.
Expand Down
7 changes: 4 additions & 3 deletions apps/docs/content/01-docs/(guides)/setup-repo.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -56,9 +56,10 @@ this flow touches the network.
verdict skill status --dir <skills-dir> --json
```

Install atomically writes `SKILL.md`, `prompts/`, `schemas/`, and
`scripts/verdict` plus a `.verdict-managed` receipt (status `create`);
`status` reads `keep` once every target is in sync. Prefer
Install atomically writes the three Bootstrap skill files (`SKILL.md`,
`NOTICE.md`, `scripts/verdict`) plus a `.verdict-managed` receipt (status
`create`); `status` reads `keep` once every target is in sync. The binary
serves the Procedure and the schemas through `verdict skill`. Prefer
`--agent claude --agent codex` to target agents, or `--detect` for every
agent whose skills dir already exists and is non-empty. The skill is
embedded in the binary, so the served copy can never skew.
Expand Down
7 changes: 4 additions & 3 deletions apps/docs/content/01-docs/(start)/glossary.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,10 @@ tools. Commands accept a unique short prefix.

## Surfaces

**Skill.** The instructions an agent loads to run a review: `SKILL.md` and
`prompts/`. The binary carries its own copy and the published `schemas/`;
`verdict skill install` writes them into an agent's skills directory.
**Bootstrap skill.** The small `SKILL.md` installed into an agent harness: it
names when to use Verdict and how to reach the binary. The binary carries the
Procedure (`PROCEDURE.md`, `prompts/`, `packs/`, `references/`) and the published
`schemas/`; `verdict skill install` writes only the Bootstrap skill.

**Envelope.** The one JSON object a `--json` run prints to stdout. It has nine
fixed keys and `schema_version: "1"`. New information goes inside `data`. See
Expand Down
2 changes: 1 addition & 1 deletion apps/docs/content/02-agents/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ verdict skill install --agent claude # write it
verdict skill status --dir <skills-dir> # keep, or drift with an update list
```

The install writes 28 files atomically — `SKILL.md`, ten review prompts, the JSON schemas, `scripts/verdict` — plus a `.verdict-managed` receipt, and refuses a directory it did not create unless you pass `--force`. It never prompts and makes no network call. Re-run it after upgrading the binary so the copy never skews from the CLI.
The install writes three files atomically — `SKILL.md`, `NOTICE.md`, `scripts/verdict` — plus a `.verdict-managed` receipt, and refuses a directory it did not create unless you pass `--force`. It never prompts and makes no network call. The binary serves the Procedure and the schemas through `verdict skill`. Re-run the install after a breaking stub change.

## The MCP server meets the host where it is

Expand Down
2 changes: 1 addition & 1 deletion bench/runner/bench_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -838,7 +838,7 @@ func checkPromptProvenance(t *testing.T, cfg *Config, rep *Report) {
if got, want := rep.PromptSHA["off-by-one"], hex.EncodeToString(sum[:]); got != want {
t.Errorf("prompt sha256 = %q, want the shipped plus derived bytes (%q)", got, want)
}
if _, err := os.Stat(filepath.Join(cfg.WorkDir, prefixPrompts, "SKILL.md")); err != nil {
if _, err := os.Stat(filepath.Join(cfg.WorkDir, prefixPrompts, "PROCEDURE.md")); err != nil {
t.Errorf("the run dir has no prompt set: %v", err)
}
}
Expand Down
2 changes: 1 addition & 1 deletion bench/runner/bundle.go
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ func repoState(ctx context.Context, root string) (string, bool) {
}
published := []string{
filepath.Join("bench", "corpus"),
filepath.Join(prefixPrompts, "SKILL.md"),
filepath.Join(prefixPrompts, "PROCEDURE.md"),
filepath.Join(prefixPrompts, "prompts"),
}
status, err := gitOut(ctx, root, append([]string{"status", "--porcelain", "--"}, published...)...)
Expand Down
4 changes: 2 additions & 2 deletions bench/runner/prompt.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ const (
prefixName = "prefix"
prefixDir = "prompt-prefix"
prefixScaffold = "scaffold.txt"
prefixPrompts = "skills/verdict"
prefixPrompts = "procedure"
fenceHeader = "<<<UNTRUSTED nonce=%s source=%s>>>\n"
fenceFooter = "<<<END UNTRUSTED nonce=%s>>>\n"
trustBoundary = "--- verdict trust boundary (tool-generated, trusted) ---"
Expand All @@ -44,7 +44,7 @@ const (

// promptFiles are the four trusted files every agent prompt opens with, in
// the order the tool concatenates them.
var promptFiles = []string{"SKILL.md", "prompts/system.md", "prompts/review.md", "prompts/diagrams.md"}
var promptFiles = []string{"PROCEDURE.md", "prompts/system.md", "prompts/review.md", "prompts/diagrams.md"}

// fenceLine matches the tool's untrusted-region header. The prompt files quote
// the marker shape in prose with `nonce=<hex>` placeholders, so the match
Expand Down
Loading
Loading