diff --git a/.changeset/sta-445-bootstrap-skill.md b/.changeset/sta-445-bootstrap-skill.md new file mode 100644 index 000000000..b416488b2 --- /dev/null +++ b/.changeset/sta-445-bootstrap-skill.md @@ -0,0 +1,13 @@ +--- +"@stainless-code/verdict": minor +--- + +**Breaking changes** + +The installed skill is now a small Bootstrap skill: it only says when to use Verdict and how to reach the binary. The binary serves the full Procedure, so the instructions always match the version that checks your work. + +Reinstall the skill on every machine. For harness installs, for each installed agent, run `verdict skill status --agent ` and delete the old `verdict` skill directory at the path printed after `->`; then run `verdict skill install --agent `. `verdict skill install --detect` is fine instead of `--agent`. `verdict skill uninstall` removes only the new Bootstrap skill files, so it would leave the old Procedure files beside the stub. + +For registry installs, delete the old `verdict` skill directory, then re-run `npx skills add stainless-code/verdict`. + +Copies went stale between releases: the Procedure lived inside the installed skill, so an older copy could hand an agent instructions that disagreed with the binary grading its work. Keeping it in the binary removes that skew. diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 464c25992..591597f77 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -5,7 +5,7 @@ - [ ] `make evals` green (or the affected `./evals/one.sh ` while iterating, full suite at the end) - [ ] `make test-race` (race + shuffle) green; `gofmt -l .` empty - [ ] `golangci-lint run ./...` clean under the repo's `.golangci.yml` -- [ ] `bun fmt --check` on touched md/json/yml (never reflowed `skills/verdict/prompts/` examples) +- [ ] `bun fmt --check` on touched md/json/yml (never reflowed `procedure/prompts/` examples) - [ ] Changeset added (`.changeset/.md`) โ€” or docs-only / eval-only / comment-only, so none needed - [ ] Golden diff reviewed (`-update` regenerations explained line-by-line, committed separately from unrelated changes) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e7cb797e4..8edb08617 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -73,7 +73,7 @@ jobs: # exempt here and in hooks/pre-commit, enforced by review, not fmt. # Extensions are the set bun fmt parses (it hard-errors on others); # bun.lock etc. are skipped the same way `bun fmt --check .` skips them. - git ls-files -z '*.md' '*.mdx' '*.json' '*.ts' '*.js' '*.mjs' '*.cjs' '*.css' '*.html' '*.yml' '*.yaml' '*.webmanifest' ':!skills/verdict/prompts/**' | xargs -r -0 bun fmt --check -- + git ls-files -z '*.md' '*.mdx' '*.json' '*.ts' '*.js' '*.mjs' '*.cjs' '*.css' '*.html' '*.yml' '*.yaml' '*.webmanifest' ':!procedure/prompts/**' | xargs -r -0 bun fmt --check -- lint: name: ๐Ÿ•ต Lint diff --git a/AGENTS.md b/AGENTS.md index 563263e62..897ff4efb 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -43,9 +43,9 @@ skip-token ban and test determinism. Read both before writing or reviewing Go. argv a case did not register (`evals/README.md`, `evals/lib/hoststub.sh`). - **No `/tmp` test dirs.** Use `t.TempDir()` for files and `internal/testenv` for child processes; a hardcoded path leaks across parallel runs. -- **Exactly six skill files are hashed into goldens:** `skills/verdict/SKILL.md` - and `prompts/{system,review,diagrams,verifier,validate}.md`; a change moves - only `prompts_hash` lines and the literal pin in `provenance/prompts_test.go`. +- **Exactly six Procedure files are hashed into goldens:** `procedure/PROCEDURE.md` + and `procedure/prompts/{system,review,diagrams,verifier,validate}.md`; a change moves + only `prompts_hash` lines and the literal pin in `internal/provenance/prompts_test.go`. ## Git and commits @@ -73,7 +73,7 @@ skip-token ban and test determinism. Read both before writing or reviewing Go. - Before commit or push, run `bun fmt --check` on changed files when Bun is available; fix only non-example hunks. -- Never reflow `skills/verdict/prompts/` examples โ€” see `CONTRIBUTING.md` ยง Formatting. +- Never reflow `procedure/prompts/` examples โ€” see `CONTRIBUTING.md` ยง Formatting. - `gofmt -l .` must be empty, and shell files gate on `shellcheck -S warning` (see `hooks/pre-commit`). - `golangci-lint run ./...` must report 0 issues under `.golangci.yml`; no new diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ae53770c2..689e21a44 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -70,10 +70,15 @@ rename in the tree. The binary installs its own embedded skill copy with ``` verdict/ - skills/verdict/ # the skill payload copied by `npx skills add`: - SKILL.md # skill entry point (feed to any agent) + skills/verdict/ # the Bootstrap skill copied by `npx skills add`: + SKILL.md # entry point: where the Procedure lives, how to reach the binary + NOTICE.md # upstream license notice + scripts/verdict # POSIX shim: VERDICT_BIN, skill bin, PATH + procedure/ # the Procedure served only by the binary: + PROCEDURE.md # index of the task-scoped instructions prompts/ # system, review, incremental, address, fix-ci, build, ci, grade, diagrams, subagent - scripts/verdict # POSIX skill shim: VERDICT_BIN, bundled bin, PATH, then bootstrap + packs/ # built-in lens pack + references/ # shared review references schemas/ # published JSON Schemas, served by `verdict skill files` at schemas/ install.sh # verified binary installer (macOS/Linux release asset) install.ps1 # verified binary installer (Windows release asset) @@ -110,7 +115,7 @@ Every PR must clear all of these; CI enforces each one: `.golangci.yml` (CI runs the pinned action; zero issues, no new `//nolint`). 4. **Formatted.** `bun fmt --check .` before pushing; fix only what it - flags, and never reflow `skills/verdict/prompts/` examples. + flags, and never reflow `procedure/prompts/` examples. 5. **A changeset when the repo's practice requires one.** See [Versioning](#versioning). diff --git a/GLOSSARY.md b/GLOSSARY.md new file mode 100644 index 000000000..8317c70aa --- /dev/null +++ b/GLOSSARY.md @@ -0,0 +1,19 @@ +# Verdict + +Verdict is a local, agent-driven pull-request review toolkit: it hands a review agent the change, validates what the agent returns, and keeps an auditable trail. + +## Guidance + +**Procedure**: +The review instructions Verdict gives an agent: what to read, how to judge, how to tag severity and rigor, and how to self-check before recording. Only the Verdict binary serves it, from its own embedded copy, so it always matches the binary in use. +_Avoid_: skill content, prompts, instructions + +**Bootstrap skill**: +The small skill file installed into an agent harness. It names when to use Verdict and how to reach the Procedure through the binary, and carries no Procedure itself. +_Avoid_: skill copy, installed skill (when it means the Procedure) + +## Review + +**Bundle**: +Everything a single review reads, written under `.verdict/state//`: the change, its metadata, the handed context, the controls, and the Procedure for that review. +_Avoid_: workspace, state dir, snapshot diff --git a/NOTICE.md b/NOTICE.md index 919803128..bf45bc0b7 100644 --- a/NOTICE.md +++ b/NOTICE.md @@ -2,7 +2,7 @@ ## Adapted review prompts -The review prompts under `skills/verdict/prompts/` are adapted from +The review prompts under `procedure/prompts/` are adapted from [Pullfrog](https://github.com/pullfrog/pullfrog), an MIT-licensed open-source GitHub Action. The skill's review discipline additionally borrows from clawpatch and pstack, as described in `docs/vision.md`. All lineage is credited, not hidden. @@ -10,11 +10,12 @@ skill's review discipline additionally borrows from clawpatch and pstack, as des - Upstream repository: - Pinned upstream commit: `bef4f79cef8c9793608ef50179bae9e925231e38` -The skill directory ships its own copy of this notice, `skills/verdict/NOTICE.md`, because -`npx skills add` copies that directory alone: the adapted prompts have to carry the upstream -license with them, not leave it behind in this repository-root file. The two copies hold the same -license block, and `TestNoticeCoversAdaptedPaths` fails when they drift or when a prompt is -missing from the payload copy. +The Bootstrap skill ships its own copy of this notice, `skills/verdict/NOTICE.md`, because +`npx skills add` copies that directory alone. The adapted Procedure is embedded in the binary and +served from `procedure/`, but the notice travels with the installed copy so the upstream license +is never left behind in this repository-root file. The two copies hold the same license block, +and `TestNoticeCoversAdaptedPaths` fails when they drift or when a prompt is missing from the +payload copy. diff --git a/README.md b/README.md index c0cba7947..497ef4c1a 100644 --- a/README.md +++ b/README.md @@ -55,14 +55,14 @@ Check it with `verdict --version`. ### 2. Add the skill -The skill is the review prompt. It ships inside the binary. +The installed skill is a small Bootstrap skill: it names when to use Verdict and how to reach the binary. The binary serves the full Procedure through `verdict skill`, so the instructions always match the version that checks your work. ```bash verdict skill install --detect ``` `--detect` installs into every agent whose skills directory already exists and -is non-empty. To target one agent, run `verdict skill install --agent claude`. The agent names are in `verdict skill --help`. You can also use the registry: `npx skills add stainless-code/verdict`. Use one route per machine, or the skill loads twice. The registry route installs only the skill, so install the binary first: `install.sh` is not shipped in the npm tarball. After `verdict upgrade`, re-run the install to refresh the copy. +is non-empty. To target one agent, run `verdict skill install --agent claude`. The agent names are in `verdict skill --help`. The shim resolves `VERDICT_BIN`, then the skill's own `bin/verdict`, then `verdict` on `PATH`; it never fetches during a review. You can also use the registry: `npx skills add stainless-code/verdict`. Use one route per machine, or the skill loads twice. The registry route installs only the skill, so install the binary first: `install.sh` is not shipped in the npm tarball. After `verdict upgrade`, re-run the install to refresh the copy. ### 3. Review a PR diff --git a/apps/docs/content/01-docs/(guides)/agent-run-review.mdx b/apps/docs/content/01-docs/(guides)/agent-run-review.mdx index 6f761b8c3..d961eb089 100644 --- a/apps/docs/content/01-docs/(guides)/agent-run-review.mdx +++ b/apps/docs/content/01-docs/(guides)/agent-run-review.mdx @@ -73,8 +73,9 @@ verdict skill install --dir --json verdict skill status --dir --json ``` -Install reports `create` with the payload files (`SKILL.md`, `prompts/`, -`schemas/`, `scripts/verdict`); `status` reads `keep` once in sync. +Install reports `create` with the three Bootstrap skill files (`SKILL.md`, +`NOTICE.md`, `scripts/verdict`); `status` reads `keep` once in sync. The binary +serves the Procedure and the schemas through `verdict skill`. ## Refusals diff --git a/apps/docs/content/01-docs/(guides)/local-review.mdx b/apps/docs/content/01-docs/(guides)/local-review.mdx index ebe5b3bbb..a8f2cbff8 100644 --- a/apps/docs/content/01-docs/(guides)/local-review.mdx +++ b/apps/docs/content/01-docs/(guides)/local-review.mdx @@ -31,7 +31,8 @@ Use this when there is no pull request yet: a branch you are shaping, a spike, o Point any agent at the bundle: it reads `full.diff` through `toc.txt` (plus - `SKILL.md` and `prompts/`) and writes exactly one + the Procedure from the binary: `verdict skill`, then `verdict skill show `) + and writes exactly one `.verdict/state/local-mywork/review.json` with `schema_version` 1, `body`, `approved`, and `comments[]`. This is the agent's turn; there is no command to run until the file lands. diff --git a/apps/docs/content/01-docs/(guides)/setup-repo.mdx b/apps/docs/content/01-docs/(guides)/setup-repo.mdx index 1efaa26d6..750e4e072 100644 --- a/apps/docs/content/01-docs/(guides)/setup-repo.mdx +++ b/apps/docs/content/01-docs/(guides)/setup-repo.mdx @@ -56,9 +56,10 @@ this flow touches the network. verdict skill status --dir --json ``` - Install atomically writes `SKILL.md`, `prompts/`, `schemas/`, and - `scripts/verdict` plus a `.verdict-managed` receipt (status `create`); - `status` reads `keep` once every target is in sync. Prefer + Install atomically writes the three Bootstrap skill files (`SKILL.md`, + `NOTICE.md`, `scripts/verdict`) plus a `.verdict-managed` receipt (status + `create`); `status` reads `keep` once every target is in sync. The binary + serves the Procedure and the schemas through `verdict skill`. Prefer `--agent claude --agent codex` to target agents, or `--detect` for every agent whose skills dir already exists and is non-empty. The skill is embedded in the binary, so the served copy can never skew. diff --git a/apps/docs/content/01-docs/(start)/glossary.mdx b/apps/docs/content/01-docs/(start)/glossary.mdx index 18a728be4..55bca8f59 100644 --- a/apps/docs/content/01-docs/(start)/glossary.mdx +++ b/apps/docs/content/01-docs/(start)/glossary.mdx @@ -46,9 +46,10 @@ tools. Commands accept a unique short prefix. ## Surfaces -**Skill.** The instructions an agent loads to run a review: `SKILL.md` and -`prompts/`. The binary carries its own copy and the published `schemas/`; -`verdict skill install` writes them into an agent's skills directory. +**Bootstrap skill.** The small `SKILL.md` installed into an agent harness: it +names when to use Verdict and how to reach the binary. The binary carries the +Procedure (`PROCEDURE.md`, `prompts/`, `packs/`, `references/`) and the published +`schemas/`; `verdict skill install` writes only the Bootstrap skill. **Envelope.** The one JSON object a `--json` run prints to stdout. It has nine fixed keys and `schema_version: "1"`. New information goes inside `data`. See diff --git a/apps/docs/content/02-agents/index.mdx b/apps/docs/content/02-agents/index.mdx index 85ed7a84f..4cef7f324 100644 --- a/apps/docs/content/02-agents/index.mdx +++ b/apps/docs/content/02-agents/index.mdx @@ -19,7 +19,7 @@ verdict skill install --agent claude # write it verdict skill status --dir # keep, or drift with an update list ``` -The install writes 28 files atomically โ€” `SKILL.md`, ten review prompts, the JSON schemas, `scripts/verdict` โ€” plus a `.verdict-managed` receipt, and refuses a directory it did not create unless you pass `--force`. It never prompts and makes no network call. Re-run it after upgrading the binary so the copy never skews from the CLI. +The install writes three files atomically โ€” `SKILL.md`, `NOTICE.md`, `scripts/verdict` โ€” plus a `.verdict-managed` receipt, and refuses a directory it did not create unless you pass `--force`. It never prompts and makes no network call. The binary serves the Procedure and the schemas through `verdict skill`. Re-run the install after a breaking stub change. ## The MCP server meets the host where it is diff --git a/bench/runner/bench_test.go b/bench/runner/bench_test.go index fd7baa670..94fb03955 100644 --- a/bench/runner/bench_test.go +++ b/bench/runner/bench_test.go @@ -838,7 +838,7 @@ func checkPromptProvenance(t *testing.T, cfg *Config, rep *Report) { if got, want := rep.PromptSHA["off-by-one"], hex.EncodeToString(sum[:]); got != want { t.Errorf("prompt sha256 = %q, want the shipped plus derived bytes (%q)", got, want) } - if _, err := os.Stat(filepath.Join(cfg.WorkDir, prefixPrompts, "SKILL.md")); err != nil { + if _, err := os.Stat(filepath.Join(cfg.WorkDir, prefixPrompts, "PROCEDURE.md")); err != nil { t.Errorf("the run dir has no prompt set: %v", err) } } diff --git a/bench/runner/bundle.go b/bench/runner/bundle.go index 9768ef136..dfb039df9 100644 --- a/bench/runner/bundle.go +++ b/bench/runner/bundle.go @@ -192,7 +192,7 @@ func repoState(ctx context.Context, root string) (string, bool) { } published := []string{ filepath.Join("bench", "corpus"), - filepath.Join(prefixPrompts, "SKILL.md"), + filepath.Join(prefixPrompts, "PROCEDURE.md"), filepath.Join(prefixPrompts, "prompts"), } status, err := gitOut(ctx, root, append([]string{"status", "--porcelain", "--"}, published...)...) diff --git a/bench/runner/prompt.go b/bench/runner/prompt.go index 2b2dde9a8..de44cc104 100644 --- a/bench/runner/prompt.go +++ b/bench/runner/prompt.go @@ -20,7 +20,7 @@ const ( prefixName = "prefix" prefixDir = "prompt-prefix" prefixScaffold = "scaffold.txt" - prefixPrompts = "skills/verdict" + prefixPrompts = "procedure" fenceHeader = "<<>>\n" fenceFooter = "<<>>\n" trustBoundary = "--- verdict trust boundary (tool-generated, trusted) ---" @@ -44,7 +44,7 @@ const ( // promptFiles are the four trusted files every agent prompt opens with, in // the order the tool concatenates them. -var promptFiles = []string{"SKILL.md", "prompts/system.md", "prompts/review.md", "prompts/diagrams.md"} +var promptFiles = []string{"PROCEDURE.md", "prompts/system.md", "prompts/review.md", "prompts/diagrams.md"} // fenceLine matches the tool's untrusted-region header. The prompt files quote // the marker shape in prose with `nonce=` placeholders, so the match diff --git a/cmd/verdict/skill.go b/cmd/verdict/skill.go index 728af964e..180fc2e68 100644 --- a/cmd/verdict/skill.go +++ b/cmd/verdict/skill.go @@ -42,8 +42,7 @@ func runSkill(ctx context.Context, r *cli.Runner, args []string, env map[string] } } -// -//nolint:revive // the show route nests the path, inventory, and SKILL.md arms +// runSkillShow routes one show request to the path, inventory, or index arm. func runSkillShow(ctx context.Context, r *cli.Runner, args []string, env map[string]string, root string) int { flags, code, done := parseSkillShowFlags(r, args) if done { @@ -51,25 +50,40 @@ func runSkillShow(ctx context.Context, r *cli.Runner, args []string, env map[str } req := &skill.Request{Runtime: skillRuntime(r, env), Version: skillVersion(root)} if flags.path != "" { - body, ref := skill.ShowFile(ctx, req, flags.path) - if ref != nil { - return r.Refuse(ref) - } - if flags.format == formatJSONName { - writeSkillEnvelope(r, skillFileData(flags.path, body)) - return cli.ExitOK - } - fmt.Fprint(r.Out, string(body)) - return cli.ExitOK + return writeSkillPath(ctx, r, req, flags) } if flags.format == formatJSONName { - inv, ref := skill.List(ctx, req) - if ref != nil { - return r.Refuse(ref) - } - writeSkillEnvelope(r, skillInventoryData(&inv)) + return writeSkillInventory(ctx, r, req) + } + return writeSkillIndex(ctx, r, req) +} + +// writeSkillPath serves one inventory file as bytes or as a JSON envelope. +func writeSkillPath(ctx context.Context, r *cli.Runner, req *skill.Request, flags skillShowFlags) int { + body, ref := skill.ShowFile(ctx, req, flags.path) + if ref != nil { + return r.Refuse(ref) + } + if flags.format == formatJSONName { + writeSkillEnvelope(r, skillFileData(flags.path, body)) return cli.ExitOK } + fmt.Fprint(r.Out, string(body)) + return cli.ExitOK +} + +// writeSkillInventory serves the full served-path inventory as one envelope. +func writeSkillInventory(ctx context.Context, r *cli.Runner, req *skill.Request) int { + inv, ref := skill.List(ctx, req) + if ref != nil { + return r.Refuse(ref) + } + writeSkillEnvelope(r, skillInventoryData(&inv)) + return cli.ExitOK +} + +// writeSkillIndex serves the Procedure index as the default show body. +func writeSkillIndex(ctx context.Context, r *cli.Runner, req *skill.Request) int { body, ref := skill.Show(ctx, req) if ref != nil { return r.Refuse(ref) @@ -441,24 +455,28 @@ func writeSkillEnvelope(r *cli.Runner, data *jq.Obj) { func skillUsageText() string { return "usage: verdict skill [show []|files|status|install|uninstall] [opts]\n" + - " skill [show] print SKILL.md (default --format md)\n" + + " skill [show] print PROCEDURE.md, the Procedure index\n" + " skill show print one embedded payload file byte-identical\n" + " (paths come from `skill files`)\n" + " skill --format json one envelope: {name, version, files}\n" + " skill files list every embedded payload path\n" + " skill status [--dir |--agent ...|--detect] [--diff] [--json]\n" + " read-only drift report per target: create/update/keep/blocked, the\n" + - " receipt versus embedded version, and the changed files. Exit 0 when\n" + - " every target is in sync, 1 when drifted. --diff adds one capped\n" + - " unified diff per changed file (a truncated diff says so).\n" + + " installed stub_version versus the embedded one, the receipt versus\n" + + " embedded version, and the changed files. A matching stub_version\n" + + " forgives SKILL.md byte edits only; NOTICE.md and scripts/verdict\n" + + " drift when missing or changed, and a missing or different version\n" + + " reports SKILL.md. Exit 0 when every target is in sync, 1 when drifted.\n" + + " --diff adds one capped unified diff per changed file (truncated says so).\n" + " skill install [--dir |--agent ...|--detect] [--force] [--dry-run] [--json]\n" + - " atomically write SKILL.md, prompts/, schemas/, and scripts/verdict plus\n" + - " a .verdict-managed receipt. Previews create/update/keep/blocked;\n" + - " --agent repeats and installs each agent in argv order (deduped);\n" + - " --detect targets every agent below whose skills dir already exists and\n" + - " is non-empty, and lists each target's result (nothing detected refuses\n" + - " typed: name one with --agent). --detect cannot be combined with\n" + - " --dir or --agent.\n" + + " atomically write the Bootstrap skill: SKILL.md, NOTICE.md, and\n" + + " scripts/verdict, plus a .verdict-managed receipt. The Procedure and\n" + + " schemas stay in this binary; read them with `skill` and `skill show`.\n" + + " Previews create/update/keep/blocked; --agent repeats and installs each\n" + + " agent in argv order (deduped); --detect targets every agent below whose\n" + + " skills dir already exists and is non-empty, and lists each target's\n" + + " result (nothing detected refuses typed: name one with --agent).\n" + + " --detect cannot be combined with --dir or --agent.\n" + " only --json is guaranteed non-interactive (this command never prompts).\n" + " A target without a verdict receipt refuses typed unless --force;\n" + " foreign files are never touched.\n" + @@ -468,8 +486,10 @@ func skillUsageText() string { " refuses unless --force, and even then only inventory paths go;\n" + " --dry-run lists the plan and deletes nothing.\n" + " agents: agents, claude, codex, cursor, opencode, copilot, gemini, goose, qwen, windsurf\n" + - " The skill is embedded in this binary, so the served copy can never skew.\n" + - " In a skill copy, scripts/verdict resolves VERDICT_BIN, then /bin/verdict,\n" + - " then verdict on PATH; otherwise run /scripts/verdict bootstrap once\n" + - " (verified; never a review-time fetch).\n" + " The Procedure and the stub are embedded in this binary, so served bytes\n" + + " cannot skew.\n" + + " In a skill copy, scripts/verdict resolves VERDICT_BIN, then\n" + + " /bin/verdict, then verdict on PATH; otherwise run\n" + + " /scripts/verdict bootstrap once (verified; never a\n" + + " review-time fetch).\n" } diff --git a/cmd/verdict/skill_management_test.go b/cmd/verdict/skill_management_test.go index 2111e0d4f..4fdd33cc3 100644 --- a/cmd/verdict/skill_management_test.go +++ b/cmd/verdict/skill_management_test.go @@ -40,6 +40,21 @@ func skillTreeHash(t *testing.T, root string) string { return hex.EncodeToString(hash.Sum(nil)) } +// TestSkillShowServesProcedurePayload pins the on-disk source of a served +// prompt: the checkout's procedure file, not an installed skill copy. +func TestSkillShowServesProcedurePayload(t *testing.T) { + hermeticEnv(t) + setTestVersion(t) + want, err := os.ReadFile(filepath.Join("..", "..", "procedure", "prompts", "review.md")) + if err != nil { + t.Fatal(err) + } + code, out, derr := runCapture(t, []string{"skill", "show", "prompts/review.md"}) + if code != 0 || derr != "" || out != string(want) { + t.Fatalf("skill show prompts/review.md = exit %d stderr %q (%d bytes), want the checkout file", code, derr, len(out)) + } +} + // TestSkillShowPath pins the byte-identical path read and its one refusal. // //nolint:cyclop,revive // the test walks the path matrix @@ -54,9 +69,13 @@ func TestSkillShowPath(t *testing.T) { if code != 0 || out != string(want) || derr != "" { t.Fatalf("skill show prompts/review.md = exit %d stderr %q (stdout %d bytes)", code, derr, len(out)) } + procedure, err := skillfs.Read(skillfs.ProcedureMD) + if err != nil { + t.Fatal(err) + } code, out, _ = runCapture(t, []string{"skill", "show"}) - if code != 0 || out == "" { - t.Fatalf("bare skill show = exit %d (%d bytes)", code, len(out)) + if code != 0 || out != string(procedure) { + t.Fatalf("bare skill show = exit %d (%d bytes), want PROCEDURE.md", code, len(out)) } for _, path := range []string{"../SKILL.md", "/etc/passwd", "prompts/../SKILL.md", "nope.md"} { code, out, _ := runCapture(t, []string{"skill", "show", path, "--json"}) @@ -110,7 +129,30 @@ func TestSkillStatusCommand(t *testing.T) { t.Fatalf("fresh status = exit %d data %v", code, data) } - if err := os.WriteFile(filepath.Join(target, "SKILL.md"), []byte("hand edited\n"), 0o644); err != nil { + stubPath := filepath.Join(target, skillfs.SkillMD) + stubBody, err := os.ReadFile(stubPath) + if err != nil { + t.Fatal(err) + } + if err := os.WriteFile(stubPath, append(stubBody, []byte("\nhand edited\n")...), 0o644); err != nil { + t.Fatal(err) + } + code, out, _ = runCapture(t, []string{"skill", "status", "--dir", dir, "--json"}) + doc = decodeSkillEnvelope(t, out) + data = skillData(t, doc) + if code != 0 || data["in_sync"] != true { + t.Fatalf("SKILL.md body edit status = exit %d data %v", code, data) + } + + stub, err := os.ReadFile(stubPath) + if err != nil { + t.Fatal(err) + } + edited := strings.Replace(string(stub), "stub_version: "+skillfs.StubVersion, "stub_version: 2", 1) + if edited == string(stub) { + t.Fatal("installed stub has no stub_version to edit") + } + if err := os.WriteFile(filepath.Join(target, "SKILL.md"), []byte(edited), 0o644); err != nil { t.Fatal(err) } before := skillTreeHash(t, target) @@ -118,7 +160,7 @@ func TestSkillStatusCommand(t *testing.T) { doc = decodeSkillEnvelope(t, out) data = skillData(t, doc) if code != 1 || data["in_sync"] != false || data["status"] != "update" { - t.Fatalf("edited status = exit %d data %v", code, data) + t.Fatalf("version edit status = exit %d data %v", code, data) } diffs, ok := data["diff"].([]any) if !ok || len(diffs) != 1 { @@ -128,7 +170,7 @@ func TestSkillStatusCommand(t *testing.T) { if !ok || entry["path"] != "SKILL.md" || entry["truncated"] != false { t.Fatalf("diff entry = %v", diffs[0]) } - if body, ok := entry["diff"].(string); !ok || !strings.Contains(body, "-hand edited") { + if body, ok := entry["diff"].(string); !ok || !strings.Contains(body, "-stub_version: 2") { t.Fatalf("diff body = %v", entry["diff"]) } if after := skillTreeHash(t, target); after != before { @@ -136,7 +178,7 @@ func TestSkillStatusCommand(t *testing.T) { } code, out, _ = runCapture(t, []string{"skill", "status", "--dir", dir, "--diff"}) - if code != 1 || !strings.Contains(out, "status: drift") || !strings.Contains(out, "-hand edited") { + if code != 1 || !strings.Contains(out, "status: drift") || !strings.Contains(out, "-stub_version: 2") { t.Fatalf("human status = exit %d\n%s", code, out) } diff --git a/cmd/verdict/skill_test.go b/cmd/verdict/skill_test.go index 2a1f96600..9698c976c 100644 --- a/cmd/verdict/skill_test.go +++ b/cmd/verdict/skill_test.go @@ -47,15 +47,12 @@ func skillData(t *testing.T, doc map[string]any) map[string]any { func TestSkillShowAndFiles(t *testing.T) { hermeticEnv(t) setTestVersion(t) - want, err := skillfs.Read(skillfs.SkillMD) + want, err := skillfs.Read(skillfs.ProcedureMD) if err != nil { t.Fatal(err) } if len(want) == 0 { - t.Fatal("embedded SKILL.md is empty") - } - if !strings.HasPrefix(string(want), "---\nname: verdict\n") { - t.Fatalf("embedded SKILL.md lost its frontmatter: %q", string(want[:min(len(want), 80)])) + t.Fatal("embedded PROCEDURE.md is empty") } code, out, derr := runCapture(t, []string{"skill"}) if code != 0 || out != string(want) || derr != "" { diff --git a/cmd/verdict/usage.go b/cmd/verdict/usage.go index c93507a00..41a943096 100644 --- a/cmd/verdict/usage.go +++ b/cmd/verdict/usage.go @@ -44,7 +44,7 @@ func usageText() string { " downstream verbs address it as local-; post and live calibrate refuse it, while\n" + " calibrate --probe grades it offline.\n" + " --show-prompts (fetch and fetch-local) renders the assembled agent prompt โ€”\n" + - " SKILL.md + prompts/system.md + prompts/review.md + prompts/diagrams.md (verbatim) + trust-boundary\n" + + " PROCEDURE.md + prompts/system.md + prompts/review.md + prompts/diagrams.md (verbatim) + trust-boundary\n" + " prologue + nonce-fenced bundle meta + toc.txt + roles.txt + full.diff +\n" + " kb.txt/sensitive.txt/panel when present (one fence per file, in that order) โ€”\n" + " to stdout (human; progress moves to stderr) or data.prompt_dump (--json).\n" + @@ -469,16 +469,19 @@ func usageText() string { " skill [show []|files|status|install|uninstall] [--format md|json] [--dir |--agent ...]\n" + " [--detect] [--diff] [--force] [--dry-run] [--json]\n" + " serve, inspect, or remove the skill embedded in this binary (never a skew):\n" + - " show prints SKILL.md (default --format md), show prints one\n" + + " show prints PROCEDURE.md (default --format md), show prints one\n" + " embedded file byte-identical, --format json emits one envelope with\n" + " name/version/files, files lists every embedded path.\n" + " status is a read-only drift report per target (create/update/keep/\n" + - " blocked) with receipt vs embedded version; it exits 0 in sync and 1\n" + - " when drifted. --diff adds a capped unified diff per changed file.\n" + - " install atomically writes SKILL.md, prompts/, schemas/, and\n" + - " scripts/verdict plus a .verdict-managed receipt; it previews\n" + - " create/update/keep/blocked, never prompts, and refuses a foreign\n" + - " target unless --force (foreign files are never touched).\n" + + " blocked) comparing the installed stub_version and the receipt against\n" + + " this binary; it exits 0 in sync and 1 when drifted. A matching\n" + + " stub_version forgives SKILL.md byte edits only; NOTICE.md and\n" + + " scripts/verdict still drift when missing or changed. A missing or\n" + + " different version reports SKILL.md. --diff adds a capped unified diff.\n" + + " install atomically writes SKILL.md, NOTICE.md, and scripts/verdict\n" + + " plus a .verdict-managed receipt; the Procedure and schemas stay in the\n" + + " binary. It previews create/update/keep/blocked, never prompts, and\n" + + " refuses a foreign target unless --force (foreign files are untouched).\n" + " uninstall removes only the payload paths the receipt owns, then the\n" + " receipt; foreign files survive, a target without a receipt refuses\n" + " unless --force, and --dry-run deletes nothing.\n" + @@ -487,8 +490,9 @@ func usageText() string { " agent whose skills dir already exists and is non-empty; with none\n" + " present it refuses typed and asks for --agent .\n" + " In a skill copy, scripts/verdict resolves VERDICT_BIN, then\n" + - " /bin/verdict, then verdict on PATH; otherwise run the one-time\n" + - " verified /scripts/verdict bootstrap (no review-time fetch).\n" + + " /bin/verdict, then verdict on PATH; otherwise run the\n" + + " one-time verified\n" + + " /scripts/verdict bootstrap (no review-time fetch).\n" + " pack [--json]\n" + " list: read-only lens-pack inventory in precedence order: built-in, org\n" + " (explicit packs[] paths in personal settings), and repo\n" + diff --git a/docs/ci-recipe.md b/docs/ci-recipe.md index f4c138f86..9a610ac3a 100644 --- a/docs/ci-recipe.md +++ b/docs/ci-recipe.md @@ -24,7 +24,8 @@ into your repo as `.github/workflows/verdict-review.yml`. `cursor`, `opencode` โ€” the shipped `run --agent` table) and the `--agent` argument that reads it. Install that CLI on the runner in an earlier step; the recipe never downloads it. The agent reads `full.diff` via `toc.txt` - (plus `PR.json`, `roles.txt`, `SKILL.md` + `prompts/`) and writes exactly + (plus `PR.json` and `roles.txt`; it reads the Procedure from the binary with + `verdict skill`, then `verdict skill show `) and writes exactly one `.verdict/state//review.json` (`schema_version: 1`, `body`, `approved`, `comments[]`, optional `resolved[]`/`triage[]`). The step fails unless that file is fresh: `run` refuses `no-review-artifact` when the agent diff --git a/docs/go-rules.md b/docs/go-rules.md index ce19ebde0..926de11f9 100644 --- a/docs/go-rules.md +++ b/docs/go-rules.md @@ -86,7 +86,7 @@ this repo, and the repo files stay the truth for what is gated: - `os.Root` confinement; `crypto/rand` secrets; validated edge input (GO-SEC-01/02/04). โ€” partly: `lint:depguard` (`math/rand`), `lint:gosec`, `lint:nosprintfhostport`; `os.Root` is review-only, and operator-supplied bundle paths are exempt by decision. - Exit-code contract; `--json/--jq` exporters; generated completions/manpages (GO-CLI-02/06/07). โ€” partly: exit codes are frozen constants in `internal/cli` with tests; no generated completions or manpages. - Start std-`flag` under a zero-dep policy; graduate to cobra when justified (GO-CLI-08). โ€” partly: `go.mod` carries no third-party requirements; argv parsing is hand-rolled in `internal/cli` to keep the frozen contract byte-identical. -- `embed` prompts; stamped `--version` + `-trimpath` builds (GO-CLI-09). โ€” partly: `skillfs.go` embeds `skills/verdict/` (`SKILL.md`, `prompts`, `scripts/verdict`) and the root `schemas/`, serving both at payload paths; `make build` and goreleaser pass `-trimpath`. +- `embed` prompts; stamped `--version` + `-trimpath` builds (GO-CLI-09). โ€” partly: `skillfs.go` embeds `skills/verdict/` (the Bootstrap skill), `procedure/` (`PROCEDURE.md`, `prompts`, `packs`, `references`) and the root `schemas/`, serving all three at payload paths; `make build` and goreleaser pass `-trimpath`. - flags > env > file > defaults; report secret sources; lenient config reads (GO-CONFIG-01/02/03). โ€” partly: `internal/cli` precedence + `internal/cli/flags_test.go`. - Advisory `flock` across read-modify-write; JSON schema libs where beyond `DisallowUnknownFields` (GO-ATOMIC-02, GO-JSON-04); tag linters apply to new types only, legacy wire fields frozen (GO-JSON-03). โ€” partly: `lint:musttag,tagalign`; the rest is review-only. - Progress as stderr attrs + JSON error shapes; log after flag parsing (GO-OBS-01/02). โ€” partly: envelope error-shape goldens. diff --git a/docs/manual.md b/docs/manual.md index 53dbc6eb5..92ab60ad0 100644 --- a/docs/manual.md +++ b/docs/manual.md @@ -417,7 +417,7 @@ list); the other providers contribute the title, body, and head ref. A lens pack is data, not a verb: `pack.json` names each lens and `lenses/.md` carries its body. The built-in pack ships at -`skills/verdict/packs/builtin/` (`name` `verdict/builtin`, `version` `1.0.0`) +`procedure/packs/builtin/` (`name` `verdict/builtin`, `version` `1.0.0`) and `fetch`/`fetch-local` load it from the embedded skill payload. The format is strict JSON (unknown fields refuse), versioned by `schema_version: "1"`, and published as `schemas/pack.schema.json`. The fixed layout is `pack.json`, @@ -1340,7 +1340,7 @@ writes โ€” the bundle is byte-identical after the run, and the bare text path never a warning. `fetch`/`fetch-local` take `--show-prompts`: after the normal bundle write, -the script renders the assembled agent reading list โ€” `SKILL.md`, +the script renders the assembled agent reading list โ€” `PROCEDURE.md`, `prompts/system.md`, `prompts/review.md`, `prompts/diagrams.md`, the bundle meta (`PR.json` or `source.json`), `toc.txt`, `roles.txt`, `full.diff`, plus `kb.txt`, `lenses.txt`, `reverts.txt`, `sensitive.txt`, and `panel` when @@ -1350,11 +1350,11 @@ per bundle file โ€” to stdout (human mode; progress moves to stderr) or `data.prompt_dump` (`--json`). No agent runs, no network beyond the fetch itself, no store writes beyond the bundle: without the flag both verbs are byte-identical. It reads the trusted prompts from the current directory, so -run it from an installed skill dir or a checkout (it falls back to the checkout's `skills/verdict/`) carrying `SKILL.md` and `prompts/`. Missing +run it from a bundle or a checkout (it falls back to the checkout's `procedure/`) carrying `PROCEDURE.md` and `prompts/`. Missing bundle files skip silently, but missing trusted prompt files fail loudly: no `prompts/system.md` refuses typed (`prompt-dump-failed` โ€” the untrusted-data preamble lives there, so a dump -without it is unsafe), while a missing `SKILL.md`, `prompts/review.md`, or +without it is unsafe), while a missing `PROCEDURE.md`, `prompts/review.md`, or `prompts/diagrams.md` warns (stderr plus envelope `warnings[]`) and the dump proceeds without it. Trusted prompts print verbatim first, then a tool-generated trust-boundary prologue naming the run's nonce, then every @@ -2694,9 +2694,10 @@ you stand). The shipped `verdict` is one static Go binary (`make build` writes `bin/verdict`) that needs no shell wrapper. `VERDICT_BIN` is not the binary's: the skill shim and the TypeScript client both consult it. The shim -(`skills/verdict/scripts/verdict`) resolves it first, then `/bin/verdict`, -then `verdict` on `PATH`; `resolveBin` in `clients/ts` takes it ahead of the -platform package. A plain `verdict` call consults neither variable. +(`skills/verdict/scripts/verdict`) resolves it first, then +`/bin/verdict`, then `verdict` on `PATH`; `resolveBin` in `clients/ts` +takes it ahead of the platform package. A plain `verdict` call consults neither +variable. - **Binary** (`install.sh` / `install.ps1`, the default): the script fetches one release archive, `checksums.txt`, `commit.txt`, and @@ -2766,8 +2767,9 @@ platform package. A plain `verdict` call consults neither variable. `rm -rf `, โ€ฆ). `--dry-run` and `--json` are supported. - **Registry** (`npx skills add stainless-code/verdict`): the primary path is the GLOBAL binary (curl/npm/brew above), which the skill copy resolves on - PATH through `scripts/verdict` (`VERDICT_BIN`, then `/bin/verdict`, - then `verdict`). `npx skills` is scoped to the skills dir and never + PATH through `scripts/verdict` (`VERDICT_BIN`, then + `/bin/verdict`, then `verdict`). + `npx skills` is scoped to the skills dir and never installs the binary; `install.sh` is not shipped in the npm tarball and never runs here. With no global binary, run the one-time verified `sh /verdict/scripts/verdict bootstrap`: it fetches the pinned @@ -2780,13 +2782,14 @@ platform package. A plain `verdict` call consults neither variable. `--detect`/`--all`, doctor-gated). Updates via `git pull`; `install-skill.sh` refuses outside a git checkout. `hooks/pre-commit` wiring applies to the checkout only. -- **Embedded skill** (`verdict skill`): the binary carries its own `SKILL.md`, - `prompts/`, `packs/`, and `scripts/verdict` from `skills/verdict/`, plus - `schemas/` from the repository root, so the served skill can never skew from - the binary. The skills CLI +- **Embedded skill** (`verdict skill`): the binary carries the Bootstrap skill + from `skills/verdict/` and the Procedure from `procedure/` (prompts, packs, + references, and `PROCEDURE.md`) plus `schemas/` from the repository root, so + every served file matches the binary that grades the review. The skills CLI (`npx skills add stainless-code/verdict`) copies only `skills/verdict/`; - `verdict skill install` writes that payload and the served `schemas/` files. - `verdict skill` prints `SKILL.md` (default + `verdict skill install` writes just the stub's three files โ€” `SKILL.md`, + `NOTICE.md`, `scripts/verdict` โ€” and a receipt, never the Procedure. + `verdict skill` prints `PROCEDURE.md` (default `--format md`); `--format json` is one standard envelope with `name`/`version`/`files`; `verdict skill files` lists every embedded path. `verdict skill show ` prints one inventory file byte-identical (a @@ -2796,14 +2799,19 @@ platform package. A plain `verdict` call consults neither variable. `skill files`. `verdict skill status [--dir |--agent ...|--detect] [--diff] [--json]` is the read-only drift check: it writes nothing and - reports each target's create/update/keep/blocked result plus the receipt - versus embedded version, reusing the install plan. It exits 0 when every + reports each target's create/update/keep/blocked result, comparing the + installed `stub_version` with the one served by this binary and the receipt + with the embedded version. A matching `stub_version` forgives `SKILL.md` + byte edits only: `NOTICE.md` and `scripts/verdict` still drift when missing + or changed, and a missing or different `stub_version` reports `SKILL.md`. + Every drifted file appears in `Changed`, so `--diff` shows the shim too. + It exits 0 when every target is in sync and 1 when drifted (a typed refusal still exits 1), so CI can gate on it without `install --dry-run`. `--diff` adds one unified diff per changed file, capped at 64 KiB overall with an explicit truncation marker and `truncated: true` in the JSON entry. `verdict skill install` (flags: `--dir `, `--agent `, - `--detect`, `--force`, `--dry-run`, `--json`) writes the payload plus a + `--detect`, `--force`, `--dry-run`, `--json`) writes the three stub files plus a `.verdict-managed` receipt (file-atomic, receipt last), previews create/update/keep/blocked (`--dry-run --json` stays a drift preview: `update` means the installed copy is stale, `keep` means current, `blocked` needs a @@ -2830,7 +2838,7 @@ platform package. A plain `verdict` call consults neither variable. absent or empty. `verdict skill uninstall [--dir |--agent ...] [--force] [--dry-run] [--json]` deletes only the inventory paths the receipt owns and - then the receipt, pruning the payload directories it empties; every foreign + then the receipt, pruning the stub directories it empties; every foreign file survives. A target that is a symlink refuses (`skill-target-file`), and an inventory path that escapes the target through a symlinked directory refuses (`skill-foreign-conflict`) before any file inside it is removed. A diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 3e28378ad..d0c59424a 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -1090,7 +1090,7 @@ Fix: re-fetch a bundle with file sections (or drop the exclusions), then re-run. Cause: the prompt dump needs the skill and prompts sources. -Fix: run from a checkout carrying `SKILL.md`/`prompts`, then re-run. +Fix: run from a checkout carrying `PROCEDURE.md`/`prompts`, then re-run. ## `provider-unimplemented` diff --git a/evals/README.md b/evals/README.md index 5a87f4d87..52c0a5453 100644 --- a/evals/README.md +++ b/evals/README.md @@ -1,6 +1,6 @@ # Offline contract evals -Smoke-grade graders for the doctrine and verb contracts in `SKILL.md` and +Smoke-grade graders for the doctrine and verb contracts in `procedure/PROCEDURE.md` and the `verdict` binary: exactly-one terminal write, the no-review-note rule, `--bind` downgrade behavior, and the `--json` envelope shape (`schema_version: "1"`). They complement calibration (which grades judgment @@ -134,7 +134,7 @@ the build. | `fetch-reverts` | `fetch` writes the bundle's `reverts.txt` from the tracked ledger | A real `fetch-local` over a two-file diff against a planted `.verdict/reverts.jsonl`: the exact-path row and the row whose stored path is the `pkg/` directory prefix land as two fenced blocks sorted by sha, the row on another path stays out (and a later fetch that matches nothing removes the file, while an absent ledger warns about nothing), `data.artifacts["reverts.txt"]` and `provenance.json` both name the file, a rerun is byte-identical, a hostile subject (backticks, `