Skip to content

refactor(skill): serve the Procedure from the binary behind a thin Bootstrap skill - #182

Merged
SutuSebastian merged 15 commits into
mainfrom
agent/sta-445-1b-stub
Nov 18, 2026
Merged

SutuSebastian merged 15 commits into
mainfrom
agent/sta-445-1b-stub

Conversation

@SutuSebastian

@SutuSebastian SutuSebastian commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The installed skill becomes a thin Bootstrap skill. The Procedure moves into the binary, so the instructions an agent reads always match the binary that checks its work (ADR-0006, STA-445 PR 1b).

 skills/verdict/            # what `skill install` writes
   SKILL.md                 # 13,427 B → 1,407 B, stub_version: 1
   NOTICE.md
   scripts/verdict
-  prompts/ (13)  references/ (2)  packs/ (4)
 procedure/                 # new: embedded, served by the binary
+  PROCEDURE.md  prompts/  references/  packs/
agent reads SKILL.md (Bootstrap skill)
  scripts/verdict          # VERDICT_BIN → <skill>/bin/verdict → PATH → missing-go-binary
  verdict skill            # Procedure index
  verdict skill show <path>

The shim never runs a binary from the reviewed checkout, such as a committed node_modules/.bin/verdict. A hostile PR cannot supply its own judge.

TestBootstrapSkillStaysThin keeps the stub from growing back into a Procedure. It checks:

  • the stub stays at 2 KiB or less;
  • the frontmatter has name, description and an integer stub_version;
  • the only verbs it mentions are verdict skill and verdict post;
  • it has no review vocabulary (severity, rigor, confidence, anchor, chunks) and no review.schema.json property names.

GLOSSARY.md at the repo root defines Procedure, Bootstrap skill and Bundle.

Evidence

  • Before: skill install copied 22 files, including the full Procedure. A copy went stale as soon as the binary changed.
    After: skill install --dir writes SKILL.md, NOTICE.md, scripts/verdict and .verdict-managed. verdict skill show prompts/review.md is byte-identical to procedure/prompts/review.md.
  • Shim: the eval shim ignores a binary committed to the reviewed tree exits 1 with missing-go-binary and empty stdout, with a committed node_modules/.bin/verdict present.
  • Stub guard: each TestBootstrapSkillStaysThin rule was mutation-checked and fails when broken.
  • Status: skill status reads stub_version from SKILL.md and still compares NOTICE.md and the shim byte for byte. A deleted shim exits 1, and a SKILL.md body edit exits 0. A six-case table covers this and fails when the old clear-all behavior is restored.
  • Reinstall: on an existing install, the changeset's recipe (delete the skill directory, then install) leaves exactly the four stub files, and status exits 0. uninstall now owns only the stub files, so it cannot clear the old Procedure.
  • Verdict dogfood review (fetch-local → run --agent pi → record → report --check, all local) found the status bug and the reinstall bug; both are fixed here.
  • Ladder at f76d63214, all exit 0:
    • make test-race
    • make evals (142 passed)
    • doc-coverage
    • make vet
    • Windows go vet
    • make deadcode
    • make cross
    • golangci-lint (native and Windows)
    • schemas-check
    • ts-types-check
    • clients-ts
    • make size
    • gofmt
    • clean tree

Merge Danger

Door: two-way

The change is a revert away. No stored data or published schema changes.

Blast Radius: installed-skills

Existing installs keep the old full skill until they reinstall. The changeset is listed under Breaking changes and gives a reinstall that clears the old directory. The run/drain directive still says to read the installed skill; PR 2 moves it to the bundle's procedure.md. linux/arm64 grows by 65,536 B, one 64 KiB page-alignment step.

@changeset-bot

changeset-bot Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: f76d632

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@stainless-code/verdict Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 210 files, which is 110 over the limit of 100.

To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.

Check out review usage here.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 03f4a32e-c84b-41ba-8ad9-47f68f8f73d4
Commits

Reviewing files that changed from the base of the PR and between 178ae11 and f76d632.

Files selected for processing: 210

  • .changeset/sta-445-bootstrap-skill.md
  • .github/PULL_REQUEST_TEMPLATE.md
  • .github/workflows/ci.yml
  • AGENTS.md
  • CONTRIBUTING.md
  • GLOSSARY.md
  • NOTICE.md
  • README.md
  • apps/docs/content/01-docs/(guides)/agent-run-review.mdx
  • apps/docs/content/01-docs/(guides)/local-review.mdx
  • apps/docs/content/01-docs/(guides)/setup-repo.mdx
  • apps/docs/content/01-docs/(start)/glossary.mdx
  • apps/docs/content/02-agents/index.mdx
  • bench/runner/bench_test.go
  • bench/runner/bundle.go
  • bench/runner/prompt.go
  • cmd/verdict/skill.go
  • cmd/verdict/skill_management_test.go
  • cmd/verdict/skill_test.go
  • cmd/verdict/usage.go
  • docs/ci-recipe.md
  • docs/go-rules.md
  • docs/manual.md
  • docs/troubleshooting.md
  • evals/README.md
  • evals/cases/chunk-envelope.sh
  • evals/cases/chunk-incremental.sh
  • evals/cases/chunk-local.sh
  • evals/cases/chunk-synthesize.sh
  • evals/cases/fix-prompt-pins.sh
  • evals/cases/install-routes.sh
  • evals/cases/prompt-dump-summary.sh
  • evals/cases/prompt-injection.sh
  • evals/cases/publish-diagram.sh
  • evals/cases/record-normalize.sh
  • evals/cases/review-provenance.sh
  • evals/cases/sandbox-recipe-docs.sh
  • evals/cases/skill-route.sh
  • evals/cases/skill-show.sh
  • evals/cases/skill-status.sh
  • evals/cases/skill-uninstall.sh
  • evals/cases/tarball-contract.sh
  • evals/cases/terminal-write.sh
  • evals/lib/common.sh
  • hooks/pre-commit
  • internal/archtest/agent_sources_test.go
  • internal/archtest/bootstrap_skill_test.go
  • internal/archtest/ci_fastpath_test.go
  • internal/archtest/docdrift_test.go
  • internal/archtest/lens_dispatch_test.go
  • internal/archtest/prompt_schema_test.go
  • internal/core/fetch/envelope.go
  • internal/core/fetch/lenses_test.go
  • internal/core/fetch/local.go
  • internal/core/fetch/promptdump.go
  • internal/core/fetch/promptdump_layout_test.go
  • internal/core/fetch/service_test.go
  • internal/core/pack/pack.go
  • internal/core/pack/pack_test.go
  • internal/core/skill/detect.go
  • internal/core/skill/panel_test.go
  • internal/core/skill/service.go
  • internal/core/skill/service_test.go
  • internal/core/skill/status.go
  • internal/core/skill/status_test.go
  • internal/core/skill/uninstall.go
  • internal/core/skill/uninstall_test.go
  • internal/fetch/golden_test.go
  • internal/fetch/testdata/author-advisory.golden
  • internal/fetch/testdata/azure-happy.golden
  • internal/fetch/testdata/base-sha-json.golden
  • internal/fetch/testdata/base-sha-note.golden
  • internal/fetch/testdata/bitbucket-happy.golden
  • internal/fetch/testdata/bitbucket-paging.golden
  • internal/fetch/testdata/budget-over-consented.golden
  • internal/fetch/testdata/budget-unknown.golden
  • internal/fetch/testdata/budget-unreadable.golden
  • internal/fetch/testdata/budget-unset.golden
  • internal/fetch/testdata/budget-within.golden
  • internal/fetch/testdata/cap-override-beats-chunk.golden
  • internal/fetch/testdata/cap-override.golden
  • internal/fetch/testdata/controls-base-absent.golden
  • internal/fetch/testdata/dump-json.golden
  • internal/fetch/testdata/dump-text.golden
  • internal/fetch/testdata/empty-diff.golden
  • internal/fetch/testdata/file-paging.golden
  • internal/fetch/testdata/forgejo-happy.golden
  • internal/fetch/testdata/gitcafe-base-note.golden
  • internal/fetch/testdata/gitcafe-happy.golden
  • internal/fetch/testdata/gitlab-base-note.golden
  • internal/fetch/testdata/gitlab-base.golden
  • internal/fetch/testdata/gitlab-happy.golden
  • internal/fetch/testdata/gitlab-text.golden
  • internal/fetch/testdata/happy-json.golden
  • internal/fetch/testdata/happy-text.golden
  • internal/fetch/testdata/local-branch.golden
  • internal/fetch/testdata/local-dump-chunk.golden
  • internal/fetch/testdata/local-dump-text.golden
  • internal/fetch/testdata/local-paths-empty.golden
  • internal/fetch/testdata/local-paths.golden
  • internal/fetch/testdata/local-range.golden
  • internal/fetch/testdata/local-range3.golden
  • internal/fetch/testdata/local-staged.golden
  • internal/fetch/testdata/local-stash.golden
  • internal/fetch/testdata/local-untracked.golden
  • internal/fetch/testdata/local-worktree-text.golden
  • internal/fetch/testdata/local-worktree.golden
  • internal/fetch/testdata/over-cap-chunk-text.golden
  • internal/fetch/testdata/over-cap-chunk.golden
  • internal/fetch/testdata/over-cap-knob.golden
  • internal/fetch/testdata/ratelimit-retry.golden
  • internal/fetch/testdata/sensitive-kb-head.golden
  • internal/fetch/testdata/sensitive-kb.golden
  • internal/fetch/testdata/since-alias.golden
  • internal/fetch/testdata/threads-degraded.golden
  • internal/golden/harness.go
  • internal/panel/panel.go
  • internal/provenance/prompts_test.go
  • internal/provenance/provenance.go
  • internal/provenance/provenance_test.go
  • internal/provenance/receipt_test.go
  • internal/record/testdata/body/mermaid-stripped.golden
  • internal/record/testdata/body/non-ascii.golden
  • internal/record/testdata/body/receipt-collision.golden
  • internal/record/testdata/chunk/drift.golden
  • internal/record/testdata/chunk/unattested.golden
  • internal/record/testdata/chunk/unknown-chunk-claim.golden
  • internal/record/testdata/chunk/valid.golden
  • internal/record/testdata/claims/verify-triage-resolved/rereview.golden
  • internal/record/testdata/degenerate/basis-bad.golden
  • internal/record/testdata/degenerate/basis-ok.golden
  • internal/record/testdata/degenerate/blob-link-local.golden
  • internal/record/testdata/degenerate/brief-valid.golden
  • internal/record/testdata/degenerate/carried-footer.golden
  • internal/record/testdata/degenerate/confidence-valid.golden
  • internal/record/testdata/degenerate/fenced-body-section.golden
  • internal/record/testdata/degenerate/missing-marker-ok.golden
  • internal/record/testdata/degenerate/normalized-fenced.golden
  • internal/record/testdata/degenerate/normalized-prose.golden
  • internal/record/testdata/degenerate/normalized-weak-shapes.golden
  • internal/record/testdata/incremental/non-touching/rereview.golden
  • internal/record/testdata/incremental/reopen/rereview.golden
  • internal/record/testdata/kb/absent.golden
  • internal/record/testdata/kb/bad-id-shape.golden
  • internal/record/testdata/kb/bad-policy-env.golden
  • internal/record/testdata/kb/bad-shape.golden
  • internal/record/testdata/kb/broken-store.golden
  • internal/record/testdata/kb/counter-adds/rereview.golden
  • internal/record/testdata/kb/duplicate.golden
  • internal/record/testdata/kb/empty.golden
  • internal/record/testdata/kb/false-positive/rereview.golden
  • internal/record/testdata/kb/joins-idempotent/rereview.golden
  • internal/record/testdata/kb/no-store.golden
  • internal/record/testdata/kb/null.golden
  • internal/record/testdata/kb/retired.golden
  • internal/record/testdata/kb/string-counters/rereview.golden
  • internal/record/testdata/kb/truncated.golden
  • internal/record/testdata/kb/unknown.golden
  • internal/record/testdata/kb/valid.golden
  • internal/record/testdata/matrix/n10/first.golden
  • internal/record/testdata/matrix/n10/rereview.golden
  • internal/record/testdata/matrix/n200/first.golden
  • internal/record/testdata/matrix/n200/rereview.golden
  • internal/record/testdata/matrix/n50/first.golden
  • internal/record/testdata/matrix/n50/rereview.golden
  • internal/record/testdata/overrides/dir-as-fragment.golden
  • internal/record/testdata/overrides/eq-threshold.golden
  • internal/record/testdata/overrides/valid.golden
  • internal/record/testdata/provider/gitlab.golden
  • internal/record/testdata/refuse/empty-prior.golden
  • internal/record/testdata/refuse/false-prior.golden
  • internal/record/testdata/refuse/invalid-utf8-review.golden
  • internal/record/testdata/refuse/null-prior.golden
  • internal/record/testdata/source/dirty-head.golden
  • internal/record/testdata/source/dirty-string.golden
  • internal/record/testdata/uncite/null-keeps/rereview.golden
  • internal/record/testdata/verify/contradict-exit/rereview.golden
  • internal/record/testdata/verify/contradict-timeout/rereview.golden
  • internal/record/testdata/verify/on-addressed/rereview.golden
  • internal/schemagen/overlay.json
  • internal/schemagen/semantics_test.go
  • internal/schemagen/testdata/baseline/pack.schema.json
  • notice_test.go
  • pkg/verdict/sdk_fetch_test.go
  • procedure/PROCEDURE.md
  • procedure/packs/builtin/lenses/api-contract.md
  • procedure/packs/builtin/lenses/perf.md
  • procedure/packs/builtin/lenses/test.md
  • procedure/packs/builtin/pack.json
  • procedure/prompts/address-reviews.md
  • procedure/prompts/build.md
  • procedure/prompts/ci.md
  • procedure/prompts/diagrams.md
  • procedure/prompts/fix-ci.md
  • procedure/prompts/grade.md
  • procedure/prompts/incremental-review.md
  • procedure/prompts/review.md
  • procedure/prompts/reviewer-subagent.md
  • procedure/prompts/rule-packs.md
  • procedure/prompts/system.md
  • procedure/prompts/validate.md
  • procedure/prompts/verifier.md
  • procedure/references/commands.md
  • procedure/references/panel.md
  • schemas/pack.schema.json
  • scripts/size-baseline.txt
  • skillfs.go
  • skillfs_test.go
  • skills/verdict/NOTICE.md
  • skills/verdict/SKILL.md

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@SutuSebastian
SutuSebastian merged commit 5407576 into main Nov 18, 2026
21 checks passed
@SutuSebastian
SutuSebastian deleted the agent/sta-445-1b-stub branch November 18, 2026 00:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant