diff --git a/.abcd/development/plans/2026-07-11-iss35-semantic-release-gate.md b/.abcd/development/plans/2026-07-11-iss35-semantic-release-gate.md new file mode 100644 index 00000000..9720e82e --- /dev/null +++ b/.abcd/development/plans/2026-07-11-iss35-semantic-release-gate.md @@ -0,0 +1,110 @@ +# iss-35 semantic release gate — SOTA-verified design (Option C, task 5) + +**Status:** design recorded 2026-07-11, maintainer signed off "full SOTA incl. +signing". Reached via the [prefer-sota](../principles/prefer-sota.md) process: +an adversary challenged generic release-engineering SOTA for repo fit, then a +`sota-researcher` verdict was taken on the fit-surviving hypothesis. This doc is +the design of record for the semantic (Direction-A) half of the iss-35 +graduation — the deterministic half shipped as the `surface_coverage` record-lint +rule. + +## The decision + +Wire the LLM brief↔surface cross-check (the semantic pass that verifies brief +*prose* still matches *binary behaviour* — flags, exit codes, schema fields, +counts) as a standing, enforced pre-tag release gate. It cannot run in CI +(GitHub Actions has no model access); the repo's enforcement plane is +deliberately split — deterministic gates in CI (`release.yml`), semantic gates +in the host/preflight environment where a model is present. + +## Why this form (adversary → SOTA verdict) + +The adversary rejected, each for a named repo preference: + +- **policy-as-code (OPA/Rego)** — a new language + runtime (ask-first) and a + second lint engine beside the one Go engine the repo paid to consolidate onto. +- **release-please / semantic-release** — bot commits to a branch, which + `release.yml`'s no-branch-commit tripwire fails closed on; and it duplicates + the adr-31 derived-versioning engine. +- **LLM-judge-in-CI** — violates host-delegated-by-default, needs a model key in + a tag-triggered workflow (trust-boundary expansion), and is a non-convergent + (flaky) blocking gate. +- **a naive `make release-gate`** — manufactures a false green: deterministic + gates pass, the model gate silently no-ops, exit 0. + +The `sota-researcher` verdict on the fit-surviving hypothesis: it is a +**Verification Summary Attestation (VSA)** — a named SLSA pattern. The VSA spec +*explicitly* decouples the verifier from the builder and sanctions verification +"at build time, at upload time, at download time, or via continuous monitoring" +by "a separate trusted party" — a host-sited verifier emitting a receipt that +release-time *checks* is the canonical VSA topology, not a workaround. The +closest exemplar is SLSA AMPEL's `prerelease` policy (validates prior VSAs +rather than re-running checks). So the design is SOTA-aligned; the only delta +from full SOTA is signing the receipt for unforgeability — which this repo +already has the primitive for (it runs SLSA build-provenance attestation in +`release.yml`). + +## The design + +Four parts, from the verdict's ranked recommendations (R1–R4): + +1. **Semantic-pass receipt = a VSA-shaped predicate** (R1, R3). A JSON record + keyed to the release commit digest, using the VSA field shape so it is + recognisable and upgradeable: `subject.digest` (the commit sha), + `verifier` (identity), `timeVerified`, `verificationResult` (PROMOTE/HOLD), + `policy`/detector version, plus the eval-standard fields — pinned + **judge-model snapshot** (e.g. `claude-opus-4-8`, never a floating alias), + judge-prompt hash, brief version, per-category result, failing-example ids. +2. **Fail-closed verification before tag** (R1, R4). A Go `record-lint` rule + (the existing engine, zero new deps) asserts a receipt exists for `HEAD`'s sha + **with `verificationResult: PROMOTE`**; absent or HOLD ⇒ BLOCK. The *gate* is + deterministic (receipt present + verdict) even though the *judge* that + produced it is stochastic — the stochastic judgement stays upstream in the + host, never re-derived at tag time. +3. **Runbook↔`release.yml` lockstep** (R2). The dev-facing pre-tag runbook and + the CI workflow must both stay human-authored (neither is a clean projection + of the other), so a consistency lint fails when their gate lists diverge — + the repo's golden-check anti-drift pattern (`gofmt -l`, `go generate` + + `git diff --exit-code`). +4. **Signed receipt (unforgeability)** (R1 caveat, maintainer chose full SOTA). + A committed sha-keyed file is trust-on-write — any committer can forge a + PROMOTE. Sign the receipt with the SLSA attestor already used in + `release.yml` (Sigstore/cosign-class), and verify the signature in-Go at gate + time — never with a Rego/CUE policy (that re-imports the rejected + new-language cost). + +## Homes (host-agnostic, dev-tier) + +- **Detector** (the LLM cross-check workflow, host-run, never CI): committed as + a dev-record artifact under `.abcd/development/` (excluded from the release + artifact), sanitised first — drop the hardcoded absolute local path + (privacy rule) and the stale skills→commands ground-truth (abcd ships zero + skills). It is invoked host-side via the agent harness, not `make`/CI. +- **Receipts**: sha-keyed, under a committed work-tier path (candidate + `.abcd/work/reviews/`), subject to the RD001–003 reviews charter. +- **Runbook**: a maintainer how-to **outside `docs/`** (developer-facing → + `.abcd/development/`), host-agnostic (docs-lint applies), referencing + `release.yml` as the source of truth for the deterministic half and owning + only the semantic gates. + +## Phased build (each phase its own gate) + +1. Sanitise + commit the detector to its dev-tier home; author the runbook + (owns semantic gates, references CI for the rest). No enforcement yet — mark + the not-yet-armed enforcement per [loud-staging](../principles/loud-staging.md). +2. Receipt schema (VSA-shaped, R3 fields) + the fail-closed `record-lint` + verification rule (TDD, mirror `surface_coverage`); arm it and watch it fire + on a missing/HOLD receipt. +3. Runbook↔`release.yml` lockstep consistency lint (TDD). +4. Receipt signing + in-Go signature verification; wire the release-time check + into `release.yml`. + +**Trust boundary:** phases 2–4 touch signing, attestation verification, and the +release workflow — the `security-reviewer` agent runs before presenting, and a +BLOCK verdict stops the change (per AGENTS.md). + +## What this is NOT + +Not an in-CI LLM judge, not policy-as-code, not a release-automation tool, not a +`make` target that re-runs the judge. The judge runs once, host-side; the gate +verifies its signed evidence. diff --git a/.abcd/development/principles/prefer-sota.md b/.abcd/development/principles/prefer-sota.md new file mode 100644 index 00000000..18156f98 --- /dev/null +++ b/.abcd/development/principles/prefer-sota.md @@ -0,0 +1,69 @@ +# Prefer SOTA + +**The rule.** The state of the art is the default bar for every design, +tooling, or practice decision: where a choice has a credible SOTA answer, that +answer is the presumptive pick. But "it is SOTA" is never on its own sufficient +warrant to adopt. Before adoption, SOTA is challenged for *fit* against this +repo's stated preferences by an adversarial reviewer whose sole job is to find +where the generic best practice collides with a deliberate local constraint. +Only the SOTA that survives that challenge — or is re-selected in light of it — +is adopted. When the best option is not obvious, the sequence is fixed and +ordered: adversarial fit-challenge first, then a SOTA verdict that already +accounts for the challenge, then the pick. + +**Why.** abcd is an *opinionated* configuration layer for development — its +value is the set of considered defaults it holds and propagates, not raw +novelty. SOTA is the right starting bar because it encodes the field's +accumulated learning cheaply; refusing it reinvents. But adopting SOTA +unfiltered is the opposite failure: a generic best practice can directly +violate a load-bearing local choice (host-delegated LLM work, minimal +dependencies, host-agnostic prose, a single source of truth). Running the +fit-challenge *before* the verdict is what stops "it's SOTA" from laundering a +preference violation — the adversary supplies the repo's own objections as an +input to the recommendation, so the verdict is fit-aware rather than fit-blind. +This is the opinion in "opinionated configuration": abcd does not chase novelty, +and it does not ossify — it takes the field's best and earns each adoption +against what the repo already believes. + +**Bounds.** + +- Binds on genuine forks — a design, tooling, or practice decision where the + best option is not obvious. Trivial or fully-specified choices with a + conventional default state the assumption in one line and proceed; they do + not summon the process. +- The adversary challenges *fit*, not SOTA's abstract correctness: its output + is the enumerated places generic SOTA would trample a *named* local + preference, plus the surviving-fit hypothesis. The SOTA verdict then ranks + options already filtered through that challenge; it never reports raw SOTA as + the answer. +- A local preference can itself be wrong. When the challenge exposes a + constraint that survives only by inertia, the response is to revisit it + deliberately — an ADR under [`../decisions`](../decisions) — never to + silently keep it and never to silently break it. +- Composes with [evaluator-outside-the-loop](evaluator-outside-the-loop.md) + (the challenger is independent of whoever proposes the option) and + [verifier-selects-gates-decide](verifier-selects-gates-decide.md): the SOTA + verdict is a proposal; the human's adoption is the gate. + +**Scope.** This is a propagated default, not a repo-local habit — it holds for +every abcd-managed repo, as one of the opinionated conventions abcd installs. +Its home in a managed repo is the marked working-conventions section abcd writes +into that repo's `AGENTS.md`; abcd's own record carries the normative statement +here. + +**Live instance.** Choosing how to wire the iss-35 semantic cross-check as a +release gate: an adversarial reviewer challenged generic release-engineering +SOTA (policy-as-code, release-automation tooling, an LLM judge in CI, an +executable gate target) against this repo's host-delegated-LLM, +minimal-dependency, single-source-of-truth, host-agnostic preferences, and only +the fit-surviving form was taken to a SOTA verdict for adoption. + +**Promotion.** MVP: a documented decision protocol — this principle, plus the +standing model-routing note naming which agents fill the challenger and +researcher roles. Tool: an abcd verb that scaffolds the adversary → SOTA-verdict +pass for a decision and records the challenge and verdict as a dated research +note beside the ADR it informs, so the fit-filter is a cheap repeatable step +rather than a remembered discipline. The managed-repo propagation path (the +`AGENTS.md` conventions section abcd installs) is itself the tool rung for +reach: the principle ships as an abcd default rather than being re-derived per +repo.