Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion MEMORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,22 @@ domain decisions.

## Current release status (2026-08-07)

- **v0.4.1 is the canonical release.** All five public packages are on npm at
- **v0.5.0 is prepared on `main`, not yet published.** PR #61 fixed a real
gap in `@changesafe/core`'s currently-published `0.4.1`: a domain adapter
could skip any universal policy (not just the two the shape permits) with
zero enforcement outside the app's own registration path, and a skip's
claimed replacement was never verified to exist. Every receipt now carries
`policyCoverage`. PR #60 added Kubernetes support to `changesafe eval` and
fixed a prompt-injection surface (label/selector values leaking into the
trusted preamble) and an eval-accuracy gap (unvalidated rollback
references) found in review. `CORE_POLICY_VERSION` moves to `core-v0.2.0`,
`TERRAFORM_POLICY_VERSION` to `terraform-v0.2.0` — see
`docs/RELEASE_NOTES_v0.5.0.md`. All nine workspace packages are bumped to
`0.5.0` on `main`; publishing still needs an explicit owner-authorized tag
and GitHub Release, after which this section should record the same
registry-smoke verification v0.4.1 got before being called canonical.
- **v0.4.1 is still the canonical release** until v0.5.0 actually publishes.
All five public packages are on npm at
`0.4.1`, published by the release workflow over trusted publishing, and
every one carries a verified provenance attestation recording
`.github/workflows/publish.yml` in `wonkwonlee/ChangeSafe` at
Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -345,7 +345,7 @@ there is no `--auto-approve`. Full usage: [packages/cli/README.md](packages/cli/

## Packages

Five public packages are on npm. As of **v0.4.1** they are published by the
Five public packages are on npm. Since **v0.4.1** they are published by the
release workflow over npm trusted publishing, and each carries a provenance
attestation naming the workflow, repository, and commit that produced it —
`npm audit signatures` verifies them, and you should rather than take this
Expand All @@ -355,6 +355,7 @@ bootstrap/remediation versions carry no attestation.
Use `0.4.1` or later. `0.4.0` published only three of the five packages
before failing, so the CLI and the Kubernetes domain do not exist at that
version; those three are genuine and attested, but the set is incomplete.
(`v0.5.0` is prepared but not yet published — see `docs/LAUNCH.md`.)

```bash
npm i @changesafe/core @changesafe/domain-terraform # embed the gate
Expand Down
24 changes: 21 additions & 3 deletions docs/LAUNCH.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,27 @@ deploys, tags, or posts automatically.

## Release truth

- `v0.4.1` is the canonical package/tag release: the complete published set of
the multi-domain workbench, and the first to go out through the trusted
publishing workflow rather than a manual publish.
- `v0.5.0` is **prepared on `main`, not yet published.** It is not the
canonical release until an owner-authorized tag and GitHub Release
actually complete the publish workflow — until then, `v0.4.1` remains
what `npm install` and the GitHub Action resolve to, and every
user-facing reference (this file's example below, `README.md`) stays
pinned to it. What v0.5.0 will contain once published: a universal-policy
skip legitimacy fix in `@changesafe/core` (a domain adapter could
previously skip any universal policy, not just the two the shape permits,
with no enforcement outside the app's own registration path), a new
Terraform `PLAN_CONTEXT_REQUIRED` policy replacing a prose-only
`VERIFICATION_REQUIRED` skip, every receipt now recording
`policyCoverage`, and `changesafe eval` gaining Kubernetes support.
`CORE_POLICY_VERSION` moves to `core-v0.2.0` and
`TERRAFORM_POLICY_VERSION` to `terraform-v0.2.0` — a v0.5.0 receipt will
not be directly comparable to an earlier one. See
[RELEASE_NOTES_v0.5.0.md](RELEASE_NOTES_v0.5.0.md). Once published and
registry-verified the same way v0.4.1 was below, update this bullet and
the pinned references to v0.5.0.
- `v0.4.1` is the canonical package/tag release: the complete published set
of the multi-domain workbench, and the first to go out through the
trusted publishing workflow rather than a manual publish.
- **Do not point anyone at `0.4.0`.** It published three of five packages
before failing, so the CLI and the Kubernetes domain do not exist at that
version. The three that shipped are genuine and carry provenance; the set is
Expand Down
116 changes: 116 additions & 0 deletions docs/RELEASE_NOTES_v0.5.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# ChangeSafe v0.5.0

A trust-model fix in `@changesafe/core`, a new Terraform policy that replaces
a prose-only skip with a real one, `changesafe eval` gaining Kubernetes
support, and two security-relevant fixes to the Kubernetes analysis prompt.

**Policy behavior changed.** `CORE_POLICY_VERSION` moves to `core-v0.2.0` and
`TERRAFORM_POLICY_VERSION` moves to `terraform-v0.2.0`. A v0.5.0 receipt is
not directly comparable to a v0.4.x one — that is the point of the version
existing, not an oversight.

## Universal-policy skip legitimacy, enforced in core

A domain adapter's `skippedUniversalPolicies` could previously name any of
the five universal policies, not just the two whose shape assumes a
model-authored proposal or an in-place patch (`ROLLBACK_COMPLETE`,
`VERIFICATION_REQUIRED`). The only check against this lived in the app's own
runtime registration (`features/domains/runtime.ts`), not in published core —
so a hand-written adapter, or `changesafe gate` run directly against one,
got no enforcement at all. A probe adapter that skipped all five universal
policies passed `evaluatePolicies` cleanly with zero findings and `LOW` risk,
despite the proposal carrying an injection attempt, invented evidence, no
rollback, and no verification.

`replacedBy` was also a bare string, so nothing verified a skip's claimed
replacement existed. Terraform's `VERIFICATION_REQUIRED` skip named `"the
pull request review"` — prose, not a policy.

Fixed:

- `evaluatePolicies` and `policyOrder` both validate every skip before doing
anything else, and throw for an illegitimate `policyId`, a duplicate skip,
or a replacement that names a policy the adapter does not declare (or that
collides with a universal policy id, which `policyOrder`'s id filter could
not otherwise distinguish from the skip itself). Only
`ROLLBACK_COMPLETE`/`VERIFICATION_REQUIRED` may ever be skipped —
`PATCH_SCHEMA`, `BLAST_RADIUS`, and `UNTRUSTED_INSTRUCTION` are structurally
answerable by every domain and no adapter may skip them.
- `SkipReplacement` is a typed shape (`{ kind: "domain-policy", policyId }`)
instead of a string. A replacement that named a non-mechanical process
rather than a real policy would have produced no finding of its own — the
gate would pass with a genuine verdict gap rather than merely an
honestly-labeled one — so that option does not exist at all.
- Every receipt now carries `policyCoverage`: the exact ordered policy ids
that ran, plus what was skipped and why. A verifier reads this from the
signed payload instead of fetching and trusting the adapter's source to
interpret an absent policy id. The field is optional on
`ChangeReceiptSchema` so a receipt issued before it existed (v0.4.1 and
earlier) still parses for hash and signature verification —
`createReceipt` always sets it going forward, and `canonicalize` drops an
`undefined` property, so a legacy receipt's hash is unaffected.

## Terraform gains `PLAN_CONTEXT_REQUIRED`

Terraform's proposal is derived mechanically from the plan with no model
involved, so it can never declare its own precondition or postcheck steps —
that is the honest reason it skips `VERIFICATION_REQUIRED`. What replaces it
used to be prose. It is now a real policy: `PLAN_CONTEXT_REQUIRED` warns when
a destructive change carries zero PR or commit context, because the pull
request review this domain relies on for verification needs something to
review. A plan that destroys nothing, or that carries any context entry,
passes.

## `changesafe eval` measures Kubernetes too

The benchmark previously imported `networkAnalysisPrompt` and `networkDomain`
only, so it measured one domain while the corpus spans three. Kubernetes now
has its own hardened prompt (`packages/ai/src/prompts/kubernetes.ts`) and is
registered in `ANALYSIS_DOMAINS`; `eval --domain kubernetes` resolves it
instead of the hardcoded network path. Terraform stays out by design — its
plan already is the proposal, so there is nothing for a model to propose.
Report schema moves to version 3, recording `corpus.domain`, so a report is
only ever compared against another from the same domain.

## Two fixes in the same change

Landed as part of getting Kubernetes eval support production-honest, not
found independently:

- **Kubernetes eval input is normalized.** The strict, already-normalized
`KubernetesSnapshotSchema` was being parsed directly against raw,
collector-shaped scenario fixtures — every bundled Kubernetes scenario
failed before ever reaching the provider. `eval`'s Kubernetes domain now
tries the strict parse first and falls back to `normalizeSnapshot`, the
same boundary `changesafe gate` already used.
- **Pod-label and Service-selector values stay inside the untrusted
block.** The Kubernetes prompt's trusted preamble — the section the model
is told to trust, ahead of `<untrusted_snapshot_data>` — was echoing
`podLabels` and `selector` values into it. Those are `Record<string,
string>` with no charset restriction beyond length, so an instruction-like
value there bypassed the untrusted-content boundary the system prompt
describes. They are no longer copied into the preamble; the full snapshot
already carries them inside the untrusted block.
- **Rollback references are validated too.** The Kubernetes prompt's local
cross-check validated a forward `replace` against the snapshot but never a
rollback `replace`, and accepted any rollback `remove` regardless of
whether it undid something the proposal actually added. A model response
with an ungrounded rollback reference was counted `accepted` by the
benchmark before the gate's `PATCH_SCHEMA` policy caught it downstream —
inflating the grounded-output metric for a proposal that was always going
to be blocked.
- **Prompt/adapter pairing is typed.** The analysis domain registry paired
each prompt with its adapter through two independent `as unknown as never`
casts, so nothing prevented registering, say, the Kubernetes prompt against
the network adapter. A single generic function now binds both under one
type parameter, so a mismatched pairing fails to typecheck.

## Known limits

- Terraform still cannot catch command smuggling: it is an external-diff
domain with no simulator, and no Terraform policy inspects planned
attribute values for command payloads. Tracked, not papered over with a
scenario the gate would not actually refuse.
- `K8S_SERVICE_SELECTOR` still only re-checks Services that existed before a
change; whether it should also check newly created Services is an open
decision, not yet made.
2 changes: 1 addition & 1 deletion lib/domain/version.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/** Kept in sync with package.json manually; embedded in receipts. */
export const APP_VERSION = "0.4.1";
export const APP_VERSION = "0.5.0";

/** Bumped whenever any frozen policy's behavior changes; embedded in receipts. */
export const POLICY_VERSION = "policies-v0.1.0";
Expand Down
62 changes: 31 additions & 31 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "changesafe",
"version": "0.4.1",
"version": "0.5.0",
"private": true,
"description": "A deterministic airlock for AI-proposed infrastructure changes: AI proposes, pure policies validate, a human decides, simulation stays sandboxed.",
"license": "MIT",
Expand Down
8 changes: 4 additions & 4 deletions packages/ai/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@changesafe/ai",
"version": "0.4.1",
"version": "0.5.0",
"description": "Provider-agnostic model adapters that produce ChangeProposals. One Zod schema drives structured output on every provider; every provider's output faces identical local validation.",
"license": "MIT",
"private": true,
Expand All @@ -17,9 +17,9 @@
"typecheck": "tsc --noEmit"
},
"dependencies": {
"@changesafe/core": "^0.4.1",
"@changesafe/domain-kubernetes": "^0.4.1",
"@changesafe/domain-network": "^0.4.1",
"@changesafe/core": "^0.5.0",
"@changesafe/domain-kubernetes": "^0.5.0",
"@changesafe/domain-network": "^0.5.0",
"zod": "^4.4.3"
},
"devDependencies": {
Expand Down
2 changes: 1 addition & 1 deletion packages/cli/dist/changesafe.js
Original file line number Diff line number Diff line change
Expand Up @@ -26181,7 +26181,7 @@ import { writeFileSync } from "node:fs";
import path2 from "node:path";

// src/version.ts
var CLI_PACKAGE_VERSION = "0.4.1";
var CLI_PACKAGE_VERSION = "0.5.0";
var CLI_APP_VERSION = `changesafe-cli-${CLI_PACKAGE_VERSION}`;
var SERVER_APP_VERSION = `changesafe-server-${CLI_PACKAGE_VERSION}`;

Expand Down
Loading
Loading