test: name every enforced behaviour in a ledger, and the test that proves it

.github/workflows/ci.yml

@@ -120,6 +120,13 @@
             | tar -xz kubeconform
           KUBECONFORM="${PWD}/kubeconform" npm run lint:manifests
 
+      # The behaviour ledger (docs/requirements.md): every row parses, names a
+      # real test that holds at least one test, its stated count matches what
+      # it holds, and every REQ-NNN cited in the tracked tree resolves to a
+      # row. See docs/adr/architecture/0103-a-behaviour-ledger-names-what-a-test-proves.md.
+      - 'name': 'Requirements ledger'
+        'run': 'npm run lint:requirements'
+
   'tests':
     'name': 'Tests'
     'runs-on': 'ubuntu-latest'

docs/adr/README.md

@@ -203,3 +203,4 @@ Decisions about the compiler's own structure, not about the model. Their
 | [0100](architecture/0100-tests-run-in-process-on-vitest.md) | Tests run in-process on Vitest, and the tooling is TypeScript that Node runs directly | settled |
 | [0101](architecture/0101-coverage-is-a-ratchet.md) | Coverage is a ratchet: the thresholds sit on what the suite reaches, and only rise | settled |
 | [0102](architecture/0102-the-gate-grows-with-the-code.md) | A new gate's script and its CI job land in the same pull request, and a test proves the two stay matched | settled |
+| [0103](architecture/0103-a-behaviour-ledger-names-what-a-test-proves.md) | A behaviour ledger names every guarantee and the test that proves it, and a meta test holds the two together | settled |

docs/adr/architecture/0103-a-behaviour-ledger-names-what-a-test-proves.md

@@ -0,0 +1,75 @@
+---
+tier: decision
+status: proposed
+claim: settled
+date: 2026-09-14
+normative: docs/architecture.md#gates
+rests-on: ["0001"]
+---
+
+# A behaviour ledger names every guarantee and the test that proves it, and a meta test holds the two together
+
+## Rests on
+
+Every guarantee this repository makes is provable by naming the one test file
+that would fail if the guarantee stopped holding, so a ledger of {id,
+sentence, test file} triples can be checked by reading the tree rather than by
+trusting whoever last touched it. False if: a guarantee exists that no single
+test file proves, which would leave a row with nothing honest to cite.
+Settled by: `docs/requirements.md` holding one row per gate this repository
+enforces today, each resolving to a real, non-empty test file, with none left
+over.
+
+## Why
+
+The coverage ratchet ([0101](0101-coverage-is-a-ratchet.md)) notices a line
+that stops running, but not a test file that is renamed, emptied or deleted
+while the code it once proved stays in place and green. Nothing before this
+connected a sentence a contributor relies on to the test that makes it true,
+so that sentence could go silently unproven and nothing in the suite would
+say so.
+
+The fix is the same shape as the boundary gate and
+[the script-to-workflow check](0102-the-gate-grows-with-the-code.md): state
+the rule as a comparison over things that already exist on disk (a markdown
+table and a directory of test files), rather than as a convention to
+remember. A row that no longer resolves, a file that exists but holds no
+test, a stated count that drifts from the rows actually present, or an id
+cited where no row backs it: each is a comparison
+a script can run, not a habit a reviewer can forget, which matters here for
+the same reason as [0001](../model/0001-estate-scale-and-ownership.md): one
+person reading their own diff later is not a second pair of eyes.
+
+This ledger is deliberately not `docs/architecture-rules.md`
+([issue #29](https://github.com/JorisJonkers-dev/deploy-kit/issues/29), not
+yet landed). That one will list the rules a tool enforces, each keyed to its
+enforcer; this one lists the behaviours a person depends on, each keyed to
+the test that fails when it stops being true. A behaviour can rest on several
+rules, and a rule can serve several behaviours, so keeping the ledgers apart
+keeps each one answerable to a different question.
+
+## Alternatives
+
+| option | cost if taken | why rejected |
+|---|---|---|
+| No ledger; trust the existing gates and their own tests | Nothing to write or maintain | Exactly the failure mode this decision exists to close: a test file can be renamed, emptied or deleted while the code it covered stays in place, and nothing today would say so |
+| A checklist item in the pull request template | Visible to a human reviewer | There is no second reviewer here, and a template item is unchecked by habit long before it is unchecked on purpose |
+| Fold behaviours into `docs/architecture-rules.md` once it lands | One document instead of two | A rule and a behaviour answer different questions; merging them would make the rule ledger's per-rule enforcer story and this ledger's per-behaviour test story compete for the same row |
+
+## Reversibility
+
+Undo cost today: deleting one document and one gate script. Becomes
+irreversible once: never; the ledger describes this repository's own tree and
+nothing outside it reads either.
+
+## Consequences
+
+- A row must name a real, non-empty test file, so adding a guarantee with no
+  test to cite is not representable; the guarantee waits until it has one.
+  Paid by whoever adds a row, once per row.
+- Renaming, emptying or deleting a test file a row names fails the suite
+  instead of merging quietly, and the failure names the row. Paid by whoever
+  touches that file, in the same pull request.
+- The stated row count is one more thing to update by hand when a row is
+  added or removed; forgetting it fails the same test rather than drifting
+  unnoticed. Paid once per row added or removed.

docs/architecture.md

@@ -206,15 +206,16 @@ proves the two never drift apart.
 | decisions | `npm run lint:adrs` | frontmatter, register integrity, citations, normative anchors per domain |
 | links | `npm run lint:links` | relative links and heading anchors across every tracked Markdown file |
 | manifests | `npm run lint:manifests` | every rendered example object against pinned Kubernetes and CRD schemas |
+| requirements | `npm run lint:requirements` | a behaviour ledger row that no longer parses, names a missing or empty test, drifts from its stated count, or is cited by an id no row carries |
 | tests | `npm run test:coverage` | behaviour, plus the coverage ratchet |
 | package contents | `node scripts/check-package-contents.ts` | `npm pack` shipping a file outside `docs/adr/` and `spec/`, the boundary the package's `files` field states but does not enforce on its own |
 | actionlint | a pinned `actionlint` binary | invalid workflow syntax, an undefined `${{ }}` expression, a shellcheck finding inside a `run:` step |
 | secret scan | a pinned `gitleaks` binary | a committed secret matching the default ruleset, or this repository's own allowlist entries |
 
-Decisions, links and manifests share one CI job, `contracts`: all three check a
-document against a rule rather than code against a graph. Boundaries runs
-alone as `architecture`, because it is the one gate that speaks for
-`docs/architecture.md` itself rather than for a document beside it.
+Decisions, links, manifests and requirements share one CI job, `contracts`:
+all four check a document against a rule rather than code against a graph.
+Boundaries runs alone as `architecture`, because it is the one gate that
+speaks for `docs/architecture.md` itself rather than for a document beside it.
 
 Coverage is a ratchet ([0101](adr/architecture/0101-coverage-is-a-ratchet.md)).
 The thresholds in `vitest.config.ts` sit on what the suite reaches, over an

docs/requirements.md

@@ -0,0 +1,36 @@
+# Behaviour ledger
+
+This is not `docs/architecture-rules.md` (issue
+[#29](https://github.com/JorisJonkers-dev/deploy-kit/issues/29), not yet
+landed). That ledger will list the rules a tool enforces: a dependency-cruiser
+check, an ESLint message, a lint's error code, each keyed to its enforcer and
+its severity. This one lists the behaviours a person depends on: a sentence a
+contributor or a consumer can rely on, keyed to the test that fails the moment
+it stops being true. A behaviour can rest on several rules, and one rule can
+serve several behaviours, so the two ledgers stay separate on purpose.
+
+A behaviour can lose its proof without anything saying so: the coverage
+ratchet notices a line that stops running, but not a test file that was
+renamed, emptied or deleted while the code it covered stayed in place. Every
+id below is greppable, and a comment naming one resolves to a row rather than
+to nothing.
+[`scripts/lint-requirements.ts`](../scripts/lint-requirements.ts) is what
+holds this document honest: every row parses; every named file exists, is a
+test file and holds at least one test; ids are unique; the count this
+document states matches the number of rows it holds; and every id cited
+anywhere in the tracked tree resolves to a row here.
+
+This ledger holds **10** rows. The compiler's behaviours join it as they land.
+
+| id | a contributor or a consumer can rely on | proved by |
+|---|---|---|
+| REQ-001 | Every decision record under `docs/adr/` satisfies its frontmatter, register and citation contract | [test/adr-contract.test.ts](../test/adr-contract.test.ts) |
+| REQ-002 | Every relative link and heading anchor in tracked Markdown resolves to a real target | [test/link-contract.test.ts](../test/link-contract.test.ts) |
+| REQ-003 | Every rendered Kubernetes manifest in the worked examples validates against its pinned schema | [test/manifest-contract.test.ts](../test/manifest-contract.test.ts) |
+| REQ-004 | The compiler's layer boundaries and module reachability are enforced on the dependency graph, not on review alone | [test/boundary-contract.test.ts](../test/boundary-contract.test.ts) |
+| REQ-005 | A pull request title and its commits use a conventional-commit type release-please reads | [test/pr-title-contract.test.ts](../test/pr-title-contract.test.ts) |
+| REQ-006 | No em-dash enters tracked text outside `docs/mde/` and `CHANGELOG.md` | [test/emdash.test.ts](../test/emdash.test.ts) |
+| REQ-007 | A test that reaches the network, is committed focused or skipped, sleeps a fixed duration, or asserts nothing never reaches a green build | [test/harness.test.ts](../test/harness.test.ts) |
+| REQ-008 | Coverage is a ratchet: no `v8`, `c8` or `istanbul` ignore comment exempts a line from it | [test/harness.test.ts](../test/harness.test.ts) |
+| REQ-009 | The npm package ships nothing outside `docs/adr/` and `spec/`, checked against what npm would really pack rather than the advisory `files` field | [test/package-contents-contract.test.ts](../test/package-contents-contract.test.ts) |
+| REQ-010 | A gate's npm script and the CI job that runs it land in the same pull request, so neither can drift from the other unnoticed | [test/pipeline-wiring.test.ts](../test/pipeline-wiring.test.ts) |

package.json

@@ -18,12 +18,13 @@
     "lint:links": "node scripts/lint-links.ts",
     "lint:manifests": "node scripts/lint-manifests.ts",
     "lint:boundaries": "node scripts/lint-boundaries.ts",
+    "lint:requirements": "node scripts/lint-requirements.ts",
     "format": "prettier --write .",
     "format:check": "prettier --check .",
     "typecheck": "tsc --noEmit",
     "test": "vitest run",
     "test:coverage": "vitest run --coverage",
-    "verify": "npm run lint && npm run format:check && npm run typecheck && npm run lint:adrs && npm run lint:links && npm run lint:manifests && npm run lint:boundaries && npm run test:coverage"
+    "verify": "npm run lint && npm run format:check && npm run typecheck && npm run lint:adrs && npm run lint:links && npm run lint:manifests && npm run lint:requirements && npm run lint:boundaries && npm run test:coverage"
   },
   "devDependencies": {
     "@eslint/js": "10.0.1",

scripts/lint-requirements.ts

@@ -0,0 +1,207 @@
+// The behaviour ledger, docs/requirements.md: what keeps it honest.
+//
+// docs/requirements.md holds one row per behaviour this repository
+// guarantees: a greppable id, one sentence a contributor or a consumer can
+// rely on, and the test file that proves it. A row can lose its proof
+// silently when the test it names is renamed, emptied or deleted while the
+// code it covered stays in place; nothing else in the suite would notice.
+// This is what notices: every row parses; every named file exists, is a test
+// file and holds at least one test; ids are unique; the document's stated row
+// count matches what it actually holds; and every id cited anywhere in the
+// tracked tree resolves to a row.
+//
+// This is not docs/architecture-rules.md (issue #29, not yet landed): that
+// ledger will list the rules a tool enforces (a lint rule, a dependency-
+// cruiser check, an ESLint message), each keyed to its enforcer and severity.
+// This one lists the behaviours a person depends on, each keyed to the test
+// that fails when it stops being true. A behaviour can rest on several rules,
+// and one rule can serve several behaviours, so the two ledgers are
+// deliberately not merged.
+//
+// A library first, like the other gates: tests call lintRequirements() in
+// -process, and `node scripts/lint-requirements.ts [root]` is the command.
+import { execFileSync } from "node:child_process";
+import { readFileSync } from "node:fs";
+import { join } from "node:path";
+import { isEntrypoint } from "./lib/entrypoint.ts";
+import { processOutput, type GateOutput } from "./lib/output.ts";
+
+/** One row of the ledger: an id, its guarantee, and the test that proves it. */
+export interface RequirementRow {
+  readonly id: string;
+  readonly sentence: string;
+  readonly test: string;
+}
+
+/** What one run found: how many rows it read, and every violation. */
+export interface RequirementsLintResult {
+  readonly rows: number;
+  readonly errors: readonly string[];
+}
+
+const REPOSITORY = join(import.meta.dirname, "..");
+const LEDGER = join("docs", "requirements.md");
+
+const CITATION = /\bREQ-\d{3}\b/g;
+const STATED_COUNT = /this ledger holds \*\*(\d+)\*\* rows?\b/i;
+
+// A row's link text is the test path relative to the repository root
+// (`test/x.test.ts`); its href is that same path relative to docs/
+// (`../test/x.test.ts`), since the ledger lives in docs/. Both are checked
+// against each other, so a row cannot link one file while claiming another.
+const ROW =
+  /^\|\s*(REQ-\d{3})\s*\|\s*(.+?)\s*\|\s*\[([^\]]+)\]\(([^)]+)\)\s*\|$/;
+
+/** Every line that opens like a ledger row, matched or not. */
+function candidateLines(text: string): string[] {
+  return text.split("\n").filter((line) => /^\|\s*REQ-\d{3}\s*\|/.test(line));
+}
+
+/**
+ * Parse the ledger body into rows. Pure and synchronous, so a malformed row
+ * is testable against a string with no filesystem involved. A line that opens
+ * like a row but does not match it is reported and dropped, not guessed at.
+ */
+export function parseRequirements(text: string): {
+  readonly rows: readonly RequirementRow[];
+  readonly errors: readonly string[];
+} {
+  const rows: RequirementRow[] = [];
+  const errors: string[] = [];
+  for (const line of candidateLines(text)) {
+    const match = ROW.exec(line);
+    const id = match?.[1];
+    const sentence = match?.[2];
+    const linkText = match?.[3];
+    const href = match?.[4];
+    if (
+      id === undefined ||
+      sentence === undefined ||
+      linkText === undefined ||
+      href === undefined
+    ) {
+      errors.push(`malformed row: ${line}`);
+      continue;
+    }
+    if (linkText !== href.replace(/^\.\.\//, ""))
+      errors.push(
+        `${id}: link text '${linkText}' does not match its target '${href}'`,
+      );
+    rows.push({ id, sentence, test: linkText });
+  }
+  return { rows, errors };
+}
+
+/**
+ * Every id cited outside the row that declares it, across the tracked tree,
+ * that no row carries. Pure over a {rel: content} map, so it is testable
+ * against a synthetic tree the way emdash.test.ts's offendersIn() is.
+ */
+export function citationErrors(
+  files: Readonly<Record<string, string>>,
+  ids: ReadonlySet<string>,
+): string[] {
+  const errors: string[] = [];
+  const reported = new Set<string>();
+  // Object keys are unique paths, so two entries are never equal: a strict
+  // less-than is enough to order them, with no equal case to fall through to.
+  const entries = Object.entries(files).sort(([a], [b]) => (a < b ? -1 : 1));
+  for (const [rel, content] of entries) {
+    for (const match of content.matchAll(CITATION)) {
+      const id = match[0];
+      const key = `${rel}\0${id}`;
+      if (ids.has(id) || reported.has(key)) continue;
+      reported.add(key);
+      errors.push(`${rel} cites ${id}, which no row carries`);
+    }
+  }
+  return errors;
+}
+
+/** Every tracked file's text, keyed by its path relative to `root`. */
+function trackedText(root: string): Record<string, string> {
+  const files: Record<string, string> = {};
+  const tracked = execFileSync("git", ["ls-files", "-z"], {
+    cwd: root,
+    encoding: "utf8",
+  }).split("\0");
+  for (const rel of tracked) {
+    if (rel === "") continue;
+    try {
+      files[rel] = readFileSync(join(root, rel), "utf8");
+    } catch {
+      // A path that cannot be read as text (a symlink to nowhere, a
+      // directory entry) carries no citation either way.
+    }
+  }
+  return files;
+}
+
+/** Lint the behaviour ledger under `root`. */
+export function lintRequirements(root: string): RequirementsLintResult {
+  const path = join(root, LEDGER);
+  let text: string;
+  try {
+    text = readFileSync(path, "utf8");
+  } catch {
+    return { rows: 0, errors: [`${LEDGER}: ledger missing`] };
+  }
+
+  const { rows, errors: parseErrors } = parseRequirements(text);
+  const errors = [...parseErrors];
+
+  const byId = new Map<string, RequirementRow>();
+  for (const row of rows) {
+    const seen = byId.get(row.id);
+    if (seen)
+      errors.push(`${row.id}: id used twice, also for '${seen.sentence}'`);
+    else byId.set(row.id, row);
+  }
+
+  for (const row of rows) {
+    const testPath = join(root, row.test);
+    let content: string;
+    try {
+      content = readFileSync(testPath, "utf8");
+    } catch {
+      errors.push(`${row.id}: names '${row.test}', which does not exist`);
+      continue;
+    }
+    if (!row.test.endsWith(".test.ts"))
+      errors.push(`${row.id}: '${row.test}' is not a test file`);
+    if (!/\b(?:test|it)\s*\(/.test(content))
+      errors.push(`${row.id}: '${row.test}' holds no test`);
+  }
+
+  const stated = STATED_COUNT.exec(text)?.[1];
+  if (stated === undefined)
+    errors.push(`${LEDGER}: does not state how many rows it holds`);
+  else if (Number(stated) !== rows.length)
+    errors.push(
+      `${LEDGER}: states ${stated} rows, holds ${String(rows.length)}`,
+    );
+
+  errors.push(...citationErrors(trackedText(root), new Set(byId.keys())));
+
+  return { rows: rows.length, errors };
+}
+
+/** Lint the tree named by argv[0], or this repository, and say what was found. */
+export function main(
+  argv: readonly string[],
+  output: GateOutput = processOutput,
+): number {
+  const { rows, errors } = lintRequirements(argv[0] ?? REPOSITORY);
+  if (errors.length > 0) {
+    output.err(
+      `requirements lint: ${errors.length} error(s)\n` +
+        errors.map((e) => `  - ${e}\n`).join(""),
+    );
+    return 1;
+  }
+  output.out(`requirements lint: ${rows} rows clean\n`);
+  return 0;
+}
+
+if (isEntrypoint(import.meta.url, process.argv[1]))
+  process.exitCode = main(process.argv.slice(2));

test/adr-contract.test.ts

@@ -3,6 +3,9 @@
 // part of the test run rather than a thing someone remembers to run. It also
 // starts the lint the way CI does, as a command, which is what proves the
 // guard at the bottom of the script still runs it.
+//
+// REQ-001 (docs/requirements.md): every decision record satisfies its
+// frontmatter, register and citation contract.
 import { execFileSync } from "node:child_process";
 import { existsSync, readFileSync, readdirSync } from "node:fs";
 import { basename, join } from "node:path";

test/boundary-contract.test.ts

@@ -4,6 +4,9 @@
 // has only ever run against a tree with no violations is untested: nothing
 // proves it would fail. Each case builds a throwaway src/ tree that crosses
 // exactly one boundary and asserts the named rule reports it.
+//
+// REQ-004 (docs/requirements.md): layer boundaries and module reachability
+// are enforced on the dependency graph, not on review alone.
 import { spawnSync } from "node:child_process";
 import { mkdirSync, mkdtempSync, symlinkSync, writeFileSync } from "node:fs";
 import { dirname, join } from "node:path";