Tie the README's version claim to the table it summarises

.github/workflows/ci.yml

@@ -153,15 +153,21 @@ jobs:
         shell: bash
 
   lint-invariants:
-    name: Shell invariants
+    name: Repo invariants
     runs-on: ubuntu-latest
     steps:
+      # fetch-depth: 0 for the tags, not the history. The version-claim
+      # check compares the roadmap's Shipped table against the newest
+      # release tag, and the default shallow checkout carries no tags --
+      # which would leave that half reporting a skip CI never reads.
       - uses: actions/checkout@v7
+        with:
+          fetch-depth: 0
 
-      # Properties shellcheck has no opinion about, each added because the
-      # pattern it rejects shipped here and was found by reading rather
-      # than by any test.
-      - name: Check shell invariants
+      # Properties no other tool has an opinion about, each added because
+      # the pattern it rejects shipped here and was found by reading
+      # rather than by any test.
+      - name: Check repo invariants
         run: ./tests/lint/run-tests.sh
 
   preflight-static:

CLAUDE.md

@@ -327,7 +327,7 @@ Per-suite requirements:
 | audit-chain | bash, sha256sum |
 | docs-index | bash, awk, diff |
 | cloud-preflight | bash |
-| lint | bash, python3 |
+| lint | bash, python3; a checkout carrying its tags |
 | preflight-static | bash, python3; shellcheck if present |
 | pki | bash, jq, openssl |
 | pki-migration | bash, jq, python3, openssl |
@@ -386,6 +386,14 @@ CI enforces several invariants worth knowing before you push:
   -rf "$D"; }` returns 1 when the test is false, and bash applies that
   to the script's exit status from an `EXIT` trap — a successful run
   reports failure, and an explicit `exit 0` does not save it.
+- **The README's version claim must match the roadmap's Shipped table**
+  (`tests/lint/check_version_claim.py`), and that table must be no older
+  than the newest git tag. The table is the authority rather than the
+  tags because a release lands its roadmap row in a PR and is tagged
+  after it merges — gating on tags would fail that PR for being correct.
+  The tag half is one-directional for the same reason: ahead is fine,
+  behind is a release nobody wrote down. This is why the lint job checks
+  out with `fetch-depth: 0`.
 - **Trivy findings at HIGH or above fail the build.** Accepted findings
   go in `.trivyignore.yaml` *with the reason* — an unjustified
   suppression is indistinguishable from never having run the scanner.

README.md

@@ -441,7 +441,7 @@ See [`.github/workflows/ci.yml`](.github/workflows/ci.yml).
 
 ## Roadmap
 
-Everything through v0.14 has shipped — see
+Everything through v0.18 has shipped — see
 [Releases](https://github.com/sethbergman/vault-reference-platform/releases)
 for the log. "Shipped" here means there is a test that fails if the
 feature breaks, not that the code exists.

docs/roadmap.md

@@ -554,5 +554,40 @@ refused. The assertion failed, correctly, because it greps the specific
 refusal rather than checking the exit code. Had it only checked that the
 script exited non-zero, it would have gone green while testing nothing.
 
+A fifth is not about an assertion at all, which is why it survived four
+releases.
+
+The Roadmap section of the README opened "Everything through v0.14 has
+shipped". v0.18 had. Four PRs each added a row to the Shipped table at
+the top of this file and left that sentence alone — and it is the first
+claim a reader meets, in the one section whose entire purpose is to say
+what is done and what is not.
+
+Nothing caught it because nothing was looking. `tests/docs-index` checks
+that every file in `docs/` is linked from the generated index and skips
+`README.md` by name; markdownlint has opinions about the line and none
+about the fact on it. The question above still finds it, but only when
+asked of the claim rather than of the test: what would have to break for
+that sentence to fail? Nothing could. There was no weak assertion here —
+there was no assertion.
+
+So this is the argument for generating `docs/README.md`, arriving a
+second time somewhere too small to generate. One sentence does not earn a
+generator, so it is asserted instead.
+[`tests/lint/check_version_claim.py`](../tests/lint/check_version_claim.py)
+requires the README to name the newest row of the table above, and
+requires that row to be no older than the newest git tag. The tag half is
+one-directional on purpose: a release lands its roadmap row in a PR and
+is tagged after that merges, so gating the README on tags would fail the
+PR for being correct, while a table *behind* the tags is a release nobody
+wrote down. Six mutations were watched to fail, including the one the
+README half cannot see — deleting the v0.18 row while v0.18 is tagged.
+
+The four earlier entries share a shape: a test existed and was weaker
+than it looked. This one is different in a way worth separating, because
+the fix is different. The table above was right the whole time; what
+drifted was a hand-written summary of it. A weak assertion gets widened.
+An unchecked claim gets tied to whatever is already correct.
+
 None of this changes what the table above claims. It changes how much the
 word "tested" in it is worth, which seemed worth writing down.

tests/lint/check_version_claim.py

@@ -0,0 +1,155 @@
+#!/usr/bin/env python3
+"""Reject a README version claim that has drifted from the roadmap.
+
+README.md carries one sentence naming the newest shipped release. It is
+the first claim a reader meets, and nothing checked it: at v0.18 it still
+said v0.14, four releases stale, and had been wrong through four PRs that
+each updated the roadmap table beside it.
+
+This is the failure docs/README.md is generated to avoid -- a
+hand-maintained claim that stops being true silently, invisible to
+everyone except the reader who relied on it. One sentence is not worth
+generating, so it is asserted instead.
+
+TWO CHECKS, AND WHY THE SOURCE OF TRUTH IS NOT THE TAGS
+
+  1. The README claim must name the newest row of the roadmap's Shipped
+     table.
+  2. That row must be at least as new as the newest git tag.
+
+The roadmap table is the authority rather than `git tag` because of the
+order releases are cut in: the roadmap row lands in a PR, and the tag is
+pushed after it merges. Gating the README on tags would fail that PR for
+being correct. Gating the roadmap on tags in one direction only -- table
+ahead of tags is fine, table behind them is not -- permits that ordering
+and still catches a release that was tagged and never written down.
+
+Check 2 needs the tags to be present. CI's lint job fetches them
+deliberately; see .github/workflows/ci.yml. A checkout with no tags is
+reported as a failure rather than skipped, because a skip here is
+indistinguishable from a pass and this file exists because something
+unchecked went stale.
+
+Exits non-zero and says which claim disagrees with which.
+"""
+
+import re
+import subprocess
+import sys
+
+# "Everything through v0.14 has shipped" -- anchored on the load-bearing
+# words rather than the whole sentence, so rewording the prose around it
+# is fine and removing the claim is not.
+CLAIM = re.compile(r'through (v\d+\.\d+) has shipped')
+
+# A row of the Shipped table: | v0.18 | Rate limit quotas, ... |
+ROW = re.compile(r'^\|\s*(v\d+\.\d+)\s*\|')
+
+TAG = re.compile(r'^v\d+\.\d+$')
+
+
+def key(version):
+    """Sort v0.9 below v0.10, which a string compare does not."""
+    return tuple(int(part) for part in version.lstrip('v').split('.'))
+
+
+def read(path):
+    with open(path, encoding='utf-8') as handle:
+        return handle.read()
+
+
+def shipped_versions(text):
+    """Every version in the roadmap's `## Shipped` table, that table only."""
+    lines = text.split('\n')
+    try:
+        start = lines.index('## Shipped')
+    except ValueError:
+        return []
+
+    found = []
+    for line in lines[start + 1:]:
+        if line.startswith('## '):
+            break
+        match = ROW.match(line)
+        if match:
+            found.append(match.group(1))
+    return found
+
+
+def newest_tag():
+    """The newest vN.N tag, or None when this is not a git checkout.
+
+    Returns [] rather than None for a checkout that is a repository but
+    carries no tags -- the caller treats those differently.
+    """
+    try:
+        out = subprocess.run(
+            ['git', 'tag', '--list'],
+            capture_output=True, text=True, check=True,
+        ).stdout
+    except (subprocess.CalledProcessError, FileNotFoundError):
+        return None
+
+    tags = [t for t in out.split('\n') if TAG.match(t.strip())]
+    return max(tags, key=key) if tags else []
+
+
+def main():
+    failures = []
+
+    shipped = shipped_versions(read('docs/roadmap.md'))
+    if not shipped:
+        print('No `## Shipped` table found in docs/roadmap.md, or no version')
+        print('rows in it. This check reads that table as the source of truth,')
+        print('so it cannot run -- fix the table or update this checker.')
+        return 1
+
+    newest = max(shipped, key=key)
+
+    claims = CLAIM.findall(read('README.md'))
+    if not claims:
+        failures.append(
+            'README.md names no shipped version.\n'
+            '  Expected a sentence matching "through vN.N has shipped".\n'
+            '  If the wording changed deliberately, update CLAIM in this file\n'
+            '  -- do not delete the claim, it is what a reader trusts first.'
+        )
+    else:
+        stale = sorted({c for c in claims if c != newest}, key=key)
+        if stale:
+            failures.append(
+                'README.md claims %s; docs/roadmap.md ships %s.\n'
+                '  The roadmap table is the source of truth. Update the README.'
+                % (', '.join(stale), newest)
+            )
+
+    tag = newest_tag()
+    if tag is None:
+        print('Not a git checkout; skipping the roadmap-versus-tag check.')
+    elif tag == []:
+        failures.append(
+            'No vN.N git tags found.\n'
+            '  This check compares the roadmap table against the newest tag,\n'
+            '  and a checkout with no tags cannot answer that. Run\n'
+            '  `git fetch --tags`; CI fetches them with fetch-depth: 0.'
+        )
+    elif key(newest) < key(tag):
+        failures.append(
+            '%s is tagged but docs/roadmap.md stops at %s.\n'
+            '  A release was cut without a Shipped row. Add it.'
+            % (tag, newest)
+        )
+
+    if failures:
+        print('Version claims disagree:')
+        for item in failures:
+            print('  %s' % item)
+        return 1
+
+    print('README names %s; roadmap ships %s; newest tag %s.'
+          % (newest, newest, tag if tag else 'n/a'))
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main())

tests/lint/run-tests.sh

@@ -1,7 +1,6 @@
 #!/usr/bin/env bash
 #
-# run-tests.sh — Invariants about shell scripts that shellcheck does not
-#                enforce
+# run-tests.sh — Repo-wide invariants no other tool has an opinion about
 #
 # Usage:
 #   ./tests/lint/run-tests.sh
@@ -13,7 +12,11 @@
 # cheap, they are repo-wide, and they fail the build with an explanation
 # rather than leaving the next person to rediscover the same thing.
 #
-# Requirements: bash, python3
+# The first was about shell, which is what this suite was named for.
+# Prose drifts the same way and the second check is about that, so the
+# scope is the invariant rather than the language it is written in.
+#
+# Requirements: bash, python3, and a checkout carrying its tags
 
 set -uo pipefail
 
@@ -55,6 +58,26 @@ else
     bad "no trap handler returns the result of a bare test" "$OUT"
 fi
 
+# ---------------------------------------------------------------------------
+printf '
+=== The README names the release the roadmap actually ships ===
+'
+# ---------------------------------------------------------------------------
+# README.md carries one sentence naming the newest shipped release, and
+# nothing checked it: at v0.18 it still said v0.14. Four PRs had updated
+# the roadmap table directly beside it and left the sentence alone, which
+# is the failure docs/README.md is generated to avoid — a hand-maintained
+# claim that stops being true silently. One sentence is not worth
+# generating, so it is asserted instead.
+#
+# The checker also requires the roadmap table to be no older than the
+# newest git tag, which is why this suite now wants a checkout with tags.
+if OUT="$(python3 "${SCRIPT_DIR}/check_version_claim.py" 2>&1)"; then
+    ok "README, roadmap and tags agree on the newest release"
+else
+    bad "README, roadmap and tags agree on the newest release" "$OUT"
+fi
+
 printf '\n'
 printf 'passed: %d   failed: %d\n' "$PASS" "$FAIL"
 [[ "$FAIL" -eq 0 ]] || exit 1