Skip to content

Tie the README's version claim to the table it summarises - #90

Merged
sethbergman merged 1 commit into
mainfrom
test/readme-version-claim-lint
Sep 16, 2026
Merged

sethbergman merged 1 commit into
mainfrom
test/readme-version-claim-lint

Conversation

@sethbergman

Copy link
Copy Markdown
Owner

The README's Roadmap section said "Everything through v0.14 has shipped"
while v0.18 had. Four PRs each added a row to the Shipped table in
docs/roadmap.md and left the sentence alone — so the first claim a
reader meets, in the one section whose purpose is to say what is done and
what is not, was four releases stale.

Nothing caught it because nothing was looking. tests/docs-index skips
README.md by name, and markdownlint has opinions about the line rather
than the fact on it. This is the failure docs/README.md is generated to
avoid, arriving somewhere too small to generate — so it is asserted
instead.

What the check does

tests/lint/check_version_claim.py, wired into the existing lint suite
so no new CI job is needed:

  1. The README's through vN.N has shipped claim must name the newest row
    of the roadmap's ## Shipped table.
  2. That table must be no older than the newest git tag.

The table is the source of truth rather than the tags because of the
order releases are cut in here: the roadmap row lands in a PR and the tag
is pushed after it merges. Gating the README on tags would fail that PR
for being correct. So the tag half is one-directional — a table ahead of
the tags is the normal state, a table behind them is a release nobody
wrote down.

The lint job now checks out with fetch-depth: 0. Without the tags that
half would report a skip, and a skip in CI is indistinguishable from a
pass: decoration standing exactly where a reader assumes a guard, which
is the thing this check exists to stop. The only remaining skip path is a
checkout with no .git at all, which CI never has.

Mutations, each watched to fail

Mutation Caught by
README one release behind (v0.17) claim ≠ table
README reworded to "The latest release is v0.18" no claim found — a reword cannot silently disable the check
README ahead of the table (v0.19) claim ≠ table
v0.18 row deleted while v0.18 is tagged the tag half; the README half stays green
## Shipped renamed to ## Releases parse finds nothing, refuses to run rather than passing
git repo with no tags fails with git fetch --tags rather than skipping

The fourth is the one that matters: it is caught by an assertion that
does not name the README at all. Numeric version sorting is confirmed by
the passing run itself — a lexicographic max over the table would pick
v0.9, not v0.18.

Also

  • README.md corrected to v0.18.
  • The lint suite's header and CI job name widened from "Shell invariants"
    to repo invariants; the scope is the invariant rather than the language
    it is written in.
  • docs/roadmap.md gains a fifth entry under "What 'done' means here".
    It is framed as different in kind from the four above it: those are a
    test that existed and was weaker than it looked, this one had no
    assertion at all. The fix differs too — a weak assertion gets widened,
    an unchecked claim gets tied to whatever is already correct.
  • CLAUDE.md records the invariant and why the job fetches tags.

Job count is unchanged at 38, so check-ci-coverage needs nothing.

🤖 Generated with Claude Code

The README's Roadmap section said "Everything through v0.14 has shipped"
while v0.18 had. Four PRs each added a row to the Shipped table in
docs/roadmap.md and left the sentence twenty lines of prose away from it
alone, so the first claim a reader meets was four releases stale.

Nothing caught it because nothing was looking. tests/docs-index skips
README.md by name, and markdownlint has opinions about the line rather
than the fact on it. This is the failure docs/README.md is generated to
avoid, arriving somewhere too small to generate — so assert it instead.

The roadmap table is the source of truth rather than git tags, because
of the order releases are cut in here: the roadmap row lands in a PR and
the tag is pushed after it merges, so gating the README on tags would
fail that PR for being correct. The tag check is therefore
one-directional — a table ahead of the tags is the normal state, a table
behind them is a release nobody wrote down.

The lint job now checks out with fetch-depth: 0. Without the tags that
half would report a skip, and a skip in CI is indistinguishable from a
pass — decoration standing exactly where a reader assumes a guard, which
is the thing this check exists to stop.

Six mutations watched to fail, including the two that matter: deleting
the v0.18 row while v0.18 is tagged, which the README half cannot see,
and rewording the claim away, which must fail rather than quietly
finding nothing to check.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@sethbergman
sethbergman merged commit 6d55ab5 into main Sep 16, 2026
38 checks passed
@sethbergman
sethbergman deleted the test/readme-version-claim-lint branch September 16, 2026 02:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant