Project every page into markdown, and count the restatements (#217)

[bdelanghe]

Closes #217.

"Also we repeat ourselves a lot the pages should project into markdown we could read those."

Two asks, and the second is why the first matters.

The projection

Every page now has a .md at the same route — /index.md, /map.md, /desk.md, /blog/….md — indexed in llms.txt and covered by the signed whole-site manifest (58 files, up from 50).

One rule, two cases:

• a page authored in markdown (blog/*.md) serves its source. Projecting the built HTML back would create a second copy of every post to keep in sync, which is the exact failure this change is about.
• every other page is projected from its built HTML, so the generated regions (nav, /map, /desk, /ledger, /conformance) are in it. Projecting the source files would silently omit most of some pages.

The projection is also committed under content/pages/ and drift-checked, like gen-seams / gen-map / gen-desk / gen-ledger already are. That is the half that pays off in review: a copy change shows up as a prose diff next to the markup one.

The honesty stamp region is dropped from the projection. It is rewritten on every build, so carrying it into a committed file would make that file stale the moment anything else merged — and dropping it makes the projection a pure function of the source, which is what lets --check mean anything.

Sample, content/pages/index.md:

Bounded Systems leans on a single mechanism. For each kind of system power, there is one sanctioned way through. We call it a capability seam: a door you hold a socket to, never the keys behind it.

room — a named bundle of doors for one kind of stay. Ask for a room it doesn't know and it fails closed — a typo never widens authority.

The measurement

The obvious check — find sentences that appear twice — finds almost nothing here: 2% of sentences repeat verbatim, and most of those are nav chrome. That number is reassuring and wrong.

This site repeats itself by restatement. The homepage says, in three places:

• "Every claim on this page is graded against the running code."
• "Every claim here is graded against the running code."
• "A solo project — every claim on this page graded against the code that backs it."

Three phrasings, no two identical. So the comparison is over content words, not strings: strip stopwords and short words, take the set, score pairs by Jaccard overlap. It reads the markdown, not the HTML, so it sees the words a reader sees and nothing about markup.

298 sentences across 13 page(s) · 19 restatement pair(s) at ≥34% shared vocabulary
                                  (8 within a page, 11 across pages)

The top of the list:

	where
83%	conformance	"SPDX · present + valid + complete + signed" / "SPDX SBOM present, valid, complete, and signed"
80%	index	"Every claim on this page is graded against the running code." / "Every claim here is graded against the running code."
80%	blog ↔ index	"So docs generate from source and fail CI on drift, and behaviour specs execute against the engine." / "Docs generate from source and fail CI on drift; guest-room's specs execute against its engine."
60%	blog ↔ index	"One sanctioned door per kind of power." / "For each kind of system power, there is one sanctioned way through."
56%	blog ↔ index	"The door checks an owner and a policy, acts or refuses, and records either way" / "That point checks your policy, acts or refuses, and records what happened."
50%	index	"The unsolved problem is keeping many bounded, agent-authored abstractions honest against each other as they evolve…" / "What nobody has solved yet is keeping many of them honest as they evolve…"

npm run repetition -- --list prints all 19.

Ratcheted, not a cliff. That count is real and large; failing the build on it today would block every unrelated change, and a gate nobody can go green against gets switched off. The committed baseline is a ceiling: a change may not add repetition, and --update lowers it when a change removes some.

A missing baseline is refused rather than silently written. That happened once here: npm run check ran before the projection existed, saw only the five blog posts, and wrote a ceiling of 4 that the next run then "regressed" against. A ratchet that picks its own starting point from whatever it saw first is worse than none.

Honest about what this is

Vocabulary overlap is a proxy for restatement, not a judge of it. Two sentences can share their nouns and say opposite things, and a deliberate callback is repetition on purpose. The report prints the pairs, not just the number, because it is a reading list rather than a verdict.

The HTML parser is small and deliberately unambitious: it handles the tags this site uses and degrades unknown ones to their text rather than throwing, because a projection that fails closed on a formatting detail would block the build for no gain.

Both gates are mutation-tested

.github/workflows/markdown.yml proves each can fail before trusting either: edit the projection → the drift check must go red; lower the ceiling → the ratchet must go red. Rule 3 of docs/agentic-code-hygiene.md — a gate's own claim about itself is not evidence. Restores are from copies, never git checkout, so running the script locally can't discard uncommitted work.

Not in scope

Cutting the repetition. This lands the instrument and records the number. What to cut is editorial and should be decided while looking at the list, not bundled into the change that produced it.

Two things the projection surfaced

1. The room/door/guest definitions are <span class="defn__term"> / <span class="defn__body"> — a description list wearing spans. In markdown they fuse into "rooma named bundle of doors", which is why the projector needs a pairing rule for them. The real fix is a <dl> in the HTML, which would also help screen readers. Worth its own ticket.
2. _headers already carried /*.md → text/markdown; charset=utf-8 before any .md existed to serve. That type is correct, and browsers download rather than render it. curl and editors read it fine. Say the word if you'd rather they render inline — that's a one-line header change, but it's site-wide policy so I've left it alone.

Verified locally

Run the way CI runs them — node build.mjs plus the four brand tools from a clone of brand at the installed version:

• npm run check (now 23 gates, including check-repetition) ✓
• node scripts/gen-markdown.mjs --check ✓
• brand meta / content / coverage --colors / a11y ✓
• axe-gate — 13 pages, 0 serious/critical ✓
• a11y-heuristic — 0 errors ✓
• structure-audit ✓ (13 pages, unchanged hash — the .md twins are not pages in the graph)
• emit-catalog --check ✓
• mutation: drift check red on an edited projection, ratchet red on a lowered ceiling ✓

---

Generated by Claude Code

[bdelanghe]

Superseded by #220, which merged this branch in whole — the legibility gate reads the markdown projection, so they had to travel together. Every commit here is in main as of c7dceb8; nothing was dropped.

The repetition ratchet went with it, and the homepage rewrite immediately paid it down: 19 → 7 pairs, ceiling lowered.

Closing #217 alongside this.

---

Generated by Claude Code