fix(harness): GT-658 ask whether the renderer works before accusing 371 diagrams, and pin it - #444
Merged
Merged
Conversation
…71 diagrams, and pin it Closes GT-658. `01-validate-docs --render-mermaid` spawned `npx -y @mermaid-js/mermaid-cli` with NO version, once per diagram. On 2026-08-08 that install came out INCOMPLETE on a runner — `Cannot find package 'import-meta-resolve'`, which @mermaid-js/mermaid-cli@11.16.0 DOES declare (verified with npm view), so the tree was half-written rather than the publish being bad. Every diagram then failed as "mermaid render failed", attributed to the document containing it. The job took 23m21s to produce those failures on the develop -> main promotion, and the same command had succeeded twenty minutes earlier in a concurrent run. Two defects in one invocation: The pin. `npx -y` with no version resolves @latest when CI runs, so what executes is chosen by whoever published most recently. The tool also appears in package-lock.json ZERO times: npm ci does not install it, npm audit cannot see it, Dependabot cannot bump it, and GT-657's audit gate — built the same day — cannot reason about it. The blame. The corpus and the renderer are different things and only one of them can be broken by a commit, but the guard asked "is each of these 371 diagrams valid?" before "does the renderer work?". A broken renderer produced hundreds of confident, wrong accusations, each naming a file and line whose content was fine. The loudest signal pointed at the only innocent party. Fixed by asking the right question first. A preflight renders one trivial diagram; a failure there is ONE error naming the renderer and stating that no diagram is implicated. Exit 0 is not accepted as proof — a renderer that writes no SVG, or an empty one, has not rendered. The version is pinned to 11.16.0 in a single constant both the preflight and the per-diagram render read, so the two cannot drift apart. Observed red, not assumed: pinning a non-existent version gives exit 1, ZERO "mermaid render failed" lines, and "405 diagram(s) were NOT checked, and none of them is implicated". The first version of this preflight had the defect it exists to remove. On a machine with no usable Chromium, mermaid-cli exits 1 having written nothing to either stream, and the message interpolated to "exited 1 on a trivial diagram: " — uninformative, exactly like the failures being replaced. It now says the process was silent, names the likely cause, and prints the command to reproduce. NOT verified locally, and stated rather than implied: the happy path does not run on this machine for that same Chromium reason. The corpus render is exercised by `Evolith Core Validation` on the runner, green on PRs #440, #441 and #442. NOT done, as a separate decision: declaring the tool a pinned devDependency is the fuller fix — lockfile, npm audit, Dependabot, and `npm ci` failing loudly on exactly the partial install that caused this — but it drags Puppeteer and a Chromium download into every npm ci, for every developer, to render diagrams only CI renders. Worth taking deliberately, not as a side effect of a bug fix. Guards 03, 04, 08, 09 --check, 42, 43, 46, 49, 63 and bilingual-terminology-lint all exit 0. Board: 643 / 656 done, 3 in progress, 3 pending, 7 deferred. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
📊 Bilingual Coverage ImpactPR Changes
Repository Coverage
✅ Good: All EN changes have ES counterparts. Generated by GitHub Actions |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes GT-658. Found in the CI of #443, the promotion carrying GT-622/656/657.
01-validate-docs --render-mermaidspawnednpx -y @mermaid-js/mermaid-cliwith no version, once per diagram. That install came out incomplete on a runner —Cannot find package 'import-meta-resolve', which@mermaid-js/mermaid-cli@11.16.0does declare (verified withnpm view), so the tree was half-written rather than the publish being bad. Every diagram then failed asmermaid render failed, attributed to the document containing it. 23m21s to produce those failures; the same command had succeeded twenty minutes earlier in a concurrent run.Two defects in one invocation
The pin.
npx -ywith no version resolves@latestwhen CI runs — what executes is chosen by whoever published most recently, not by any commit here. The tool also appears inpackage-lock.jsonzero times:npm cidoes not install it,npm auditcannot see it, Dependabot cannot bump it, and GT-657's audit gate — built the same day — cannot reason about it. A binary that downloads Chromium and runs in CI sat outside every supply-chain control this repo has.The blame. The corpus and the renderer are different things and only one of them can be broken by a commit. The guard asked "is each of these 371 diagrams valid?" before "does the renderer work?", so a broken renderer produced hundreds of confident, wrong accusations — each naming a file and line whose content was fine. The loudest signal pointed at the only innocent party.
Fixed by asking the right question first
A preflight renders one trivial diagram. A failure there is one error naming the renderer and stating explicitly that no diagram is implicated. Exit 0 is not accepted as proof — a renderer that writes no SVG, or an empty one, has not rendered. The version is pinned to
11.16.0in a single constant both the preflight and the per-diagram render read, so the two cannot drift apart.Observed red, not assumed — pinning a non-existent version:
exit 1, with zero
mermaid render failedlines.The first version of this preflight had the defect it exists to remove
On a machine with no usable Chromium, mermaid-cli exits 1 having written nothing to either stream, and my message interpolated to
"exited 1 on a trivial diagram: "— uninformative, exactly like the failures being replaced. It now says the process was silent, names the likely cause, and prints the command to reproduce.Not verified locally, stated rather than implied
The happy path does not run on this machine, for that same Chromium reason. The corpus render is exercised by
Evolith Core Validationon the runner — that check on this PR is the verification, and it was green on #440, #441 and #442.Not done, as a separate decision
Declaring the tool a pinned devDependency is the fuller fix — lockfile,
npm audit, Dependabot, andnpm cifailing loudly on exactly the partial install that caused this. It also drags Puppeteer and a Chromium download into everynpm ci, for every developer, to render diagrams only CI renders. Worth taking deliberately, not as a side effect of a bug fix.Guards 03, 04, 08, 09
--check, 42, 43, 46, 49, 63 andbilingual-terminology-lintall exit 0. Board: 643 / 656 done.🤖 Generated with Claude Code