From d39042801ab28acdf67360030a943329cf300fdc Mon Sep 17 00:00:00 2001 From: quality Date: Sun, 11 Oct 2026 00:20:46 -0400 Subject: [PATCH 1/3] test(coverage): ratchet the unit region floors to the 100% scripts/ and src/ already measure `test:unit:coverage:check` is the gate ci.yml runs, and three of its eight floors sat below what the suite achieves. scripts/ and src/ are at 2753/2753 regions -- every region executed, every file in both trees at 100.00% on its own -- while the gate asked for 99% in aggregate and 97% per file, so 27 source regions could go dark before a run turned red. tests/coverage-gate-thresholds.test.mjs states the rule this breaks in its own header: raise these floors in the same change when coverage climbs. The climb happened and the ratchet did not; its recorded measurement was stale by 278 regions (2475/2476 against 2753/2753 today). --check-regions 95 -> 96 (measured 96.12) --check-source-regions 99 -> 100 (measured 100.00) --check-source-file-regions 97 -> 100 (lowest source file 100.00) Unlike the browser gate, which stays under its ceiling because 18 regions are uncreditable (#1256), the unit source number has no residual list, so 100 is the measured floor rather than an aspiration. Measured twice -- node v22.21.1, the major ci.yml pins, and node v26.10.0 -- because region counts are a V8 artifact and a floor at exactly 100 is only safe if it does not move with the runtime. The two runs agree to the region. The two --check-harness-* floors are deliberately left at 97: #1294 rescopes harness scoring to take in tests/helpers*.mjs, whose lowest file is 97.06%, and ratcheting them before that lands would collide with it. CONTRIBUTING.md and AGENTS.md restate these thresholds in prose and are gated by tests/docs-threshold-parity.test.mjs, so they move in this commit too. Closes #1297 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: quality --- AGENTS.md | 4 +-- CONTRIBUTING.md | 2 +- package.json | 2 +- tests/coverage-gate-thresholds.test.mjs | 34 +++++++++++++++++-------- 4 files changed, 28 insertions(+), 14 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index b3b8bbed..95f8371e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -37,7 +37,7 @@ Before opening a PR, read [`CONTRIBUTING.md#making-changes`](CONTRIBUTING.md#making-changes) for the full fork/branch/PR process. Two CI-enforced requirements to note up front: the "Validate repository" check runs `npm run test:unit:coverage:check` (coverage -thresholds 99% lines / 100% source / 95% regions / 99% source regions) on every +thresholds 99% lines / 100% source / 96% regions / 100% source regions) on every PR, and every commit must carry a DCO `Signed-off-by` trailer (`git commit -s`) or CI will fail the PR. @@ -50,7 +50,7 @@ or CI will fail the PR. - **Build**: `npm run build` - **Unit Tests**: `npm run test:unit:coverage:check` — required "Validate repository" CI check on every PR; enforces coverage thresholds (99% lines / - 100% source / 95% regions / 99% source regions), plus separate floors for the + 100% source / 96% regions / 100% source regions), plus separate floors for the `tests/tools/` harness (100% lines / 97% regions, 93% regions per harness file). `npm run test:unit` alone skips the coverage gate. - **End-to-end Tests**: required "End-to-end tests" CI check on every PR. Needs diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6e45bb76..93c8701f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -177,7 +177,7 @@ enough for a first contribution. 2. Make your change and run the relevant validation script. 3. Run `npm run test:unit:coverage:check` — the required "Validate repository" check runs this on every PR, and it fails the build if unit-test coverage - drops below its thresholds (99% lines / 100% source / 95% regions / 99% + drops below its thresholds (99% lines / 100% source / 96% regions / 100% source regions). The harness under `tests/tools/` is floored separately (100% lines / 97% regions, and 93% regions per harness file): it is measured like source but reported in its own `harness files` row, so a gap in the test diff --git a/package.json b/package.json index a0a13301..386251a5 100644 --- a/package.json +++ b/package.json @@ -70,7 +70,7 @@ "pr-queue-hygiene": "node scripts/pr-queue-hygiene.mjs", "test:unit": "TZ=UTC node --test", "test:unit:coverage": "TZ=UTC node tests/tools/coverage-report.mjs", - "test:unit:coverage:check": "TZ=UTC node tests/tools/coverage-report.mjs --check 99 --check-source 100 --check-regions 95 --check-source-regions 99 --check-source-file-regions 97 --check-harness 100 --check-harness-regions 97 --check-harness-file-regions 97 --require-source-files", + "test:unit:coverage:check": "TZ=UTC node tests/tools/coverage-report.mjs --check 99 --check-source 100 --check-regions 96 --check-source-regions 100 --check-source-file-regions 100 --check-harness 100 --check-harness-regions 97 --check-harness-file-regions 97 --require-source-files", "test:e2e": "playwright test", "build:e2e:coverage": "npm run build:e2e:coverage:site && npm run build:e2e:coverage:fixtures", "build:e2e:coverage:site": "DOCUSAURUS_NO_PERSISTENT_CACHE=1 E2E_COVERAGE=1 npm run build:production", diff --git a/tests/coverage-gate-thresholds.test.mjs b/tests/coverage-gate-thresholds.test.mjs index 6a427887..81cc5d6c 100644 --- a/tests/coverage-gate-thresholds.test.mjs +++ b/tests/coverage-gate-thresholds.test.mjs @@ -28,22 +28,36 @@ const GATE_SCRIPT = 'test:unit:coverage:check'; // Measured on the suite this commit ships, floored to a whole percent so a // rounding difference between runs cannot fail the build: -// all files 99.42% lines / 95.07% regions (48197/48476 lines, 8060/8478 regions) -// src files 100.00% lines / 99.96% regions (8721/8721 lines, 2475/2476 regions) +// all files 99.65% lines / 96.12% regions (57649/57853 lines, 9305/9681 regions) +// src files 100.00% lines / 100.00% regions (9135/9135 lines, 2753/2753 regions) // -// Only `--check-regions` has cleared a whole percent since the floors were -// last set; the other three sit under their next whole percent (99.42, 99.96) -// or are already at 100, so flooring leaves them where they are. +// Both figures were taken twice, on node v22.21.1 -- the major ci.yml pins -- +// and on node v26.10.0, and the two runs agree to the region. Region counts +// are a V8 artifact, so a floor set at exactly 100 is only safe once it has +// been shown not to move with the runtime; these did not. +// +// `--check` sits under its next whole percent (99.65) and `--check-source` is +// already at 100, so flooring leaves both where they are. The three region +// floors below have all cleared a whole percent since they were last set. const FLOORS = { '--check': 99, '--check-source': 100, - '--check-regions': 95, - '--check-source-regions': 99, + '--check-regions': 96, + // scripts/ and src/ are at 2753/2753: every region executed, with no + // uncreditable residual of the kind that keeps the browser gate (#1256) and + // the harness per-file gate below their own ceilings. 100 is therefore the + // measured floor here rather than an aspiration. A new defensive fallback in + // either tree now has to be reached by a test or argued for by lowering this + // deliberately -- which is the point: at 99 the gate let 27 source regions + // go dark before it noticed. + '--check-source-regions': 100, // Per-file rather than aggregate, so its floor is set by the worst source // file rather than by the whole of scripts/ and src/ together. Measured on - // the suite this commit ships, the lowest is - // scripts/lib/architecture-content.mjs at 97.78%, floored to 97. - '--check-source-file-regions': 97, + // the suite this commit ships, every file under both trees is at 100.00%, + // so the worst is 100. Kept alongside the aggregate even though an aggregate + // of 100 implies it: the per-file check is what names the offending file and + // its uncovered regions in the failure message. + '--check-source-file-regions': 100, // The harness tree (tests/tools/) is walked by --require-source-files and // entered none of the ratios above, which are all built from // isSourceFile(). These are the same three shapes over its own row. From 5b11b2b14a870b92e3df9db6a132c39a6e0758e7 Mon Sep 17 00:00:00 2001 From: Bob Killen Date: Sun, 11 Oct 2026 09:04:02 -0500 Subject: [PATCH 2/3] docs: record the 97% per-file harness region floor in AGENTS.md and CONTRIBUTING.md Both files still said 93%, stale since #1250 ratcheted --check-harness-file-regions to 97. They are the lines this change already edits, so the fix rides here. Closes #1306 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Bob Killen --- AGENTS.md | 2 +- CONTRIBUTING.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 95f8371e..f8bad0ea 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -51,7 +51,7 @@ or CI will fail the PR. - **Unit Tests**: `npm run test:unit:coverage:check` — required "Validate repository" CI check on every PR; enforces coverage thresholds (99% lines / 100% source / 96% regions / 100% source regions), plus separate floors for the - `tests/tools/` harness (100% lines / 97% regions, 93% regions per harness + `tests/tools/` harness (100% lines / 97% regions, 97% regions per harness file). `npm run test:unit` alone skips the coverage gate. - **End-to-end Tests**: required "End-to-end tests" CI check on every PR. Needs a production build first, then a one-time browser install: diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 93c8701f..809e32d2 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -179,7 +179,7 @@ enough for a first contribution. check runs this on every PR, and it fails the build if unit-test coverage drops below its thresholds (99% lines / 100% source / 96% regions / 100% source regions). The harness under `tests/tools/` is floored separately (100% - lines / 97% regions, and 93% regions per harness file): it is measured like + lines / 97% regions, and 97% regions per harness file): it is measured like source but reported in its own `harness files` row, so a gap in the test apparatus cannot be paid for out of `src/`'s slack. `npm run test:unit` alone skips the coverage gate, so a green `test:unit` locally does not guarantee a From b37ce6efb236cabd3b3e7c9767ad7e118d02e060 Mon Sep 17 00:00:00 2001 From: Bob Killen Date: Sun, 11 Oct 2026 09:19:30 -0500 Subject: [PATCH 3/3] test(coverage): re-record the gate measurements after #1294 and #1305 Rebased onto main: src is 2777/2777 regions, the harness row now includes tests/helpers*.mjs (5336/5336 lines, 1867/1875 regions, lowest file helpers-script-sandbox.mjs at 97.06), and e2e-coverage-report.mjs's ceiling is 98.57 with 7 residual regions. All three ratcheted floors still clear. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Bob Killen --- tests/coverage-gate-thresholds.test.mjs | 33 ++++++++++++++++--------- 1 file changed, 21 insertions(+), 12 deletions(-) diff --git a/tests/coverage-gate-thresholds.test.mjs b/tests/coverage-gate-thresholds.test.mjs index 81cc5d6c..ca7455aa 100644 --- a/tests/coverage-gate-thresholds.test.mjs +++ b/tests/coverage-gate-thresholds.test.mjs @@ -28,8 +28,8 @@ const GATE_SCRIPT = 'test:unit:coverage:check'; // Measured on the suite this commit ships, floored to a whole percent so a // rounding difference between runs cannot fail the build: -// all files 99.65% lines / 96.12% regions (57649/57853 lines, 9305/9681 regions) -// src files 100.00% lines / 100.00% regions (9135/9135 lines, 2753/2753 regions) +// all files 99.65% lines / 96.14% regions (58948/59153 lines, 9515/9897 regions) +// src files 100.00% lines / 100.00% regions (9233/9233 lines, 2777/2777 regions) // // Both figures were taken twice, on node v22.21.1 -- the major ci.yml pins -- // and on node v26.10.0, and the two runs agree to the region. Region counts @@ -43,7 +43,7 @@ const FLOORS = { '--check': 99, '--check-source': 100, '--check-regions': 96, - // scripts/ and src/ are at 2753/2753: every region executed, with no + // scripts/ and src/ are at 2777/2777: every region executed, with no // uncreditable residual of the kind that keeps the browser gate (#1256) and // the harness per-file gate below their own ceilings. 100 is therefore the // measured floor here rather than an aspiration. A new defensive fallback in @@ -62,16 +62,25 @@ const FLOORS = { // entered none of the ratios above, which are all built from // isSourceFile(). These are the same three shapes over its own row. // Measured on the suite this commit ships: harness files 100.00% lines / - // 99.53% regions (3925/3925 lines, 1467/1474 regions), and the lowest - // harness file is tests/tools/e2e-coverage-report.mjs at 98.57%, floored - // to 97. Every other harness file is at 100.00%. + // 99.57% regions (5336/5336 lines, 1867/1875 regions), now that #1294 has + // brought tests/helpers*.mjs into the row. The lowest harness file is + // tests/helpers-script-sandbox.mjs at 97.06%, floored to 97; next is + // tests/tools/e2e-coverage-report.mjs at 98.57%, and every other harness + // file is at 100.00%. // - // 98.57% is also that file's ceiling, not a backlog: its 7 uncovered - // regions are the defensive fallbacks the note at the top of the file lists - // as unreachable (#1253; three more were deleted rather than listed in - // #1304), so no test can lift it. The per-file floor must stay at or below - // 98 until that list shrinks; a probe that lands one of them should raise - // the floor and shorten the list in the same change. + // 98.57% is e2e-coverage-report.mjs's ceiling, not a backlog: its 7 + // uncovered regions are the defensive fallbacks the note at the top of the + // file lists as unreachable (#1253; three more were deleted rather than + // listed in #1304), so no test can lift it. The per-file floor must stay at + // or below 97 while helpers-script-sandbox.mjs sits at 97.06, and at or + // below 98 until that list shrinks; a probe that lands one of them should + // raise the floor and shorten the list in the same change. + // + // --check-harness-regions is left at 97 against a measured 99.57 on purpose: + // #1294 only just rescoped the row, and the helper tree's lowest file is + // 0.06 points above the per-file floor, so the aggregate is left with its + // slack until that file has been lifted rather than ratcheted into a gate + // the next helper edit trips. '--check-harness': 100, '--check-harness-regions': 97, '--check-harness-file-regions': 97,