Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Before opening a PR, read
[`CONTRIBUTING.md#making-changes`](CONTRIBUTING.md#making-changes) for the full
fork/branch/PR process. Two CI-enforced requirements to note up front: the
"Validate repository" check runs `npm run test:unit:coverage:check` (coverage
thresholds 99% lines / 100% source / 95% regions / 99% source regions) on every
thresholds 99% lines / 100% source / 96% regions / 100% source regions) on every
PR, and every commit must carry a DCO `Signed-off-by` trailer (`git commit -s`)
or CI will fail the PR.

Expand All @@ -50,8 +50,8 @@ or CI will fail the PR.
- **Build**: `npm run build`
- **Unit Tests**: `npm run test:unit:coverage:check` — required "Validate
repository" CI check on every PR; enforces coverage thresholds (99% lines /
100% source / 95% regions / 99% source regions), plus separate floors for the
`tests/tools/` harness (100% lines / 97% regions, 93% regions per harness
100% source / 96% regions / 100% source regions), plus separate floors for the
`tests/tools/` harness (100% lines / 97% regions, 97% regions per harness
file). `npm run test:unit` alone skips the coverage gate.
- **End-to-end Tests**: required "End-to-end tests" CI check on every PR. Needs
a production build first, then a one-time browser install:
Expand Down
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,9 +177,9 @@ enough for a first contribution.
2. Make your change and run the relevant validation script.
3. Run `npm run test:unit:coverage:check` — the required "Validate repository"
check runs this on every PR, and it fails the build if unit-test coverage
drops below its thresholds (99% lines / 100% source / 95% regions / 99%
drops below its thresholds (99% lines / 100% source / 96% regions / 100%
source regions). The harness under `tests/tools/` is floored separately (100%
lines / 97% regions, and 93% regions per harness file): it is measured like
lines / 97% regions, and 97% regions per harness file): it is measured like
source but reported in its own `harness files` row, so a gap in the test
apparatus cannot be paid for out of `src/`'s slack. `npm run test:unit` alone
skips the coverage gate, so a green `test:unit` locally does not guarantee a
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@
"pr-queue-hygiene": "node scripts/pr-queue-hygiene.mjs",
"test:unit": "TZ=UTC node --test",
"test:unit:coverage": "TZ=UTC node tests/tools/coverage-report.mjs",
"test:unit:coverage:check": "TZ=UTC node tests/tools/coverage-report.mjs --check 99 --check-source 100 --check-regions 95 --check-source-regions 99 --check-source-file-regions 97 --check-harness 100 --check-harness-regions 97 --check-harness-file-regions 97 --require-source-files",
"test:unit:coverage:check": "TZ=UTC node tests/tools/coverage-report.mjs --check 99 --check-source 100 --check-regions 96 --check-source-regions 100 --check-source-file-regions 100 --check-harness 100 --check-harness-regions 97 --check-harness-file-regions 97 --require-source-files",
"test:e2e": "playwright test",
"build:e2e:coverage": "npm run build:e2e:coverage:site && npm run build:e2e:coverage:fixtures",
"build:e2e:coverage:site": "DOCUSAURUS_NO_PERSISTENT_CACHE=1 E2E_COVERAGE=1 npm run build:production",
Expand Down
61 changes: 42 additions & 19 deletions tests/coverage-gate-thresholds.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -28,36 +28,59 @@ const GATE_SCRIPT = 'test:unit:coverage:check';

// Measured on the suite this commit ships, floored to a whole percent so a
// rounding difference between runs cannot fail the build:
// all files 99.42% lines / 95.07% regions (48197/48476 lines, 8060/8478 regions)
// src files 100.00% lines / 99.96% regions (8721/8721 lines, 2475/2476 regions)
// all files 99.65% lines / 96.14% regions (58948/59153 lines, 9515/9897 regions)
// src files 100.00% lines / 100.00% regions (9233/9233 lines, 2777/2777 regions)
//
// Only `--check-regions` has cleared a whole percent since the floors were
// last set; the other three sit under their next whole percent (99.42, 99.96)
// or are already at 100, so flooring leaves them where they are.
// Both figures were taken twice, on node v22.21.1 -- the major ci.yml pins --
// and on node v26.10.0, and the two runs agree to the region. Region counts
// are a V8 artifact, so a floor set at exactly 100 is only safe once it has
// been shown not to move with the runtime; these did not.
//
// `--check` sits under its next whole percent (99.65) and `--check-source` is
// already at 100, so flooring leaves both where they are. The three region
// floors below have all cleared a whole percent since they were last set.
const FLOORS = {
'--check': 99,
'--check-source': 100,
'--check-regions': 95,
'--check-source-regions': 99,
'--check-regions': 96,
// scripts/ and src/ are at 2777/2777: every region executed, with no
// uncreditable residual of the kind that keeps the browser gate (#1256) and
// the harness per-file gate below their own ceilings. 100 is therefore the
// measured floor here rather than an aspiration. A new defensive fallback in
// either tree now has to be reached by a test or argued for by lowering this
// deliberately -- which is the point: at 99 the gate let 27 source regions
// go dark before it noticed.
'--check-source-regions': 100,
// Per-file rather than aggregate, so its floor is set by the worst source
// file rather than by the whole of scripts/ and src/ together. Measured on
// the suite this commit ships, the lowest is
// scripts/lib/architecture-content.mjs at 97.78%, floored to 97.
'--check-source-file-regions': 97,
// the suite this commit ships, every file under both trees is at 100.00%,
// so the worst is 100. Kept alongside the aggregate even though an aggregate
// of 100 implies it: the per-file check is what names the offending file and
// its uncovered regions in the failure message.
'--check-source-file-regions': 100,
// The harness tree (tests/tools/) is walked by --require-source-files and
// entered none of the ratios above, which are all built from
// isSourceFile(). These are the same three shapes over its own row.
// Measured on the suite this commit ships: harness files 100.00% lines /
// 99.53% regions (3925/3925 lines, 1467/1474 regions), and the lowest
// harness file is tests/tools/e2e-coverage-report.mjs at 98.57%, floored
// to 97. Every other harness file is at 100.00%.
// 99.57% regions (5336/5336 lines, 1867/1875 regions), now that #1294 has
// brought tests/helpers*.mjs into the row. The lowest harness file is
// tests/helpers-script-sandbox.mjs at 97.06%, floored to 97; next is
// tests/tools/e2e-coverage-report.mjs at 98.57%, and every other harness
// file is at 100.00%.
//
// 98.57% is e2e-coverage-report.mjs's ceiling, not a backlog: its 7
// uncovered regions are the defensive fallbacks the note at the top of the
// file lists as unreachable (#1253; three more were deleted rather than
// listed in #1304), so no test can lift it. The per-file floor must stay at
// or below 97 while helpers-script-sandbox.mjs sits at 97.06, and at or
// below 98 until that list shrinks; a probe that lands one of them should
// raise the floor and shorten the list in the same change.
//
// 98.57% is also that file's ceiling, not a backlog: its 7 uncovered
// regions are the defensive fallbacks the note at the top of the file lists
// as unreachable (#1253; three more were deleted rather than listed in
// #1304), so no test can lift it. The per-file floor must stay at or below
// 98 until that list shrinks; a probe that lands one of them should raise
// the floor and shorten the list in the same change.
// --check-harness-regions is left at 97 against a measured 99.57 on purpose:
// #1294 only just rescoped the row, and the helper tree's lowest file is
// 0.06 points above the per-file floor, so the aggregate is left with its
// slack until that file has been lifted rather than ratcheted into a gate
// the next helper edit trips.
'--check-harness': 100,
'--check-harness-regions': 97,
'--check-harness-file-regions': 97,
Expand Down
Loading