[quality] ten data-dependent test.skip() sites in tests/e2e can retire browser coverage with a green run #1184

Finding

Ten end-to-end cases in tests/e2e/** select the record they drive out of the
committed data rather than naming it, and call test.skip() when the data
holds no such record. Playwright reports a skipped case as a non-failure, so
the day an upstream refresh removes the last matching record, the case stops
exercising the browser and CI stays green with no signal at all.

The repository already recognises the hazard — for one of the ten.
tests/e2e/member-directory.spec.js:20-27 states it outright:

on the day the landscape gives both of them one, a test.skip reading the
real file would quietly retire the dialog's only browser coverage of the
initials arm. A green run would report nothing, because a skipped case is not
a failing one.

and defends that one case with tests/e2e/fixtures/data/members.json, which
appends a logo-less member so the coverage build cannot lose it. That defence
is conditional on E2E_COVERAGE=1 (member-directory.spec.js:76-86): in the
plain End-to-end tests job even that case falls back to a silent skip. The
other nine sites have no defence in either job.

The ten sites

Enumerated mechanically at 03cfcfe by scanning tests/e2e/*.spec.js for
test.skip( with comments stripped (so prose mentions and commented-out calls
are not counted). test.describe.skip is excluded: those blocks turn on the
build (process.env.E2E_COVERAGE === '1'), not on the data.

Every one of the ten data files involved is regenerated from upstream on a
schedule rather than edited in a commit — npm run fetch:community-people
(.github/workflows/refresh-community-people.yml), npm run generate:members
from the pinned landscape snapshot, npm run collect:radar-reports
(.github/workflows/refresh-radar-reports.yml), and the metrics collector. The
condition each skip reads is therefore upstream state, not a source edit, and
no review sees it change.

Evidence and provenance

• main = 03cfcfe, checked 2026-10-08.
• All ten currently run. Each predicate was evaluated against the committed
  data/*.json at 03cfcfe (node v26.10.0): awards.verifiedAt and
  verifiedAgainst both present; 11 TAB people, with a bio-carrying one, a
  bio-less one, a located one, an unlocated one, and one missing a handle; 101
  members, with both a logo-carrying one and a logo-less one;
  referenceArchitectureLifecycle.omitted non-empty; radar.generatedAt
  present. So this is a latent failure mode, not a live coverage loss — which
  is exactly when it is cheap to close.
• This is not a coverage-gap finding and carries no coverage-gap priority:
  no source path is under-tested today. It is test-infrastructure — the
  mechanism that keeps the existing end-to-end coverage from retiring without
  a signal.

Recommendation

Add one unit guard, tests/e2e-data-dependent-skips.test.mjs. It runs in
npm run test:unit, which CI runs on every pull request (ci.yml:40), so it
reports in both the e2e and e2e-coverage worlds and in neither's absence.
Two halves, both needed:

1. Predicate checks. One case per registered site asserting the committed
   data still satisfies it, so a silent skip becomes a failing unit test that
   names the data shape the end-to-end case needs.
2. Inventory check. Scan tests/e2e/*.spec.js for test.skip( and assert
   the set of call sites matches the registered list, keyed on each skip's own
   message rather than its line number (messages are stable across edits;
   line numbers are not). Without this half, the next test.skip() added to a
   spec reintroduces the very hazard the guard exists to remove.

Both halves were verified to fail when they should, at 03cfcfe: inserting an
unregistered test.skip(false, 'an unregistered probe skip') into
awards.spec.js fails the inventory case, and giving every member in
data/members.json a logo fails exactly the
member-directory.spec.js:82 predicate case and no other. Both probes were
reverted.

Nothing outside tests/ changes, and no workflow file is involved.

Coordination

Disjoint from every open hold-gated PR. The new file is the only file touched;
none of #1161 (scripts/audit-gate.mjs), #1163
(tests/tools/e2e-coverage-report.mjs), #1165
(tests/architecture-content-mirror.test.mjs), #1166 (CONTRIBUTING.md),
#1168/#1176 (scripts/lib/svg-active-content.mjs), #1171
(tests/e2e-data-fixtures.test.mjs, the metrics.json overlays and the
metrics-sparkline* specs), #1174 (tests/e2e-coverage-run.test.mjs), #1178
(tests/e2e/focus-trap-trigger-unmounted.spec.js), #1181
(scripts/lib/architecture-content.mjs) or #1183 (package.json) touches it,
and none of them edits any of the five specs this guard reads. It is also
disjoint from the region-attribution work tracked by #1066/#1079: this guard
asserts nothing about the coverage report.

Priority

• Impact: medium (ten end-to-end cases can retire themselves on a scheduled
  data refresh with a green run and no annotation; the repository has already
  paid to defend one of them)
• Effort: low

🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: 03cfcfe

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88