Skip to content

[quality] tests/tools/ is required to be measured but scored by no source coverage gate #1192

Description

@hivecommons-hive

Finding

tests/tools/ is a source root for one of the reporter's guarantees and
invisible to all the others.

const SOURCE_ROOTS = ['scripts', 'src', join('tests', 'tools')];

— tests/tools/coverage-report.mjs:91

// Coverage records exist for the test files too. They are not the code this
// repository ships, so they are summarised separately rather than folded
// into totals the suite's own near-complete self-coverage would dominate.
export function isSourceFile(file) {
  return !file.startsWith('tests/');
}

— tests/tools/coverage-report.mjs:448-452

SOURCE_ROOTS is what --require-source-files walks, so the gate insists
every module under tests/tools/ be measured — tests/coverage-report-source-files.test.mjs:95-99
spells out why: "The harness tree is enumerated too: a tool nothing imports
is the measuring apparatus going unmeasured."

isSourceFile() is what builds the src files row, and it drops everything
under tests/. So --check-source, --check-source-regions and the
per-file floor added in #1190 all score scripts/ and src/ and never
tests/tools/. The apparatus must be measured and is then held to no
standard. The only floor it falls under is --check-regions 95, an all-files
aggregate over 8708 regions that the suite's own test files dominate.

This is not symmetry for its own sake. tests/tools/ is where the coverage
reporters, the Playwright coverage fixture, the data-overlay loader and the
JSX/DOM harness live: a gap there is a gap in the instrument every other gate
reads.

Evidence and provenance

  • Unit — measured. npm run test:unit:coverage (TZ=UTC node tests/tools/coverage-report.mjs, node v26.10.0) run locally on 2026-10-08
    at 03cfcfe. Every tests/tools/ row:

    file line % region %
    tests/tools/e2e-coverage-report.mjs 100.00 93.20
    tests/tools/e2e-coverage-run.mjs 100.00 98.65
    the other 13 modules 100.00 100.00

    The same run reports src files | 100.00 | 99.92 | 8984/8984 lines | 2545/2547 regions, which is how a 93.20% file sits in the tree while the
    gated row reads 99.92%.

  • Both sub-100% harness files are at 100% lines, so the line gates are
    structurally blind to them as well: their gaps are sub-line regions
    (an unexecuted ?? fallback or ternary arm), the same shape [quality] the unit coverage gate is aggregate-only: 23 source regions can go uncovered without failing it #1190
    describes.

  • E2E — not applicable. This is a property of the reporter's gate
    configuration, not of a code path, so no end-to-end coverage claim is made
    and this is a coverage-reporting finding rather than a coverage-gap.

Recommendation

Give the harness a floor of its own rather than folding it into src files
— the comment at coverage-report.mjs:448-450 is right that the suite's
self-coverage would dominate a merged total, and that reasoning does not
extend to leaving the harness ungated:

  • tests/tools/coverage-report.mjs summarises tests/tools/ as its own
    row beside src files and all files, built from the same
    SOURCE_ROOTS entry --require-source-files already walks
  • it accepts a floor for that row (line and region), failing with the
    file, ratio and uncovered region start lines the way the other gates do
  • package.json's test:unit:coverage:check passes the floor, set from
    what the harness measures at the time it lands
  • tests/coverage-gate-thresholds.test.mjs records the matching floor

Coordination — blocked on #1191

This must not start until #1191
(hold-gated, for #1190) lands or closes. That PR edits
tests/tools/coverage-report.mjs's flag table, report() return value and
gate block, package.json's scripts.test:unit:coverage:check line and
tests/coverage-gate-thresholds.test.mjs — the same four places every box
above edits. Two PRs over report() and the same gate string would conflict
on every merge.

It is disjoint from the other open hold-gated PRs. #1163 and #1174 touch
tests/tools/e2e-coverage-report.mjs and tests/tools/e2e-coverage-run.mjs
— the two files this finding measures, not the reporter that would gate
them; whatever those PRs do to their region percentages, the structural hole
is unchanged and the floor is set after they land. #1161, #1165, #1166,
#1168, #1171, #1176, #1178, #1181 and #1185 touch neither the reporter nor
the gate. #1183 touches package.json but adds an engines block, not a
scripts entry.

No workflow file is involved: .github/workflows/ci.yml:40 already runs
npm run test:unit:coverage:check.

Priority

  • Impact: medium (the measuring apparatus is required to be measured and then
    scored by nothing; two of its fifteen modules already sit below every floor
    the repository applies to shipped code)
  • Effort: medium

🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: 1e91668

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

Activity

  1. added
    qualityApproved by a Hive merger/owner for auto-merge on green CI
    testingApproved by a Hive merger/owner for auto-merge on green CI
    agent/qualityApproved by a Hive merger/owner for auto-merge on green CI
    on Oct 8, 2026
  2. mrbobbytables commented on Oct 8, 2026

    @mrbobbytables
    Member

    Status: the stated blocker has cleared — PR #1191 (per-file source-region floor, for #1190) merged. This is now unblocked and actionable.

    Two notes for whoever picks it up:

    1. test(coverage): floor source region coverage per file, not only in aggregate #1191 already reshaped the four surfaces this issue edits (tests/tools/coverage-report.mjs flag table / report() / gate block, package.json's test:unit:coverage:check, and tests/coverage-gate-thresholds.test.mjs), so re-check the exact flag surface before implementing — the gate line now reads --check 99 --check-source 100 --check-regions 95 --check-source-regions 99 --check-source-file-regions 97 --require-source-files.
    2. The floor for the new tests/tools row should be measured after the currently-open coverage-tooling work lands; PR test: cover the CSS scanners' unterminated-string fallback arms #1200 (unit tests only) does not touch the reporter.

    Related: #1187 also extends the coverage harness under tests/tools/ (e2e fixture builds); the two are disjoint in files but both ratchet the measuring apparatus, so cross-linking for sequencing awareness.

  3. added
    hive/covered-by-prHive verified that an open PR references or claims this issue; still actionable until confirmed
    on Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/qualityApproved by a Hive merger/owner for auto-merge on green CIhive/covered-by-prHive verified that an open PR references or claims this issue; still actionable until confirmedhive/hosted-available-lke648397-260827-5n31Approved by a Hive merger/owner for auto-merge on green CIqualityApproved by a Hive merger/owner for auto-merge on green CItestingApproved by a Hive merger/owner for auto-merge on green CI

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions