Repository navigation
Add a doc-parity unit test pinning documented coverage thresholds to the enforced gates #1193
Copy link
Copy link
Closed
Labels
documentationImprovements or additions to documentationImprovements or additions to documentationqualityApproved by a Hive merger/owner for auto-merge on green CIApproved by a Hive merger/owner for auto-merge on green CItestingApproved by a Hive merger/owner for auto-merge on green CIApproved by a Hive merger/owner for auto-merge on green CI🐝 covered by PRHive verified that an open PR references or claims this issue; still actionable until confirmedHive verified that an open PR references or claims this issue; still actionable until confirmed
Description
Problem: docs drift is being fixed per-instance, never as a class
10 of the last 100 closed issues were documentation-drift reports, and the same spots drift repeatedly:
- [guide] CONTRIBUTING.md says e2e coverage 'has no percentage thresholds' — CI gates it at --check-source 100 --check-source-regions 80 #1037: CONTRIBUTING.md said e2e coverage had 'no percentage thresholds' while CI gated at
--check-source 100 --check-source-regions 80 - [guide] CONTRIBUTING.md and AGENTS.md still document the unit region gate as 94% after #1152 ratcheted it to 95% #1169 (now Docs drift: sync LAUNCH.md verification dates and coverage-gate numbers with current state #1189): CONTRIBUTING.md and AGENTS.md still say the unit region gate is 94% after test: ratchet the unit region gate to the measured 95 #1152 ratcheted it to 95%
- [guide] CONTRIBUTING.md pre-PR checklist and CI-failure guidance omit the new required check:audit gate #1160: CONTRIBUTING.md omitted the new
check:auditgate - [guide] SECURITY.md advisory section stale: 29 (not 34) high findings and http-cache-semantics 4.3.0 fix now published #1045, [sec-check] http-cache-semantics 4.3.0 patches GHSA-ch52-4w7c-c8xp, but the lockfile is still on 4.2.0 and SECURITY.md says no fix exists #1049: SECURITY.md advisory counts went stale twice
The repository already proves the fix pattern works where it is applied:
tests/audit-gate.test.mjs:174pins the audit allowlist to SECURITY.md — that drift class has not recurred sincetests/workflow-scripts.test.mjs:355pins--check-source-regions 91to ci.ymltests/dev-environment.test.mjspins the Node major across devcontainer/workflows/docs (test(dev-environment): pin package.json engines.node into the Node-major parity chain #1183)
But nothing pins the threshold numbers stated in CONTRIBUTING.md and AGENTS.md to the values the gate actually enforces (tests/coverage-gate-thresholds.test.mjs / ci.yml). As of today, AGENTS.md:40 and CONTRIBUTING.md:176 still say '94% regions' while the enforced gate is 95 — the exact drift #1189 is about to fix for the second time.
Proposed fix
Add a unit test (e.g. tests/docs-threshold-parity.test.mjs) that:
- Extracts the single source of truth for each gate (the
--check-*flags asserted intests/coverage-gate-thresholds.test.mjsand.github/workflows/ci.yml). - Asserts every place CONTRIBUTING.md, AGENTS.md, and .github/copilot-instructions.md states those numbers matches.
Then the next ratchet PR fails locally until the docs are updated in the same commit, and the recurring guide-issue class ends.
Completion criteria
- A unit test fails when any documented coverage threshold diverges from the enforced value.
- The test covers CONTRIBUTING.md, AGENTS.md, and copilot-instructions.md at minimum.
- Lands with or after Docs drift: sync LAUNCH.md verification dates and coverage-gate numbers with current state #1189 so the current 94-vs-95 drift does not fail it on arrival.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
documentationImprovements or additions to documentationImprovements or additions to documentationqualityApproved by a Hive merger/owner for auto-merge on green CIApproved by a Hive merger/owner for auto-merge on green CItestingApproved by a Hive merger/owner for auto-merge on green CIApproved by a Hive merger/owner for auto-merge on green CI🐝 covered by PRHive verified that an open PR references or claims this issue; still actionable until confirmedHive verified that an open PR references or claims this issue; still actionable until confirmed
Activity