Repository navigation
quality: remove pragma-based coverage suppression from owned Python tooling #493
Description
Finding
Fresh review of #488@2f699fa6818abd4a8b1a20f915a2596177ebb544 found that the newly added raw-string LCOV regression correctly removed its copied # pragma: no cover, but the repository-wide Python coverage policy still permits the same suppression pattern for owned tooling.
Exact-head .coveragerc@8f5ce471f2fca5639e7e8857e39d404698af521a has fail_under = 100, branch coverage enabled, and exclude_lines = pragma: no cover. The covered production source set includes scripts.check_workspace_contract, scripts.check_docstrings, scripts.check_coverage, scripts.prepare_agent_pr_message, scripts.release_evidence, and scripts.actions_workflow_fleet.
At least these covered production entrypoints retain # pragma: no cover on their if __name__ == "__main__" guards: scripts/check_docstrings.py, scripts/check_workspace_contract.py, scripts/release_evidence.py, and scripts/actions_workflow_fleet.py. prepare_agent_pr_message already has an in-process runpy.run_module(..., run_name="__main__") regression pattern that demonstrates the preferred direction: execute the real CLI boundary rather than remove it from the denominator.
This is quality/operability evidence, not a Validation Evidence arithmetic finding. It must not be repaired by lowering fail_under, excluding more files/lines, moving source outside [run].source, adding skip/xfail, or rewriting the source denominator.
Required RED → GREEN
- Add focused in-process CLI-entrypoint tests for every owned covered script whose
__main__guard is presently suppressed. Use deterministic local fixtures/mocks so tests do not require live network mutation or provider credentials. - Remove the corresponding
# pragma: no coverannotations only after the real entrypoint path is exercised. - Remove the global
exclude_lines = pragma: no coverpolicy once the covered production source set has no legitimate reliance on it; a regression must fail if an owned production source reintroduces pragma-based suppression. - Preserve branch coverage,
fail_under = 100, covered source membership, current LCOV authored-line semantics, and all scientific arithmetic/contracts. - Prove exact-head Python line+branch 100% without exclusion-based denominator manipulation, then update
TEST_STRATEGY/operability traceability if the coverage policy is documented there.
Boundary
Do not fold unrelated test-file unittest.main() convenience guards into production coverage just to increase denominator. The acceptance target is the owned production source set selected by .coveragerc; test files may be cleaned separately for consistency, but they are not a substitute for exercising production CLI boundaries.
Current #488 CI is runner-queued, so this issue records a separate verified repair frontier without churning that exact scientific head merely to retrigger Actions.
Activity
seonghobae commentedon Sep 15, 2026
Status: the fix exists and is green on everything a branch controls
Measured against
origin/maintoday, this issue is still open in substance:and five of the six modules under
[run] sourcestill carry a pragma:pragma: no coveroccurrences on mainscripts/check_workspace_contract.pyscripts/check_docstrings.pyscripts/check_coverage.pyscripts/release_evidence.pyscripts/actions_workflow_fleet.pyscripts/prepare_agent_pr_message.pyThe repair is #497. It removes the
exclude_linesstanza and the pragmas, and replaces the suppression withtests/quality/test_cli_entrypoints.py, which executes each covered module's__main__boundary in-process throughrunpy.run_module(..., run_name="__main__")and asserts the exit status, so the boundary is measured rather than removed from the denominator.Its current check state is worth recording precisely, because it is not a code problem:
dependency-reviewNothing else fails. The Python gate, the coverage floor, and the contract checkers all pass.
So this issue is gated on #497's base merging plus the two infrastructure conditions above, not on further implementation work.
One related change since
While repairing a separate coverage-gate hole I removed two exclusions from
scripts/check_coverage.pythat had the same character as the pragma suppression this issue is about: a match arm whose body is one string literal, andOk(())as a function's success return. Both were dropped from the authored-line denominator, so a zero count on either could never fail the contract. That work is in #525 and is independent of this issue, but it is the same failure mode, and worth knowing about when judging whether the suppression problem is fully retired.seonghobae commentedon Sep 16, 2026
Fresh current-head verification: coverage owner vehicle #538 is now Ready/main-target at exact
0a6947b11f2dba9e161f2abe8bdec0f652d4b47a..coveragercstill hasexclude_lines = pragma: no cover, andscripts/check_docstrings.pystill carriesif __name__ == "__main__": # pragma: no cover, so this issue remains valid and is not inherited by #538 yet. Do not open a competing coverage sibling while #538 exact-head gates are queued. Once that head reaches a practical terminal point, the next coverage-policy batch should use the surviving #538 lineage (or its verified successor): public/in-process CLI REDs first, then remove owned production pragmas and finally the global pragma exclusion with a regression forbidding reintroduction. No loweringfail_under, source-set shrink, skip/xfail, or denominator rewrite.seonghobae commentedon Sep 16, 2026
#497 is now repaired onto the surviving foundation lineage instead of the unrelated #492 base.
Current implementation authority:
#497@6f6ee499e80b0537c363baf8d94fd4c46c91ce9d, direct base#538@0a6947b11f2dba9e161f2abe8bdec0f652d4b47a, still Draft.Fresh compare against #538 is ahead/behind
10/0and the effective product delta remains exactly nine paths:.coveragerc,.gitignore, one changelog fragment, five covered CLI scripts, andtests/quality/test_cli_entrypoints.py. The first reconciliation tree accidentally dropped three explanatory comments fromscripts/check_coverage.py; ordinary-forward6f6ee499...restores them. Currentscripts/check_coverage.pydiffers from #538 only at the__main__pragma marker plus terminal-newline normalization; the later LCOV classifier/reconciliation logic is preserved.Because #497 is Draft while #538 is unlanded, Rust Foundation and Documentation Quality on
6f6ee499...are intentionally skipped. Those skips are lifecycle evidence, not GREEN promotion evidence. Keep #493 open until #538 lands, #497 is restacked to the surviving protected-main authority if needed, and the unchanged exact head proves real CLI entrypoint execution, no pragma suppression/global exclusion, branch coverage +fail_under=100, all required terminal gates, and independent approval.