ci(report): a pull request says what it changed and what that did to coverage

[ExtraToast]

Closes #1326.

A reviewer opening a pull request here saw a diff breakdown spliced into the body by a bot, and no coverage at all. The coverage exists — four suites measure it on every run — but it only ever reached an artifact nobody downloads and a floor that fails a job. A floor says whether a number cleared a line. It never says whether this change moved it, and that is the question a reviewer actually has.

nedap/moves-dev-cli (ADRs 0019 and 0020) and JorisJonkers-dev/deploy-kit both answer this with one comment. The second of those names this repository as the prior art for the change-breakdown half; this brings the coverage half back the other way.

What this achieves

One comment per pull request, found again by a marker and updated in place rather than appended to, carrying three things:

• Changes — the existing service · category taxonomy as a Markdown table with bar cells, a total that excludes generated files, and the test-lines-per-production-line ratio.
• Coverage — statements, branches, functions and lines for each of the four suites, each with the main baseline and a delta beneath it.
• Patch coverage — how many of the lines this pull request added are covered, and which are not.

The pull request body belongs to its author again. pr-diff-stats.yml and pr_diff_stats.py are deleted, and the pull_request_target privilege and the edited-retrigger guard that the splice needed go with them.

How

.github/scripts/pr_report.py, stdlib only, in the shape pr_diff_stats.py already established: a script the workflow calls rather than logic in YAML. The taxonomy is not rewritten — .github/diff-stats.yml is renamed to .github/pr-report-rules.yml and read unchanged, along with its glob translator and rules parser, so every classification the repository already made still holds.

The four suites arrive in two formats. JaCoCo's INSTRUCTION, BRANCH, METHOD and LINE fill the statements, branches, functions and lines columns. Instructions are not statements and methods are not functions, but they answer the same question per column, and the alternative is two tables a reviewer cannot read across. Only the report-level counters are read: JaCoCo repeats every counter at package, class and method level, so summing all of them multiplies the same numbers several times over.

Patch coverage reads line-level data from both formats — DA: from lcov, <line ci=…> from JaCoCo — and matches their paths onto the changed files by longest common tail. No report spells a path the repository's way (frontend/src/App.vue from one, net/blueshell/api/… from the other), and a tail matching more than one changed file is dropped rather than guessed.

It runs on workflow_run because validate.yml runs on pull_request, so a fork's run holds a read-only token and cannot comment — and forks and bots are most of this repository's volume. Nothing in it checks out or executes head-ref code: it reads artifacts and the pulls API, both data. The checkout is pinned to the default branch with no ref: and carries a comment saying not to add one.

ADR 0020 caches a baseline from main runs, and validate.yml has no push: main trigger to take one from. A merge-queue entry is the merged result about to become main, so that is what gets cached, and only from a run that went green. It is written from this workflow rather than from Validate on purpose: a workflow_run job runs on the default branch, so its cache is visible to every pull request. One written from a gh-readonly-queue/** ref would not be.

Not in scope

No Python test harness. pr_diff_stats.py had none and inventing one inside a workflow that holds a write token is a larger decision than this; the script was exercised against real data instead, below.

Open pull requests keep the old spliced block in their bodies until they are next edited. Nothing removes it and it does no harm.

Worth a reviewer's attention

• This pull request cannot show its own comment. A workflow_run workflow only ever runs from the version on the default branch, so GitHub will not run pr-report.yml from this branch. The first real comment appears on whatever pull request opens after this merges. That is why the verification below is local rather than a green check.
• The job gates nothing. The Gradle floors and the vitest thresholds stay the enforcement, and every artifact is optional — an absence is named in the comment (No coverage reached this report for **api integration**) rather than rendered as zero. That distinction is not academic: an empty coverage set rendering as 0.00% is exactly the trap that hid a broken JaCoCo fan-in on ci(api): the integration suite runs in six shards, and the floors read all six #1322 earlier today.
• pr-report-meta is a new one-line artifact from the changes job carrying the pull request number, because workflow_run.pull_requests is empty for fork pull requests.
• The comment is capped at 65000 characters; over that the uncovered-line list is dropped first and a note says so.

Verification

The script was run against the real artifacts from runs 35339810953 and 35337672075, and against real pull requests:

case	result
#1322, CI-only change	renders; no measured lines, patch section says so
#1279, api production and tests	JaCoCo paths resolve; 6 of 6 added lines covered; ratio 2.18
#1283, frontend	lcov paths resolve; 19 of 28 covered; uncovered lines listed per file
a perturbed baseline	deltas +1.50 / -2.25; an exact zero renders 0.00 without a sign, per ADR 0020
a suite absent from the baseline	row renders with em dashes and no delta row
missing artifacts, and a corrupt XML	both reported as notes, neither as zero

The validate.yml half is proven in CI on this branch: pr-report-meta is produced and its contents round-trip to 1327.

The api unit row showing 40.49% for both statements and branches is not a bug. INSTRUCTION is 40.4867% and BRANCH is 40.4948%, and both round to the same two decimals.

[ExtraToast]

One limitation worth stating plainly before review: a workflow_run workflow only ever runs from the version on the default branch. GitHub will not run pr-report.yml from this branch, so this pull request cannot demonstrate its own comment. The first real comment appears on whatever pull request opens after this merges.

That is why the verification above is local rather than a green check: the script was run against the actual artifacts from runs 35339810953 and 35337672075 and against four real pull requests, including the degraded paths. What CI on this branch does verify is the validate.yml half — the pr-report-meta artifact appearing on the run.

If you would rather see it working before it lands, the alternative is to merge it and let the next pull request be the proof, with a quick revert if it misbehaves. It gates nothing, so a failure is a missing comment rather than a blocked merge.

[ExtraToast]

Changes

Bucket	Files	Added	Removed
ci · build & config	6	███████ 796	░░░ 384
total	6	+796	−384

No production or test lines added.

Coverage

	Statements	Branches	Functions	Lines
api unit	40.49%	40.52%	32.79%	42.11%
main	—	—	—	—
api integration	68.44%	51.85%	68.34%	76.57%
main	—	—	—	—
frontend unit	67.64%	58.80%	60.46%	68.82%
main	—	—	—	—
frontend e2e	60.60%	37.81%	58.04%	64.12%
main	—	—	—	—

No baseline is cached from main yet, so there is nothing to compare against.

Patch coverage: this pull request changes no line that coverage measures.