Skip to content

👷 ci: hold native coverage to LLVM's total - #2162

Open
gaborbernat wants to merge 1 commit into
mainfrom
fix/enforce-total-line-coverage-1745
Open

👷 ci: hold native coverage to LLVM's total#2162
gaborbernat wants to merge 1 commit into
mainfrom
fix/enforce-total-line-coverage-1745

Conversation

@gaborbernat

Copy link
Copy Markdown
Member

The native coverage gate reads a narrower thing than its name suggests. --fail-uncovered-lines 0 counts a line that no function reached, and cargo-llvm-cov drops a line from that list as soon as any function or macro expansion on it ran, which cargo-llvm-cov#404 describes. A physical line shared by a covered call and an uncovered closure leaves the list while LLVM still counts it missed.

A ten-line crate shows the size of that gap. One function, v.map_or(0, |x| x * 2), tested only with None, puts the file at six covered lines of seven. cargo-llvm-cov reports 85.71%, prints no Uncovered Lines section at all, and the gate exits zero. 📉 Adding --fail-under-lines 100 to the same profile exits one. On this workspace the two measures agreed everywhere I sampled, so the divergence is rare rather than absent, which is what makes it worth a gate rather than a review habit.

The gate also now reports to the terminal instead of into the lcov file. In lcov mode a failure prints nothing whatsoever, so a red gate would hand back an exit code and no file to look at. Running the report twice over one profile costs a second report step and none of the measurement, which is where the sixteen minutes go. A failing file then shows in the per-file table, and a file carrying missed lines while listing none under Uncovered Lines is the shared-line kind. That comparison is the only way to locate them, since LLVM does not expose those source lines and llvm-project#126307 tracks the omission.

The contract that already checks this recipe's target directory now also holds both properties. Removing the flag fails it, and folding the gate back into the lcov write fails it, so the gate cannot quietly return to reporting nothing.

Two things for a reviewer. This PR's own coverage job is the measurement that matters: it runs just coverage-native on x86_64-linux, which is the only platform this gate covers, and it will say whether the workspace already sits at LLVM's 100% or whether files remain. If it comes back red, the per-file table now names them and the remediation belongs in its own change rather than buried here. I did not run the full native profile locally, since it is the machine-wide heavy command the brief reserves.

Closes #1745

`--fail-uncovered-lines 0` reads a narrower thing than its name suggests. It
counts a line no function reached, and cargo-llvm-cov drops a line from that
list once any function or macro expansion on it ran. A physical line shared by a
covered call and an uncovered closure therefore leaves the list while LLVM still
counts it missed, so the gate passes on a file the tool reports at 85.71% line
coverage. A ten-line crate reproduces it: one `map_or` whose closure never runs
puts a file at six lines of seven and the gate returns zero.

Adding LLVM's own total closes that, since the total counts what the list
discards. The two together are the bound the name always implied.

The gate now reports to the terminal rather than into the lcov file. In lcov
mode the failure prints nothing at all, so a red gate would hand back an exit
code and no file to look at. Reporting twice over one profile costs a second run
of the report step and nothing of the measurement, which is where the time goes.
A failing file is then visible in the per-file table, and one carrying missed
lines with nothing listed under Uncovered Lines is the shared-line kind, which is
the only way to find them: LLVM does not expose those source lines, which
llvm-project#126307 tracks.

The contract that already checks this recipe's target directory now keeps both
properties, so neither the flag nor the terminal report can be dropped without
failing.
@gaborbernat gaborbernat added the enhancement New feature or request label Sep 2, 2026
@codspeed-hq

codspeed-hq Bot commented Sep 2, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 29 untouched benchmarks
⏩ 133 skipped benchmarks1


Comparing fix/enforce-total-line-coverage-1745 (227835c) with main (892226e)

Open in CodSpeed

Footnotes

  1. 133 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enforce LLVM total line coverage

1 participant