Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ cargo-gamma-process = { path = "crates/cargo-gamma-process", default-features =
cargo-gamma-rt = { path = "crates/cargo-gamma-rt", default-features = false, version = "0.1.0" }
cargo-gamma-unsafe = { path = "crates/cargo-gamma-unsafe", default-features = false, version = "0.1.0" }
cargo-heather = { path = "crates/cargo-heather", default-features = false, version = "0.2.1" }
cargo-platform = { version = "0.3.3", default-features = false }
cargo_metadata = { version = "0.23.1", default-features = false }
cel-interpreter = { version = "0.10.0", default-features = false }
chrono = { version = "0.4.40", default-features = false }
Expand Down
23 changes: 23 additions & 0 deletions crates/cargo-coverage-gate/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1 +1,24 @@
# Changelog
## [0.4.0] - 2026-08-28

- ✨ Features

- add target-specific policies
- add cargo-aprz and cargo-ensure-no-default-features ([#76](https://github.com/microsoft/ox-tools/pull/76))
- run a command per workspace member with cargo-style selection ([#61](https://github.com/microsoft/ox-tools/pull/61))
- add expect-no-coverable-lines assertion ([#51](https://github.com/microsoft/ox-tools/pull/51))
Comment thread
martin-kolinek marked this conversation as resolved.

- 🐛 Bug Fixes

- preserve cross-package coverage
- resolve effective target policies lazily

- 📚 Documentation

- document configuration capabilities

- ♻️ Code Refactoring

- tighten target policy contracts
- reuse thresholds for target opt-outs

3 changes: 2 additions & 1 deletion crates/cargo-coverage-gate/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
[package]
name = "cargo-coverage-gate"
description = "A cargo subcommand that gates pull requests on per-package line coverage measured by cargo-llvm-cov"
version = "0.3.0"
version = "0.4.0"
readme = "README.md"
Comment thread
martin-kolinek marked this conversation as resolved.
keywords = ["oxidizer", "cargo", "subcommand", "coverage", "ci"]
categories = ["command-line-utilities", "development-tools::cargo-plugins"]
Expand All @@ -31,6 +31,7 @@ name = "cargo-coverage-gate"
path = "src/bin/cargo-coverage-gate/main.rs"

[dependencies]
cargo-platform = { workspace = true }
cargo_metadata = { workspace = true }
clap = { workspace = true, features = ["derive", "std", "help", "usage", "error-context"] }
lcov = { workspace = true }
Expand Down
114 changes: 92 additions & 22 deletions crates/cargo-coverage-gate/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,25 @@ coverage lcov tracefile, resolves each package’s threshold from a small
three-layer lookup, and emits a verdict table to stdout (and,
optionally, to a Markdown summary file for CI step summaries).

### Threshold resolution
### Configuration

#### Numeric thresholds

A workspace can define the default line-coverage threshold:

```toml
# Illustrative workspace policy.
[workspace.metadata.coverage-gate]
min-lines-percent = 80
```

Individual packages can override it:

```toml
# Illustrative package policy, intentionally stricter than the workspace.
[package.metadata.coverage-gate]
min-lines-percent = 95
Comment thread
martin-kolinek marked this conversation as resolved.
```

For each workspace member, the effective threshold is the first match
among:
Expand All @@ -34,13 +52,61 @@ among:
1. The built-in default of `100.0` — full coverage required.

Setting `min-lines-percent = 0.0` explicitly opts a package out of
gating (it always passes, regardless of attributed data). A package
that legitimately contains no coverable lines (pure re-exports, type
definitions, a thin binary shim) instead declares
`expect-no-coverable-lines = true`: the gate passes only while that
holds and fails — as a regression — if coverable lines later appear.
The two keys are mutually exclusive, and `expect-no-coverable-lines`
is package-scoped only.
gating: it always passes, regardless of attributed data. Thresholds must
be in the inclusive range `0.0..=100.0`.

#### Packages with no coverable lines

A package that legitimately contains no coverable lines (pure re-exports,
type definitions, or a thin binary shim) can make that invariant explicit:

```toml
[package.metadata.coverage-gate]
expect-no-coverable-lines = true
```

The gate passes only while the package has no attributed coverable lines
and fails as a regression if coverable code later appears. This differs
from `min-lines-percent = 0`, which keeps passing if the package grows
coverable code. The two keys are mutually exclusive, and
`expect-no-coverable-lines` is package-scoped only.

#### Target-specific policies

A package can replace that policy for a Cargo-style target selector:

```toml
[package.metadata.coverage-gate]
min-lines-percent = 100

[package.metadata.coverage-gate.target.'cfg(not(windows))']
min-lines-percent = 0

[package.metadata.coverage-gate.target.x86_64-unknown-linux-gnu]
min-lines-percent = 100
```

A target-specific no-coverable-lines assertion uses the same nesting:

```toml
[package.metadata.coverage-gate.target.thumbv7em-none-eabihf]
expect-no-coverable-lines = true
```

Target keys accept exact Rust target triples or quoted `cfg(...)`
expressions using Cargo’s target grammar. A target table sets either
`min-lines-percent` or `expect-no-coverable-lines = true`, replacing the
package’s base policy for that target. Exact triples take precedence over
matching `cfg(...)` expressions. Multiple matching cfg policies are a
configuration error rather than depending on declaration order.

A zero target-specific threshold disables gating on the matching target,
but does not disable test execution or instrumentation. Those test binaries
remain instrumented because they may contribute coverage to other packages.
If cargo-llvm-cov reports that an instrumented run produced no coverage
data, automation can supply an empty lcov tracefile: zero-threshold and
`expect-no-coverable-lines` packages pass, while positively gated packages
report `NO DATA`.

### Why lcov, not the JSON?

Expand All @@ -60,6 +126,7 @@ Codecov / ADO numbers confusing.

```text
cargo coverage-gate [--lcov <path>]... [-p|--package <spec>]...
[--target <triple>]
[--summary-file <path>] [--quiet]
```

Expand Down Expand Up @@ -91,25 +158,28 @@ let code = report.verdict().as_exit_code();

### Public API

The library exposes [`evaluate`][__link1], which returns an
[`EvaluatedReport`][__link2]. The report can be rendered as plain text via
[`EvaluatedReport::render_text`][__link3] or as GitHub-flavored Markdown
via [`EvaluatedReport::render_markdown`][__link4], and reduced to a single
[`Verdict`][__link5] via [`EvaluatedReport::verdict`][__link6]. The accompanying
binary loads the lcov tracefile from disk and orchestrates rendering
plus the appropriate exit code.
[`evaluate`][__link1] gates one lcov tracefile for the rustc host target, while
[`evaluate_many`][__link2] merges multiple tracefiles at line level.
[`evaluate_many_for_target`][__link3] evaluates an explicit Rust target triple.
Evaluation returns an [`EvaluatedReport`][__link4], which renders as plain
text via [`EvaluatedReport::render_text`][__link5] or GitHub-flavored Markdown via
[`EvaluatedReport::render_markdown`][__link6] and reduces to a [`Verdict`][__link7] via
[`EvaluatedReport::verdict`][__link8]. The accompanying binary loads tracefiles from
disk and orchestrates rendering plus the appropriate exit code.


<hr/>
<sub>
This crate was developed as part of <a href="../..">The Oxidizer Project</a>. Browse this crate's <a href="https://github.com/microsoft/ox-tools/tree/main/crates/cargo-coverage-gate">source code</a>.
</sub>

[__cargo_doc2readme_dependencies_info]: ggGmYW0CYXZlMC43LjJhdIQbFhzZ8rzWNNYbuRaDSGWynFgbH4PMdoT7GNcbVwNPtPjAhvFhYvRhcoQbDzRwf0qddWQbQiTzhu0-bE0bX-rutkvfDuYbITgXvtMXiRVhZIGDc2NhcmdvLWNvdmVyYWdlLWdhdGVlMC4zLjBzY2FyZ29fY292ZXJhZ2VfZ2F0ZQ
[__cargo_doc2readme_dependencies_info]: ggGmYW0CYXZlMC43LjJhdIQbFhzZ8rzWNNYbuRaDSGWynFgbH4PMdoT7GNcbVwNPtPjAhvFhYvRhcoQbF3P3j2nLNgEbcLNXR38OFU8bmdHWagvGfrMbqUEVVagSr0lhZIGDc2NhcmdvLWNvdmVyYWdlLWdhdGVlMC40LjBzY2FyZ29fY292ZXJhZ2VfZ2F0ZQ
[__link0]: https://github.com/taiki-e/cargo-llvm-cov
[__link1]: https://docs.rs/cargo-coverage-gate/0.3.0/cargo_coverage_gate/fn.evaluate.html
[__link2]: https://docs.rs/cargo-coverage-gate/0.3.0/cargo_coverage_gate/struct.EvaluatedReport.html
[__link3]: https://docs.rs/cargo-coverage-gate/0.3.0/cargo_coverage_gate/?search=EvaluatedReport::render_text
[__link4]: https://docs.rs/cargo-coverage-gate/0.3.0/cargo_coverage_gate/?search=EvaluatedReport::render_markdown
[__link5]: https://docs.rs/cargo-coverage-gate/0.3.0/cargo_coverage_gate/enum.Verdict.html
[__link6]: https://docs.rs/cargo-coverage-gate/0.3.0/cargo_coverage_gate/?search=EvaluatedReport::verdict
[__link1]: https://docs.rs/cargo-coverage-gate/0.4.0/cargo_coverage_gate/fn.evaluate.html
[__link2]: https://docs.rs/cargo-coverage-gate/0.4.0/cargo_coverage_gate/fn.evaluate_many.html
[__link3]: https://docs.rs/cargo-coverage-gate/0.4.0/cargo_coverage_gate/fn.evaluate_many_for_target.html
[__link4]: https://docs.rs/cargo-coverage-gate/0.4.0/cargo_coverage_gate/struct.EvaluatedReport.html
[__link5]: https://docs.rs/cargo-coverage-gate/0.4.0/cargo_coverage_gate/?search=EvaluatedReport::render_text
[__link6]: https://docs.rs/cargo-coverage-gate/0.4.0/cargo_coverage_gate/?search=EvaluatedReport::render_markdown
[__link7]: https://docs.rs/cargo-coverage-gate/0.4.0/cargo_coverage_gate/enum.Verdict.html
[__link8]: https://docs.rs/cargo-coverage-gate/0.4.0/cargo_coverage_gate/?search=EvaluatedReport::verdict
96 changes: 84 additions & 12 deletions crates/cargo-coverage-gate/docs/design/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,7 @@ want to reproduce the gate locally.

```text
cargo coverage-gate [--lcov <path>]... [-p <spec>]... [--package <spec>]...
[--target <triple>]
[--summary-file <path>] [--quiet]
```

Expand Down Expand Up @@ -126,6 +127,9 @@ Flags:
test-impact step so that impact-scoped runs only gate the packages
whose tests actually ran. A selector that matches no member is a
configuration error (exit 2).
- `--target <triple>` — evaluate target-specific package policies for the
supplied Rust target. Defaults to the rustc host target when target
policies exist; workspaces without target policies do not invoke rustc.
- `--summary-file <path>` — write a Markdown verdict table to this file.
When unset, the tool honors the environment variables
`GITHUB_STEP_SUMMARY` (GitHub Actions) and
Expand Down Expand Up @@ -161,9 +165,10 @@ min-lines-percent = 75.0
min-lines-percent = 80.0
```

The schema today is one key, `min-lines-percent`, an integer or float percentage
(`0.0`–`100.0` inclusive). Future extensions can add `min-functions`,
`min-regions` symmetrically.
The base schema accepts `min-lines-percent`, an integer or float percentage
(`0.0`–`100.0` inclusive), or the package-only
`expect-no-coverable-lines` assertion described below. Future extensions can
add `min-functions` and `min-regions` symmetrically.

The built-in default of `100.0` means **gating is on by default**: a new
package with no metadata anywhere will only pass if every measured line is
Expand Down Expand Up @@ -223,6 +228,55 @@ Rules:
- A non-boolean value is a configuration error (exit `2`). An explicit
`false` is identical to omitting the key.

#### Target-specific policy

Packages whose implementation exists only on selected compilation targets use
Cargo-style target selectors nested under their package metadata:

```toml
[package.metadata.coverage-gate]
min-lines-percent = 100

[package.metadata.coverage-gate.target.'cfg(not(windows))']
min-lines-percent = 0
```

Selectors use the same grammar as Cargo's target-specific dependency tables:
an exact target triple (`x86_64-pc-windows-msvc`) or a quoted `cfg(...)`
expression (`cfg(windows)`, `cfg(target_os = "linux")`,
`cfg(all(unix, target_arch = "x86_64"))`). Coverage-gate uses the
`cargo-platform` parser and matches `cfg(...)` expressions against
`rustc --print cfg --target <triple>`, so it does not maintain a second target
language. As in Cargo target-specific dependency selection, selectors must
describe target properties. Build-context predicates such as `cfg(feature =
"...")`, `cfg(test)`, `cfg(debug_assertions)`, and `cfg(proc_macro)` are
configuration errors because a standalone target query cannot evaluate them.

`min-lines-percent = 0` disables gating for that package on the matching
target. It does not disable tests or coverage instrumentation. Test binaries
from zero-threshold packages remain part of the instrumented run because they
may contribute coverage to other workspace packages. This is distinct from
`expect-no-coverable-lines = true`, which asserts that the selected package
itself owns no coverable lines.

Target tables replace the base policy with either `min-lines-percent` or
`expect-no-coverable-lines = true`. The two are mutually exclusive. An exact
target that overrides a broader `cfg(...)` opt-out repeats its positive
threshold.

Resolution follows Cargo's precedence:

1. An exact target-triple table wins over every matching `cfg(...)` table.
2. Otherwise one matching `cfg(...)` table supplies the target policy.
3. Multiple matching `cfg(...)` tables are a configuration error rather than
depending on TOML declaration order.
4. With no matching target table, the ordinary package → workspace → built-in
policy applies.

The CLI accepts `--target <triple>`. When omitted, it obtains the rustc host
target from `rustc -vV`. Rust target discovery is lazy: if no package declares
target policies, evaluation does not invoke rustc.

### 5.4 The verdict table

The tool prints a table to stdout (and to the summary file when
Expand Down Expand Up @@ -378,6 +432,19 @@ state and classifies as a pass (`EMPTY` / `➖`), not the no-data
configuration error. Conversely, if such a package *does* have attributed
coverable lines, it fails the gate (exit `1`) rather than passing.

A package whose effective target policy sets `min-lines-percent = 0` passes,
including when it has no attributed data. Its tests remain instrumented so
they can contribute coverage to other packages.

An empty lcov tracefile is valid input. Each in-scope package is classified
from its effective policy: zero-threshold and `expect-no-coverable-lines`
packages pass, while a package with a positive threshold reports `NO DATA`
and makes the result a configuration error. This lets an orchestrator recover
from cargo-llvm-cov's "no coverage data found" export outcome by supplying an
empty tracefile to the gate. It must not remove test binaries from
instrumentation preemptively; if those tests produce coverage for another
package, the normal export succeeds and that coverage remains visible.

### 6.4 Cross-package test attribution

Per-package aggregation groups measurements by **source-file ownership**
Expand Down Expand Up @@ -582,12 +649,17 @@ comparison rounds to the same precision before comparing — see

### 10.2 Security

The tool reads `Cargo.toml` files and a coverage lcov tracefile. It never
writes; the only output channels are stdout and the optional summary
file. No network access, no privileged operations. The only subprocess
invocation is the read-only `cargo metadata` call performed by
`cargo_metadata::MetadataCommand::exec()` during workspace discovery
(used to enumerate workspace members and resolve the workspace root).
The tool reads `Cargo.toml` files and coverage lcov tracefiles. It never writes;
the only output channels are stdout and the optional summary file. It performs
no network access or privileged operations.

Workspace discovery invokes the read-only `cargo metadata` command through
`cargo_metadata::MetadataCommand::exec()` to enumerate workspace members and
resolve the workspace root. When any package declares target-specific policy,
the tool also invokes the executable selected by `RUSTC` (or `rustc` when
unset). It runs `rustc -vV` only when it must discover the host target, then
runs `rustc --print cfg --target <triple>` for both explicit and discovered
targets. Workspaces without target-specific policy do not invoke rustc.

### 10.3 Monorepo / multi-workspace

Expand All @@ -605,9 +677,9 @@ file) and `DA:` (line count) records; everything else is ignored, so
new record types added by future cargo-llvm-cov releases will not
break parsing.

If a tracefile contains no `SF:` sections — empty file, or a corrupted
upload — the tool exits with a configuration error (no files attributed
to any package). Structural parse errors (e.g., malformed `DA:`
If a tracefile contains no `SF:` sections, every in-scope package is evaluated
as having no attributed data. The effective package policies determine the
result as described in §6.3. Structural parse errors (e.g., malformed `DA:`
records) are hard errors with exit code 2.

#### Tooling requirements
Expand Down
6 changes: 6 additions & 0 deletions crates/cargo-coverage-gate/src/bin/cargo-coverage-gate/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,12 @@ pub(crate) struct CoverageGateArgs {
#[arg(long = "package", short = 'p', value_name = "SPEC")]
pub(crate) packages: Vec<String>,

/// Rust target triple whose coverage policy should be evaluated.
///
/// Defaults to the rustc host target.
#[arg(long, value_name = "TRIPLE")]
pub(crate) target: Option<String>,
Comment thread
martin-kolinek marked this conversation as resolved.

/// Write the Markdown verdict table to this file.
///
/// When unset, the tool falls back to `$GITHUB_STEP_SUMMARY` and then
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@ pub(crate) fn run(args: &CoverageGateArgs) -> Result<ExitCode, AppError> {
}
let lcov_refs: Vec<&str> = lcov_texts.iter().map(String::as_str).collect();

let report = cargo_coverage_gate::evaluate_many(&lcov_refs, None, &args.packages).into_app_err("failed to evaluate coverage")?;
let report = cargo_coverage_gate::evaluate_many_for_target(&lcov_refs, None, &args.packages, args.target.as_deref())
.into_app_err("failed to evaluate coverage")?;

write_text_output(&report, args.quiet).into_app_err("failed to write verdict to stdout")?;

Expand Down
Loading
Loading