ci(coverage): build coverage on push to master so Codecov compares the right base#854
Conversation
The "Tests and Codecov" workflow ran only on `pull_request` to master and `workflow_dispatch`, so coverage was uploaded to Codecov solely for PR-head commits. Master HEAD (the post-merge state of the default branch) was never built directly, so Codecov never received a coverage report for it. Because PRs land on master via merge commits, those PR-head commits sit on the second-parent side of history, off master's first-parent line. When a new PR opens, its base is the current master tip -- a first-parent merge commit that has no coverage report. Codecov then walks back the ancestry to the nearest commit that does have a report and uses that as the comparison base. In practice this resolves to an arbitrary old PR-head commit far behind master (e.g. ~200 commits), so the project coverage diff reflects hundreds of commits of drift rather than the PR under review, making the comparison useless. Add a `push: branches: [master]` trigger so the full build runs on master after every merge and uploads coverage for the actual master commit. The PR-only "Merge with latest target branch" step is already guarded by `github.event_name == 'pull_request'`, so push builds test master directly. With a coverage report attached to each master first-parent commit, Codecov can use the true PR base and report a meaningful diff. Co-authored-by: Claude <noreply@anthropic.com>
The job set `defaults.run.shell: bash -l {0}` to get a login shell for
conda activation. Overriding the shell discards GitHub Actions' default
`bash --noprofile --norc -eo pipefail {0}`, so multi-line `run:` blocks
no longer aborted on the first failing command and pipe failures were
masked by the exit status of the last pipe stage. A dependency install
or build command could fail mid-step while the step still reported
success.
Switch to `bash -leo pipefail {0}`: keep the login shell (-l) for conda,
add -e so a step fails on the first non-zero command, and -o pipefail so
failures propagate through pipes. This matches the fail-fast behavior
conda-incubator/setup-miniconda documents (`bash -el {0}`) while also
restoring pipefail.
Co-authored-by: Claude <noreply@anthropic.com>
|
Note Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported. |
Codecov Report✅ All modified and coverable lines are covered by tests.
Additional details and impacted files@@ Coverage Diff @@
## master #854 +/- ##
==========================================
- Coverage 37.16% 27.96% -9.20%
==========================================
Files 213 241 +28
Lines 30671 35495 +4824
Branches 12486 14781 +2295
==========================================
- Hits 11399 9927 -1472
- Misses 17438 18586 +1148
- Partials 1834 6982 +5148
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
| # flags: -e aborts the step on the first failing command and | ||
| # -o pipefail propagates failures through pipes. Overriding the | ||
| # shell drops GitHub's default `-eo pipefail`, so set it back here. | ||
| shell: bash -leo pipefail {0} |
There was a problem hiding this comment.
Shell-flag inconsistency across workflows:
conda_build.yml / conda_build_release.yml → bash -el {0} (no pipefail)
coverity-scan.yml → step-level bash -l {0} in 4 places (neither errexit nor pipefail)
The other workflows have the same latent issue this PR fixes — fail-fast not enabled.
There was a problem hiding this comment.
-el is fine if there is no pipe in the script. We should check if -l is needed for each workflow. Just use the default value for the workflow that doesn't need -l.
|
The build on master failed, however: |
Problem
The
Tests and Codecovworkflow (.github/workflows/ci-cmake_tests.yml) ran only onpull_request(tomaster) andworkflow_dispatch. Coverage was therefore uploaded to Codecov only for PR-head commits —masterHEAD (the post-merge state of the default branch) was never built directly, so Codecov never received a coverage report for it.Because PRs land on
mastervia merge commits, those PR-head commits sit on the second-parent side of history, offmaster's first-parent line. When a new PR opens, its base is the currentmastertip — a first-parent merge commit with no coverage report. Codecov then walks back the ancestry to the nearest commit that does have a report and uses that as the comparison base, which resolves to an arbitrary old PR-head commit far behindmaster.Observed on #849: Codecov picked base
98e5842, which is 194 commits behindmasterand not on the first-parent line. Its own comment said "Report is 211 commits behind head." The "project" diff then showedFiles +29, Lines +4844, Coverage 37.16% → 28.15% (−9.01%)— hundreds of commits of drift, not the 3-file / +172−17 change in the PR. The comparison is effectively useless.Changes
Add a
push: branches: [master]trigger so the full build runs onmasterafter every merge and uploads a coverage report for the actualmastercommit. The PR-only "Merge with latest target branch" step is already guarded bygithub.event_name == 'pull_request', so push builds testmasterdirectly. With a report attached to eachmasterfirst-parent commit, Codecov can use the true PR base and produce a meaningful diff.Harden the workflow shell:
defaults.run.shellwasbash -l {0}. Overriding the shell discards GitHub Actions' default-eo pipefail, so multi-linerun:blocks no longer aborted on the first failing command and pipe failures were masked. Switch tobash -leo pipefail {0}— keep the login shell (-l) for conda activation, add-e(fail on first non-zero command) and-o pipefail(propagate failures through pipes). This matches the fail-fast behaviorconda-incubator/setup-minicondadocuments (bash -el {0}) while also restoring pipefail.Note on rollout
This fix takes effect going forward: Codecov gets a usable base for a PR once the
mastercommit it is based on has its own coverage report — i.e. after this lands and the next merge fires the new push build. The first PR or two after merge may still show drift untilmasteraccumulates push-built reports.Test plan
.github/workflows/ci-cmake_tests.ymlparses as valid YAML; triggers arepush,pull_request,workflow_dispatch; job shell isbash -leo pipefail {0}.This is a CI-config-only change; no library code is touched, so the C++/Python test suites are unaffected.
Generated by Claude Code