-
Notifications
You must be signed in to change notification settings - Fork 19
ci: parallelise the workflow and stop discarding the build cache #202
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
b917caf
14e2f95
766afc5
9bc54ab
84a3e7c
d990da1
79e0a9a
a894a25
3ff4bce
949d5c0
1a33158
43f19ca
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,21 +2,46 @@ name: CI | |
|
|
||
| on: | ||
| push: | ||
| # Pull requests are covered by the `pull_request` trigger below. Building | ||
| # every branch push as well duplicated the entire suite for each PR commit | ||
| # (two identical ~23 minute runs), so pushes only build the mainline, which | ||
| # is also what keeps a shared, warm cache for PR branches to restore from. | ||
| branches: [main] | ||
| pull_request: | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }} | ||
| # Keyed by pull-request *identity*, not by branch name: two PRs from | ||
| # different forks can share a head branch name (`main`, `feature`), and | ||
| # keying on `github.head_ref` would put them in one group where | ||
| # `cancel-in-progress` lets either one cancel the other's required checks. | ||
| # `github.event.pull_request.number` is unset for pushes, which fall back to | ||
| # the ref. | ||
| group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| env: | ||
| CARGO_TERM_COLOR: always | ||
| # Incremental compilation only pays off across edits on one machine; in CI it | ||
| # costs codegen time and inflates the cached target directory. | ||
| CARGO_INCREMENTAL: 0 | ||
| # Debug info dominates link time and target-directory size. Dropping it | ||
| # roughly halves both, and nothing in CI reads line numbers out of a | ||
| # backtrace. | ||
| CARGO_PROFILE_DEV_DEBUG: 0 | ||
| CARGO_PROFILE_TEST_DEBUG: 0 | ||
| RUST_BACKTRACE: 1 | ||
|
|
||
| jobs: | ||
| rust-sdk: | ||
| name: Rust SDK | ||
| # Static analysis only: `cargo clippy` and `cargo doc` stop at metadata, so | ||
| # this job never links a binary and finishes long before the test matrix. | ||
| lint: | ||
| name: Lint | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v7 | ||
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| # This job executes repo code (cargo build/test); don't persist the | ||
| # token in git config. | ||
|
|
@@ -27,11 +52,12 @@ jobs: | |
| # vendor/tinytools/crates/tinytools/Cargo.toml". | ||
| submodules: recursive | ||
|
|
||
| - uses: dtolnay/rust-toolchain@stable | ||
| - uses: dtolnay/rust-toolchain@6bed0761d98439e5a578e2877258200ad565ba87 # stable | ||
| with: | ||
| toolchain: stable | ||
| components: rustfmt, clippy | ||
|
|
||
| - uses: Swatinem/rust-cache@v2 | ||
| - uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2 | ||
| with: | ||
| workspaces: . | ||
|
|
||
|
|
@@ -44,57 +70,108 @@ jobs: | |
| - name: Clippy all features | ||
| run: cargo clippy --workspace --all-targets --all-features -- -D warnings | ||
|
|
||
| - name: Build | ||
| run: cargo build --workspace --all-targets | ||
| # `--all-features` so links into feature-gated items resolve; without | ||
| # it rustdoc cannot see the very modules the docs point at. | ||
| - name: Doc lints | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do not make known rustdoc warnings gate CI The workspace has approximately 160 existing broken intra-doc link warnings (as noted in earlier review context). This change removes the [RULE] known-issues-gate-ci · |
||
| env: | ||
| RUSTDOCFLAGS: -D rustdoc::broken_intra_doc_links | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do not make broken intra-doc links block merges without fixing them first This change removes the previous non-blocking behavior for the known broken intra-doc-link set and places the check in the required lint job. Existing broken links therefore prevent the aggregate [RULE] broken-intra-doc-links-gate · |
||
| run: cargo doc --workspace --no-deps --all-features | ||
|
senamakel marked this conversation as resolved.
|
||
|
|
||
| # Scoped to `crates`, this repo's own workspace: an unscoped run also | ||
| # walks `vendor/`, where it reports findings against the vendored | ||
| # submodules that have to be fixed in their own repositories. | ||
| - name: Unused dependencies | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Keep known unused-dependency findings non-blocking The old step used [RULE] nonblocking-known-ci-failure · There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Keep known unused-dependency findings non-blocking The previous step was explicitly non-blocking because the workspace has existing unused-dependency findings. This revision removes Additional
|
||
| uses: bnjbvr/cargo-machete@ac30a525c0a8d163a92d727b3ff079ee3f6ecb08 # v0.9.2 | ||
| with: | ||
| args: crates | ||
|
|
||
| # Every feature selection links a distinct workspace graph, so they cannot | ||
| # share a target directory. Previously they ran back to back with a | ||
| # `cargo clean` between each, which serialised five cold builds into one job | ||
| # and left the cache holding whichever graph happened to be built last. As | ||
| # separate jobs they build concurrently and each keeps its own warm cache. | ||
| test: | ||
| name: Test (${{ matrix.name }}) | ||
| runs-on: ubuntu-latest | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| include: | ||
| - name: default | ||
| features: "" | ||
| - name: all-features | ||
| features: "--all-features" | ||
| - name: sqlite | ||
| features: "--no-default-features --features sqlite" | ||
| - name: tools | ||
| features: "--no-default-features --features tools" | ||
| - name: multimodal | ||
| features: "--no-default-features --features multimodal" | ||
| steps: | ||
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| persist-credentials: false | ||
| submodules: recursive | ||
|
|
||
| - uses: dtolnay/rust-toolchain@6bed0761d98439e5a578e2877258200ad565ba87 # stable | ||
| with: | ||
| toolchain: stable | ||
|
|
||
| - name: Build all features | ||
| run: cargo build --workspace --all-targets --all-features | ||
| - uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2 | ||
| with: | ||
| workspaces: . | ||
| # Without a distinct key every matrix leg would race to overwrite one | ||
| # cache entry with its own feature graph. | ||
| key: ${{ matrix.name }} | ||
|
|
||
| - name: Build | ||
| run: cargo build --workspace --all-targets ${{ matrix.features }} | ||
|
|
||
| - name: Test | ||
| run: cargo test --workspace | ||
| run: cargo test --workspace ${{ matrix.features }} | ||
|
|
||
| coverage: | ||
| name: Coverage | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| persist-credentials: false | ||
| submodules: recursive | ||
|
|
||
| - name: Test all features | ||
| run: cargo test --workspace --all-features | ||
| - uses: dtolnay/rust-toolchain@6bed0761d98439e5a578e2877258200ad565ba87 # stable | ||
| with: | ||
| toolchain: stable | ||
| components: llvm-tools-preview | ||
|
|
||
| - name: Test optional features independently | ||
| run: | | ||
| # Each feature set links a distinct workspace graph. Clearing the | ||
| # preceding graph keeps the runner's linker working set bounded and | ||
| # makes these genuinely independent builds. | ||
| cargo clean | ||
| cargo test --workspace --no-default-features --features sqlite | ||
| cargo clean | ||
| cargo test --workspace --no-default-features --features tools | ||
| cargo clean | ||
| cargo test --workspace --no-default-features --features multimodal | ||
|
|
||
| - name: Free build artifacts before coverage | ||
| run: cargo clean | ||
|
|
||
| - name: Coverage gate | ||
| uses: taiki-e/install-action@cargo-llvm-cov | ||
| - uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2 | ||
| with: | ||
| workspaces: . | ||
| # Instrumented objects are not interchangeable with the test matrix's, | ||
| # so coverage keeps its own cache rather than invalidating theirs. | ||
| key: coverage | ||
|
|
||
| - uses: taiki-e/install-action@7623a79cdfecb99d681017af368ca353d9f49bb5 # v2.87.19 | ||
| with: | ||
| tool: cargo-llvm-cov | ||
|
|
||
| - name: Verify line coverage | ||
| env: | ||
| CARGO_PROFILE_DEV_DEBUG: 0 | ||
| CARGO_PROFILE_TEST_DEBUG: 0 | ||
| run: >- | ||
| cargo llvm-cov --all-features --workspace | ||
| --ignore-filename-regex '(^|/)(tests?|examples)/|/test(_.*)?\.rs$' | ||
| --fail-under-lines 80 | ||
|
|
||
| # TODO: drop `continue-on-error` once the ~160 existing broken | ||
| # intra-doc link warnings across the workspace are fixed and this can | ||
| # block merges instead of only reporting. | ||
| - name: Doc lints | ||
| env: | ||
| RUSTDOCFLAGS: -D rustdoc::broken_intra_doc_links | ||
| run: cargo doc --workspace --no-deps | ||
| continue-on-error: true | ||
|
|
||
| # TODO: drop `continue-on-error` once the existing unused-dependency | ||
| # findings across the workspace are cleaned up and this can block | ||
| # merges instead of only reporting. | ||
| - name: Unused dependencies | ||
| uses: bnjbvr/cargo-machete@main | ||
| continue-on-error: true | ||
| # Single required status check: branch protection can depend on this one job | ||
| # instead of being updated every time the matrix gains or loses a leg. | ||
| ci: | ||
| name: CI | ||
| runs-on: ubuntu-latest | ||
| if: always() | ||
| needs: [lint, test, coverage] | ||
| steps: | ||
| - name: Check results | ||
| run: | | ||
| echo "lint: ${{ needs.lint.result }}" | ||
| echo "test: ${{ needs.test.result }}" | ||
| echo "coverage: ${{ needs.coverage.result }}" | ||
| [ "${{ contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled') }}" = "false" ] | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do not make known rustdoc warnings gate CI
The previous workflow explicitly marked this check as non-blocking because the workspace has roughly 160 existing broken intra-doc links. Moving it into the new lint job without
continue-on-errormakes those known warnings fail the required CI status and blocks every merge until unrelated documentation is fixed. Restore non-blocking behavior or fix all existing warnings before enabling the gate.Additional
testsobservationDo not make broken intra-doc links block merges without fixing them first
[RULE] ci-regression
The old CI kept this step with
continue-on-error: truebecause there were ~160 existing broken intra-doc link warnings across the workspace. This revision removes the soft-fail and makes it a hard gate, but only fixes links in the files it touches. The remaining broken links elsewhere will cause thelintjob to fail, blocking all merges until they are all cleaned up. Either fix every broken link across the workspace in this PR, or restorecontinue-on-error: truewith the TODO comment until a dedicated cleanup covers the rest.[RULE] nonblocking-known-ci-failure ·