Skip to content

ci: keep live cloud tests off unrelated changes and serialize the Azure probe - #971

Merged
Xuanwo merged 6 commits into
mainfrom
xuanwo/ci-concurrency-live-path-filters
Nov 19, 2026
Merged

Xuanwo merged 6 commits into
mainfrom
xuanwo/ci-concurrency-live-path-filters

Conversation

@Xuanwo

@Xuanwo Xuanwo commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

Live workflows ran on every push and pull request, and the Azure Pipelines
probe competes for a single-agent pool with main and the other pull requests.
A workflow-level cancellation also killed the probe after it had queued its
Azure run, so the agent stayed busy and the probe cleanup could not run.

A newer run now cancels its own counterpart jobs instead of the whole
workflow, and the probe runs in one group shared by pull requests, main and
reruns that never cancels an entry in progress. queue: max keeps pending
entries: cancel-in-progress: false alone still cancels the pending entry of a
group, and dropping a pending probe would drop live coverage that main or a
pull request is waiting for.

Preserving pending entries makes a stale probe possible, so before any
1Password, OIDC, blob or Azure work the job lists this workflow's runs for its
branch and skips only when a newer run for the same event is still in flight.
A change that the path filters drop creates no run, and an unrelated ref or an
unclear API answer keeps the probe, so the check cannot remove the only live
validation. The summary reports a skipped probe as skipped, not as passed.
This adds one token scope, actions: read on the probe job.

A queued probe now also checks out the commit that triggered it rather than
the ref it was triggered from. The pipeline fetched the mutable ref, so a
successor push made it abort on the SHA mismatch, and a replaced merge ref
could only resolve newer code. The fetch is by commit, with the ref kept only
as a fallback and still required to resolve to the triggering commit; the
ref/SHA shape checks are unchanged.

The live workflows now trigger on their service, the shared workspace crates,
the workspace manifest, lockfile and toolchain, and the probe scripts they
use. Offline workflows keep their triggers and ci.yml still builds, lints
and tests the whole workspace for every change, so this only removes live
coverage that cannot have changed. Trust gating, fork skips, token scopes
outside the probe job, secrets and the other summaries are unchanged.

Verified locally: 38 fixture tests, shellcheck, actionlint, the 28-case
path matrix, and targeted mutations (dropping the in-flight rule, an unguarded
remote step, a ref-only fetch, a missing SHA check) each fail them.
actionlint's vendored schema predates queue, so that one key is ignored
there. Commit-by-SHA fetching was confirmed read-only against GitHub for an
advanced branch and for displaced and closed pull request merge commits before
the pipeline relied on it.

The Azure Pipelines probe queues a run on a pool that has a single agent.
A workflow-level concurrency group cancels that probe together with the
rest of a superseded run, so the queued or running Azure run keeps the
agent and the probe's cancel/delete cleanup never gets to run.

Give every job its own ref-scoped group so a newer run still supersedes an
older run's work, and serialize the probe in one group shared by pull
requests, main and reruns. The probe is never cancelled in progress, and
queue: max keeps pending probes instead of replacing them, so a newer run
cannot drop live coverage that is already waiting for the agent.
Every push and pull request queued all thirteen live workflows, including
changes that cannot affect a provider. Gate the live cloud workflows on
their service, the shared workspace crates, the workspace manifest and
toolchain, and the workflow and probe scripts the service uses.

Offline-only workflows keep their current triggers, and the workspace
build, lint and unit tests in ci.yml still cover every crate, so coverage
is only dropped where the provider code cannot have changed.
The probe group keeps pending entries so a newer run on another ref cannot
drop live coverage, but that also lets an older entry take the single pool
agent for a revision a newer run already replaced.

Before any 1Password, OIDC, blob or Azure work, list this workflow's runs for
the branch and skip the probe only when a newer run for the same event is
still in flight. A change the path filters drop creates no run, and an
unrelated ref or an unclear API answer keeps the probe, so the check cannot
remove the only live validation. The skip costs one read-only API call and
nothing else; the summary reports it as skipped instead of passed.
The pipeline read the mutable ref it was triggered from. A push that
advanced the ref made the checkout abort on the SHA mismatch, and a merge ref
replaced by the next push could only resolve newer code, so a queued probe
could neither validate its own revision nor prove it had not moved.

Fetch the commit the GitHub workflow ran and require the resolved commit to
match it before checking out detached. The ref stays only as a fetch fallback
and still has to resolve to that commit, and the ref/SHA shape checks are
unchanged. The tests drive the step against a fixture remote where the branch
and a pull request merge ref advance after queueing.
…-live-path-filters

# Conflicts:
#	.github/azure-live-tests/test_scripts.py
@Xuanwo
Xuanwo merged commit efd0e60 into main Nov 19, 2026
62 checks passed
@Xuanwo
Xuanwo deleted the xuanwo/ci-concurrency-live-path-filters branch November 19, 2026 21:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants