Repository navigation
ci: keep live cloud tests off unrelated changes and serialize the Azure probe - #971
Merged
Merged
Conversation
The Azure Pipelines probe queues a run on a pool that has a single agent. A workflow-level concurrency group cancels that probe together with the rest of a superseded run, so the queued or running Azure run keeps the agent and the probe's cancel/delete cleanup never gets to run. Give every job its own ref-scoped group so a newer run still supersedes an older run's work, and serialize the probe in one group shared by pull requests, main and reruns. The probe is never cancelled in progress, and queue: max keeps pending probes instead of replacing them, so a newer run cannot drop live coverage that is already waiting for the agent.
Every push and pull request queued all thirteen live workflows, including changes that cannot affect a provider. Gate the live cloud workflows on their service, the shared workspace crates, the workspace manifest and toolchain, and the workflow and probe scripts the service uses. Offline-only workflows keep their current triggers, and the workspace build, lint and unit tests in ci.yml still cover every crate, so coverage is only dropped where the provider code cannot have changed.
PsiACE
approved these changes
Nov 19, 2026
…-live-path-filters
The probe group keeps pending entries so a newer run on another ref cannot drop live coverage, but that also lets an older entry take the single pool agent for a revision a newer run already replaced. Before any 1Password, OIDC, blob or Azure work, list this workflow's runs for the branch and skip the probe only when a newer run for the same event is still in flight. A change the path filters drop creates no run, and an unrelated ref or an unclear API answer keeps the probe, so the check cannot remove the only live validation. The skip costs one read-only API call and nothing else; the summary reports it as skipped instead of passed.
The pipeline read the mutable ref it was triggered from. A push that advanced the ref made the checkout abort on the SHA mismatch, and a merge ref replaced by the next push could only resolve newer code, so a queued probe could neither validate its own revision nor prove it had not moved. Fetch the commit the GitHub workflow ran and require the resolved commit to match it before checking out detached. The ref stays only as a fetch fallback and still has to resolve to that commit, and the ref/SHA shape checks are unchanged. The tests drive the step against a fixture remote where the branch and a pull request merge ref advance after queueing.
…-live-path-filters # Conflicts: # .github/azure-live-tests/test_scripts.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Live workflows ran on every push and pull request, and the Azure Pipelines
probe competes for a single-agent pool with main and the other pull requests.
A workflow-level cancellation also killed the probe after it had queued its
Azure run, so the agent stayed busy and the probe cleanup could not run.
A newer run now cancels its own counterpart jobs instead of the whole
workflow, and the probe runs in one group shared by pull requests, main and
reruns that never cancels an entry in progress.
queue: maxkeeps pendingentries:
cancel-in-progress: falsealone still cancels the pending entry of agroup, and dropping a pending probe would drop live coverage that main or a
pull request is waiting for.
Preserving pending entries makes a stale probe possible, so before any
1Password, OIDC, blob or Azure work the job lists this workflow's runs for its
branch and skips only when a newer run for the same event is still in flight.
A change that the path filters drop creates no run, and an unrelated ref or an
unclear API answer keeps the probe, so the check cannot remove the only live
validation. The summary reports a skipped probe as skipped, not as passed.
This adds one token scope,
actions: readon the probe job.A queued probe now also checks out the commit that triggered it rather than
the ref it was triggered from. The pipeline fetched the mutable ref, so a
successor push made it abort on the SHA mismatch, and a replaced merge ref
could only resolve newer code. The fetch is by commit, with the ref kept only
as a fallback and still required to resolve to the triggering commit; the
ref/SHA shape checks are unchanged.
The live workflows now trigger on their service, the shared workspace crates,
the workspace manifest, lockfile and toolchain, and the probe scripts they
use. Offline workflows keep their triggers and
ci.ymlstill builds, lintsand tests the whole workspace for every change, so this only removes live
coverage that cannot have changed. Trust gating, fork skips, token scopes
outside the probe job, secrets and the other summaries are unchanged.
Verified locally: 38 fixture tests,
shellcheck,actionlint, the 28-casepath matrix, and targeted mutations (dropping the in-flight rule, an unguarded
remote step, a ref-only fetch, a missing SHA check) each fail them.
actionlint's vendored schema predatesqueue, so that one key is ignoredthere. Commit-by-SHA fetching was confirmed read-only against GitHub for an
advanced branch and for displaced and closed pull request merge commits before
the pipeline relied on it.