Skip to content

Add ACA qualification leak sweep - #198

Draft
Laveesh Rohra (larohra) wants to merge 3 commits into
larohra/aca-qualification-cifrom
larohra/aca-qualification-sweep
Draft

Add ACA qualification leak sweep#198
Laveesh Rohra (larohra) wants to merge 3 commits into
larohra/aca-qualification-cifrom
larohra/aca-qualification-sweep

Conversation

@larohra

Copy link
Copy Markdown
Contributor

Stack

Stack layer 5/5, targeting parent draft PR #197 (larohra/aca-qualification-ci). This replaces the leak-sweep and cleanup-observability portion of draft PR #169 without modifying or closing #169 or any prior replacement-layer PR.

Change

  • Starts an independent, nonblocking AcaSweep stage alongside Build; AcaQualification waits for both while gating only on the Build result, so sweep infrastructure failure cannot suppress qualification.
  • Lists the configured CI-dedicated Sandbox Group in the authored region and deletes only resources proven strictly older than six hours.
  • Requires the exact exclusive-ci-qualification acknowledgment before unfiltered inspection/deletion. This asserts an external infrastructure prerequisite; the ACA data-plane API cannot prove group ownership or exclusivity.
  • Emits durable Azure DevOps warnings for inspection failures, delete failures, and every unknown-age resource, with redacted errors or hashed resource references. Summaries include incomplete and delete_failures and cannot present failed inspection as a clean group.
  • Keeps current-run cleanup assertions in the qualification suites and uses the next pre-run sweep as the accumulated-leftover signal. No destructive post-run cleanup, report-only final audit, or rollback is added.

Validation

  • 79 passedtests/test_aca_qualification_pipeline.py
  • ruff check src tests
  • ruff check eng/scripts/aca_qualification_pipeline.py
  • mypy src — 98 source files
  • git diff --check
  • ADO preview compile: pipeline 1777 expanded AcaSweep, Build+AcaSweep dependency, Build-only qualification gate, Python 3.13/3.14 maxParallel: 2, one explicit acknowledgment, and no ACA variable-group reference.
  • ADO preview compile: pipeline 1733 remained the unaffected official-build graph with no ACA stages.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Snapshot leaks remain unhandled, and concurrent sandbox deletion can be incorrectly reported as a failure.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds a nonblocking pre-run leak sweep to ACA qualification CI.

Changes:

  • Runs AcaSweep alongside Build.
  • Deletes resources older than six hours with durable warnings.
  • Adds tests and operational documentation.
File summaries
File Description
tests/test_aca_qualification_pipeline.py Tests sweep behavior and pipeline wiring.
eng/templates/official/jobs/aca-sweep.yml Defines the sweep job.
eng/scripts/README.md Documents the sweep command.
eng/scripts/aca_qualification_pipeline.py Implements cleanup and reporting. Requires handling already-absent sandboxes as successful and sweeping orphaned snapshots.
eng/ci/e2e-tests.yml Wires sweep and qualification dependencies.
eng/ci/docs/aca-qualification.md Documents operations and troubleshooting.
docs/frds/0008-aca-sandbox-session-runtime.md Records the finalized design decision.
Review details
  • Files reviewed: 7/7 changed files
  • Comments generated: 2
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread eng/scripts/aca_qualification_pipeline.py
Comment thread eng/scripts/aca_qualification_pipeline.py
@larohra
Laveesh Rohra (larohra) force-pushed the larohra/aca-qualification-sweep branch from bb296b6 to bf5313e Compare September 4, 2026 19:06
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: dc895b41-586d-4824-97b1-6761d99a35de
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: dc895b41-586d-4824-97b1-6761d99a35de
@larohra
Laveesh Rohra (larohra) force-pushed the larohra/aca-qualification-sweep branch from df6ec3b to 70113a9 Compare September 8, 2026 23:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants