Add deployed ACA qualification assets

[Laveesh Rohra (larohra)]

Stack layer 3/5

Parent: #195 (larohra/aca-runtime-reliability)

Adds the isolated, manually runnable deployed ACA qualification fixture, live client/suites, pinned Python 3.13/3.14 fixture constraints, and package/ZIP deployment helpers. Qualification covers fresh-session acceptance, authenticated public turn, lifecycle reuse/reclaim, active backing loss, and N=5 concurrent admission/idempotency/events/results/cleanup.

Provenance is intentionally lightweight and in-package: build ID, commit SHA, and live Python minor are checked only after cold-start timing completes. Wheel digest, installed package version, deploy-input manifest, deployment-storage chain, and rollback are out of scope.

No ACA qualification/sweep stages, CI trigger/gating/service-connection policy, strategy matrix, variable-group changes, or group-wide sweep are included. Those remain in layers 4/5.

Validation:

• uv run python -m ruff check src tests
• uv run python -m mypy src
• 298 targeted qualification tests passed
• 6 live qualification modules collected
• git diff --check

Replaces the fixture/tooling portion of draft #169 without modifying or closing it.

[hallvictoria]

The deployed helper accepts a pathless Function App origin in tests/live/aca_deployed_agent_support.py, but the URL builders append /agents/... and the management paths directly to base_url. Since the fixture retains the default Functions /api route prefix, a pathless value produces /agents/... rather than /api/agents/.... Could we either normalize pathless origins by adding /api, or require the configured URL to include /api? Please also cover both accepted forms in the helper tests.

[Copilot]

🟡 Changes recommended

Provenance failures do not stop later suites, and validation permits unsupported qualification load shapes.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds manually deployable ACA qualification assets for Python 3.13/3.14 without adding CI pipeline wiring.

Changes:

• Adds the deployable fixture, pinned dependencies, packaging/deployment helpers, and build provenance checks.
• Expands live qualification for cold start, lifecycle, backing loss, load, retries, events, and cleanup.
• Documents the layer-3 qualification contract in FRD 0008.

File summaries

File	Description
tests/test_aca_smoke_support.py	Tests fixture discovery with region forwarding.
tests/test_aca_qualification_pipeline.py	Covers qualification tooling and fixture contracts.
tests/test_aca_deployed_load.py	Extends load and retry tests.
tests/test_aca_deployed_load_support.py	Tests visibility and throttling helpers.
tests/test_aca_deployed_cold_start.py	Tests post-timing provenance checks.
tests/test_aca_deployed_agent_turn.py	Tests lifecycle retention behavior.
tests/test_aca_deployed_agent_support.py	Covers JSON and SSE retries.
tests/live/test_aca_deployed_loss.py	Refreshes public loss projection.
tests/live/test_aca_deployed_load.py	Adds admission, event, and result resilience.
tests/live/test_aca_deployed_lifecycle.py	Verifies terminal result retention.
tests/live/test_aca_deployed_cold_start.py	Adds deployed-build verification.
tests/live/README.md	Documents manual qualification usage.
tests/live/apps/aca-qualification/tools/qualification_hold.py	Adds the active-run hold tool.
tests/live/apps/aca-qualification/requirements.txt	Defines deploy-time requirements generation.
tests/live/apps/aca-qualification/README.md	Documents the fixture app.
tests/live/apps/aca-qualification/host.json	Configures the Functions host.
tests/live/apps/aca-qualification/function_app.py	Adds app creation and build-info route.
tests/live/apps/aca-qualification/deployed_turn.agent.md	Defines the no-tools turn agent.
tests/live/apps/aca-qualification/deployed_load.agent.md	Defines the load/loss agent.
tests/live/apps/aca-qualification/agents.config.yaml	Configures ACA runtime and authentication.
tests/live/apps/aca-qualification/.gitignore	Excludes generated build metadata.
tests/live/apps/aca-qualification/.funcignore	Excludes local artifacts from deployment.
tests/live/aca_deployed_loss_support.py	Improves loss-state diagnostics.
tests/live/aca_deployed_load_support.py	Adds throttling and visibility metrics.
tests/live/aca_deployed_cold_start_support.py	Adds expected-build and report support.
tests/live/aca_deployed_agent_support.py	Adds bounded HTTP and SSE retries.
eng/scripts/README.md	Documents qualification commands.
eng/scripts/aca_qualification_pipeline.py	Implements assembly, deployment, and provenance tooling.
eng/scripts/aca_deployed_qualification.py	Adds cold start to the ordered suite.
eng/constraints/README.md	Documents fixture dependency pins.
eng/constraints/aca-fixture-requirements.txt	Pins the deployable dependency closure.
docs/frds/0008-aca-sandbox-session-runtime.md	Records layer-3 design decisions.

Review details

Suppressed comments (1)

eng/scripts/aca_deployed_qualification.py:182

• Running all modules in one pytest invocation does not make provenance a gate: pytest continues to the turn, lifecycle, loss, and load modules after the cold-start test reports a marker mismatch. That can emit qualification evidence for a deployment already known to be stale. Run the cold-start/attestation module first and invoke the remaining modules only when it succeeds (or otherwise enable fail-fast behavior).

    return _run_pytest(
        (
            "tests/live/test_aca_deployed_cold_start.py",
            "tests/live/test_aca_deployed_agent_turn.py",
            "tests/live/test_aca_deployed_lifecycle.py",
            "tests/live/test_aca_deployed_loss.py",
            "tests/live/test_aca_deployed_load.py",
        ),

• Files reviewed: 32/32 changed files
• Comments generated: 3
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Laveesh Rohra (larohra)]

Addressed the Function base URL feedback in 09b4b86: pathless HTTPS site origins now normalize to the fixture's default /api route root, while explicit /api remains accepted. Tests cover both forms and the manual example now includes /api. The same update also makes cold-start/provenance a separate hard gate, so later qualification suites do not run after a provenance failure.