Repository navigation
feat(scanner): migrate security operations and supply chain rules (AZ-SECOPS-001 to 010, AZ-SC-001 to 008) to the evaluate() contract #375
Description
What problem does this solve?
The security operations rules (AZ-SECOPS-001 to AZ-SECOPS-010: Defender, Sentinel, logging, retention) and supply chain rules (AZ-SC-001 to AZ-SC-008: registries and Azure DevOps service connections) depend on dedicated collectors outside AzureClient. These rules only implement scan(), so the engine reports their coverage as UNKNOWN / LEGACY_RULE_NOT_MIGRATED, and every compliance control mapped to them reads UNKNOWN in the evaluation-derived reports introduced in #310.
Describe the solution
Migrate the rules below to the evaluate() contract defined in scanner/evaluation.py, following the storage reference implementation (STORAGE_REF).
Rules in scope: az_secops_001 … az_secops_010, az_sc_001 … az_sc_008, plus _security_operations_common.py.
Notes: DevOpsClient.get_service_endpoints() already returns None on failure. When Azure DevOps is not configured at all, report NOT_APPLICABLE / INTEGRATION_NOT_CONFIGURED, which is distinct from ERROR / INVENTORY_UNAVAILABLE when it is configured but unreachable. Apply the same rule to scanner/security_operations.py collectors.
Status mapping
Every path that scan() currently handles with a silent continue must become an explicit status:
| Situation | Status | reason_code |
|---|---|---|
Inventory call failed (returned None) |
ERROR |
INVENTORY_UNAVAILABLE |
| Inventory succeeded but is empty | NOT_APPLICABLE |
NO_RESOURCES_FOUND |
| Setting is compliant | PASS |
— |
| Setting is a violation | FAIL + the existing finding dict |
— |
A per-resource detail call returned None, or a required field is missing |
UNKNOWN |
EVIDENCE_UNAVAILABLE / MISSING_PROPERTIES |
Opt-in policy tag not set (policy_required(...) is false) |
NOT_APPLICABLE |
POLICY_NOT_REQUIRED |
| Approved exception tag set | NOT_APPLICABLE |
APPROVED_EXCEPTION |
A failed inventory call must never produce PASS or NOT_APPLICABLE.
Acceptance criteria
- Each rule in scope exposes
evaluate(azure_client, subscription_id) -> List[RuleEvaluation]using the shared helpers inscanner/evaluation.py. -
scan()becomes a thin wrapper returning the findings attached toFAILevaluations. - Detection is unchanged: for the existing test fixtures,
scan()returns exactly the same findings as before the change. - Each
FAIL/PASSevaluation records the inspected value(s) inevidence, for example{"allow_blob_public_access": true}. - New tests cover the PASS, FAIL, UNKNOWN, NOT_APPLICABLE and inventory-failure paths for every rule in scope.
- The rules are picked up automatically by
tests/test_rule_evaluation_contract.pyand it passes. -
pytest,ruff check .andruff format --check .pass. -
CHANGELOG.mdupdated underUnreleased.
Alternatives considered
Migrating one rule at a time was considered. Grouping by family lets each PR add the family's inventory helper once and review the shared common module once.
Additional context
- Blocked by feat(scanner): evaluate() contract foundation - fallible inventory, shared helpers, single engine path #369 STORAGE_REF=feat(scanner): migrate storage rules AZ-STOR-001 to 010 to the evaluate() contract #370.
- Existing tests:
tests/test_rules_security_operations_*.py,tests/test_rules_supply_chain.py,tests/test_devops_client.py - Core-team scope: these rules depend on shared collectors and multi-call evidence, so the status mapping needs careful review.
Activity