Repository navigation
feat(scanner): migrate compute and PQC rules (AZ-CMP-001 to 007, AZ-PQC-001 to 003) to the evaluate() contract #374
Description
What problem does this solve?
The compute rules (AZ-CMP-001 to AZ-CMP-007) and post-quantum cryptography rules (AZ-PQC-001 to AZ-PQC-003) each combine several Azure calls per resource (VM → NIC → subnet → NSG, disks, extensions, patch status, JIT policies, Key Vault keys and certificates). These rules only implement scan(), so the engine reports their coverage as UNKNOWN / LEGACY_RULE_NOT_MIGRATED, and every compliance control mapped to them reads UNKNOWN in the evaluation-derived reports introduced in #310.
Describe the solution
Migrate the rules below to the evaluate() contract defined in scanner/evaluation.py, following the storage reference implementation (STORAGE_REF).
Rules in scope: az_cmp_001 … az_cmp_007, az_pqc_001 … az_pqc_003.
Notes: these rules are 120–275 lines each with several dependent lookups per resource. A failure in a dependent lookup (for example get_network_interface() or get_vm_patch_status()) must produce UNKNOWN / EVIDENCE_UNAVAILABLE for that resource only. Add list_virtual_machines() / list_web_apps() variants returning Optional[List] where the existing helper returns [] on failure. The PQC rules reuse list_key_vaults() from the foundation issue.
Status mapping
Every path that scan() currently handles with a silent continue must become an explicit status:
| Situation | Status | reason_code |
|---|---|---|
Inventory call failed (returned None) |
ERROR |
INVENTORY_UNAVAILABLE |
| Inventory succeeded but is empty | NOT_APPLICABLE |
NO_RESOURCES_FOUND |
| Setting is compliant | PASS |
— |
| Setting is a violation | FAIL + the existing finding dict |
— |
A per-resource detail call returned None, or a required field is missing |
UNKNOWN |
EVIDENCE_UNAVAILABLE / MISSING_PROPERTIES |
Opt-in policy tag not set (policy_required(...) is false) |
NOT_APPLICABLE |
POLICY_NOT_REQUIRED |
| Approved exception tag set | NOT_APPLICABLE |
APPROVED_EXCEPTION |
A failed inventory call must never produce PASS or NOT_APPLICABLE.
Acceptance criteria
- Each rule in scope exposes
evaluate(azure_client, subscription_id) -> List[RuleEvaluation]using the shared helpers inscanner/evaluation.py. -
scan()becomes a thin wrapper returning the findings attached toFAILevaluations. - Detection is unchanged: for the existing test fixtures,
scan()returns exactly the same findings as before the change. - Each
FAIL/PASSevaluation records the inspected value(s) inevidence, for example{"allow_blob_public_access": true}. - New tests cover the PASS, FAIL, UNKNOWN, NOT_APPLICABLE and inventory-failure paths for every rule in scope.
- The rules are picked up automatically by
tests/test_rule_evaluation_contract.pyand it passes. -
pytest,ruff check .andruff format --check .pass. -
CHANGELOG.mdupdated underUnreleased.
Alternatives considered
Migrating one rule at a time was considered. Grouping by family lets each PR add the family's inventory helper once and review the shared common module once.
Additional context
- Blocked by feat(scanner): evaluate() contract foundation - fallible inventory, shared helpers, single engine path #369 STORAGE_REF=feat(scanner): migrate storage rules AZ-STOR-001 to 010 to the evaluate() contract #370.
- Existing tests:
tests/test_rules_compute.py,tests/test_jit_policy_merge.py,tests/test_pqc_rules.py,tests/test_az_pqc_001.py - Core-team scope: these rules depend on shared collectors and multi-call evidence, so the status mapping needs careful review.
Activity