Repository navigation
feat(scanner): migrate storage rules AZ-STOR-001 to 010 to evaluate() (#370) - #382
parthrohit22 wants to merge 1 commit into
Conversation
TFT444
left a comment
There was a problem hiding this comment.
Storage rules migration looks clean. scan() parity confirmed for all 10 rules, the intentional AZ-STOR-006 empty-resource-id change is correctly handled and tested, contract test and branch tests cover every UNKNOWN/MISSING_PROPERTIES boundary, and there are no security concerns.
Important: this PR is stacked on #381 and must NOT merge before #381 merges. Please rebase onto dev and retarget to dev once #381 lands. Approving now so the review is on record.
dd9a4be to
737164a
Compare
…#370) Each storage rule now reports PASS, FAIL, UNKNOWN, ERROR or NOT_APPLICABLE for every account (per sub-service for AZ-STOR-004 and per blob container for AZ-STOR-009), records the inspected value as evidence, and reports a failed storage inventory as ERROR. Paths that scan() skipped silently now carry explicit reason codes. Findings are unchanged, except that AZ-STOR-006 no longer emits a finding with an empty resource_id for an account returned without an ID; that account is reported as UNKNOWN. Add policy_exemption() and missing_resource_id() helpers and register the storage rules in the evaluation contract test. Signed-off-by: parthrohit22 <parthrohit60@gmail.com>
737164a to
adf8938
Compare
ritiksah141
left a comment
There was a problem hiding this comment.
Requesting changes on process, not on the code. The storage migration itself looks solid: I reviewed the full diff at head adf8938 locally and ran the suites (contract and storage tests 94/94, full suite 1598 passed / 93 skipped / 0 failed, ruff check and format clean). Findings parity holds for all ten rules, the previously silent skip paths now report explicit statuses, the per-sub-service (AZ-STOR-004) and per-container (AZ-STOR-009) evaluations are correct, and the policy_exemption() refactor preserves the old policy_required() semantics exactly.
Blockers:
-
This PR targets feat/369-evaluate-contract-foundation and #381 has not merged. Merging #382 as-is would push the storage commits into #381's branch and expand that PR beyond what was reviewed and approved there. The PR body itself promises a rebase onto dev and a retarget once #381 merges; please hold until then (#381 still has an outstanding CHANGELOG fix requested in review).
-
There is no CI on this PR. The pull_request trigger in ci.yml only fires when the base is dev or main, so the current head has zero automated validation on GitHub. Retargeting onto dev fixes this; please have the checks green before re-requesting review.
-
The existing approval is stale: it points at commit 3cce17d, the head from before the 2026-10-04 rebase, so adf8938 has not actually been reviewed. After the retarget it needs a fresh approval.
Minor items to fold into the same pass, none blocking on their own:
- The CHANGELOG entry says findings are unchanged "except that AZ-STOR-006 no longer emits a finding with an empty resource_id", but the same change applies to AZ-STOR-001, 007, 008 and 010, whose old scan() paths also used account.id directly in findings. Suggest rewording to say any account returned without an ID is now reported as UNKNOWN/MISSING_PROPERTIES instead of producing an empty-resource_id finding.
- In az_stor_003, az_stor_004 and az_stor_005 the UNKNOWN branches do not attach the evidence dict they compute (for example {"lifecycle_policy": None}), while the PASS and FAIL branches do. Consider recording the inspected value on UNKNOWN rows too, for consistency with the evidence contract.
- UNRECOGNIZED_TLS_VERSION, UNRECOGNIZED_KEY_SOURCE and PUBLIC_ACCESS_DISABLED are inline string literals. Fine for rule-specific codes, but module-level constants would keep them greppable next to the shared codes in scanner/evaluation.py.
Once #381 lands and this is retargeted onto dev with CI green and a fresh review, this should merge quickly.
SHAURYAKSHARMA24
left a comment
There was a problem hiding this comment.
Ran this head (adf8938) locally. With the storage SDK client raising a 403, a 429 or an expired-credential error, all ten storage rules come back as ERROR/INVENTORY_UNAVAILABLE and show up in failed_rule_ids, and an empty subscription gives NOT_APPLICABLE/NO_RESOURCES_FOUND. In a mixed run (Key Vault fine, storage 403) the Key Vault FAIL still comes through next to the storage errors, which is what we want. Full suite matches dev apart from the same two Windows-only failures.
Agree with ritiksah141 on the process side: this should wait for #381, get retargeted to dev so CI actually runs, and then needs a fresh approval on the final head. I'll take another look after the retarget.
What does this PR do?
Migrates the ten storage rules
AZ-STOR-001toAZ-STOR-010to theevaluate()coverage contract, so each reports a status for every resource it checks, PASS included, instead of only listing violations.What changes per rule
AZ-STOR-001public blob accessUNKNOWN / MISSING_PROPERTIESAZ-STOR-002HTTPS onlyUNKNOWN / MISSING_PROPERTIES(Nonestill fails, as before)AZ-STOR-003lifecycle policyUNKNOWN / EVIDENCE_UNAVAILABLE; no ID / name / resource group →UNKNOWN / MISSING_PROPERTIESAZ-STOR-004diagnostic loggingUNKNOWN / EVIDENCE_UNAVAILABLEAZ-STOR-005geo-redundancyUNKNOWN / MISSING_PROPERTIESAZ-STOR-006shared keyNonestill fails (documented Azure default)AZ-STOR-007minimum TLSUNKNOWN / UNRECOGNIZED_TLS_VERSIONAZ-STOR-008customer-managed keyNOT_APPLICABLE / POLICY_NOT_REQUIRED; exception →APPROVED_EXCEPTION; no key source →UNKNOWN; unknown source →UNKNOWN / UNRECOGNIZED_KEY_SOURCEAZ-STOR-009blob immutabilityPOLICY_NOT_REQUIRED/APPROVED_EXCEPTION; containers unreadable →UNKNOWN; no containers →NOT_APPLICABLE / NO_RESOURCES_FOUNDAZ-STOR-010private endpointNOT_APPLICABLE / PUBLIC_ACCESS_DISABLED; connections unavailable →UNKNOWN / EVIDENCE_UNAVAILABLEEvery rule also:
ERROR / INVENTORY_UNAVAILABLEwhen the storage account list call fails;NOT_APPLICABLE / NO_RESOURCES_FOUNDwhen the list is empty;evidence;scan()as a thin wrapper over theFAILevaluations.Shared helpers
policy_exemption()inscanner/rules/_storage_common.pyreturns theNOT_APPLICABLEreason code for opt-in policy tags, orNonewhen the policy applies.policy_required()keeps its behaviour and now delegates to it. The DB, Cosmos and Cache migrations (feat(scanner): migrate SQL and PostgreSQL rules AZ-DB-001 to 007 to the evaluate() contract #377, feat(scanner): migrate Cosmos DB, Redis Cache and data link rules to the evaluate() contract #378) can reuse it.missing_resource_id()inscanner/evaluation.pyhandles resources returned without an ID.Findings are unchanged, with one intentional exception
I recorded every rule's
scan()output before and after the change over a fixture of nine accounts covering every branch: SDK enums,Noneand missing fields, a missing resource group, exception tags, nested container properties, and empty and failed container lists. The output is identical and in the same order, with one exception:AZ-STOR-006used to emit a finding withresource_id: ""for an account returned without an ID. Such a finding cannot be remediated and collides with every other ID-less finding. That account is nowUNKNOWN / MISSING_PROPERTIESwith no finding. Azure always returns an ID, so this only affects malformed input.Type of change
Also a scanner change: storage rules migrated to the evaluation contract.
Rule details (if applicable)
Testing
Local results:
tests/test_rules_storage.py.tests/test_rule_evaluation_contract.py, which checks inventory failure, empty inventory, finding shape andscan()parity.ruff check .andruff format --check .are clean.dev. The local failures are the Postgres-backed tests CI runs against its ephemeral database.Related issue
Closes #370
Part of #380. Depends on #381.
Checklist
Signed-off-bytrailer (git commit -s; seedocs/dco.md)