Skip to content

feat: add rule AZ-CMP-007 management ports open without JIT VM access - #307

Merged
TFT444 merged 2 commits into
openshield-org:devfrom
shariqueahmad108-ship-it:rule/az-cmp-007-jit
Aug 28, 2026
Merged

feat: add rule AZ-CMP-007 management ports open without JIT VM access#307
TFT444 merged 2 commits into
openshield-org:devfrom
shariqueahmad108-ship-it:rule/az-cmp-007-jit

Conversation

@shariqueahmad108-ship-it

Copy link
Copy Markdown
Contributor

What

Implements #270 as AZ-CMP-007 — flags VMs whose management ports (SSH 22 / RDP 3389) are open to the internet via their NIC's NSG but are not covered by a Microsoft Defender for Cloud Just-In-Time (JIT) VM access policy. Standing-open management ports are continuously scanned and brute-forced; JIT limits exposure to approved, time-boxed windows.

Detection logic

Reuses the repo's own building blocks rather than reinventing them:

  • Management-port exposure — the same "allow inbound <port> from any source (*/0.0.0.0/0/Internet)" test that AZ-NET-001/002 use, applied to ports 22 and 3389.
  • VM → NIC → NSG resolution — the AZ-CMP-001 path: each VM's NIC is resolved, its NSG looked up by id from get_network_security_groups(), and its security_rules scanned.
  • JIT cross-reference — a new AzureClient.get_jit_network_access_policies() getter (Defender for Cloud SecurityCenter SDK). Each policy's virtual_machines[].id + ports[].number build a per-VM covered-port set. A VM is flagged only for management ports that are open and not covered.

Edge cases handled deliberately:

  • No management ports open → NOT_APPLICABLE (not flagged).
  • Trusted-source-only rule (e.g. a corporate CIDR, not the internet) → not flagged.
  • Partial coverage → only the uncovered port is reported (metadata lists open_management_ports, uncovered_ports, jit_policy_present).
  • Defender for Cloud unreadable (None) → coverage is indeterminate, so the VM is not flagged — the rule never false-positives when it can't confirm "no JIT". This mirrors the repo's existing "never treat indeterminate as a violation" stance.

Compliance mappings (all four framework JSONs)

Framework Control
NIST CSF PR.AC-3 Remote access is managed
ISO 27001 A.13.1.1 Network controls
SOC 2 CC6.6 Restricts access from outside the network boundary
CIS N/A-CMP-007 JIT is a Defender-for-Cloud recommendation; CIS Azure Foundations 2.0.0 has no numbered control, so this uses the repo's existing N/A convention (as AZ-KV-001). Happy to map to a preferred id.

Notes

Tests

Six unit tests in tests/test_rules_compute.py (mock-based, no network): open-SSH-no-JIT, JIT-covered, no-mgmt-ports (NOT_APPLICABLE), trusted-source, partial-coverage, and indeterminate-JIT.

pytest tests/test_rules_compute.py   -> 22 passed
ruff check .            -> clean
ruff format --check .   -> clean

Honest testing note

I don't have an Azure subscription, so I could not run the issue's "tested against a real Azure free-trial subscription" step — that box is left for a maintainer to confirm. The SDK shapes the code relies on (SecurityCenter(credential, subscription_id), jit_network_access_policies.list(), JitNetworkAccessPolicyVirtualMachine.id/.ports, JitNetworkAccessPortRule.number, and the NSG rule fields) were verified against the installed azure-mgmt-security==7.0.0 and azure-mgmt-network models, and the rule logic is fully covered by the unit tests above.

Test plan (from the issue)

  • Compliant case returns no findings (unit)
  • Non-compliant case returns correct finding (unit)
  • VMs with no management ports open return NOT_APPLICABLE (unit)
  • CLI playbook written (fix_az_cmp_007.sh)
  • All four compliance framework JSONs updated
  • Tested against a real Azure free-trial subscription (needs maintainer — no Azure access on my side)

Closes #270

Adds a Compute scan rule that flags VMs whose management ports (SSH 22 /
RDP 3389) are open to the internet through their NIC's NSG but are not
covered by a Microsoft Defender for Cloud Just-In-Time (JIT) VM access
policy, leaving those ports standing open instead of only during an
approved, time-boxed request.

Detection reuses the repo's proven building blocks: az_net_001's
"allow inbound port from any source" test and az_cmp_001's VM -> NIC -> NSG
resolution, cross-referenced against JIT policies from a new
AzureClient.get_jit_network_access_policies() getter (Defender for Cloud
SecurityCenter SDK; azure-mgmt-security is already a dependency). VMs with
no management ports open are NOT_APPLICABLE, and when Defender for Cloud
cannot be queried (None) coverage is indeterminate and the VM is not
flagged, so the rule never false-positives on unknown coverage.

Includes the remediation playbook (playbooks/cli/fix_az_cmp_007.sh, with
guarded args per the fix_az_net_016.sh convention), mock support
(set_jit_policies / get_jit_network_access_policies), six unit tests
(open-no-JIT, JIT-covered, NOT_APPLICABLE, trusted-source, partial-coverage,
indeterminate), and mappings across the four compliance frameworks
(NIST CSF PR.AC-3, ISO 27001 A.13.1.1, SOC 2 CC6.6; CIS uses the repo's N/A
convention as Azure Foundations 2.0.0 has no numbered JIT control).

Closes openshield-org#270

Signed-off-by: shariqueahmad108-ship-it <shariqueahmad108@gmail.com>

@ritiksah141 ritiksah141 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Port ranges are missed

The detection only compares exact values and *. A valid NSG range such as 20-30 exposes SSH, but the rule returns false. I reproduced this directly.

Suggested inline comment around az_cmp_007.py:52:

This only recognises an exact port or *. Azure NSG rules can express destination ranges, so a rule such as 20-30 exposes SSH but this implementation treats
port 22 as closed. Please parse ranges in both destination_port_range and destination_port_ranges, and add regression coverage for 22/3389 contained inside a
range.

  1. Subnet-level NSGs are ignored

The implementation only follows nic.network_security_group. A VM can be exposed through the NSG attached to its NIC’s subnet while having no NIC-level NSG,
Suggested inline comment around az_cmp_007.py:91:

This only checks an NSG attached directly to the NIC. A VM can have no NIC-level NSG while its subnet NSG permits inbound SSH/RDP, so the rule would
incorrectly return NOT_APPLICABLE for an exposed VM. Please include the NIC IP configuration’s subnet NSG/effective exposure path and add a subnet-only
regression test.

  1. The remediation can overwrite existing JIT coverage

The script sends a complete PUT to a shared policy named default, with only the selected VM in properties.virtualMachines. The Azure API defines this as “Create
or Update” and requires the complete VM configuration array, so existing entries need to be preserved. Microsoft API documentation
(https://learn.microsoft.com/en-us/rest/api/defenderforcloud/jit-network-access-policies/create-or-update?view=rest-defenderforcloud-2020-01-01).

Suggested inline comment around fix_az_cmp_007.sh:25:

This PUT sends a complete default policy containing only the selected VM. If that policy already protects other VMs, this can replace its virtualMachines
array and remove their JIT configuration. Please fetch the current policy and merge/update this VM without dropping existing entries, or use a safely scoped
policy name. Add a test or documented validation proving an existing multi-VM policy is preserved.

Suggested overall review in your style:

@shariqueahmad108-ship-it, the overall rule structure, framework mappings and categorized tests look good, but there are three cases holding my approval:

  • NSG destination port ranges such as 20-30 are not evaluated, which causes false negatives for SSH/RDP.
  • Only NIC-level NSGs are checked, so exposure through a subnet-level NSG is missed.
  • The remediation PUT writes the shared default JIT policy with only one VM and can remove existing VM entries.

Please address the inline comments and add regression coverage for the range and subnet cases, plus make the playbook preserve existing JIT policy entries.
Also the full repository CI checks still need to run; currently only the Semgrep check is shown.

@TFT444 TFT444 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Three correctness blockers. First, port-range detection does not handle range strings like '20-30', so a rule covering port 22 via a range never fires. Second, only NIC-level NSGs are checked and subnet-attached NSGs exposing the same port are silently missed. Third, the remediation PUT replaces the entire jitNetworkAccessPolicies/default VM list, which destroys existing JIT entries for other VMs in the policy. Scope the PUT to only the target VM.

…tion

Addresses @ritiksah141's review on openshield-org#307:

- Port ranges: _rule_allows_port_from_any now parses '*', single ports, and
  inclusive ranges (e.g. 20-30 exposing SSH) in both destination_port_range and
  destination_port_ranges, instead of only exact/'*' matches.
- Subnet-level NSGs: exposure is evaluated across every NSG that governs the VM
  -- the NIC's NSG and the NSG on the NIC IP configuration's subnet (resolved via
  each NSG's own .subnets back-reference) -- so a VM with no NIC-level NSG but an
  exposing subnet NSG is no longer a false NOT_APPLICABLE.
- Remediation: fix_az_cmp_007.sh now GETs the existing 'default' JIT policy and
  merges this VM in via jit_policy_merge.py (preserving every other VM's entry)
  before the create-or-update PUT, instead of overwriting the shared policy with
  a single VM.

Adds regression tests: 22 inside a range, 3389 inside a ranges-list entry, a
subnet-only NSG exposure case, and jit_policy_merge tests proving an existing
multi-VM policy is preserved (and that re-remediation updates in place).

Signed-off-by: shariqueahmad108-ship-it <shariqueahmad108@gmail.com>
@shariqueahmad108-ship-it

Copy link
Copy Markdown
Contributor Author

Thanks @ritiksah141 and @TFT444 — all three are fixed in 0466e42c:

  1. Port ranges_rule_allows_port_from_any now parses *, single ports, and inclusive ranges (e.g. 20-30 covering 22) in both destination_port_range and destination_port_ranges. Regression tests added for 22 inside 20-30 and 3389 inside a 3380-3400 ranges-list entry.
  2. Subnet-level NSGs — exposure is now evaluated across every NSG that governs the VM: the NIC's NSG and the subnet's NSG (resolved via each NSG's .subnets back-reference, so no extra API calls). Added a subnet-only regression test — no NIC-level NSG, exposing subnet NSG → correctly flagged instead of NOT_APPLICABLE.
  3. Remediationfix_az_cmp_007.sh now GETs the existing default policy and merges this VM in via the new jit_policy_merge.py (preserving every other VM's entry) before the create-or-update PUT, instead of overwriting the shared policy. Added tests/test_jit_policy_merge.py proving an existing multi-VM policy is preserved, and that re-remediating the same VM updates its entry in place rather than duplicating it.

Local: 25 compute-rule tests + 4 merge tests pass, ruff clean. Thanks for the thorough reviews!

@parthrohit22 parthrohit22 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked this against az_net_001.py/az_net_002.py's "allow from any" helper and azure_client.py's existing caching/error-handling conventions, and verified the JIT and NSG SDK model shapes this rule relies on actually match azure-mgmt-security/azure-mgmt-network (jit_network_access_policies.list(), JitNetworkAccessPolicy.virtual_machines[].id/.ports[].number, NetworkSecurityGroup.subnets back-reference).

What stands out as solid:

  • Indeterminate-vs-violation handling is correct: get_jit_network_access_policies() returns None on API/permission failure (distinct from [] for "no policies exist"), and scan() treats None as indeterminate rather than a confirmed finding — matches this codebase's convention, and it's directly tested.
  • The remediation script (jit_policy_merge.py) does a GET-then-merge before PUT rather than a naive replace — a naive single-VM PUT would have silently dropped every other VM's JIT config off the shared default policy. Good catch, and it's covered by tests proving existing VMs survive re-remediation.
  • Port-range matching (a 20-30 range covering port 22) and subnet-level NSG resolution (VM with only a subnet NSG, no NIC-level NSG) are both handled, with regression tests for each.
  • Six original + seven regression tests exercise real scenarios (open-no-JIT, JIT-covered, NOT_APPLICABLE, trusted-source-only, partial coverage, indeterminate JIT, range-based exposure) rather than being tautological.

Two non-blocking notes, no action needed to merge:

  • _rule_allows_port_from_any doesn't check protocol (a UDP-only allow on port 22 would still flag) or NSG rule priority ordering — inherited as-is from the existing az_net_001/az_net_002 helper this rule reuses, not something new here.
  • docs/rules-reference.md and website/content.js still only list AZ-CMP-001..004 — AZ-CMP-007 isn't in the public rule catalog docs yet. Not enforced by CI, so not a blocker, but worth a quick follow-up.

Full suite passes (724 passed, 3 skipped, all pre-existing/unrelated), ruff clean, playbook and merge-helper script both valid, no RULE_ID or PLAYBOOK path issues.

@parthrohit22

Copy link
Copy Markdown
Member

@TFT444 @ritiksah141
I think this PR is ready for merge now, re-requesting reviews from you guys,
if agreed with my previous comments update it, I will merge it .

@ritiksah141 ritiksah141 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All issues flagged by me is being addressed, so approving it

@shariqueahmad108-ship-it

Copy link
Copy Markdown
Contributor Author

Friendly nudge — this one looks ready whenever a maintainer has a moment:

  • All three review points (port-range parsing, subnet-level NSGs, and the JIT-preserving remediation) are fixed in 0466e42c, with regression tests for each.
  • @ritiksah141 re-reviewed after the fixes and approved; @parthrohit22 also approved and noted it's ready to merge.
  • All 20 CI checks are green.

@TFT444 — since your review flagged the same three items, could you take another look (or dismiss if satisfied)? Happy to address anything else. Thanks all for the thorough reviews!

@Vishnu2707 Vishnu2707 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

@TFT444

TFT444 commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

merging now as everything good to go

@TFT444
TFT444 merged commit efc93d7 into openshield-org:dev Aug 28, 2026
20 checks passed
parthrohit22 added a commit to parthrohit22/openshield that referenced this pull request Aug 28, 2026
…ontract v1

Rebases this branch onto dev now that PR openshield-org#308 (severity contract v1,
d8e4f6a1b2c3) has merged, and repoints 3a76ff935bf6's down_revision at
it as planned in the migration's own docstring, so `alembic heads`
resolves to a single head again.

Reconciles openshield-org#308's atomic/idempotent save_scan() and score_counts()
usage with this branch's compliance-mapping-snapshot and evidence-
schema work in api/models/finding.py, api-reference.md, and
architecture.md, and merges the CI step lists in ci.yml so both
suites run.

Two issues surfaced while reconciling the two branches' code, fixed
here rather than deferred:

- get_compliance_score()'s severity/category grouping query was about
  to run against openshield-org#308's finding.py through the RealDictCursor this
  branch used elsewhere in the same method. RealDictRow has no
  __iter__ override, so positional unpacking of its rows silently
  reads back key names instead of values. Restored openshield-org#308's own plain
  cursor for that one query, matching its tested convention, instead
  of carrying the bug or rewriting openshield-org#308's already-merged code.
- .github/scripts/validate_mapping_pack.py flagged AZ-CMP-007 (added
  by the already-merged openshield-org#307) as missing the evidence-schema fields
  this branch's mapping-pack validation requires, across all four
  framework files. Filled them in following the existing sibling-rule
  conventions in each file.

Updates the affected tests in test_clean_scan.py,
test_compliance_scoring.py, and test_severity_contract.py for the new
call shapes, and fixes frontend/src/utils/api.test.mjs's module-
loading harness for api.js's new severity.js import from openshield-org#308
(stubbed, since the functions under test here don't call it and
severity.js has its own dedicated suite).

Verified: alembic heads (single head) and a full upgrade/downgrade/
upgrade/heads cycle against a local Postgres instance; full pytest
suite (818 passed, 3 skipped); ruff check and format --check; mapping-
pack validator; and the full frontend test/lint/build suite.

Signed-off-by: Parth J Rohit <parthrohit60@gmail.com>
Signed-off-by: parthrohit22 <parthrohit60@gmail.com>
parthrohit22 added a commit to parthrohit22/openshield that referenced this pull request Aug 29, 2026
…Z-SECOPS-010

The dev merge into this branch pulled in openshield-org#277 (enterprise network and
perimeter controls) and openshield-org#320, both merged since this branch was last
updated. openshield-org#277's ten new AZ-NET-018..027 rules and the previously-merged
AZ-SECOPS-010 carry compliance mappings across all four framework
files with no mapping_type/evidence_type/primary_source/rationale/
review_status - this PR's own validate_mapping_pack.py correctly
rejects that, since neither rule existed yet when this PR's evidence-
schema requirement was written. Same root cause as the AZ-CMP-007 gap
from openshield-org#307 fixed earlier in this branch's history.

Filled in following the exact conventions of each entry's nearest
sibling in the same file:
- cis_azure_benchmark.json: mapping_type "not_applicable" for the
  N/A-* control_ids (all ten AZ-NET-0xx entries), "direct" for
  AZ-SECOPS-010's real numbered control (2.1.20) - matching this PR's
  existing framework-level-default convention for non-N/A CIS entries.
- nist_csf.json / iso27001.json / soc2.json: mapping_type "supporting",
  evidence_type "automated_configuration_scan", with primary_source/
  rationale text following the exact template of each file's sibling
  entries (e.g. AZ-NET-016, AZ-SECOPS-009).
- owner: null, review_status: "pending_review", review_date: null,
  matching every other unreviewed entry in these files.

Also cleans up a merge artifact in CHANGELOG.md: openshield-org#277 had added a
second, malformed top-level "## Unreleased" section (missing brackets,
no ### subsections) above the file's existing well-formed
"## [Unreleased]" - folded its one entry into the correct section.

Verified: mapping-pack validator clean (0 errors, was 55). Full
backend suite (892 passed, 5 skipped - pre-existing/environment-only).
alembic heads still resolves to exactly one head (3a76ff935bf6).
ruff check and format --check clean. Diff confirmed scoped to exactly
the 11 affected rule_ids across the 4 framework files plus the
CHANGELOG cleanup - no other entries touched.

Signed-off-by: Parth J Rohit <parthrohit60@gmail.com>
Signed-off-by: parthrohit22 <parthrohit60@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(scanner): add AZ-XXX-000 Just-In-Time VM access not configured

5 participants