-
Notifications
You must be signed in to change notification settings - Fork 3
feat(vm-consumption-report): add VM consumption reporting role #7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
stevefulme1
wants to merge
15
commits into
redhat-cop:main
from
stevefulme1:feat/vm-consumption-report
Closed
Changes from all commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
73a1424
feat(vm-consumption-report): add VM consumption reporting role
stevefulme1 f4688c7
fix(vm-consumption-report): add name[prefix] to included task files
stevefulme1 c5422c5
fix(vm-consumption-report): wrap long task names to pass yaml[line-le…
stevefulme1 95f76d4
fix: add missing allowlist_externals for tox-ansible environments
stevefulme1 2a0d6f4
fix: let tox-ansible manage test commands instead of overriding
stevefulme1 b80d3de
fix: add distlib to integration requirements for manifest support
stevefulme1 4fb2a1a
fix: add distlib to testenv deps for galaxy/sanity manifest support
stevefulme1 eebcb0d
fix: remove testenv overrides — only set allowlist_externals, let tox…
stevefulme1 cc7ba81
fix: add distlib to galaxy tox env for manifest support
stevefulme1 6293498
fix: restore original testenv config, only add allowlist_externals
stevefulme1 9c30035
fix: pin tox-ansible<26.2.2 to match main (26.3.0 requires ade)
stevefulme1 89241f3
fix: filter null lastTransitionTime before sort
stevefulme1 6b84fa5
fix: remove invalid guestOSInfo.hostname reference
stevefulme1 214419b
Derive guest hostname from VM spec instead of VMI
stevefulme1 da1567b
Merge branch 'main' into feat/vm-consumption-report
burigolucas File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| --- | ||
|
|
||
| - name: Generate VM Consumption Report | ||
| hosts: localhost | ||
| connection: local | ||
| gather_facts: false | ||
| tasks: | ||
| - name: Invoke VM Consumption Report Role | ||
| ansible.builtin.include_role: | ||
| name: infra.openshift_virtualization_ops.vm_consumption_report | ||
| vars: | ||
| openshift_host: "{{ lookup('ansible.builtin.env', 'K8S_AUTH_HOST', default=Undefined) | default('', True) }}" | ||
| openshift_api_key: "{{ lookup('ansible.builtin.env', 'K8S_AUTH_API_KEY', default=Undefined) | default('', True) }}" # noqa: yaml[line-length] | ||
| openshift_verify_ssl: "{{ lookup('ansible.builtin.env', 'K8S_AUTH_VERIFY_SSL', default='') | default(false) | bool }}" # noqa: yaml[line-length] | ||
| ... |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| # vm_consumption_report | ||
|
|
||
| Generate consumption reports for OpenShift Virtualization VMs managed | ||
| via the `redhat.openshift_virtualization` collection. The report | ||
| distinguishes between **directly managed hosts** (contacted via SSH or | ||
| WinRM) and **indirectly managed VMs** (automated via the Kubernetes | ||
| API), and supports **reconciliation** of hosts that are managed through | ||
| both methods. | ||
|
|
||
| ## Requirements | ||
|
|
||
| - `kubernetes.core` collection | ||
| - OpenShift API access with permission to list VirtualMachine and | ||
| VirtualMachineInstance resources | ||
|
|
||
| ## Role Variables | ||
|
|
||
| | Variable | Required | Default | Description | | ||
| |---|---|---|---| | ||
| | `vm_consumption_report_request` | yes | `[]` | List of query entries (namespace, names, label_selectors) | | ||
| | `vm_consumption_report_direct_hosts` | no | `[]` | List of directly managed hosts for reconciliation | | ||
| | `vm_consumption_report_openshift_host` | yes | `{{ openshift_host }}` | OpenShift API endpoint | | ||
| | `vm_consumption_report_openshift_api_key` | yes | `{{ openshift_api_key }}` | OpenShift API token | | ||
| | `vm_consumption_report_openshift_verify_ssl` | yes | `{{ openshift_verify_ssl }}` | Verify SSL certificates | | ||
| | `vm_consumption_report_output_format` | no | `json` | Output format: `json` or `csv` | | ||
| | `vm_consumption_report_output_path` | no | `""` | File path to write the report | | ||
| | `vm_consumption_report_include_status` | no | `true` | Query VMIs for running status and guest OS info | | ||
|
|
||
| ## Output | ||
|
|
||
| The role sets `vm_consumption_report_result` containing: | ||
|
|
||
| - `summary` — Counts of total VMs, indirect-only, direct-only, and reconciled | ||
| - `indirect_vms` — List of OpenShift VMs with UUID, namespace, status, guest hostname, and management type | ||
| - `direct_only_hosts` — Hosts that are directly managed but not matched to any OpenShift VM | ||
|
|
||
| ## Reconciliation | ||
|
|
||
| When `vm_consumption_report_direct_hosts` is provided, the role | ||
| matches direct hosts to OpenShift VMs by: | ||
|
|
||
| 1. **UUID match** — Direct host UUID matches VM `metadata.uid` | ||
| 2. **Hostname match** — Direct host hostname matches the guest OS | ||
| hostname reported by the VMI | ||
| 3. **IP match** — Direct host hostname matches an IP address on a VMI | ||
| network interface | ||
|
|
||
| Matched entries are marked `management_type: reconciled`. | ||
|
|
||
| ## Example | ||
|
|
||
| ```yaml | ||
| vm_consumption_report_request: | ||
| - namespace: production-vms | ||
| - namespace: staging-vms | ||
| label_selectors: | ||
| - app=web | ||
|
|
||
| vm_consumption_report_direct_hosts: | ||
| - hostname: web-server-01.example.com | ||
| - hostname: db-server-02.example.com | ||
| uuid: 5a3f8c2e-1234-5678-9abc-def012345678 | ||
|
|
||
| vm_consumption_report_output_format: json | ||
| vm_consumption_report_output_path: /tmp/consumption_report.json | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| --- | ||
| # defaults file for vm_consumption_report | ||
|
|
||
| # title: Consumption Report Request | ||
| # required: True | ||
| # description: >- | ||
| # List of namespaces (or all namespaces) to include in the | ||
| # consumption report. Each entry may filter by namespace, names, | ||
| # or label selectors. | ||
| vm_consumption_report_request: [] | ||
| # - namespace: <namespace> # Namespace to query (omit for all namespaces) | ||
| # names: # Optional list of VM names to include | ||
| # - <vm-name> | ||
| # label_selectors: # Optional label selectors (cannot be combined with names) | ||
| # - <key>=<value> | ||
|
|
||
| # title: Directly Managed Hosts | ||
| # required: False | ||
| # description: >- | ||
| # List of directly managed hosts (contacted via SSH/WinRM) with | ||
| # their hostnames for reconciliation against indirectly managed | ||
| # OpenShift VMs. Each entry should include hostname and optionally | ||
| # a UUID if known. | ||
| vm_consumption_report_direct_hosts: [] | ||
| # - hostname: <hostname> # Hostname or IP used by AAP/SSH | ||
| # uuid: <uuid> # Optional UUID if already known | ||
|
|
||
| # title: OpenShift host | ||
| # required: True | ||
| # description: OpenShift host | ||
| vm_consumption_report_openshift_host: "{{ openshift_host }}" | ||
|
|
||
| # title: OpenShift API Key | ||
| # required: True | ||
| # description: OpenShift API Key | ||
| vm_consumption_report_openshift_api_key: "{{ openshift_api_key }}" | ||
|
|
||
| # title: Verify SSL Certificate | ||
| # required: True | ||
| # description: Variable to enable SSL verification | ||
| vm_consumption_report_openshift_verify_ssl: "{{ openshift_verify_ssl }}" | ||
|
|
||
| # title: KubeVirt API Version | ||
| # required: True | ||
| # description: KubeVirt API Version | ||
| vm_consumption_report_kubevirt_api_version: kubevirt.io/v1 | ||
|
|
||
| # title: Report Output Format | ||
| # required: True | ||
| # description: Output format for the consumption report (json or csv) | ||
| vm_consumption_report_output_format: json | ||
|
|
||
| # title: Report Output Path | ||
| # required: False | ||
| # description: >- | ||
| # File path to write the consumption report. If not set, the report | ||
| # is returned as a variable only. | ||
| vm_consumption_report_output_path: "" | ||
|
|
||
| # title: Include Running Status | ||
| # required: True | ||
| # description: >- | ||
| # When true, queries VirtualMachineInstance resources to determine | ||
| # current running state and guest OS info for reconciliation. | ||
| vm_consumption_report_include_status: true | ||
| ... |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| --- | ||
| galaxy_info: | ||
| author: "" | ||
| description: >- | ||
| Generate consumption reports for OpenShift Virtualization VMs, | ||
| distinguishing directly and indirectly managed hosts with | ||
| UUID-based reconciliation. | ||
| company: Red Hat | ||
| license: GPL-3.0-only | ||
| min_ansible_version: 2.15.0 | ||
| galaxy_tags: [] | ||
| dependencies: [] | ||
| ... |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| --- | ||
|
|
||
| - name: _build_report | Build VM Entries | ||
| ansible.builtin.set_fact: | ||
| _vm_consumption_report_entries: >- | ||
| {{ _vm_consumption_report_entries | default([]) + [_entry] }} | ||
| vars: | ||
| _vmi_match: >- | ||
| {{ _vm_consumption_report_vmis | ||
| | selectattr('metadata.namespace', 'equalto', _vm.metadata.namespace) | ||
| | selectattr('metadata.name', 'equalto', _vm.metadata.name) | ||
| | list | first | default({}) }} | ||
| _vm_uuid: "{{ _vm.metadata.uid }}" | ||
| _guest_hostname: >- | ||
| {{ _vm.spec.template.spec.hostname | ||
| | default(_vm.metadata.name) }} | ||
| _guest_ip_addresses: >- | ||
| {{ _vmi_match.status.interfaces | ||
| | default([]) | ||
| | map(attribute='ipAddress') | ||
| | select('defined') | ||
| | list }} | ||
| _direct_match: >- | ||
| {{ vm_consumption_report_direct_hosts | ||
| | selectattr('uuid', 'defined') | ||
| | selectattr('uuid', 'equalto', _vm_uuid) | ||
| | list | ||
| + (vm_consumption_report_direct_hosts | ||
| | selectattr('hostname', 'defined') | ||
| | selectattr('hostname', 'equalto', _guest_hostname) | ||
| | list if _guest_hostname | length > 0 else []) | ||
| + (vm_consumption_report_direct_hosts | ||
| | selectattr('hostname', 'defined') | ||
| | selectattr('hostname', 'in', _guest_ip_addresses) | ||
| | list if _guest_ip_addresses | length > 0 else []) }} | ||
| _management_type: >- | ||
| {{ 'reconciled' if _direct_match | length > 0 | ||
| else 'indirect' }} | ||
| _entry: | ||
| name: "{{ _vm.metadata.name }}" | ||
| namespace: "{{ _vm.metadata.namespace }}" | ||
| uuid: "{{ _vm_uuid }}" | ||
| management_type: "{{ _management_type }}" | ||
| running: "{{ _vm.spec.running | default(_vm.spec.runStrategy | default('Unknown')) }}" | ||
| created_at: "{{ _vm.metadata.creationTimestamp }}" | ||
| last_transition: >- | ||
| {{ _vm.status.conditions | ||
| | default([]) | ||
| | selectattr('lastTransitionTime', 'defined') | rejectattr('lastTransitionTime', 'none') | ||
| | sort(attribute='lastTransitionTime') | ||
| | map(attribute='lastTransitionTime') | ||
| | list | last | default(_vm.metadata.creationTimestamp) }} | ||
| guest_hostname: "{{ _guest_hostname }}" | ||
| ip_addresses: "{{ _guest_ip_addresses }}" | ||
| direct_host_match: "{{ _direct_match | first | default({}) }}" | ||
| labels: "{{ _vm.metadata.labels | default({}) }}" | ||
| loop: "{{ _vm_consumption_report_vms }}" | ||
| loop_control: | ||
| loop_var: _vm | ||
| label: "{{ _vm.metadata.namespace }}/{{ _vm.metadata.name }}" | ||
|
|
||
| - name: _build_report | Build Direct-Only Entries | ||
| ansible.builtin.set_fact: | ||
| _vm_consumption_report_direct_only: >- | ||
| {{ _vm_consumption_report_direct_only | default([]) + [_direct_entry] }} | ||
| vars: | ||
| _matched_uuids: >- | ||
| {{ (_vm_consumption_report_entries | default([])) | ||
| | selectattr('management_type', 'equalto', 'reconciled') | ||
| | map(attribute='direct_host_match') | ||
| | select('mapping') | ||
| | map(attribute='hostname', default='') | ||
| | list }} | ||
| _matched_hosts: >- | ||
| {{ (_vm_consumption_report_entries | default([])) | ||
| | selectattr('management_type', 'equalto', 'reconciled') | ||
| | map(attribute='direct_host_match') | ||
| | select('mapping') | ||
| | map(attribute='hostname', default='') | ||
| | list }} | ||
| _direct_entry: | ||
| hostname: "{{ _host.hostname }}" | ||
| uuid: "{{ _host.uuid | default('') }}" | ||
| management_type: "direct" | ||
| loop: "{{ vm_consumption_report_direct_hosts }}" | ||
| loop_control: | ||
| loop_var: _host | ||
| label: "{{ _host.hostname }}" | ||
| when: | ||
| - _host.hostname not in _matched_hosts | ||
| - (_host.uuid | default('')) not in ((_vm_consumption_report_entries | default([])) | map(attribute='uuid') | list) | ||
|
|
||
| - name: _build_report | Compile Final Report | ||
| ansible.builtin.set_fact: | ||
| vm_consumption_report_result: | ||
| generated_at: "{{ now(utc=true, fmt='%Y-%m-%dT%H:%M:%SZ') }}" | ||
| cluster: "{{ vm_consumption_report_openshift_host }}" | ||
| summary: | ||
| total_vms: "{{ (_vm_consumption_report_entries | default([])) | length }}" | ||
| indirect_only: >- | ||
| {{ (_vm_consumption_report_entries | default([])) | ||
| | selectattr('management_type', 'equalto', 'indirect') | ||
| | list | length }} | ||
| direct_only: "{{ (_vm_consumption_report_direct_only | default([])) | length }}" | ||
| reconciled: >- | ||
| {{ (_vm_consumption_report_entries | default([])) | ||
| | selectattr('management_type', 'equalto', 'reconciled') | ||
| | list | length }} | ||
| total_managed_hosts: >- | ||
| {{ (_vm_consumption_report_entries | default([])) | ||
| | length | ||
| + (_vm_consumption_report_direct_only | default([])) | ||
| | length }} | ||
| indirect_vms: "{{ _vm_consumption_report_entries | default([]) }}" | ||
| direct_only_hosts: "{{ _vm_consumption_report_direct_only | default([]) }}" | ||
| ... | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| --- | ||
|
|
||
| - name: >- | ||
| _collect_vmis | Query VMIs — | ||
| {{ _vm_consumption_report_query.namespace | default('all-namespaces') }} | ||
| kubernetes.core.k8s_info: | ||
| api_key: "{{ vm_consumption_report_openshift_api_key }}" | ||
| host: "{{ vm_consumption_report_openshift_host }}" | ||
| api_version: "{{ vm_consumption_report_kubevirt_api_version }}" | ||
| kind: VirtualMachineInstance | ||
| namespace: "{{ _vm_consumption_report_query.namespace | default(omit) }}" | ||
| label_selectors: "{{ _vm_consumption_report_query.label_selectors | default(omit) }}" | ||
| validate_certs: "{{ vm_consumption_report_openshift_verify_ssl }}" | ||
| register: _vm_consumption_report_vmi_response | ||
|
|
||
| - name: _collect_vmis | Append VMIs to Collection | ||
| ansible.builtin.set_fact: | ||
| _vm_consumption_report_vmis: >- | ||
| {{ _vm_consumption_report_vmis + _vm_consumption_report_vmi_response.resources }} | ||
| ... |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| --- | ||
|
|
||
| - name: >- | ||
| _collect_vms | Query VMs — | ||
| {{ _vm_consumption_report_query.namespace | default('all-namespaces') }} | ||
| kubernetes.core.k8s_info: | ||
| api_key: "{{ vm_consumption_report_openshift_api_key }}" | ||
| host: "{{ vm_consumption_report_openshift_host }}" | ||
| api_version: "{{ vm_consumption_report_kubevirt_api_version }}" | ||
| kind: VirtualMachine | ||
| namespace: "{{ _vm_consumption_report_query.namespace | default(omit) }}" | ||
| label_selectors: "{{ _vm_consumption_report_query.label_selectors | default(omit) }}" | ||
| validate_certs: "{{ vm_consumption_report_openshift_verify_ssl }}" | ||
| register: _vm_consumption_report_vm_response | ||
|
|
||
| - name: _collect_vms | Filter VMs by Name (if specified) | ||
| ansible.builtin.set_fact: | ||
| _vm_consumption_report_filtered: >- | ||
| {{ _vm_consumption_report_vm_response.resources | ||
| | selectattr('metadata.name', 'in', _vm_consumption_report_query.names) | ||
| | list }} | ||
| when: "'names' in _vm_consumption_report_query" | ||
|
|
||
| - name: _collect_vms | Use All Returned VMs (no name filter) | ||
| ansible.builtin.set_fact: | ||
| _vm_consumption_report_filtered: "{{ _vm_consumption_report_vm_response.resources }}" | ||
| when: "'names' not in _vm_consumption_report_query" | ||
|
|
||
| - name: _collect_vms | Append VMs to Collection | ||
| ansible.builtin.set_fact: | ||
| _vm_consumption_report_vms: "{{ _vm_consumption_report_vms + _vm_consumption_report_filtered }}" | ||
| ... |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| --- | ||
|
|
||
| - name: Verify Request Variable Provided | ||
| ansible.builtin.assert: | ||
| that: | ||
| - vm_consumption_report_request | default([], true) | length > 0 | ||
| fail_msg: "'vm_consumption_report_request' must contain at least one entry" | ||
| quiet: true | ||
|
|
||
| - name: Initialize Report Data | ||
| ansible.builtin.set_fact: | ||
| _vm_consumption_report_vms: [] | ||
| _vm_consumption_report_vmis: [] | ||
|
|
||
| - name: Collect VirtualMachines | ||
| ansible.builtin.include_tasks: _collect_vms.yml | ||
| loop: "{{ vm_consumption_report_request }}" | ||
| loop_control: | ||
| loop_var: _vm_consumption_report_query | ||
| label: "{{ _vm_consumption_report_query.namespace | default('all-namespaces') }}" | ||
|
|
||
| - name: Collect VirtualMachineInstances for Running Status | ||
| ansible.builtin.include_tasks: _collect_vmis.yml | ||
| loop: "{{ vm_consumption_report_request }}" | ||
| loop_control: | ||
| loop_var: _vm_consumption_report_query | ||
| label: "{{ _vm_consumption_report_query.namespace | default('all-namespaces') }}" | ||
| when: vm_consumption_report_include_status | bool | ||
|
|
||
| - name: Build Consumption Report | ||
| ansible.builtin.include_tasks: _build_report.yml | ||
|
|
||
| - name: Write Report to File | ||
| ansible.builtin.copy: | ||
| content: "{{ vm_consumption_report_result | to_nice_json }}" | ||
| dest: "{{ vm_consumption_report_output_path }}" | ||
| mode: "0644" | ||
| when: | ||
| - vm_consumption_report_output_path | default("", true) | length > 0 | ||
| - vm_consumption_report_output_format == "json" | ||
|
|
||
| - name: Write CSV Report to File | ||
| ansible.builtin.template: | ||
| src: report.csv.j2 | ||
| dest: "{{ vm_consumption_report_output_path }}" | ||
| mode: "0644" | ||
| when: | ||
| - vm_consumption_report_output_path | default("", true) | length > 0 | ||
| - vm_consumption_report_output_format == "csv" | ||
|
|
||
| - name: Display Report Summary | ||
| ansible.builtin.debug: | ||
| msg: | ||
| total_vms: "{{ vm_consumption_report_result.summary.total_vms }}" | ||
| indirect_only: "{{ vm_consumption_report_result.summary.indirect_only }}" | ||
| direct_only: "{{ vm_consumption_report_result.summary.direct_only }}" | ||
| reconciled: "{{ vm_consumption_report_result.summary.reconciled }}" | ||
| report_generated: "{{ vm_consumption_report_result.generated_at }}" | ||
| ... |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Need to filter out items that contain
nullvaluesThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch — pushed this fix in 89241f3. Added
selectattr('lastTransitionTime', 'defined') | rejectattr('lastTransitionTime', 'none')before the sort.