Skip to content

Cache libvirt domain state#819

Open
ben-grande wants to merge 3 commits into
QubesOS:mainfrom
ben-grande:cache-running
Open

Cache libvirt domain state#819
ben-grande wants to merge 3 commits into
QubesOS:mainfrom
ben-grande:cache-running

Conversation

@ben-grande
Copy link
Copy Markdown
Contributor

@ben-grande ben-grande commented Jun 1, 2026

The state changes rarely, but querying it can take a considerable time, that is blocking, when looping through the state of multiple domains.

For: QubesOS/qubes-issues#10569
For: QubesOS/qubes-issues#9902


Didn't run openqa locally, only experiment some things:

  • start
  • shutdown
  • kill
  • pause
  • unpause
  • checking Domains Widget
  • restart qubesd
  • qubed-query -e dom0 admin.vm.CurrentState QUBE

@ben-grande
Copy link
Copy Markdown
Contributor Author

PipelineRetryFailed

@ben-grande
Copy link
Copy Markdown
Contributor Author

Didn't run openqa locally, only experiment some things, start, shutdown, kill, checking Domains Widget.

@ben-grande
Copy link
Copy Markdown
Contributor Author

PipelineRetryFailed

@codecov
Copy link
Copy Markdown

codecov Bot commented Jun 1, 2026

Codecov Report

❌ Patch coverage is 30.76923% with 54 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.30%. Comparing base (b4626b6) to head (6011ca3).
⚠️ Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
qubes/vm/qubesvm.py 31.42% 48 Missing ⚠️
qubes/vm/adminvm.py 20.00% 4 Missing ⚠️
qubes/app.py 33.33% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #819      +/-   ##
==========================================
- Coverage   70.47%   70.30%   -0.17%     
==========================================
  Files          61       61              
  Lines       14139    14189      +50     
==========================================
+ Hits         9964     9976      +12     
- Misses       4175     4213      +38     
Flag Coverage Δ
unittests 70.30% <30.76%> (-0.17%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ben-grande ben-grande force-pushed the cache-running branch 2 times, most recently from 5eff83f to aa2e823 Compare June 1, 2026 21:26
@ben-grande
Copy link
Copy Markdown
Contributor Author

PipelineRetryFailed

1 similar comment
@ben-grande
Copy link
Copy Markdown
Contributor Author

PipelineRetryFailed

@ben-grande ben-grande marked this pull request as draft June 2, 2026 07:55
@ben-grande
Copy link
Copy Markdown
Contributor Author

Marked as draft as there still is some caching issues for the _power_state.

@marmarek
Copy link
Copy Markdown
Member

marmarek commented Jun 2, 2026

If necessary (or simpler), it's IMO okay to use cache only for is_running() (that is used frequently), but still do active check for get_power_state(). It could be also a safety valve for cases when cache are not reliable (like during startup or shutdown).

@ben-grande
Copy link
Copy Markdown
Contributor Author

If necessary (or simpler), it's IMO okay to use cache only for is_running() (that is used frequently), but still do active check for get_power_state(). It could be also a safety valve for cases when cache are not reliable (like during startup or shutdown).

Experimenting with cache a bit more and it's getting better, but if I don't get it to a hundred percent, will not cache the power state.

@ben-grande
Copy link
Copy Markdown
Contributor Author

ben-grande commented Jun 2, 2026

From my tests, it seems to be working. My nemesis, OpenQA, should try me. Do you know a subset of tests that are interesting before joining this PR in the full run?

Looking at this recent test:

  • system_tests_suspend (soft-failed)
  • system_tests_usbproxy
  • system_tests_devices
  • system_tests_gui_tools
  • system_tests_guivm_gui_interactive

Line to use after this PR is not draft anymore:

openQArun TEST=system_tests_suspend,system_tests_usbproxy,system_tests_devices,system_tests_gui_tools,system_tests_guivm_gui_interactive

@marmarek
Copy link
Copy Markdown
Member

marmarek commented Jun 2, 2026

I'd include also at least one of the guivm tests

@ben-grande ben-grande force-pushed the cache-running branch 3 times, most recently from b960953 to 9538d21 Compare June 2, 2026 20:57
Useful when debugging to know what's happening. I thought of logging the
pretty name of the detail, but that got really big and I think it's out
of scope from the Qubes OS project and in scope of the python-libvirt
package.
The state changes rarely, but querying it can take a considerable
blocking time, worse when looping through the state of multiple domains.

For: QubesOS/qubes-issues#10569
For: QubesOS/qubes-issues#9902
These data only changes once when domain is running.
@ben-grande
Copy link
Copy Markdown
Contributor Author

PipelineRetryFailed

@ben-grande
Copy link
Copy Markdown
Contributor Author

Last CI failed on Fedora 43 to due to:

======================================================================
ERROR: qubes.tests.api_admin/TC_00_VMs/test_150_pool_info
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/python3.14/contextlib.py", line 85, in inner
    return func(*args, **kwds)
  File "/home/gitlab-runner/builds/QubesOS/qubes-core-admin/qubes/tests/api_admin.py", line 76, in setUp
    super().setUp()
    ~~~~~~~~~~~~~^^
  File "/home/gitlab-runner/builds/QubesOS/qubes-core-admin/qubes/tests/__init__.py", line 516, in setUp
    self.loop = asyncio.get_event_loop()
                ~~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib64/python3.14/asyncio/events.py", line 715, in get_event_loop
    raise RuntimeError('There is no current event loop in thread %r.'
                       % threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'MainThread'.

So it finally reached this repo.

@ben-grande
Copy link
Copy Markdown
Contributor Author

PipelineRetryFailed

@ben-grande ben-grande marked this pull request as ready for review June 3, 2026 08:41
@ben-grande
Copy link
Copy Markdown
Contributor Author

openQArun TEST=system_tests_suspend,system_tests_usbproxy,system_tests_devices,system_tests_gui_tools,system_tests_guivm_gui_interactive

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants