Repository navigation
tests: wait for the fake OpenVMM's whole command line (flaky state read) - #1511
Merged
Pedro Henrique Penna (ppenna) merged 1 commit intoNov 12, 2026
Merged
Pedro Henrique Penna (ppenna) merged 1 commit into
Pedro Henrique Penna (ppenna) merged 1 commit into
Conversation
Contributor
This pull request changes existing testsIt modifies or removes 1 existing test asset(s), listed below. Review each with care: a change to a test can hide a regression that the test caught. See doc/contribute.md#changes-to-existing-tests. Test code
Relocated unchanged: 0; added: 0. Compared the merge base |
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues are recorded for the test-only change to wait for complete CLI argument JSON.
No open issues 🎉
What changed in this PR
This test-only change fixes a flaky CLI state read by waiting for the fake OpenVMM’s complete JSON command-line record rather than merely waiting for the file to exist.
Changes:
- Uses
wait_for_jsonto obtain the arguments, preserving existing assertions and their order. - Clarifies the
wait_fordocumentation to distinguish file existence from readable JSON content. - Leaves production code, sleeps, and timeouts unchanged.
| File | Description |
|---|---|
nvx-rs/tests/cli.rs |
Waits for complete argument JSON in the stable caller-path test and clarifies the wait helper’s documentation. |
Review effort: Balanced
The Windows job of CI run 43131964028, on #1496, which doesn't touch this code, failed in the CLI test start_opens_the_stable_caller_paths_that_provision_checked: tests\cli\harness.rs:538:33: ...\build\fake-openvmm-state.json: EOF while parsing a value at line 1 column 0 The test starts a sandbox in the background, waits until the fake OpenVMM's record of its command line, fake-openvmm-state.json, exists, and then parses it. The fake writes the record with fs::write, which creates the file before it writes its content, so a test that reads it in between finds it empty. #1460 added the parse right after that wait, which until then only told the test that OpenVMM had started. wait_for_json, which the same file already uses for the fake's record of its block devices, waits for exactly this. The test now takes the arguments from wait_for_json, which returns once the record holds a whole JSON document, and checks the same things in the same order: the record of the block devices still doesn't exist then, since the fake opens its images 1.5 s after it starts. wait_for's comment now sends a test that reads a file's content to wait_for_json. The fake keeps writing its records with fs::write. Its other readers run once the command that launched it has returned, or once it has exited, so after it wrote them. A temporary file and a rename would spare future readers the wait, but on Windows a rename fails while another process, such as a scanner, holds the record that it replaces open without sharing its deletion, and the fake ignores its write errors, so a second launch of the same sandbox could leave the first one's record in place. Evidence, at 14ca267, with a fake that waits 300 ms between creating the record and writing it: the base test failed with the CI symptom on a Windows 11 workstation and on nvx-kvm-test (Linux, KVM), and this change passed on both. Without that wait, this change passed 50 of 50 runs of the test and 3 of 3 runs of the CLI suite on each. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/slim-cli-state-race
branch
from
November 12, 2026 03:35
1c8c07e to
08ba447
Compare
Pedro Henrique Penna (ppenna)
marked this pull request as ready for review
November 12, 2026 03:35
Pedro Henrique Penna (ppenna)
merged commit Nov 12, 2026
ccea875
into
feature/nvx-refactor
83 of 84 checks passed
This was referenced Nov 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #1076. Fixes a flaky test of the CLI's suite (
cli-state-race) that #1460 (SL3.5b) introduced.What and why
Symptom. Run 43131964028, attempt 1, of #1496, which doesn't touch this code, failed in the job
nvx-rs / windows-latest, step "Check nvx-rs":The test passes in other runs, and 24 parallel local runs didn't reproduce it.
Root cause. The test starts
nvx-rs sandbox startin the background and, to look at the VM while it starts, waits withwait_foruntil the fake OpenVMM's record of its command line,build/fake-openvmm-state.json, exists. It then parses the record withRoot::openvmm_arguments, which panics on a parse error. The fake writes the record first thing in itsmain, withfs::write, which creates the file and then writes its content, so a test that reads it in between finds it empty.wait_for_json, besidewait_forin the same file, says exactly this ("the fake OpenVMM writes its dumps withfs::write, which creates a file before it writes the file's content"), and the next test already waits with it for the fake's record of its block devices.Origin. #1460 (SL3.5b), in
597ec1bfbf1("Test the approved stable caller-path ownership contract"), turnedstart_hands_openvmm_the_images_that_it_checkedinto this test and added the parse right after thewait_for, which #752 had added only to know that OpenVMM had started.Fix. Only the test changes:
wait_for_json, which returns once the record holds a whole JSON document. The record is a JSON array, so no prefix of it parses.fake_block_delay_ms=1500) after it starts, and the record's arguments still name the stable caller paths.wait_for's comment now sends a test that reads a file's content towait_for_json.No sleep and no timeout changes.
Why the fake keeps
fs::write. No other reader of the fake's records waits for one to exist and then parses it (see below), so this reader's fix covers them all. Writing the records through a temporary file and a rename would spare future readers the wait, but on Windows a rename fails while another process, such as a scanner, holds the file that it replaces open without sharing its deletion, asrename_away_and_backinchecked_handoff.rsallows for. The fake ignores its write errors, so the second start of a sandbox that a lifecycle test starts twice could then leave the first start's record in place.Other readers of the same kind. In
nvx-rs/tests/and the fake OpenVMM:fake-openvmm-TOKEN.json,-blocks.json,-maps.json):tests/cli/{lifecycle,formats,sandbox_run,one_shot,openvmm}.rs,tests/cli.rsandtests/template_capture.rsread them once the command that launched the fake has returned, which a start does only after the guest's handshake and mapping table that the fake serves after it wrote them, or once the fake has exited, astests/fake_openvmm_behavior.rsdoes.an_interrupted_start_keeps_its_claim_until_recovery_confirms_exitalready waits withwait_for_json. None waits for a record to exist and then parses it, so this PR fixes the only one.touchmarkers oftests/cli/exec_lock.rs, of theinterruptstests and ofa_run_passes_openvmm_no_handle_of_its_caller, andfake-openvmm-.jsonininterrupting_a_cold_boot_ends_it_without_guidance), or wait for files written through a temporary file and a rename (the fake's foreground gate'sphase.json, the fixtures' reports inchecked_handoff.rsandsupervisor.rs, the CLI'slaunch.json).tests/template_capture.rs,crash_adapter: it polls until the fake's capture markeractiveexists and the journal records the VMM, kills the adapter, then parses the process ID in the marker withactive_pid. The fake creates the marker withFile::create_newand then writes the ID (Active::begininsrc/testing/fake_openvmm.rs), so the marker is empty for a moment.nvx-rs/tests/, in the Linux guest unit tests ofsrc/guest/managed_agent/dispatch/tests.rs:a_reset_does_not_wait_for_output_from_an_escaped_direct_descendantanddirect_endings_do_not_wait_for_an_escaped_descendant_to_close_outputwait untilescapedexists, and later parse the process ID that a shell wrote there withecho $$ > "$0". A third test in that file already waits for the file to be non-empty.Size
0 core lines in 0 counted files: the change is in
nvx-rs/tests/cli.rsalone, with 5 lines added and 3 removed.Changes to existing tests
python scripts\check_test_changes.py --base origin/feature/nvx-refactorlists 1 change to an existing test, hencetests-modified:start_opens_the_stable_caller_paths_that_provision_checkedwaits for the fake OpenVMM's whole record of its command line instead of its existence, and parses the arguments from it. It keeps every assertion, in the same order.doc/acceptance-case-matrix.mdlists no test ofnvx-rs/tests/cli.rs, so no row changes.Not in this PR
The fake OpenVMM's writes, and the two readers listed under "Not fixed here".
Validation
Reproduction. A temporary, uncommitted edit of the fake widens the window between the record's creation and its content to 300 ms:
14ca2671205harness.rs:538:33,EOF while parsing a value at line 1 column 0)Checks, in
nvx-rs/withNVX_BUNDLE=skip, without the widened window. On Windows they ran at3ab9d37df43, on14ca2671205; on nvx-kvm-test, in a worktree of14ca2671205with this patch applied, the same tree. The head then moved only by a rebase onto012123bf9ac, whose three commits don't touchnvx-rs/tests/cli.rs,nvx-rs/tests/cli/or the fake, with no change to the patch (git range-diff), and the test passed again there on Windows. The head,1c8c07e5990, differs from that rebase only by its commit message.start_opens_the_stable_caller_paths_that_provision_checked, 50 runs in a rowclisuite (cargo test --locked --all-features --test cli), 3 runscargo fmt --all --checkcargo clippy --locked --all-targets --all-features -- -D warningscargo clippy --locked --lib --no-default-features -- -D warningscargo +1.89 check --locked --all-targets --all-featurespython scripts\check_test_layout.pypython scripts\check_test_changes.py --base origin/feature/nvx-refactorLeft to CI: the full Linux and Windows
nvx-rssuites at this head, the microVM scenarios, and the gates that a change to one integration test can't affect (artifact bundling, rustdoc, the runtime adapter,nvx-cliand Markdown links).