Skip to content

host: stop VMs only after verified exit - #1426

Closed
Enrique Saurez (esaurez) wants to merge 6 commits into
feature/nvx-refactorfrom
esaurez/slim-2.2.2-sl2.10-20261109
Closed

Enrique Saurez (esaurez) wants to merge 6 commits into
feature/nvx-refactorfrom
esaurez/slim-2.2.2-sl2.10-20261109

Conversation

@esaurez

@esaurez Enrique Saurez (esaurez) commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Part of #1100.

Return stop outcomes only after a verified process wait confirms exit. Process::wait returns opaque Exited proof or None; vm::stop preserves graceful/forced ordering and reports an unconfirmed exit as StillRunning. Outcome-report errors remain separate from proved exit: only a missing report is Ok(None), while unreadable or malformed reports retain their I/O causes.

Keep the four legacy lifecycle callers on wait_bool and preserve existing best-effort report policies. Update the directly affected callers, both public-API snapshots, documentation, and real-process tests. No report schema, restart-proof process-open API, background service, native runtime, or dependency change.

This is a linear change on #1421: the five original commits are preserved, followed by one coupled test-only correction. The OpenVMM pin is unchanged. Size: 101 production lines across seven production files, 29 changed paths; this includes the runtime adapter's ten production lines rather than treating them as tests.

Both Copilot and the independent local review found one inherited reattach test still asserting the old boolean wait result. The append-only correction asserts is_some() instead, retaining verified-exit confirmation without changing the original five commits.

Existing tests

The 49 modified tests/helpers below are retained and adapted to typed wait/report results without changing their assertions, deadlines, parallelism, or hardware skips. The one fixture-mechanism change copies the real POSIX executable through an owned child, keeping writable descriptors out of concurrent CLI children.

File Retained tests/helpers
nvx-rs/src/error/tests.rs every_kind_has_a_code_of_its_own_spelling
nvx-rs/src/host/platform/windows/tests.rs a_launch_whose_vmm_exited_ends_while_a_process_spawned_in_its_window_holds_its_log
nvx-rs/src/host/process/tests.rs hosts_without_pidfds_poll_and_never_signal
nvx-rs/src/host/process/tests/linux.rs a_process_runs_until_its_last_thread_exits
nvx-rs/src/host/report/tests.rs reports_are_summarized_without_free_form_fields
nvx-rs/src/host/template/clone/tests/launch.rs VmmDir::end_and_wait, VmmDir::wait_exit, VmmDir::drop, a_clone_whose_launch_failed_before_it_began_launches_no_more_until_its_release, a_launch_whose_caller_crashes_once_its_vmm_started_ends_the_vmm_as_reattach_recovers_it, a_restore_whose_caller_crashes_before_its_commit_ends_its_vmm_as_reattach_recovers_it
nvx-rs/src/host/template/clone/tests/restore.rs assert_vmm_ended
nvx-rs/src/host/template/store/leases/tests.rs VmmDir::drop, a_clone_that_crashes_after_its_spawn_keeps_its_lease_until_its_unrecorded_vmm_exits, a_clone_that_crashes_after_recording_its_vmm_keeps_its_lease_until_reattach_ends_it
nvx-rs/src/host/vm/tests.rs spawn_caller_helper, an_unrelated_process_started_in_the_library_spawn_window_gets_no_vm_claim, current_cold_spawn_boots_a_guest_accepts_shell_input_and_exits_cleanly
nvx-rs/src/host/vm/tests/reattach.rs reattaching_returns_the_held_owned_child_and_authenticated_connection (retains exit confirmation with is_some())
nvx-rs/tests/checked_handoff.rs a_prepared_launch_hands_the_vmm_its_checked_images_alone, a_vmm_keeps_the_scratch_images_claim_once_its_launcher_is_gone
nvx-rs/tests/cli.rs a_stop_under_another_identity_never_ends_a_process_that_its_record_does_not_name, stop_reports_its_timeout_and_leaves_the_vm_running, a_sandbox_whose_openvmm_exited_reports_it_once_and_starts_again
nvx-rs/tests/cli/download.rs provision_and_deprovision_need_no_artifact_root (owned-child executable-copy fixture; existing premise and assertions retained)
nvx-rs/tests/cli/lifecycle.rs check_lifecycle, recycled_process_id, lifecycle_of_a_sandbox_whose_upper_is_in_ram, stops_keep_an_outcome_report_that_the_caller_reads_whatever_its_umask
nvx-rs/tests/fake_openvmm.rs outcome_reports_of_another_runtime_are_never_read, records_that_name_another_process_never_end_it
nvx-rs/tests/store.rs kill, a_committed_launch_names_its_vm_for_another_build
nvx-rs/tests/supervisor.rs VmmDirectory::drop, check_vmm, long_stores_launch_with_a_short_working_directory, a_child_that_exits_wakes_its_waiter, a_process_that_is_not_a_child_wakes_its_waiters, a_restarted_launcher_finds_its_vmm_again_and_ends_it, a_reused_process_id_is_never_signaled, references_that_name_no_process_are_never_opened, an_unrecorded_vmm_that_serves_no_endpoint_is_never_signaled, a_vmm_in_its_orchestrators_job_ends_with_it, a_job_that_ends_its_vmm_wakes_the_waiters, a_process_that_this_user_may_not_terminate_is_waited_for_but_never_signaled
tests/runtime-adapter/tests/vm_clone.rs Early::activate, a_host_program_of_another_runtime_ends_the_clone_through_its_verified_identity, a_clone_whose_vm_exits_once_a_stop_found_it_fails_before_its_lease_is_released, a_run_whose_clone_s_vm_exits_during_its_execution_ends_with_the_clone_s_failure

New coverage exercises self-exit and report outcomes, verified forced exit after grace, and unconfirmed exit, plus missing, malformed, and unreadable reports.

Validation

  • Passed on the current head: source/submodule verification, test-layout check, test-change inventory, linear ancestry and whitespace checks.
  • Passed locally on Windows and Linux: the complete Rust feature matrices, Clippy, documentation, MSRV, public APIs, runtime-adapter configurations and native-source comparisons. The focused VM module passed 31 Windows tests and 36 Linux tests, with three existing hardware tests ignored on each.
  • Passed locally on Windows: the complete CLI suite (1,823 tests, 70 skipped), adversarial suite (84 tests, 11 skipped), four host-resolver tests, eight help checks, Ruff, both Pyright platforms, repository quality checks and all six PowerShell syntax checks.
  • Full CI, attempt 1, passed on e4af8689. Current-head Copilot feedback has no unresolved threads; the independent whole-owned review and its correction review are clean. The PR remains Draft.
  • Passed locally on Linux as the native unprivileged caller: the complete CLI suite (1,823 tests, 34 skipped), adversarial suite (84 tests, none skipped), four host-resolver tests, eight help checks, all repository quality checks, 25 Specula tests and all six PowerShell syntax checks. Earlier temporary-socket and isolated-UID environment failures were corrected without changing the product, test assertions, timeouts or parallelism.
  • Existing hardware-ignored tests remain ignored, not reported as passes. No historical-head qualification is claimed for this candidate.

Enrique Saurez and others added 5 commits November 9, 2026 14:04
Return typed exit proof from process waits and keep outcome-report read failures separate from confirmed exits. Preserve legacy lifecycle waits and best-effort report handling.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 0ad4b260-76ef-4afa-bd83-887489696103
Adapt the remaining direct process-wait assertions and test helpers without changing their deadlines or expected lifecycle behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 0ad4b260-76ef-4afa-bd83-887489696103
Record the generated Linux and Windows API surfaces. Keep exit-proof construction explicitly conditional and avoid importing error kinds that make existing documentation links redundant.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 0ad4b260-76ef-4afa-bd83-887489696103
Keep the CLI and library error inventories consistent and place process-wait and outcome-reader details in their owning module sections.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 0ad4b260-76ef-4afa-bd83-887489696103
Keep writable fixture descriptors out of concurrently forked CLI children while preserving the real copied executable, root-discovery premise and existing assertions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 0ad4b260-76ef-4afa-bd83-887489696103
@github-actions github-actions Bot added the tests-modified Changes existing tests, test data, or test wiring label Nov 9, 2026
@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

This pull request changes existing tests

It modifies or removes 49 existing test asset(s), listed below. Review each with care: a change to a test can hide a regression that the test caught. See doc/contribute.md#changes-to-existing-tests.

Test code

Change What Before After
modified helper fn end_and_wait in host::template::clone::tests::launch, impl VmmDir`` nvx-rs/src/host/template/clone/tests/launch.rs:362 nvx-rs/src/host/template/clone/tests/launch.rs:362
modified helper fn wait_exit in host::template::clone::tests::launch, impl VmmDir`` nvx-rs/src/host/template/clone/tests/launch.rs:372 nvx-rs/src/host/template/clone/tests/launch.rs:372
modified helper fn drop in host::template::clone::tests::launch, impl Drop for VmmDir`` nvx-rs/src/host/template/clone/tests/launch.rs:412 nvx-rs/src/host/template/clone/tests/launch.rs:412
modified helper fn assert_vmm_ended in host::template::clone::tests::restore nvx-rs/src/host/template/clone/tests/restore.rs:422 nvx-rs/src/host/template/clone/tests/restore.rs:422
modified helper fn drop in host::template::store::leases::tests, impl Drop for VmmDir`` nvx-rs/src/host/template/store/leases/tests.rs:2459 nvx-rs/src/host/template/store/leases/tests.rs:2459
modified helper fn check_lifecycle in cli/lifecycle (tests) nvx-rs/tests/cli/lifecycle.rs:550 nvx-rs/tests/cli/lifecycle.rs:550
modified helper fn recycled_process_id in cli/lifecycle (tests) nvx-rs/tests/cli/lifecycle.rs:1425 nvx-rs/tests/cli/lifecycle.rs:1428
modified helper fn kill in store (tests) nvx-rs/tests/store.rs:176 nvx-rs/tests/store.rs:176
modified helper fn drop in supervisor (tests), impl Drop for VmmDirectory`` nvx-rs/tests/supervisor.rs:449 nvx-rs/tests/supervisor.rs:449
modified helper fn check_vmm in supervisor (tests) nvx-rs/tests/supervisor.rs:1226 nvx-rs/tests/supervisor.rs:1234
modified helper fn activate in vm_clone (tests), impl CloneControl for Early`` tests/runtime-adapter/tests/vm_clone.rs:2314 tests/runtime-adapter/tests/vm_clone.rs:2314
modified test fn every_kind_has_a_code_of_its_own_spelling in error::tests nvx-rs/src/error/tests.rs:50 nvx-rs/src/error/tests.rs:50
modified test fn a_launch_whose_vmm_exited_ends_while_a_process_spawned_in_its_window_holds_its_log in host::platform::windows::tests nvx-rs/src/host/platform/windows/tests.rs:1049 nvx-rs/src/host/platform/windows/tests.rs:1060
modified test fn hosts_without_pidfds_poll_and_never_signal in host::process::tests nvx-rs/src/host/process/tests.rs:259 nvx-rs/src/host/process/tests.rs:259
modified test fn a_process_runs_until_its_last_thread_exits in host::process::tests::linux nvx-rs/src/host/process/tests/linux.rs:112 nvx-rs/src/host/process/tests/linux.rs:112
modified test fn reports_are_summarized_without_free_form_fields in host::report::tests nvx-rs/src/host/report/tests.rs:4 nvx-rs/src/host/report/tests.rs:4
modified test fn a_clone_whose_launch_failed_before_it_began_launches_no_more_until_its_release in host::template::clone::tests::launch nvx-rs/src/host/template/clone/tests/launch.rs:1547 nvx-rs/src/host/template/clone/tests/launch.rs:1549
modified test fn a_launch_whose_caller_crashes_once_its_vmm_started_ends_the_vmm_as_reattach_recovers_it in host::template::clone::tests::launch nvx-rs/src/host/template/clone/tests/launch.rs:2299 nvx-rs/src/host/template/clone/tests/launch.rs:2301
modified test fn a_restore_whose_caller_crashes_before_its_commit_ends_its_vmm_as_reattach_recovers_it in host::template::clone::tests::launch nvx-rs/src/host/template/clone/tests/launch.rs:2332 nvx-rs/src/host/template/clone/tests/launch.rs:2334
modified test fn a_clone_that_crashes_after_its_spawn_keeps_its_lease_until_its_unrecorded_vmm_exits in host::template::store::leases::tests nvx-rs/src/host/template/store/leases/tests.rs:2623 nvx-rs/src/host/template/store/leases/tests.rs:2625
modified test fn a_clone_that_crashes_after_recording_its_vmm_keeps_its_lease_until_reattach_ends_it in host::template::store::leases::tests nvx-rs/src/host/template/store/leases/tests.rs:2654 nvx-rs/src/host/template/store/leases/tests.rs:2656
modified test fn spawn_caller_helper in host::vm::tests nvx-rs/src/host/vm/tests.rs:1088 nvx-rs/src/host/vm/tests.rs:1088
modified test fn an_unrelated_process_started_in_the_library_spawn_window_gets_no_vm_claim in host::vm::tests nvx-rs/src/host/vm/tests.rs:1139 nvx-rs/src/host/vm/tests.rs:1140
modified test fn current_cold_spawn_boots_a_guest_accepts_shell_input_and_exits_cleanly in host::vm::tests nvx-rs/src/host/vm/tests.rs:1195 nvx-rs/src/host/vm/tests.rs:1197
modified test fn reattaching_returns_the_held_owned_child_and_authenticated_connection in host::vm::tests::reattach nvx-rs/src/host/vm/tests/reattach.rs:229 nvx-rs/src/host/vm/tests/reattach.rs:229
modified test fn a_prepared_launch_hands_the_vmm_its_checked_images_alone in checked_handoff (tests) nvx-rs/tests/checked_handoff.rs:487 nvx-rs/tests/checked_handoff.rs:487
modified test fn a_vmm_keeps_the_scratch_images_claim_once_its_launcher_is_gone in checked_handoff (tests) nvx-rs/tests/checked_handoff.rs:717 nvx-rs/tests/checked_handoff.rs:718
modified test fn a_stop_under_another_identity_never_ends_a_process_that_its_record_does_not_name in cli (tests) nvx-rs/tests/cli.rs:978 nvx-rs/tests/cli.rs:978
modified test fn stop_reports_its_timeout_and_leaves_the_vm_running in cli (tests) nvx-rs/tests/cli.rs:1139 nvx-rs/tests/cli.rs:1141
modified test fn a_sandbox_whose_openvmm_exited_reports_it_once_and_starts_again in cli (tests) nvx-rs/tests/cli.rs:1186 nvx-rs/tests/cli.rs:1190
modified test fn provision_and_deprovision_need_no_artifact_root in cli/download (tests) nvx-rs/tests/cli/download.rs:165 nvx-rs/tests/cli/download.rs:165
modified test fn lifecycle_of_a_sandbox_whose_upper_is_in_ram in cli/lifecycle (tests) nvx-rs/tests/cli/lifecycle.rs:636 nvx-rs/tests/cli/lifecycle.rs:637
modified test fn stops_keep_an_outcome_report_that_the_caller_reads_whatever_its_umask in cli/lifecycle (tests) nvx-rs/tests/cli/lifecycle.rs:718 nvx-rs/tests/cli/lifecycle.rs:720
modified test fn outcome_reports_of_another_runtime_are_never_read in fake_openvmm (tests) nvx-rs/tests/fake_openvmm.rs:458 nvx-rs/tests/fake_openvmm.rs:458
modified test fn records_that_name_another_process_never_end_it in fake_openvmm (tests) nvx-rs/tests/fake_openvmm.rs:498 nvx-rs/tests/fake_openvmm.rs:498
modified test fn a_committed_launch_names_its_vm_for_another_build in store (tests) nvx-rs/tests/store.rs:497 nvx-rs/tests/store.rs:499
modified test fn long_stores_launch_with_a_short_working_directory in supervisor (tests) nvx-rs/tests/supervisor.rs:596 nvx-rs/tests/supervisor.rs:598
modified test fn a_child_that_exits_wakes_its_waiter in supervisor (tests) nvx-rs/tests/supervisor.rs:664 nvx-rs/tests/supervisor.rs:672
modified test fn a_process_that_is_not_a_child_wakes_its_waiters in supervisor (tests) nvx-rs/tests/supervisor.rs:691 nvx-rs/tests/supervisor.rs:699
modified test fn a_restarted_launcher_finds_its_vmm_again_and_ends_it in supervisor (tests) nvx-rs/tests/supervisor.rs:720 nvx-rs/tests/supervisor.rs:728
modified test fn a_reused_process_id_is_never_signaled in supervisor (tests) nvx-rs/tests/supervisor.rs:751 nvx-rs/tests/supervisor.rs:759
modified test fn references_that_name_no_process_are_never_opened in supervisor (tests) nvx-rs/tests/supervisor.rs:791 nvx-rs/tests/supervisor.rs:799
modified test fn an_unrecorded_vmm_that_serves_no_endpoint_is_never_signaled in supervisor (tests) nvx-rs/tests/supervisor.rs:851 nvx-rs/tests/supervisor.rs:859
modified test fn a_vmm_in_its_orchestrators_job_ends_with_it in supervisor (tests) nvx-rs/tests/supervisor.rs:1483 nvx-rs/tests/supervisor.rs:1491
modified test fn a_job_that_ends_its_vmm_wakes_the_waiters in supervisor (tests) nvx-rs/tests/supervisor.rs:1510 nvx-rs/tests/supervisor.rs:1518
modified test fn a_process_that_this_user_may_not_terminate_is_waited_for_but_never_signaled in supervisor (tests) nvx-rs/tests/supervisor.rs:1566 nvx-rs/tests/supervisor.rs:1574
modified test fn a_host_program_of_another_runtime_ends_the_clone_through_its_verified_identity in vm_clone (tests) tests/runtime-adapter/tests/vm_clone.rs:3882 tests/runtime-adapter/tests/vm_clone.rs:3882
modified test fn a_clone_whose_vm_exits_once_a_stop_found_it_fails_before_its_lease_is_released in vm_clone (tests) tests/runtime-adapter/tests/vm_clone.rs:3930 tests/runtime-adapter/tests/vm_clone.rs:3932
modified test fn a_run_whose_clone_s_vm_exits_during_its_execution_ends_with_the_clone_s_failure in vm_clone (tests) tests/runtime-adapter/tests/vm_clone.rs:5320 tests/runtime-adapter/tests/vm_clone.rs:5324

Relocated unchanged: 0; added: 12. Compared the merge base c7e971607a0e with e4af86899218.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The changed wait return type leaves a test passing Option<Exited> to assert!, preventing host-enabled library tests from compiling on both Linux and Windows.

1 open issue
What changed in this PR

This PR advances #1100 by requiring verified process exit before the host VM stop API returns success, while preserving legacy lifecycle behavior.

Changes:

  • Introduces opaque exit proof, vm::stop, and StillRunning for unconfirmed exits.
  • Separates missing outcome reports from read or parse errors without hiding confirmed exit.
  • Adapts callers and tests, adds real-process coverage, and updates documentation and public API snapshots.
File Description
tests/​runtime-adapter/​tests/​vm_clone.rs Adapts clone tests to typed exit results.
tests/​runtime-adapter/​src/​bin/​runtime-adapter-host.rs Handles typed waits and report errors.
nvx-rs/​tests/​supervisor.rs Updates supervisor wait assertions and cleanup.
nvx-rs/​tests/​store.rs Adapts store process-wait helpers.
nvx-rs/​tests/​fake_openvmm.rs Updates process identity and exit assertions.
nvx-rs/​tests/​cli/​lifecycle.rs Adapts lifecycle exit checks.
nvx-rs/​tests/​cli/​download.rs Copies the Unix executable through a child process.
nvx-rs/​tests/​cli.rs Updates CLI process-wait assertions.
nvx-rs/​tests/​checked_handoff.rs Adapts handoff exit checks.
nvx-rs/​src/​host/​vm/​tests.rs Adds real-process stop and report coverage.
nvx-rs/​src/​host/​vm.rs Adds verified stop outcomes and bounded termination.
nvx-rs/​src/​host/​template/​store/​leases/​tests.rs Updates lease cleanup and exit checks.
nvx-rs/​src/​host/​template/​clone/​tests/​restore.rs Adapts restore exit confirmation.
nvx-rs/​src/​host/​template/​clone/​tests/​launch.rs Updates launch wait and report assertions.
nvx-rs/​src/​host/​template/​clone/​records.rs Preserves best-effort report reads.
nvx-rs/​src/​host/​report/​tests.rs Tests missing, malformed, and unreadable reports.
nvx-rs/​src/​host/​report.rs Returns distinct report absence and error results.
nvx-rs/​src/​host/​process/​tests/​linux.rs Updates last-thread exit assertions.
nvx-rs/​src/​host/​process/​tests.rs Adapts polling fallback assertions.
nvx-rs/​src/​host/​process.rs Introduces Exited and typed wait results.
nvx-rs/​src/​host/​platform/​windows/​tests.rs Adds a wait-only handle helper and updates assertions.
nvx-rs/​src/​host/​instance/​locked.rs Retains legacy best-effort report handling.
nvx-rs/​src/​error/​tests.rs Includes StillRunning in error-code checks.
nvx-rs/​src/​error.rs Adds StillRunning and its code mappings.
nvx-rs/​README.md Documents stop, exit proof, and report semantics.
nvx-rs/​public-api/​windows.txt Records new Windows public API symbols.
nvx-rs/​public-api/​linux.txt Records new Linux public API symbols.
doc/​usage.md Documents the still-running error code.

Review effort: Balanced

Comment thread nvx-rs/src/host/process.rs Resolved
Adapt the parent-provided reattach assertion to the opaque exit-proof result without weakening its verified-exit requirement.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 6a8293af-330b-47af-99ea-b48bfe7ebbbd

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The review identifies no active issues with the VM stop API and exit-confirmation changes.

1 open issue

Review effort: Balanced

Base automatically changed from esaurez/slim-3.3.2-sl7.6b-20261109 to feature/nvx-refactor November 10, 2026 04:40
@esaurez
Enrique Saurez (esaurez) deleted the esaurez/slim-2.2.2-sl2.10-20261109 branch November 13, 2026 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feat-nvx-refactor tests-modified Changes existing tests, test data, or test wiring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants