Skip to content

host: restore a VM from a snapshot and a VmConfig (SL2.16) - #1470

Merged
Pedro Henrique Penna (ppenna) merged 2 commits into
feature/nvx-refactorfrom
esaurez/sl216-production-0f0b8f59-20261110
Nov 11, 2026
Merged

Pedro Henrique Penna (ppenna) merged 2 commits into
feature/nvx-refactorfrom
esaurez/sl216-production-0f0b8f59-20261110

Conversation

@esaurez

@esaurez Enrique Saurez (esaurez) commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Part of #1103 (SL2.16, "A VM restores from a snapshot and a VmConfig"). Parent #1081, umbrella #1076.

Merging this PR completes phase 2 of #1076: once SL2.16 lands, every PR of phases 1 and 2 has merged into feature/nvx-refactor, so its merge is the API-ready commit that the PRs of phases 4 and 5 wait for.

What and why

host::vm::create refused every snapshot, which left callers on the store-coupled clone launch. With from: Some(snapshot) it now prepares a restore from a Snapshot and a VmConfig:

  • Refusals and eligibility. It refuses Boot, and calls Snapshot::accepts with the host's real time (HostTime::now) before it creates any output.
  • Restore view. It hard-links only the snapshot's fixed OpenVMM inventory (manifest.bin, memory.bin, state.bin) into a new private <vm_dir>/openvmm/. A view on another filesystem fails with the native error (EXDEV, ERROR_NOT_SAME_DEVICE); nothing is copied, and there is no snapshot-parent fallback.
  • Restore arguments follow the clone launch's rules (host/template/clone/launch.rs): the devices at the snapshot's processor capacity, --restore-network-policy with the restore's own policy in place of --net and the guest's address, --restore-absent-scratch for an upper in RAM, --restore-processors only when the snapshot records its boot processors, --restore-memory, the console attachment that the snapshot allows (a snapshot captured Disconnected refuses a listener), the snapshot's backend and CPU profile, and the launch options. No kernel, initramfs, command line or --memory.
  • NIC and activation record. Prepared::nic is the snapshot's NIC, and activation_record takes the restore policy's DNS, publication and proxy.
  • Ready listener. create binds the restore-ready listener before any spawn and moves that same listener, never reopened nor in the child's inheritance allowlist, through VmmCommand::ready to an external launcher, or through Launched::ready from vm::spawn.
  • Rollback. A failed creation removes only what it created (the links, the view, the listener's socket and the log), keeps every pre-existing entry and caller input, and reports a cleanup failure with the original error.
  • Docs. create's rustdoc and the README state the caller-owned Linux per-snapshot lock (held from create, including adding the links, through the peer-verified restore-ready event or the VM's verified exit, and around the view's removal; NVX takes none and Windows needs none) and why no durable in-flight record is needed: OpenVMM checks the snapshot's files only before its one restore-ready connection, which it does not retry, so a caller that dies before it leaves a restore that cannot complete.
  • Example. examples/external-launch.rs gains a restore: given a snapshot directory and an EROFS image, it takes the snapshot's key.bin lock on Linux before opening the snapshot, restores through vm::create(Some) and its own launcher, verifies the restore-ready event with ReadyListener::receive, then stops the VM, confirms its exit and removes the view. It restores the snapshots that nvx-rs snapshot capture produces (disconnected boot console).

Public API and a Scope extension

The public-API snapshots change by three entries on each platform:

 nvx-rs/public-api/linux.txt and nvx-rs/public-api/windows.txt
 method nvx::host::restore::ReadyListener::into_std
+method nvx::host::restore::ReadyListener::receive
 field nvx::host::vm::Launched::process
+field nvx::host::vm::Launched::ready
 field nvx::host::vm::VmmCommand::program
+field nvx::host::vm::VmmCommand::ready

Launched::ready and VmmCommand::ready are what the Scope names. Making ReadyListener::receive public extends the Scope: #1076's design has callers drive restores themselves, with their own launchers, and the Linux lock rule that create documents needs such a caller to verify the restore-ready peer before it releases the lock. Without receive, an external launcher has only into_std() and must reimplement the SO_PEERPIDFD and named-pipe client checks; with it, the example verifies the peer through the library. receive was crate-private for SL4.3's blocking driver, which can still use it. Nothing else becomes public.

The example requires a snapshot captured with a disconnected boot console, as nvx-rs snapshot capture captures one, rather than the listener that the handed-off version required: no NVX tool captures with a listener, and Snapshot::accepts refuses a console attachment that the capture did not record, so the example now restores the snapshots that NVX's own tooling produces.

Commits

  1. 828725de config: share console and runtime option rendering: a preparatory refactoring. It moves the console attachment checks and the runtime options into console_backend and runtime_arguments, which the restore reuses. A cold boot's arguments do not change (the config tests compare them whole).
  2. b37d0fa3 host: restore a VM from a snapshot and a VmConfig (SL2.16): the restore, its tests, docs and example.

History

This PR was handed off by Enrique Saurez (@esaurez) (checkpoint on #1103). Its earlier head a3130238 sat on #1465's input-join branch. The inputs that it composed have since merged into feature/nvx-refactor (#1434, #1437, #1455, #1457, #1297), so the branch now holds only SL2.16's own work, replayed onto feature/nvx-refactor 5947e4e8 and squashed into the two commits above with their authorship; the corrections of this qualification are folded into the second. #1465's own commit (a formatting fix of the composed imports) is not needed. The base is now feature/nvx-refactor.

Size

Measured against feature/nvx-refactor: 357 core lines (estimate 420, stop 600) in 5 counted files: host/vm.rs 139, examples/external-launch.rs 103, host/config/mod.rs 93, host/snapshot.rs 20, host/restore.rs 2. host/mod.rs changes only its rustdoc (0 core lines).

Not in this PR

Owning or acquiring the per-snapshot lock; durable in-flight restore records; copying snapshot files, cross-filesystem restores or a snapshot-parent fallback; NIC re-addressing or a console listener absent from the capture; driving activation to Ready, retries or cancellation (SL2.18a's RestoreMachine, SL4.3's blocking driver); tiers other than Fresh.

Validation

Artifacts: CI run 42956031193 of feature/nvx-refactor 5947e4e8 (guest kernel, Alpine initramfs and Ubuntu EROFS layer, and OpenVMM 14dc4116 for linux-gnu, linux-musl and windows-msvc, each checked against its provenance). This change touches neither the guest nor OpenVMM. nvx-rs and the example were built from this branch on each host. The head b37d0fa3 has the same tree as bf30d9ea, on which the Windows checks ran. The runtime checks and the Linux rerun ran on a939ec07, which differs from it only in README paragraphs.

Host (type) Backend Check Result
nvx-kvm-test (VM) KVM nvx-rs snapshot capture, then external-launch restoring that snapshot through vm::create(Some) and its own launcher to the peer-verified restore-ready event, then stop, confirmed exit and view removal passed; the snapshot's link counts return to 1 and no OpenVMM is left
nvx-mshv-test (VM) MSHV same passed
nvx-whp-test (VM) WHP same passed
all three KVM, MSHV, WHP external-launch cold launch (the refactored cold renderer) passed
all three KVM, MSHV, WHP nvx.py test-microvm --scenario l3-l4-egress-policy --scenario filesystem-comma-paths --scenario run-console-restore (nvx-rs run through vm::create) passed, 3/3 on each
nvx-kvm-test Linux ignored snapshot_creation_refuses_a_true_cross_filesystem_view_without_copying with NVX_VM_TEST_OTHER_FILESYSTEM (tmpfs to ext4) passed, EXDEV
nvx-whp-test Windows same (C: to F:) passed, ERROR_NOT_SAME_DEVICE

Gates, with the commands of this branch's check-quality, check-nvx-rs and validate-nvx actions:

  • Linux (nvx-kvm-test): on 7c09bca1, test layout, Markdown links, test changes, rustfmt, the five warning-denied Clippy configurations, default and no-default tests, bundling, rustdoc with warnings denied, the public API, MSRV 1.89, the macOS check, the guest-binary build and checks, and the runtime adapter's protocol, host and guest configurations passed. cargo test --all-features and the guest-binary tests failed the same 28 guest:: tests (a process-group kill returns EINVAL on that host) that fail at the base 5947e4e8 on the same host, so they are environmental there; CI passed them at the base. On a939ec07, the gates that the later changes affect passed again: test layout, links, test changes, rustfmt, Clippy, the host:: tests (1125), rustdoc, the public API, MSRV and the macOS check.
  • Windows (nvx-whp-test, bf30d9ea): cargo test --all-features and cargo test passed (1440 and 1219 library tests, and every integration test).
  • Windows (local workstation, bf30d9ea): test layout, Markdown links, test changes, rustfmt, the five Clippy configurations, no-default build and tests, bundling, rustdoc, MSRV 1.89, and the runtime adapter's protocol and host configurations passed. Under the workstation's heavy parallel load, cargo test --all-features and cargo test each failed two timing-sensitive tests (children_inherit_only_their_standard_handles, and a_capture_ack_and_successful_output_without_timely_exit_are_stopped_but_never_finished or the_restore_to_ready_deadline_runs_from_the_launch_through_every_step), which pass alone there and in the full runs on nvx-whp-test. Of the validate-nvx CLI suite, compileall and the modules that read the test-layout allowlist, the test changes, the public API and the links (test_test_layout, test_test_changes, test_public_api, test_markdown_links: 178 tests) passed; the full module list did not finish within 70 minutes on the overloaded workstation, so it is left to CI.
  • After folding the fixups (b37d0fa3, the same tree as bf30d9ea): rustfmt, Clippy (--lib --no-default-features and --all-targets --all-features) and the test layout passed again, and rebase_check.py verify found both commits signed on 5947e4e8.

Left to CI: the full microVM and runtime-adapter suites on KVM, MSHV and WHP, and the benchmarks.

Changes to existing tests (tests-modified)

python3 scripts/check_test_changes.py --base origin/feature/nvx-refactor lists 9 changes, and 14 tests are added:

  • RestoreFixture (host::snapshot::tests) becomes pub(in crate::host), with current, retime, all_processors_online and console_listener, so that the VM tests restore real fixtures at the host's real time. Its existing uses are unchanged.
  • cold_creation_is_inspectable_without_a_process_listener_or_capability_file and spawn_returns_the_same_outputs_and_capability_with_a_verified_process also assert that a cold launch has no ready listener.
  • a_missing_boot_and_a_snapshot_are_refused_before_any_output_is_created is renamed a_missing_boot_and_boot_with_a_snapshot_are_refused_before_any_output_is_created, with the same body: a snapshot is no longer refused as such, so its case now checks the refusal of Boot with a snapshot (the same Unsupported kind and "Snapshot" fragment, and no output).
  • a_library_vm_outlives_the_caller_that_spawned_it_without_retaining_its_output_pipe initializes the new ready: None field of its Launched literal.
  • The test hooks (#[cfg_attr(not(test), expect(dead_code, ...))]) of ReadyListener::bind, endpoint and receive and of Snapshot::dir are removed, since production code uses them now; scripts/check_test_layout.py --tighten removed their two allowlist entries.

The clone path's launch, retry and failure tests are kept unchanged: this PR deletes nothing of the clone path.

@esaurez Enrique Saurez (esaurez) added tests-modified Changes existing tests, test data, or test wiring feat-nvx-refactor labels Nov 11, 2026
@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

This pull request changes existing tests

It modifies or removes 9 existing test asset(s), listed below. Review each with care: a change to a test can hide a regression that the test caught. See doc/contribute.md#changes-to-existing-tests.

Test code

Change What Before After
modified helper struct RestoreFixture in host::snapshot::tests nvx-rs/src/host/snapshot/tests.rs:1431 nvx-rs/src/host/snapshot/tests.rs:1431
modified test fn cold_creation_is_inspectable_without_a_process_listener_or_capability_file in host::vm::tests nvx-rs/src/host/vm/tests.rs:817 nvx-rs/src/host/vm/tests.rs:817
modified test fn a_missing_boot_and_a_snapshot_are_refused_before_any_output_is_created in host::vm::tests (renamed to a_missing_boot_and_boot_with_a_snapshot_are_refused_before_any_output_is_created) nvx-rs/src/host/vm/tests.rs:854 nvx-rs/src/host/vm/tests.rs:855
modified test fn spawn_returns_the_same_outputs_and_capability_with_a_verified_process in host::vm::tests nvx-rs/src/host/vm/tests.rs:1354 nvx-rs/src/host/vm/tests.rs:1781
modified test fn a_library_vm_outlives_the_caller_that_spawned_it_without_retaining_its_output_pipe in host::vm::tests nvx-rs/src/host/vm/tests.rs:1548 nvx-rs/src/host/vm/tests.rs:1976
removed hook the test hooks of fn bind in host::restore (impl ReadyListener) nvx-rs/src/host/restore.rs:528
removed hook the test hooks of fn endpoint in host::restore (impl ReadyListener) nvx-rs/src/host/restore.rs:547
removed hook the test hooks of fn receive in host::restore (impl ReadyListener) nvx-rs/src/host/restore.rs:568
removed hook the test hooks of struct Snapshot in host::snapshot nvx-rs/src/host/snapshot.rs:107

Relocated unchanged: 0; added: 14. Compared the merge base 5947e4e834ef with b37d0fa3c7d3.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The new restore path and external-launch example lack VM runtime validation on the pinned OpenVMM, leaving Linux hard-link and dead-controller safety checks unresolved and requiring human review.

No open issues 🎉

What changed in this PR

Implements SL2.16 of #1103: caller-owned snapshot restore preparation through host::vm::create, building on the composed launch and capture APIs without requiring a sandbox store.

Changes:

  • Adds strict restore eligibility checks, hard-linked snapshot views, rollback, and restore arguments using captured NIC addressing with current network policy.
  • Transfers the prebound restore-ready listener through both launch paths and extends the external-launch example.
  • Shares VM directory ownership below snapshot and launch modules, updates tests and documentation, and includes named-snapshot CLI groundwork.

The PR remains a draft handoff; new-path VM runtime validation and full CI/dependency validation are outstanding.

File Description
scripts/​test-layout-allowlist.json Updates allowances for removed hooks and CLI groundwork.
nvx-rs/​src/​host/​vm/​tests.rs Adds restore eligibility, rollback, arguments, and listener-ownership coverage.
nvx-rs/​src/​host/​vm.rs Implements restore preparation and listener handoff.
nvx-rs/​src/​host/​vm_directory/​tests.rs Relocates directory privacy and endpoint tests.
nvx-rs/​src/​host/​vm_directory.rs Extracts shared VM directory and output ownership.
nvx-rs/​src/​host/​snapshot/​tests.rs Adds reusable restore fixtures and hard-link tests.
nvx-rs/​src/​host/​snapshot/​capture.rs Uses the shared directory module to avoid a launch dependency.
nvx-rs/​src/​host/​snapshot.rs Adds fixed-inventory restore-view hard linking.
nvx-rs/​src/​host/​restore.rs Uses shared directory ownership and exposes ready-event receipt.
nvx-rs/​src/​host/​mod.rs Registers the private directory module.
nvx-rs/​src/​host/​config/​mod.rs Shares launch rendering and adds restore-device arguments.
nvx-rs/​src/​host/​cli/​store/​mod.rs Exposes locked state internally for snapshot storage.
nvx-rs/​src/​host/​cli/​snapshots/​tests.rs Tests snapshot naming, publication, locking, and privacy.
nvx-rs/​src/​host/​cli/​snapshots.rs Adds immutable named-snapshot storage groundwork.
nvx-rs/​src/​host/​cli/​sandbox/​state/​tests.rs Tests source-snapshot serialization and validation.
nvx-rs/​src/​host/​cli/​sandbox/​state.rs Records optional source-snapshot names and shares utilities.
nvx-rs/​src/​host/​cli/​mod.rs Registers snapshot storage and centralizes utility functions.
nvx-rs/​README.md Documents restore ownership, locking, and example usage.
nvx-rs/​public-api/​windows.txt Records the ready receiver and listener fields.
nvx-rs/​public-api/​linux.txt Records the ready receiver and listener fields.
nvx-rs/​examples/​external-launch.rs Demonstrates restore-ready verification, confirmed exit, and view cleanup.

Review effort: Balanced

Move the console attachment checks and the runtime options (the CPU
profile, the guest shutdown action and the control authentication
timeout) out of VmConfig::openvmm_arguments into console_backend and
runtime_arguments, so that a snapshot restore can render them by the
same rules as a cold boot. A cold boot's arguments do not change.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ppenna
Pedro Henrique Penna (ppenna) force-pushed the esaurez/sl216-production-0f0b8f59-20261110 branch from a313023 to 9bc9d40 Compare November 11, 2026 04:21
@ppenna
Pedro Henrique Penna (ppenna) changed the base branch from esaurez/sl216-inputs-411c6e97-20261110 to feature/nvx-refactor November 11, 2026 04:21
@ppenna Pedro Henrique Penna (ppenna) changed the title host: prepare caller-owned snapshot restores (SL2.16) host: restore a VM from a snapshot and a VmConfig (SL2.16) Nov 11, 2026
vm::create refused every snapshot, which left callers on the
store-coupled clone launch. With from: Some(snapshot), it now refuses
Boot, checks Snapshot::accepts at the host's real time, hard-links the
snapshot's fixed OpenVMM files into a new <vm_dir>/openvmm/ view, binds
the restore-ready listener before any spawn, and renders the restore's
arguments by the clone launch's rules: the devices at the snapshot's
processor capacity, --restore-network-policy with the restore's own
policy instead of --net, --restore-absent-scratch for an upper in RAM,
--restore-processors only for a snapshot that records its boot
processors, --restore-memory, the console attachment that the snapshot
allows, and the snapshot's backend and CPU profile. A view on another
filesystem fails; nothing is copied.

The snapshot's NIC feeds Prepared::nic and activation_record, whose
network carries the restore policy's DNS, publication and proxy. The
listener moves, never reopened nor inherited by the child, through
VmmCommand::ready to an external launcher, or through Launched::ready
from vm::spawn. ReadyListener::receive becomes public, so that such a
caller verifies the restore-ready peer. A failed creation removes only
what it created: the links, the view, the listener's socket and the
log.

create's docs state the caller's Linux per-snapshot lock, held from
create through the peer-verified restore-ready event or the VM's
verified exit, and around the view's removal. OpenVMM checks the
snapshot's files only before its one restore-ready connection, which it
does not retry, so a restore whose caller dies before that connection
cannot complete, and the caller needs no durable in-flight record. The
external-launch example takes the snapshot's lock before opening it,
restores a snapshot that `nvx-rs snapshot capture` captured, with its
disconnected boot console, up to the verified ready event, then stops
the VM and removes the view.

Tests cover the refusals before any output exists, the view's links
and rollback, the restore arguments, the NIC and activation record,
the listener's handoff through both launch paths, and, when
NVX_VM_TEST_OTHER_FILESYSTEM names a second filesystem, the refusal of
a view across filesystems. A snapshot captured with nvx-rs snapshot
capture restored through the example to its verified ready event on
KVM, MSHV and WHP test VMs, with OpenVMM 14dc4116 and the guest
artifacts of CI run 42956031193.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the snapshot-based VM restore changes.

No open issues 🎉

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feat-nvx-refactor tests-modified Changes existing tests, test data, or test wiring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants