Repository navigation
cli: restore named snapshots once with run (SL3.8b) - #1489
Pedro Henrique Penna (ppenna) merged 2 commits into
Conversation
This pull request changes existing testsIt modifies or removes 50 existing test asset(s), listed below. Review each with care: a change to a test can hide a regression that the test caught. See doc/contribute.md#changes-to-existing-tests. Test code
End-to-end tests
CI test wiring
Relocated unchanged: 0; added: 11. Compared the merge base |
There was a problem hiding this comment.
🟡 Changes recommended
Changes are needed to prevent deletion of a state directory created by another process, correctly forward signal exit statuses, and migrate a terminal-restoration regression test that now fails in Linux CLI validation.
3 open issues
What changed in this PR
This PR implements SL3.8b of #1114, routing run --restore-snapshot through the managed named-snapshot restore path introduced by #1486 and #1487.
Changes:
- Adds one-off restoration with shared sandbox options, workload-status forwarding, and cleanup.
- Rejects raw snapshots and incompatible options, removing the obsolete raw-restore implementation.
- Updates documentation and tests, and retires the raw console-restore scenario.
| File | Description |
|---|---|
scripts/test_microvm_tests.py |
Removes tests for the retired console-restore scenario. |
scripts/nvx_tools/microvm_tests.py |
Removes the raw console-restore runner and dispatch. |
scripts/nvx_tools/microvm_test_scripts/run-console-restore.sh |
Deletes the retired guest script. |
scripts/nvx_tools/microvm_shards.json |
Removes the scenario and rebalances shards. |
nvx-rs/tests/cli/validation.rs |
Tests named-restore requirements and refusals. |
nvx-rs/tests/cli/usage.rs |
Updates help and option-parsing expectations. |
nvx-rs/tests/cli/openvmm.rs |
Removes raw-snapshot fixtures and expectations. |
nvx-rs/tests/cli/one_shot.rs |
Removes obsolete raw-restore launch cases. |
nvx-rs/tests/cli/dry_run.rs |
Removes unsupported restore dry-run cases. |
nvx-rs/tests/cli/download.rs |
Checks rejected restores download nothing. |
nvx-rs/tests/cli.rs |
Updates the failed-restore CPU-guidance case. |
nvx-rs/src/host/template/mod.rs |
Removes the raw attachment-reader re-export. |
nvx-rs/src/host/snapshot/manifest/tests.rs |
Moves attachment checks to the retained manifest reader. |
nvx-rs/src/host/snapshot/manifest.rs |
Removes the raw-snapshot attachment reader. |
nvx-rs/src/host/cli/tests.rs |
Checks dynamically shared run options. |
nvx-rs/src/host/cli/snapshots.rs |
Exposes restore options and store-boundary validation internally. |
nvx-rs/src/host/cli/sandbox/mod.rs |
Adds one-off restore orchestration and cleanup. |
nvx-rs/src/host/cli/sandbox/lifecycle/start/tests.rs |
Tests restore policy, NIC refusals, and status forwarding. |
nvx-rs/src/host/cli/sandbox/lifecycle/start.rs |
Adds optional workload-status forwarding. |
nvx-rs/src/host/cli/run/tests.rs |
Tests shared restore parsing and network handling. |
nvx-rs/src/host/cli/run.rs |
Replaces raw restores with named one-off restores. |
nvx-rs/src/host/cli/network.rs |
Removes obsolete raw network-argument generation. |
nvx-rs/src/host/cli/mod.rs |
Updates the run command’s module documentation. |
nvx-rs/README.md |
Documents named restores and revised CLI behavior. |
doc/usage.md |
Documents restore options, cleanup, and interruption recovery. |
doc/run.md |
Moves raw-restore examples to direct OpenVMM commands. |
Review effort: Balanced
bf63b6a to
12f0f38
Compare
8741bb3 to
824fa35
Compare
12f0f38 to
8315693
Compare
8315693 to
fc50a29
Compare
`run --restore-snapshot PATH` handed a raw OpenVMM snapshot directory to OpenVMM with the options of a cold boot, outside the store, the locks, and the restore driver of the named snapshots that `snapshot capture` captures and `snapshot restore` restores. It now takes the name of a snapshot of `--store-dir` and restores it once, through the path of `snapshot restore`: `sandbox::restore_once` provisions a managed sandbox in `--state-dir`, starts it from the snapshot under the snapshot's lock, through the launch journal and the blocking restore driver, which forwards the workload's output, stops the VM once the workload has exited with status 0, deprovisions the sandbox once it is provisioned, whatever happened, and ends with the workload's status. The restore inputs are options of `snapshot restore`, taken from its own option list and parsed by `sandbox` for both: `--state-dir`, `--layer`, `--layer-format`, `--scratch`, `--upper`, `--ram-upper-percent`, `--entrypoint`, `--arg`, `--hostname`, `--workload-user`, and `--timeout`, so a restore of `run` takes none that `snapshot restore` refuses, such as the exec-only `--wait`; `--memory-mib` takes the sandbox's default for a restore. Both commands prepare a restore through one function, into which `snapshot restore`'s own preparation moves: it resolves the store, refuses a state directory in the store, checks the workload as the exec request that each restored start sends, which may thus hold whitespace and quotes, and opens the snapshot, before anything is provisioned. The restore links the snapshot's files into `--state-dir`, which must therefore be on the store's filesystem, and outside the store, which SL3.8a requires of every restored sandbox, at its provision, start, and deprovision. Since the run removes it, it must not exist, and an empty path, which names the working directory, is refused: a sandbox that it names is never removed. A cold boot refuses `--store-dir`, `--state-dir`, and the restore inputs, and a restore refuses the options of a cold boot that the snapshot or its sandbox fixes: `--guest`, `--memory-capacity-mib`, `--processors`, `--cpu-profile`, `--net`, the share options, `--outcome-report`, `--cmdline`, and `--dry-run`. A raw OpenVMM snapshot is refused before anything is read or launched: `--restore-snapshot` without `--store-dir`, or with a value that is no snapshot name, such as a path, fails with a message that says that the option names a snapshot of the store that `snapshot capture` captured. `--restore-processors`, `--restore-memory-mib`, `--restore-ready-path`, and `--restore-network-policy` only served the raw restore, and go: the named restore takes its processors from the snapshot's key and its memory from `--memory-mib`, owns its restore-ready endpoint, and always installs the restore's own network policy. A restored workload that does not exit with status 0 still fails the start, which ends its VM and launch, but the start that `run` asks for ends with the workload's status, which `run` exits with, as `sandbox run` does: 128 plus the signal's number for a workload that a signal ends. `snapshot restore`, and each later `sandbox start` of a restored sandbox, keep their error, such as "the restored workload ended with Exit status N", and status 1. The snapshot fixes the NIC. The restore of a snapshot with a NIC takes the NIC's address, and lowers the run's DNS server, egress, forwards, and published listeners, which need `--network-profile`, into the restore's own policy, as `sandbox` lowers them; that of a snapshot without one refuses network options before it provisions anything. `snapshot capture` gives its VM no NIC today, so only fake-backed tests cover the policy that reaches the guest of a networked restore. A `run` that dies before it removes its sandbox, as `kill -9` ends it, leaves `--state-dir` and its VM, which keeps running until a command of the sandbox ends it, and the snapshot's lock goes with the `run`. While the workload runs, the start is unfinished, and `sandbox deprovision` ends the VM, through the recovery of the start's launch journal, and removes the directory; once the workload has exited with status 0, `sandbox stop` must end the VM first, as it must after Ctrl-C while the run stops the VM. usage.md says so. The raw restore's code goes with it: the restore body and console reattachment of `run`, the OpenVMM arguments of the network options, the reader of a snapshot manifest's attachments that only it used, and `sandbox::one_shot`, which has run OpenVMM on the terminal for it alone since `sandbox run` runs its workload through the managed VM API, with the Windows `spawn_foreground` and `standard_handle` that only `one_shot` used. The `run-console-restore` microVM scenario, which restored a raw snapshot with its boot console on the terminal, is retired with its shard entries. esaurez's partial implementation on esaurez/private-sl38b-0325-b2 (ef52ba6) restored blockless snapshots through unmerged prerequisites. This keeps its refusal of raw snapshots and of the options that a restore fixes, and restores through the layered sandbox path of SL3.8a instead. Evidence, on nvx-kvm-test (KVM, Azure Linux 3, kernel 6.6) and nvx-mshv-test (MSHV, kernel 6.18), virtual machines, and on WHP on a Windows 11 developer machine, with OpenVMM 14dc4116, the kernel of release v0.1.0-dev.53ad09678bc1, and this tree's Alpine initramfs: `snapshot capture` published `base`, and `run --restore-snapshot base` restored the release's Ubuntu EROFS layer and a GPT image of its tree, over a RAM upper and over a fresh scratch (GPT over a scratch on KVM and MSHV), as nobody and as daemon (`--workload-user 1:1`), forwarded the workload's output and its statuses 0, 7, and 1, and left no directory, VM, or link to the snapshot's files behind, while `snapshot restore` of /usr/bin/false still failed with its error and status 1. The raw snapshot directory, with and without `--store-dir`, network options for the snapshot without a NIC, a missing, existing, or in-store `--state-dir`, and `--wait` were refused before launch. On KVM and MSHV also: `--memory-mib 384` of a snapshot of 512 MiB capacity (MemTotal 372564 kB, against 241492 kB), three concurrent one-offs of one snapshot, SIGINT (status 130, the directory removed), and a state directory of 89 bytes, whose socket fails the VM's creation and which the run removes. A `run` that kill -9, or Stop-Process -Force on Windows, ended while its workload ran left its VM running 120 s later, while another restore of the snapshot succeeded; `sandbox deprovision` alone ended that VM and removed the directory, as `sandbox stop`, which then reports sandbox-not-running, and `deprovision` also did. A restored sandbox whose workload had exited with status 0 refused `deprovision` until `sandbox stop` ended it. Tests: unit tests parse the restore options of `run` as those of `snapshot restore`, give the restore of a snapshot the snapshot's NIC with the policy of the run's options, and refuse network options for a snapshot without a NIC; fake-backed restores install the run's network options as the restore's own policy, with the DNS server and published listeners of the activation record, and OpenVMM's egress, forward, and host-loopback options without `--net`, refuse network options for a snapshot without a NIC before they provision anything, and end with the status of their workload, 3, or 143 for one that SIGTERM ends, once the VM and its launch have ended, where `snapshot restore` keeps its error, and take a workload with whitespace and quotes, as `snapshot restore` does. CLI tests cover the raw refusals, a missing store, the state directory's refusals, a workload that is not a valid exec request, the options that each kind of run refuses, the removed options and `--wait`, and the help. The dry runs and launches of raw restores, the Python test that a raw restore left OpenVMM the terminal, which no `nvx-rs run` gives OpenVMM anymore, the proptest of a restore's policy paths, the tests of the attachment reader, and the test of the handles that `spawn_foreground` passes, with its case in the test of refused batch files, are retired with the code that they tested. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
SL3.8b removes `nvx-rs run`'s restore of a raw OpenVMM snapshot and the boot console that it reattached on the terminal, which the `run-console-restore` microVM scenario checked, so the case matrix gives that scenario the fate retired, and its summary of the microVM scenarios counts 48 kept and 1 retired. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
a69c751 to
ce25a7f
Compare
There was a problem hiding this comment.
🟡 Changes recommended
The post-workload stop/deprovision sequence lacks end-to-end fake-backed tests, leaving regressions that leak VMs, state directories, or snapshot hard links undetected.
1 open issue
3 resolved since last review
Review effort: Balanced
1b902ac
into
feature/nvx-refactor


Part of #1114 (SL3.8b:
run --restore-snapshotrestores a one-off VM from a strict snapshot).It builds on SL3.8a1 (#1486) and SL3.8a2 (#1487), which add the restored start and the
snapshot restorecommand whose restore path and options this one-off reuses; both have merged.What and why
nvx-rs run --restore-snapshot PATHhanded a raw OpenVMM snapshot directory to OpenVMM with the options of a cold boot, outside the store, the locks, and the restore driver of the named snapshots thatsnapshot capture(SL3.7b) publishes andsnapshot restore(SL3.8a) restores. This PR does SL3.8b's Scope, as Pedro Henrique Penna (@ppenna) decided it (option A: the layered restore of SL3.8a, which #1114's Notes call for, since a restored VM gets its image, upper and workload at activation):One restore path.
run --restore-snapshot NAME --store-dir STORE --state-dir DIRrestores the named snapshot once throughsnapshot restore's own path.sandbox::restore_onceprovisions a managed sandbox inDIRand starts it from the snapshot, under the snapshot's lock, through the launch journal and the blocking restore driver, which forwards the workload's output. It stops the VM once the workload has exited with status 0, deprovisions the sandbox (removingDIR) once it is provisioned, whatever happened, and ends with the workload's status.--state-dir, and why. The restore hard-links the snapshot's files into the sandbox's directory, so that directory must be on the store's filesystem. SL3.8a2 refuses a restored sandbox's directory inside the store, beforesnapshot restoreprovisions it, at each start and at deprovision (outside_store, throughLockedSnapshot::remove_view), so the one-off can no longer keep its sandbox below the store, as an earlier draft of this PR did inSTORE/runs/<hex>. Only the caller can name a directory outside the store on its filesystem, soruntakessnapshot restore's own--state-dir, with its definition and parsing. Its rules: it must lie outside the store (checked with SL3.8a2'soutside_storebefore anything is read or provisioned) and on the store's filesystem; it must not exist, and an empty path, which names the working directory, is refused, since the run creates it and removes it once it ends, so a sandbox or directory that it names already is never removed. The existence check guards against reusing an existing state directory. Preparing the state directory is shared with every sandbox lifecycle command, and outside this PR's scope. On Linux, the VM's sockets in it must fit a Unix socket path (below 108 bytes), as forsnapshot restore: a longer one fails at the VM's creation, and the run removes the directory. The 66-byte limit on the store's path, which the earlier draft'sruns/imposed, no longer applies.Exit status. A restored workload that doesn't exit with status 0 still fails the start, which ends the VM and its launch. But the start that
runasks for (forward_status, carried in the start'sRestore) ends with the workload's status, whichrunexits with, whatever ended the workload. That's whatsandbox runandsandbox execdo, which exit with the guest's status whatever its category: 128 plus the signal's number for a workload that a signal ends.snapshot restore, and each latersandbox startof a restored sandbox, keep SL3.8a's error, such aserror[other]: the restored workload ended with Exit status N, and status 1.Restore inputs reuse
snapshot restore's options.--state-dir,--layer,--layer-format,--scratch,--upper,--ram-upper-percent,--entrypoint,--arg,--hostname,--workload-userand--timeoutaresandbox's option definitions, names and parsing (sandbox::OPTIONSandsandbox::Options::take), taken fromsnapshot restore's own option list (snapshots::restore_options), so a restore ofruntakes none thatsnapshot restorerefuses, such as SL3.9's exec-only--wait.runkeeps its own--hypervisorand--memory-mib, which defaults to the sandbox's 256 MiB for a restore. It adds no option that neithersnapshot restorenor today'srunhas:--store-dirand--state-diraresnapshot restore's.One preparation, so both restore commands accept and refuse the same workloads.
snapshot restore's own preparation moves intosnapshots::restore_source, which both commands call before they provision anything. It resolves the store, refuses a state directory in the store, checks the workload as the exec request that each restored start sends, and opens the snapshot, returning the restore'sSourceand the snapshot's NIC. Since cli: restore managed sandboxes from named snapshots (SL3.8a2) #1487's last fix, a restored sandbox's launch takes the default workload, and its own workload travels over NVXC as an exec request. Sorun --restore-snapshot, likesnapshot restore, takes an entrypoint and arguments with whitespace and quotes, whichsandbox provisionrefuses. And it refuses a workload that isn't a valid exec request, such as a relative entrypoint or 65 arguments, before it opens the store or provisions anything; before this,runwould have provisioned and launched the VM first.outside_storekeeps its original visibility, since onlyrestore_sourcecalls it now.Refusals. A cold boot refuses
--store-dir,--state-dirand the restore inputs. A restore refuses the options of a cold boot that the snapshot or its sandbox fixes:--guest,--memory-capacity-mib,--processorsand--cpu-profile(the snapshot's),--net(the snapshot's NIC), the share options (run's policy paths mean other paths thansandbox's, and shares are not in the Scope),--outcome-report,--cmdlineand--dry-run.Raw snapshots are refused before launch.
--restore-snapshotwithout--store-dir, or with a value that is no snapshot name, such as a directory, fails before anything is read or launched:--restore-snapshot requires --store-dir: it names a snapshot of --store-dir, which `snapshot capture` captured, and no longer a raw OpenVMM snapshot.Raw-only options go.
--restore-network-policy(as the Scope says), and--restore-ready-path,--restore-processorsand--restore-memory-mib, which only the raw path used: the named restore owns its restore-ready endpoint, takes its processors from the snapshot's key (the online count thatvm::createrestores), and its memory target from--memory-mib, a multiple of 128 between the snapshot's boot memory and its capacity.Network. The snapshot fixes the NIC. For a snapshot with a NIC,
run's DNS, egress, forwards and published listeners, which need--network-profile, take the NIC's address and become the restore's own network policy through the normal lowering (sandbox_network).snapshot capturegives its VM no NIC (capture.rs,network: None), andSnapshotrequires the restore's NIC presence to match, so network options for a snapshot without a NIC are refused before anything is provisioned:the snapshot has no NIC, so its restore takes no network options. Limitation: only fake-backed tests cover the policy that reaches the guest of a networked restore; a NIC forsnapshot captureis out of scope (follow-up on nvx-rs slimming 3.2.2: the CLI captures and restores named snapshots #1114).What a run leaves when it cannot clean up (measured on KVM, MSHV and WHP). A
runthatkill -9(orStop-Process -Forceon Windows) ends while its workload runs leaves--state-dir(config.json, the launch journallaunch.json,control.capability, the control endpoint,openvmm.log,sandbox.lock, theopenvmm/view, whose hard links keep the snapshot's files at 2 links, and on Linuxrestore-ready.sock) and its VM: OpenVMM was still running 120 s later, 100 s after its workload ended, and nothing ends it but a command of the sandbox. The snapshot's lock goes with therun, so another one-off of the same snapshot restored meanwhile, and a laterrunrefuses the existing directory. The start was unfinished, andalone ended that VM, through its lock's recovery of the start's launch journal, and removed
DIR, the snapshot's links back to 1. (nvx-rs sandbox stop --state-dir DIRalso ends it, through the same recovery, and then reportssandbox-not-running.) Once the workload has exited with status 0, the start has committed and the VM runs as a started sandbox does, whichdeprovisionrefuses (sandbox must be stopped before deprovision): arunkilled then, or one that Ctrl-C interrupts while it stops the VM, which also leaves the sandbox and its running VM and prints a warning that names the directory, needsas shown with a restored sandbox of
snapshot restoreon KVM and MSHV; the Ctrl-C case is from the code, since that window is too short to time on a VM. Ctrl-C while the workload runs ends the VM and removes the directory (exit 130).doc/usage.mdsays all this. A sweep of leftover directories would be the restore reconciliation that SL3.8b's "Not in this PR" excludes.Code that goes with the raw path:
run's restore body and console reattachment,NetworkOptions::extend_arguments, andsnapshot::manifest::read_attachmentswith its re-export, which only the raw path used. Since SL3.4b (sandbox: run a workload once through the managed VM API (SL3.4b) #1490) runssandbox run's workload through the managed VM API, the raw restore was also the last caller ofhost::cli::sandbox::one_shot, which ran OpenVMM on the terminal. The module goes (run_openvmm,OneShot,end_interruptedandspawn_error, with itsmodline and the re-export ofrun_openvmminsandbox/mod.rs). So do the Windowsplatform::spawn_foregroundand its helperstandard_handle, which onlyone_shotused, with theircli-gated imports and the re-export inplatform/windows/mod.rs. Without them, Clippy with warnings denied fails on this stack: the import ofrun_openvmmis unused, andrun_openvmm,OneShot,end_interrupted,spawn_error,spawn_foregroundandstandard_handleare never used.Docs:
doc/usage.md(run's synopsis, options, restore and leftover paragraphs;snapshot restore's pointer to it),doc/run.md(raw restores go through OpenVMM directly; the networked-snapshot and readiness examples use./bin/openvmm),nvx-rs/README.md(### run, the differences fromnvx.py, the fake's restores).Not in this PR (per #1114): blockless restores; a NIC in
snapshot capture; automatic removal of a killed one-off's sandbox; shares for a restore.Takeover. Enrique Saurez (@esaurez)'s partial implementation on
esaurez/private-sl38b-0325-b2(ef52ba6b5a, checkpoint "WIP - Enrique Saurez (@esaurez) (partial implementation)" on #1114, untouched since 2026-11-09) restored blockless snapshots through unmerged prerequisite branches (esaurez/private-blockless-driver-0325,esaurez/private-blockless-snapshot-0325). This PR keeps its refusal of raw snapshots and of the options that a restore fixes, and restores through SL3.8a's layered path instead, so it needs nothing unmerged beyond SL3.8a.Size. 210 core lines (estimate 150, stop 300), measured against
feature/nvx-refactor(64ad98afacc):host/cli/run.rs152,host/cli/sandbox/mod.rs30 (restore_once),host/cli/snapshots.rs15 (restore_source, whichsnapshot restore's preparation moves into, andrestore_optionsmade visible torun), andhost/cli/sandbox/lifecycle/start.rs13 (forward_status). Counted files: 4 (the estimate was 3).host/cli/network.rs,host/snapshot/manifest.rsandhost/template/mod.rsonly lose code that leaves with the raw restore, and don't count, as for SL3.4b. Nor dohost/cli/sandbox/one_shot.rs, which this PR deletes, andhost/platform/windows/spawn.rsandhost/platform/windows/mod.rs, which only losespawn_foregroundandstandard_handle. The deletions add no core lines.host/cli/mod.rschanges rustdoc only.BOOT_CONTRACT_VERSION,CONTROL_CONTRACT_REVISIONand the public API are unchanged (python3 scripts/public_api.pypasses).Validation
At this head (
ce25a7fa114, onfeature/nvx-refactor64ad98afacc, where SL3.8a1 and SL3.8a2 have merged), on Windows and on nvx-kvm-test (KVM, Azure Linux 3, kernel 6.6), these passed:cargo fmt --checkand the five Clippy variants below with-D warnings.host::cli::run,host::cli::sandbox(with the restore tests),host::cli::snapshots,host::snapshot::manifestandhost::spawn, plushost::platform::windowson Windows: 122 passed on Windows and 92 on Linux.cliintegration tests: 110 passed on Windows and 119 on Linux.NvxRsTestswithNVX_RSset (a debug build on Windows, the release build on Linux),public_api.py(on KVM),check_test_layout.pyandcheck_markdown_links.py.nvx-rsand Alpine initramfs, all aftersnapshot capture base:run --restore-snapshot baseof the Ubuntu EROFS layer over a RAM upper ran/bin/echo 'a "restored" sandbox'(status 0), and, as daemon, a/bin/sh -cscript with spaces that exited with 7, whichrunforwarded.snapshot restoreran the same workload.runrefused the raw snapshot.At
824fa351753, on SL3.8a2 (12f0f38578f, onfeature/nvx-refactord4edeeb9ab9, with SL3.4b), on Windows and on nvx-kvm-test (KVM, Azure Linux 3), these passed:cargo fmt --checkand the five Clippy variants below with-D warnings.host::cli::run,host::cli::sandbox,host::cli::snapshots,host::snapshot::manifestandhost::spawn, plushost::platform::windowson Windows: 117 passed on Windows and 88 on Linux.cliintegration tests: 110 passed on Windows and 119 on Linux. One Windows run, at the previous push of this rebase, failedstop_forces_the_verified_vm_after_its_grace_period_even_after_an_acknowledgement(cannot confirm exit of VM process, asandbox stop --timeout 1that this PR doesn't change); it passed alone three times and in every full run since.NvxRsTestsofscripts/test_nvx_tools.pywithNVX_RSset, as CI's CLI jobs run them: a debug build on Windows, the release build on Linux. They includetest_usage_documents_every_option_of_nvx_rs,test_run_help_states_each_guest_memory_defaultandtest_the_real_nvx_rs_prints_what_nvx_py_reads, which checkrun's options, help and docs and run only withNVX_RSset.public_api.py(on KVM),check_test_layout.py, Ruff ontest_nvx_tools.py, and the scenario-wiring tests:test_microvm_tests(on Windows and Linux) andtest_ci_runs_each_default_microvm_suite_in_its_shards.The table's rows ran at
d2a2ef3266d, on SL3.8a2bf63b6aa.824fa351753carried that change, apart from the test imports that SL3.4b changed and the restore tests that SL3.8a2 moved tostart/tests/restore.rs. It added the deletions above and two changes from Copilot's review: the forwarded status of a workload that a signal ends, and the retired Python terminal test. This head adds the shared restore preparation (snapshots::restore_source) and its tests.cargo fmt --check; Clippy-D warnings:--all-targets --all-features,--lib,--lib --no-default-features,--lib --bins --features cli,--lib --bins --no-default-features --features testingcargo test --all-features --no-fail-fastcli102). Linux, nvx-kvm-test (kernel 6.6): the lib's 28guest::tests fail as they do atfeature/nvx-refactoron that host (kill ProcessGroup: Invalid argument), and nothing else does. Windows: every test passed in one of two full runs; the others failed only under the full suite's load and pass alone (host::cli::store::tests::recovery::process::a_busy_endpoint_cannot_extend_recovery_or_signal_its_live_process, 312 ms against 250 ms;exec_lock::exec_processes_contend_but_other_sandboxes_and_stop_proceed, a stop with--timeout 0.1), or are the Windows timer bug that3340bbd7ee4fixes on the feature branch after SL3.8a2's base (host::restore_driver::tests::ready_has_sixty_seconds_and_start_thirty_from_ready_for_each_pending_accept_and_read, Expired at 59.9998 s)cargo test(default features),cargo test --lib --no-default-features,cargo docwith-D warnings, MSRV 1.89, macOS check, artifact bundling,python3 scripts/public_api.py, runtime adapter protocol-only and host-only configurationscheck_test_layout.py,check_markdown_links.py, Ruff check and format, Pyright (Linux and Windows),compileallpython -m unittesttest_microvm_tests,test_nvx_tools,test_test_changes,test_test_layout,test_markdown_links(1094 tests),test_adversarial.py,nvx.py ... --helpmicrovm_shards.jsonre-planned withnvx.py plan-microvm-shards14dc4116, the kernel of release v0.1.0-dev.53ad09678bc1, this tree's Alpine initramfs; this head's treesnapshot capture base, thenrun --restore-snapshot base --state-dir ...restored the release's Ubuntu EROFS layer and a GPT image of its tree, over a RAM upper and over a fresh scratch (GPT over a scratch on KVM and MSHV), as nobody and as daemon (--workload-user 1:1; also65534:65534, andsnapshot restoreas daemon, on KVM and MSHV), forwarded the workload's output and its status (0, 7, and 1 for/usr/bin/false), and left no directory, VM, or link to the snapshot's files behind;snapshot restoreof/usr/bin/falsestill failed witherror[other]: the restored workload ended with Exit status 1. Refused before launch: the store's raw OpenVMM snapshot directory with and without--store-dir, network options for the NIC-less snapshot, a missing, existing or in-store--state-dir, and--wait; on KVM and MSHV also--processors,--restore-network-policy, a missing snapshot, and--state-dirwithout a restore. On KVM and MSHV also:--memory-mib 384of a snapshot of 512 MiB capacity (MemTotal 372564 kB against 241492 kB), three concurrent one-offs of one snapshot, SIGINT (exit 130, directory removed), and an 89-byte--state-dir, whose socket fails the VM's creation and which the run removes. The kill -9 results above are from KVM, MSHV and WHP at this tree.Found on the way, not caused here:
--workload-user 1000:1000fails a restore at activation (NVX-MANAGED-ERROR: stage=data status=71, guest exit 125), forrun --restore-snapshotas forsnapshot restore: the NVX Ubuntu image has onlyroot,daemonandnobody, and a layered root needs an existing account (protocol::features::WORKLOAD_ACCOUNT), so a coldsandbox startfails the same way at the base. It is a known pre-existing limitation, which Pedro Henrique Penna (@ppenna) is recording as a follow-up; the checks above use1:1(daemon) and65534:65534(nobody).Left to CI: the microVM scenario matrix (
test-microvmon kvm, mshv and whp), the guest builds and their determinism check, and the benchmarks. Theopenvmmgitlink does not move.Changes to existing tests (
tests-modified)check_test_changes.py: 50 changes to existing tests and 11 added, none relocated, againstfeature/nvx-refactor(64ad98afacc). Every other test of the area is kept.host::cli::sandbox::lifecycle::start::tests::restore, where SL3.8a keeps its restore tests and this PR adds its four: the fixture'sstartdelegates to a newstart_with, which also takesforward_status(startpasses false, SL3.8a's behavior);a_restore_whose_spawn_cannot_confirm_that_its_vm_ended_recovers_it_before_its_snapshot_goes_freenamesforward_status: falsein theRestorethat it builds, anda_restored_start_refuses_a_share_of_its_snapshot_storepasses it tostart, assandbox startdoes. No assertion changes. The fake guest'sservegains asignaledbehavior, whose workload SIGTERM ends (status 143, categorysignal), for this PR's test of the statuses that a one-off forwards.host::platform::windows::tests:foreground_children_inherit_only_the_allowed_handles_besides_their_standard_ones, which covered onlyspawn_foreground, retires with it, andbatch_files_are_refused_before_cmd_could_read_their_argumentsdrops itsspawn_foregroundcase and keeps itsspawn_with_stdioone.scripts/test_nvx_tools.py:NvxRsTests.test_run_leaves_openvmm_the_terminal_to_restoreretires with the raw restore andrun_openvmm, along with theselectimport that only it used. It ran a raw restore on a pseudo-terminal to show thatnvx-rs runleft OpenVMM the terminal on its standard error. Nonvx-rs rungives OpenVMM the terminal anymore: a cold boot gives it a pipe for its standard input and the run's log for its standard output and error (host::spawn), and a named restore starts it detached through the restore driver. The test ran only withNVX_RSset, as in CI's CLI jobs.host::cli::run::tests::restore_policy_paths_attach_to_their_shares(proptest): retired with the raw restore's share and policy-path arguments; a restore refuses shares.host::cli::tests::usages_are_one_line_and_helps_list_every_option: readsrun's options fromrun::options(), which adds the restore inputs thatsandboxdefines.host::snapshot::manifest::tests:the_attachments_that_openvmm_writes_are_read_in_their_orderand its helperattachmentread the attachments throughSnapshotManifest::read, the remaining reader, so they no longer need theclifeature.the_attachments_of_any_snapshot_are_readis rewritten asmalformed_attachments_are_refused, which keeps its attachment bound and non-UTF-8 checks throughSnapshotManifest::parse; its acceptance of non-template manifests, the helperterminal_console_snapshot, andthe_attachments_of_unreadable_manifests_are_refusedretire withread_attachments, whose missing, foreign-format and malformed manifestsSnapshotManifest::read's own tests cover for templates.cli/dry_run:dry_runs_print_the_openvmm_command_and_launch_nothing,run_dry_runsanddry_runretire the raw restores' dry runs, which a restore now refuses: "run: restore", "... of a boot console on the terminal", "... of a boot console on the terminal with every option", "... of a disconnected boot console", "... without guest artifacts", "... with every option", "... of the Alpine guest", "... with equal processors", "... ignores the memory", "... paths", "... with shares", "... with shares and a command line", "... of a networked snapshot", "... with a network policy", "... with an egress policy file", "... that replaces the network policy", "... retains quoted command line and delimiter", "... without the budget"; the helperwith_snapshotand the raw snapshot fixture go.cli/one_shot:one_shot_runs_exit_with_the_workload_or_openvmm_status,run_launchesandfails_a_cold_bootretire the raw restores' launches ("run: restore", "... without guest artifacts", "... of a boot console on the terminal", "... of a disconnected boot console", "... outcome report", "... of a networked snapshot", "... that replaces the network policy").cli/openvmm: the raw restore's expectations and snapshot builders go (restore,TERMINAL_CONSOLE,DISCONNECTED_CONSOLE,NO_CONSOLE,snapshot,snapshot_manifest).cli/usage: "run: restore memory not an integer" and "run: restore processors not an integer" become "... option removed" (unrecognized arguments); "run: restore processors not a choice" retires; "run: missing path" becomes "run: missing snapshot name"; new: the removed--restore-ready-pathand--restore-network-policy,--wait, whichsnapshot restorerefuses too, a restore upper that is not a choice, a zero restore timeout.HELPlists--restore-snapshot NAME,--store-dir PATH,--state-dir STATE_DIR, and the restore inputs.cli/validation(run_refusals): "run: net of a restore" and "run: net without profile of a restore" restore a named snapshot of a store and expect--net is not accepted for restore; "run: memory capacity of a restore" expects--memory-capacity-mib is not accepted for restore; retired with their options or the manifest reading: "run: ready path without snapshot", "run: network policy replacement without snapshot", "run: restore processors without snapshot", "run: restore memory without snapshot", "run: restore processors above processors", "run: restore of a missing snapshot", "... without a dry run", "run: restore of a snapshot without a manifest", "run: restore of a malformed manifest", "run: restore of a manifest of another format"; new: a store, a state directory or restore inputs without a snapshot, a raw snapshot without a store and in place of a name, a missing store, a restore without a state directory, with an empty one, into an existing one and into the store, a workload that isn't a valid exec request (a relative entrypoint, with a missing store, which shows that the check comes first), and a dry run, a share, a command line and processors of a restore.cli::run_guides_failed_cold_boots_on_cpus_that_no_built_in_profile_serves: its restore case, after which no guidance prints, restores a named snapshot of a missing store instead of a raw snapshot.cli/download:restores_download_nothing_for_snapshots_that_they_cannot_readis rewritten asrestores_download_nothing_for_snapshots_that_they_cannot_open: a restore without--store-dirand one of a missing store download nothing, in place of an unreadable raw manifest.run-console-restore, which restored a raw snapshot with its boot console on the terminal throughnvx-rs run, retires: its script, runner, marker and dispatch inscripts/nvx_tools/microvm_tests.py, its tests (RunConsoleRestoreScenarioTests) inscripts/test_microvm_tests.py, and its entries on kvm, mshv and whp inmicrovm_shards.json, whichnvx.py plan-microvm-shardsre-planned. The real-VM checks above cover the named restore. In the case matrix (doc/acceptance-case-matrix.md, nvx-rs: build every feature set, compile the documented examples, and give every old test a fate (SL5.1a) #1488), its row becomes retired, and the summary of the microVM scenarios counts 48 kept and 1 retired. No other row names a test that this PR changes or deletes.