diff --git a/doc/acceptance-case-matrix.md b/doc/acceptance-case-matrix.md index 11a9aada4..4503cb534 100644 --- a/doc/acceptance-case-matrix.md +++ b/doc/acceptance-case-matrix.md @@ -30,11 +30,11 @@ its own in turn. | Section | Tests | Kept | Rewritten | Retired | | --- | ---: | ---: | ---: | ---: | -| [The adapter's scenarios](#the-adapters-scenarios) | 157 | 0 | 114 | 43 | +| [The adapter's scenarios](#the-adapters-scenarios) | 157 | 0 | 113 | 44 | | [The adapter's other tests](#the-adapters-other-tests) | 169 | 0 | 9 | 160 | | [MicroVM scenarios](#microvm-scenarios) | 54 | 50 | 3 | 1 | | [CI](#ci) | 69 | 43 | 3 | 23 | -| [Docs](#docs) | 36 | 26 | 3 | 7 | +| [Docs](#docs) | 36 | 25 | 3 | 8 | ## The adapter's scenarios @@ -129,9 +129,9 @@ The adapter's README said what each proved. | `a_launch_refuses_shares_that_it_cannot_export` | rewritten | `nvx-rs/src/host/filesystem/tests.rs::policy_paths_hide_show_and_narrow_writes`; `nvx-rs/tests/security_checklist.rs::an_export_refuses_a_read_only_path_inside_a_writable_directory` | | `a_launch_renders_each_element_of_its_network_through_the_plan` | rewritten | `nvx-rs/src/host/config/tests.rs::cold_boot_arguments_lower_layered_roots_networks_and_exports_by_path`; `nvx-rs/src/host/network/tests.rs::proxy_urls_name_the_gateway_as_the_guest_sees_it` | | `a_launch_shares_host_paths_inside_the_authorized_directories` | rewritten | `nvx-rs/src/host/filesystem/tests.rs::policy_paths_hide_show_and_narrow_writes`; `nvx-rs/src/host/config/tests.rs::cold_boot_arguments_lower_layered_roots_networks_and_exports_by_path` | -| `a_layered_launch_hands_openvmm_the_checked_images` | rewritten | `nvx-rs/tests/checked_handoff.rs::a_prepared_launch_hands_the_vmm_its_checked_images_alone`; `nvx-rs/src/host/config/tests.rs::cold_boot_arguments_lower_layered_roots_networks_and_exports_by_path` | +| `a_layered_launch_hands_openvmm_the_checked_images` | rewritten | `nvx-rs/src/host/config/tests.rs::cold_boot_arguments_lower_layered_roots_networks_and_exports_by_path`. A launch names its images by path: the checked-file handoff leaves (SL6.3, #1122), as `images_replaced_after_the_checks_never_reach_the_vm` says | | `a_layered_launch_passes_what_the_guest_checks_as_it_is` | rewritten | `nvx-rs/src/guest/init/boot/tests.rs::cold_boot_reads_the_actual_root_view_export_maps_and_cli_contracts` | -| `a_layered_launch_plans_its_blocks_and_boot_tokens_with_the_adapters_plan` | rewritten | `nvx-rs/src/host/config/tests.rs::cold_command_lines_keep_the_old_root_view_and_mapping_values`; `nvx-rs/src/host/config/tests.rs::cold_boot_arguments_lower_layered_roots_networks_and_exports_by_path` | +| `a_layered_launch_plans_its_blocks_and_boot_tokens_with_the_adapters_plan` | rewritten | `nvx-rs/src/host/config/tests.rs::cold_command_lines_carry_the_root_the_view_and_the_mapping_count`; `nvx-rs/src/host/config/tests.rs::cold_boot_arguments_lower_layered_roots_networks_and_exports_by_path` | | `an_execution_finds_the_url_of_its_network_s_proxy_in_its_environment` | rewritten | `nvx-rs/src/guest/managed_agent/launch/tests.rs::restored_proxy_defaults_replace_template_values_before_exec_overrides`; `test-microvm --scenario sandbox-proxy-only` | | `shares_read_their_host_path_their_target_and_their_access` | rewritten | `nvx-rs/src/host/cli/run/guest_tokens/tests.rs::planned_boot_shares_keep_table_sources_targets_access_and_file_kind` | | `the_host_program_refuses_a_network_before_it_launches` | rewritten | `nvx-rs/tests/cli.rs::network_policies_that_the_planner_refuses_fail_before_launch` | @@ -152,7 +152,7 @@ The adapter's README said what each proved. | Scenario | Fate | Coverage, or why it retires | | --- | --- | --- | | `a_launch_that_the_hook_ends_after_its_checks_leaves_no_launch` | rewritten | A launch that fails after its checks leaves nothing of its own: `nvx-rs/src/host/vm/tests.rs::creation_rolls_back_its_log_and_never_removes_preexisting_outputs_or_inputs`; `nvx-rs/tests/security_checklist.rs::a_scratch_image_is_claimed_by_one_launch_and_checked_again_after_a_release`. The step hook retires | -| `images_replaced_after_the_checks_never_reach_the_vm` | rewritten | `nvx-rs/tests/checked_handoff.rs::a_vmm_finds_the_checked_images_past_replaced_links_from_another_directory` | +| `images_replaced_after_the_checks_never_reach_the_vm` | retired | The checked-file handoff leaves (SL6.3, #1122): a launch names its images by path, so what takes an image's path after its check is what OpenVMM opens. Callers stage their images in a directory that only their own account can write, as the docs of `host::sandbox::distro`, `host::sandbox::scratch` and `host::seal` require, so that no other account can replace a checked image | | `the_hook_pauses_a_launch_at_each_of_its_steps` | retired | The adapter's step hook | ### `vm_layered` (43) @@ -358,7 +358,7 @@ suite in its shards, and the public sandbox scenarios in its guest and sandbox j | `virtio-features` | kept | | | `virtio-net` | kept | | | `vm-lifecycle` | kept | A [cold boot](#cold-boots) through the public API: `nvx-rs/tests/vm_lifecycle.rs::a_cold_vm_is_reattached_from_saved_values_and_stopped` | -| `workload-identity` | kept | | +| `workload-identity` | kept | Since SL6.3 (#1122), it also checks that a guest given OpenVMM's workload identity without `nvx_identity=`, or with another one, fails closed | ### Cold boots @@ -500,9 +500,9 @@ this matrix. The README's own code is no doctest. | `nvx-rs/src/guest/spawn.rs - guest::spawn::LaunchStage::from_code (line 201)` | rewritten | `nvx-rs/src/guest/spawn/tests.rs::launch_stages_keep_their_numbers`. The exec engine's launch failure and outcome, which the example read, leave with the engine (SL6.11a) | | `nvx-rs/src/guest/terminal.rs - guest::terminal (line 10)` | kept | | | `nvx-rs/src/guest/time/mod.rs - guest::time (line 9)` | kept | SL6.11c removed its capture and restore finish, which became crate-private: `guest::snapshot::capture` runs them | -| `nvx-rs/src/host/sandbox/prepared.rs - host::sandbox::prepared::PreparedLaunch (line 92)` | kept | Its launch type leaves with the old launch paths (#1087), whose PR retires it | +| `nvx-rs/src/host/sandbox/prepared.rs - host::sandbox::prepared::PreparedLaunch (line 92)` | retired | `PreparedLaunch` leaves with the checked-file handoff (SL6.3, #1122) | | `nvx-rs/src/host/store/mod.rs - host::store (line 72)` | retired | The sandbox store leaves (SL6.1, #1121) | -| `nvx-rs/src/host/supervisor/mod.rs - host::supervisor (line 32)` | retired | The store-coupled supervisor leaves with the sandbox store (SL6.1, #1121); `host::supervisor` stays only as the transitional public path of the low-level spawn, until SL6.3 (#1122), with no doc test | +| `nvx-rs/src/host/supervisor/mod.rs - host::supervisor (line 32)` | retired | The store-coupled supervisor leaves with the sandbox store (SL6.1, #1121), and `host::supervisor`, the transitional public path of the low-level spawn, with SL6.3 (#1122): `host::vm` spawns VMMs, and the tests of the spawn moved to `nvx-rs/src/host/spawn/tests.rs` and `nvx-rs/tests/process.rs` | | `nvx-rs/src/host/template/clone.rs - host::template::clone::ClonePlan (line 247)` | retired | `ClonePlan` and `CloneRequest` leave with the old clone path (SL6.2b, #1121) | | `nvx-rs/src/host/template/plan.rs - host::template::plan::TemplatePlan (line 137)` | retired | `TemplatePlan` leaves with the template store (SL6.2a, #1121) | | `nvx-rs/src/host/template/store.rs - host::template::store::Store (line 473)` | retired | The template store leaves (SL6.2a, #1121) | @@ -512,4 +512,4 @@ this matrix. The README's own code is no doctest. | `nvx-rs/src/host/snapshot/capture.rs - host::snapshot::capture::finish_capture (line 48)` | kept | | | `nvx-rs/src/host/store/mod.rs - host::store (line 104)` | retired | The sandbox store leaves (SL6.1, #1121) | | `nvx-rs/README.md`, its Example: an instance through `LaunchSpec` and the sandbox store | rewritten | Nothing compiles it. `nvx-rs/examples/host-workflows.rs` compiles the documented host workflows, and the library's guide (SL6.10a) replaces the example. SL6.1 (#1121) made it the cold boot that `nvx-rs/src/lib.rs - (line 57)` compiles | -| `nvx-rs/examples/external-launch.rs` | kept | Compiled with the crate's targets, and run by hand on a host | +| `nvx-rs/examples/external-launch.rs` | kept | Compiled with the crate's targets, and run by hand on a host. Since SL6.3 (#1122), which deletes `host::supervisor`, a minimal launcher of its own on `std::process` starts its VMM | diff --git a/doc/design/2.1-machine-model-and-configuration.md b/doc/design/2.1-machine-model-and-configuration.md index 469fca199..33c957e01 100644 --- a/doc/design/2.1-machine-model-and-configuration.md +++ b/doc/design/2.1-machine-model-and-configuration.md @@ -60,13 +60,17 @@ lifetime is one of: | Host attachments: console endpoints, the network endpoint, block files, and exported host directories | Host, matching the recorded identities | VM process | | Online processor and RAM activation targets | Host, within snapshot limits | Restore | -The host fixes the workload policy of a fresh boot. An optional nonzero -numeric UID and GID become the fixed workload identity, and an optional -one-shot or managed lifecycle selects whether the guest agents run one -workload or stay resident for sequential requests over the control console. -Both are command-line tokens that the machine profile owns and that caller -arguments cannot supply. A managed lifecycle requires a fixed workload -identity and a live, authenticated control console. +The host fixes the workload policy of a fresh boot. NVX's launchers name a +fixed workload identity, a nonzero numeric UID and GID, and a one-shot or +managed lifecycle, which selects whether the guest agents run one workload or +stay resident for sequential requests over the control console, with +command-line tokens of their own (see +[Effective Command Line](5.1-boot-protocol.md#512-effective-command-line)). A +managed lifecycle requires a fixed workload identity and a live, authenticated +control console. The machine profile also takes an identity and a lifecycle as +tokens of its own, which caller arguments cannot supply. NVX's guests select +neither from them, and fail closed when the profile names an identity that the +launcher's token does not. ## 2.1.3 Storage Configuration diff --git a/doc/design/4.2-guest-agents-and-workload-model.md b/doc/design/4.2-guest-agents-and-workload-model.md index cb2754024..667e4a538 100644 --- a/doc/design/4.2-guest-agents-and-workload-model.md +++ b/doc/design/4.2-guest-agents-and-workload-model.md @@ -12,15 +12,10 @@ obligations; the one exception is the time daemon, which [guest-agent guidelines](../guest-agent-guidelines.md) give the init's contract, with nvx-init as the worked example. -A layered root's guest runs one of two inits. nvx-init, NVX's PID 1 for -layered roots, runs the new launches, and the restores of platform captures, -in one VM-wide space, as Figure 4.2 shows; it also serves the Fresh boot that -a platform capture takes, which has no root yet. A layered launch that still -carries OpenVMM's older lifecycle token, `nvx_lifecycle=`, which NVX's CLI no -longer adds, runs the older init agent, whose sandbox agent gives each -execution private namespaces and a cgroup in the -[layered sandbox root](4.1-guest-filesystem-and-image-model.md#411-layered-sandbox-root). -The agent of the +A layered root's guest runs nvx-init, NVX's PID 1 for layered roots, which +runs its launches, and the restores of platform captures, in one VM-wide +space, as Figure 4.2 shows; it also serves the Fresh boot that a platform +capture takes, which has no root yet. The agent of the [blockless managed root](4.1-guest-filesystem-and-image-model.md#413-blockless-managed-root) is PID 1 in the guest's initramfs root, and runs its executions with nvx-init's plain-process core. @@ -55,9 +50,7 @@ Launching an execution belongs to the init, not to the library's capture or activation. nvx-init's executions, and those of the blockless managed root, are plain processes of their agent: each child takes its credentials and enters its working directory itself, in its agent's space, without a root of -its own or a placement barrier. The older layered sandbox root keeps its -execution entry, which enters private namespaces and the workload root, and -its placement barrier. +its own or a placement barrier. [Run the executions](../guest-agent-guidelines.md#run-the-executions) gives nvx-init's steps. @@ -68,5 +61,4 @@ resource boundary. An execution's process group, which its end, cancellation, timeout and reset kill, is a cleanup scope, not a namespace or a cgroup: a process that leaves the group escapes it. The init owns termination, capture admission and cleanup; no workspace or per-execution -enforcement of `nvx` supplies them. The older layered sandbox root keeps its -namespace and cgroup cleanup. \ No newline at end of file +enforcement of `nvx` supplies them. \ No newline at end of file diff --git a/doc/design/5.1-boot-protocol.md b/doc/design/5.1-boot-protocol.md index d864b016c..4686ae5a2 100644 --- a/doc/design/5.1-boot-protocol.md +++ b/doc/design/5.1-boot-protocol.md @@ -85,16 +85,18 @@ Launchers add runtime tokens of their own: the [boot tokens of a layered root](4.1-guest-filesystem-and-image-model.md#411-layered-sandbox-root), one share token per host-share child, the number of host mappings for a blockless managed root, and a request to create the workload account. A -launcher that passes OpenVMM neither the workload identity nor the lifecycle -may name them for a blockless guest with tokens of its own, -`nvx_identity=UID:GID` and `nvx_mode=one-shot` or `nvx_mode=managed`, which -the guest's `/init` reads as it reads the host-owned tokens. In the one-shot -mode, `/init` runs the workload that `nvx_entrypoint=` and its `nvx_arg=` -tokens name once, as that identity, after the network and the shares, and -powers the VM off with the workload's status. Each `nvx_arg=` token carries -one argument, which cannot contain a space, since `/init` splits the command -line on spaces. A caller may also declare the RAM-backed overlay layout, whose -overlay upper the guest agents keep on a tmpfs, with exactly one +launcher names the workload identity and the lifecycle of a blockless guest +with tokens of its own, `nvx_identity=UID:GID` and `nvx_mode=one-shot` or +`nvx_mode=managed`, which the guest's `/init` reads. It ignores the lifecycle +token that OpenVMM appends, and fails closed on OpenVMM's identity tokens: a +boot that has them stops with `NVX-WORKLOAD-ERROR` and status 125 before it +starts anything, unless `nvx_identity=` names the same UID and GID. In the +one-shot mode, `/init` runs the workload that `nvx_entrypoint=` and its +`nvx_arg=` tokens name once, as that identity, after the network and the +shares, and powers the VM off with the workload's status. Each `nvx_arg=` +token carries one argument, which cannot contain a space, since `/init` splits +the command line on spaces. A caller may also declare the RAM-backed overlay +layout, whose overlay upper the guest agents keep on a tmpfs, with exactly one `nvx_overlay_upper=ramfs` token; any other form of that token is rejected. Embedded NULs are rejected, and the complete NUL-terminated command line must diff --git a/doc/project-structure.md b/doc/project-structure.md index fc726a13e..d63adc7bd 100644 --- a/doc/project-structure.md +++ b/doc/project-structure.md @@ -232,8 +232,8 @@ verification of the pinned OpenVMM release. initramfs builds, plus the Azure Linux variants of the container-entry scripts. The ten compiled `/sbin/nvx-*` programs live under `nvx-rs/src/guest` and `nvx-rs/src/bin`. `init` controls early boot and then starts an -initramfs program named by `nvx_exec=`, the Alpine-only `nvx-init-agent` -sandbox profile, `nvx-managed-agent` for a managed workload, a one-shot +initramfs program named by `nvx_exec=`, `nvx-init` for a layered root, +`nvx-managed-agent` for a managed workload, a one-shot workload named by `nvx_entrypoint=`, a virtio restore probe for benchmark profiling, or the interactive shell. On an interactive boot, `init` prints a stable distribution boot marker for Alpine and Azure Linux, and @@ -242,7 +242,8 @@ the container launch and entry scripts that only the Alpine initramfs installs; the entry script stages Alpine's musl loader and `setpriv` inside the workload's root. -The sandbox scripts resolve fixed virtio-blk roles through sysfs, mount the +The older sandbox scripts, `nvx-init-agent` and the container scripts, which +`init` no longer starts, resolve fixed virtio-blk roles through sysfs, mount the EROFS `distro` layer as the overlay's only lower layer, under the upper that the host declares, an ext4 scratch disk or a tmpfs, place the workload in its cgroup before release, construct its mount/PID/UTS namespaces, enter its diff --git a/guest/common/init b/guest/common/init index 38e818f7e..807e5963d 100644 --- a/guest/common/init +++ b/guest/common/init @@ -35,13 +35,6 @@ case "$cmdline" in *" nvx_sandbox=1 "*) sandbox_mode=true ;; *) sandbox_mode=false ;; esac -new_init=false -if [ "$sandbox_mode" = true ]; then - case "$cmdline" in - *" nvx_lifecycle="*) ;; - *) new_init=true ;; - esac -fi fatal() { echo "$*" >&2 @@ -160,17 +153,17 @@ nvx_exec= nvx_exec_seen=false workload_uid= workload_gid= +openvmm_uid= +openvmm_gid= workload_lifecycle=one-shot control_tty= workload_account= entrypoint= entrypoints=0 -# Kernel command-line tokens are deliberately split on spaces. OpenVMM's -# options give the lifecycle and the workload identity tokens of their own; a -# blockless launch of the CLI gives the CLI's instead, nvx_mode=one-shot or -# nvx_mode=managed and nvx_identity=UID:GID, and names its one-shot workload -# with nvx_entrypoint= and nvx_arg= tokens, whose values become the positional -# parameters, in order. +# Kernel command-line tokens are deliberately split on spaces. A blockless +# launch of the CLI gives nvx_mode=one-shot or nvx_mode=managed and +# nvx_identity=UID:GID, and names its one-shot workload with nvx_entrypoint= +# and nvx_arg= tokens, whose values become the positional parameters, in order. # shellcheck disable=SC2086 set -f set -- @@ -182,10 +175,10 @@ for token in $cmdline; do nvx_exec=${token#*=} [ -n "$nvx_exec" ] || fatal "nvx_exec must not be empty" ;; - nvx_workload_uid=*) workload_uid=${token#*=} ;; - nvx_workload_gid=*) workload_gid=${token#*=} ;; nvx_identity=*) read_workload_identity "${token#*=}" ;; - nvx_lifecycle=* | nvx_mode=*) workload_lifecycle=${token#*=} ;; + nvx_workload_uid=*) openvmm_uid=${token#*=} ;; + nvx_workload_gid=*) openvmm_gid=${token#*=} ;; + nvx_mode=*) workload_lifecycle=${token#*=} ;; nvx_entrypoint=*) entrypoint=${token#*=} entrypoints=$((entrypoints + 1)) @@ -196,7 +189,14 @@ for token in $cmdline; do esac done set +f -if [ -n "$nvx_exec" ] && [ "$new_init" = false ]; then +# OpenVMM's --microvm-workload-identity appends nvx_workload_uid= and +# nvx_workload_gid=, which no longer name the workload's identity. A boot that +# has them fails closed unless nvx_identity= names the same one. +if [ -n "$openvmm_uid$openvmm_gid" ] && + [ "$openvmm_uid:$openvmm_gid" != "$workload_uid:$workload_gid" ]; then + workload_fatal "nvx_identity must name OpenVMM's workload identity" +fi +if [ -n "$nvx_exec" ] && [ "$sandbox_mode" = false ]; then case "$nvx_exec" in /*) ;; *) fatal "nvx_exec must be an absolute initramfs path" ;; @@ -317,11 +317,6 @@ if [ -n "$nip" ]; then fi fi if [ "$sandbox_mode" = true ]; then - if [ "$new_init" = false ]; then - [ -x /sbin/nvx-init-agent ] || - workload_fatal "selected guest cannot act as sandbox control" - exec /sbin/nvx-init-agent - fi [ -x /sbin/nvx-init ] || workload_fatal "selected guest cannot act as sandbox control" # nvx-time boot returns before its child publishes the daemon PID. diff --git a/nvx-rs/README.md b/nvx-rs/README.md index cdaf2ac36..9c8437b89 100644 --- a/nvx-rs/README.md +++ b/nvx-rs/README.md @@ -44,11 +44,11 @@ nvx-rs/ │ ├── hypervisor.rs │ ├── error.rs nvx::Error, its ErrorKind, and its stable ErrorCode │ ├── host/ feature `host` (default): the runtime that every launch shares, VM -│ │ directories and configurations, artifacts, launch, network and egress policy -│ │ files, filesystem, NVXS connection, session, process, platform, templates' +│ │ directories and configurations, artifacts, network and egress policy files, +│ │ filesystem, NVXS connection, session, process, platform, templates' │ │ manifests and compatibility keys, outcome reports, file seals, and the -│ │ sandbox launch profile of `nvx-rs sandbox`, its distro and scratch bindings, -│ │ and its prepared launches; with feature `cli`, the `nvx-rs` CLI +│ │ sandbox launch profile of `nvx-rs sandbox` and its distro and scratch +│ │ bindings; with feature `cli`, the `nvx-rs` CLI │ ├── testing/ feature `testing`: the fake OpenVMM │ ├── guest/ feature `guest`: the programs that run in Linux guests, the guest's NVXS │ │ connection, the libraries through which a guest init or agent opens the @@ -119,17 +119,15 @@ changes only when the module's purpose does. | [`nvx::host::config`](#nvxhostconfig) | `VmConfig` and `CaptureConfig`: pure configuration, cold-boot kernel/OpenVMM arguments, portable length checks, and capture identity | | [`nvx::host::restore`](#nvxhostrestore) | A caller-driven restore to Ready without I/O or a clock, and its peer-verified restore-ready listener | | [`nvx::host::artifacts`](#nvxhostartifacts) | Locating the OpenVMM executable and the guest artifacts that a VM runs, and their runtime identity | -| [`nvx::host::launch`](#nvxhostlaunch) | `LaunchSpec` and `VmmLaunch`: the OpenVMM command line of a managed microVM, its checks, and the inputs that it shares with an adapter's launch | | [`nvx::host::network`](#nvxhostnetwork) | The one lowering of a network posture into OpenVMM options, for the CLI and external orchestrators | | [`nvx::host::network::policy`](#nvxhostnetworkpolicy) | Egress policy files and their compilation into OpenVMM's native rules | | [`nvx::host::filesystem`](#nvxhostfilesystem) | The planner of the host paths that every NVX caller exports to the guest through OpenVMM's virtio-fs device | | [`nvx::host::nvxs`](#nvxhostnvxs) | Compatibility paths for the host NVXS connection and its retained cause details | | [`nvx::host::transport`](#nvxhosttransport) | A verified VM endpoint connection and the blocking host NVXS driver, with the existing timed transport | | [`nvx::host::process`](#nvxhostprocess) | Verified references to OpenVMM processes, through which callers wait for them and end them | -| [`nvx::host::supervisor`](#nvxhostsupervisor) | The transitional public path of the low-level spawn of a VMM, which code that stays uses until SL6.3 (#1122) | | [`nvx::host::seal`](#nvxhostseal) | `FileSeal`: what the host reports about an open file from its metadata alone, without reading it | | [`nvx::host::platform`, `report`](#nvxhostplatform-report) | Linux and Windows primitives, and OpenVMM's outcome report | -| [`nvx::host::sandbox`](#nvxhostsandbox) | The launch profile of `nvx-rs sandbox`: its plans, live shares, option checks, bindings, and prepared launches | +| [`nvx::host::sandbox`](#nvxhostsandbox) | The launch profile of `nvx-rs sandbox`: its layer, live shares, option checks, and bindings | | [`nvx::guest`](#nvxguest) | The programs that run in Linux guests; see [Guest binaries](#guest-binaries) | | [`nvx::guest::mappings`](#nvxguestmappings) | `Mapping` and `MappingTable`: host mappings, and the account of a host's mapping table, for any guest init or agent | | [`nvx::guest::nvxs`](#nvxguestnvxs) | `GuestConnection`: the guest's role of the NVXS session over a byte stream, such as the control console | @@ -571,8 +569,15 @@ Read the resolved NIC or capture context, or build an activation record, before consuming `Prepared` into the command. The compiled [external-launch example](examples/external-launch.rs) consumes -this command, launches it with the transitional `supervisor::spawn_vm`, checks the -cold guest's boot marker and shell response, then asks it to exit cleanly. +this command, starts it with a minimal launcher of its own on `std::process`, +checks the cold guest's boot marker and shell response, then asks it to exit +cleanly. Its launcher passes the VMM no descriptor or handle besides its +standard streams, and starts it neither in a new session with the owner's bits +cleared from its umask on Linux nor in its log's job on Windows: a production +launcher adds what the command's contract asks, as `vm::spawn` does, and the +example's docs list what it leaves out. It holds the VMM through a `ProcessRef` +whose start time it reads as `ProcessRef` defines it: the `starttime` field of +`/proc//stat` on Linux, and the process's creation time on Windows. It requires the VMM's actual exit status 0 and a successful outcome with complete teardown. Build with `cargo build --locked --example external-launch` and supply the backend, OpenVMM, @@ -598,16 +603,28 @@ failure is reported with the original error. Spawn removes no output path. Dropping `Launched` closes the process handle but does not stop the VM. On Linux the detached child gets the caller's umask with the owner's bits -cleared; the caller's own umask is unchanged. On Windows it starts without a +cleared; the caller's own umask is unchanged. The umask still narrows the +group's and others' permissions to the files that OpenVMM creates, such as its +outcome report, its snapshots, and the files that a guest creates in a share, +but never the owner's, which a umask such as 0400 would clear, so that the +caller could not read the report, nor OpenVMM use its own files. In a directory +that has a default POSIX ACL, Linux narrows those permissions by the ACL +instead of the umask. On Windows it starts without a console in its log-named job, refusing a taken or untrusted job name, and leaves the caller's job when the runtime requests breakaway. The inheritance lock is internal to spawn. Windows callers coordinate their own other spawns, for example with their own lock around VM and other process starts: an unrelated process started during the inheritance window may inherit handles, -but never the VM's job claim. +but never the VM's job claim. Windows' process-creation API limits the VM +directory, in which OpenVMM starts, to 258 UTF-16 code units in its absolute +path, including any verbatim prefix and trailing separator: the spawn refuses +a longer one before it creates a process. The [VM tests](src/host/vm/tests.rs) cover cleanup after child creation, -ownership, caller survival, the Linux umask, and Windows job claims. Their +ownership, caller survival, the Linux umask, and Windows job claims, and the +[spawn tests](src/host/spawn/tests.rs) what the VMM inherits while other +threads spawn, the working-directory limit, the survival of a VMM whose +launcher crashed, and the waits and cleanup of its child. Their ignored cold-spawn smoke test runs the current library against explicit `NVX_VM_SPAWN_OPENVMM`, `NVX_VM_SPAWN_KERNEL`, and `NVX_VM_SPAWN_INITRAMFS` paths, using a fresh `NVX_VM_SPAWN_PROOF_DIR` and `NVX_HYPERVISOR` or the @@ -628,34 +645,26 @@ mistyped, or repeated field, truncated bytes, and a capture that is not a template's: a platform-tier capture for clones, with a read-only `distro` whose identity is unbound, a fresh scratch, and the dormant filesystem slot. -`CaptureClass` holds the invariant inputs of a capture, known before it: the -runtime, the guest's build, the backend, the requested CPU profile, the -processors, those online at boot, and RAM, the geometries of the disks, the -network, of which it takes what the template captures alone, its NIC at the -guest's address under the complete deny-all policy, and the adapter's other -`Invariants`. - -`SnapshotKey::new` checks that the capture made the machine that its class -requested, and derives the template's compatibility key, from metadata alone: -the runtime and contract versions through which its clones are restored, the -backend, CPU profile and effective CPUID, topology, rates, resources, kernel -command line, disks, and devices of its machine, with the NIC's addresses and -the identity of the complete deny-all policy under which the capture attached -it, which each clone's restore replaces with the clone's own, so that a template -with a NIC serves the clones of every network policy at its guest's address. - -A key and a class each have one canonical, versioned encoding, whose -domain-separated SHA-256 digest is its identity, a `KeyIdentity`, or a -`ClassIdentity`; the +`SnapshotKey` checks that a capture made the machine that its context, the +`CaptureContext` with which `vm::create` prepared it, requested, and derives +the template's compatibility key from metadata alone: the runtime and contract +versions through which its clones are restored, the backend, CPU profile and +effective CPUID, topology, rates, resources, kernel command line, disks, and +devices of its machine, with the NIC's addresses and the identity of the +complete deny-all policy under which the capture attached it, which each +clone's restore replaces with the clone's own, so that a template with a NIC +serves the clones of every network policy at its guest's address. + +A key has one canonical, versioned encoding, whose domain-separated SHA-256 +digest is its identity, a `KeyIdentity`; the [conformance vectors](tests/vectors/README.md) of version 1, -`tests/vectors/snapshot-key-v1.txt`, pin both encodings and identities. The -key's encoding, version 1, leaves the network policy out, since templates -capture their NIC under the complete deny-all policy. +`tests/vectors/snapshot-key-v1.txt`, pin its encoding and identity. The key's +encoding, version 1, leaves the network policy out, since templates capture +their NIC under the complete deny-all policy. The key's canonical encoding and identity use the snapshot names and omit `guest_build`. Version 1 changes in place: no snapshot outlives a deployment, -and no reader or migration of the prior encoding is supported. `CaptureClass` -still records the legacy capture inputs, including its guest build. +and no reader or migration of the prior encoding is supported. The persisted key includes the complete per-snapshot time contract and must agree with the manifest's time. Its compatibility identity excludes capture @@ -745,8 +754,8 @@ percent of the VM's memory, 50 when the caller has no other A `NetworkConfig` holds the `NetworkPolicy` that OpenVMM enforces and the NIC's address, an `nvx::net::Ipv4Cidr`, if the caller names one: without one, a cold -boot takes `DEFAULT_GUEST_NETWORK`, `10.0.0.2/24`, which `nvx::host::launch` -also exports, and a restore the snapshot's. A `NicConfig` is the NIC that a VM +boot takes `DEFAULT_GUEST_NETWORK`, `10.0.0.2/24`, and a restore the +snapshot's. A `NicConfig` is the NIC that a VM settled on, with its address. A `Boot` holds the caller's own kernel command-line tokens, to which NVX adds those of what it lowers: a VM that boots needs one, and a VM that a snapshot restores takes none. @@ -785,8 +794,8 @@ entries. The table's `HostMap` and its wire codecs are accessible through `nvx::protocol::maps`, and remain re-exported from `nvx::protocol`. `COMMAND_LINE_BUDGET`, in `nvx::host::config`, is the 1,024-byte x86 kernel -command-line budget, including the terminating NUL. The legacy launch checks -continue to use it. +command-line budget, including the terminating NUL. The CLI checks the command +lines that it launches against it. The internal whole-config renderer used by `vm::create` renders a cold boot's OpenVMM arguments from `Runtime` and `VmConfig`. Its borrowed control endpoint, @@ -818,10 +827,12 @@ every host, before any host change, with 2048 UTF-16 code units reserved for the runtime's files, launch options, and VM output names. The reserve foresees file and endpoint names of up to 260 code units, not arbitrary runtime paths or CPU-profile strings. The whole renderer checks the complete actual command -line without a reserve too. `PROCESS_COMMAND_LINE_LIMIT`, -`process_command_line_units` and `fit_process_command_line` now belong to -`nvx::host::config`; their existing `nvx::host::launch` paths remain usable, -with the same Windows quoting and terminating-NUL count. +line without a reserve too. `process_command_line_units`, in +`nvx::host::config`, counts the Windows command line that runs a program with +its arguments, in UTF-16 code units, as Windows quotes them, with the +terminating NUL, and `fit_process_command_line` checks it against +`PROCESS_COMMAND_LINE_LIMIT`, on every host, with the reserve that its caller +keeps. A `Boot` may also hold a `CaptureConfig`, what a capture of the VM adds to the VM's configuration, which keeps the rest, such as the NIC, the console, and the @@ -933,47 +944,6 @@ manifest, in a checkout's `build/` and `openvmm/target/release/` or a release's executable, kernel, and Alpine initramfs from the install metadata, a release's `SHA256SUMS` or a checkout's build provenance, without hashing an artifact. -### `nvx::host::launch` - -`LaunchSpec` and OpenVMM's command line: the managed lifecycle, the -`WorkloadIdentity`, the 1024-byte kernel command line, its reserved parameters, -the 32,767-character limit of a Windows command line, and `LaunchSpec::check`, -which every command line applies to the memory size, the non-root identity, the -hostname, the guest network, and the kernel parameters. - -`VmmLaunch` holds the inputs that a cold launch's command line names besides the -VM's devices and its kernel command line: the OpenVMM executable, the -hypervisor, the memory, the kernel, the initramfs, the endpoint of the control -console, and the outcome report. `VmmLaunch::arguments` renders them for a -`LaunchSpec`'s launch, after its managed lifecycle, and for an adapter's -prepared launch, after its devices, and `LaunchSpec::vmm_launch` gives a -specification's. - -`process_command_line_units` counts the Windows command line that runs a program -with its arguments, in UTF-16 code units, as Windows quotes them, and -`fit_process_command_line` checks it against `PROCESS_COMMAND_LINE_LIMIT`, on -every host, with the reserve that its caller keeps. The helpers live in -`nvx::host::config` and remain re-exported here for existing callers. -`LaunchSpec::check_process_command_line`, which a caller runs before it -provisions a VM, keeps 1024 code units for what may change before the VM -starts. - -`VmmLaunch::RESERVE`, 2048 code units, is what the planning of an adapter's -launch keeps for the rest of the launch: the executable and the arguments that -`VmmLaunch::arguments` renders besides the kernel command line, with paths of up -to 260 code units, Windows' `MAX_PATH`, and the longer names by which a -prepared launch gives OpenVMM the checked files of the plan's block devices. - -`kernel_parameters_problem` checks the extra kernel parameters as the guest's -kernel reads them, with `nvx::protocol::cmdline`: trimmed, split at every byte -that the kernel takes for whitespace, including the 0xa0 that ends the UTF-8 -encoding of characters such as `à`, with names that match whether they spell a -dash or an underscore. It refuses NUL characters, quotes, and the `--` -delimiter, after which the kernel would leave the `hostname=` parameter that -follows the extra parameters to init, and the reserved parameters: those named -`tsc` or `hostname`, with or without a value, and those whose names start with -`nvx_` or `nvx-`. - ### `nvx::host::network` `NetworkPolicy` and `openvmm_arguments`, the one lowering of a network posture @@ -1086,64 +1056,6 @@ holds what the process's children read once it reaped them, as the kernel accounts them to their parent, while Windows counts the process's own reads alone. -### `nvx::host::supervisor` - -The transitional public path of the low-level spawn of a VMM, which code that -stays uses until SL6.3 (#1122): `spawn` and `spawn_vm`, the `VmmCommand` that -they start and the `Launched` VMM that they return, `Inherited`, and -`spawn_command`. The module re-exports them and has no code of its own: it -supervises nothing. [`nvx::host::vm`](#nvxhostvm) creates, spawns, reattaches -to, and stops a VM in a private directory that its caller owns. SL6.3 deletes -`spawn_command` and the checked-file handoff of `PreparedLaunch`, and settles -the launcher of the `external-launch` example: the last users of this path, -besides the tests of the spawn in `tests/supervisor.rs`. - -`spawn_vm` starts a caller-owned command without a store, with the same strict -Windows log-job requirement as `vm::spawn`: an untrusted job name or a log that -cannot name a job is refused before child creation. The legacy `spawn` keeps -its fallback without a job for those logs and names. Other spawn behavior is -shared. If setup fails after a child is created, including failed process -verification, `Launched::kill_and_confirm_exit` waits without a deadline for -its exit and reaps it on Unix. The caller reports any cleanup error; the -legacy `kill_and_wait` remains time-bounded and may release a running child. - -`VmmCommand::inherit` lists the descriptors (Linux) or handles (Windows) that -the VMM inherits under their numbers, besides its standard streams, and it -inherits nothing else of the caller's: on Linux, they must be close-on-exec, a -flag that only the VMM's own process clears; on Windows, they join the VMM's -explicit handle list. - -On Linux, the VMM starts with the caller's umask without the owner's bits, -which only the VMM's own process clears. The umask still narrows the group's -and others' permissions to the files that OpenVMM creates, such as its outcome -report, its snapshots, and the files that a guest creates in a share, but never -the owner's, which a umask such as 0400 would clear, so that the caller could -not read the report, nor OpenVMM use its own files. The usual umasks, such as -022 and 077, keep the owner's bits, so they apply unchanged. In a directory -that has a default POSIX ACL, Linux narrows the permissions of what is created -there by that ACL instead of the umask, so the ACL decides them, the owner's -included. - -On Windows, the VMM's standard handles and its allowed handles are inheritable -while it is created, and a process that another thread creates meanwhile -without a handle list, as `std::process::Command` creates one, inherits them -and holds them until it exits: the read end of the VMM's capability pipe, its -log, and the handles that it allows. The VMM runs in a job object of its own, -named after its log, whose processes are the launch's claim while such a -process holds the log, so a recovery still ends the launch once the VMM exits. -Each spawn of a VMM holds the crate's spawn lock throughout that window, so a -caller that launches VMMs spawns its other processes with `spawn_command`, -which spawns a `std::process::Command` under it. On other hosts, the lock holds -nothing. - -#### Windows working-directory limit - -Windows' process-creation API limits `VmmCommand::working_dir` to 258 UTF-16 -code units in its absolute path, including any verbatim prefix and trailing -separator. It may name a shorter directory than those of the VMM's other files. -A raw `spawn` refuses an overlong working directory with `InvalidInput`, and -never changes the requested directory. - ### `nvx::host::seal` `FileSeal`: what the host reports about an open file from its metadata alone, @@ -1183,48 +1095,14 @@ it, and NVX does not depend on an asynchronous I/O runtime. ### `nvx::host::sandbox` -The launch profile of `nvx-rs sandbox`, with native input validation: -`SandboxLaunch`, with the distro `SandboxLayer`, an EROFS layer or a GPT disk, -the one layer that a sandbox takes, its `SandboxUpper`, the ext4 scratch disk -or RAM, the entrypoint, arguments, hostname, workload identity, and -`SandboxSettings` resource limits (`Option` for memory and `Option` -for processes), and the `SandboxPlan` that it makes of them. - -`plan`: `SandboxPlan`, the typed plan of a cold layered launch, which is -persistable in a versioned serde form: one read-only `distro` block device, an -EROFS filesystem or a GPT disk with the identity that the guest expects of it, -if any, an upper, the ext4 scratch disk or RAM, the launch settings, the -`ExportPlan` of its live shares, and optionally a `SandboxNetwork`, the guest -network and its typed `NetworkPolicy`, whose proxy's URL as the guest sees it -the plan reports. The plan of an adapter that boots its own guest, -`SandboxPlan::adapter`, holds no launch settings and no export plan. - -A plan knows only the `distro` and `scratch` `BlockRole`s, and gives, for every -combination, its `--microvm-sandbox-block` arguments, with -`--microvm-workload-identity` when it has launch settings, with no scratch block -device for a RAM upper, and those of its network, which `nvx::host::network` -lowers, and its kernel command line, with the boot tokens of -`nvx::protocol::root`, within the 1024-byte budget that OpenVMM leaves it, -refusing one whose root tokens the guest's parser refuses. Both deliver the live -shares as the managed lifecycle does: the arguments export them at -`nvx::host::filesystem::GUEST_EXPORT`, the command line announces the number of -their mappings with `nvx_maps=`, and `mapping_table` gives the ordered table -that `start` and `sandbox run` send once the guest agent is ready. - -An adapter's plan renders the caller's tokens before those of its root, and -refuses tokens that its root or OpenVMM's shares own. -`SandboxPlan::check_adapter_launch` checks, as an adapter plans its launch, that -the plan's arguments, those that export the adapter's shares, and the kernel -command line leave `nvx::host::launch::VmmLaunch::RESERVE` of the limit of a -Windows command line to the rest of the launch, so that planning refuses a plan -whose network rules or mapped, hidden, or allowed paths leave it too little. +The launch profile of `nvx-rs sandbox`, with native input validation: the +distro `SandboxLayer`, an EROFS layer or a GPT disk, the one layer that a +sandbox takes, and the workload identity that `--workload-user` names. `mount`: `SandboxMount` requests of live shares of host directories and files, their guest targets and the reserved ones, their access policies and owner, and -the checks of their host paths; `plan_mounts`, which plans them through -`nvx::host::filesystem` for the container root, and `boot_shares`, which renders -a plan as the OpenVMM arguments and boot tokens of one share or of the aggregate -of several, as the one-shot lifecycle delivers them. +the checks of their host paths; and `plan_mounts`, which plans them through +`nvx::host::filesystem` for the container root. `options`: the checks that `nvx-rs sandbox` and `nvx-rs run` apply to their options: the attribution of `--mount-deny`, `--mount-allow`, and `--mount-write` @@ -1247,17 +1125,6 @@ a nonzero multiple of 512 bytes, logical blocks of 512 bytes, and physical blocks of the I/O block size of their filesystem on Linux, and of 4096 bytes on Windows. -`prepared`: `PreparedLaunch`, the launch of a plan with the checked files of its -block devices, which renders the plan's OpenVMM arguments and kernel command -line, naming the checked files as `/proc/self/fd/N` on Linux and by their final -paths on Windows, lists the descriptors or handles that the VMM inherits through -`VmmCommand::inherit`, and releases the checked files, and on Windows the VMM's -copies of them, at authenticated Ready; see [Checked handoff](#checked-handoff). -For an adapter's plan, `PreparedLaunch::adapter_arguments` renders the complete -arguments of the launch: the plan's, those that export the adapter's shares, -and those of the launch's `VmmLaunch`, and refuses a command line that Windows -would not start OpenVMM with. - ### `nvx::guest` The programs that run in Linux guests; see [Guest binaries](#guest-binaries). @@ -1867,9 +1734,9 @@ path, as it records every path, with the binding of the layer there, under `distro_binding` in `config.json`. When the path given is a link to a layer of another name, `start` checks the manifest beside the link's target, so `provision` verifies the layer there as well, and it records only a binding -that the check accepts there. `start` checks the binding before it launches -OpenVMM, and hands OpenVMM the checked layer; see -[Checked handoff](#checked-handoff). `run` and `start` +that the check accepts there. `start` checks the binding just before it +launches OpenVMM, which opens the layer at its path; see +[Images by path](#images-by-path). `run` and `start` launch the image of the format and the identity that the binding records, an EROFS layer or a GPT disk, over a scratch or a RAM upper, the layered root that nvx-init assembles. A configuration @@ -1879,14 +1746,25 @@ after its distro binding was made: its length differs`; both add `deprovision the sandbox and provision it again`, since a new provision verifies the layer again. -## Checked handoff - -A launch hands the VMM exactly the objects that its checks found, never a path -that something could replace between the check and the VMM's open of it. - -`SandboxPlan::validated` rejects disk image paths containing commas, semicolons, -or improperly nested square brackets with `ErrorKind::Unsupported`, including -the resolved paths that `provision` records. Balanced brackets, as in +## Images by path + +A launch names its images by path: OpenVMM opens the distro layer and the +scratch image at the paths that the VM's configuration holds. Their checks, +`DistroBinding::check` and `ScratchBinding::check`, find a staged image +unchanged from its metadata alone and return it open, but what takes an image's +path after its check is what OpenVMM opens. Callers therefore stage their +images in a directory that only their own account can write, as +[Distro bindings](#distro-bindings) and the docs of `nvx::host::sandbox::distro`, +`nvx::host::sandbox::scratch`, and `nvx::host::seal` require, so that no other +account can replace a checked image. On Windows, a checked distro layer also +denies other opens for writing or deleting the layer while it is held, and the +scratch image's guard denies deleting and renaming the image. `nvx-rs sandbox +start` and `run` check the images just before they create the VM, and hold the +checked files until the guest is ready, or, for `run`, until its VM has exited. + +`nvx-rs sandbox` rejects disk image paths containing commas, semicolons, or +improperly nested square brackets with `ErrorKind::Unsupported`, including the +resolved paths that `provision` records. Balanced brackets, as in `layer[1].erofs` or `layer[[1][2]].erofs`, are supported; an unclosed `[`, an unmatched `]`, or reversed `][` is not. `run` and `provision` apply these checks before launching OpenVMM or recording a sandbox, and `start` checks recorded @@ -1912,92 +1790,39 @@ handle is a guard that holds no access to the image's content, shares reading and writing, and denies deleting and renaming the image, and the check makes sure that no other process has the image open for writing, as another sandbox's VMM does, then opens the image for writing, sharing every access, as the claim's -handle. The VMM inherits the claim's handle and the guard: so it holds the -claim, and the guard the image's path, from its start, even if the launcher ends -before the VMM has opened the image. Claims exclude the launches of NVX, not a -process that writes the image on its own. - -`PreparedLaunch::new` takes a `SandboxPlan`, the `CheckedDistro` that -`DistroBinding::check` returned for its image, and, for an upper on scratch, -the `CheckedScratch` that `ScratchBinding::check` returned, and refuses files -that are not those of the plan. It renders the plan's OpenVMM arguments and -kernel command line, holds the checked files, and lists the descriptors that -the VMM inherits: - -- On Linux, the VMM inherits the checked descriptors through - `VmmCommand::inherit`, `PreparedLaunch::inherited`, and the arguments name - each `/proc/self/fd/N`, `distro:file:/proc/self/fd/N,ro` and - `scratch:file:/proc/self/fd/M`, which OpenVMM opens: the link reaches the - checked file whatever later takes its path. Every other descriptor stays - close-on-exec, as the supervisor guarantees. A checked descriptor that took a - standard stream's number, as when the caller had closed that stream, first - moves above the standard streams, with its open file description and its - claim. -- On Windows, the arguments name each checked file by its final path, the - absolute `\\?\` path that its handle gives, which resolves every link that - the plan's path went through, and does not depend on the VMM's directory. - The checked files hold that path in place: the distro layer's file denies - writing and deleting the layer, the scratch image's guard denies deleting and - renaming it, and NTFS renames no directory while a file below it is open; - OpenVMM's opens share them. The VMM also inherits the handles of the checked - files and the scratch image's claim, though no argument names a handle, - through `VmmCommand::inherit` and `PreparedLaunch::inherited`, so that its - copies hold the paths and the claim from its start: a launcher that ends - before OpenVMM has opened the images, as when it crashes, closes only its own - checked files, and a replacement at their paths still fails until OpenVMM - exits; `start` never commits such a launch, and the sandbox's next transition - ends its VMM. At authenticated Ready, `PreparedLaunch::release` closes - OpenVMM's copies of the checked files with `DuplicateHandle` and - `DUPLICATE_CLOSE_SOURCE`, through the VMM's process handle that its launch - holds, once a copy of each is found to hold the checked file's object, and - then the checked files themselves, so that the guards are released while - OpenVMM keeps the claim. A final path that OpenVMM could not take as the - path of a raw image, with a comma, semicolon, or unbalanced brackets, or - ending in `.vhd`, `.vhdx`, `.vmgs`, or `.iso`, which a link, the directory of - a relative path, or the case of a name on disk can bring, fails the preparation - with `ErrorKind::Unsupported`. - -Images are raw: OpenVMM attaches a file's bytes as they are, and chooses a -container format by the file's extension, which `/proc/self/fd/N` lacks, so a -caller whose images use a container format, such as a fixed VHD, stages a raw -file. The paths in the arguments hold for the launch alone: the plan, which -callers persist, keeps the real paths. The caller adds its own arguments, or, -for an adapter's plan, has `PreparedLaunch::adapter_arguments` render the -complete arguments of its launch, starts the VMM, and releases the prepared -launch with `PreparedLaunch::release` once the guest reports authenticated -Ready, or drops it once it has torn down a launch that failed; the VMM's own -opens then keep the images, and what it inherited keeps the scratch image's -claim. Staging, the checks, and the preparation of a launch read no content of -the images. +handle. The VMM inherits the claim's handle, or for a restore, whose scratch +image OpenVMM opens only while no other handle may write it, the guard, which +then holds the claim: so it holds the claim from its start, even if the +launcher ends before the VMM has opened the image. Claims exclude the launches +of NVX, not a process that writes the image on its own. + +Images are raw: OpenVMM attaches a file's bytes as they are, and would choose a +container format by a file's extension, which the checks of a launch refuse, so +a caller whose images use a container format, such as a fixed VHD, stages a raw +file. Staging and the checks read no content of the images. `nvx-rs sandbox provision` records the scratch image's binding under `scratch_binding` in `config.json`, at the image's resolved path, and `start` -checks it after the distro binding, and launches OpenVMM through a prepared -launch, which it releases once the guest is ready, or drops once OpenVMM has -exited. A scratch image that changed fails as a distro layer does, such as with -`IMAGE changed after its scratch binding was made: its length differs; +checks it after the distro binding, and launches OpenVMM, which inherits the +image's claim. A scratch image that changed fails as a distro layer does, such +as with `IMAGE changed after its scratch binding was made: its length differs; deprovision the sandbox and provision it again`, and one that another sandbox claims with `another sandbox claims the ext4 scratch image IMAGE`. `run` binds the scratch image itself, checks the binding and claims the image as `start` does, and holds the claim until its VM exits. A sandbox whose upper is in RAM, `--upper ram`, has no scratch image: `provision` records `null` for the image and its binding, and neither `start` nor `run` binds or claims one. - ## Sandbox input contract The sandbox launch profile specifies its own inputs rather than emulating Python. -UID/GID and workload resource limits contain only ASCII decimal digits, without -signs, whitespace, underscores, or Unicode digits. Each UID/GID is in -`1..4294967294`, memory in `1..18446744073709551615` bytes, and the process -limit in `1..4194303`. A layer UUID is canonical lowercase hyphenated text; -braces, a prefix, uppercase, and unhyphenated forms are refused. - -API change: `nvx::host::sandbox::{Int, IntError}` are no longer exported. -`SandboxSettings::memory_max` is `Option` and `pids_max` is `Option`; -`None` leaves the respective limit unset, and zero is refused. Persisted plans -still encode these limits as canonical positive decimal strings. The CLI's -sandbox configuration records hold no limits: the VM is the limit of a -sandbox's workload. See [Specification tests](#specification-tests). +A UID or GID contains only ASCII decimal digits, without signs, whitespace, +underscores, or Unicode digits, and is in `1..4294967294`. A layer UUID is +canonical lowercase hyphenated text; braces, a prefix, uppercase, and +unhyphenated forms are refused. + +API change: `nvx::host::sandbox::{Int, IntError}` are no longer exported. The +CLI's sandbox configuration records hold no resource limits: the VM is the +limit of a sandbox's workload. See [Specification tests](#specification-tests). Managed exec environment files and distro manifests are strict UTF-8 JSON, read with `serde_json`; non-finite numbers, numbers beyond the range of a @@ -2015,14 +1840,14 @@ formatting and retain native JSON/I/O error details rather than Python's Entrypoints, workload arguments, and guest mount targets must each remain one unquoted kernel token: no whitespace, control characters, double quotes, or -UTF-8 byte that Linux treats as whitespace (including `0xa0`). Caller-supplied -sandbox command lines use Linux's parameter splitting, including quoted values -and hyphen/underscore name equivalence. The exact trimmed text is validated -before rendering, and the guest bootstrap's whitespace-only token view is -checked too. Reserved `nvx_*` and `virtfs_*` names cannot be hidden inside -quoted values or behind leading Unicode whitespace; a `--` or unclosed quote -that hides the appended root tokens is refused. Adapter plans reserve only -their root and OpenVMM share parameters, as before. +UTF-8 byte that Linux treats as whitespace (including `0xa0`). `nvx-rs sandbox +start` and `run` check the kernel command line that they launch: the caller's +`--cmdline` uses Linux's parameter splitting, including quoted values and +hyphen/underscore name equivalence, and is checked as the launched line holds +it, untrimmed, and through the guest bootstrap's whitespace-only token view +too. Reserved `nvx_*` and `virtfs_*` names cannot be hidden inside quoted +values or behind leading Unicode whitespace; a `--` or unclosed quote that +hides the tokens that OpenVMM appends is refused. Host paths are the host's own, which `std::path` holds: spelled with their components alone, as `Path::components` reads them, with backslashes on @@ -2384,7 +2209,7 @@ The state directory is the CLI's own [store](src/host/cli/store/mod.rs) of the sandbox, which keeps the runtime records of its VM, and `config.json` is the CLI's record of the sandbox beside them. It also records the [distro binding](#distro-bindings) and the -[scratch binding](#checked-handoff) of the sandbox, and its network as the +[scratch binding](#images-by-path) of the sandbox, and its network as the network options from which each `start` builds the policy again, as `provision` built it. The typed configuration names one EROFS or GPT image by path and identity, and a scratch upper by path or a RAM upper with its cap from 1 through @@ -2492,7 +2317,7 @@ fails with `the guest image lacks control features that layered sandboxes need (...)`, which names the missing features. `nvx.py` does not ask; this, the checks of the [distro binding](#distro-bindings) and of the -[scratch binding](#checked-handoff), and the claim of the scratch image are the +[scratch binding](#images-by-path), and the claim of the scratch image are the intended changes of behavior. The guest agents of NVX advertise bits 0, 4, and 6 in sandbox mode. @@ -2603,12 +2428,12 @@ guest agents of NVX advertise bits 0, 4, and 6 in sandbox mode. [Sandbox networking](../doc/run.md#sandbox-networking). - A sandbox's scratch image is claimed while its VM runs, so a second sandbox, or a `run`, on the same image fails; see - [Checked handoff](#checked-handoff). + [Images by path](#images-by-path). - On Linux, OpenVMM starts with the caller's umask without the owner's bits, where `nvx.py` passed the umask unchanged, so that a umask that clears the owner's own permissions, such as 0400, no longer keeps the caller, or `start` and `stop`, from reading OpenVMM's outcome report; see - [`nvx::host::supervisor`](#nvxhostsupervisor). + [`nvx::host::vm`](#nvxhostvm). - `start` and `exec` refuse, and `stop` terminates, a VM that another runtime identity started; see [Runtime identity](#runtime-identity). - `run --restore-snapshot` names a snapshot of `--store-dir` that @@ -3190,10 +3015,8 @@ own facilities replaced it: and of the export planner that paths that are not valid Unicode are refused. The sandbox specification tests, those of `nvx::host::sandbox`, specify the -documented option grammar and the contracts with OpenVMM and the guest. -Properties cover the restore-stable aggregate child names, reserved targets -and tokens, every generated launch token and its exact 1024-byte budget, -policy option sequences, and paths on the host's own filesystem. They check +documented option grammar. Properties cover canonical layer UUIDs, reserved +targets, policy option sequences, and paths on the host's own filesystem. They check that a launch refuses both systemd entrypoints whatever the distro layer's manifest lists, and that verification refuses a manifest that lists `systemd` whatever the entrypoint, as [Distro bindings](#distro-bindings) describes. The @@ -3210,10 +3033,7 @@ and `parameq`, `parameqn`, and `repair_env_string`, with which the kernel matches parameters with their handlers. `nvx::protocol::cmdline::parameters` must read as the model does every command line of up to six bytes among a letter and those that `next_arg` treats apart, and random ones. With the model, -properties check that the kernel reads no reserved parameter in the extra -parameters that a `LaunchSpec` accepts, whatever separates it, and reads the -`hostname=` parameter after them; that the time ABI's check of the guest's -command line fails exactly when the kernel applies a clock parameter that the -ABI forbids; and that the kernel reads every token of the command lines of a -sandbox whose entrypoint, arguments, and share targets a launch accepts as one -parameter, whole. +properties check that the time ABI's check of the guest's command line fails +exactly when the kernel applies a clock parameter that the ABI forbids, and +that the share targets that a sandbox accepts hold nothing at which the kernel +would end a parameter. diff --git a/nvx-rs/examples/external-launch.rs b/nvx-rs/examples/external-launch.rs index ee8bd1ef5..9074024e4 100644 --- a/nvx-rs/examples/external-launch.rs +++ b/nvx-rs/examples/external-launch.rs @@ -1,4 +1,5 @@ -//! A caller-owned launch, using the strict low-level spawn primitive without a store. +//! A caller-owned launch: `vm::create` prepares a cold boot or a restore, and the caller's own +//! launcher starts the VMM of the `vm::VmmCommand` that it hands over. //! //! Run the compiled example with a backend, OpenVMM, kernel, initramfs, new VM directory, and //! the three artifact SHA-256 digests from their installation metadata, in that order. @@ -6,12 +7,30 @@ //! event, then intentionally stops and verifies the VM's exit without driving guest activation. //! The snapshot must have captured a disconnected boot console, as `nvx-rs snapshot capture` //! does. +//! +//! Its launcher, `spawn`, is a minimal one on `std::process`, enough for these two VMs: it writes +//! the capability into a pipe that it closes to writers before the spawn, gives the VMM that pipe +//! as its standard input and the log as its standard output and error, holds the VMM through a +//! `process::ProcessRef` whose start time it reads itself, as `ProcessRef` defines it, and ends +//! and reaps the VMM if anything fails once it runs. It leaves out the rest of the launch contract +//! that +//! `vm::VmmCommand` states, which a production launcher should add, as `vm::spawn` does: +//! +//! - It passes the VMM no descriptor or handle besides its standard streams, and refuses a +//! command that allows any. These two VMs need none. +//! - On Linux, it starts the VMM in a process group of its own, but not in a new session, and +//! leaves the VMM the caller's umask rather than clearing the owner's bits from it. +//! - On Windows, it starts the VMM without a console, but lets it inherit every inheritable handle +//! of the caller rather than an explicit list, and starts it in no job named after its log. +//! +//! The example spawns nothing else while the VMM starts, so it takes no lock around its spawn. use std::error::Error; #[cfg(target_os = "linux")] use std::fs::File; use std::io::{self, Write}; use std::path::Path; +use std::process::{Child, Command}; use std::time::{Duration, Instant}; use nvx::Hypervisor; @@ -20,10 +39,10 @@ use nvx::host::config::{ Boot, ConsoleAttachment, LayeredRoot, NetworkConfig, RootConfig, UpperConfig, VmConfig, }; use nvx::host::network::{Access, NetworkPolicy}; -use nvx::host::platform::Transport; +use nvx::host::process::{BootId, Opened, Process, ProcessRef}; use nvx::host::runtime::{ArtifactIdentity, LaunchOptions, Runtime, RuntimeIdentity}; -use nvx::host::supervisor::{self, Inherited}; -use nvx::host::{artifacts, report, snapshot, transport, vm}; +use nvx::host::transport::{self, Transport}; +use nvx::host::{artifacts, report, snapshot, vm}; use nvx::protocol::root::RootImage; /// One byte past OpenVMM's 25-byte restore-ready event, so that a longer event fails to parse. @@ -119,26 +138,9 @@ fn main() -> Result<(), Box> { } vm::create_private_dir(Path::new(&args[5]))?; let dir = vm::VmDir::open(Path::new(&args[5]))?; - let vm::VmmCommand { - program, - arguments, - working_dir, - capability, - outputs, - inherit, - log, - breakaway_from_job, - mut ready, - } = vm::create(&runtime, &config, snapshot.as_ref(), &dir)?.into_command(); - let command = supervisor::VmmCommand { - program, - arguments, - working_dir, - breakaway_from_job, - inherit: inherit.iter().map(Inherited::new).collect(), - }; - let mut launched = match supervisor::spawn_vm(&command, capability.as_bytes(), log) { - Ok(launched) => launched, + let command = vm::create(&runtime, &config, snapshot.as_ref(), &dir)?.into_command(); + let mut child = match spawn(&command) { + Ok(child) => child, Err(error) => { if snapshot.is_some() && let Err(cleanup) = remove_view(dir.path()) @@ -151,8 +153,11 @@ fn main() -> Result<(), Box> { return Err(error.into()); } }; + let vm::VmmCommand { + outputs, mut ready, .. + } = command; let result = (|| -> Result<(), Box> { - let process = launched.process()?; + let process = held(&child)?; let deadline = Instant::now() + Duration::from_secs(60); if let Some(listener) = &mut ready { let mut event = Vec::new(); @@ -168,8 +173,7 @@ fn main() -> Result<(), Box> { } nvx::protocol::restore_ready::parse_ready_event(&event)?; process.kill()?; - if process.wait(deadline)?.is_none() || launched.wait_timeout(Duration::ZERO)?.is_none() - { + if process.wait(deadline)?.is_none() || child.try_wait()?.is_none() { return Err(io::Error::other("restored VM exit is unconfirmed").into()); } println!( @@ -190,10 +194,10 @@ fn main() -> Result<(), Box> { if process.wait(deadline)?.is_none() { return Err(io::Error::new(io::ErrorKind::TimedOut, "VM did not exit").into()); } - match launched.wait_timeout(Duration::ZERO)? { - Some(0) => {} + match child.try_wait()? { + Some(status) if status.success() => {} Some(status) => { - return Err(io::Error::other(format!("VMM exited with status {status}")).into()); + return Err(io::Error::other(format!("VMM exited with {status}")).into()); } None => return Err(io::Error::other("VMM exit status is unavailable").into()), } @@ -210,7 +214,10 @@ fn main() -> Result<(), Box> { Ok(()) })(); if let Err(error) = result { - if let Err(cleanup) = launched.kill_and_confirm_exit() { + // `std::process` signals the child only until it reaps it, so the kill reaches the VMM + // alone. + let _ = child.kill(); + if let Err(cleanup) = child.wait() { return Err(io::Error::other(format!( "{error}; could not confirm exit of the launched VM: {cleanup}" )) @@ -226,7 +233,6 @@ fn main() -> Result<(), Box> { } return Err(error); } - launched.detach(); if snapshot.is_some() { remove_view(dir.path())?; } @@ -237,6 +243,119 @@ fn main() -> Result<(), Box> { Ok(()) } +/// Starts the VMM of `command` with `std::process`, as the minimal launcher that the module +/// describes: with the capability on its standard input, through a pipe that is closed to writers +/// before the spawn, and the log as its standard output and error, in a process group of its own, +/// and on Windows without a console, outside the caller's job if the command asks for it. +/// +/// Fails with [`io::ErrorKind::Unsupported`] for a command that allows the VMM a descriptor or +/// handle, which this launcher does not pass. +fn spawn(command: &vm::VmmCommand) -> io::Result { + if !command.inherit.is_empty() { + return Err(io::Error::new( + io::ErrorKind::Unsupported, + "this launcher passes the VMM no descriptor or handle", + )); + } + let (stdin, mut writer) = io::pipe()?; + writer.write_all(command.capability.as_bytes())?; + drop(writer); + let mut child = Command::new(&command.program); + child + .args(&command.arguments) + .current_dir(&command.working_dir) + .stdin(stdin) + .stdout(command.log.try_clone()?) + .stderr(command.log.try_clone()?); + #[cfg(unix)] + std::os::unix::process::CommandExt::process_group(&mut child, 0); + #[cfg(windows)] + { + use std::os::windows::process::CommandExt; + use windows_sys::Win32::System::Threading::{ + CREATE_BREAKAWAY_FROM_JOB, CREATE_NEW_PROCESS_GROUP, CREATE_NO_WINDOW, + }; + + let breakaway = if command.breakaway_from_job { + CREATE_BREAKAWAY_FROM_JOB + } else { + 0 + }; + child.creation_flags(CREATE_NEW_PROCESS_GROUP | CREATE_NO_WINDOW | breakaway); + } + child.spawn() +} + +/// Holds the VMM that `child` runs through a handle of its own: until the child is reaped, its ID +/// and start time name it alone. +fn held(child: &Child) -> io::Result { + let reference = ProcessRef { + pid: child.id(), + start_time: start_time(child)?, + boot: BootId::current()?, + }; + match Process::open(&reference)? { + Opened::Running(process) => Ok(process), + Opened::Exited(_) => Err(io::Error::other("the VMM exited before it could be held")), + } +} + +/// Returns the start time of the process that `child` runs, as `ProcessRef` defines it: on Linux, +/// the `starttime` field of `/proc//stat`, in clock ticks since boot, which the process keeps +/// until its parent reaps it. +#[cfg(target_os = "linux")] +fn start_time(child: &Child) -> io::Result { + let path = format!("/proc/{}/stat", child.id()); + let stat = std::fs::read(&path)?; + // The command name may hold spaces and parentheses, so the fields that follow it start after + // the last closing parenthesis, with the state, field 3; the start time is field 22. + stat.iter() + .rposition(|&byte| byte == b')') + .and_then(|end| std::str::from_utf8(&stat[end + 1..]).ok()) + .and_then(|fields| fields.split_whitespace().nth(19)) + .and_then(|field| field.parse().ok()) + .ok_or_else(|| io::Error::other(format!("{path} has an unexpected format"))) +} + +/// Returns the start time of the process that `child` runs, as `ProcessRef` defines it: on +/// Windows, the creation time of the process as a 64-bit `FILETIME`. +#[cfg(windows)] +fn start_time(child: &Child) -> io::Result { + use std::os::windows::io::AsRawHandle; + use windows_sys::Win32::Foundation::FILETIME; + use windows_sys::Win32::System::Threading::GetProcessTimes; + + let zero = FILETIME { + dwLowDateTime: 0, + dwHighDateTime: 0, + }; + let (mut created, mut exited, mut kernel, mut user) = (zero, zero, zero, zero); + // SAFETY: the handle of `child` has every access to its process, which it keeps until the + // child drops, and each output points to a `FILETIME` that the call may write. + let succeeded = unsafe { + GetProcessTimes( + child.as_raw_handle(), + &mut created, + &mut exited, + &mut kernel, + &mut user, + ) + } != 0; + if !succeeded { + return Err(io::Error::last_os_error()); + } + Ok((u64::from(created.dwHighDateTime) << 32) | u64::from(created.dwLowDateTime)) +} + +/// Fails on the hosts other than Linux and Windows, where NVX runs no VMs. +#[cfg(not(any(target_os = "linux", windows)))] +fn start_time(_child: &Child) -> io::Result { + Err(io::Error::new( + io::ErrorKind::Unsupported, + "NVX runs VMs on Linux and Windows hosts only", + )) +} + /// Removes only the successful fixed restore view, while the caller still holds its lock. fn remove_view(directory: &Path) -> io::Result<()> { let view = directory.join(snapshot::OPENVMM_DIR); diff --git a/nvx-rs/public-api/linux.txt b/nvx-rs/public-api/linux.txt index f46f3964e..ca8c7f54f 100644 --- a/nvx-rs/public-api/linux.txt +++ b/nvx-rs/public-api/linux.txt @@ -1831,53 +1831,6 @@ field nvx::host::filesystem::Target::Guest::0 fn nvx::host::filesystem::guest_path fn nvx::host::filesystem::plan fn nvx::host::filesystem::resolve_guest_path -mod nvx::host::launch -const nvx::host::launch::COMMAND_LINE_BUDGET -const nvx::host::launch::DEFAULT_GUEST_NETWORK -const nvx::host::launch::DEFAULT_HOSTNAME -const nvx::host::launch::DEFAULT_MEMORY_MIB -struct nvx::host::launch::LaunchSpec -field nvx::host::launch::LaunchSpec::breakaway_from_job -method nvx::host::launch::LaunchSpec::check -method nvx::host::launch::LaunchSpec::check_process_command_line -field nvx::host::launch::LaunchSpec::filesystem -field nvx::host::launch::LaunchSpec::guest_network -field nvx::host::launch::LaunchSpec::hostname -field nvx::host::launch::LaunchSpec::hypervisor -field nvx::host::launch::LaunchSpec::identity -field nvx::host::launch::LaunchSpec::initrd -field nvx::host::launch::LaunchSpec::kernel -method nvx::host::launch::LaunchSpec::kernel_command_line -field nvx::host::launch::LaunchSpec::kernel_parameters -field nvx::host::launch::LaunchSpec::memory_mib -field nvx::host::launch::LaunchSpec::network -method nvx::host::launch::LaunchSpec::new -field nvx::host::launch::LaunchSpec::openvmm -method nvx::host::launch::LaunchSpec::openvmm_arguments -method nvx::host::launch::LaunchSpec::vmm_launch -const nvx::host::launch::MAX_TIMEOUT -const nvx::host::launch::PROCESS_COMMAND_LINE_LIMIT -struct nvx::host::launch::VmmLaunch -const nvx::host::launch::VmmLaunch::RESERVE -method nvx::host::launch::VmmLaunch::arguments -field nvx::host::launch::VmmLaunch::endpoint -field nvx::host::launch::VmmLaunch::hypervisor -field nvx::host::launch::VmmLaunch::initrd -field nvx::host::launch::VmmLaunch::kernel -field nvx::host::launch::VmmLaunch::memory_mib -field nvx::host::launch::VmmLaunch::openvmm -field nvx::host::launch::VmmLaunch::report -struct nvx::host::launch::WorkloadIdentity -const nvx::host::launch::WorkloadIdentity::DEFAULT -field nvx::host::launch::WorkloadIdentity::create_account -field nvx::host::launch::WorkloadIdentity::gid -method nvx::host::launch::WorkloadIdentity::select -field nvx::host::launch::WorkloadIdentity::uid -fn nvx::host::launch::base_command_line -fn nvx::host::launch::fit_process_command_line -fn nvx::host::launch::kernel_parameters_problem -fn nvx::host::launch::process_command_line_units -fn nvx::host::launch::timeout_problem mod nvx::host::network enum nvx::host::network::Access variant nvx::host::network::Access::Allow @@ -2153,16 +2106,9 @@ method nvx::host::runtime::RuntimeIdentity::new field nvx::host::runtime::RuntimeIdentity::openvmm fn nvx::host::runtime::probe_hypervisor mod nvx::host::sandbox -enum nvx::host::sandbox::BlockRole -variant nvx::host::sandbox::BlockRole::Distro -variant nvx::host::sandbox::BlockRole::Scratch -method nvx::host::sandbox::BlockRole::as_str -method nvx::host::sandbox::BlockRole::is_read_only -method nvx::host::sandbox::BlockRole::mmio_base const nvx::host::sandbox::DEFAULT_ENTRYPOINT const nvx::host::sandbox::DEFAULT_HOSTNAME const nvx::host::sandbox::DEFAULT_WORKLOAD_IDENTITY -const nvx::host::sandbox::KERNEL_COMMAND_LINE_MAX_SIZE const nvx::host::sandbox::LAYER_ROLES enum nvx::host::sandbox::LayerRole variant nvx::host::sandbox::LayerRole::Distro @@ -2189,43 +2135,12 @@ field nvx::host::sandbox::MountPolicy::denied method nvx::host::sandbox::MountPolicy::is_empty method nvx::host::sandbox::MountPolicy::paths field nvx::host::sandbox::MountPolicy::writable -const nvx::host::sandbox::OPENVMM_COMMAND_LINE_RESERVE -const nvx::host::sandbox::PLAN_VERSION enum nvx::host::sandbox::PolicyKind variant nvx::host::sandbox::PolicyKind::Allowed variant nvx::host::sandbox::PolicyKind::Denied variant nvx::host::sandbox::PolicyKind::Writable method nvx::host::sandbox::PolicyKind::as_str method nvx::host::sandbox::PolicyKind::option -struct nvx::host::sandbox::PreparedLaunch -method nvx::host::sandbox::PreparedLaunch::adapter_arguments -method nvx::host::sandbox::PreparedLaunch::distro -method nvx::host::sandbox::PreparedLaunch::inherited -method nvx::host::sandbox::PreparedLaunch::kernel_command_line -method nvx::host::sandbox::PreparedLaunch::new -method nvx::host::sandbox::PreparedLaunch::openvmm_arguments -method nvx::host::sandbox::PreparedLaunch::plan -method nvx::host::sandbox::PreparedLaunch::release -method nvx::host::sandbox::PreparedLaunch::scratch -const nvx::host::sandbox::SANDBOX_COMMAND_LINE_MAX_SIZE -struct nvx::host::sandbox::SandboxDistro -method nvx::host::sandbox::SandboxDistro::image -method nvx::host::sandbox::SandboxDistro::new -method nvx::host::sandbox::SandboxDistro::path -struct nvx::host::sandbox::SandboxLaunch -method nvx::host::sandbox::SandboxLaunch::bound_plan -method nvx::host::sandbox::SandboxLaunch::distro -method nvx::host::sandbox::SandboxLaunch::export -method nvx::host::sandbox::SandboxLaunch::layers -method nvx::host::sandbox::SandboxLaunch::network -method nvx::host::sandbox::SandboxLaunch::new -method nvx::host::sandbox::SandboxLaunch::plan -method nvx::host::sandbox::SandboxLaunch::scratch -method nvx::host::sandbox::SandboxLaunch::settings -method nvx::host::sandbox::SandboxLaunch::upper -method nvx::host::sandbox::SandboxLaunch::validated -method nvx::host::sandbox::SandboxLaunch::with_export -method nvx::host::sandbox::SandboxLaunch::with_network struct nvx::host::sandbox::SandboxLayer method nvx::host::sandbox::SandboxLayer::format method nvx::host::sandbox::SandboxLayer::gpt @@ -2247,44 +2162,6 @@ method nvx::host::sandbox::SandboxMount::parse method nvx::host::sandbox::SandboxMount::policy method nvx::host::sandbox::SandboxMount::validated method nvx::host::sandbox::SandboxMount::with_kind -struct nvx::host::sandbox::SandboxNetwork -method nvx::host::sandbox::SandboxNetwork::gateway -method nvx::host::sandbox::SandboxNetwork::guest_network -method nvx::host::sandbox::SandboxNetwork::new -method nvx::host::sandbox::SandboxNetwork::openvmm_arguments -method nvx::host::sandbox::SandboxNetwork::policy -method nvx::host::sandbox::SandboxNetwork::proxy_url -method nvx::host::sandbox::SandboxNetwork::template_arguments -struct nvx::host::sandbox::SandboxPlan -method nvx::host::sandbox::SandboxPlan::adapter -method nvx::host::sandbox::SandboxPlan::blocks -method nvx::host::sandbox::SandboxPlan::check_adapter_launch -method nvx::host::sandbox::SandboxPlan::distro -method nvx::host::sandbox::SandboxPlan::export -method nvx::host::sandbox::SandboxPlan::kernel_command_line -method nvx::host::sandbox::SandboxPlan::layout -method nvx::host::sandbox::SandboxPlan::mapping_table -method nvx::host::sandbox::SandboxPlan::network -method nvx::host::sandbox::SandboxPlan::new -method nvx::host::sandbox::SandboxPlan::openvmm_arguments -method nvx::host::sandbox::SandboxPlan::proxy_url -method nvx::host::sandbox::SandboxPlan::settings -method nvx::host::sandbox::SandboxPlan::upper -method nvx::host::sandbox::SandboxPlan::validated -method nvx::host::sandbox::SandboxPlan::with_export -method nvx::host::sandbox::SandboxPlan::with_network -struct nvx::host::sandbox::SandboxSettings -field nvx::host::sandbox::SandboxSettings::args -field nvx::host::sandbox::SandboxSettings::entrypoint -field nvx::host::sandbox::SandboxSettings::hostname -field nvx::host::sandbox::SandboxSettings::memory_max -field nvx::host::sandbox::SandboxSettings::pids_max -field nvx::host::sandbox::SandboxSettings::workload_identity -struct nvx::host::sandbox::SandboxUpper -const nvx::host::sandbox::SandboxUpper::RAM -method nvx::host::sandbox::SandboxUpper::kind -method nvx::host::sandbox::SandboxUpper::scratch -method nvx::host::sandbox::SandboxUpper::scratch_path mod nvx::host::sandbox::distro const nvx::host::sandbox::distro::BINDING_VERSION struct nvx::host::sandbox::distro::CheckedDistro @@ -2329,10 +2206,6 @@ method nvx::host::sandbox::layer::SandboxLayer::role method nvx::host::sandbox::layer::SandboxLayer::uuid mod nvx::host::sandbox::mount const nvx::host::sandbox::mount::AGGREGATE_MOUNT_TARGET -struct nvx::host::sandbox::mount::BootShares -field nvx::host::sandbox::mount::BootShares::arguments -field nvx::host::sandbox::mount::BootShares::fragment -field nvx::host::sandbox::mount::BootShares::tokens const nvx::host::sandbox::mount::MAX_MOUNT_POLICY_PATHS const nvx::host::sandbox::mount::MAX_MOUNT_TARGET_BYTES const nvx::host::sandbox::mount::MOUNT_TAG @@ -2379,12 +2252,6 @@ method nvx::host::sandbox::mount::SandboxMount::parse method nvx::host::sandbox::mount::SandboxMount::policy method nvx::host::sandbox::mount::SandboxMount::validated method nvx::host::sandbox::mount::SandboxMount::with_kind -fn nvx::host::sandbox::mount::aggregate_child_argument -fn nvx::host::sandbox::mount::aggregate_child_name -fn nvx::host::sandbox::mount::aggregate_command_line_fragment -fn nvx::host::sandbox::mount::aggregate_share_tokens -fn nvx::host::sandbox::mount::boot_shares -fn nvx::host::sandbox::mount::mount_argument fn nvx::host::sandbox::mount::mount_kind fn nvx::host::sandbox::mount::parse_mount_value fn nvx::host::sandbox::mount::plan_mounts @@ -2434,62 +2301,7 @@ fn nvx::host::sandbox::options::sandbox_mounts fn nvx::host::sandbox::options::validate_exec_environment fn nvx::host::sandbox::options::validate_run_share_targets fn nvx::host::sandbox::options::validate_sandbox_entrypoint -fn nvx::host::sandbox::options::validate_sandbox_systemd_policy fn nvx::host::sandbox::parse_workload_identity -mod nvx::host::sandbox::plan -enum nvx::host::sandbox::plan::BlockRole -variant nvx::host::sandbox::plan::BlockRole::Distro -variant nvx::host::sandbox::plan::BlockRole::Scratch -method nvx::host::sandbox::plan::BlockRole::as_str -method nvx::host::sandbox::plan::BlockRole::is_read_only -method nvx::host::sandbox::plan::BlockRole::mmio_base -const nvx::host::sandbox::plan::PLAN_VERSION -struct nvx::host::sandbox::plan::SandboxDistro -method nvx::host::sandbox::plan::SandboxDistro::image -method nvx::host::sandbox::plan::SandboxDistro::new -method nvx::host::sandbox::plan::SandboxDistro::path -struct nvx::host::sandbox::plan::SandboxNetwork -method nvx::host::sandbox::plan::SandboxNetwork::gateway -method nvx::host::sandbox::plan::SandboxNetwork::guest_network -method nvx::host::sandbox::plan::SandboxNetwork::new -method nvx::host::sandbox::plan::SandboxNetwork::openvmm_arguments -method nvx::host::sandbox::plan::SandboxNetwork::policy -method nvx::host::sandbox::plan::SandboxNetwork::proxy_url -method nvx::host::sandbox::plan::SandboxNetwork::template_arguments -struct nvx::host::sandbox::plan::SandboxPlan -method nvx::host::sandbox::plan::SandboxPlan::adapter -method nvx::host::sandbox::plan::SandboxPlan::blocks -method nvx::host::sandbox::plan::SandboxPlan::check_adapter_launch -method nvx::host::sandbox::plan::SandboxPlan::distro -method nvx::host::sandbox::plan::SandboxPlan::export -method nvx::host::sandbox::plan::SandboxPlan::kernel_command_line -method nvx::host::sandbox::plan::SandboxPlan::layout -method nvx::host::sandbox::plan::SandboxPlan::mapping_table -method nvx::host::sandbox::plan::SandboxPlan::network -method nvx::host::sandbox::plan::SandboxPlan::new -method nvx::host::sandbox::plan::SandboxPlan::openvmm_arguments -method nvx::host::sandbox::plan::SandboxPlan::proxy_url -method nvx::host::sandbox::plan::SandboxPlan::settings -method nvx::host::sandbox::plan::SandboxPlan::upper -method nvx::host::sandbox::plan::SandboxPlan::validated -method nvx::host::sandbox::plan::SandboxPlan::with_export -method nvx::host::sandbox::plan::SandboxPlan::with_network -struct nvx::host::sandbox::plan::SandboxUpper -const nvx::host::sandbox::plan::SandboxUpper::RAM -method nvx::host::sandbox::plan::SandboxUpper::kind -method nvx::host::sandbox::plan::SandboxUpper::scratch -method nvx::host::sandbox::plan::SandboxUpper::scratch_path -mod nvx::host::sandbox::prepared -struct nvx::host::sandbox::prepared::PreparedLaunch -method nvx::host::sandbox::prepared::PreparedLaunch::adapter_arguments -method nvx::host::sandbox::prepared::PreparedLaunch::distro -method nvx::host::sandbox::prepared::PreparedLaunch::inherited -method nvx::host::sandbox::prepared::PreparedLaunch::kernel_command_line -method nvx::host::sandbox::prepared::PreparedLaunch::new -method nvx::host::sandbox::prepared::PreparedLaunch::openvmm_arguments -method nvx::host::sandbox::prepared::PreparedLaunch::plan -method nvx::host::sandbox::prepared::PreparedLaunch::release -method nvx::host::sandbox::prepared::PreparedLaunch::scratch fn nvx::host::sandbox::require_file mod nvx::host::sandbox::scratch const nvx::host::sandbox::scratch::CLAIM_RELEASE_WAIT @@ -2537,29 +2349,9 @@ struct nvx::host::snapshot::BlockGeometry field nvx::host::snapshot::BlockGeometry::length field nvx::host::snapshot::BlockGeometry::logical_block_size field nvx::host::snapshot::BlockGeometry::physical_block_size -struct nvx::host::snapshot::CaptureClass -const nvx::host::snapshot::CaptureClass::VERSION -field nvx::host::snapshot::CaptureClass::boot_processors -method nvx::host::snapshot::CaptureClass::check -field nvx::host::snapshot::CaptureClass::cpu_profile -field nvx::host::snapshot::CaptureClass::distro -field nvx::host::snapshot::CaptureClass::guest_build -field nvx::host::snapshot::CaptureClass::hypervisor -method nvx::host::snapshot::CaptureClass::identity -field nvx::host::snapshot::CaptureClass::invariants -field nvx::host::snapshot::CaptureClass::memory -field nvx::host::snapshot::CaptureClass::memory_capacity -field nvx::host::snapshot::CaptureClass::network -field nvx::host::snapshot::CaptureClass::processors -field nvx::host::snapshot::CaptureClass::runtime -field nvx::host::snapshot::CaptureClass::scratch -method nvx::host::snapshot::CaptureClass::to_bytes struct nvx::host::snapshot::CaptureContext struct nvx::host::snapshot::CapturedSnapshot field nvx::host::snapshot::CapturedSnapshot::key -struct nvx::host::snapshot::ClassIdentity -method nvx::host::snapshot::ClassIdentity::as_bytes -method nvx::host::snapshot::ClassIdentity::from_bytes const nvx::host::snapshot::DESCRIPTION_FILE struct nvx::host::snapshot::Downtime field nvx::host::snapshot::Downtime::nanos @@ -2629,7 +2421,6 @@ method nvx::host::snapshot::SnapshotKey::invariants method nvx::host::snapshot::SnapshotKey::memory method nvx::host::snapshot::SnapshotKey::memory_block_size method nvx::host::snapshot::SnapshotKey::memory_capacity -method nvx::host::snapshot::SnapshotKey::new method nvx::host::snapshot::SnapshotKey::processors method nvx::host::snapshot::SnapshotKey::runtime method nvx::host::snapshot::SnapshotKey::scratch @@ -2659,27 +2450,6 @@ field nvx::host::snapshot::TimeContract::lineage field nvx::host::snapshot::TimeContract::tsc_frequency_hz field nvx::host::snapshot::TimeContract::tsc_tolerance_ppm fn nvx::host::snapshot::finish_capture -mod nvx::host::supervisor -struct nvx::host::supervisor::Inherited -method nvx::host::supervisor::Inherited::new -method nvx::host::supervisor::Inherited::proc_path -struct nvx::host::supervisor::Launched -method nvx::host::supervisor::Launched::detach -method nvx::host::supervisor::Launched::id -method nvx::host::supervisor::Launched::kill_and_confirm_exit -method nvx::host::supervisor::Launched::kill_and_wait -method nvx::host::supervisor::Launched::process -method nvx::host::supervisor::Launched::wait_timeout -struct nvx::host::supervisor::VmmCommand -field nvx::host::supervisor::VmmCommand::arguments -field nvx::host::supervisor::VmmCommand::breakaway_from_job -field nvx::host::supervisor::VmmCommand::inherit -method nvx::host::supervisor::VmmCommand::new -field nvx::host::supervisor::VmmCommand::program -field nvx::host::supervisor::VmmCommand::working_dir -fn nvx::host::supervisor::spawn -fn nvx::host::supervisor::spawn_command -fn nvx::host::supervisor::spawn_vm mod nvx::host::transport struct nvx::host::transport::HostConnection method nvx::host::transport::HostConnection::attach @@ -2834,7 +2604,6 @@ field nvx::protocol::OuterHeader::sequence struct nvx::protocol::ProtocolError method nvx::protocol::ProtocolError::into_message method nvx::protocol::ProtocolError::message -const nvx::protocol::WORKLOAD_ACCOUNT_TOKEN mod nvx::protocol::cmdline const nvx::protocol::cmdline::DNS_TOKEN const nvx::protocol::cmdline::GATEWAY_TOKEN diff --git a/nvx-rs/public-api/windows.txt b/nvx-rs/public-api/windows.txt index 526f2453f..5af2af9b2 100644 --- a/nvx-rs/public-api/windows.txt +++ b/nvx-rs/public-api/windows.txt @@ -559,53 +559,6 @@ field nvx::host::filesystem::Target::Guest::0 fn nvx::host::filesystem::guest_path fn nvx::host::filesystem::plan fn nvx::host::filesystem::resolve_guest_path -mod nvx::host::launch -const nvx::host::launch::COMMAND_LINE_BUDGET -const nvx::host::launch::DEFAULT_GUEST_NETWORK -const nvx::host::launch::DEFAULT_HOSTNAME -const nvx::host::launch::DEFAULT_MEMORY_MIB -struct nvx::host::launch::LaunchSpec -field nvx::host::launch::LaunchSpec::breakaway_from_job -method nvx::host::launch::LaunchSpec::check -method nvx::host::launch::LaunchSpec::check_process_command_line -field nvx::host::launch::LaunchSpec::filesystem -field nvx::host::launch::LaunchSpec::guest_network -field nvx::host::launch::LaunchSpec::hostname -field nvx::host::launch::LaunchSpec::hypervisor -field nvx::host::launch::LaunchSpec::identity -field nvx::host::launch::LaunchSpec::initrd -field nvx::host::launch::LaunchSpec::kernel -method nvx::host::launch::LaunchSpec::kernel_command_line -field nvx::host::launch::LaunchSpec::kernel_parameters -field nvx::host::launch::LaunchSpec::memory_mib -field nvx::host::launch::LaunchSpec::network -method nvx::host::launch::LaunchSpec::new -field nvx::host::launch::LaunchSpec::openvmm -method nvx::host::launch::LaunchSpec::openvmm_arguments -method nvx::host::launch::LaunchSpec::vmm_launch -const nvx::host::launch::MAX_TIMEOUT -const nvx::host::launch::PROCESS_COMMAND_LINE_LIMIT -struct nvx::host::launch::VmmLaunch -const nvx::host::launch::VmmLaunch::RESERVE -method nvx::host::launch::VmmLaunch::arguments -field nvx::host::launch::VmmLaunch::endpoint -field nvx::host::launch::VmmLaunch::hypervisor -field nvx::host::launch::VmmLaunch::initrd -field nvx::host::launch::VmmLaunch::kernel -field nvx::host::launch::VmmLaunch::memory_mib -field nvx::host::launch::VmmLaunch::openvmm -field nvx::host::launch::VmmLaunch::report -struct nvx::host::launch::WorkloadIdentity -const nvx::host::launch::WorkloadIdentity::DEFAULT -field nvx::host::launch::WorkloadIdentity::create_account -field nvx::host::launch::WorkloadIdentity::gid -method nvx::host::launch::WorkloadIdentity::select -field nvx::host::launch::WorkloadIdentity::uid -fn nvx::host::launch::base_command_line -fn nvx::host::launch::fit_process_command_line -fn nvx::host::launch::kernel_parameters_problem -fn nvx::host::launch::process_command_line_units -fn nvx::host::launch::timeout_problem mod nvx::host::network enum nvx::host::network::Access variant nvx::host::network::Access::Allow @@ -881,16 +834,9 @@ method nvx::host::runtime::RuntimeIdentity::new field nvx::host::runtime::RuntimeIdentity::openvmm fn nvx::host::runtime::probe_hypervisor mod nvx::host::sandbox -enum nvx::host::sandbox::BlockRole -variant nvx::host::sandbox::BlockRole::Distro -variant nvx::host::sandbox::BlockRole::Scratch -method nvx::host::sandbox::BlockRole::as_str -method nvx::host::sandbox::BlockRole::is_read_only -method nvx::host::sandbox::BlockRole::mmio_base const nvx::host::sandbox::DEFAULT_ENTRYPOINT const nvx::host::sandbox::DEFAULT_HOSTNAME const nvx::host::sandbox::DEFAULT_WORKLOAD_IDENTITY -const nvx::host::sandbox::KERNEL_COMMAND_LINE_MAX_SIZE const nvx::host::sandbox::LAYER_ROLES enum nvx::host::sandbox::LayerRole variant nvx::host::sandbox::LayerRole::Distro @@ -917,43 +863,12 @@ field nvx::host::sandbox::MountPolicy::denied method nvx::host::sandbox::MountPolicy::is_empty method nvx::host::sandbox::MountPolicy::paths field nvx::host::sandbox::MountPolicy::writable -const nvx::host::sandbox::OPENVMM_COMMAND_LINE_RESERVE -const nvx::host::sandbox::PLAN_VERSION enum nvx::host::sandbox::PolicyKind variant nvx::host::sandbox::PolicyKind::Allowed variant nvx::host::sandbox::PolicyKind::Denied variant nvx::host::sandbox::PolicyKind::Writable method nvx::host::sandbox::PolicyKind::as_str method nvx::host::sandbox::PolicyKind::option -struct nvx::host::sandbox::PreparedLaunch -method nvx::host::sandbox::PreparedLaunch::adapter_arguments -method nvx::host::sandbox::PreparedLaunch::distro -method nvx::host::sandbox::PreparedLaunch::inherited -method nvx::host::sandbox::PreparedLaunch::kernel_command_line -method nvx::host::sandbox::PreparedLaunch::new -method nvx::host::sandbox::PreparedLaunch::openvmm_arguments -method nvx::host::sandbox::PreparedLaunch::plan -method nvx::host::sandbox::PreparedLaunch::release -method nvx::host::sandbox::PreparedLaunch::scratch -const nvx::host::sandbox::SANDBOX_COMMAND_LINE_MAX_SIZE -struct nvx::host::sandbox::SandboxDistro -method nvx::host::sandbox::SandboxDistro::image -method nvx::host::sandbox::SandboxDistro::new -method nvx::host::sandbox::SandboxDistro::path -struct nvx::host::sandbox::SandboxLaunch -method nvx::host::sandbox::SandboxLaunch::bound_plan -method nvx::host::sandbox::SandboxLaunch::distro -method nvx::host::sandbox::SandboxLaunch::export -method nvx::host::sandbox::SandboxLaunch::layers -method nvx::host::sandbox::SandboxLaunch::network -method nvx::host::sandbox::SandboxLaunch::new -method nvx::host::sandbox::SandboxLaunch::plan -method nvx::host::sandbox::SandboxLaunch::scratch -method nvx::host::sandbox::SandboxLaunch::settings -method nvx::host::sandbox::SandboxLaunch::upper -method nvx::host::sandbox::SandboxLaunch::validated -method nvx::host::sandbox::SandboxLaunch::with_export -method nvx::host::sandbox::SandboxLaunch::with_network struct nvx::host::sandbox::SandboxLayer method nvx::host::sandbox::SandboxLayer::format method nvx::host::sandbox::SandboxLayer::gpt @@ -975,44 +890,6 @@ method nvx::host::sandbox::SandboxMount::parse method nvx::host::sandbox::SandboxMount::policy method nvx::host::sandbox::SandboxMount::validated method nvx::host::sandbox::SandboxMount::with_kind -struct nvx::host::sandbox::SandboxNetwork -method nvx::host::sandbox::SandboxNetwork::gateway -method nvx::host::sandbox::SandboxNetwork::guest_network -method nvx::host::sandbox::SandboxNetwork::new -method nvx::host::sandbox::SandboxNetwork::openvmm_arguments -method nvx::host::sandbox::SandboxNetwork::policy -method nvx::host::sandbox::SandboxNetwork::proxy_url -method nvx::host::sandbox::SandboxNetwork::template_arguments -struct nvx::host::sandbox::SandboxPlan -method nvx::host::sandbox::SandboxPlan::adapter -method nvx::host::sandbox::SandboxPlan::blocks -method nvx::host::sandbox::SandboxPlan::check_adapter_launch -method nvx::host::sandbox::SandboxPlan::distro -method nvx::host::sandbox::SandboxPlan::export -method nvx::host::sandbox::SandboxPlan::kernel_command_line -method nvx::host::sandbox::SandboxPlan::layout -method nvx::host::sandbox::SandboxPlan::mapping_table -method nvx::host::sandbox::SandboxPlan::network -method nvx::host::sandbox::SandboxPlan::new -method nvx::host::sandbox::SandboxPlan::openvmm_arguments -method nvx::host::sandbox::SandboxPlan::proxy_url -method nvx::host::sandbox::SandboxPlan::settings -method nvx::host::sandbox::SandboxPlan::upper -method nvx::host::sandbox::SandboxPlan::validated -method nvx::host::sandbox::SandboxPlan::with_export -method nvx::host::sandbox::SandboxPlan::with_network -struct nvx::host::sandbox::SandboxSettings -field nvx::host::sandbox::SandboxSettings::args -field nvx::host::sandbox::SandboxSettings::entrypoint -field nvx::host::sandbox::SandboxSettings::hostname -field nvx::host::sandbox::SandboxSettings::memory_max -field nvx::host::sandbox::SandboxSettings::pids_max -field nvx::host::sandbox::SandboxSettings::workload_identity -struct nvx::host::sandbox::SandboxUpper -const nvx::host::sandbox::SandboxUpper::RAM -method nvx::host::sandbox::SandboxUpper::kind -method nvx::host::sandbox::SandboxUpper::scratch -method nvx::host::sandbox::SandboxUpper::scratch_path mod nvx::host::sandbox::distro const nvx::host::sandbox::distro::BINDING_VERSION struct nvx::host::sandbox::distro::CheckedDistro @@ -1057,10 +934,6 @@ method nvx::host::sandbox::layer::SandboxLayer::role method nvx::host::sandbox::layer::SandboxLayer::uuid mod nvx::host::sandbox::mount const nvx::host::sandbox::mount::AGGREGATE_MOUNT_TARGET -struct nvx::host::sandbox::mount::BootShares -field nvx::host::sandbox::mount::BootShares::arguments -field nvx::host::sandbox::mount::BootShares::fragment -field nvx::host::sandbox::mount::BootShares::tokens const nvx::host::sandbox::mount::MAX_MOUNT_POLICY_PATHS const nvx::host::sandbox::mount::MAX_MOUNT_TARGET_BYTES const nvx::host::sandbox::mount::MOUNT_TAG @@ -1107,12 +980,6 @@ method nvx::host::sandbox::mount::SandboxMount::parse method nvx::host::sandbox::mount::SandboxMount::policy method nvx::host::sandbox::mount::SandboxMount::validated method nvx::host::sandbox::mount::SandboxMount::with_kind -fn nvx::host::sandbox::mount::aggregate_child_argument -fn nvx::host::sandbox::mount::aggregate_child_name -fn nvx::host::sandbox::mount::aggregate_command_line_fragment -fn nvx::host::sandbox::mount::aggregate_share_tokens -fn nvx::host::sandbox::mount::boot_shares -fn nvx::host::sandbox::mount::mount_argument fn nvx::host::sandbox::mount::mount_kind fn nvx::host::sandbox::mount::parse_mount_value fn nvx::host::sandbox::mount::plan_mounts @@ -1162,62 +1029,7 @@ fn nvx::host::sandbox::options::sandbox_mounts fn nvx::host::sandbox::options::validate_exec_environment fn nvx::host::sandbox::options::validate_run_share_targets fn nvx::host::sandbox::options::validate_sandbox_entrypoint -fn nvx::host::sandbox::options::validate_sandbox_systemd_policy fn nvx::host::sandbox::parse_workload_identity -mod nvx::host::sandbox::plan -enum nvx::host::sandbox::plan::BlockRole -variant nvx::host::sandbox::plan::BlockRole::Distro -variant nvx::host::sandbox::plan::BlockRole::Scratch -method nvx::host::sandbox::plan::BlockRole::as_str -method nvx::host::sandbox::plan::BlockRole::is_read_only -method nvx::host::sandbox::plan::BlockRole::mmio_base -const nvx::host::sandbox::plan::PLAN_VERSION -struct nvx::host::sandbox::plan::SandboxDistro -method nvx::host::sandbox::plan::SandboxDistro::image -method nvx::host::sandbox::plan::SandboxDistro::new -method nvx::host::sandbox::plan::SandboxDistro::path -struct nvx::host::sandbox::plan::SandboxNetwork -method nvx::host::sandbox::plan::SandboxNetwork::gateway -method nvx::host::sandbox::plan::SandboxNetwork::guest_network -method nvx::host::sandbox::plan::SandboxNetwork::new -method nvx::host::sandbox::plan::SandboxNetwork::openvmm_arguments -method nvx::host::sandbox::plan::SandboxNetwork::policy -method nvx::host::sandbox::plan::SandboxNetwork::proxy_url -method nvx::host::sandbox::plan::SandboxNetwork::template_arguments -struct nvx::host::sandbox::plan::SandboxPlan -method nvx::host::sandbox::plan::SandboxPlan::adapter -method nvx::host::sandbox::plan::SandboxPlan::blocks -method nvx::host::sandbox::plan::SandboxPlan::check_adapter_launch -method nvx::host::sandbox::plan::SandboxPlan::distro -method nvx::host::sandbox::plan::SandboxPlan::export -method nvx::host::sandbox::plan::SandboxPlan::kernel_command_line -method nvx::host::sandbox::plan::SandboxPlan::layout -method nvx::host::sandbox::plan::SandboxPlan::mapping_table -method nvx::host::sandbox::plan::SandboxPlan::network -method nvx::host::sandbox::plan::SandboxPlan::new -method nvx::host::sandbox::plan::SandboxPlan::openvmm_arguments -method nvx::host::sandbox::plan::SandboxPlan::proxy_url -method nvx::host::sandbox::plan::SandboxPlan::settings -method nvx::host::sandbox::plan::SandboxPlan::upper -method nvx::host::sandbox::plan::SandboxPlan::validated -method nvx::host::sandbox::plan::SandboxPlan::with_export -method nvx::host::sandbox::plan::SandboxPlan::with_network -struct nvx::host::sandbox::plan::SandboxUpper -const nvx::host::sandbox::plan::SandboxUpper::RAM -method nvx::host::sandbox::plan::SandboxUpper::kind -method nvx::host::sandbox::plan::SandboxUpper::scratch -method nvx::host::sandbox::plan::SandboxUpper::scratch_path -mod nvx::host::sandbox::prepared -struct nvx::host::sandbox::prepared::PreparedLaunch -method nvx::host::sandbox::prepared::PreparedLaunch::adapter_arguments -method nvx::host::sandbox::prepared::PreparedLaunch::distro -method nvx::host::sandbox::prepared::PreparedLaunch::inherited -method nvx::host::sandbox::prepared::PreparedLaunch::kernel_command_line -method nvx::host::sandbox::prepared::PreparedLaunch::new -method nvx::host::sandbox::prepared::PreparedLaunch::openvmm_arguments -method nvx::host::sandbox::prepared::PreparedLaunch::plan -method nvx::host::sandbox::prepared::PreparedLaunch::release -method nvx::host::sandbox::prepared::PreparedLaunch::scratch fn nvx::host::sandbox::require_file mod nvx::host::sandbox::scratch const nvx::host::sandbox::scratch::CLAIM_RELEASE_WAIT @@ -1265,29 +1077,9 @@ struct nvx::host::snapshot::BlockGeometry field nvx::host::snapshot::BlockGeometry::length field nvx::host::snapshot::BlockGeometry::logical_block_size field nvx::host::snapshot::BlockGeometry::physical_block_size -struct nvx::host::snapshot::CaptureClass -const nvx::host::snapshot::CaptureClass::VERSION -field nvx::host::snapshot::CaptureClass::boot_processors -method nvx::host::snapshot::CaptureClass::check -field nvx::host::snapshot::CaptureClass::cpu_profile -field nvx::host::snapshot::CaptureClass::distro -field nvx::host::snapshot::CaptureClass::guest_build -field nvx::host::snapshot::CaptureClass::hypervisor -method nvx::host::snapshot::CaptureClass::identity -field nvx::host::snapshot::CaptureClass::invariants -field nvx::host::snapshot::CaptureClass::memory -field nvx::host::snapshot::CaptureClass::memory_capacity -field nvx::host::snapshot::CaptureClass::network -field nvx::host::snapshot::CaptureClass::processors -field nvx::host::snapshot::CaptureClass::runtime -field nvx::host::snapshot::CaptureClass::scratch -method nvx::host::snapshot::CaptureClass::to_bytes struct nvx::host::snapshot::CaptureContext struct nvx::host::snapshot::CapturedSnapshot field nvx::host::snapshot::CapturedSnapshot::key -struct nvx::host::snapshot::ClassIdentity -method nvx::host::snapshot::ClassIdentity::as_bytes -method nvx::host::snapshot::ClassIdentity::from_bytes const nvx::host::snapshot::DESCRIPTION_FILE struct nvx::host::snapshot::Downtime field nvx::host::snapshot::Downtime::nanos @@ -1357,7 +1149,6 @@ method nvx::host::snapshot::SnapshotKey::invariants method nvx::host::snapshot::SnapshotKey::memory method nvx::host::snapshot::SnapshotKey::memory_block_size method nvx::host::snapshot::SnapshotKey::memory_capacity -method nvx::host::snapshot::SnapshotKey::new method nvx::host::snapshot::SnapshotKey::processors method nvx::host::snapshot::SnapshotKey::runtime method nvx::host::snapshot::SnapshotKey::scratch @@ -1387,26 +1178,6 @@ field nvx::host::snapshot::TimeContract::lineage field nvx::host::snapshot::TimeContract::tsc_frequency_hz field nvx::host::snapshot::TimeContract::tsc_tolerance_ppm fn nvx::host::snapshot::finish_capture -mod nvx::host::supervisor -struct nvx::host::supervisor::Inherited -method nvx::host::supervisor::Inherited::new -struct nvx::host::supervisor::Launched -method nvx::host::supervisor::Launched::detach -method nvx::host::supervisor::Launched::id -method nvx::host::supervisor::Launched::kill_and_confirm_exit -method nvx::host::supervisor::Launched::kill_and_wait -method nvx::host::supervisor::Launched::process -method nvx::host::supervisor::Launched::wait_timeout -struct nvx::host::supervisor::VmmCommand -field nvx::host::supervisor::VmmCommand::arguments -field nvx::host::supervisor::VmmCommand::breakaway_from_job -field nvx::host::supervisor::VmmCommand::inherit -method nvx::host::supervisor::VmmCommand::new -field nvx::host::supervisor::VmmCommand::program -field nvx::host::supervisor::VmmCommand::working_dir -fn nvx::host::supervisor::spawn -fn nvx::host::supervisor::spawn_command -fn nvx::host::supervisor::spawn_vm mod nvx::host::transport struct nvx::host::transport::HostConnection method nvx::host::transport::HostConnection::attach @@ -1561,7 +1332,6 @@ field nvx::protocol::OuterHeader::sequence struct nvx::protocol::ProtocolError method nvx::protocol::ProtocolError::into_message method nvx::protocol::ProtocolError::message -const nvx::protocol::WORKLOAD_ACCOUNT_TOKEN mod nvx::protocol::cmdline const nvx::protocol::cmdline::DNS_TOKEN const nvx::protocol::cmdline::GATEWAY_TOKEN diff --git a/nvx-rs/src/host/cli/sandbox/state.rs b/nvx-rs/src/host/cli/sandbox/state.rs index dfe30b68e..15729d333 100644 --- a/nvx-rs/src/host/cli/sandbox/state.rs +++ b/nvx-rs/src/host/cli/sandbox/state.rs @@ -8,8 +8,8 @@ //! caller keeps those paths stable while the VM uses them. //! //! Provision checks the images, the workload and the export plan of its options with -//! [`check_images`] and [`check_workload`], as the plan of the sandbox profile checked them, and -//! each start checks them again in the record, with [`Config::check`]. +//! [`check_images`] and [`check_workload`], and each start checks them again in the record, with +//! [`Config::check`]. //! //! The network is the [`NetworkRecord`] of the sandbox's network options, from which each start //! builds the network's policy again, as provision built it. @@ -117,10 +117,10 @@ fn resolved(path: &Path) -> Result { path_string(&paths::resolve(path).map_err(Exit::from_os_error)?) } -/// Refuses the workload of a sandbox as the plan of the sandbox profile refused it: an -/// `entrypoint` that is not an absolute path or that holds whitespace, a control character, or a -/// quote, an argument of `args` that is empty or holds one, a `hostname` that is not a lowercase -/// RFC 1123 label of up to 63 characters, and the root's UID or GID in `identity`. +/// Refuses the workload of a sandbox: an `entrypoint` that is not an absolute path or that holds +/// whitespace, a control character, or a quote, an argument of `args` that is empty or holds one, +/// a `hostname` that is not a lowercase RFC 1123 label of up to 63 characters, and the root's UID +/// or GID in `identity`. pub(crate) fn check_workload( entrypoint: &str, args: &[String], @@ -152,8 +152,7 @@ pub(crate) fn check_workload( Ok(()) } -/// Refuses the images of a sandbox as the plan of the sandbox profile refused them before its -/// launch: `distro`, and `scratch`, the scratch image of an upper on scratch, must each be a +/// Refuses the images of a sandbox before its launch: `distro`, and `scratch`, the scratch image of an upper on scratch, must each be a /// regular file whose path, and the path to which it resolves, has no comma, semicolon, or /// unbalanced bracket, and does not end in an extension with which OpenVMM would not attach it as /// a raw image. It also refuses `export`, if any, once its directories and files overlap, by path diff --git a/nvx-rs/src/host/cli/snapshots/capture/tests.rs b/nvx-rs/src/host/cli/snapshots/capture/tests.rs index 09f8ac9c2..94c82360f 100644 --- a/nvx-rs/src/host/cli/snapshots/capture/tests.rs +++ b/nvx-rs/src/host/cli/snapshots/capture/tests.rs @@ -13,7 +13,7 @@ use super::*; use crate::host::config::NicConfig; use crate::host::platform; use crate::host::process::{BootId, Opened, ProcessRef}; -use crate::host::snapshot::tests::{ManifestSpec, SnapshotFixture, class}; +use crate::host::snapshot::tests::{ManifestSpec, SnapshotFixture, context}; use crate::host::snapshot::{MANIFEST_NAME, Snapshot}; use crate::host::spawn::{self, VmmCommand}; use crate::protocol::{ @@ -81,7 +81,8 @@ impl Fixture { options.timeout = timeout; let result = capture(&options, &runtime(self.root.path()), |prepared| { write_recipe(self.root.path(), &prepared, behavior); - // As `spawn_vm` does, this hands the VMM its capability on its standard input. + // As the crate's spawn of a VMM does, this hands the VMM its capability on its standard + // input. stdin.write_all(prepared.capability().as_bytes()).unwrap(); drop(stdin); Ok(process) @@ -187,7 +188,7 @@ fn runtime(root: &Path) -> Runtime { openvmm: std::env::current_exe().unwrap(), kernel: root.join("fake-kernel"), initramfs: root.join("fake-initramfs"), - identity: class().runtime, + identity: context().runtime, hypervisor: if cfg!(windows) { crate::Hypervisor::Whp } else { @@ -262,17 +263,20 @@ fn write_recipe(root: &Path, prepared: &Prepared, behavior: &str) { fn fake_spawn(root: &Path, prepared: Prepared, behavior: &str) -> Process { write_recipe(root, &prepared, behavior); - let command = VmmCommand::new( - std::env::current_exe().unwrap(), - ["--exact", HELPER, "--ignored", "--nocapture"] + let command = VmmCommand { + program: std::env::current_exe().unwrap(), + arguments: ["--exact", HELPER, "--ignored", "--nocapture"] .map(Into::into) .to_vec(), - prepared.outputs().log.parent().unwrap(), - ); - let child = spawn::spawn_vm( + working_dir: prepared.outputs().log.parent().unwrap().to_path_buf(), + breakaway_from_job: false, + inherit: Vec::new(), + }; + let child = spawn::spawn_vm_with_mode( &command, prepared.capability().as_bytes(), prepared.log().try_clone().unwrap(), + false, ) .unwrap(); let process = child.process().unwrap(); diff --git a/nvx-rs/src/host/cli/store/tests/liveness.rs b/nvx-rs/src/host/cli/store/tests/liveness.rs index fcc522311..5f054d1b8 100644 --- a/nvx-rs/src/host/cli/store/tests/liveness.rs +++ b/nvx-rs/src/host/cli/store/tests/liveness.rs @@ -110,18 +110,21 @@ fn the_spawned_process_inherits_the_preparations_claim() { let mut locked = state.lock(Deadline::after(10.0)).unwrap(); drop(locked.begin_launch(&prepared, &runtime).unwrap()); let journal = locked.journal().unwrap().unwrap(); - let command = VmmCommand::new( - std::env::current_exe().unwrap(), - ["--exact", HELPER, "--ignored", "--nocapture"] + let command = VmmCommand { + program: std::env::current_exe().unwrap(), + arguments: ["--exact", HELPER, "--ignored", "--nocapture"] .map(Into::into) .to_vec(), - state.dir(), - ); + working_dir: state.dir().to_path_buf(), + breakaway_from_job: false, + inherit: Vec::new(), + }; let child = Running(Some( - spawn::spawn_vm( + spawn::spawn_vm_with_mode( &command, prepared.capability().as_bytes(), prepared.log().try_clone().unwrap(), + false, ) .unwrap(), )); diff --git a/nvx-rs/src/host/cli/store/tests/recovery.rs b/nvx-rs/src/host/cli/store/tests/recovery.rs index 04aaadc86..91b726e05 100644 --- a/nvx-rs/src/host/cli/store/tests/recovery.rs +++ b/nvx-rs/src/host/cli/store/tests/recovery.rs @@ -143,18 +143,21 @@ mod process { &prepared.outputs().endpoint, ) .unwrap(); - let command = VmmCommand::new( - std::env::current_exe().unwrap(), - ["--exact", HELPER, "--ignored", "--nocapture"] + let command = VmmCommand { + program: std::env::current_exe().unwrap(), + arguments: ["--exact", HELPER, "--ignored", "--nocapture"] .map(Into::into) .to_vec(), - state.dir(), - ); + working_dir: state.dir().to_path_buf(), + breakaway_from_job: false, + inherit: Vec::new(), + }; let child = Running(Some( - spawn::spawn_vm( + spawn::spawn_vm_with_mode( &command, prepared.capability().as_bytes(), prepared.log().try_clone().unwrap(), + false, ) .unwrap(), )); diff --git a/nvx-rs/src/host/config/mod.rs b/nvx-rs/src/host/config/mod.rs index f7152cbde..e81c48e8c 100644 --- a/nvx-rs/src/host/config/mod.rs +++ b/nvx-rs/src/host/config/mod.rs @@ -38,7 +38,8 @@ pub const COMMAND_LINE_BUDGET: usize = 1024; /// OpenVMM's command line is held to it on every host, so a policy behaves alike on all of them. pub const PROCESS_COMMAND_LINE_LIMIT: usize = 32_767; -/// Room for the runtime's files and the VM's output paths, as legacy adapter planning reserves. +/// Room for the runtime's files and the VM's output paths, which +/// [`VmConfig::check_command_line`] keeps. pub(crate) const OPENVMM_COMMAND_LINE_RESERVE: usize = 2048; /// The guest address of a VM's NIC at a cold boot whose configuration names none, in the form @@ -674,8 +675,8 @@ impl VmConfig { /// host, without reading or changing the host, with 2048 UTF-16 code units reserved for the /// runtime's files, launch options, and the VM's output paths. /// - /// Like legacy adapter planning, the reserve foresees runtime-file and output names of up to - /// 260 code units each, Windows' `MAX_PATH`. It does not certify arbitrary runtime paths or + /// The reserve foresees runtime-file and output names of up to 260 code units each, Windows' + /// `MAX_PATH`. It does not certify arbitrary runtime paths or /// CPU-profile names: [`crate::host::vm::create`] checks the complete command line with the /// actual runtime and output names, without a reserve. /// A listener's backend prefix is counted here; its endpoint name stays in the reserve. @@ -1160,11 +1161,15 @@ fn reject_container_backend( } /// Preserves the legacy sandbox refusal set used by snapshot and clone flows. +#[cfg_attr( + not(feature = "cli"), + allow(dead_code, reason = "only the CLI's sandboxes refuse container images") +)] pub(crate) fn reject_container_path(path: &Path, shown: &str) -> Result<()> { reject_container_backend(path, shown, true, false) } -/// Common cold-boot inputs, shared with the legacy VmmLaunch without narrowing its public types. +/// The OpenVMM arguments of the inputs that every cold boot names. pub(crate) fn boot_arguments( hypervisor: Hypervisor, memory_mib: u64, diff --git a/nvx-rs/src/host/config/tests.rs b/nvx-rs/src/host/config/tests.rs index 2ffbdafbc..51b8d2eb0 100644 --- a/nvx-rs/src/host/config/tests.rs +++ b/nvx-rs/src/host/config/tests.rs @@ -7,12 +7,9 @@ use std::time::Duration; use super::*; use crate::activation::MachineIdMode; -use crate::host::artifacts::Artifacts; use crate::host::filesystem::{Bind, ExportChild, Owner}; -use crate::host::launch::LaunchSpec; use crate::host::network::{Access, EgressRule, Protocol}; use crate::host::runtime::{ArtifactIdentity, GuestPowerAction, LaunchOptions, RuntimeIdentity}; -use crate::host::sandbox::{SandboxDistro, SandboxPlan, SandboxSettings, SandboxUpper}; use crate::hypervisor::Hypervisor; use crate::protocol::{cmdline, parse_host_mapping_count}; use crate::uuid::Uuid; @@ -509,44 +506,8 @@ fn each_refusal_of_a_capture_names_what_it_refuses() { } } -impl VmConfig { - fn legacy_command_line(&self) -> String { - let hostname = String::from_utf8(self.view.hostname.clone()).unwrap(); - match &self.root { - RootConfig::Blockless => { - let mut spec = LaunchSpec::new( - Artifacts::new("/openvmm", "/vmlinux", "/initramfs"), - Hypervisor::Kvm, - ); - spec.hostname = hostname; - spec.filesystem = self.filesystem.clone(); - spec.kernel_command_line().unwrap() - } - RootConfig::Layered(root) => { - let upper = match &root.upper { - UpperConfig::Scratch { path, .. } => SandboxUpper::scratch_at(path.clone()), - UpperConfig::Ram { .. } => SandboxUpper::RAM, - }; - SandboxPlan::new( - SandboxDistro::at(root.image.clone(), root.format), - upper, - SandboxSettings { - hostname, - ..SandboxSettings::default() - }, - ) - .unwrap() - .with_export(self.filesystem.clone()) - .unwrap() - .kernel_command_line("") - .unwrap() - } - } - } -} - #[test] -fn cold_command_lines_keep_the_old_root_view_and_mapping_values() { +fn cold_command_lines_carry_the_root_the_view_and_the_mapping_count() { let mut configs = vec![blockless()]; let uppers = [ UpperConfig::Scratch { @@ -602,19 +563,18 @@ fn cold_command_lines_keep_the_old_root_view_and_mapping_values() { ..config.clone() }; config.view.machine_id = machine_id; - let old = config.legacy_command_line(); let actual = config.kernel_command_line().unwrap(); - let old_root = RootLayout::parse(old.as_bytes()).unwrap(); let root = RootLayout::parse(actual.as_bytes()).unwrap(); assert_eq!( - root.map(|root| (root.image, root.upper)), - old_root.map(|root| (root.image, root.upper)) + root.is_some(), + matches!(config.root, RootConfig::Layered(_)) ); - if let Some(old_root) = old_root { - assert!(actual.starts_with(&old_root.tokens().join(" "))); + if let Some(root) = root { + assert!(actual.starts_with(&root.tokens().join(" "))); } if let RootConfig::Layered(configured) = &config.root { let root = root.unwrap(); + assert_eq!(root.image, configured.format); assert_eq!(root.overlay, Some(configured.overlay)); assert_eq!( root.gpt_first_partition_bound, @@ -622,10 +582,12 @@ fn cold_command_lines_keep_the_old_root_view_and_mapping_values() { ); match configured.upper { UpperConfig::Scratch { journal, .. } => { + assert_eq!(root.upper, RootUpper::Scratch); assert_eq!(root.journal, Some(journal)); assert_eq!(root.max_memory_percent, None); } UpperConfig::Ram { max_memory_percent } => { + assert_eq!(root.upper, RootUpper::Ram); assert_eq!(root.journal, None); assert_eq!(root.max_memory_percent, Some(max_memory_percent)); } @@ -633,14 +595,7 @@ fn cold_command_lines_keep_the_old_root_view_and_mapping_values() { } let decoded = view::parse(actual.as_bytes()).unwrap().unwrap(); assert_eq!(decoded, config.view); - let old_hostname = cmdline::init_tokens(old.as_bytes()) - .find(|token| matches!(token.name, b"hostname" | b"nvx_hostname")) - .unwrap() - .value - .unwrap(); - assert_eq!(decoded.hostname, old_hostname); let count = parse_host_mapping_count(actual.as_bytes()).unwrap(); - assert_eq!(count, parse_host_mapping_count(old.as_bytes()).unwrap()); assert_eq!(count, if filesystem.is_some() { 2 } else { 0 }); let names: Vec<_> = cmdline::init_tokens(actual.as_bytes()) .skip(root.map_or(0, |root| root.tokens().len())) diff --git a/nvx-rs/src/host/launch.rs b/nvx-rs/src/host/launch.rs deleted file mode 100644 index 1bd263e99..000000000 --- a/nvx-rs/src/host/launch.rs +++ /dev/null @@ -1,456 +0,0 @@ -//! OpenVMM command line of a managed microVM, and the inputs that an adapter's launch shares. -//! -//! The VM runs the guest's own Alpine Linux userland directly, with no sandbox layers or scratch -//! disk. OpenVMM's managed lifecycle adds the control tty and workload identity tokens; the guest -//! init then starts the managed agent, which runs workloads as the configured non-root identity. -//! The arguments mirror the managed-lifecycle check of `scripts/nvx_tools/microvm_tests.py`. -//! -//! [`LaunchSpec::check_process_command_line`] holds a managed microVM's command line to -//! [`PROCESS_COMMAND_LINE_LIMIT`], the longest that Windows starts a process with, on every host, -//! so that a launch behaves alike on all of them, as [`process_command_line_units`] counts it and -//! [`fit_process_command_line`] checks it, and any other caller may check a command line with -//! them. -//! -//! An adapter's [`PreparedLaunch`](super::sandbox::PreparedLaunch) names the inputs of its launch -//! as a [`LaunchSpec`] does, whatever its devices: [`VmmLaunch`] renders them for both, and -//! [`PreparedLaunch::adapter_arguments`](super::sandbox::PreparedLaunch::adapter_arguments) holds -//! an adapter's command line to the limit too. - -use std::ffi::OsString; -use std::path::{Path, PathBuf}; -use std::time::Duration; - -use super::artifacts::Artifacts; -use super::config; -use super::filesystem::ExportPlan; -use super::network::{self, NetworkPolicy}; -use crate::error::{Error, Result}; -use crate::hypervisor::Hypervisor; -use crate::net::{is_valid_guest_network, is_valid_hostname}; -use crate::protocol::WORKLOAD_ACCOUNT_TOKEN; -use crate::protocol::cmdline::{self, Parameter}; - -pub(crate) use super::config::fit_arguments; -pub use super::config::{ - COMMAND_LINE_BUDGET, DEFAULT_GUEST_NETWORK, PROCESS_COMMAND_LINE_LIMIT, - fit_process_command_line, process_command_line_units, -}; -/// Part of that limit kept for what may change between provision and start, such as the -/// configured OpenVMM and guest paths. -const PROCESS_COMMAND_LINE_RESERVE: usize = 1024; -/// Longest accepted start, control, or stop timeout, or extra time to wait for a workload's -/// outcome: 30 days. -/// -/// The bound keeps every deadline derived from these timeouts, including a workload's own -/// timeout plus the extra time, representable as an [`Instant`](std::time::Instant) and within -/// the roughly 49-day range of Windows' millisecond waits. -pub const MAX_TIMEOUT: Duration = Duration::from_secs(30 * 24 * 60 * 60); -/// Default guest memory in MiB. -pub const DEFAULT_MEMORY_MIB: u32 = 256; -/// Default workload hostname. -pub const DEFAULT_HOSTNAME: &str = "nvx-sandbox"; -/// The name of the kernel parameter that sets the workload hostname, which follows the extra -/// kernel parameters on every command line. -const HOSTNAME_PARAMETER: &str = "hostname"; - -pub use crate::protocol::workload::WorkloadIdentity; - -impl WorkloadIdentity { - /// Alpine's `nobody`, which every guest image has. - pub const DEFAULT: Self = Self { - uid: 65534, - gid: 65534, - create_account: false, - }; - - /// Returns the identity of the workloads of a VM. - /// - /// On Linux hosts, virtio-fs shows host owners and modes unchanged, so a fixed identity could - /// not read the calling user's private files or write to their directories. With - /// `map_host_identity`, the workloads of a VM that maps host paths therefore run under the - /// calling user's own IDs, and the guest creates an account for them, so host file - /// permissions apply to them as they do to that user. Root callers, VMs that map nothing, - /// and Windows hosts, which present mapped files as accessible to every guest user, keep - /// `uid` and `gid`. - pub fn select(uid: u32, gid: u32, map_host_identity: bool, maps_host_paths: bool) -> Self { - match host_identity(map_host_identity && maps_host_paths) { - Some((uid, gid)) => Self { - uid, - gid, - create_account: true, - }, - None => Self { - uid, - gid, - create_account: false, - }, - } - } -} - -/// Returns the calling user's IDs when `wanted`, unless the caller is root or the host is not a -/// Unix host. -fn host_identity(wanted: bool) -> Option<(u32, u32)> { - if !wanted { - return None; - } - #[cfg(unix)] - { - let (uid, gid) = super::sys::effective_ids(); - (uid != 0 && gid != 0).then_some((uid, gid)) - } - #[cfg(not(unix))] - None -} - -/// Everything that OpenVMM's command line for one VM depends on. -/// -/// Paths are used as given, and OpenVMM runs in the instance's directory, so make them absolute, -/// for example with [`absolute`](super::artifacts::absolute). The other settings must pass -/// [`check`](Self::check), which every method that builds a command line applies. -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct LaunchSpec { - /// The `openvmm` executable. - pub openvmm: PathBuf, - /// The NVX guest kernel (`vmlinux`). - pub kernel: PathBuf, - /// The NVX Alpine initramfs (`initramfs.cpio.gz`): the guest userland and its control agent. - pub initrd: PathBuf, - /// Hypervisor used by OpenVMM. - pub hypervisor: Hypervisor, - /// Guest memory in MiB; positive. - pub memory_mib: u32, - /// Identity of the workloads; neither ID may be root's. - pub identity: WorkloadIdentity, - /// UTS hostname of the workloads, set through the kernel's `hostname=` parameter: a lowercase - /// RFC 1123 label of up to 63 characters. - pub hostname: String, - /// Extra kernel parameters; see [`kernel_parameters_problem`] for the ones that are refused. - pub kernel_parameters: String, - /// Static guest IPv4 address and prefix used when a network device is attached, in the form - /// that [`is_valid_guest_network`] accepts. - /// - /// OpenVMM derives the guest's IPv6 address by embedding this one in `fd00::/96`: the default - /// `10.0.0.2/24` yields `fd00::a00:2/120`, whose gateway is `fd00::a00:1`. - pub guest_network: String, - /// Network posture; `None` attaches no network device. - pub network: Option, - /// Host paths to export; `None` maps nothing. - pub filesystem: Option, - /// Windows only: start OpenVMM outside the caller's job object so the VM survives the - /// caller. The job must permit breakaway. - pub breakaway_from_job: bool, -} - -impl LaunchSpec { - /// Creates a specification that runs `artifacts` under `hypervisor` with the defaults: - /// [`DEFAULT_MEMORY_MIB`], [`WorkloadIdentity::DEFAULT`], [`DEFAULT_HOSTNAME`], no extra - /// kernel parameters, [`DEFAULT_GUEST_NETWORK`], and neither a network device nor host paths. - pub fn new(artifacts: Artifacts, hypervisor: Hypervisor) -> Self { - Self { - openvmm: artifacts.openvmm, - kernel: artifacts.kernel, - initrd: artifacts.initrd, - hypervisor, - memory_mib: DEFAULT_MEMORY_MIB, - identity: WorkloadIdentity::DEFAULT, - hostname: DEFAULT_HOSTNAME.to_owned(), - kernel_parameters: String::new(), - guest_network: DEFAULT_GUEST_NETWORK.to_owned(), - network: None, - filesystem: None, - breakaway_from_job: false, - } - } - - /// Checks the settings that OpenVMM's command line carries: a positive memory size, a - /// non-root workload identity, a hostname that [`is_valid_hostname`] accepts, a guest network - /// that [`is_valid_guest_network`] accepts, and extra kernel parameters that - /// [`kernel_parameters_problem`] finds no problem with. - /// - /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) naming the first - /// setting that does not pass. - pub fn check(&self) -> Result<()> { - let unsupported = |message: String| Err(Error::unsupported(message)); - if self.memory_mib == 0 { - return unsupported("memory_mib must be positive".to_owned()); - } - if self.identity.uid == 0 || self.identity.gid == 0 { - return unsupported("workloads must run as a non-root UID and GID".to_owned()); - } - if !is_valid_hostname(&self.hostname) { - return unsupported(format!( - "hostname {:?} must be a lowercase RFC 1123 label of up to 63 characters", - self.hostname - )); - } - if !is_valid_guest_network(&self.guest_network) { - return unsupported(format!( - "guest_network {:?} must be an IPv4 address with a /1 to /30 prefix that is not \ - its network's network, broadcast, or gateway (first) address", - self.guest_network - )); - } - if let Some(problem) = kernel_parameters_problem(&self.kernel_parameters, &self.hostname) { - return unsupported(format!("kernel_parameters {problem}")); - } - Ok(()) - } - - /// Builds the kernel command line: the extra parameters, the workload hostname, and the - /// tokens that configure the guest agent, such as the number of host mappings that it - /// receives over the control channel. - /// - /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) when the - /// specification fails [`check`](Self::check), or when the command line exceeds - /// [`COMMAND_LINE_BUDGET`]. - pub fn kernel_command_line(&self) -> Result { - self.check()?; - self.checked_kernel_command_line() - } - - /// Builds the kernel command line of a specification that passed [`check`](Self::check). - fn checked_kernel_command_line(&self) -> Result { - let mut tokens = vec![base_command_line(&self.kernel_parameters, &self.hostname)]; - if self.identity.create_account { - tokens.push(WORKLOAD_ACCOUNT_TOKEN.to_owned()); - } - if let Some(plan) = &self.filesystem { - tokens.push(plan.kernel_token()); - } - let command_line = tokens.join(" "); - if command_line.len() + 1 > COMMAND_LINE_BUDGET { - return Err(Error::unsupported(format!( - "the kernel command line exceeds its {COMMAND_LINE_BUDGET}-byte budget; shorten \ - the extra kernel parameters or the hostname" - ))); - } - Ok(command_line) - } - - /// Builds the complete OpenVMM argument list for a VM whose control console listens on - /// `endpoint` and whose outcome report goes to `report`. - /// - /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) when the - /// specification fails [`check`](Self::check), or when the kernel command line or the network - /// posture cannot be expressed. - pub fn openvmm_arguments(&self, endpoint: &str, report: &Path) -> Result> { - self.check()?; - let mut arguments: Vec = vec!["--machine".into(), "microvm".into()]; - arguments.push("--microvm-workload-identity".into()); - arguments.push(format!("{}:{}", self.identity.uid, self.identity.gid).into()); - arguments.push("--microvm-lifecycle".into()); - arguments.push("managed".into()); - arguments.extend( - self.vmm_launch(endpoint, report) - .arguments(&self.checked_kernel_command_line()?), - ); - arguments.extend( - network::openvmm_arguments(self.network.as_ref(), &self.guest_network)? - .into_iter() - .map(OsString::from), - ); - if let Some(plan) = &self.filesystem { - arguments.extend(plan.openvmm_arguments().into_iter().map(OsString::from)); - } - Ok(arguments) - } - - /// Returns the inputs of the launch of a VM of the specification whose control console listens - /// on `endpoint` and whose outcome report goes to `report`: its OpenVMM executable, - /// hypervisor, memory, kernel, and initramfs, the endpoint, and the report. - /// - /// An adapter that launches its own guest with the specification's artifacts gives them to - /// [`PreparedLaunch::adapter_arguments`](super::sandbox::PreparedLaunch::adapter_arguments). - pub fn vmm_launch<'a>(&'a self, endpoint: &'a str, report: &'a Path) -> VmmLaunch<'a> { - VmmLaunch { - openvmm: &self.openvmm, - hypervisor: self.hypervisor, - memory_mib: self.memory_mib, - kernel: &self.kernel, - initrd: &self.initrd, - endpoint, - report, - } - } - - /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) if OpenVMM's command - /// line would not fit [`PROCESS_COMMAND_LINE_LIMIT`] with room to spare, as can happen with - /// many filesystem paths. - pub fn check_process_command_line(&self, endpoint: &str, report: &Path) -> Result<()> { - let arguments = self.openvmm_arguments(endpoint, report)?; - self.fit_process_command_line(&arguments, PROCESS_COMMAND_LINE_RESERVE) - } - - /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) unless OpenVMM's - /// command line with `arguments` leaves `reserve` of [`PROCESS_COMMAND_LINE_LIMIT`] unused, as - /// [`fit_process_command_line`] checks it. - pub(crate) fn fit_process_command_line( - &self, - arguments: &[OsString], - reserve: usize, - ) -> Result<()> { - fit_process_command_line(self.openvmm.as_os_str(), arguments, reserve) - } -} - -/// The inputs of a cold launch that OpenVMM's command line names besides the VM's devices and -/// its kernel command line: the OpenVMM executable that runs the VM, the hypervisor, the guest's -/// memory, kernel, and initramfs, the endpoint on which the control console listens, and the -/// path of OpenVMM's outcome report. -/// -/// [`arguments`](Self::arguments) renders them for a [`LaunchSpec`]'s launch, after the managed -/// lifecycle, and for an adapter's, after the devices of its plan, with -/// [`PreparedLaunch::adapter_arguments`](super::sandbox::PreparedLaunch::adapter_arguments). -/// Paths are used as given, and OpenVMM runs in the directory that its launch chooses, so make -/// them absolute. -#[derive(Clone, Copy, Debug, PartialEq, Eq)] -pub struct VmmLaunch<'a> { - /// The `openvmm` executable, which the command line runs. - pub openvmm: &'a Path, - /// Hypervisor used by OpenVMM. - pub hypervisor: Hypervisor, - /// Guest memory in MiB; positive. - pub memory_mib: u32, - /// The guest kernel (`vmlinux`). - pub kernel: &'a Path, - /// The initramfs. - pub initrd: &'a Path, - /// The endpoint on which the control console listens, such as the path of a socket. - pub endpoint: &'a str, - /// The path of OpenVMM's outcome report. - pub report: &'a Path, -} - -impl VmmLaunch<'_> { - /// The part of [`PROCESS_COMMAND_LINE_LIMIT`] that the planning of an adapter's launch keeps - /// for the rest of the launch, in UTF-16 code units, as - /// [`SandboxPlan::check_adapter_launch`](super::sandbox::SandboxPlan::check_adapter_launch) - /// leaves it: what the plan does not render. - /// - /// It holds the rest of a launch whose OpenVMM executable, kernel, initramfs, endpoint, and - /// outcome report take up to 260 code units each, Windows' `MAX_PATH`: the executable and the - /// arguments that [`arguments`](Self::arguments) renders besides the kernel command line, the - /// longest memory and hypervisor included, and the names by which a - /// [`PreparedLaunch`](super::sandbox::PreparedLaunch) gives OpenVMM the checked files of the - /// plan's two block devices, which take up to 262 code units more each than the paths of the - /// plan: a quoted final path of up to `MAX_PATH` behind `\\?\` in place of a path of at least - /// 4. A launch whose inputs are longer may still not fit, which its complete check refuses - /// before OpenVMM starts. - pub const RESERVE: usize = config::OPENVMM_COMMAND_LINE_RESERVE; - - /// Returns the arguments that name the launch's inputs, with the kernel command line - /// `kernel_command_line`: the VM in one process, `--single-process`, on the hypervisor, - /// `--hypervisor`, with the memory, `--memory {memory_mib}M`, the kernel, `--kernel`, the - /// initramfs, `--initrd`, and the kernel command line, `--cmdline`, then no virtio console, - /// `--virtio-console none`, the control console listening on the endpoint, - /// `--microvm-control-console listen={endpoint}`, authenticated by the capability that - /// OpenVMM reads from its standard input, `--microvm-control-auth-stdin`, and the outcome - /// report, `--microvm-report`. - pub fn arguments(&self, kernel_command_line: &str) -> Vec { - let mut arguments = config::boot_arguments( - self.hypervisor, - u64::from(self.memory_mib), - self.kernel, - self.initrd, - kernel_command_line, - ); - arguments.extend(config::console_arguments( - "none", - self.endpoint, - self.report, - )); - arguments - } -} - -/// Returns the start of every kernel command line that `extra` parameters and `hostname` -/// determine alone: the parameters and the workload hostname. -pub fn base_command_line(extra: &str, hostname: &str) -> String { - let hostname = format!("{HOSTNAME_PARAMETER}={hostname}"); - match extra.trim() { - "" => hostname, - extra => format!("{extra} {hostname}"), - } -} - -/// Describes why `extra` kernel parameters are unacceptable for VMs named `hostname`, if they -/// are. -/// -/// The parameters are checked as the guest's kernel reads them in the command line that -/// [`base_command_line`] renders, as [`cmdline::parameters`] reads it: trimmed, split at every -/// byte that the kernel takes for whitespace, including the 0xa0 that ends the UTF-8 encoding of -/// characters such as `à`, with names that match whether they spell a dash or an underscore. The -/// parameters must not contain NUL characters, quotes, or the `--` delimiter, after which the -/// kernel would leave the `hostname=` parameter that follows them to init. The parameters named -/// `tsc` or `hostname`, with or without a value, and those whose names start with `nvx_` or -/// `nvx-` are reserved. They share the guest's 1024-byte kernel command line with the `hostname=` -/// parameter and the tokens that configure the guest agent, so parameters and a hostname that -/// alone leave no room in it are refused too. The description continues a sentence whose subject -/// names the parameters. -pub fn kernel_parameters_problem(extra: &str, hostname: &str) -> Option { - if extra.contains('\0') { - return Some("contains a NUL character".to_owned()); - } - if extra.contains(['"', '\'']) { - return Some("must not contain quotes".to_owned()); - } - let extra = extra.trim(); - if let Some(reserved) = - cmdline::parameters(extra.as_bytes()).find(|¶meter| is_reserved(parameter)) - { - return Some(format!("uses the reserved token {:?}", text(reserved))); - } - // Every VM's command line starts with these parameters and the hostname, and ends with a - // NUL; launching then checks the tokens that each VM adds. - let command_line = base_command_line(extra, hostname); - // None of the parameters is named `hostname`, so the kernel reads none only when a `--` ends - // its parameters before the hostname's. - if !cmdline::parameters(command_line.as_bytes()) - .any(|parameter| parameter.is_named(HOSTNAME_PARAMETER)) - { - return Some("must not contain the -- delimiter".to_owned()); - } - (command_line.len() + 1 > COMMAND_LINE_BUDGET).then(|| { - format!( - "and the hostname leave no room in the {COMMAND_LINE_BUDGET}-byte kernel command line" - ) - }) -} - -/// Returns whether extra kernel parameters may not hold `parameter`: the workload hostname's, -/// which the launch sets; `tsc`, which the time ABI governs; and those that configure the guest, -/// whose names start with `nvx_` or, since the kernel matches them alike, `nvx-`. -fn is_reserved(parameter: Parameter<'_>) -> bool { - parameter.name.starts_with(b"nvx_") - || parameter.name.starts_with(b"nvx-") - || parameter.is_named("tsc") - || parameter.is_named(HOSTNAME_PARAMETER) -} - -/// Returns the text of `parameter` as the kernel reads it: `NAME=VALUE`, or `NAME` alone. -fn text(parameter: Parameter<'_>) -> String { - let name = String::from_utf8_lossy(parameter.name); - match parameter.value { - Some(value) => format!("{name}={}", String::from_utf8_lossy(value)), - None => name.into_owned(), - } -} - -/// Describes why a start, control, or stop timeout, or the extra time to wait for a workload's -/// outcome, is unacceptable, if it is: it must be at most [`MAX_TIMEOUT`], and positive when -/// `positive` is set. The description continues a sentence whose subject names the timeout. -pub fn timeout_problem(value: Duration, positive: bool) -> Option { - if positive && value.is_zero() { - return Some("must be positive".to_owned()); - } - (value > MAX_TIMEOUT).then(|| { - format!( - "must be at most {} days", - MAX_TIMEOUT.as_secs() / (24 * 60 * 60) - ) - }) -} - -#[cfg(test)] -mod tests; diff --git a/nvx-rs/src/host/launch/tests.rs b/nvx-rs/src/host/launch/tests.rs deleted file mode 100644 index 2cc5fa0ca..000000000 --- a/nvx-rs/src/host/launch/tests.rs +++ /dev/null @@ -1,550 +0,0 @@ -use proptest::prelude::*; - -use super::*; -use crate::ErrorKind; -use crate::host::config::arguments_units; -use crate::host::filesystem::{Bind, ExportChild, Owner}; -use crate::protocol::cmdline::tests::model; - -fn spec() -> LaunchSpec { - LaunchSpec::new( - Artifacts::new("/o", "/k/vmlinux", "/k/initrd"), - Hypervisor::Kvm, - ) -} - -#[test] -fn kernel_command_line_sets_the_hostname() { - let mut spec = spec(); - spec.kernel_parameters = "quiet loglevel=0".to_owned(); - assert_eq!( - spec.kernel_command_line().unwrap(), - "quiet loglevel=0 hostname=nvx-sandbox" - ); - spec.kernel_parameters = String::new(); - assert_eq!(spec.kernel_command_line().unwrap(), "hostname=nvx-sandbox"); - spec.kernel_parameters = "x".repeat(COMMAND_LINE_BUDGET); - assert_eq!( - spec.kernel_command_line().unwrap_err().kind(), - ErrorKind::Unsupported - ); -} - -#[test] -fn arguments_select_the_managed_direct_guest() { - let arguments = spec() - .openvmm_arguments("/state/t/control.sock", Path::new("/state/t/outcome.json")) - .unwrap(); - let arguments: Vec<&str> = arguments.iter().map(|a| a.to_str().unwrap()).collect(); - assert_eq!( - arguments, - [ - "--machine", - "microvm", - "--microvm-workload-identity", - "65534:65534", - "--microvm-lifecycle", - "managed", - "--single-process", - "--hypervisor", - "kvm", - "--memory", - "256M", - "--kernel", - "/k/vmlinux", - "--initrd", - "/k/initrd", - "--cmdline", - "hostname=nvx-sandbox", - "--virtio-console", - "none", - "--microvm-control-console", - "listen=/state/t/control.sock", - "--microvm-control-auth-stdin", - "--microvm-report", - "/state/t/outcome.json", - ] - ); - assert!(!arguments.contains(&"--microvm-sandbox-block")); -} - -#[test] -fn mappings_add_the_export_and_the_mapping_count() { - let mut spec = spec(); - spec.filesystem = Some(ExportPlan { - children: vec![ExportChild { - root: "/host/work".into(), - file: false, - writable: false, - denied: vec!["/host/work/secret".into()], - denied_identities: vec![None], - allowed: Vec::new(), - write: Vec::new(), - }], - binds: vec![Bind { - child: 0, - source: String::new(), - target: "/host/work".to_owned(), - read_only: true, - identity: None, - }], - owner: Owner::Vmm, - }); - assert_eq!( - spec.kernel_command_line().unwrap(), - "hostname=nvx-sandbox nvx_maps=1" - ); - spec.identity.create_account = true; - assert_eq!( - spec.kernel_command_line().unwrap(), - "hostname=nvx-sandbox nvx_workload_account=create nvx_maps=1" - ); - spec.identity.create_account = false; - let arguments = spec.openvmm_arguments("/e", Path::new("/r")).unwrap(); - let arguments: Vec<&str> = arguments.iter().map(|a| a.to_str().unwrap()).collect(); - assert_eq!( - arguments[arguments.len() - 6..], - [ - "--mount-aggregate", - "/run/nvx/hostfs/root", - "--mount-child", - "0,/host/work,ro", - "--mount-deny", - "/host/work/secret", - ] - ); -} - -#[test] -fn oversized_process_command_lines_are_rejected() { - let mut spec = spec(); - let report = Path::new("/state/t/outcome.json"); - spec.check_process_command_line("/state/t/control.sock", report) - .unwrap(); - let denied: Vec = (0..128) - .map(|index| format!("/host/work/{index:03}-{}", "d".repeat(240)).into()) - .collect(); - spec.filesystem = Some(ExportPlan { - children: vec![ExportChild { - root: "/host/work".into(), - file: false, - writable: false, - denied_identities: vec![None; denied.len()], - denied, - allowed: Vec::new(), - write: Vec::new(), - }], - binds: Vec::new(), - owner: Owner::Vmm, - }); - let error = spec - .check_process_command_line("/state/t/control.sock", report) - .unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported); -} - -#[test] -fn launches_may_use_the_reserve_that_provisioning_keeps() { - let endpoint = "/state/t/control.sock"; - let report = Path::new("/state/t/outcome.json"); - let mut spec = spec(); - let arguments = spec.openvmm_arguments(endpoint, report).unwrap(); - let base = process_command_line_units(spec.openvmm.as_os_str(), &arguments); - // Lengthening the program's path lengthens the command line alike. - let program = |extra: usize| PathBuf::from(format!("/o{}", "o".repeat(extra))); - spec.openvmm = program(PROCESS_COMMAND_LINE_LIMIT - base); - assert_eq!( - process_command_line_units(spec.openvmm.as_os_str(), &arguments), - PROCESS_COMMAND_LINE_LIMIT - ); - spec.fit_process_command_line(&arguments, 0).unwrap(); - let reserved = spec - .check_process_command_line(endpoint, report) - .unwrap_err(); - assert!( - reserved - .to_string() - .contains("allows, with 1024 held in reserve; map"), - "{reserved}" - ); - spec.openvmm = program(PROCESS_COMMAND_LINE_LIMIT - base + 1); - let error = spec.fit_process_command_line(&arguments, 0).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported); - assert!( - error - .to_string() - .contains("a Windows process command line allows; map"), - "{error}" - ); -} - -#[test] -fn every_launch_names_its_own_inputs_alike() { - let launch = VmmLaunch { - openvmm: Path::new("/o/openvmm"), - hypervisor: Hypervisor::Mshv, - memory_mib: 512, - kernel: Path::new("/k/vmlinux"), - initrd: Path::new("/k/initramfs.cpio.gz"), - endpoint: "/state/t/control.sock", - report: Path::new("/state/t/outcome.json"), - }; - let arguments = launch.arguments("quiet nvx_sandbox=1"); - let arguments: Vec<&str> = arguments.iter().map(|a| a.to_str().unwrap()).collect(); - assert_eq!( - arguments, - [ - "--single-process", - "--hypervisor", - "mshv", - "--memory", - "512M", - "--kernel", - "/k/vmlinux", - "--initrd", - "/k/initramfs.cpio.gz", - "--cmdline", - "quiet nvx_sandbox=1", - "--virtio-console", - "none", - "--microvm-control-console", - "listen=/state/t/control.sock", - "--microvm-control-auth-stdin", - "--microvm-report", - "/state/t/outcome.json", - ] - ); - // A managed launch names them after its lifecycle, with its own command line. - let spec = spec(); - let endpoint = "/state/t/control.sock"; - let report = Path::new("/state/t/outcome.json"); - let launch = spec.vmm_launch(endpoint, report); - assert_eq!( - launch, - VmmLaunch { - openvmm: Path::new("/o"), - hypervisor: Hypervisor::Kvm, - memory_mib: DEFAULT_MEMORY_MIB, - kernel: Path::new("/k/vmlinux"), - initrd: Path::new("/k/initrd"), - endpoint, - report, - } - ); - let managed = spec.openvmm_arguments(endpoint, report).unwrap(); - let own = launch.arguments(&spec.kernel_command_line().unwrap()); - assert_eq!(managed[6..6 + own.len()], own); - assert_eq!(managed[4..6], ["--microvm-lifecycle", "managed"]); -} - -#[test] -fn the_reserve_of_a_plan_holds_the_rest_of_a_launch_of_inputs_of_max_path() { - // Each input of `MAX_PATH` code units, with spaces, which the command line quotes. - let path = |name: &str| format!("C:\\{name} {}", "x".repeat(260 - 4 - name.len())); - let [openvmm, kernel, initrd, endpoint, report] = - ["openvmm", "vmlinux", "initrd", "control", "report"].map(path); - for input in [&openvmm, &kernel, &initrd, &endpoint, &report] { - assert_eq!(input.encode_utf16().count(), 260); - } - let launch = VmmLaunch { - openvmm: Path::new(&openvmm), - hypervisor: Hypervisor::Mshv, - memory_mib: u32::MAX, - kernel: Path::new(&kernel), - initrd: Path::new(&initrd), - endpoint: &endpoint, - report: Path::new(&report), - }; - // The plan's options count the kernel command line, whatever it is: the rest is the - // program and the other arguments. - let kernel_command_line = "nvx_sandbox=1"; - let own = launch.arguments(kernel_command_line); - let cmdline: Vec = ["--cmdline", kernel_command_line] - .map(OsString::from) - .into(); - let rest = - process_command_line_units(launch.openvmm.as_os_str(), &own) - arguments_units(&cmdline); - // A checked file's name that takes the most more than its path in the plan: a quoted - // final path of `MAX_PATH` behind `\\?\`, for a path of 4. - let named = |path: &str| arguments_units(&[OsString::from(format!("distro:file:{path}"))]); - let final_path = format!(r"\\?\{}", path("distro")); - let growth = named(&final_path) - named(r"C:\d"); - assert_eq!(growth, 262); - assert!( - rest + 2 * growth <= VmmLaunch::RESERVE, - "{rest} + 2 * {growth}" - ); -} - -#[test] -fn kernel_parameters_are_checked() { - let hostname = DEFAULT_HOSTNAME; - assert!(kernel_parameters_problem("quiet loglevel=0", hostname).is_none()); - for reserved in [ - "tsc=reliable", - "hostname=other", - "quiet nvx_exec=/bin/sh", - "a -- b", - "say \"hi\"", - "nul\0", - ] { - assert!( - kernel_parameters_problem(reserved, hostname).is_some(), - "{reserved:?}" - ); - } - // The parameters, a space, the hostname token, and a NUL fill the command line exactly. - let room = COMMAND_LINE_BUDGET - base_command_line("", hostname).len() - 2; - assert!(kernel_parameters_problem(&"x".repeat(room), hostname).is_none()); - assert_eq!( - kernel_parameters_problem(&"x".repeat(room + 1), hostname).unwrap(), - "and the hostname leave no room in the 1024-byte kernel command line" - ); -} - -#[test] -fn kernel_parameters_are_read_as_the_kernel_reads_them() { - let hostname = DEFAULT_HOSTNAME; - // `à`, U+00E0, is C3 A0 in UTF-8, and the kernel takes the byte 0xa0 for whitespace, so it - // reads the text after it as another parameter (microsoft/nvx#723). So do U+00A0 and - // U+0120, and the vertical tab, the form feed, and the carriage return. - for (extra, problem) in [ - ( - "quiet\u{e0}tsc=unstable", - "uses the reserved token \"tsc=unstable\"", - ), - ("quiet\u{e0}--", "must not contain the -- delimiter"), - ( - "quiet\u{a0}-- loglevel=0", - "must not contain the -- delimiter", - ), - ( - "quiet\u{a0}hostname=other", - "uses the reserved token \"hostname=other\"", - ), - ( - "quiet\u{120}nvx_maps=5", - "uses the reserved token \"nvx_maps=5\"", - ), - ( - "quiet\u{b}nvx_workload_account=create", - "uses the reserved token \"nvx_workload_account=create\"", - ), - ( - "quiet\u{c}tsc=reliable", - "uses the reserved token \"tsc=reliable\"", - ), - ( - "quiet\rhostname=other", - "uses the reserved token \"hostname=other\"", - ), - // The kernel matches names whether they spell a dash or an underscore. A launch also - // reserves `tsc` and `hostname` without a value: the early handler of `hostname` - // copies the value that such a parameter lacks. - ("nvx-maps=5", "uses the reserved token \"nvx-maps=5\""), - ("tsc", "uses the reserved token \"tsc\""), - ("quiet hostname", "uses the reserved token \"hostname\""), - // The command line starts with the parameters trimmed of any whitespace. - ( - "\u{2003}tsc=unstable", - "uses the reserved token \"tsc=unstable\"", - ), - ] { - assert_eq!( - kernel_parameters_problem(extra, hostname).as_deref(), - Some(problem), - "{extra:?}" - ); - } - // The kernel reads one parameter where none of its whitespace separates two, and a `--` - // with a value is no delimiter. - for extra in [ - "quiet\u{2003}tsc=unstable", - "quiet\u{3000}nvx_maps=5", - "quiet\u{800}tsc=unstable", - "x=tsc=unstable", - "tscx=1", - "hostnames=a", - "xnvx_maps=5", - "a--", - "--=1", - ] { - assert_eq!( - kernel_parameters_problem(extra, hostname), - None, - "{extra:?}" - ); - } -} - -/// Returns a strategy for what the kernel takes for whitespace between two parameters: an -/// ASCII byte that `isspace` accepts, or a character above U+007F whose code point ends with -/// the six bits 0x20, which UTF-8 encodes in a last byte of 0xa0. -fn kernel_separator() -> impl Strategy { - prop_oneof![ - prop::sample::select(vec![' ', '\t', '\n', '\u{b}', '\u{c}', '\r']), - (0x80_u32..0x11_0000) - .prop_filter_map("a surrogate", |code| char::from_u32((code & !0x3f) | 0x20)), - ] - .prop_map(String::from) -} - -/// Returns a strategy for extra kernel parameters: mostly letters, equals signs, dashes, -/// underscores, the names that a launch reserves, and the whitespace of the kernel and of -/// Unicode, and now and then a quote or a NUL. -fn extra_parameters() -> impl Strategy { - /// The pieces of the parameters. U+0800 holds the byte 0xa0 before its last byte, and - /// U+2003 and U+3000 hold none. - const PIECES: [&str; 23] = [ - "a", "b", "=", "-", "--", "_", "nvx_", "nvx-", "tsc", "hostname", " ", "\t", "\n", "\u{b}", - "\u{c}", "\r", "\u{a0}", "\u{e0}", "\u{120}", "\u{2020}", "\u{800}", "\u{2003}", - "\u{3000}", - ]; - prop::collection::vec( - prop_oneof![ - 24 => prop::sample::select(PIECES.to_vec()), - 1 => prop::sample::select(vec!["\"", "'", "\0"]), - ], - 0..24, - ) - .prop_map(|pieces| pieces.concat()) -} - -proptest! { - /// No reserved parameter passes, whatever separates it from the parameter before it. - #[test] - fn reserved_parameters_are_refused_whatever_separates_them( - before in "[a-z]{1,8}", - separator in kernel_separator(), - reserved in prop::sample::select(vec![ - "tsc=unstable", - "tsc", - "hostname=other", - "hostname", - "nvx_maps=5", - "nvx-maps=5", - "nvx_workload_account=create", - "--", - ]), - after in "( [a-z]{1,8})?", - ) { - let extra = format!("{before}{separator}{reserved}{after}"); - prop_assert!(kernel_parameters_problem(&extra, DEFAULT_HOSTNAME).is_some(), "{:?}", extra); - } - - /// The parameters that a launch accepts are those in which the kernel reads no reserved - /// parameter, as it compares names, and after which it reads the `hostname=` parameter: - /// the parameters hold no NUL and no quote, and no `--` ends them. - #[test] - fn the_kernel_reads_the_hostname_after_accepted_parameters(extra in extra_parameters()) { - let hostname = DEFAULT_HOSTNAME; - let reading = model::read(base_command_line(&extra, hostname).as_bytes()); - let reserved = |name: &[u8]| { - model::parameqn(name, b"nvx_", 4) - || model::parameq(name, b"tsc") - || model::parameq(name, b"hostname") - }; - let readable = !extra.contains(['\0', '"', '\'']) - && reading.rest.is_none() - && reading.parameters.split_last().is_some_and(|((name, value), others)| { - name == b"hostname" - && value.as_deref() == Some(hostname.as_bytes()) - && !others.iter().any(|(name, _)| reserved(name)) - }); - prop_assert_eq!( - kernel_parameters_problem(&extra, hostname).is_none(), - readable, - "{:?}", - extra - ); - } -} - -#[test] -fn every_command_line_checks_the_specification() { - let endpoint = "/state/t/control.sock"; - let report = Path::new("/state/t/outcome.json"); - spec().check().unwrap(); - let changes: [fn(&mut LaunchSpec); 11] = [ - |spec| spec.memory_mib = 0, - |spec| spec.identity.uid = 0, - |spec| spec.identity.gid = 0, - // A hostname cannot smuggle further kernel parameters in. - |spec| spec.hostname = "a nvx_maps=1".to_owned(), - |spec| spec.hostname = "Sandbox".to_owned(), - |spec| spec.guest_network = "10.0.0.1/24".to_owned(), - |spec| spec.kernel_parameters = "nvx_maps=5".to_owned(), - |spec| spec.kernel_parameters = "nvx_workload_account=create".to_owned(), - |spec| spec.kernel_parameters = "quiet hostname=other".to_owned(), - // The kernel reads the byte 0xa0 of `à` as whitespace (microsoft/nvx#723). - |spec| spec.kernel_parameters = "quiet\u{e0}tsc=unstable".to_owned(), - |spec| spec.kernel_parameters = "quiet\u{e0}--".to_owned(), - ]; - for change in changes { - let mut spec = spec(); - change(&mut spec); - for error in [ - spec.check().unwrap_err(), - spec.kernel_command_line().unwrap_err(), - spec.openvmm_arguments(endpoint, report).unwrap_err(), - spec.check_process_command_line(endpoint, report) - .unwrap_err(), - ] { - assert_eq!(error.kind(), ErrorKind::Unsupported, "{spec:?}"); - } - } - let mut root = spec(); - root.identity.gid = 0; - assert_eq!( - root.check().unwrap_err().to_string(), - "workloads must run as a non-root UID and GID" - ); - let mut reserved = spec(); - reserved.kernel_parameters = "nvx_maps=5".to_owned(); - assert_eq!( - reserved.check().unwrap_err().to_string(), - "kernel_parameters uses the reserved token \"nvx_maps=5\"" - ); -} - -#[test] -fn timeouts_are_bounded() { - assert_eq!(timeout_problem(Duration::ZERO, false), None); - assert_eq!( - timeout_problem(Duration::ZERO, true).unwrap(), - "must be positive" - ); - assert_eq!(timeout_problem(MAX_TIMEOUT, true), None); - for beyond in [MAX_TIMEOUT + Duration::from_nanos(1), Duration::MAX] { - assert_eq!( - timeout_problem(beyond, false).unwrap(), - "must be at most 30 days" - ); - } -} - -#[test] -fn workloads_that_map_host_paths_take_the_host_identity() { - let configured = WorkloadIdentity::select(1000, 1000, false, true); - assert_eq!( - configured, - WorkloadIdentity { - uid: 1000, - gid: 1000, - create_account: false, - } - ); - assert_eq!(WorkloadIdentity::select(7, 8, true, false).uid, 7); - let mapped = WorkloadIdentity::select(7, 8, true, true); - match host_identity(true) { - Some((uid, gid)) => assert_eq!( - mapped, - WorkloadIdentity { - uid, - gid, - create_account: true, - } - ), - None => assert_eq!(mapped, WorkloadIdentity::select(7, 8, false, false)), - } -} diff --git a/nvx-rs/src/host/mod.rs b/nvx-rs/src/host/mod.rs index baca8e201..1a372a8cb 100644 --- a/nvx-rs/src/host/mod.rs +++ b/nvx-rs/src/host/mod.rs @@ -11,18 +11,12 @@ //! that its workload has of it, its network, its export, its boot console, and what only its //! cold boot takes, with what a capture of it adds, a [`CaptureConfig`](config::CaptureConfig), //! whose identity names the capture's snapshots. -//! - [`launch`] builds OpenVMM's command line from a [`LaunchSpec`](launch::LaunchSpec): the -//! managed lifecycle, the workload identity, the kernel command line, and the network and -//! filesystem options. //! - [`network`] lowers egress rules, and the egress policy files that `nvx-rs run` and //! `nvx-rs sandbox` accept, into OpenVMM's rules, and [`filesystem`] plans the export of host //! paths through OpenVMM's aggregate virtio-fs device. //! - [`report`] reads OpenVMM's outcome report. -//! - [`supervisor`] is the transitional public path, until SL6.3 (#1122), of the low-level spawn -//! of the private `spawn` module, which starts a VMM detached, passing it only the descriptors -//! or handles that the caller allows, and no other descriptor or handle of the caller's. //! - [`snapshot`] holds the compatibility key, [`SnapshotKey`](snapshot::SnapshotKey), derived -//! from a [`CaptureClass`](snapshot::CaptureClass) and [`SnapshotManifest`](snapshot::SnapshotManifest), +//! from a capture's context and [`SnapshotManifest`](snapshot::SnapshotManifest), //! the strict manifest reader, and the capture's public time and eligibility information. //! - [`transport`] verifies a VM's endpoint and runs the host's role of the NVXS session for it, //! for any application protocol, over a platform transport. The private `session` (feature @@ -36,9 +30,7 @@ //! layers and a private ext4 scratch disk, with live shares, and the checks of its options. //! [`sandbox::distro`] binds a distro layer to what explicit verification found, so that a //! launch checks the layer through [`seal`], which seals open files from their metadata alone, -//! and [`sandbox::scratch`] binds and claims a scratch image. A -//! [`PreparedLaunch`](sandbox::PreparedLaunch) hands the VMM the files that those checks -//! opened, which it inherits through the [`supervisor`]'s allowlist. +//! and [`sandbox::scratch`] binds and claims a scratch image. //! - [`vm`] holds a caller-owned VM directory, private to the current user, the fixed names of //! its files and endpoints, and a checked cold, capture or restore [`Prepared`](vm::Prepared) //! launch with its opened log and control credential, which the library never writes. A @@ -52,7 +44,9 @@ //! sandboxes, looks up and downloads artifacts, and compiles egress policies. //! //! The private `paths` module makes host paths absolute and resolves their links, with -//! `std::path` and `std::fs`, for the sandbox launch profile and the CLI. +//! `std::path` and `std::fs`, for the sandbox launch profile and the CLI. The private `spawn` +//! module starts a VMM for [`vm`], detached, passing it only the descriptors or handles that the +//! caller allows, and no other descriptor or handle of the caller's. //! //! The children of `host` depend on one another without cycles: each builds only on children //! beneath it, as [`vm`] does on `spawn`, and `spawn` on [`platform`]. An item that @@ -73,7 +67,6 @@ pub mod artifacts; pub mod cli; pub mod config; pub mod filesystem; -pub mod launch; pub mod network; pub mod nvxs; pub(crate) mod paths; @@ -90,7 +83,6 @@ pub mod seal; pub(crate) mod session; pub mod snapshot; mod spawn; -pub mod supervisor; #[cfg(unix)] #[allow(unsafe_code)] mod sys; diff --git a/nvx-rs/src/host/platform/inherit.rs b/nvx-rs/src/host/platform/inherit.rs index 3b70684a2..b761f9829 100644 --- a/nvx-rs/src/host/platform/inherit.rs +++ b/nvx-rs/src/host/platform/inherit.rs @@ -1,18 +1,14 @@ -//! The descriptors (Unix) or handles (Windows) of a caller that a VMM inherits when -//! [`supervisor::spawn`](crate::host::supervisor::spawn) starts it, and the lock that keeps the -//! caller's other spawns out of the window in which they are inheritable. +//! The descriptors (Unix) or handles (Windows) of a caller that a VMM inherits when the crate's +//! spawn of a VMM, which [`vm::spawn`](crate::host::vm::spawn) makes, starts it, and the lock that +//! keeps the caller's other spawns out of the window in which they are inheritable. #[cfg(windows)] use std::cell::RefCell; -use std::io; use std::marker::PhantomData; #[cfg(unix)] use std::os::fd::{AsFd, BorrowedFd}; #[cfg(windows)] use std::os::windows::io::{AsHandle, BorrowedHandle}; -#[cfg(target_os = "linux")] -use std::path::PathBuf; -use std::process::{Child, Command}; use std::sync::MutexGuard; #[cfg(windows)] use std::sync::{Mutex, PoisonError}; @@ -40,7 +36,7 @@ thread_local! { /// makes a handle of this process inheritable, so that a process that the holder spawns /// meanwhile, through any API, inherits no handle of a VMM. /// -/// On Windows, [`spawn`](crate::host::supervisor::spawn) makes the VMM's standard handles, the +/// On Windows, a spawn of a VMM makes the VMM's standard handles, the /// read end of the pipe that carries its capability and two handles of its log, and the handles /// that it allows, inheritable while it creates the VMM. A process that another thread creates in /// that window with inheritance but without a list of handles, as `std::process::Command` creates @@ -49,9 +45,8 @@ thread_local! { /// holds the VMM's log, and the handles that the VMM is allowed, such as the claim of a scratch /// image, after the VMM has exited. It holds no claim of the VMM's launch on the log, which the /// VMM's job then decides, so a recovery that finds the VMM gone still ends the launch. Each spawn -/// of a VMM holds this lock throughout that window, so a process that the caller spawns under it, -/// as [`spawn_command`] spawns one, inherits none of those handles, and one that the caller spawns -/// otherwise still can. +/// of a VMM holds this lock throughout that window, so a process that the caller spawns under it +/// inherits none of those handles, and one that the caller spawns otherwise still can. /// /// On other hosts it holds nothing. On Linux, a process that another thread spawns holds a copy /// of a VMM's log only from its fork until it executes its program, which closes the copy, and @@ -63,11 +58,25 @@ thread_local! { /// while spawning. #[must_use = "the lock is released when it is dropped"] #[derive(Debug)] +#[cfg_attr( + not(windows), + allow( + dead_code, + reason = "outside the tests, which take it on every host, only the spawns of VMMs on Windows take it" + ) +)] pub(crate) struct SpawnLock(PhantomData>); /// Waits until no spawn of a VMM of this process has handles inheritable for it, and returns a /// [`SpawnLock`], which keeps the spawns of VMMs of other threads from making any inheritable /// until it is dropped, as it describes. +#[cfg_attr( + not(windows), + allow( + dead_code, + reason = "outside the tests, which take it on every host, only the spawns of VMMs on Windows take it" + ) +)] pub(crate) fn lock_spawns() -> SpawnLock { #[cfg(windows)] HELD.with_borrow_mut(|(locks, held)| { @@ -93,18 +102,6 @@ impl Drop for SpawnLock { } } -/// Spawns `command`, as [`Command::spawn`] does, under the crate's spawn lock, so that the process -/// inherits no handle of a VMM that this process spawns meanwhile. -/// -/// A caller that launches VMMs on Windows should spawn its other processes this way: a process -/// that it spawns otherwise while a VMM starts can hold the VMM's handles until it exits, its -/// capability's pipe, its log, and the handles that it allows. The lock lasts only until the -/// process is created, not until it exits. -pub fn spawn_command(command: &mut Command) -> io::Result { - let _lock = lock_spawns(); - command.spawn() -} - /// The descriptor of the caller that an [`Inherited`] borrows. #[cfg(unix)] type Borrowed<'a> = BorrowedFd<'a>; @@ -120,50 +117,38 @@ struct Borrowed<'a>(std::convert::Infallible, PhantomData<&'a ()>); /// [`File`](std::fs::File), that a VMM inherits under the number that it has in the caller, which /// the caller can therefore name in the VMM's arguments. /// -/// [`spawn`](crate::host::supervisor::spawn) passes the VMM these and its standard streams, and -/// nothing else of the caller's: +/// A spawn of a VMM passes the VMM these and its standard streams, and nothing else of the +/// caller's: /// /// - On Linux, the descriptor must be close-on-exec in the caller, as Rust opens its files, and /// stays so: only the VMM's own process clears the flag, between its fork and its exec, so no /// process that another thread spawns meanwhile inherits it. A spawn refuses a descriptor that -/// is not close-on-exec, since any process that the caller spawns would inherit it. The VMM -/// opens it again as `/proc/self/fd/N`, which `proc_path` returns. +/// is not close-on-exec, since any process that the caller spawns would inherit it. /// - On Windows, the handle joins the explicit list of handles that the VMM inherits, and is /// inheritable only while the VMM is created, after which the spawn restores its earlier /// inheritability. Spawns of VMMs take that window one at a time, under the crate's spawn lock, /// so several may allow the same handle. A process that another thread creates in that window /// without such a list, as `std::process::Command` does, can inherit it too, and holds it until -/// it exits, unless the caller spawns that process with [`spawn_command`]. +/// it exits, unless the caller spawns that process under the crate's spawn lock. /// /// Other hosts fail a spawn that passes any. #[derive(Debug, Clone, Copy)] -pub struct Inherited<'a>(Borrowed<'a>); +pub(crate) struct Inherited<'a>(Borrowed<'a>); impl<'a> Inherited<'a> { /// Names the descriptor of `object`, such as a [`File`](std::fs::File). #[cfg(unix)] - pub fn new(object: &'a impl AsFd) -> Self { + pub(crate) fn new(object: &'a impl AsFd) -> Self { Self(object.as_fd()) } /// Names the handle of `object`, such as a [`File`](std::fs::File). #[cfg(windows)] - pub fn new(object: &'a impl AsHandle) -> Self { + pub(crate) fn new(object: &'a impl AsHandle) -> Self { Self(object.as_handle()) } } -#[cfg(target_os = "linux")] -impl Inherited<'_> { - /// Returns the path through which the VMM opens the descriptor again: `/proc/self/fd/N`, - /// where `N` is its number in the caller and in the VMM. - pub fn proc_path(&self) -> PathBuf { - use std::os::fd::AsRawFd; - - PathBuf::from(format!("/proc/self/fd/{}", self.0.as_raw_fd())) - } -} - #[cfg(unix)] impl<'a> From> for Inherited<'a> { fn from(descriptor: BorrowedFd<'a>) -> Self { diff --git a/nvx-rs/src/host/platform/windows/process.rs b/nvx-rs/src/host/platform/windows/process.rs index cf2ce65fb..c979b086c 100644 --- a/nvx-rs/src/host/platform/windows/process.rs +++ b/nvx-rs/src/host/platform/windows/process.rs @@ -3,13 +3,12 @@ //! boot and host, and the host's monotonic time. use std::io; -use std::os::windows::io::{AsRawHandle, BorrowedHandle, OwnedHandle}; +use std::os::windows::io::OwnedHandle; use std::thread; use std::time::{Duration, Instant}; use std::{mem, ptr}; use windows_sys::Win32::Foundation::{ - CompareObjectHandles, DUPLICATE_CLOSE_SOURCE, DUPLICATE_SAME_ACCESS, DuplicateHandle, ERROR_ACCESS_DENIED, ERROR_INVALID_PARAMETER, ERROR_SUCCESS, FILETIME, HANDLE, WAIT_OBJECT_0, WAIT_TIMEOUT, }; @@ -23,7 +22,7 @@ use windows_sys::Win32::System::Threading::{ }; use windows_sys::Win32::System::WindowsProgramming::QueryInterruptTimePrecise; -use super::{last_error, open_handle, owned, raw, wide}; +use super::{last_error, open_handle, raw, wide}; use crate::host::platform::{parse_host_id, unless_exited}; /// Returns the process creation `FILETIME`, used with the PID as a stable identity. @@ -219,73 +218,6 @@ impl ProcessHandle { pub(crate) fn bytes_read(&self, _pid: u32, _start_time: u64) -> io::Result { read_transfer(raw(&self.process)) } - - /// Closes the handle of process `pid` that has the value of this process's handle `ours`, - /// which the process inherited from this one: a handle keeps its value in a child that - /// inherits it, and a child that never uses it, as a VMM never uses the guards of the checked - /// files that it inherits, keeps it there. - /// - /// The process loses no other handle: the handle is closed only once a copy of it is found to - /// hold the object of `ours`, so a process that did not inherit `ours` keeps every handle that - /// it has, and the close fails with [`io::ErrorKind::InvalidInput`], or as `DuplicateHandle` - /// fails for a value that names no handle of the process. A process that has exited has closed - /// its handles, so the close succeeds. Fails with [`io::ErrorKind::PermissionDenied`] through - /// a handle that may not duplicate the handles of the process, as one that [`open_process`] - /// returns: only that of [`LaunchedProcess::pin`] may. - /// - /// [`LaunchedProcess::pin`]: crate::host::platform::windows::LaunchedProcess::pin - pub(crate) fn close_inherited(&self, pid: u32, ours: BorrowedHandle<'_>) -> io::Result<()> { - let value = ours.as_raw_handle() as HANDLE; - let exited = || has_exited(&self.process); - let mut copy = ptr::null_mut(); - // SAFETY: the value is looked up among the handles of the process alone, and the output - // pointer is valid. - let duplicated = unsafe { - DuplicateHandle( - raw(&self.process), - value, - GetCurrentProcess(), - &mut copy, - 0, - 0, - DUPLICATE_SAME_ACCESS, - ) - } != 0; - if !duplicated { - return unless_exited(Err(io::Error::last_os_error()), exited); - } - let copy = - owned(copy).ok_or_else(|| io::Error::other("DuplicateHandle returned no handle"))?; - // SAFETY: both handles are open. - if unsafe { CompareObjectHandles(raw(©), value) } == 0 { - return Err(io::Error::new( - io::ErrorKind::InvalidInput, - format!( - "process {pid} did not inherit handle {:#x} of this process", - value as usize - ), - )); - } - drop(copy); - // SAFETY: without a target process, DuplicateHandle duplicates nothing, and only closes - // the handle of the process, which holds the object of `ours`. - let closed = unsafe { - DuplicateHandle( - raw(&self.process), - value, - ptr::null_mut(), - ptr::null_mut(), - 0, - 0, - DUPLICATE_CLOSE_SOURCE, - ) - } != 0; - if closed { - Ok(()) - } else { - unless_exited(Err(io::Error::last_os_error()), exited) - } - } } /// Returns the outcome of ending process `pid`, as [`ProcessHandle::kill`] does through a handle diff --git a/nvx-rs/src/host/platform/windows/spawn.rs b/nvx-rs/src/host/platform/windows/spawn.rs index 310524909..63f0b4ea9 100644 --- a/nvx-rs/src/host/platform/windows/spawn.rs +++ b/nvx-rs/src/host/platform/windows/spawn.rs @@ -56,15 +56,6 @@ impl LaunchedProcess { )) } - /// Terminates the process and returns whether it exited within `timeout`. - pub(crate) fn terminate(self, timeout: Duration) -> bool { - // SAFETY: CreateProcessW returned the handle with full access. - unsafe { TerminateProcess(raw(&self.process), 1) }; - let milliseconds = u32::try_from(timeout.as_millis()).unwrap_or(INFINITE - 1); - // SAFETY: as above. - unsafe { WaitForSingleObject(raw(&self.process), milliseconds) == WAIT_OBJECT_0 } - } - /// Terminates this owned child and waits for its exit before releasing its handle. pub(crate) fn terminate_and_wait(self) -> io::Result<()> { // SAFETY: CreateProcessW returned the owned handle with full access. @@ -88,6 +79,13 @@ impl LaunchedProcess { /// Waits up to `timeout` for the process to exit, and returns its exit status if it has, /// as `Popen.wait` reads it with `GetExitCodeProcess`. + #[cfg_attr( + not(feature = "cli"), + allow( + dead_code, + reason = "only the CLI's one-shot runs wait for the status of the VMMs that they start" + ) + )] pub(crate) fn wait_timeout(&self, timeout: Duration) -> io::Result> { let milliseconds = u32::try_from(timeout.as_millis()).unwrap_or(INFINITE - 1); // SAFETY: CreateProcessW returned the handle with full access. @@ -207,8 +205,8 @@ impl Drop for InheritableForSpawn { /// and two that allow the same handle restore what it was before either. A process that the caller /// spawns concurrently without a handle list, as `std::process` does, inherits them if it is /// created in that window, and holds them until it exits, the log of OpenVMM's standard output and -/// error among them, unless the caller spawns it under a `SpawnLock`, as [`spawn_command`] does; a -/// process that it spawns with a handle list does not. +/// error among them, unless the caller spawns it under a `SpawnLock`; a process that it spawns with +/// a handle list does not. /// /// OpenVMM starts in `job`, which the process attribute that names it assigns before OpenVMM /// runs, and which every process that OpenVMM starts then joins too, but no process that the @@ -223,7 +221,6 @@ impl Drop for InheritableForSpawn { /// /// [`create_log_job`]: crate::host::platform::windows::create_log_job /// [`SpawnLock`]: crate::host::platform::inherit::SpawnLock -/// [`spawn_command`]: crate::host::platform::inherit::spawn_command /// [`log_claim_held`]: crate::host::platform::windows::log_claim_held pub(crate) fn spawn_with_stdio( program: &Path, diff --git a/nvx-rs/src/host/platform/windows/tests.rs b/nvx-rs/src/host/platform/windows/tests.rs index 41becd1f2..4dffc90fe 100644 --- a/nvx-rs/src/host/platform/windows/tests.rs +++ b/nvx-rs/src/host/platform/windows/tests.rs @@ -5,7 +5,6 @@ use std::io::{self, Read, Write}; use std::mem; #[cfg(feature = "cli")] use std::os::windows::fs::OpenOptionsExt; -use std::os::windows::io::AsHandle; use std::path::Path; use std::rc::Rc; use std::thread; @@ -267,7 +266,7 @@ fn children_inherit_only_their_standard_handles() { let _ = sender.send(leaked_reader.read_to_end(&mut rest).map(|_| rest)); }); let rest = receiver.recv_timeout(Duration::from_secs(5)); - assert!(child.terminate(Duration::from_secs(10))); + child.terminate_and_wait().unwrap(); assert!( rest.expect("the child inherited the caller's pipe") .unwrap() @@ -322,57 +321,6 @@ fn make_inheritable(handle: &impl AsRawHandle) { assert_ne!(set, 0, "{}", io::Error::last_os_error()); } -#[test] -fn a_child_loses_only_the_inherited_handle_that_is_closed_in_it() { - let directory = tempfile::tempdir().unwrap(); - let closed = std::fs::File::create(directory.path().join("closed")).unwrap(); - let kept = std::fs::File::create(directory.path().join("kept")).unwrap(); - let (_other_reader, other) = io::pipe().unwrap(); - let (stdin, _stdin_writer) = io::pipe().unwrap(); - let (_stdout_reader, stdout) = io::pipe().unwrap(); - let stderr = stdout.try_clone().unwrap(); - let system = std::env::var_os("SystemRoot").unwrap(); - let ping = Path::new(&system).join("System32").join("PING.EXE"); - let child = spawn_with_stdio( - &ping, - &["-n", "30", "127.0.0.1"].map(OsString::from), - &std::env::temp_dir(), - [stdin.into(), stdout.into(), stderr.into()], - &[Inherited::new(&closed), Inherited::new(&kept)], - SpawnMode::Detached { - breakaway_from_job: false, - }, - None, - ) - .unwrap(); - let pid = child.id(); - let (process, start_time) = child.pin().unwrap(); - let value = |handle: &std::fs::File| handle.as_raw_handle() as HANDLE; - let held = |handle: &std::fs::File| holds(&child.process, value(handle), handle); - let inherited = (held(&closed), held(&kept)); - // A handle that the child did not inherit is not closed in it, and neither is any of its - // own. - let refused = process.close_inherited(pid, other.as_handle()); - // A process found again, through a handle that may not duplicate its handles, keeps them. - let found = open_process(pid, start_time).unwrap().unwrap(); - let denied = found.close_inherited(pid, closed.as_handle()); - let closing = process.close_inherited(pid, closed.as_handle()); - let after = (held(&closed), held(&kept)); - assert!(child.terminate(Duration::from_secs(10))); - // Once the child has exited, it holds no handle, so a close succeeds. - let exited = process.close_inherited(pid, kept.as_handle()); - assert_eq!(inherited, (true, true), "the child did not inherit both"); - assert!(refused.is_err(), "{refused:?}"); - assert_eq!(denied.unwrap_err().kind(), io::ErrorKind::PermissionDenied); - closing.unwrap(); - assert_eq!( - after, - (false, true), - "the close did not take the closed handle alone" - ); - exited.unwrap(); -} - #[test] fn children_inherit_the_allowed_handles_while_they_are_created() { let (_allowed_reader, allowed) = io::pipe().unwrap(); @@ -412,7 +360,7 @@ fn children_inherit_the_allowed_handles_while_they_are_created() { holds(&child.process, value(&other), &other), ); let (allowed_inheritable, kept_inheritable) = (inheritable(&allowed), inheritable(&kept)); - assert!(child.terminate(Duration::from_secs(10))); + child.terminate_and_wait().unwrap(); assert!( allowed_held && kept_held, "an allowed handle was not inherited" @@ -497,7 +445,7 @@ fn concurrent_spawns_that_allow_one_handle_all_inherit_it_and_restore_it() { ) .unwrap(); let held = holds(&child.process, value, &shared); - assert!(child.terminate(Duration::from_secs(10))); + child.terminate_and_wait().unwrap(); assert!(held, "a spawn did not inherit the allowed handle"); } }); @@ -551,7 +499,9 @@ fn a_spawn_that_panics_in_its_window_restores_the_handles_and_lets_other_spawns_ .recv_timeout(WINDOW_TIMEOUT) .expect("a spawn after the panic did not take the lock") .unwrap(); - assert!(other.terminate(WINDOW_TIMEOUT), "the stand-in did not exit"); + other + .terminate_and_wait() + .expect("the stand-in did not exit"); } /// The longest wait of the tests of a spawn's window. @@ -586,8 +536,8 @@ fn spawn_locked(command: &mut std::process::Command) -> io::Result Self { let system = std::env::var_os("SystemRoot").unwrap(); @@ -682,10 +632,9 @@ impl HeldSpawn { /// log and its identity. fn finish(self) -> (std::path::PathBuf, (u64, u64)) { let (stand_in, log, identity) = self.stand_in(); - assert!( - stand_in.terminate(WINDOW_TIMEOUT), - "the stand-in did not exit" - ); + stand_in + .terminate_and_wait() + .expect("the stand-in did not exit"); (log, identity) } } @@ -769,10 +718,9 @@ fn a_process_spawned_in_the_window_of_a_vmm_in_its_logs_job_holds_the_log_but_no let (stand_in, log, identity) = held.stand_in(); let mut other = other.unwrap(); let held_while_running = log_claim_held(&log, identity).unwrap(); - assert!( - stand_in.terminate(WINDOW_TIMEOUT), - "the stand-in did not exit" - ); + stand_in + .terminate_and_wait() + .expect("the stand-in did not exit"); // Once the stand-in has exited, the job holds no process, and the claim is released, while // the process still holds the log. let released = claim_released_within(&log, identity, WINDOW_TIMEOUT); @@ -948,7 +896,7 @@ fn local_link(name: &[u16], target: &[u16]) -> OwnedHandle { #[test] fn vm_spawns_refuse_untrusted_log_job_names_before_creating_a_child() { - use crate::host::supervisor::{self, VmmCommand}; + use crate::host::spawn::{self, VmmCommand}; use crate::protocol::CAPABILITY_LEN; let directory = tempfile::tempdir().unwrap(); @@ -956,22 +904,33 @@ fn vm_spawns_refuse_untrusted_log_job_names_before_creating_a_child() { let id = log_job_id(&log).unwrap().unwrap(); let name = log_job_name(id); let system = std::env::var_os("SystemRoot").unwrap(); - let command = VmmCommand::new( - Path::new(&system).join("System32").join("cmd.exe"), - ["/c", "exit", "0"].map(OsString::from).to_vec(), - directory.path(), - ); + let command = VmmCommand { + program: Path::new(&system).join("System32").join("cmd.exe"), + arguments: ["/c", "exit", "0"].map(OsString::from).to_vec(), + working_dir: directory.path().to_path_buf(), + breakaway_from_job: false, + inherit: Vec::new(), + }; let refused = || { let entered = Rc::new(Cell::new(false)); let observed = Rc::clone(&entered); let result = with_spawn_window( move || observed.set(true), - || supervisor::spawn_vm(&command, &[7; CAPABILITY_LEN], log.try_clone().unwrap()), + || { + spawn::spawn_vm_with_mode( + &command, + &[7; CAPABILITY_LEN], + log.try_clone().unwrap(), + false, + ) + }, ); let error = match result { Err(error) => error, Ok(child) => { - assert!(child.kill_and_wait(), "the unexpected child did not exit"); + child + .kill_and_confirm_exit() + .expect("the unexpected child did not exit"); panic!("an untrusted log-job name started a child"); } }; @@ -1011,10 +970,12 @@ fn vm_spawns_refuse_untrusted_log_job_names_before_creating_a_child() { let observed = Rc::clone(&entered); let child = with_spawn_window( move || observed.set(true), - || supervisor::spawn_vm(&command, &[7; CAPABILITY_LEN], log), + || spawn::spawn_vm_with_mode(&command, &[7; CAPABILITY_LEN], log, false), ) .unwrap(); - assert!(child.kill_and_wait(), "the trusted child did not exit"); + child + .kill_and_confirm_exit() + .expect("the trusted child did not exit"); assert!( entered.get(), "the successful spawn bypassed the observation" @@ -1094,8 +1055,8 @@ fn a_vmm_never_starts_outside_a_job_that_a_claim_check_takes_for_its_logs() { } #[test] -fn a_vmm_whose_log_is_no_file_of_a_disk_volume_starts_without_a_job() { - use crate::host::supervisor::{self, VmmCommand}; +fn a_vmm_whose_log_is_no_file_of_a_disk_volume_is_refused() { + use crate::host::spawn::{self, VmmCommand}; use crate::protocol::CAPABILITY_LEN; let null = OpenOptions::new().write(true).open("NUL").unwrap(); @@ -1111,23 +1072,24 @@ fn a_vmm_whose_log_is_no_file_of_a_disk_volume_starts_without_a_job() { assert_eq!(log_job_id(&pipe).unwrap(), None); let system = std::env::var_os("SystemRoot").unwrap(); let cmd = Path::new(&system).join("System32").join("cmd.exe"); - let command = VmmCommand::new( - cmd, - ["/c", "exit", "0"].map(OsString::from).to_vec(), - std::env::temp_dir(), - ); + let command = VmmCommand { + program: cmd, + arguments: ["/c", "exit", "0"].map(OsString::from).to_vec(), + working_dir: std::env::temp_dir(), + breakaway_from_job: false, + inherit: Vec::new(), + }; for log in [null, pipe] { - let error = - match supervisor::spawn_vm(&command, &[7; CAPABILITY_LEN], log.try_clone().unwrap()) { - Err(error) => error, - Ok(child) => { - assert!(child.kill_and_wait(), "the unexpected child did not exit"); - panic!("a strict spawn accepted a log without a job"); - } - }; + let error = match spawn::spawn_vm_with_mode(&command, &[7; CAPABILITY_LEN], log, false) { + Err(error) => error, + Ok(child) => { + child + .kill_and_confirm_exit() + .expect("the unexpected child did not exit"); + panic!("a spawn accepted a log without a job"); + } + }; assert_eq!(error.kind(), io::ErrorKind::PermissionDenied, "{error}"); - let launched = supervisor::spawn(&command, &[7; CAPABILITY_LEN], log).unwrap(); - assert!(launched.kill_and_wait(), "the stand-in did not exit"); } } @@ -1184,7 +1146,7 @@ fn a_thread_that_holds_the_spawn_lock_spawns_under_it_until_its_last_lock_is_dro drop(second); let late = acquired.recv_timeout(WINDOW_TIMEOUT).is_ok(); waiter.join().unwrap(); - assert!(vmm.terminate(WINDOW_TIMEOUT), "the stand-in did not exit"); + vmm.terminate_and_wait().expect("the stand-in did not exit"); drop(other); assert!( !early, diff --git a/nvx-rs/src/host/process.rs b/nvx-rs/src/host/process.rs index a24ed8776..ce2b91dd5 100644 --- a/nvx-rs/src/host/process.rs +++ b/nvx-rs/src/host/process.rs @@ -20,14 +20,11 @@ //! system's read calls, and [`own_bytes_read`] how many the calling process has, so that a caller //! can tell how much of its files a VMM, or the caller itself, read. //! -//! The [`supervisor`](super::supervisor) launches OpenVMM, and its -//! [`Launched::process`](super::supervisor::Launched::process) holds the child that it started -//! through a handle of its own. +//! [`vm::spawn`](super::vm::spawn) launches OpenVMM, and the [`Launched`](super::vm::Launched) +//! that it returns holds the child that it started through a handle of its own. use std::fmt; use std::io; -#[cfg(windows)] -use std::os::windows::io::BorrowedHandle; use std::str::FromStr; use std::sync::OnceLock; use std::thread; @@ -39,10 +36,17 @@ use super::platform; use crate::error::Error; /// Longest wait for a terminated OpenVMM process to exit. +#[cfg_attr( + not(feature = "cli"), + allow( + dead_code, + reason = "only the CLI waits for the OpenVMM processes that it terminates" + ) +)] pub(crate) const KILL_TIMEOUT: Duration = Duration::from_secs(10); /// Interval at which the exit of a process is polled where no exit event serves: the exit of any -/// process on Linux hosts without pidfds, and the exit of a child that a -/// [`Launched`](super::supervisor::Launched) waits for with a timeout on Linux, since +/// process on Linux hosts without pidfds, and the exit of a child that the crate's spawn of a VMM +/// waits for with a timeout on Linux, since /// [`std::process::Child`] can wait only without one. An exit is noticed at most this late, for /// about 40 checks a second. pub(crate) const POLL_INTERVAL: Duration = Duration::from_millis(25); @@ -319,21 +323,6 @@ impl Process { self.handle .bytes_read(self.reference.pid, self.reference.start_time) } - - /// Closes the process's handle that has the value of this process's handle `ours`, which the - /// process inherited from this one, and never uses, as a VMM never uses the guards of the - /// checked files that it inherits on Windows. - /// - /// The process loses no other handle: a process that did not inherit `ours` keeps every handle - /// that it has, and the close fails. A process that has exited has closed its handles, so the - /// close succeeds. Fails with [`io::ErrorKind::PermissionDenied`] for a process that - /// [`open`](Self::open) found: only the `Process` of a launch, which - /// [`Launched::process`](super::supervisor::Launched::process) returns, may close the - /// process's handles. - #[cfg(windows)] - pub(crate) fn close_inherited(&self, ours: BorrowedHandle<'_>) -> io::Result<()> { - self.handle.close_inherited(self.reference.pid, ours) - } } impl fmt::Debug for Process { diff --git a/nvx-rs/src/host/restore/tests.rs b/nvx-rs/src/host/restore/tests.rs index 4c5a2bbb9..3e0b9d40f 100644 --- a/nvx-rs/src/host/restore/tests.rs +++ b/nvx-rs/src/host/restore/tests.rs @@ -14,8 +14,8 @@ use crate::activation::{ ActivationRecord, Configuration, ExpectedResources, Export, ExportKind, Image, MachineIdMode, OverlayMode, ReadyActivation, RestoreSection, TransactionId, Upper, View, }; -use crate::host::platform::inherit::spawn_command; use crate::host::process::{BootId, Opened}; +use crate::host::tests::spawn_locked; use crate::host::vm::create_private_dir; use crate::protocol::restore_ready::parse_ready_event; @@ -90,7 +90,7 @@ impl Client { .env("NVX_TEST_READY_CLOSE", if close { "1" } else { "0" }) .stdin(Stdio::piped()) .stdout(Stdio::piped()); - let child = spawn_command(&mut command).unwrap(); + let child = spawn_locked(&mut command).unwrap(); let process = held_process(child.id()); let mut client = Self { child, process }; client @@ -233,7 +233,7 @@ fn a_queued_peer_cannot_be_the_vm_that_reuses_its_pid() { .stdin(Stdio::null()) .stdout(Stdio::piped()) .stderr(Stdio::piped()); - let output = spawn_command(&mut command) + let output = spawn_locked(&mut command) .and_then(Child::wait_with_output) .unwrap_or_else(|error| panic!("{diagnostics}\n{route} helper: {error}")); let stdout = String::from_utf8_lossy(&output.stdout); @@ -278,7 +278,7 @@ fn a_queued_peer_cannot_be_the_vm_that_reuses_its_pid() { ]) .stdin(Stdio::piped()) .stdout(Stdio::piped()); - let child = spawn_command(&mut command).unwrap(); + let child = spawn_locked(&mut command).unwrap(); assert_eq!(child.id(), recycled); let process = held_process(child.id()); let mut vm = Client { child, process }; @@ -323,7 +323,7 @@ fn the_receiver_fails_closed_without_original_peer_pidfds() { .stdout(Stdio::piped()) .stderr(Stdio::piped()); crate::host::sys::tests::refuse_peer_pidfd_on_spawn(&mut command, errno); - let output = spawn_command(&mut command) + let output = spawn_locked(&mut command) .unwrap() .wait_with_output() .unwrap(); diff --git a/nvx-rs/src/host/restore/tests/live_init.rs b/nvx-rs/src/host/restore/tests/live_init.rs index 91bfbe176..ce6c83c6d 100644 --- a/nvx-rs/src/host/restore/tests/live_init.rs +++ b/nvx-rs/src/host/restore/tests/live_init.rs @@ -15,7 +15,7 @@ use crate::activation::{ use crate::host::nvxs::HostConnection; use crate::host::session; use crate::host::snapshot::SnapshotManifest; -use crate::host::supervisor::{Launched, VmmCommand, spawn}; +use crate::host::spawn::{Launched, VmmCommand, spawn_vm_with_mode}; use crate::protocol::{ APP_ERROR, APP_EXEC, APP_EXIT, APP_MAPS, APP_READY, APP_STDERR, APP_STDOUT, APP_STOP, APP_STOPPED, CAPABILITY_LEN, HostMap, Workload, WorkloadEnvironment, decode_app, encode_app, @@ -36,7 +36,7 @@ impl Drop for OwnedVm { .is_ok_and(|done| done) { vm.detach(); - } else if !vm.kill_and_wait() { + } else if vm.kill_and_confirm_exit().is_err() { eprintln!("OWNED_RESTORE_TEARDOWN_FAILED"); } } @@ -296,8 +296,14 @@ fn run() -> Result<(), AnyError> { ]; let mut capability = [0; CAPABILITY_LEN]; getrandom::fill(&mut capability)?; - let command = VmmCommand::new(vmm, options, &root); - let mut owned = OwnedVm(Some(spawn(&command, &capability, log)?)); + let command = VmmCommand { + program: vmm, + arguments: options, + working_dir: root.clone(), + breakaway_from_job: false, + inherit: Vec::new(), + }; + let mut owned = OwnedVm(Some(spawn_vm_with_mode(&command, &capability, log, false)?)); let running = owned.0.as_ref().ok_or("missing owned VM")?; let process = running.process()?; let pid = running.id(); diff --git a/nvx-rs/src/host/restore_driver/tests.rs b/nvx-rs/src/host/restore_driver/tests.rs index f32cc7dfa..718a7c2be 100644 --- a/nvx-rs/src/host/restore_driver/tests.rs +++ b/nvx-rs/src/host/restore_driver/tests.rs @@ -181,7 +181,11 @@ fn record() -> ActivationRecord { fn workload() -> Workload<'static> { Workload { - identity: WorkloadIdentity::DEFAULT, + identity: WorkloadIdentity { + uid: 65534, + gid: 65534, + create_account: false, + }, exec: protocol::Workload { argv: vec!["/bin/true".to_owned()], timeout_ms: 0, diff --git a/nvx-rs/src/host/sandbox/distro.rs b/nvx-rs/src/host/sandbox/distro.rs index 751afe3bc..1a2798914 100644 --- a/nvx-rs/src/host/sandbox/distro.rs +++ b/nvx-rs/src/host/sandbox/distro.rs @@ -181,12 +181,7 @@ pub struct DistroBinding { /// The file is the object that the check compared with the binding, open for reading, whatever /// later takes its path. On Windows, it denies other opens for writing or deleting the layer, so /// that nothing modifies, renames, or replaces the layer while it is held; OpenVMM's read-only -/// open of the layer shares it. Dropping it closes the file, and a VMM that inherited it from a -/// [`PreparedLaunch`](super::PreparedLaunch) keeps its copy until the prepared launch is released -/// or the VMM exits. -/// -/// A [`PreparedLaunch`](super::PreparedLaunch) takes it, with the path at which the check opened -/// the layer and the format and identity that the binding names, and hands the layer to the VMM. +/// open of the layer shares it. Dropping it closes the file. #[derive(Debug)] pub struct CheckedDistro { /// The checked layer. @@ -195,8 +190,6 @@ pub struct CheckedDistro { path: PathBuf, /// The layer's format, as the binding names it. format: DistroFormat, - /// The identity that the guest expects of the layer, as the binding names it. - identity: String, } impl CheckedDistro { @@ -210,12 +203,6 @@ impl CheckedDistro { &self.path } - /// Returns the layer's format and the identity that the guest expects of it, as the binding - /// names them. - pub(crate) fn bound(&self) -> (DistroFormat, &str) { - (self.format, &self.identity) - } - /// Returns the block geometry that OpenVMM presents for the layer as the read-only `distro` /// block device, read through the checked file, as OpenVMM reads it through its own open of /// the layer: the layer's length, which must be a nonzero multiple of 512 bytes; logical @@ -239,12 +226,6 @@ impl CheckedDistro { block_geometry(&self.file, &self.path, description) } - /// Returns the checked layer, for a prepared launch to move. - #[cfg(target_os = "linux")] - pub(crate) fn file_mut(&mut self) -> &mut File { - &mut self.file - } - /// Returns the checked layer, which the caller then holds: an owned descriptor on Linux, and /// an owned handle on Windows, which it converts into an `OwnedFd` or an `OwnedHandle`. pub fn into_file(self) -> File { @@ -447,7 +428,6 @@ impl DistroBinding { file, path: path.to_path_buf(), format: self.format, - identity: self.identity.clone(), }) } } diff --git a/nvx-rs/src/host/sandbox/distro/tests.rs b/nvx-rs/src/host/sandbox/distro/tests.rs index 6d119468d..15b4b88d6 100644 --- a/nvx-rs/src/host/sandbox/distro/tests.rs +++ b/nvx-rs/src/host/sandbox/distro/tests.rs @@ -756,11 +756,10 @@ fn gpt_disks_bind_by_their_disk_guid() { binding ); - // The check returns the disk, with the format and disk GUID that the binding names. + // The check returns the disk that the binding seals. let checked = binding .check_with(&gpt, DISTRO_POLICY_REVISION, METADATA_ONLY) .unwrap(); - assert_eq!(checked.bound(), (DistroFormat::Gpt, UUID)); assert_eq!(&FileSeal::of(checked.file()).unwrap(), binding.image()); // A binding of one format checks no image of the other, even at its path and identity. diff --git a/nvx-rs/src/host/sandbox/layer.rs b/nvx-rs/src/host/sandbox/layer.rs index e05aba25a..20c5ad3e7 100644 --- a/nvx-rs/src/host/sandbox/layer.rs +++ b/nvx-rs/src/host/sandbox/layer.rs @@ -11,8 +11,8 @@ use crate::uuid::Uuid; /// The role of a read-only layer of `nvx-rs sandbox`. /// -/// A sandbox takes exactly one layer, the distro layer, which its plan attaches as the `distro` -/// [`BlockRole`](super::BlockRole). +/// A sandbox takes exactly one layer, the distro layer, which OpenVMM attaches as the `distro` +/// sandbox block device. #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)] pub enum LayerRole { /// The distribution, the only layer of a sandbox. diff --git a/nvx-rs/src/host/sandbox/mod.rs b/nvx-rs/src/host/sandbox/mod.rs index 95c31f2a3..28055573c 100644 --- a/nvx-rs/src/host/sandbox/mod.rs +++ b/nvx-rs/src/host/sandbox/mod.rs @@ -1,32 +1,18 @@ //! The sandbox launch profile: a microVM that runs one workload in a container over a read-only //! distro image and a writable upper, with live virtio-fs shares. //! -//! This is the profile of `nvx-rs sandbox`: its documented limits, option semantics, OpenVMM -//! arguments, and guest kernel tokens, with native Rust input validation and diagnostics: +//! This is the profile of `nvx-rs sandbox`: its documented limits and option semantics, with +//! native Rust input validation and diagnostics: //! -//! - [`plan`]: the typed, persistable [`SandboxPlan`] of a cold layered launch: one read-only -//! `distro` block device, an EROFS filesystem or a GPT disk with the identity that the guest -//! expects of it, over an upper on an ext4 scratch image or in RAM, with the launch settings, the -//! export plan of its live shares, and optionally a network, a [`SandboxNetwork`] whose typed -//! policy may be proxy-only. It renders the OpenVMM arguments, -//! [`openvmm_arguments`](SandboxPlan::openvmm_arguments), and the kernel command line, -//! [`kernel_command_line`](SandboxPlan::kernel_command_line), whose boot tokens -//! [`protocol::root`](crate::protocol::root) defines. An adapter that runs its own guest plans -//! the same root with [`SandboxPlan::adapter`], without the profile's workload settings. //! - [`SandboxLayer`]: the distro layer of a sandbox, the one [`LayerRole`] that a sandbox takes: //! an EROFS filesystem, as `nvx-rs sandbox` names it with `ROLE,PATH,EROFS_UUID`, or a GPT disk. //! - [`SandboxMount`]: a requested live share of a host directory or regular file at a guest //! target, with its access mode, owner, and access-policy paths; [`mount`] holds the rules of -//! guest targets and the checks of host paths, plans the shares of a sandbox through the planner -//! of every NVX caller, [`host::filesystem`](crate::host::filesystem), and renders the plan as -//! the guest's shell reads it at boot. -//! - [`SandboxLaunch`]: the launch of `nvx-rs sandbox`, its layers over its upper, an ext4 -//! scratch image or RAM, with its settings and the export plan of its shares, which plans its -//! distro layer, in the layer's format, over that upper. +//! guest targets and the checks of host paths, and plans the shares of a sandbox through the +//! planner of every NVX caller, [`host::filesystem`](crate::host::filesystem). //! - [`options`]: the checks that `nvx-rs sandbox` and `nvx-rs run` apply to their options: the -//! attribution of access-policy paths to shares, the systemd entrypoint check and the preflight -//! that verifies the distro layer, the targets of `nvx-rs run` shares, and the managed exec -//! environment. +//! attribution of access-policy paths to shares, the systemd entrypoint check, the targets of +//! `nvx-rs run` shares, and the managed exec environment. //! - [`distro`]: the binding of the distro layer, which explicit verification makes from the //! layer and its manifest, and which a launch checks from metadata alone. //! - [`scratch`]: the binding of the scratch image, its identity, kind, and length, which a @@ -35,14 +21,9 @@ //! and physical block sizes, which [`CheckedDistro::geometry`](distro::CheckedDistro::geometry) //! and [`CheckedScratch::geometry`](scratch::CheckedScratch::geometry) read through the file, //! as OpenVMM reads it. -//! - [`PreparedLaunch`]: the launch of a plan with the files that the checks of its block devices -//! returned, which hands the VMM those objects rather than paths that something could replace -//! meanwhile: the OpenVMM arguments that name them, and the descriptors that the VMM inherits; -//! [`prepared`] explains the handoff on each host. //! -//! Workload IDs and resource limits are ASCII decimal integers in their documented ranges. -//! Layer UUIDs use their canonical lowercase hyphenated text, and resource limits use native -//! unsigned Rust integers. Host paths use the host's native flavor. +//! Workload IDs are ASCII decimal integers in their documented ranges. Layer UUIDs use their +//! canonical lowercase hyphenated text. Host paths use the host's native flavor. //! //! # Errors //! @@ -57,42 +38,19 @@ pub(crate) mod geometry; pub mod layer; pub mod mount; pub mod options; -pub mod plan; -pub mod prepared; pub mod scratch; use std::fs::File; use std::path::Path; -use self::distro::{DistroBinding, DistroFormat}; use self::geometry::BlockGeometry; pub use self::layer::{LAYER_ROLES, LayerRole, SandboxLayer}; use self::mount::os_error; pub use self::mount::{MountAccess, MountKind, MountOwner, MountPolicy, PolicyKind, SandboxMount}; -pub use self::plan::{ - BlockRole, PLAN_VERSION, SandboxDistro, SandboxNetwork, SandboxPlan, SandboxUpper, -}; -pub use self::prepared::PreparedLaunch; use crate::error::{Error, ErrorCode, Result}; use crate::host::artifacts::require_existing; -pub(crate) use crate::host::config::{reject_container_path, reject_disk_path}; -use crate::host::filesystem::ExportPlan; -use crate::net::is_valid_hostname; use crate::protocol::cmdline; -use crate::protocol::root::RootImage; -use crate::uuid::Uuid; -/// The guest adds one for its init process; Linux rejects `pids.max` at or above 4,194,305. -pub(crate) const PIDS_MAX_LIMIT: u32 = 4_194_303; - -/// The size of the x86 kernel command line, including its NUL. -pub const KERNEL_COMMAND_LINE_MAX_SIZE: usize = 2048; -/// The part of the kernel command line that OpenVMM keeps for its own tokens. -pub const OPENVMM_COMMAND_LINE_RESERVE: usize = 1024; -/// The part of the kernel command line that the sandbox profile may use, including the NUL and -/// the bootstrap tokens that OpenVMM appends for live shares. -pub const SANDBOX_COMMAND_LINE_MAX_SIZE: usize = - KERNEL_COMMAND_LINE_MAX_SIZE - OPENVMM_COMMAND_LINE_RESERVE; /// The workload identity by default, `65534:65534`. pub const DEFAULT_WORKLOAD_IDENTITY: (u32, u32) = (65534, 65534); /// The entrypoint by default. @@ -221,254 +179,5 @@ pub(crate) fn block_geometry(file: &File, path: &Path, description: &str) -> Res } } -/// The launch settings of a [`SandboxPlan`] or a [`SandboxLaunch`], besides its block devices and -/// its shares, with the defaults of `nvx-rs sandbox`. -#[derive(Clone, Debug, PartialEq, Eq)] -pub struct SandboxSettings { - /// The absolute path of the workload's program in the container, without whitespace. - pub entrypoint: String, - /// The workload's arguments, each nonempty and without whitespace. - pub args: Vec, - /// The workload's hostname: a lowercase RFC 1123 label of up to 63 characters. - pub hostname: String, - /// The workload's UID and GID, neither of them root's. - pub workload_identity: (u32, u32), - /// The workload's memory limit in bytes, `memory.max` of its cgroup, if positive. - pub memory_max: Option, - /// The most processes that the workload may run, if positive; the guest's own init counts - /// too, so its `pids.max` is one more. - pub pids_max: Option, -} - -impl Default for SandboxSettings { - fn default() -> Self { - Self { - entrypoint: DEFAULT_ENTRYPOINT.to_owned(), - args: Vec::new(), - hostname: DEFAULT_HOSTNAME.to_owned(), - workload_identity: DEFAULT_WORKLOAD_IDENTITY, - memory_max: None, - pids_max: None, - } - } -} - -/// Checks the launch settings of a plan, in this order: the entrypoint, the arguments, the -/// hostname, a non-root identity, positive limits, limits that the guest's cgroups can apply, and -/// at most [`u64::MAX`] bytes of memory and 4,194,303 processes. -/// -/// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported). -fn validate_settings(settings: &SandboxSettings) -> Result<()> { - let SandboxSettings { - entrypoint, - args, - hostname, - workload_identity: (uid, gid), - memory_max, - pids_max, - } = settings; - if !entrypoint.starts_with('/') || has_kernel_separator(entrypoint) { - return Err(refused( - "sandbox entrypoint must be an absolute path without whitespace, control characters, or quotes", - )); - } - if args - .iter() - .any(|argument| argument.is_empty() || has_kernel_separator(argument)) - { - return Err(refused( - "sandbox arguments must be nonempty and contain no whitespace, control characters, or quotes", - )); - } - if !is_valid_hostname(hostname) { - return Err(refused( - "sandbox hostname must be a lowercase RFC 1123 label up to 63 characters", - )); - } - if *uid == 0 || *gid == 0 { - return Err(refused( - "sandbox workload identity must be a non-root UID:GID", - )); - } - for (name, value) in [ - ("memory-max", *memory_max), - ("pids-max", pids_max.map(u64::from)), - ] { - if value == Some(0) { - return Err(refused(format!("--{name} must be positive"))); - } - } - if pids_max - .as_ref() - .is_some_and(|&value| value > PIDS_MAX_LIMIT) - { - return Err(refused(format!( - "--pids-max must be at most {PIDS_MAX_LIMIT}" - ))); - } - Ok(()) -} - -/// The launch of a sandbox of `nvx-rs sandbox`: its read-only layers, its upper, an ext4 scratch -/// image or RAM, its settings, the export plan of its shares, and its network, which it plans as -/// its distro layer, in the layer's format, over its upper. -/// -/// Its layers are keyed by the role of each `--layer` value, of which a launch takes only the -/// distro layer. [`plan`](Self::plan) returns its [`SandboxPlan`], which renders its OpenVMM -/// arguments and kernel command line, and [`bound_plan`](Self::bound_plan) the plan with the image -/// that the binding of its distro layer records. A launch has no network device until -/// [`with_network`](Self::with_network) gives it one. -#[derive(Clone, Debug, PartialEq, Eq)] -pub struct SandboxLaunch { - /// The read-only layers, in the order given: the distro layer alone. - layers: Vec, - /// The plan: the distro layer, in its format and with its identity, over the upper, with the - /// settings and the export plan. - plan: SandboxPlan, -} - -impl SandboxLaunch { - /// Returns the launch of `layers` over `upper`, an ext4 scratch image or RAM, without shares, - /// which [`with_export`](Self::with_export) adds. - /// - /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported), checking in this - /// order: no duplicate roles, exactly one layer, the distro layer, a UUID of the distro layer - /// in its canonical text, as `--layer` writes it, and the settings, as [`SandboxPlan::new`] - /// checks them. - pub fn new( - layers: Vec, - upper: SandboxUpper, - settings: SandboxSettings, - ) -> Result { - let mut duplicates: Vec<&str> = LAYER_ROLES - .iter() - .filter(|&&role| layers.iter().filter(|layer| layer.role() == role).count() > 1) - .map(|role| role.as_str()) - .collect(); - if !duplicates.is_empty() { - duplicates.sort_unstable(); - return Err(refused(format!( - "duplicate sandbox layer role(s): {}", - duplicates.join(", ") - ))); - } - let distro = match &layers[..] { - [layer] if layer.role() == LayerRole::Distro => layer, - _ => { - return Err(refused( - "a sandbox requires exactly one read-only distro layer", - )); - } - }; - let Some(uuid) = Uuid::parse(distro.uuid()) else { - return Err(refused(format!( - "{} layer UUID is invalid: {:?}", - distro.role(), - distro.uuid() - ))); - }; - let image = match distro.format() { - DistroFormat::Erofs => RootImage::erofs(Some(uuid)), - DistroFormat::Gpt => RootImage::gpt(Some(uuid)), - }; - let plan = SandboxPlan::new( - SandboxDistro::at(distro.path().to_path_buf(), image), - upper, - settings, - )?; - Ok(Self { layers, plan }) - } - - /// Returns the layers, in the order given. - pub fn layers(&self) -> &[SandboxLayer] { - &self.layers - } - - /// Returns the distro layer, the one layer that a launch holds. - pub fn distro(&self) -> &SandboxLayer { - &self.layers[0] - } - - /// Returns the upper: an ext4 scratch image, or RAM. - pub fn upper(&self) -> &SandboxUpper { - self.plan.upper() - } - - /// Returns the scratch image, or `None` for an upper in RAM. - pub fn scratch(&self) -> Option<&Path> { - self.plan.upper().scratch_path() - } - - /// Returns the settings besides the layers and the upper. - pub fn settings(&self) -> &SandboxSettings { - self.plan - .settings() - .expect("a launch plans the sandbox profile, with its settings") - } - - /// Returns the export plan of the shares, if any. - pub fn export(&self) -> Option<&ExportPlan> { - self.plan.export() - } - - /// Returns the launch with `export`, the export plan of its live shares, if any, such as - /// [`mount::plan_mounts`] makes. - /// - /// Fails as [`SandboxPlan::with_export`] checks the export plan. - pub fn with_export(self, export: Option) -> Result { - let Self { layers, plan } = self; - Ok(Self { - layers, - plan: plan.with_export(export)?, - }) - } - - /// Returns the launch with `network`, or with no network device for `None`. - #[must_use] - pub fn with_network(self, network: Option) -> Self { - Self { - plan: self.plan.with_network(network), - ..self - } - } - - /// Returns the network, or `None` for no network device. - pub fn network(&self) -> Option<&SandboxNetwork> { - self.plan.network() - } - - /// Returns the plan of the launch: its distro layer, in the format and with the identity of - /// its layer, as a `--layer` value names an EROFS layer and its UUID, over its upper, with its - /// settings, the export plan of its shares, and its network. - pub fn plan(&self) -> &SandboxPlan { - &self.plan - } - - /// Returns the plan of the launch with the image that `binding`, the binding of its distro - /// layer, records: the staged format, and the identity that the guest expects of the layer. - /// - /// A binding that [`DistroBinding::check`] accepts for the distro layer names the format and - /// identity that [`plan`](Self::plan) takes from the layer's `--layer` value, so both plans - /// render alike; callers that hold the binding plan from it. Fails with - /// [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) for a binding whose identity is - /// not a UUID in its canonical text. - pub fn bound_plan(&self, binding: &DistroBinding) -> Result { - Ok(self.plan.with_image(plan::bound_image(binding)?)) - } - - /// Checks the launch against the host, as [`SandboxPlan::validated`] checks its plan: the - /// distro layer and the scratch image of an upper on scratch must be regular files whose paths - /// have no comma or semicolon, and whose names, and those of the files to which their paths - /// resolve, do not end in an extension with which OpenVMM would not attach them as raw images, - /// and the export plan's directories and files must still not overlap, and it must pass - /// [`ExportPlan::verify`]. - /// - /// Fails with [`ErrorKind::Unavailable`](crate::ErrorKind::Unavailable) for a missing image, - /// and otherwise as those checks do. - pub fn validated(&self) -> Result<()> { - self.plan.validated() - } -} - #[cfg(test)] mod tests; diff --git a/nvx-rs/src/host/sandbox/mount/aggregate.rs b/nvx-rs/src/host/sandbox/mount/aggregate.rs index ce52062d7..b730b64e3 100644 --- a/nvx-rs/src/host/sandbox/mount/aggregate.rs +++ b/nvx-rs/src/host/sandbox/mount/aggregate.rs @@ -1,11 +1,8 @@ -//! The children of an aggregate of shares, the values of OpenVMM's options that attach shares, -//! and the planning of the export of a sandbox's shares. +//! The planning of the export of a sandbox's shares. use std::io; use std::path::PathBuf; -use sha2::{Digest, Sha256}; - use super::{ MountAccess, MountKind, MountOwner, PolicyKind, SandboxMount, validate_mount_host_paths, validate_mounts, @@ -17,40 +14,6 @@ use crate::host::filesystem::{ use crate::host::paths; use crate::host::sandbox::refused; -/// Returns the name of the aggregate child at `index` that the guest binds at `target`: the -/// index, a hyphen, and the first 32 hexadecimal digits of the SHA-256 digest of the target. -/// -/// A snapshot records every child's name, so it pins each target, and OpenVMM refuses a restore -/// that requests other targets before the guest runs, rather than keep the snapshot's binds. -pub fn aggregate_child_name(index: usize, target: &str) -> String { - let digest = Sha256::digest(target.as_bytes()); - let hex: String = digest[..16] - .iter() - .map(|byte| format!("{byte:02x}")) - .collect(); - format!("{index}-{hex}") -} - -/// Returns the `--mount-child` value that attaches `host_path` as the child `name`. It always -/// gives the mode, which OpenVMM reads after the last comma or before the `file` flag, so the -/// host path may contain commas. A file child carries OpenVMM's `file` flag, so OpenVMM refuses -/// a directory that replaced the file instead of exporting all of it. -pub fn aggregate_child_argument( - name: &str, - host_path: &str, - access: MountAccess, - kind: MountKind, -) -> String { - let file = if kind == MountKind::File { ",file" } else { "" }; - format!("{name},{host_path},{access}{file}") -} - -/// Returns the `--mount` value that attaches `host_path` at `target`. It always gives the mode, -/// which OpenVMM reads after the last comma, so the host path may contain commas. -pub fn mount_argument(target: &str, host_path: &str, access: MountAccess) -> String { - format!("{target},{host_path},{access}") -} - /// Plans the export of `mounts`, the live shares of one sandbox, through the planner of every /// NVX caller, [`filesystem::plan`], after it checks them with [`validate_mounts`], each share /// with [`SandboxMount::validated`], and their host paths with [`validate_mount_host_paths`]. diff --git a/nvx-rs/src/host/sandbox/mount/boot.rs b/nvx-rs/src/host/sandbox/mount/boot.rs index f4ae4e5df..e1363a00c 100644 --- a/nvx-rs/src/host/sandbox/mount/boot.rs +++ b/nvx-rs/src/host/sandbox/mount/boot.rs @@ -1,145 +1,10 @@ -//! The shares of an export plan in the form that a launch delivers at boot, and the bootstrap -//! tokens that OpenVMM appends to the kernel command line for them. +//! The text of the host paths of an export plan, which OpenVMM's options carry. use std::path::Path; -use super::request::command_line_fragment; -use super::{ - AGGREGATE_COMMAND_LINE_SUFFIX, AGGREGATE_MOUNT_TARGET, MountAccess, MountKind, MountOwner, - PolicyKind, aggregate_child_argument, aggregate_child_name, mount_argument, - validate_mount_target, -}; use crate::error::Result; -use crate::host::filesystem::{ExportPlan, GUEST_EXPORT, Owner}; use crate::host::sandbox::refused; -/// The form in which a launch delivers an export plan at boot: OpenVMM's arguments that attach -/// its directories and files, the kernel parameters that the plan adds for them, and the -/// bootstrap tokens that OpenVMM appends to the kernel command line. -#[derive(Clone, Debug, Default, PartialEq, Eq)] -pub struct BootShares { - /// OpenVMM's arguments: one directory share or an aggregate with its children, in order, the - /// access-policy paths of each, absolute, and the owner unless it is `vmm`. - pub arguments: Vec, - /// The kernel parameters: as [`boot_shares`] renders them, the `nvx_share=` tokens of an - /// aggregate, one for each child, in order, and none for one directory share; for a mapping - /// table, the `nvx_maps=` token of its number of mappings. - pub tokens: Vec, - /// The bootstrap tokens that OpenVMM appends after the kernel command line. - pub fragment: String, -} - -/// Renders `plan` in the form that the guest's shell reads at boot. -/// -/// The shell binds each exported directory or file at one target, so every child of the plan -/// must be mapped exactly once, at its root, with the access with which OpenVMM exports it, at a -/// target that [`validate_mount_target`] accepts. -/// One directory then uses OpenVMM's virtio-fs device directly, and anything else shares it as -/// the children of an aggregate, named by [`aggregate_child_name`]. -/// -/// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) for a plan that the boot -/// tokens cannot express, such as one that maps a path below a share's root, rather than extend -/// what the shell reads. -pub fn boot_shares(plan: &ExportPlan) -> Result { - let mut shares = Vec::with_capacity(plan.children.len()); - for (index, child) in plan.children.iter().enumerate() { - let root = path_text(&child.root)?; - let binds: Vec<_> = plan - .binds - .iter() - .filter(|bind| bind.child == index) - .collect(); - let bind = match binds[..] { - [bind] if bind.source.is_empty() => bind, - [bind] => { - return Err(refused(format!( - "the boot tokens of a share cannot map {}, a path below the share's root {root}", - child.root.join(&bind.source).display() - ))); - } - _ => { - return Err(refused(format!( - "the boot tokens of a share map {root} at one target, not {}", - binds.len() - ))); - } - }; - validate_mount_target(&bind.target)?; - // The tokens mount a share with the access with which OpenVMM exports it, so a bind of - // another access, such as a read-only bind of a directory whose writes OpenVMM narrows to - // a path inside it, would let the guest write what the plan keeps read-only. - if bind.read_only == child.writable { - let (bound, exported) = if child.writable { - ("read-only", "read-write") - } else { - ("read-write", "read-only") - }; - return Err(refused(format!( - "the boot tokens of a share cannot map {root} {bound} while OpenVMM exports it \ - {exported}" - ))); - } - let access = if child.writable { - MountAccess::ReadWrite - } else { - MountAccess::ReadOnly - }; - shares.push((child, root, bind.target.as_str(), access)); - } - if plan.binds.len() != shares.len() { - return Err(refused( - "the boot tokens of a share cannot map a path of an unknown host directory", - )); - } - let mut boot = BootShares::default(); - match shares[..] { - [] => return Ok(boot), - [(child, root, target, access)] if !child.file => { - boot.arguments = vec!["--mount".to_owned(), mount_argument(target, root, access)]; - boot.fragment = command_line_fragment(target, access); - } - _ => { - boot.arguments = vec![ - "--mount-aggregate".to_owned(), - AGGREGATE_MOUNT_TARGET.to_owned(), - ]; - for (index, &(child, root, target, access)) in shares.iter().enumerate() { - let name = aggregate_child_name(index, target); - let (kind, file) = if child.file { - (MountKind::File, ",file") - } else { - (MountKind::Directory, "") - }; - boot.arguments.push("--mount-child".to_owned()); - boot.arguments - .push(aggregate_child_argument(&name, root, access, kind)); - boot.tokens - .push(format!("nvx_share={name},{target},{access}{file}")); - } - let accesses: Vec = - shares.iter().map(|&(_, _, _, access)| access).collect(); - boot.fragment = aggregate_command_line_fragment(&accesses); - } - } - for &(child, ..) in &shares { - for (kind, paths) in [ - (PolicyKind::Denied, &child.denied), - (PolicyKind::Allowed, &child.allowed), - (PolicyKind::Writable, &child.write), - ] { - for path in paths { - boot.arguments.push(kind.option().to_owned()); - boot.arguments.push(path_text(path)?.to_owned()); - } - } - } - if plan.owner == Owner::Caller { - boot.arguments.push("--mount-owner".to_owned()); - boot.arguments.push(MountOwner::Caller.as_str().to_owned()); - } - Ok(boot) -} - /// Returns the text of a host path of an export plan, which OpenVMM's options carry. /// /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) for a path that is not @@ -148,55 +13,3 @@ pub(super) fn path_text(path: &Path) -> Result<&str> { path.to_str() .ok_or_else(|| refused(format!("{} is not valid UTF-8", path.display()))) } - -/// Returns the kernel command-line tokens that tell the guest where to bind each child of an -/// aggregate, given each share's guest target, access, and kind. A file child's token ends in -/// `,file`, so the guest binds it at a file. -pub fn aggregate_share_tokens(shares: &[(&str, MountAccess, MountKind)]) -> Vec { - shares - .iter() - .enumerate() - .map(|(index, &(target, access, kind))| { - let file = if kind == MountKind::File { ",file" } else { "" }; - format!( - "nvx_share={},{target},{access}{file}", - aggregate_child_name(index, target) - ) - }) - .collect() -} - -/// Returns the bootstrap tokens that OpenVMM appends for an aggregate of shares with -/// `accesses`, which it mounts read-write if any child is. -pub fn aggregate_command_line_fragment(accesses: &[MountAccess]) -> String { - aggregate_fragment(AGGREGATE_MOUNT_TARGET, accesses) -} - -/// Returns the bootstrap tokens that OpenVMM appends for the export of `plan` at -/// [`GUEST_EXPORT`], the aggregate from which a managed sandbox's guest agent maps the plan's -/// mapping table. -pub(crate) fn export_command_line_fragment(plan: &ExportPlan) -> String { - let accesses: Vec = plan - .children - .iter() - .map(|child| { - if child.writable { - MountAccess::ReadWrite - } else { - MountAccess::ReadOnly - } - }) - .collect(); - aggregate_fragment(GUEST_EXPORT, &accesses) -} - -/// Returns the bootstrap tokens that OpenVMM appends for an aggregate at `target` whose children -/// have `accesses`, which it mounts read-write if any child is. -fn aggregate_fragment(target: &str, accesses: &[MountAccess]) -> String { - let access = if accesses.contains(&MountAccess::ReadWrite) { - MountAccess::ReadWrite - } else { - MountAccess::ReadOnly - }; - command_line_fragment(target, access) + AGGREGATE_COMMAND_LINE_SUFFIX -} diff --git a/nvx-rs/src/host/sandbox/mount/host_paths.rs b/nvx-rs/src/host/sandbox/mount/host_paths.rs index 915f051e9..d98c32209 100644 --- a/nvx-rs/src/host/sandbox/mount/host_paths.rs +++ b/nvx-rs/src/host/sandbox/mount/host_paths.rs @@ -102,6 +102,13 @@ pub(crate) fn validate_mount_host_paths_with( /// [`ErrorKind::Io`](crate::ErrorKind::Io) if a path cannot be examined. /// /// [`plan_mounts`]: crate::host::sandbox::mount::plan_mounts +#[cfg_attr( + not(feature = "cli"), + allow( + dead_code, + reason = "only the CLI's sandboxes check their shares again" + ) +)] pub(crate) fn check_distinct_export_roots( export: &ExportPlan, host: &HostQueries<'_>, diff --git a/nvx-rs/src/host/sandbox/mount/mod.rs b/nvx-rs/src/host/sandbox/mount/mod.rs index 5a3835c83..a679bc790 100644 --- a/nvx-rs/src/host/sandbox/mount/mod.rs +++ b/nvx-rs/src/host/sandbox/mount/mod.rs @@ -6,15 +6,8 @@ //! runtime keeps the [`RESERVED_MOUNT_TARGETS`] and the [`RESERVED_EXACT_MOUNT_TARGETS`], and the //! sandbox records the resulting [`ExportPlan`]. //! -//! [`boot_shares`] renders such a plan in the form that the guest's shell reads at boot, as the -//! one-shot lifecycle delivers it. OpenVMM attaches one virtio-fs device, with the tag -//! [`MOUNT_TAG`]. One directory uses it directly, and the guest mounts it at its target. Several -//! directories or files, or a file, share it as the children of an aggregate, each named by -//! [`aggregate_child_name`], which the guest mounts at [`AGGREGATE_MOUNT_TARGET`], a directory -//! that only the guest's root can enter, before it binds each child at its target, as the -//! `nvx_share=` kernel parameters tell it. The managed lifecycle exports the same plan at -//! [`GUEST_EXPORT`] instead, from which the guest agent maps it as the plan's mapping table -//! orders; see [`SandboxPlan`](super::SandboxPlan). +//! OpenVMM attaches one virtio-fs device, with the tag [`MOUNT_TAG`], which exports the plan at +//! [`GUEST_EXPORT`], from which the guest maps it as the plan's mapping table orders. //! //! A share's access-policy paths are host paths: `--mount-deny` hides a path, `--mount-allow` //! exposes a path inside a denied path again, and `--mount-write` makes a path one of the only @@ -37,14 +30,15 @@ use crate::error::Result; pub use crate::protocol::share::{RESERVED_EXACT_MOUNT_TARGETS, RESERVED_MOUNT_TARGETS}; -pub use self::aggregate::{ - aggregate_child_argument, aggregate_child_name, mount_argument, plan_mounts, -}; -pub(crate) use self::boot::export_command_line_fragment; -pub use self::boot::{ - BootShares, aggregate_command_line_fragment, aggregate_share_tokens, boot_shares, -}; +pub use self::aggregate::plan_mounts; pub use self::host_paths::validate_mount_host_paths; +#[cfg_attr( + not(feature = "cli"), + allow( + unused_imports, + reason = "only the CLI's sandboxes check their shares again" + ) +)] pub(crate) use self::host_paths::{HostQueries, check_distinct_export_roots}; pub use self::request::{ SandboxMount, mount_kind, parse_mount_value, require_mount_owner_supported, @@ -63,12 +57,6 @@ pub const MAX_MOUNT_POLICY_PATHS: usize = 128; /// The longest guest target, in UTF-8 bytes. pub const MAX_MOUNT_TARGET_BYTES: usize = 4096; -/// The bootstrap tokens that OpenVMM appends to the kernel command line for its share, with the -/// guest directory, the tag, and the mode. -const MOUNT_COMMAND_LINE_FRAGMENT: [&str; 3] = [" virtfs_dir=", " virtfs_tag=", " virtfs_mode="]; -/// The token that OpenVMM appends after those of an aggregate. -const AGGREGATE_COMMAND_LINE_SUFFIX: &str = " virtfs_aggregate=1"; - /// The access mode of a share. #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] pub enum MountAccess { diff --git a/nvx-rs/src/host/sandbox/mount/request.rs b/nvx-rs/src/host/sandbox/mount/request.rs index e7050f6fa..9d940d5fc 100644 --- a/nvx-rs/src/host/sandbox/mount/request.rs +++ b/nvx-rs/src/host/sandbox/mount/request.rs @@ -5,8 +5,8 @@ use std::io; use std::path::{Component, Path, PathBuf}; use super::{ - MAX_MOUNT_POLICY_PATHS, MAX_MOUNT_TARGET_BYTES, MOUNT_COMMAND_LINE_FRAGMENT, MOUNT_TAG, - MountAccess, MountKind, MountOwner, MountPolicy, POLICY_KINDS, PolicyKind, + MAX_MOUNT_POLICY_PATHS, MAX_MOUNT_TARGET_BYTES, MountAccess, MountKind, MountOwner, + MountPolicy, POLICY_KINDS, PolicyKind, }; use crate::error::{Error, Result}; use crate::host::filesystem::within; @@ -421,12 +421,6 @@ fn absolute_host_path(directory: &Path, path: &Path) -> Result { paths::absolute_in(directory, path).map_err(|error| os_error(&error, Some(path))) } -/// Returns the bootstrap tokens of a share at `target` with `access`. -pub(super) fn command_line_fragment(target: &str, access: MountAccess) -> String { - let [directory, tag, mode] = MOUNT_COMMAND_LINE_FRAGMENT; - format!("{directory}{target}{tag}{MOUNT_TAG}{mode}{access}") -} - /// Checks the live shares that one sandbox attaches, in child order: no target may overlap /// another, so that no share hides another, and every share has the same owner. /// diff --git a/nvx-rs/src/host/sandbox/mount/tests.rs b/nvx-rs/src/host/sandbox/mount/tests.rs index d52e46cd6..ab845ecf7 100644 --- a/nvx-rs/src/host/sandbox/mount/tests.rs +++ b/nvx-rs/src/host/sandbox/mount/tests.rs @@ -8,9 +8,11 @@ pub(crate) use super::host_paths::validate_mount_host_paths_with; use super::request::require_mount_owner_supported_on; use super::*; use crate::ErrorKind; -use crate::host::filesystem::{Bind, Owner, ReservedTargets}; +use crate::host::filesystem::{Owner, ReservedTargets}; use crate::host::paths; -use crate::host::sandbox::tests::{deny, exported, host, host_text, native, policy}; +#[cfg(windows)] +use crate::host::sandbox::tests::host_text; +use crate::host::sandbox::tests::{host, native, policy}; fn parse(value: &str, policy: MountPolicy) -> Result { SandboxMount::parse(value, policy, MountOwner::Vmm, MountKind::Directory) @@ -37,31 +39,6 @@ fn mount_parser_defaults_to_read_only_and_accepts_rw() { .unwrap(); assert_eq!(writable.access(), MountAccess::ReadWrite); assert_eq!(writable.policy().denied, ["logs", "secrets"]); - // One directory uses the device directly, with the absolute denied paths of its plan. - let mut plan = exported(&[( - "/opt/hostedtoolcache", - "/host/dir", - MountAccess::ReadWrite, - MountKind::Directory, - )]); - deny(&mut plan, 0, &["/host/dir/logs", "/host/dir/secrets"]); - let boot = boot_shares(&plan).unwrap(); - assert_eq!( - boot.arguments, - [ - "--mount".to_owned(), - format!("/opt/hostedtoolcache,{},rw", host_text("/host/dir")), - "--mount-deny".to_owned(), - host_text("/host/dir/logs"), - "--mount-deny".to_owned(), - host_text("/host/dir/secrets"), - ] - ); - assert!(boot.tokens.is_empty()); - assert_eq!( - boot.fragment, - " virtfs_dir=/opt/hostedtoolcache virtfs_tag=microvm virtfs_mode=rw" - ); } #[test] @@ -163,70 +140,13 @@ fn mount_parser_accepts_host_paths_with_commas() { let mount = parse(&value, policy(&["secret"], &[], &[])).unwrap(); assert_eq!(mount.host_path(), Path::new(&native(host_path))); assert_eq!(mount.access(), access); - // OpenVMM finds the end of the host path before the mode, which NVX therefore always - // gives. - let mut plan = exported(&[(target, "/unused", access, MountKind::Directory)]); - plan.children[0].root = PathBuf::from(native(host_path)); - assert_eq!( - boot_shares(&plan).unwrap().arguments, - [ - "--mount".to_owned(), - format!("{target},{},{access}", native(host_path)), - ] - ); } - let mut plan = exported(&[ - ( - "/work", - "/unused/a", - MountAccess::ReadWrite, - MountKind::Directory, - ), - ( - "/reports", - "/unused/b", - MountAccess::ReadOnly, - MountKind::Directory, - ), - ]); - plan.children[0].root = PathBuf::from(native(alpha)); - plan.children[1].root = PathBuf::from(native(reports)); - let arguments = boot_shares(&plan).unwrap().arguments; - let children: Vec<&String> = arguments - .iter() - .zip(arguments.iter().skip(1)) - .filter(|(option, _)| *option == "--mount-child") - .map(|(_, value)| value) - .collect(); - assert_eq!( - children, - [ - &format!("{},{},rw", aggregate_child_name(0, "/work"), native(alpha)), - &format!( - "{},{},ro", - aggregate_child_name(1, "/reports"), - native(reports) - ), - ] - ); } #[test] fn mount_owner_defaults_to_vmm_and_forwards_caller() { let default = parse("/workspace,host-dir,rw", MountPolicy::default()).unwrap(); assert_eq!(default.owner(), MountOwner::Vmm); - let mut plan = exported(&[( - "/workspace", - "/host/dir", - MountAccess::ReadWrite, - MountKind::Directory, - )]); - assert!( - !boot_shares(&plan) - .unwrap() - .arguments - .contains(&"--mount-owner".to_owned()) - ); let caller = SandboxMount::parse( "/workspace,host-dir,rw", policy(&["secrets"], &[], &[]), @@ -234,23 +154,7 @@ fn mount_owner_defaults_to_vmm_and_forwards_caller() { MountKind::Directory, ) .unwrap(); - deny(&mut plan, 0, &["/host/dir/secrets"]); - let vmm = boot_shares(&plan).unwrap(); - plan.owner = Owner::Caller; - let boot = boot_shares(&plan).unwrap(); - assert_eq!( - boot.arguments, - [ - "--mount".to_owned(), - format!("/workspace,{},rw", host_text("/host/dir")), - "--mount-deny".to_owned(), - host_text("/host/dir/secrets"), - "--mount-owner".to_owned(), - "caller".to_owned(), - ] - ); assert_eq!(caller.absolute().unwrap().owner(), MountOwner::Caller); - assert_eq!(boot.fragment, vmm.fragment); assert_eq!( message(MountOwner::from_name("root")), "unsupported sandbox mount owner \"root\"; choose vmm or caller" @@ -268,40 +172,9 @@ fn mount_access_policy_is_validated_and_forwarded() { policy(&["mcp-logs"], &["mcp-logs/payloads"], &["agent", "cache"]), ) .unwrap(); - let mut plan = exported(&[( - "/tmp/mcp-runtime", - "/host/dir", - MountAccess::ReadWrite, - MountKind::Directory, - )]); - plan.children[0].write = vec![host("/host/dir/agent"), host("/host/dir/cache")]; - deny(&mut plan, 0, &["/host/dir/mcp-logs"]); - plan.children[0].allowed = vec![host("/host/dir/mcp-logs/payloads")]; - let boot = boot_shares(&plan).unwrap(); - assert_eq!( - boot.arguments, - [ - "--mount".to_owned(), - format!("/tmp/mcp-runtime,{},rw", host_text("/host/dir")), - "--mount-deny".to_owned(), - host_text("/host/dir/mcp-logs"), - "--mount-allow".to_owned(), - host_text("/host/dir/mcp-logs/payloads"), - "--mount-write".to_owned(), - host_text("/host/dir/agent"), - "--mount-write".to_owned(), - host_text("/host/dir/cache"), - ] - ); let absolute = mount.absolute().unwrap(); assert_eq!(absolute.policy().allowed, ["mcp-logs/payloads"]); assert_eq!(absolute.policy().writable, ["agent", "cache"]); - // OpenVMM enforces the policy on the host, so the guest's bootstrap tokens are those of the - // plain share. - assert_eq!( - boot.fragment, - " virtfs_dir=/tmp/mcp-runtime virtfs_tag=microvm virtfs_mode=rw" - ); let paths: Vec = (0..129).map(|index| format!("path-{index}")).collect(); let many: Vec<&str> = paths.iter().map(String::as_str).collect(); for (value, policy, expected) in [ @@ -596,41 +469,6 @@ fn mount_target_validation_rejects_unsafe_and_reserved_targets() { validate_mount_target(&format!("/{}", "x".repeat(4095))).unwrap(); } -#[test] -fn aggregate_child_names_pin_their_targets() { - let name = aggregate_child_name(0, "/workspace"); - assert_eq!(name, "0-c52ddf65534b7b46035084358ab7902b"); - // Equal targets give equal names, so a restore names its children as the boot did, and - // another target or position gives another name. - assert_eq!(aggregate_child_name(0, "/workspace"), name); - assert_ne!(aggregate_child_name(0, "/workspace2"), name); - assert_ne!(aggregate_child_name(1, "/workspace"), name); - // Every name follows OpenVMM's rule for the name of a child. - for (index, target) in [ - (0, "/w".to_owned()), - (999, format!("/{}", "x".repeat(1000))), - ] { - let name = aggregate_child_name(index, &target); - let (number, digest) = name.split_once('-').unwrap(); - assert!(number.chars().all(|c| c.is_ascii_digit())); - assert_eq!(digest.len(), 32); - assert!(digest.chars().all(|c| matches!(c, '0'..='9' | 'a'..='f'))); - } - assert!(aggregate_child_name(999, "/w").len() <= 64); - assert_eq!( - aggregate_child_argument("n", "/h,x", MountAccess::ReadWrite, MountKind::File), - "n,/h,x,rw,file" - ); - assert_eq!( - mount_argument("/t", "/h", MountAccess::ReadOnly), - "/t,/h,ro" - ); - assert_eq!( - aggregate_command_line_fragment(&[MountAccess::ReadOnly, MountAccess::ReadOnly]), - " virtfs_dir=/run/nvx/shares virtfs_tag=microvm virtfs_mode=ro virtfs_aggregate=1" - ); -} - /// Returns the resolved form of `path`, without Windows' verbatim prefix, as the planner records /// host paths. fn resolved(path: &Path) -> PathBuf { @@ -726,34 +564,6 @@ fn shares_are_planned_through_the_export_planner() { plan.verify().unwrap(); plan.check(ReservedTargets::CONTAINER_ROOT).unwrap(); - // The guest's shell binds each child from its token, in child order. - let names = [ - aggregate_child_name(0, "/etc/app/settings.json"), - aggregate_child_name(1, "/opt/tools"), - aggregate_child_name(2, "/workspace"), - ]; - let boot = boot_shares(&plan).unwrap(); - assert_eq!( - boot.tokens, - [ - format!("nvx_share={},/etc/app/settings.json,ro,file", names[0]), - format!("nvx_share={},/opt/tools,ro", names[1]), - format!("nvx_share={},/workspace,rw", names[2]), - ] - ); - let text = |path: &Path| path.to_str().unwrap().to_owned(); - assert_eq!( - boot.arguments[boot.arguments.len() - 6..], - [ - "--mount-deny".to_owned(), - text(&work.join("secrets")), - "--mount-allow".to_owned(), - text(&work.join("secrets").join("public")), - "--mount-write".to_owned(), - text(&work.join("out")), - ] - ); - // The planner refuses what OpenVMM would: here, a writable path that a denied path hides. let error = plan_mounts(&[share( format!("/workspace,{},rw", joined("work").display()), @@ -774,10 +584,6 @@ fn shares_are_planned_through_the_export_planner() { if owned.uid() != 0 && owned.gid() != 0 { let plan = plan_mounts(&mounts(MountOwner::Caller)).unwrap().unwrap(); assert_eq!(plan.owner, Owner::Caller); - assert_eq!( - boot_shares(&plan).unwrap().arguments[boot.arguments.len()..], - ["--mount-owner", "caller"] - ); } } } @@ -857,77 +663,3 @@ fn planned_shares_keep_their_owner_and_kind() { "{error}" ); } - -#[test] -fn boot_tokens_refuse_what_the_shell_cannot_bind() { - let plan = exported(&[( - "/workspace", - "/host/work", - MountAccess::ReadWrite, - MountKind::Directory, - )]); - // A path below a share's root. - let mut below = plan.clone(); - below.binds[0].source = "sub".to_owned(); - assert_eq!( - message(boot_shares(&below)), - format!( - "the boot tokens of a share cannot map {}, a path below the share's root {}", - host("/host/work").join("sub").display(), - host_text("/host/work") - ) - ); - // One child at two targets. - let mut twice = plan.clone(); - twice.binds.push(Bind { - target: "/workspace2".to_owned(), - ..twice.binds[0].clone() - }); - assert_eq!( - message(boot_shares(&twice)), - format!( - "the boot tokens of a share map {} at one target, not 2", - host_text("/host/work") - ) - ); - // A bind of no child. - let mut unknown = plan.clone(); - unknown.binds.push(Bind { - child: 1, - target: "/other".to_owned(), - ..unknown.binds[0].clone() - }); - assert_eq!( - message(boot_shares(&unknown)), - "the boot tokens of a share cannot map a path of an unknown host directory" - ); - // A read-only bind of a directory whose writes OpenVMM narrows to a path inside it, which the - // tokens would mount read-write, and a read-write bind of a read-only directory. - let mut narrowed = plan.clone(); - narrowed.children[0].write = vec![host("/host/work/out")]; - narrowed.binds[0].read_only = true; - assert_eq!( - message(boot_shares(&narrowed)), - format!( - "the boot tokens of a share cannot map {} read-only while OpenVMM exports it \ - read-write", - host_text("/host/work") - ) - ); - assert!(narrowed.check(ReservedTargets::CONTAINER_ROOT).is_err()); - let mut read_only = plan.clone(); - read_only.children[0].writable = false; - assert_eq!( - message(boot_shares(&read_only)), - format!( - "the boot tokens of a share cannot map {} read-write while OpenVMM exports it \ - read-only", - host_text("/host/work") - ) - ); - assert!(read_only.check(ReservedTargets::CONTAINER_ROOT).is_err()); - // A target that a token cannot carry. - let mut comma = plan; - comma.binds[0].target = "/work,space".to_owned(); - assert!(message(boot_shares(&comma)).contains("invalid sandbox mount target")); -} diff --git a/nvx-rs/src/host/sandbox/options.rs b/nvx-rs/src/host/sandbox/options.rs index 1c18be835..a35fa871f 100644 --- a/nvx-rs/src/host/sandbox/options.rs +++ b/nvx-rs/src/host/sandbox/options.rs @@ -5,9 +5,7 @@ //! environment, from `--environment` or `--environment-file`. //! - [`sandbox_mounts`]: the shares of the `--mount` options, with each `--mount-deny`, //! `--mount-allow`, and `--mount-write` attributed to its share, and each share's kind. -//! - [`validate_sandbox_entrypoint`]: the refusal of systemd entrypoints, and -//! [`validate_sandbox_systemd_policy`]: the preflight of `run` and `provision`, which also -//! verifies the distro layer and returns its [`DistroBinding`]. +//! - [`validate_sandbox_entrypoint`]: the refusal of systemd entrypoints. //! - [`validate_run_share_targets`]: the guest targets of several `nvx-rs run` shares, or of a //! shared file. //! - [`validate_exec_environment`] and [`exec_environment_from_file`]: the environment of a @@ -18,12 +16,11 @@ use std::fs::File; use std::io::Read; use std::path::Path; -use super::distro::DistroBinding; use super::mount::{ AGGREGATE_MOUNT_TARGET, MountKind, MountOwner, MountPolicy, POLICY_KINDS, PolicyKind, SandboxMount, has_forbidden_character, has_irregular_component, mount_kind, targets_overlap, }; -use super::{SandboxLaunch, refused}; +use super::refused; use crate::error::Result; use crate::protocol::root::RootUpper; @@ -374,18 +371,6 @@ pub fn validate_sandbox_entrypoint(entrypoint: &str) -> Result<()> { Ok(()) } -/// Refuses a launch that would run systemd, and returns the binding of its distro layer: the -/// preflight of `nvx-rs sandbox run` and `provision`. -/// -/// It refuses a systemd entrypoint with [`validate_sandbox_entrypoint`], and then verifies the -/// distro layer explicitly with [`DistroBinding::verify`], which refuses a layer whose manifest, -/// `LAYER.manifest.json` next to it, lists the `systemd` package, and hashes the layer only when -/// that manifest exists. Fails as they do. -pub fn validate_sandbox_systemd_policy(launch: &SandboxLaunch) -> Result { - validate_sandbox_entrypoint(&launch.settings().entrypoint)?; - DistroBinding::verify(launch.distro()) -} - /// Checks a managed exec environment, as `encode_exec_environment` does: at most /// [`MAX_EXEC_ENVIRONMENT`] entries, each a `KEY=VALUE` string with a nonempty key, without NUL, /// of at most [`MAX_EXEC_ENVIRONMENT_BYTES`] bytes, and no key twice. diff --git a/nvx-rs/src/host/sandbox/options/tests.rs b/nvx-rs/src/host/sandbox/options/tests.rs index 111bc9f4b..e54f01680 100644 --- a/nvx-rs/src/host/sandbox/options/tests.rs +++ b/nvx-rs/src/host/sandbox/options/tests.rs @@ -10,9 +10,10 @@ use sha2::{Digest, Sha256}; use super::*; use crate::ErrorKind; use crate::host::paths::{self, resolve}; +use crate::host::sandbox::SandboxLayer; +use crate::host::sandbox::distro::{DISTRO_POLICY_REVISION, DistroBinding, DistroFormat}; use crate::host::sandbox::layer::LayerRole; use crate::host::sandbox::mount::{plan_mounts, validate_mounts}; -use crate::host::sandbox::{SandboxLayer, SandboxSettings, SandboxUpper}; fn message(result: Result) -> String { let error = result.unwrap_err(); @@ -76,20 +77,19 @@ impl Layer { path.to_str().unwrap().to_owned() } - fn launch(&self, entrypoint: &str) -> SandboxLaunch { - SandboxLaunch::new( - vec![SandboxLayer::new( - LayerRole::Distro, - &self.path("distro.erofs"), - "11111111-1111-1111-1111-111111111111", - )], - SandboxUpper::scratch(&self.path("scratch.ext4")), - SandboxSettings { - entrypoint: entrypoint.to_owned(), - ..SandboxSettings::default() - }, + fn layer(&self) -> SandboxLayer { + SandboxLayer::new( + LayerRole::Distro, + &self.path("distro.erofs"), + "11111111-1111-1111-1111-111111111111", ) - .unwrap() + } + + /// Refuses a systemd `entrypoint`, and then verifies the layer, as the preflight of + /// `nvx-rs sandbox run` and `provision` does. + fn preflight(&self, entrypoint: &str) -> Result { + validate_sandbox_entrypoint(entrypoint)?; + DistroBinding::verify(&self.layer()) } fn manifest(&self, text: &str) { @@ -196,7 +196,6 @@ fn environment_files_hold_a_json_array_of_strings() { #[test] fn distro_manifest_binds_the_layer() { let layer = Layer::new(); - let launch = layer.launch("/bin/sh"); let valid = json!({ "format": 1, "artifact": "distro.erofs", @@ -204,15 +203,12 @@ fn distro_manifest_binds_the_layer() { "packages": [{"name": "bash"}, {"name": "coreutils"}], }); layer.manifest(&valid.to_string()); - let binding = validate_sandbox_systemd_policy(&launch).unwrap(); + let binding = layer.preflight("/bin/sh").unwrap(); assert_eq!(binding.identity(), "11111111-1111-1111-1111-111111111111"); - assert_eq!(binding.format(), super::super::DistroFormat::Erofs); + assert_eq!(binding.format(), DistroFormat::Erofs); assert!(binding.manifest().is_some()); binding - .check( - launch.distro(), - super::super::distro::DISTRO_POLICY_REVISION, - ) + .check(&layer.layer(), DISTRO_POLICY_REVISION) .unwrap(); for (field, value) in [ ("format", json!(2)), @@ -229,7 +225,7 @@ fn distro_manifest_binds_the_layer() { manifest[field] = value; layer.manifest(&manifest.to_string()); assert_eq!( - validate_sandbox_systemd_policy(&launch).unwrap_err().kind(), + layer.preflight("/bin/sh").unwrap_err().kind(), ErrorKind::Unsupported, "{field}: {manifest}", ); @@ -239,14 +235,14 @@ fn distro_manifest_binds_the_layer() { manifest.as_object_mut().unwrap().remove(field); layer.manifest(&manifest.to_string()); assert_eq!( - validate_sandbox_systemd_policy(&launch).unwrap_err().kind(), + layer.preflight("/bin/sh").unwrap_err().kind(), ErrorKind::Unsupported, ); } layer.manifest(&valid.to_string()); fs::write(layer.path("distro.erofs"), b"changed distro").unwrap(); assert_eq!( - validate_sandbox_systemd_policy(&launch).unwrap_err().kind(), + layer.preflight("/bin/sh").unwrap_err().kind(), ErrorKind::Unsupported, ); } @@ -266,13 +262,11 @@ fn systemd_entrypoints_and_images_are_refused_independently() { ); for entrypoint in ["/usr/lib/systemd/systemd", "/lib/systemd/systemd"] { assert_eq!( - validate_sandbox_systemd_policy(&layer.launch(entrypoint)) - .unwrap_err() - .kind(), + layer.preflight(entrypoint).unwrap_err().kind(), ErrorKind::Unsupported, ); } - let result = validate_sandbox_systemd_policy(&layer.launch("/sbin/init")); + let result = layer.preflight("/sbin/init"); assert_eq!(result.is_ok(), packages.as_array().unwrap().is_empty()); } } @@ -364,11 +358,9 @@ proptest! { #[test] fn sandbox_allows_non_systemd_sbin_init() { let layer = Layer::new(); - validate_sandbox_systemd_policy(&layer.launch("/sbin/init")).unwrap(); + layer.preflight("/sbin/init").unwrap(); assert_eq!( - message(validate_sandbox_systemd_policy( - &layer.launch("/lib/systemd/systemd") - )), + message(layer.preflight("/lib/systemd/systemd")), "systemd entrypoints are unsupported by the sandbox security profile" ); } @@ -385,20 +377,16 @@ fn the_entrypoint_check_needs_no_manifest_and_precedes_the_distro_verification() validate_sandbox_entrypoint("/sbin/init").unwrap(); // A layer without a manifest passes the distro verification unread, and its binding records // the manifest's absence, but the entrypoint check still refuses systemd. - let binding = validate_sandbox_systemd_policy(&layer.launch("/sbin/init")).unwrap(); + let binding = layer.preflight("/sbin/init").unwrap(); assert_eq!(binding.manifest(), None); assert_eq!( - message(validate_sandbox_systemd_policy( - &layer.launch("/usr/lib/systemd/systemd") - )), + message(layer.preflight("/usr/lib/systemd/systemd")), "systemd entrypoints are unsupported by the sandbox security profile" ); // The entrypoint check precedes the manifest, which it never reads. layer.manifest("not JSON"); assert_eq!( - message(validate_sandbox_systemd_policy( - &layer.launch("/lib/systemd/systemd") - )), + message(layer.preflight("/lib/systemd/systemd")), "systemd entrypoints are unsupported by the sandbox security profile" ); } @@ -412,7 +400,7 @@ fn sandbox_rejects_systemd_from_bound_distro_metadata() { "{{\"format\": {format}, \"artifact\": {artifact}, \"artifact_sha256\": {sha256}, \ \"packages\": {packages}}}" )); - validate_sandbox_systemd_policy(&layer.launch("/sbin/init")) + layer.preflight("/sbin/init") }; let sha256 = format!("\"{digest}\""); assert_eq!( @@ -464,22 +452,22 @@ fn sandbox_rejects_systemd_from_bound_distro_metadata() { } layer.manifest("[]"); assert_eq!( - message(validate_sandbox_systemd_policy(&layer.launch("/sbin/init"))), + message(layer.preflight("/sbin/init")), "invalid sandbox distro manifest: expected a JSON object" ); layer.manifest("{\r\n\"format\": }"); - let error = message(validate_sandbox_systemd_policy(&layer.launch("/sbin/init"))); + let error = message(layer.preflight("/sbin/init")); assert!(error.starts_with("invalid sandbox distro manifest: ")); assert!(error.contains("line 2 column 11"), "{error}"); fs::write(layer.path("distro.erofs.manifest.json"), b"{\xff}").unwrap(); - let error = message(validate_sandbox_systemd_policy(&layer.launch("/sbin/init"))); + let error = message(layer.preflight("/sbin/init")); assert!( error.starts_with("invalid sandbox distro manifest: "), "{error}" ); // A layer without a manifest passes. fs::remove_file(layer.path("distro.erofs.manifest.json")).unwrap(); - validate_sandbox_systemd_policy(&layer.launch("/sbin/init")).unwrap(); + layer.preflight("/sbin/init").unwrap(); } #[test] @@ -490,7 +478,7 @@ fn distro_manifests_are_strict_json() { layer.manifest(&format!( r#"{{"format":1,"artifact":"distro.erofs","artifact_sha256":"{digest}","packages":[],"extra":{extra}}}"# )); - message(validate_sandbox_systemd_policy(&layer.launch("/bin/sh"))) + message(layer.preflight("/bin/sh")) }; for extra in [ "NaN", @@ -514,7 +502,7 @@ fn distro_manifests_are_strict_json() { r#"{{"format":1,"artifact":"distro.erofs","artifact_sha256":"{digest}","packages":[],"extra":{}}}"#, nested(crate::host::network::policy::MAX_JSON_DEPTH - 1) )); - validate_sandbox_systemd_policy(&layer.launch("/bin/sh")).unwrap(); + layer.preflight("/bin/sh").unwrap(); let error = with_extra(&nested(crate::host::network::policy::MAX_JSON_DEPTH)); assert!( error.starts_with( @@ -526,14 +514,14 @@ fn distro_manifests_are_strict_json() { r#"{{"format":1,"format":1,"artifact":"distro.erofs","artifact_sha256":"{digest}","packages":[]}}"# )); assert_eq!( - message(validate_sandbox_systemd_policy(&layer.launch("/bin/sh"))), + message(layer.preflight("/bin/sh")), "invalid sandbox distro manifest: manifest has duplicate field \"format\"" ); layer.manifest(&format!( r#"{{"format":1,"artifact":"distro.erofs","artifact_sha256":"{digest}","packages":[{{"name":"bash","name":"systemd"}}]}}"# )); assert_eq!( - message(validate_sandbox_systemd_policy(&layer.launch("/bin/sh"))), + message(layer.preflight("/bin/sh")), "invalid sandbox distro manifest: packages[0] has duplicate field \"name\"" ); fs::write( @@ -541,7 +529,7 @@ fn distro_manifests_are_strict_json() { b"{\"format\":\xff}", ) .unwrap(); - let error = message(validate_sandbox_systemd_policy(&layer.launch("/bin/sh"))); + let error = message(layer.preflight("/bin/sh")); assert!( error.starts_with("invalid sandbox distro manifest: not UTF-8: "), "{error}" @@ -903,8 +891,7 @@ fn sandbox_command_rejects_misplaced_mount_options() { } } -/// Host directories for shares in a temporary directory, with a distro layer and a scratch -/// image. +/// Host directories for shares in a temporary directory. struct Shares { /// The temporary directory. directory: tempfile::TempDir, @@ -916,8 +903,6 @@ impl Shares { fn new() -> Self { let directory = tempfile::tempdir().unwrap(); let root = resolve(directory.path()).unwrap(); - fs::write(root.join("distro.erofs"), b"distro").unwrap(); - fs::write(root.join("scratch.ext4"), b"scratch").unwrap(); Self { directory, root } } @@ -933,9 +918,9 @@ impl Shares { path } - /// Runs the steps of `nvx-rs sandbox --dry-run` that build the launch of these shares, and - /// returns the OpenVMM arguments and the kernel command line of its plan, which exports the - /// shares for the mapping table that the host sends once the guest is ready. + /// Runs the steps of `nvx-rs sandbox --dry-run` that plan these shares, and returns the + /// OpenVMM arguments that export them and the kernel token that counts the mappings that the + /// host sends once the guest is ready. fn run( &self, mounts: &[String], @@ -959,24 +944,10 @@ impl Shares { ..launching(SandboxOperation::Run) } .check()?; - let launch = SandboxLaunch::new( - vec![SandboxLayer::new( - LayerRole::Distro, - &self.path("distro.erofs"), - "11111111-1111-1111-1111-111111111111", - )], - SandboxUpper::scratch(&self.path("scratch.ext4")), - SandboxSettings::default(), - )?; let mounts = sandbox_mounts(&mounts, &policy_paths, owner)?; validate_mounts(&mounts)?; - launch.validated()?; - let launch = launch.with_export(plan_mounts(&mounts)?)?; - validate_sandbox_systemd_policy(&launch)?; - Ok(( - launch.plan().openvmm_arguments()?, - launch.plan().kernel_command_line("")?, - )) + let export = plan_mounts(&mounts)?.expect("shares have an export plan"); + Ok((export.openvmm_arguments(), export.kernel_token())) } } @@ -1029,7 +1000,7 @@ fn sandbox_command_shares_a_file() { let shares = Shares::new(); let settings = shares.path("settings.json"); fs::write(&settings, b"{}").unwrap(); - let (arguments, command_line) = shares + let (arguments, token) = shares .run( &[format!("/etc/app/settings.json,{settings},ro")], &[], @@ -1037,14 +1008,13 @@ fn sandbox_command_shares_a_file() { ) .unwrap(); // The regular file is a child of its own, without its directory, which OpenVMM must find to - // be a regular file, and the guest maps it from the table that the command line counts. + // be a regular file, and the guest maps it from the table that the kernel token counts. assert!(!arguments.contains(&"--mount".to_owned())); assert_eq!( values(&arguments, "--mount-child"), [format!("0,{settings},ro,file")] ); - assert!(command_line.ends_with(" nvx_maps=1"), "{command_line}"); - assert!(!command_line.contains("nvx_share="), "{command_line}"); + assert_eq!(token, "nvx_maps=1"); let error = shares .run( &[format!("/etc/app/settings.json,{settings},rw")], @@ -1294,9 +1264,9 @@ fn sandbox_command_attributes_mount_deny_to_the_preceding_mount() { (PolicyKind::Denied, Some(1), "credentials"), (PolicyKind::Denied, Some(1), "logs"), ]; - let (arguments, command_line) = shares.run(&mounts, &policy, None).unwrap(); + let (arguments, token) = shares.run(&mounts, &policy, None).unwrap(); // The children follow their resolved host paths, each with its own denied paths, and the - // command line counts their mappings rather than naming their targets. + // kernel token counts their mappings rather than naming their targets. let (toolcache, workspace) = ("0", "1"); assert!(!arguments.contains(&"--mount".to_owned())); assert_eq!( @@ -1318,8 +1288,7 @@ fn sandbox_command_attributes_mount_deny_to_the_preceding_mount() { shares.path("work/secrets"), ] ); - assert!(command_line.ends_with(" nvx_maps=2"), "{command_line}"); - assert!(!command_line.contains("nvx_share="), "{command_line}"); + assert_eq!(token, "nvx_maps=2"); // With one share, a --mount-deny may come first; with several, it must follow the share // whose directory it hides. diff --git a/nvx-rs/src/host/sandbox/plan.rs b/nvx-rs/src/host/sandbox/plan.rs deleted file mode 100644 index 0c42af9b4..000000000 --- a/nvx-rs/src/host/sandbox/plan.rs +++ /dev/null @@ -1,870 +0,0 @@ -//! The typed plan of a cold layered launch: its block devices, which image over which upper, its -//! launch settings, its live shares, and its network, from which come OpenVMM's arguments and the -//! kernel command line. -//! -//! A [`SandboxPlan`] holds: -//! -//! - one read-only `distro` block device, a [`SandboxDistro`]: a raw host image, an EROFS -//! filesystem or a GPT disk, with the identity that the guest expects of it, if any, as its -//! [`RootImage`] names them; -//! - an upper, a [`SandboxUpper`]: a whole-device ext4 scratch image on the `scratch` block -//! device, or RAM, with no scratch block device; -//! - the launch settings of NVX's sandbox profile, [`SandboxSettings`]: the entrypoint and its -//! arguments, the hostname, the workload identity, and the resource limits. An adapter's plan, -//! [`SandboxPlan::adapter`], has none: the adapter's own guest runs its workloads; -//! - the export plan of the live shares, an [`ExportPlan`] that [`filesystem::plan`](crate::host::filesystem::plan) made for the -//! container root, if the sandbox has shares: the children of OpenVMM's aggregate export, the -//! ordered binds of the guest, and the identities that each start checks again. An adapter's -//! plan has none: the adapter attaches its shares itself; -//! - optionally, a network, [`SandboxNetwork`]: the guest's address and the [`NetworkPolicy`] -//! that OpenVMM enforces there, which one lowering, [`network::openvmm_arguments`], renders for -//! every caller. A proxy-only policy reports the URL of its proxy as the guest sees it, -//! [`proxy_url`](SandboxPlan::proxy_url). -//! -//! A plan keys its block devices by role, [`BlockRole`], and knows only the `distro` and -//! `scratch` roles. Every combination of format and upper plans and renders: -//! [`openvmm_arguments`](SandboxPlan::openvmm_arguments) attaches the block devices, with no -//! scratch block device for a RAM upper, and the network device, and -//! [`kernel_command_line`](SandboxPlan::kernel_command_line) describes the root with the boot -//! tokens of [`protocol::root`](crate::protocol::root), within the budget that OpenVMM leaves the -//! profile. Both render the shares as the managed lifecycle delivers them: OpenVMM exports them -//! at [`GUEST_EXPORT`](crate::host::filesystem::GUEST_EXPORT), where the guest mounts the export, -//! the kernel command line announces the number of their mappings with `nvx_maps=`, and the host -//! sends the plan's [`mapping_table`](SandboxPlan::mapping_table) in `MAPS` requests once the -//! guest agent is ready, which maps each in order beneath the container's root. An adapter's plan -//! renders its root and devices alone, and lets -//! the adapter's own boot tokens through, as -//! [`kernel_command_line`](SandboxPlan::kernel_command_line) says, and -//! [`check_adapter_launch`](SandboxPlan::check_adapter_launch) checks, as the adapter plans its -//! launch, that what the plan renders leaves the rest of the launch room on a Windows command -//! line. The plan's arguments name the block devices by their paths; a launch renders them -//! through a [`PreparedLaunch`](super::PreparedLaunch), which names the files that the checks of -//! the images opened instead. -//! -//! A plan is persistable. Its serde form, of version [`PLAN_VERSION`], requires every field and -//! refuses unknown ones and other versions, and reading it checks the plan as -//! [`SandboxPlan::new`] or [`SandboxPlan::adapter`], [`SandboxPlan::with_export`], and -//! [`SandboxNetwork::new`] do: -//! -//! ```json -//! {"version": 3, -//! "distro": {"path": "/images/distro.erofs", "format": "erofs", -//! "identity": "11111111-1111-1111-1111-111111111111"}, -//! "upper": {"kind": "scratch", "path": "/sandboxes/one/scratch.ext4"}, -//! "settings": {"entrypoint": "/bin/sh", "args": [], "hostname": "nvx-sandbox", -//! "workloadIdentity": {"uid": 65534, "gid": 65534}, -//! "memoryMax": "268435456", "pidsMax": null}, -//! "export": EXPORT, -//! "network": NETWORK} -//! ``` -//! -//! The `format` of the image is `erofs` or `gpt`, its `identity` a UUID in its canonical text or -//! `null`, and a RAM upper is `{"kind": "ram"}`. The limits are decimal strings or `null`, since a -//! memory limit may exceed the integers that a JSON number holds exactly. The `settings` of an -//! adapter's plan are `null`, as is its `EXPORT`. `EXPORT` is `null` -//! without shares, and otherwise the serde form of an [`ExportPlan`]: -//! -//! ```json -//! {"children": [{"root": "/home/user/work", "writable": true, -//! "denied": ["/home/user/work/secrets"], -//! "deniedIdentities": [{"device": 2049, "index": 1835013}], -//! "allowed": [], "write": []}], -//! "binds": [{"child": 0, "source": "", "target": "/workspace", "readOnly": false}]} -//! ``` -//! -//! The `NETWORK` is `null` for no network device, or keeps the fields of a [`SandboxNetwork`] and -//! its [`NetworkPolicy`]: -//! -//! ```json -//! {"guestNetwork": "10.0.0.2/24", "egress": "allow", -//! "allow": {"name": "egress.allow", "rules": []}, -//! "deny": {"name": "egress.deny", -//! "rules": [{"to": [{"cidr": "192.0.2.0/24", "except": ["192.0.2.128/25"]}], -//! "ports": [{"protocol": "tcp", "port": 443, "endPort": null}]}]}, -//! "ingress": "deny", "hostLoopback": "allow", -//! "hostLoopbackForwards": [{"protocol": "tcp", "hostPort": 18080, "guestPort": 8080}], -//! "proxy": null} -//! ``` -//! -//! `egress` and `ingress` are `allow` or `deny`, `hostLoopback` is `deny`, `outbound`, or -//! `allow`, a port's `protocol` is `tcp`, `udp`, `icmp`, or `any`, and a forward's `tcp` or `udp`. -//! The rules' `name`s name them in messages. A proxy-only network denies everything else, and -//! names the proxy's port on host loopback and the scheme of its URL, `http`, `https`, or `null`: -//! -//! ```json -//! {"guestNetwork": "10.0.0.2/24", "egress": "deny", -//! "allow": {"name": "egress.allow", "rules": []}, -//! "deny": {"name": "egress.deny", "rules": []}, -//! "ingress": "deny", "hostLoopback": "deny", "hostLoopbackForwards": [], -//! "proxy": {"port": 3128, "scheme": "http"}} -//! ``` -//! -//! A plan of another version is not read: versions 1 and 2 held the shares as they were -//! requested, which the plan of each launch now resolves. - -mod form; - -use std::ffi::OsString; -use std::fmt; -use std::net::Ipv4Addr; -use std::path::{Path, PathBuf}; - -use serde::{Deserialize, Serialize}; - -use super::distro::{DistroBinding, DistroFormat}; -use super::mount::{self, BootShares}; -use super::{ - SANDBOX_COMMAND_LINE_MAX_SIZE, SandboxSettings, refused, reject_container_path, - reject_disk_path, require_file, validate_settings, -}; -use crate::error::Result; -use crate::host::filesystem::{ExportPlan, ReservedTargets}; -use crate::host::launch::{self, VmmLaunch}; -use crate::host::network::{self, NetworkPolicy}; -use crate::host::paths; -use crate::protocol::root::{ - DISTRO_MMIO_BASE, ImageFormat, LAYER_TOKEN, RAM_UPPER_TOKEN, RootImage, RootLayout, RootUpper, - SCRATCH_MMIO_BASE, SCRATCH_TOKEN, -}; -use crate::protocol::{HostMap, cmdline}; -use crate::uuid::Uuid; - -/// The version of the serde form of [`SandboxPlan`] that this build writes and reads. -pub const PLAN_VERSION: u32 = 3; - -/// How a launch delivers the shares of a plan to the guest. -#[derive(Clone, Copy, Debug, PartialEq, Eq)] -pub(crate) enum Delivery { - /// The managed lifecycle's: OpenVMM exports the shares at - /// [`GUEST_EXPORT`](crate::host::filesystem::GUEST_EXPORT), the kernel command line announces - /// the number of their mappings, and the host sends the mapping table once the guest agent is - /// ready. - MappingTable, -} - -/// The role of a block device of a plan, which fixes its MMIO base and its access. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)] -pub enum BlockRole { - /// The read-only image. - Distro, - /// The writable ext4 upper. - Scratch, -} - -impl BlockRole { - /// Returns the role's name, which OpenVMM's `--microvm-sandbox-block` takes: `distro` or - /// `scratch`. - pub const fn as_str(self) -> &'static str { - match self { - Self::Distro => "distro", - Self::Scratch => "scratch", - } - } - - /// Returns the MMIO base of the role's block device: [`DISTRO_MMIO_BASE`] or - /// [`SCRATCH_MMIO_BASE`]. - pub const fn mmio_base(self) -> u64 { - match self { - Self::Distro => DISTRO_MMIO_BASE, - Self::Scratch => SCRATCH_MMIO_BASE, - } - } - - /// Returns whether the role's block device is read-only, as the distro image is. - pub const fn is_read_only(self) -> bool { - matches!(self, Self::Distro) - } -} - -impl fmt::Display for BlockRole { - fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { - formatter.write_str(self.as_str()) - } -} - -/// The read-only `distro` block device of a plan: a raw host image, and its format and the -/// identity that the guest expects of it. -#[derive(Clone, Debug, PartialEq, Eq)] -pub struct SandboxDistro { - /// The image, with its components normalized. - path: PathBuf, - /// The image's format and expected identity. - image: RootImage, -} - -impl SandboxDistro { - /// Returns the image at `path`, spelled with its components alone, as [`Path::components`] - /// reads them, in the format and with the expected identity of `image`. - /// - /// The image is raw: OpenVMM attaches the file's bytes as they are. An image in a container - /// format, such as a VHD, is staged as a raw file first, under a name whose extension does not - /// make OpenVMM read a container, which [`SandboxPlan::validated`] checks. - pub fn new(path: &str, image: RootImage) -> Self { - Self::at(paths::normalized(Path::new(path)), image) - } - - /// Returns the image at `path`, which is normalized already. - pub(crate) fn at(path: PathBuf, image: RootImage) -> Self { - Self { path, image } - } - - /// Returns the image's path. - pub fn path(&self) -> &Path { - &self.path - } - - /// Returns the image's format and expected identity. - pub fn image(&self) -> RootImage { - self.image - } - - /// Returns how messages name the image: `distro EROFS layer` or `distro GPT image`. - fn description(&self) -> &'static str { - match self.image.format { - ImageFormat::Erofs => "distro EROFS layer", - ImageFormat::Gpt => "distro GPT image", - } - } -} - -/// Where a plan keeps the upper of its overlay: in an ext4 scratch image on the `scratch` block -/// device, or in RAM. -#[derive(Clone, Debug, PartialEq, Eq)] -pub struct SandboxUpper { - /// The scratch image, with its components normalized, or `None` for a RAM upper. - scratch: Option, -} - -impl SandboxUpper { - /// An upper in RAM, with no scratch block device. - pub const RAM: Self = Self { scratch: None }; - - /// Returns an upper on the preformatted ext4 scratch image at `path`, spelled with its - /// components alone, as [`Path::components`] reads them, whose whole device the guest mounts. - pub fn scratch(path: &str) -> Self { - Self::scratch_at(paths::normalized(Path::new(path))) - } - - /// Returns an upper on the scratch image at `path`, which is normalized already. - pub(crate) fn scratch_at(path: PathBuf) -> Self { - Self { - scratch: Some(path), - } - } - - /// Returns where the upper is, as the boot tokens name it. - pub fn kind(&self) -> RootUpper { - match self.scratch { - Some(_) => RootUpper::Scratch, - None => RootUpper::Ram, - } - } - - /// Returns the scratch image, or `None` for a RAM upper. - pub fn scratch_path(&self) -> Option<&Path> { - self.scratch.as_deref() - } -} - -/// The network of a plan: the guest's static IPv4 address on its network, and the policy that -/// OpenVMM enforces there, which [`network::openvmm_arguments`] lowers. -/// -/// Its serde form is the `network` of the persisted plan of the [module](self). -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] -#[serde(into = "form::NetworkForm", try_from = "form::NetworkForm")] -pub struct SandboxNetwork { - /// The guest's address and prefix, such as `10.0.0.2/24`. - guest_network: String, - /// The first address of the guest's network, where the guest reaches its gateway. - gateway: Ipv4Addr, - /// The policy. - policy: NetworkPolicy, -} - -impl SandboxNetwork { - /// Returns the network of a guest at `guest_network`, an IPv4 address and prefix such as - /// `10.0.0.2/24`, under `policy`. - /// - /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) for a guest network - /// that OpenVMM refuses, one for which [`crate::net::guest_gateway`] finds no gateway, and - /// then as [`network::openvmm_arguments`] fails for a policy that it cannot lower. - pub fn new(guest_network: &str, policy: NetworkPolicy) -> Result { - let gateway = guest_gateway(guest_network)?; - network::openvmm_arguments(Some(&policy), guest_network)?; - Ok(Self { - guest_network: guest_network.to_owned(), - gateway, - policy, - }) - } - - /// Returns the guest's address and prefix. - pub fn guest_network(&self) -> &str { - &self.guest_network - } - - /// Returns the address at which the guest reaches its gateway: the first address of its - /// network. - pub fn gateway(&self) -> Ipv4Addr { - self.gateway - } - - /// Returns the policy. - pub fn policy(&self) -> &NetworkPolicy { - &self.policy - } - - /// Returns the URL of the policy's proxy as the guest sees it, at its gateway: - /// `SCHEME://GATEWAY:PORT` with a scheme and `GATEWAY:PORT` without one, or `None` without a - /// proxy. How workloads learn it is the caller's choice. - pub fn proxy_url(&self) -> Option { - self.policy.proxy.map(|proxy| proxy.url(self.gateway)) - } - - /// Returns the OpenVMM arguments that attach the network device and enforce the policy, none - /// for a policy that lets nothing through, as [`network::openvmm_arguments`] renders them. - pub fn openvmm_arguments(&self) -> Result> { - network::openvmm_arguments(Some(&self.policy), &self.guest_network) - } - - /// Returns the OpenVMM arguments with which a template's capture attaches its NIC for the - /// network: the guest's address under the complete deny-all policy, whatever the network's - /// own policy, which each restore of the template replaces with its clone's; or none for a - /// policy that lets nothing through, for which [`openvmm_arguments`](Self::openvmm_arguments) - /// attaches no NIC either, so that a template of the network has none. - /// - /// Fails as [`openvmm_arguments`](Self::openvmm_arguments) does. - pub fn template_arguments(&self) -> Result> { - Ok(if self.openvmm_arguments()?.is_empty() { - Vec::new() - } else { - network::template_arguments(&self.guest_network) - }) - } -} - -/// Returns the gateway of `guest_network`, a static guest address that OpenVMM accepts, as -/// [`crate::net::guest_gateway`] finds it. -/// -/// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) for any other value, as -/// `guest network "10.0.0.1/24" must be an IPv4 address with a /1 to /30 prefix that is not its -/// network's network, broadcast, or gateway (first) address`. -pub(crate) fn guest_gateway(guest_network: &str) -> Result { - crate::net::guest_gateway(guest_network).ok_or_else(|| { - refused(format!( - "guest network {guest_network:?} must be an IPv4 address with a /1 to /30 prefix that is not its \ - network's network, broadcast, or gateway (first) address", - )) - }) -} - -/// The plan of a cold layered launch: everything that its OpenVMM arguments and kernel command -/// line depend on. -/// -/// A plan launches either NVX's sandbox profile, whose guest runs one workload with the plan's -/// [`SandboxSettings`], [`new`](Self::new), or an adapter's own guest, which runs its workloads -/// itself, [`adapter`](Self::adapter). -/// -/// Its serde form is the persisted plan of the [module](self). -#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] -#[serde(into = "form::PlanForm", try_from = "form::PlanForm")] -pub struct SandboxPlan { - /// The image. - distro: SandboxDistro, - /// The upper. - upper: SandboxUpper, - /// The launch settings of the sandbox profile, or `None` for an adapter's plan. - settings: Option, - /// The export plan of the live shares, if any, which an adapter's plan never has. - export: Option, - /// The network, or `None` for no network device. - network: Option, -} - -impl SandboxPlan { - /// Returns the plan of `distro` over `upper`, with `settings`, without live shares, and with - /// no network device; [`with_export`](Self::with_export) and - /// [`with_network`](Self::with_network) add them. - /// - /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported), checking the settings - /// in this order: the entrypoint, the arguments, the hostname, a non-root identity, positive - /// limits, limits that the guest's cgroups can apply, and at most [`u64::MAX`] bytes of memory - /// and 4,194,303 processes. - pub fn new( - distro: SandboxDistro, - upper: SandboxUpper, - settings: SandboxSettings, - ) -> Result { - validate_settings(&settings)?; - Ok(Self { - distro, - upper, - settings: Some(settings), - export: None, - network: None, - }) - } - - /// Returns the plan of `distro` over `upper` for an adapter's own guest, with no network - /// device, which [`with_network`](Self::with_network) adds. - /// - /// An adapter's plan renders the root and its devices alone. Its guest chooses the identity, - /// the program, and the limits of its workloads, so the plan names none of them to the VMM or - /// the guest, and the adapter attaches its live shares itself, with the arguments that - /// [`ExportPlan::openvmm_arguments`] renders, so the plan holds no export plan. Its kernel - /// command line takes the adapter's own `nvx_*` tokens beside those of the root, as - /// [`kernel_command_line`](Self::kernel_command_line) says. - pub fn adapter(distro: SandboxDistro, upper: SandboxUpper) -> Self { - Self { - distro, - upper, - settings: None, - export: None, - network: None, - } - } - - /// Returns the plan with `export`, the export plan of its live shares, if any, which - /// [`filesystem::plan`](crate::host::filesystem::plan) made for the container root. - /// - /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) for an export plan - /// that [`ExportPlan::check`] refuses for the container root, - /// [`ReservedTargets::CONTAINER_ROOT`], and for any export plan of an adapter's plan. - pub fn with_export(self, export: Option) -> Result { - if let Some(export) = &export { - if self.settings.is_none() { - return Err(refused( - "an adapter's plan holds no export plan: the adapter attaches its shares itself", - )); - } - export.check(ReservedTargets::CONTAINER_ROOT)?; - } - Ok(Self { export, ..self }) - } - - /// Returns the plan with `network`, or with no network device for `None`. - #[must_use] - pub fn with_network(mut self, network: Option) -> Self { - self.network = network; - self - } - - /// Returns the image. - pub fn distro(&self) -> &SandboxDistro { - &self.distro - } - - /// Returns the upper. - pub fn upper(&self) -> &SandboxUpper { - &self.upper - } - - /// Returns the launch settings of the sandbox profile, or `None` for an adapter's plan. - pub fn settings(&self) -> Option<&SandboxSettings> { - self.settings.as_ref() - } - - /// Returns the export plan of the live shares, if any. - pub fn export(&self) -> Option<&ExportPlan> { - self.export.as_ref() - } - - /// Returns the network, or `None` for no network device. - pub fn network(&self) -> Option<&SandboxNetwork> { - self.network.as_ref() - } - - /// Returns the URL of the network's proxy as the guest sees it, as - /// [`SandboxNetwork::proxy_url`] does, or `None` without one. - pub fn proxy_url(&self) -> Option { - self.network.as_ref().and_then(SandboxNetwork::proxy_url) - } - - /// Returns the layout of the root that the plan boots: its image and its upper. It names none - /// of the root's settings, so the guest assembles the root with its own defaults. - pub fn layout(&self) -> RootLayout { - RootLayout { - image: self.distro.image, - upper: self.upper.kind(), - overlay: None, - journal: None, - gpt_first_partition_bound: None, - max_memory_percent: None, - } - } - - /// Returns the block devices of the plan in role order: the image, then the scratch image of - /// an upper on scratch. - pub fn blocks(&self) -> Vec<(BlockRole, &Path)> { - let mut blocks = vec![(BlockRole::Distro, self.distro.path.as_path())]; - if let Some(scratch) = &self.upper.scratch { - blocks.push((BlockRole::Scratch, scratch.as_path())); - } - blocks - } - - /// Returns the plan with another image, over the same upper and with the same settings and - /// shares. - pub(crate) fn with_image(&self, image: RootImage) -> Self { - Self { - distro: SandboxDistro::at(self.distro.path.clone(), image), - ..self.clone() - } - } - - /// Checks the plan against the host: each image of its block devices must be a regular file - /// whose path and resolved path have no comma, semicolon, or unbalanced brackets, and whose - /// names do not end in `.vhd`, `.vhdx`, `.vmgs`, or `.iso`, with which OpenVMM would not - /// attach it as a raw image; and the export plan's - /// directories and files must still not overlap, by path or through a hard link or bind - /// mount, and the plan must pass [`ExportPlan::verify`], which checks again the host objects - /// that it recorded. - /// - /// Fails with [`ErrorKind::Unavailable`](crate::ErrorKind::Unavailable) for a missing image, as - /// `distro EROFS layer not found: PATH`, `distro GPT image not found: PATH`, or `ext4 scratch - /// image not found: PATH`, and otherwise as those checks do. - pub fn validated(&self) -> Result<()> { - self.validated_with(&mount::HostQueries::native()) - } - - /// Implements [`validated`](Self::validated) with the given host queries. - pub(crate) fn validated_with(&self, host: &mount::HostQueries<'_>) -> Result<()> { - for (role, path) in self.blocks() { - let description = match role { - BlockRole::Distro => self.distro.description(), - BlockRole::Scratch => "ext4 scratch image", - }; - require_file(path, description)?; - let shown = path.display().to_string(); - reject_disk_path(path, &shown)?; - reject_container_path(path, &shown)?; - // A provisioned sandbox records the resolved path, which must also fit OpenVMM's - // block device option and name a raw image. - let resolved = - (host.resolve)(path).map_err(|error| mount::os_error(&error, Some(path)))?; - if resolved.as_path() != path { - let shown = format!("{shown} resolves to {}", resolved.display()); - reject_disk_path(&resolved, &shown)?; - reject_container_path(&resolved, &shown)?; - } - } - match &self.export { - Some(export) => { - mount::check_distinct_export_roots(export, host)?; - export.verify() - } - None => Ok(()), - } - } - - /// Returns the shares as `delivery` delivers them; nothing without shares. - fn shares(&self, delivery: Delivery) -> Result { - let Some(export) = &self.export else { - return Ok(BootShares::default()); - }; - Ok(match delivery { - Delivery::MappingTable => BootShares { - arguments: export.openvmm_arguments(), - tokens: vec![export.kernel_token()], - fragment: mount::export_command_line_fragment(export), - }, - }) - } - - /// Returns the OpenVMM arguments of the plan as the managed lifecycle launches it: the - /// microVM machine, a sandbox block device for each block device of the plan in role order, - /// `ROLE:file:PATH` with `,ro` for the read-only image, the workload identity of the sandbox - /// profile, the arguments that export the shares at - /// [`GUEST_EXPORT`](crate::host::filesystem::GUEST_EXPORT), which - /// [`ExportPlan::openvmm_arguments`] renders, and from which the guest agent maps them as - /// [`mapping_table`](Self::mapping_table) orders, and those of the network, - /// [`SandboxNetwork::openvmm_arguments`]. An adapter's plan names no workload identity and - /// exports no shares. - /// - /// Fails as [`SandboxNetwork::openvmm_arguments`] does. - pub fn openvmm_arguments(&self) -> Result> { - self.render_arguments( - &|_, path| path.display().to_string(), - Delivery::MappingTable, - ) - } - - /// Renders the OpenVMM arguments of the plan, with `block` naming the file of each block - /// device, given its role and its path in the plan, and with the shares as `delivery` delivers - /// them. - pub(crate) fn render_arguments( - &self, - block: &dyn Fn(BlockRole, &Path) -> String, - delivery: Delivery, - ) -> Result> { - let mut arguments = vec!["--machine".to_owned(), "microvm".to_owned()]; - for (role, path) in self.blocks() { - let access = if role.is_read_only() { ",ro" } else { "" }; - arguments.push("--microvm-sandbox-block".to_owned()); - arguments.push(format!("{role}:file:{}{access}", block(role, path))); - } - if let Some(settings) = &self.settings { - let (uid, gid) = settings.workload_identity; - arguments.extend([ - "--microvm-workload-identity".to_owned(), - format!("{uid}:{gid}"), - ]); - } - arguments.extend(self.shares(delivery)?.arguments); - if let Some(network) = &self.network { - arguments.extend(network.openvmm_arguments()?); - } - Ok(arguments) - } - - /// Returns the plan's mapping table, the binds of its export plan in their order, which - /// [`ExportPlan::guest_table`] renders: what the managed lifecycle sends in `MAPS` requests - /// once the guest agent is ready, and before the launch is usable. A plan without shares has - /// none, and its launch maps nothing. - pub fn mapping_table(&self) -> Vec { - self.export - .as_ref() - .map_or_else(Vec::new, ExportPlan::guest_table) - } - - /// Returns the kernel command line of the plan as the managed lifecycle boots it: - /// `user_command_line`, trimmed, then `nvx_sandbox=1`, the boot tokens - /// of the root's image and upper, `nvx_entrypoint=`, `nvx_hostname=`, a `nvx_arg=` token for - /// each argument, `nvx_memory_max=` and `nvx_pids_max=` when set, and with shares the - /// `nvx_maps=` token that announces the number of mappings of - /// [`mapping_table`](Self::mapping_table). - /// - /// The user's command line may hold no NUL and no `nvx_*` or `virtfs_*` token, which the - /// profile and OpenVMM own. Parameters are read as Linux reads them, including double - /// quotes and the equivalence of hyphens and underscores in names, and as the guest's - /// whitespace-only bootstrap reads them. A `--` or unclosed quote - /// that hides the appended root tokens is refused. The command line, the bootstrap tokens that OpenVMM appends for - /// the shares, and a NUL must fit in [`SANDBOX_COMMAND_LINE_MAX_SIZE`] bytes. The guest must - /// then read the plan's root from the command line: one that [`RootLayout::parse`] refuses - /// fails too, as when the user's command line adds a virtio-mmio device at the scratch base - /// beside a RAM upper. Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported). - /// - /// For an adapter's plan, the command line is `user_command_line`, stripped, then the boot - /// tokens of the root alone. The adapter's command line may hold its own `nvx_*` tokens, but - /// no NUL, none of the root's tokens, `nvx_layer=`, `nvx_scratch=`, and `nvx_overlay_upper=`, - /// and no `virtfs_*` token, which OpenVMM appends for the shares that the adapter attaches. - /// The command line and a NUL must fit in [`SANDBOX_COMMAND_LINE_MAX_SIZE`] bytes, which - /// leaves the bootstrap tokens of the adapter's shares to the VMM's own check of the whole - /// command line; the guest must read the plan's root from it, as for the sandbox profile. - pub fn kernel_command_line(&self, user_command_line: &str) -> Result { - self.command_line(user_command_line, Delivery::MappingTable) - } - - /// Checks, as an adapter plans its launch, that the part of OpenVMM's command line that it - /// knows then leaves [`VmmLaunch::RESERVE`] of - /// [`PROCESS_COMMAND_LINE_LIMIT`](crate::host::launch::PROCESS_COMMAND_LINE_LIMIT) to the rest - /// of the launch, which - /// [`PreparedLaunch::adapter_arguments`](super::PreparedLaunch::adapter_arguments) renders: - /// the plan's [arguments](Self::openvmm_arguments), with the paths of its block devices, those - /// that export `export`, the export of the host paths that the adapter shares with its guest's - /// root, if any, and the plan's [kernel command line](Self::kernel_command_line) with - /// `user_command_line`. So planning refuses a plan whose network rules, or mapped, hidden, or - /// allowed paths, leave the rest of the launch too little, before the adapter stages or - /// launches anything, and the launch's complete check refuses one whose inputs are longer than - /// the reserve foresees. - /// - /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) for the plan of NVX's - /// sandbox profile, which its own start launches, as - /// [`openvmm_arguments`](Self::openvmm_arguments) and - /// [`kernel_command_line`](Self::kernel_command_line) fail, and if that part takes more. - pub fn check_adapter_launch( - &self, - user_command_line: &str, - export: Option<&ExportPlan>, - ) -> Result<()> { - self.require_adapter()?; - let mut arguments: Vec = self - .openvmm_arguments()? - .into_iter() - .map(OsString::from) - .collect(); - if let Some(export) = export { - arguments.extend(export.openvmm_arguments().into_iter().map(OsString::from)); - } - arguments.push("--cmdline".into()); - arguments.push(self.kernel_command_line(user_command_line)?.into()); - launch::fit_arguments(&arguments, VmmLaunch::RESERVE) - } - - /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) for the plan of NVX's - /// sandbox profile, which an adapter does not launch. - pub(super) fn require_adapter(&self) -> Result<()> { - match self.settings { - Some(_) => Err(refused( - "the plan of NVX's sandbox profile launches with the managed lifecycle, not as \ - an adapter's", - )), - None => Ok(()), - } - } - - /// Returns the kernel command line of the plan, with the shares as `delivery` delivers them. - pub(crate) fn command_line( - &self, - user_command_line: &str, - delivery: Delivery, - ) -> Result { - let user_command_line = user_command_line.trim(); - if user_command_line.contains('\0') { - return Err(refused("kernel command line contains an embedded NUL")); - } - let Some(settings) = &self.settings else { - return self.adapter_command_line(user_command_line); - }; - for parameter in requested_parameters(user_command_line) { - let name = String::from_utf8_lossy(parameter.name); - if parameter.name.starts_with(b"nvx_") || parameter.name.starts_with(b"nvx-") { - return Err(refused(format!("{name} is owned by the sandbox profile"))); - } - if parameter.name.starts_with(b"virtfs_") || parameter.name.starts_with(b"virtfs-") { - return Err(refused(format!( - "{name} is owned by the sandbox --mount option" - ))); - } - } - let mut tokens = vec![user_command_line.to_owned(), "nvx_sandbox=1".to_owned()]; - tokens.extend(self.layout().tokens()); - tokens.extend([ - format!("nvx_entrypoint={}", settings.entrypoint), - format!("nvx_hostname={}", settings.hostname), - ]); - tokens.extend( - settings - .args - .iter() - .map(|argument| format!("nvx_arg={argument}")), - ); - if let Some(memory_max) = &settings.memory_max { - tokens.push(format!("nvx_memory_max={memory_max}")); - } - if let Some(pids_max) = &settings.pids_max { - tokens.push(format!("nvx_pids_max={}", pids_max + 1)); - } - let BootShares { - tokens: shares, - fragment, - .. - } = self.shares(delivery)?; - tokens.extend(shares); - tokens.retain(|token| !token.is_empty()); - let command_line = tokens.join(" "); - // OpenVMM appends the shares' bootstrap tokens after this command line, so they consume - // the same x86 budget. - if command_line.len() + fragment.len() + 1 > SANDBOX_COMMAND_LINE_MAX_SIZE { - let advice = match delivery { - Delivery::MappingTable => "use shorter arguments and kernel parameters", - }; - return Err(refused(format!( - "sandbox kernel command line exceeds its 1024-byte x86 budget; {advice}" - ))); - } - let layout = RootLayout::parse(command_line.as_bytes()).map_err(|error| { - refused(format!( - "sandbox kernel command line conflicts with the boot tokens of its root: {error}" - )) - })?; - if layout != Some(self.layout()) || !self.root_is_visible(&command_line) { - return Err(refused( - "sandbox kernel command line hides the boot tokens of its root", - )); - } - Ok(command_line) - } - - /// Implements [`kernel_command_line`](Self::kernel_command_line) for an adapter's plan. - fn adapter_command_line(&self, user_command_line: &str) -> Result { - for parameter in requested_parameters(user_command_line) { - let name = String::from_utf8_lossy(parameter.name); - if is_root_parameter(parameter) { - return Err(refused(format!("{name} is owned by the plan's root"))); - } - if parameter.name.starts_with(b"virtfs_") || parameter.name.starts_with(b"virtfs-") { - return Err(refused(format!("{name} is owned by OpenVMM's shares"))); - } - } - let mut tokens = vec![user_command_line.to_owned()]; - tokens.extend(self.layout().tokens()); - tokens.retain(|token| !token.is_empty()); - let command_line = tokens.join(" "); - if command_line.len() + 1 > SANDBOX_COMMAND_LINE_MAX_SIZE { - return Err(refused( - "the kernel command line of an adapter's plan exceeds its 1024-byte x86 budget", - )); - } - let layout = RootLayout::parse(command_line.as_bytes()).map_err(|error| { - refused(format!( - "the kernel command line of an adapter's plan conflicts with the boot tokens of \ - its root: {error}" - )) - })?; - if layout != Some(self.layout()) || !self.root_is_visible(&command_line) { - return Err(refused( - "the kernel command line of an adapter's plan hides the boot tokens of its root", - )); - } - Ok(command_line) - } - - /// Checks that Linux can still see every generated root parameter after the user's text. - fn root_is_visible(&self, command_line: &str) -> bool { - cmdline::parameters(command_line.as_bytes()) - .filter(|¶meter| is_root_parameter(parameter)) - .count() - == self.layout().tokens().len() - } -} - -/// Returns both readers' views: Linux honors quotes, but the guest's shell bootstrap splits -/// expanded command-line text on IFS whitespace without interpreting its quote characters, as -/// [`cmdline::init_tokens`] splits it. -fn requested_parameters(command_line: &str) -> impl Iterator> { - let guest = cmdline::init_tokens(command_line.as_bytes()).map(|token| cmdline::Parameter { - name: token.name, - value: token.value, - }); - cmdline::parameters(command_line.as_bytes()).chain(guest) -} - -/// Returns whether `parameter` names one of the boot tokens of a plan's root, which -/// [`RootLayout::tokens`] renders: `nvx_layer`, `nvx_scratch`, or `nvx_overlay_upper`. -fn is_root_parameter(parameter: cmdline::Parameter<'_>) -> bool { - [LAYER_TOKEN, SCRATCH_TOKEN, RAM_UPPER_TOKEN] - .iter() - .any(|token| { - token - .split('=') - .next() - .is_some_and(|name| parameter.is_named(name)) - }) -} - -/// Returns the image that a distro binding records: the staged format, and the identity that the -/// guest expects of the layer. -/// -/// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) for an identity that is -/// not a UUID in its canonical text. -pub(crate) fn bound_image(binding: &DistroBinding) -> Result { - image_of(binding.format(), binding.identity()) -} - -/// Returns the image of a layer of `format` whose binding names `identity`, as [`bound_image`] -/// does. -pub(crate) fn image_of(format: DistroFormat, identity: &str) -> Result { - let identity = Uuid::parse(identity).ok_or_else(|| { - refused(format!( - "the distro binding names the identity {identity:?}, which is not a UUID in its canonical text", - )) - })?; - Ok(match format { - DistroFormat::Erofs => RootImage::erofs(Some(identity)), - DistroFormat::Gpt => RootImage::gpt(Some(identity)), - }) -} - -#[cfg(test)] -mod tests; diff --git a/nvx-rs/src/host/sandbox/plan/form.rs b/nvx-rs/src/host/sandbox/plan/form.rs deleted file mode 100644 index ba33cab56..000000000 --- a/nvx-rs/src/host/sandbox/plan/form.rs +++ /dev/null @@ -1,546 +0,0 @@ -//! The serde form of a [`SandboxPlan`], in which callers persist plans, as the [plan -//! module](super) describes it. - -use serde::{Deserialize, Serialize}; - -use super::{PLAN_VERSION, SandboxDistro, SandboxNetwork, SandboxPlan, SandboxUpper}; -use crate::host::filesystem::ExportPlan; -use crate::host::network::{ - Access, Destination, EgressRule, EgressRules, ForwardProtocol, HostLoopback, LoopbackForward, - NetworkPolicy, NetworkProxy, PortSelector, Protocol, ProxyScheme, -}; -use crate::host::sandbox::{SandboxSettings, parse_decimal}; -use crate::protocol::root::{ImageFormat, RootImage}; -use crate::uuid::Uuid; - -/// The serde form of a [`SandboxPlan`]. -#[derive(Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -pub(super) struct PlanForm { - /// [`PLAN_VERSION`]. - version: u32, - /// The image. - distro: DistroForm, - /// The upper. - upper: UpperForm, - /// The launch settings of the sandbox profile, or `None` for an adapter's plan, which the form - /// requires as `null`. - #[serde(deserialize_with = "Option::deserialize")] - settings: Option, - /// The export plan of the live shares, or `None`, which the form requires as `null`. - #[serde(deserialize_with = "Option::deserialize")] - export: Option, - /// The network, or `None`, which the form requires as `null`. - #[serde(deserialize_with = "Option::deserialize")] - network: Option, -} - -/// The serde form of a [`SandboxDistro`]. -#[derive(Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -struct DistroForm { - /// The image's path. - path: String, - /// The image's format. - format: FormatForm, - /// The canonical text of the identity that the guest expects of the image, or `None`, which - /// the form requires as `null`. - #[serde(deserialize_with = "Option::deserialize")] - identity: Option, -} - -/// The serde form of an [`ImageFormat`]. -#[derive(Clone, Copy, Serialize, Deserialize)] -#[serde(rename_all = "lowercase")] -enum FormatForm { - /// [`ImageFormat::Erofs`]. - Erofs, - /// [`ImageFormat::Gpt`]. - Gpt, -} - -/// The serde form of a [`SandboxUpper`], tagged with its kind. -#[derive(Serialize, Deserialize)] -#[serde(tag = "kind", rename_all = "lowercase", deny_unknown_fields)] -enum UpperForm { - /// An upper on a scratch image. - Scratch { - /// The scratch image's path. - path: String, - }, - /// An upper in RAM, which has no other field. - Ram {}, -} - -/// The serde form of [`SandboxSettings`]. -#[derive(Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -struct SettingsForm { - /// The workload's program. - entrypoint: String, - /// The workload's arguments. - args: Vec, - /// The workload's hostname. - hostname: String, - /// The workload's UID and GID. - workload_identity: IdentityForm, - /// The decimal text of the memory limit, or `None`, which the form requires as `null`. - #[serde(deserialize_with = "Option::deserialize")] - memory_max: Option, - /// The decimal text of the process limit, or `None`, which the form requires as `null`. - #[serde(deserialize_with = "Option::deserialize")] - pids_max: Option, -} - -/// The serde form of a workload identity. -#[derive(Serialize, Deserialize)] -#[serde(deny_unknown_fields)] -struct IdentityForm { - /// The UID. - uid: u32, - /// The GID. - gid: u32, -} - -/// The serde form of a [`SandboxNetwork`] and its [`NetworkPolicy`]. -#[derive(Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -pub(super) struct NetworkForm { - /// The guest's address and prefix. - guest_network: String, - /// The default for guest-initiated connections. - egress: AccessForm, - /// The rules that permit destinations. - allow: RulesForm, - /// The rules that block destinations. - deny: RulesForm, - /// The default for connections that clients initiate to the guest. - ingress: AccessForm, - /// The access between the guest and host loopback. - host_loopback: HostLoopbackForm, - /// The ports on host loopback that are forwarded to guest ports. - host_loopback_forwards: Vec, - /// The proxy of a proxy-only network, or `None`, which the form requires as `null`. - #[serde(deserialize_with = "Option::deserialize")] - proxy: Option, -} - -/// The serde form of an [`Access`]. -#[derive(Clone, Copy, Serialize, Deserialize)] -#[serde(rename_all = "lowercase")] -enum AccessForm { - /// [`Access::Allow`]. - Allow, - /// [`Access::Deny`]. - Deny, -} - -/// The serde form of a [`HostLoopback`]. -#[derive(Clone, Copy, Serialize, Deserialize)] -#[serde(rename_all = "lowercase")] -enum HostLoopbackForm { - /// [`HostLoopback::Deny`]. - Deny, - /// [`HostLoopback::Outbound`]. - Outbound, - /// [`HostLoopback::Allow`]. - Allow, -} - -/// The serde form of [`EgressRules`]. -#[derive(Serialize, Deserialize)] -#[serde(deny_unknown_fields)] -struct RulesForm { - /// The name of the rules in messages. - name: String, - /// The rules. - rules: Vec, -} - -/// The serde form of an [`EgressRule`]. -#[derive(Serialize, Deserialize)] -#[serde(deny_unknown_fields)] -struct RuleForm { - /// The destinations, of which none matches every destination. - to: Vec, - /// The protocols and ports, of which none matches every one. - ports: Vec, -} - -/// The serde form of a [`Destination`]. -#[derive(Serialize, Deserialize)] -#[serde(deny_unknown_fields)] -struct DestinationForm { - /// The destination address or network. - cidr: String, - /// The sub-networks that the destination excludes. - except: Vec, -} - -/// The serde form of a [`PortSelector`]. -#[derive(Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -struct PortForm { - /// The protocol. - protocol: ProtocolForm, - /// The port, or the first port of a range, or `None`, which the form requires as `null`. - #[serde(deserialize_with = "Option::deserialize")] - port: Option, - /// The last port of a range, or `None`, which the form requires as `null`. - #[serde(deserialize_with = "Option::deserialize")] - end_port: Option, -} - -/// The serde form of a [`Protocol`]. -#[derive(Clone, Copy, Serialize, Deserialize)] -#[serde(rename_all = "lowercase")] -enum ProtocolForm { - /// [`Protocol::Tcp`]. - Tcp, - /// [`Protocol::Udp`]. - Udp, - /// [`Protocol::Icmp`]. - Icmp, - /// [`Protocol::Any`]. - Any, -} - -/// The serde form of a [`LoopbackForward`]. -#[derive(Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -struct ForwardForm { - /// The protocol. - protocol: ForwardProtocolForm, - /// The port on host loopback. - host_port: u16, - /// The guest port. - guest_port: u16, -} - -/// The serde form of a [`ForwardProtocol`]. -#[derive(Clone, Copy, Serialize, Deserialize)] -#[serde(rename_all = "lowercase")] -enum ForwardProtocolForm { - /// [`ForwardProtocol::Tcp`]. - Tcp, - /// [`ForwardProtocol::Udp`]. - Udp, -} - -/// The serde form of a [`NetworkProxy`]. -#[derive(Serialize, Deserialize)] -#[serde(deny_unknown_fields)] -struct ProxyForm { - /// The proxy's port on host loopback. - port: u16, - /// The scheme of the proxy's URL, which [`ProxyScheme::from_name`] reads, or `None`, which - /// the form requires as `null`. - #[serde(deserialize_with = "Option::deserialize")] - scheme: Option, -} - -impl From for PlanForm { - fn from(plan: SandboxPlan) -> Self { - let SandboxPlan { - distro, - upper, - settings, - export, - network, - } = plan; - let RootImage { format, identity } = distro.image; - // A plan's paths are text, which `SandboxDistro::new` and `SandboxUpper::scratch` take. - Self { - version: PLAN_VERSION, - distro: DistroForm { - path: distro.path.to_string_lossy().into_owned(), - format: match format { - ImageFormat::Erofs => FormatForm::Erofs, - ImageFormat::Gpt => FormatForm::Gpt, - }, - identity: identity.map(|identity| identity.to_string()), - }, - upper: match upper.scratch { - Some(path) => UpperForm::Scratch { - path: path.to_string_lossy().into_owned(), - }, - None => UpperForm::Ram {}, - }, - settings: settings.map(SettingsForm::from), - export, - network: network.map(NetworkForm::from), - } - } -} - -impl TryFrom for SandboxPlan { - type Error = String; - - /// Reads a plan, which must be of [`PLAN_VERSION`], hold an identity in its canonical text and - /// limits in decimal text, and pass the checks of [`SandboxPlan::new`], or be an adapter's - /// plan, [`SandboxPlan::adapter`], and then pass those of [`SandboxPlan::with_export`] and - /// [`SandboxNetwork::new`]. - fn try_from(form: PlanForm) -> Result { - if form.version != PLAN_VERSION { - return Err(format!( - "the sandbox plan has version {}, which this build does not read", - form.version - )); - } - let DistroForm { - path, - format, - identity, - } = form.distro; - let identity = identity - .map(|text| { - Uuid::parse(&text).ok_or_else(|| { - format!( - "the distro identity of the sandbox plan, {text:?}, is not a UUID in its \ - canonical text", - ) - }) - }) - .transpose()?; - let format = match format { - FormatForm::Erofs => ImageFormat::Erofs, - FormatForm::Gpt => ImageFormat::Gpt, - }; - let distro = SandboxDistro::new(&path, RootImage { format, identity }); - let upper = match form.upper { - UpperForm::Scratch { path } => SandboxUpper::scratch(&path), - UpperForm::Ram {} => SandboxUpper::RAM, - }; - let plan = match form.settings { - Some(settings) => { - let settings = SandboxSettings::try_from(settings)?; - Self::new(distro, upper, settings) - } - None => Ok(Self::adapter(distro, upper)), - } - .and_then(|plan| plan.with_export(form.export)) - .map_err(|error| error.to_string())?; - let network = form.network.map(SandboxNetwork::try_from).transpose()?; - Ok(plan.with_network(network)) - } -} - -impl From for NetworkForm { - fn from(network: SandboxNetwork) -> Self { - let NetworkPolicy { - egress, - allow, - deny, - ingress, - host_loopback, - host_loopback_forwards, - proxy, - } = network.policy; - let access = |access| match access { - Access::Allow => AccessForm::Allow, - Access::Deny => AccessForm::Deny, - }; - Self { - guest_network: network.guest_network, - egress: access(egress), - allow: RulesForm::from(allow), - deny: RulesForm::from(deny), - ingress: access(ingress), - host_loopback: match host_loopback { - HostLoopback::Deny => HostLoopbackForm::Deny, - HostLoopback::Outbound => HostLoopbackForm::Outbound, - HostLoopback::Allow => HostLoopbackForm::Allow, - }, - host_loopback_forwards: host_loopback_forwards - .into_iter() - .map(|forward| ForwardForm { - protocol: match forward.protocol { - ForwardProtocol::Tcp => ForwardProtocolForm::Tcp, - ForwardProtocol::Udp => ForwardProtocolForm::Udp, - }, - host_port: forward.host_port, - guest_port: forward.guest_port, - }) - .collect(), - proxy: proxy.map(|proxy| ProxyForm { - port: proxy.port, - scheme: proxy.scheme.map(|scheme| scheme.as_str().to_owned()), - }), - } - } -} - -impl TryFrom for SandboxNetwork { - type Error = String; - - /// Reads a network whose proxy's scheme, if any, [`ProxyScheme::from_name`] reads, and which - /// passes the checks of [`SandboxNetwork::new`]. - fn try_from(form: NetworkForm) -> Result { - let message = |error: crate::Error| error.to_string(); - let access = |access| match access { - AccessForm::Allow => Access::Allow, - AccessForm::Deny => Access::Deny, - }; - let proxy = form - .proxy - .map(|proxy| { - let scheme = proxy - .scheme - .as_deref() - .map(ProxyScheme::from_name) - .transpose() - .map_err(message)?; - Ok::<_, String>(NetworkProxy { - port: proxy.port, - scheme, - }) - }) - .transpose()?; - let policy = NetworkPolicy { - egress: access(form.egress), - allow: EgressRules::from(form.allow), - deny: EgressRules::from(form.deny), - ingress: access(form.ingress), - host_loopback: match form.host_loopback { - HostLoopbackForm::Deny => HostLoopback::Deny, - HostLoopbackForm::Outbound => HostLoopback::Outbound, - HostLoopbackForm::Allow => HostLoopback::Allow, - }, - host_loopback_forwards: form - .host_loopback_forwards - .into_iter() - .map(|forward| LoopbackForward { - protocol: match forward.protocol { - ForwardProtocolForm::Tcp => ForwardProtocol::Tcp, - ForwardProtocolForm::Udp => ForwardProtocol::Udp, - }, - host_port: forward.host_port, - guest_port: forward.guest_port, - }) - .collect(), - proxy, - }; - Self::new(&form.guest_network, policy).map_err(message) - } -} - -impl From for RulesForm { - fn from(rules: EgressRules) -> Self { - Self { - name: rules.name, - rules: rules - .rules - .into_iter() - .map(|rule| RuleForm { - to: rule - .to - .into_iter() - .map(|destination| DestinationForm { - cidr: destination.cidr, - except: destination.except, - }) - .collect(), - ports: rule - .ports - .into_iter() - .map(|port| PortForm { - protocol: match port.protocol { - Protocol::Tcp => ProtocolForm::Tcp, - Protocol::Udp => ProtocolForm::Udp, - Protocol::Icmp => ProtocolForm::Icmp, - Protocol::Any => ProtocolForm::Any, - }, - port: port.port, - end_port: port.end_port, - }) - .collect(), - }) - .collect(), - } - } -} - -impl From for EgressRules { - fn from(form: RulesForm) -> Self { - Self { - name: form.name, - rules: form - .rules - .into_iter() - .map(|rule| EgressRule { - to: rule - .to - .into_iter() - .map(|destination| Destination { - cidr: destination.cidr, - except: destination.except, - }) - .collect(), - ports: rule - .ports - .into_iter() - .map(|port| PortSelector { - protocol: match port.protocol { - ProtocolForm::Tcp => Protocol::Tcp, - ProtocolForm::Udp => Protocol::Udp, - ProtocolForm::Icmp => Protocol::Icmp, - ProtocolForm::Any => Protocol::Any, - }, - port: port.port, - end_port: port.end_port, - }) - .collect(), - }) - .collect(), - } - } -} - -impl From for SettingsForm { - fn from(settings: SandboxSettings) -> Self { - let (uid, gid) = settings.workload_identity; - Self { - entrypoint: settings.entrypoint, - args: settings.args, - hostname: settings.hostname, - workload_identity: IdentityForm { uid, gid }, - memory_max: settings.memory_max.map(|limit| limit.to_string()), - pids_max: settings.pids_max.map(|limit| limit.to_string()), - } - } -} - -impl TryFrom for SandboxSettings { - type Error = String; - - /// Reads settings whose limits are positive integers in decimal text, without a sign or - /// leading zeros. - fn try_from(form: SettingsForm) -> Result { - fn limit( - name: &str, - text: Option, - ) -> Result, String> { - text.map(|text| { - (!text.starts_with('0')) - .then(|| parse_decimal(&text)) - .flatten() - .ok_or_else(|| { - format!( - "the {name} of the sandbox plan, {text:?}, is not a positive integer in \ - decimal text", - ) - }) - }) - .transpose() - } - let IdentityForm { uid, gid } = form.workload_identity; - Ok(Self { - entrypoint: form.entrypoint, - args: form.args, - hostname: form.hostname, - workload_identity: (uid, gid), - memory_max: limit("memoryMax", form.memory_max)?, - pids_max: limit("pidsMax", form.pids_max)?, - }) - } -} diff --git a/nvx-rs/src/host/sandbox/plan/tests.rs b/nvx-rs/src/host/sandbox/plan/tests.rs deleted file mode 100644 index e7d729146..000000000 --- a/nvx-rs/src/host/sandbox/plan/tests.rs +++ /dev/null @@ -1,2074 +0,0 @@ -//! Tests of the typed plan: every combination of format and upper, the guest's reading of the -//! command lines that plans render, their budget, their checks, and their persisted form. - -use std::fs; -use std::path::Path; - -use serde_json::{Value, json}; - -use super::*; -use crate::ErrorKind; -use crate::host::filesystem::{Bind, FileIdentity}; -use crate::host::network::{ - Access, Destination, EgressRule, ForwardProtocol, HostLoopback, LoopbackForward, NetworkProxy, - Protocol, ProxyScheme, -}; -use crate::host::sandbox::distro::DistroBinding; -use crate::host::sandbox::mount::export_command_line_fragment; -use crate::host::sandbox::tests::{deny, exported, host, host_text, native}; -use crate::host::sandbox::{LayerRole, MountAccess, MountKind, SandboxLaunch, SandboxLayer}; - -const EROFS_UUID: &str = "11111111-1111-1111-1111-111111111111"; -const GPT_GUID: &str = "fedcba98-7654-3210-fedc-ba9876543210"; - -fn uuid(text: &str) -> Uuid { - Uuid::parse(text).unwrap() -} - -fn message(result: Result) -> String { - result.unwrap_err().to_string() -} - -/// Returns the images of every format, with and without an identity. -fn images() -> [RootImage; 4] { - [ - RootImage::erofs(Some(uuid(EROFS_UUID))), - RootImage::erofs(None), - RootImage::gpt(Some(uuid(GPT_GUID))), - RootImage::gpt(None), - ] -} - -/// Returns the uppers: on `scratch.ext4`, and in RAM. -fn uppers() -> [SandboxUpper; 2] { - [SandboxUpper::scratch("scratch.ext4"), SandboxUpper::RAM] -} - -fn plan(image: RootImage, upper: SandboxUpper, settings: SandboxSettings) -> SandboxPlan { - exporting(image, upper, settings, None) -} - -fn exporting( - image: RootImage, - upper: SandboxUpper, - settings: SandboxSettings, - export: Option, -) -> SandboxPlan { - SandboxPlan::new( - SandboxDistro::new("images/distro.img", image), - upper, - settings, - ) - .and_then(|plan| plan.with_export(export)) - .unwrap() -} - -/// Returns the export plan of a read-write directory at `/workspace`, and of a read-only one at -/// `/srv/cache` unless `one`. -fn shares(one: bool) -> ExportPlan { - let workspace = ( - "/workspace", - "/host/work", - MountAccess::ReadWrite, - MountKind::Directory, - ); - let cache = ( - "/srv/cache", - "/host/cache", - MountAccess::ReadOnly, - MountKind::Directory, - ); - if one { - exported(&[workspace]) - } else { - exported(&[workspace, cache]) - } -} - -/// Returns a read-write bind of `/host/work/out`, inside the first child of [`shares`], at -/// `/workspace/out`, which pins `identity`. -fn nested(identity: Option) -> Bind { - Bind { - child: 0, - source: "out".to_owned(), - target: "/workspace/out".to_owned(), - read_only: false, - identity, - } -} - -/// Returns the bootstrap tokens that OpenVMM appends for the shares of `plan` in the managed -/// lifecycle, which exports them for the guest agent's mapping table. -fn fragment(plan: &SandboxPlan) -> String { - plan.export() - .map(export_command_line_fragment) - .unwrap_or_default() -} - -/// A rendering of a plan's kernel command line, and of the bootstrap tokens that OpenVMM appends -/// for its export plan. -type Rendering = ( - fn(&SandboxPlan, &str) -> Result, - fn(&ExportPlan) -> String, -); - -/// Returns the renderings of the managed lifecycle, which announces a mapping table, by name. -fn renderings() -> [(&'static str, Rendering); 1] { - [( - "managed", - ( - SandboxPlan::kernel_command_line, - export_command_line_fragment, - ), - )] -} - -#[test] -fn every_format_and_upper_plans_and_renders() { - let settings = SandboxSettings { - entrypoint: "/bin/workload".to_owned(), - args: vec!["--serve".to_owned()], - hostname: "example".to_owned(), - workload_identity: (1000, 1001), - memory_max: Some(268_435_456), - pids_max: Some(64), - }; - let distro = "nvx_layer=distro,0xd0003000"; - let cases = [ - ( - 0, - 0, - format!("{distro},erofs:{EROFS_UUID} nvx_scratch=0xd0006000,ext4"), - ), - (1, 0, format!("{distro},erofs nvx_scratch=0xd0006000,ext4")), - ( - 2, - 0, - format!("{distro},gpt:{GPT_GUID} nvx_scratch=0xd0006000,ext4"), - ), - (3, 0, format!("{distro},gpt nvx_scratch=0xd0006000,ext4")), - ( - 0, - 1, - format!("{distro},erofs:{EROFS_UUID} nvx_overlay_upper=ramfs"), - ), - (1, 1, format!("{distro},erofs nvx_overlay_upper=ramfs")), - ( - 2, - 1, - format!("{distro},gpt:{GPT_GUID} nvx_overlay_upper=ramfs"), - ), - (3, 1, format!("{distro},gpt nvx_overlay_upper=ramfs")), - ]; - for (image, upper, tokens) in cases { - let (image, upper) = (images()[image], uppers()[upper].clone()); - let plan = plan(image, upper.clone(), settings.clone()); - // The plan names none of the root's settings. - assert_eq!( - plan.layout(), - RootLayout { - image, - upper: upper.kind(), - overlay: None, - journal: None, - gpt_first_partition_bound: None, - max_memory_percent: None, - } - ); - assert_eq!( - plan.kernel_command_line("quiet").unwrap(), - format!( - "quiet nvx_sandbox=1 {tokens} nvx_entrypoint=/bin/workload nvx_hostname=example \ - nvx_arg=--serve nvx_memory_max=268435456 nvx_pids_max=65" - ) - ); - let distro_block = paths::normalized(Path::new("images/distro.img")); - let mut expected = vec![ - "--machine".to_owned(), - "microvm".to_owned(), - "--microvm-sandbox-block".to_owned(), - format!("distro:file:{},ro", distro_block.display()), - ]; - let mut blocks = vec![(BlockRole::Distro, distro_block.as_path())]; - // A RAM upper has no scratch block device. - if let Some(scratch) = upper.scratch_path() { - expected.extend([ - "--microvm-sandbox-block".to_owned(), - format!("scratch:file:{}", scratch.display()), - ]); - blocks.push((BlockRole::Scratch, scratch)); - } - expected.extend([ - "--microvm-workload-identity".to_owned(), - "1000:1001".to_owned(), - ]); - assert_eq!(plan.openvmm_arguments().unwrap(), expected); - assert_eq!(plan.blocks(), blocks); - } -} - -#[test] -fn the_guest_reads_the_root_of_every_plan_from_its_command_line() { - let settings = SandboxSettings { - args: vec!["-c".to_owned(), "true".to_owned()], - ..SandboxSettings::default() - }; - for image in images() { - for upper in uppers() { - let plan = exporting(image, upper.clone(), settings.clone(), Some(shares(false))); - let command_line = plan.kernel_command_line("console=hvc0 quiet").unwrap(); - assert_eq!( - RootLayout::parse(command_line.as_bytes()), - Ok(Some(plan.layout())), - "{command_line}" - ); - // OpenVMM appends its own tokens, the discovery of the plan's block devices among - // them, which the guest reads alike. - let mut effective = - format!("{command_line} nr_cpus=1 virtio_mmio.device=0x1000@0xd0003000:19"); - if upper.scratch_path().is_some() { - effective.push_str(" virtio_mmio.device=0x1000@0xd0006000:22"); - } - effective.push_str(&fragment(&plan)); - assert_eq!( - RootLayout::parse(effective.as_bytes()), - Ok(Some(plan.layout())), - "{effective}" - ); - } - } -} - -#[test] -fn plans_refuse_the_command_lines_that_the_guest_refuses() { - // The user's command line may add a virtio-mmio device, which the guest reads as the kernel - // does: one at the scratch base conflicts with a RAM upper. - let conflict = "sandbox kernel command line conflicts with the boot tokens of its root: a RAM \ - upper takes no scratch block device, but the command line adds a virtio-mmio \ - device at 0xd0006000"; - for image in images() { - let plan = plan(image, SandboxUpper::RAM, SandboxSettings::default()); - for user in [ - "virtio_mmio.device=0x1000@0xd0006000:22", - "quiet virtio-mmio.device=4K@0xd0006000:11:3", - "\"virtio_mmio.device=0x1000@0xd0006000:22\"", - "virtio_mmio.device=\"0x1000@ 0xd0006000:11\"", - "virtio_mmio.device=0x1000@3489685504:11", - ] { - let error = plan.kernel_command_line(user).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported, "{user}"); - assert_eq!(error.to_string(), conflict, "{user}"); - } - for user in [ - // The driver refuses these values, so the kernel adds no device from them. - "virtio_mmio.device=0x1000@0xd0006000:0", - "virtio_mmio.device=0x1000@0xd0006000", - // A device elsewhere, and a token that the kernel reads inside another value. - "virtio_mmio.device=0x1000@0xd0007000:23", - "x=\"y virtio_mmio.device=0x1000@0xd0006000:22\"", - ] { - let command_line = plan.kernel_command_line(user).unwrap(); - assert_eq!( - RootLayout::parse(command_line.as_bytes()), - Ok(Some(plan.layout())), - "{user}" - ); - } - } -} - -#[test] -fn the_longest_boot_tokens_fit_the_command_line_budget_to_the_byte() { - // The tokens of an identified EROFS image over scratch are the longest: they take the most - // of the budget, which every other plan keeps the rest of. - let longest = plan(images()[0], uppers()[0].clone(), SandboxSettings::default()); - let longest_base = longest.kernel_command_line("").unwrap().len(); - for (lifecycle, (render, appended)) in renderings() { - for export in [None, Some(shares(true)), Some(shares(false))] { - for image in images() { - for upper in uppers() { - let plan = exporting(image, upper, SandboxSettings::default(), export.clone()); - let base = render(&plan, "").unwrap(); - let fragment = plan.export().map(appended).unwrap_or_default(); - // The user's command line, a space, the plan's tokens, OpenVMM's share - // tokens, and a NUL fill the budget exactly. - let fitting = - "x".repeat(SANDBOX_COMMAND_LINE_MAX_SIZE - base.len() - fragment.len() - 2); - let command_line = render(&plan, &fitting).unwrap(); - assert_eq!( - command_line.len() + fragment.len() + 1, - SANDBOX_COMMAND_LINE_MAX_SIZE, - "{lifecycle}" - ); - assert_eq!( - RootLayout::parse(command_line.as_bytes()), - Ok(Some(plan.layout())) - ); - assert!( - message(render(&plan, &format!("{fitting}x"))).contains("1024-byte"), - "{lifecycle}" - ); - if plan.export().is_none() { - assert!(base.len() <= longest_base, "{base}"); - } - } - } - } - } -} - -#[test] -fn the_managed_lifecycle_exports_the_shares_and_announces_their_mapping_table() { - let export = shares(false); - let bare = plan(images()[0], uppers()[0].clone(), SandboxSettings::default()); - let plan = exporting( - images()[0], - uppers()[0].clone(), - SandboxSettings::default(), - Some(export.clone()), - ); - // OpenVMM exports the shares where the guest agent maps them from, after the block devices - // and the workload identity. - let blocks = bare.openvmm_arguments().unwrap(); - let arguments = plan.openvmm_arguments().unwrap(); - assert_eq!(arguments[..blocks.len()], blocks[..]); - assert_eq!(arguments[blocks.len()..], export.openvmm_arguments()[..]); - assert_eq!( - arguments[blocks.len()..][..2], - ["--mount-aggregate", crate::host::filesystem::GUEST_EXPORT] - ); - // The kernel command line announces the number of mappings, not their targets, and OpenVMM - // appends the export's tokens, read-write since a child is. - let base = bare.kernel_command_line("quiet").unwrap(); - assert_eq!( - plan.kernel_command_line("quiet").unwrap(), - format!("{base} nvx_maps=2") - ); - assert_eq!( - fragment(&plan), - " virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=1" - ); - let read_only = exporting( - images()[0], - uppers()[0].clone(), - SandboxSettings::default(), - Some(exported(&[( - "/srv/cache", - "/host/cache", - MountAccess::ReadOnly, - MountKind::Directory, - )])), - ); - assert!(fragment(&read_only).contains(" virtfs_mode=ro ")); - // The mapping table is the export plan's binds, in order: parents first. - assert_eq!(plan.mapping_table(), export.guest_table()); - assert_eq!( - plan.mapping_table() - .iter() - .map(|map| map.target.as_str()) - .collect::>(), - ["/workspace", "/srv/cache"] - ); - // Without shares, nothing is mapped. - assert!(bare.mapping_table().is_empty()); - assert!(!base.contains("nvx_maps=")); -} - -#[test] -fn plans_check_their_settings_as_launches_do() { - let checked = |change: fn(&mut SandboxSettings)| { - let mut settings = SandboxSettings::default(); - change(&mut settings); - SandboxPlan::new( - SandboxDistro::new("d.img", RootImage::gpt(None)), - SandboxUpper::RAM, - settings, - ) - }; - for (change, expected) in [ - ( - (|settings: &mut SandboxSettings| settings.entrypoint = "bin/sh".to_owned()) - as fn(&mut SandboxSettings), - "sandbox entrypoint must be an absolute path without whitespace, control characters, or quotes", - ), - ( - |settings| settings.args = vec![String::new()], - "sandbox arguments must be nonempty and contain no whitespace, control characters, or quotes", - ), - ( - |settings| settings.entrypoint = "/opt/app\"".to_owned(), - "sandbox entrypoint must be an absolute path without whitespace, control characters, or quotes", - ), - ( - |settings| settings.args = vec!["say\"hi".to_owned()], - "sandbox arguments must be nonempty and contain no whitespace, control characters, or quotes", - ), - ( - |settings| settings.entrypoint = "/bin/\u{e0}--".to_owned(), - "sandbox entrypoint must be an absolute path without whitespace, control characters, or quotes", - ), - ( - |settings| settings.args = vec!["voil\u{e0}--".to_owned()], - "sandbox arguments must be nonempty and contain no whitespace, control characters, or quotes", - ), - ( - |settings| settings.hostname = "Invalid".to_owned(), - "sandbox hostname must be a lowercase RFC 1123 label up to 63 characters", - ), - ( - |settings| settings.workload_identity = (0, 1), - "sandbox workload identity must be a non-root UID:GID", - ), - ( - |settings| settings.memory_max = Some(0), - "--memory-max must be positive", - ), - ] { - let error = checked(change).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported); - assert_eq!(error.to_string(), expected); - } - checked(|_| {}).unwrap(); -} - -#[test] -fn plans_check_their_export_plan_for_the_container_root() { - let checked = |change: fn(&mut ExportPlan)| { - let mut export = shares(false); - change(&mut export); - SandboxPlan::new( - SandboxDistro::new("d.img", RootImage::gpt(None)), - SandboxUpper::RAM, - SandboxSettings::default(), - ) - .and_then(|plan| plan.with_export(Some(export))) - }; - checked(|_| {}).unwrap(); - // A hidden root shows the paths that it shows again. - checked(|export| { - deny(export, 0, &["/host/work"]); - export.children[0].allowed = vec![host("/host/work/public")]; - }) - .unwrap(); - // A child that the guest may write all of pins the denied and mapped objects inside it. - checked(|export| { - deny(export, 0, &["/host/work/secrets"]); - export.binds.push(nested(Some(FileIdentity { - device: 1, - index: 9, - }))); - }) - .unwrap(); - // Writes narrowed to `out` move only what lies strictly inside it, not `out` itself. - checked(|export| { - export.children[0].write = vec![host("/host/work/out")]; - deny(export, 0, &["/host/work/secrets", "/host/work/out/secret"]); - export.binds.push(nested(None)); - }) - .unwrap(); - // A child whose root is mapped read-only writes the paths that it maps read-write. - checked(|export| { - export.binds[0].read_only = true; - export.children[0].write = vec![host("/host/work/out")]; - export.binds.push(nested(None)); - }) - .unwrap(); - // A source holds the host's names joined with `/`: a Windows host reads a backslash in it as a - // separator, which another host keeps in the name. - let backslash = checked(|export| { - export.binds.push(Bind { - source: r"src\host".to_owned(), - target: r"/workspace/src\host".to_owned(), - ..nested(Some(FileIdentity { - device: 1, - index: 9, - })) - }); - }); - if cfg!(windows) { - let error = backslash.unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported); - assert!( - error.to_string().contains( - r#"it maps the path "src\\host" of child 0, which the host reads as other names"# - ), - "{error}" - ); - } else { - backslash.unwrap(); - } - for (change, expected) in [ - ( - (|export: &mut ExportPlan| export.binds[0].target = "/etc".to_owned()) - as fn(&mut ExportPlan), - "its guest target /etc would cover the guest's /etc", - ), - ( - |export| export.binds[0].target = "/run".to_owned(), - "its guest target /run would cover the guest's /run", - ), - ( - |export| export.binds[1].target = "/proc/self".to_owned(), - "its guest target /proc/self would cover the guest's /proc", - ), - ( - |export| export.binds[1].target = "/etc/resolv.conf".to_owned(), - "its guest target /etc/resolv.conf would cover the guest's /etc/resolv.conf", - ), - ( - |export| export.binds[0].target = "workspace".to_owned(), - "its guest target \"workspace\" is not absolute", - ), - ( - |export| export.binds[1].target = "/workspace/cache".to_owned(), - "it maps /workspace/cache inside /workspace, where it would hide part of that path", - ), - ( - |export| export.binds.swap(0, 1), - "it maps /workspace after /srv/cache, out of order or twice", - ), - ( - |export| export.binds[1].child = 7, - "it maps no path at the root of child 1", - ), - ( - |export| export.binds[0].source = "sub".to_owned(), - "it maps no path at the root of child 0", - ), - ( - |export| export.children[0].denied = vec![host("/host/work/secrets")], - "child 0 has 1 denied paths but 0 identities", - ), - ( - |export| deny(export, 0, &["/host/cache/secrets"]), - "child 0 has the policy path", - ), - ( - |export| export.children[1].write = vec![host("/host/cache/out")], - "child 1 is read-only but has writable paths", - ), - ( - |export| export.binds[1].read_only = false, - "child 1 is read-only but maps a path read-write", - ), - ( - // A read-only bind of a directory whose writes OpenVMM narrows to a path inside it. - |export| { - export.children[0].write = vec![host("/host/work/out")]; - export.binds[0].read_only = true; - }, - "child 0 is writable but maps no path read-write", - ), - ( - |export| { - export.children[0].denied = (0..129) - .map(|index| host(&format!("/host/work/secret-{index}"))) - .collect(); - export.children[0].denied_identities = vec![None; 129]; - }, - "child 0: OpenVMM can hide at most 128 paths in one exported directory", - ), - ( - |export| { - export.children[0].write = vec![host(&format!("/host/work/{}", "x".repeat(4097)))]; - }, - "child 0: OpenVMM cannot make writable", - ), - ( - |export| { - export.children[0].write = (0..5) - .map(|index| host(&format!("/host/work/{index}{}", "x".repeat(3999)))) - .collect(); - }, - "exceed its 16384-byte limit", - ), - ( - |export| { - export.binds.insert( - 0, - Bind { - child: 0, - source: "sub\0file".to_owned(), - target: "/data".to_owned(), - read_only: false, - identity: None, - }, - ); - }, - "which contains a NUL character", - ), - ( - |export| { - export.binds.insert( - 0, - Bind { - child: 0, - source: "x".repeat(4090), - target: "/data".to_owned(), - read_only: false, - identity: None, - }, - ); - }, - "which is too long for a guest path", - ), - ( - |export| export.children[1].root = host("/host/work/cache"), - "overlaps child 0", - ), - ( - |export| deny(export, 0, &["/host/work/secrets", "/host/work/secrets"]), - "lists the hidden path", - ), - ( - |export| { - deny( - export, - 0, - &["/host/work/secrets", "/host/work/secrets/keys"], - ) - }, - "which it hides already", - ), - ( - |export| export.children[0].allowed = vec![host("/host/work/public")], - "again, but hides no path that contains it", - ), - ( - |export| { - deny(export, 0, &["/host/work/secrets"]); - export.children[0].allowed = vec![host("/host/work/secrets")]; - }, - "it both hides and shows", - ), - ( - |export| { - deny(export, 0, &["/host/work/secrets"]); - export.children[0].allowed = vec![ - host("/host/work/secrets/public"), - host("/host/work/secrets/public/docs"), - ]; - }, - "which it shows already", - ), - ( - // OpenVMM makes no hidden path writable. - |export| { - deny(export, 0, &["/host/work/secrets"]); - export.children[0].write = vec![host("/host/work/secrets/public")]; - }, - "writable, but", - ), - ( - |export| export.children[0].write = vec![host("/host/work")], - "writable through a writable path", - ), - ( - |export| { - export.children[0].write = - vec![host("/host/work/out"), host("/host/work/out/logs")]; - }, - "which it makes writable already", - ), - ( - |export| deny(export, 0, &["/host/work"]), - "/workspace, but child 0 hides it", - ), - ( - // A path that a hidden directory holds is hidden too, unless it shows a path again. - |export| { - deny(export, 0, &["/host/work"]); - export.children[0].allowed = vec![host("/host/work/public")]; - export.binds.push(Bind { - child: 0, - source: "private".to_owned(), - target: "/workspace/private".to_owned(), - read_only: false, - identity: None, - }); - }, - "/workspace/private, but child 0 hides it", - ), - ( - |export| export.children.clear(), - "it exports 0 directories and files", - ), - ( - // A read-only child whose denied path no process argument could carry. - |export| deny(export, 1, &["/host/cache/secret\0suffix"]), - r#"secret\0suffix", which contains a NUL character"#, - ), - ( - |export| export.children[1].root = host("/host/ca\0che"), - "which contains a NUL character", - ), - ( - // A path that lies inside a child by its text, but not on the host. - |export| deny(export, 0, &["/host/work/../cache/secret"]), - "which has a `..` component", - ), - ( - |export| export.children[1].root = host("/host/work/../cache"), - "which has a `..` component", - ), - ( - // The guest would write `other`, which no bind maps read-write, but not `out`. - |export| { - export.binds[0].read_only = true; - export.children[0].write = vec![host("/host/work/other")]; - export.binds.push(nested(None)); - }, - "child 0 lets the guest write other paths than those that it maps read-write", - ), - ( - // Without write paths, the guest would write all of the child. - |export| { - export.binds[0].read_only = true; - export.binds.push(nested(None)); - }, - "child 0 lets the guest write other paths than those that it maps read-write", - ), - ( - |export| { - export.children[0].write = vec![host("/host/work/other")]; - export.binds.push(nested(None)); - }, - "which it maps read-write, stays read-only", - ), - ( - |export| { - export.children[0].denied = vec![host("/host/work/secrets")]; - export.children[0].denied_identities = vec![None]; - }, - "child 0 does not pin the denied path", - ), - ( - |export| { - export.children[0].write = vec![host("/host/work/out")]; - export.children[0].denied = vec![host("/host/work/out/secret")]; - export.children[0].denied_identities = vec![None]; - }, - "child 0 does not pin the denied path", - ), - ( - |export| export.binds.push(nested(None)), - "it does not pin the object that it maps at /workspace/out, which a workload could \ - move", - ), - ] { - let error = checked(change).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported); - let message = error.to_string(); - assert!( - message.starts_with("the recorded export plan is invalid: ") - && message.contains(expected), - "{message} lacks {expected}" - ); - } - // On Windows, `plan` resolves paths on local drives, without the verbatim prefix. - if cfg!(windows) { - for change in [ - (|export: &mut ExportPlan| export.children[1].root = r"\\?\C:\host\cache".into()) - as fn(&mut ExportPlan), - |export| export.children[1].root = r"\\server\share\cache".into(), - ] { - let message = checked(change).unwrap_err().to_string(); - assert!( - message.ends_with("which is not a path on a local drive"), - "{message}" - ); - } - } -} - -#[test] -fn plans_require_properly_nested_disk_path_brackets() { - let directory = tempfile::tempdir().unwrap(); - let path = |name: &str| native(directory.path().join(name).to_str().unwrap()); - let plain = path("plain.img"); - fs::write(&plain, b"image").unwrap(); - for (name, accepted) in [ - ("image[1.img", false), - ("image]1.img", false), - ("image][1.img", false), - ("image[[1].img", false), - ("image[1]].img", false), - ("image.img", true), - ("image[].img", true), - ("image[1].img", true), - ("image[[1][2]].img", true), - ("image[1][2].img", true), - ] { - let image = path(name); - fs::write(&image, b"image").unwrap(); - for plan in [ - SandboxPlan::adapter( - SandboxDistro::new(&image, RootImage::gpt(None)), - SandboxUpper::RAM, - ), - SandboxPlan::adapter( - SandboxDistro::new(&plain, RootImage::gpt(None)), - SandboxUpper::scratch(&image), - ), - ] { - let result = plan.validated(); - if accepted { - result.unwrap(); - } else { - let error = result.unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported); - assert_eq!( - error.to_string(), - format!( - "disk paths containing commas, semicolons, or unbalanced brackets are \ - unsupported: {image}" - ) - ); - } - } - } -} - -#[test] -fn plans_check_the_images_of_their_block_devices_against_the_host() { - let directory = tempfile::tempdir().unwrap(); - let path = |name: &str| native(directory.path().join(name).to_str().unwrap()); - let plan = |name: &str, image: RootImage, upper: SandboxUpper| { - SandboxPlan::new( - SandboxDistro::new(&path(name), image), - upper, - SandboxSettings::default(), - ) - .unwrap() - }; - for (image, description) in [ - (RootImage::erofs(None), "distro EROFS layer"), - (RootImage::gpt(Some(uuid(GPT_GUID))), "distro GPT image"), - ] { - let missing = plan("missing.img", image, SandboxUpper::RAM); - let error = missing.validated().unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unavailable); - assert_eq!( - error.to_string(), - format!("{description} not found: {}", path("missing.img")) - ); - } - fs::write(path("distro.img"), b"image").unwrap(); - // A RAM upper has no scratch image to check. - plan("distro.img", RootImage::gpt(None), SandboxUpper::RAM) - .validated() - .unwrap(); - let scratch = plan( - "distro.img", - RootImage::gpt(None), - SandboxUpper::scratch(&path("scratch.ext4")), - ); - assert_eq!( - message(scratch.validated()), - format!("ext4 scratch image not found: {}", path("scratch.ext4")) - ); - fs::write(path("scratch.ext4"), b"scratch").unwrap(); - scratch.validated().unwrap(); - let delimited = path("distro,ro.img"); - fs::write(&delimited, b"image").unwrap(); - assert_eq!( - message(plan("distro,ro.img", RootImage::gpt(None), SandboxUpper::RAM).validated()), - format!( - "disk paths containing commas, semicolons, or unbalanced brackets are unsupported: \ - {delimited}" - ) - ); - // OpenVMM would read a container from a file with one of these extensions. - for extension in ["vhd", "vhdx", "vmgs", "iso"] { - let refusal = |name: &str| { - format!( - "disk paths ending in .{extension} are unsupported, since OpenVMM would not \ - attach them as raw images: {}", - path(name) - ) - }; - let image = format!("distro.{extension}"); - fs::write(path(&image), b"image").unwrap(); - assert_eq!( - message(plan(&image, RootImage::gpt(None), SandboxUpper::RAM).validated()), - refusal(&image) - ); - let scratch = format!("scratch.{extension}"); - fs::write(path(&scratch), b"scratch").unwrap(); - let upper = SandboxUpper::scratch(&path(&scratch)); - assert_eq!( - message(plan("distro.img", RootImage::erofs(None), upper).validated()), - refusal(&scratch) - ); - } - // OpenVMM reads the extension of the name alone, case-sensitively, and none of a name that - // starts with its only dot. - fs::create_dir(path("images.vhd")).unwrap(); - for name in ["raw.VHD", ".vhd", "distro.vhd.img", "images.vhd/distro.img"] { - fs::write(path(name), b"image").unwrap(); - plan(name, RootImage::gpt(None), SandboxUpper::RAM) - .validated() - .unwrap(); - } - let resolved = |name: &str| { - paths::resolve(Path::new(&path(name))) - .unwrap() - .to_str() - .unwrap() - .to_owned() - }; - // Windows resolves a name to the case of the file's own, which `provision` records. - #[cfg(windows)] - { - assert_eq!( - message(plan("distro.VHD", RootImage::gpt(None), SandboxUpper::RAM).validated()), - format!( - "disk paths ending in .vhd are unsupported, since OpenVMM would not attach them \ - as raw images: {} resolves to {}", - path("distro.VHD"), - resolved("distro.VHD") - ) - ); - } - // A provisioned sandbox gives OpenVMM the paths to which its images' paths resolve, so a link - // to a container's name is refused too, as the distro image or the scratch image. - let linked = |link: &str, target: &str| { - let (link, target) = (path(link), path(target)); - move |candidate: &Path| { - if candidate.as_os_str() == link.as_str() { - Ok(PathBuf::from(&target)) - } else { - paths::resolve(candidate) - } - } - }; - let refusal = |extension: &str, link: &str, target: &str| { - format!( - "disk paths ending in .{extension} are unsupported, since OpenVMM would not attach \ - them as raw images: {} resolves to {}", - path(link), - path(target) - ) - }; - let resolve = linked("distro.img", "image.vhd"); - let host = mount::HostQueries { - resolve: &resolve, - ..mount::HostQueries::native() - }; - let raw = plan("distro.img", RootImage::gpt(None), SandboxUpper::RAM); - assert_eq!( - message(raw.validated_with(&host)), - refusal("vhd", "distro.img", "image.vhd") - ); - let resolve = linked("scratch.ext4", "upper.vhdx"); - let host = mount::HostQueries { - resolve: &resolve, - ..mount::HostQueries::native() - }; - let upper = SandboxUpper::scratch(&path("scratch.ext4")); - let scratch = plan("distro.img", RootImage::erofs(None), upper); - assert_eq!( - message(scratch.validated_with(&host)), - refusal("vhdx", "scratch.ext4", "upper.vhdx") - ); - scratch.validated().unwrap(); - for (directory, accepted) in [ - ("images,raw", false), - ("images;raw", false), - ("images[1", false), - ("images]1", false), - ("images][1", false), - ("images[[1][2]]", true), - ] { - for (link, candidate) in [("distro.img", &raw), ("scratch.ext4", &scratch)] { - let target = format!("{directory}/{link}"); - let resolve = linked(link, &target); - let host = mount::HostQueries { - resolve: &resolve, - ..mount::HostQueries::native() - }; - let result = candidate.validated_with(&host); - if accepted { - result.unwrap(); - } else { - let error = result.unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported); - assert_eq!( - error.to_string(), - format!( - "disk paths containing commas, semicolons, or unbalanced brackets are \ - unsupported: {} resolves to {}", - path(link), - path(&target) - ) - ); - } - } - } - #[cfg(unix)] - { - std::os::unix::fs::symlink(path("distro.vhd"), path("link.img")).unwrap(); - assert_eq!( - message(plan("link.img", RootImage::gpt(None), SandboxUpper::RAM).validated()), - format!( - "disk paths ending in .vhd are unsupported, since OpenVMM would not attach them \ - as raw images: {} resolves to {}", - path("link.img"), - resolved("link.img") - ) - ); - } -} - -#[test] -fn launches_plan_their_distro_layer_over_their_scratch_image() { - let launch = SandboxLaunch::new( - vec![SandboxLayer::new( - LayerRole::Distro, - "distro.erofs", - EROFS_UUID, - )], - SandboxUpper::scratch("scratch.ext4"), - SandboxSettings::default(), - ) - .unwrap(); - let plan = launch.plan(); - assert_eq!(plan.distro().path(), Path::new("distro.erofs")); - assert_eq!( - plan.distro().image(), - RootImage::erofs(Some(uuid(EROFS_UUID))) - ); - assert_eq!(plan.upper().kind(), RootUpper::Scratch); - assert_eq!(plan.upper().scratch_path(), Some(Path::new("scratch.ext4"))); - assert_eq!(launch.scratch(), Some(Path::new("scratch.ext4"))); - assert_eq!(plan.settings(), Some(&SandboxSettings::default())); - // The guest takes the UUID in its canonical text, which `--layer` writes, and nothing else. - for invalid in ["", "not-a-uuid", "11111111-1111-1111-1111-11111111111A"] { - let error = SandboxLaunch::new( - vec![SandboxLayer::new( - LayerRole::Distro, - "distro.erofs", - invalid, - )], - SandboxUpper::scratch("scratch.ext4"), - SandboxSettings { - entrypoint: "relative".to_owned(), - ..SandboxSettings::default() - }, - ) - .unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported); - assert_eq!( - error.to_string(), - format!("distro layer UUID is invalid: {invalid:?}") - ); - } -} - -#[test] -fn launches_plan_their_distro_layer_over_ram() { - let launch = SandboxLaunch::new( - vec![SandboxLayer::new( - LayerRole::Distro, - "distro.erofs", - EROFS_UUID, - )], - SandboxUpper::RAM, - SandboxSettings::default(), - ) - .unwrap(); - assert_eq!(launch.upper(), &SandboxUpper::RAM); - assert_eq!(launch.scratch(), None); - let plan = launch.plan(); - assert_eq!(plan.upper().kind(), RootUpper::Ram); - // The distro layer is the one block device, and the guest keeps the upper in RAM. - assert_eq!( - plan.blocks(), - [(BlockRole::Distro, Path::new("distro.erofs"))] - ); - let command_line = plan.kernel_command_line("").unwrap(); - let tokens: Vec<&str> = command_line.split(' ').collect(); - assert!(tokens.contains(&RAM_UPPER_TOKEN), "{command_line}"); - assert!( - !tokens.iter().any(|token| token.starts_with(SCRATCH_TOKEN)), - "{command_line}" - ); -} - -#[test] -fn launches_plan_the_image_that_the_binding_of_their_distro_layer_records() { - let directory = tempfile::tempdir().unwrap(); - let layer = directory.path().join("distro.erofs"); - fs::write(&layer, &b"layer"[..]).unwrap(); - let launch = SandboxLaunch::new( - vec![SandboxLayer::new( - LayerRole::Distro, - layer.to_str().unwrap(), - EROFS_UUID, - )], - SandboxUpper::scratch("scratch.ext4"), - SandboxSettings::default(), - ) - .unwrap(); - let binding = DistroBinding::verify(launch.distro()).unwrap(); - let bound = launch.bound_plan(&binding).unwrap(); - assert_eq!( - bound.distro().image(), - RootImage::erofs(Some(uuid(EROFS_UUID))) - ); - assert_eq!(&bound, launch.plan()); - - // The binding, not the `--layer` value, names the identity. - let mut recorded = serde_json::to_value(&binding).unwrap(); - recorded["identity"] = json!(GPT_GUID); - let other: DistroBinding = serde_json::from_value(recorded.clone()).unwrap(); - assert_eq!( - launch.bound_plan(&other).unwrap().distro().image(), - RootImage::erofs(Some(uuid(GPT_GUID))) - ); - recorded["identity"] = json!("11111111-1111-1111-1111-11111111111A"); - let invalid: DistroBinding = serde_json::from_value(recorded).unwrap(); - assert_eq!( - message(launch.bound_plan(&invalid)), - "the distro binding names the identity \"11111111-1111-1111-1111-11111111111A\", which is \ - not a UUID in its canonical text" - ); -} - -/// Returns a plan of every kind of setting, persisted as it records them. -fn persisted() -> (SandboxPlan, Value) { - // The largest limits that the guest's cgroups apply, of which the memory limit exceeds the - // integers that a JSON number holds exactly. - let settings = SandboxSettings { - entrypoint: "/bin/workload".to_owned(), - args: vec!["--serve".to_owned()], - hostname: "example".to_owned(), - workload_identity: (1000, 1001), - memory_max: Some(u64::MAX), - pids_max: Some(4_194_303), - }; - let mut export = exported(&[ - ( - "/workspace", - "/host/work", - MountAccess::ReadWrite, - MountKind::Directory, - ), - ( - "/etc/app/settings.json", - "/host/settings.json", - MountAccess::ReadOnly, - MountKind::File, - ), - ]); - export.children[0].write = vec![host("/host/work/out")]; - deny(&mut export, 0, &["/host/work/secrets"]); - export.children[0].allowed = vec![host("/host/work/secrets/public")]; - let plan = SandboxPlan::new( - SandboxDistro::new("/images/distro.img", RootImage::gpt(Some(uuid(GPT_GUID)))), - SandboxUpper::RAM, - settings, - ) - .and_then(|plan| plan.with_export(Some(export))) - .unwrap() - .with_network(Some(direct_network())); - let path = native; - let form = json!({ - "version": 3, - "distro": {"path": path("/images/distro.img"), "format": "gpt", "identity": GPT_GUID}, - "upper": {"kind": "ram"}, - "settings": { - "entrypoint": "/bin/workload", - "args": ["--serve"], - "hostname": "example", - "workloadIdentity": {"uid": 1000, "gid": 1001}, - "memoryMax": "18446744073709551615", - "pidsMax": "4194303", - }, - "export": { - "children": [ - { - "root": host_text("/host/work"), - "writable": true, - "denied": [host_text("/host/work/secrets")], - "deniedIdentities": [null], - "allowed": [host_text("/host/work/secrets/public")], - "write": [host_text("/host/work/out")], - }, - { - "root": host_text("/host/settings.json"), - "file": true, - "writable": false, - "denied": [], - "deniedIdentities": [], - "allowed": [], - "write": [], - }, - ], - "binds": [ - {"child": 0, "source": "", "target": "/workspace", "readOnly": false}, - {"child": 1, "source": "", "target": "/etc/app/settings.json", "readOnly": true}, - ], - }, - "network": direct_network_form(), - }); - (plan, form) -} - -/// Returns a network with every field of a policy that allows direct egress. -fn direct_network() -> SandboxNetwork { - let mut policy = NetworkPolicy::new(Access::Allow) - .with_allow( - EgressRule { - to: vec![Destination { - cidr: "192.0.2.0/24".to_owned(), - except: vec!["192.0.2.128/25".to_owned()], - }], - ports: Vec::new(), - } - .on_port_range(Protocol::Any, 8000, 8010), - ) - .with_allow(EgressRule::default().on_port(Protocol::Tcp, 443)) - .with_deny(EgressRule::to("198.51.100.0/24").on_port(Protocol::Udp, 53)) - .with_deny(EgressRule::to("2001:db8::/32").on_protocol(Protocol::Icmp)); - policy.ingress = Access::Allow; - policy.host_loopback = HostLoopback::Allow; - policy.host_loopback_forwards = vec![LoopbackForward { - protocol: ForwardProtocol::Tcp, - host_port: 18080, - guest_port: 8080, - }]; - SandboxNetwork::new("10.0.0.2/24", policy).unwrap() -} - -/// Returns the persisted form of [`direct_network`]. -fn direct_network_form() -> Value { - json!({ - "guestNetwork": "10.0.0.2/24", - "egress": "allow", - "allow": { - "name": "egress.allow", - "rules": [ - { - "to": [{"cidr": "192.0.2.0/24", "except": ["192.0.2.128/25"]}], - "ports": [{"protocol": "any", "port": 8000, "endPort": 8010}], - }, - {"to": [], "ports": [{"protocol": "tcp", "port": 443, "endPort": null}]}, - ], - }, - "deny": { - "name": "egress.deny", - "rules": [ - { - "to": [{"cidr": "198.51.100.0/24", "except": []}], - "ports": [{"protocol": "udp", "port": 53, "endPort": null}], - }, - { - "to": [{"cidr": "2001:db8::/32", "except": []}], - "ports": [{"protocol": "icmp", "port": null, "endPort": null}], - }, - ], - }, - "ingress": "allow", - "hostLoopback": "allow", - "hostLoopbackForwards": [{"protocol": "tcp", "hostPort": 18080, "guestPort": 8080}], - "proxy": null, - }) -} - -/// Returns the network of a guest at `10.0.0.2/24` whose one way out is a proxy at port 3128 of -/// host loopback, with `scheme`. -fn proxy_network(scheme: Option) -> SandboxNetwork { - let proxy = NetworkProxy { port: 3128, scheme }; - SandboxNetwork::new("10.0.0.2/24", NetworkPolicy::proxy_only(proxy)).unwrap() -} - -#[test] -fn plans_persist_in_their_versioned_form() { - let (plan, form) = persisted(); - assert_eq!(serde_json::to_value(&plan).unwrap(), form); - assert_eq!(serde_json::from_value::(form).unwrap(), plan); - // Every combination round-trips, and renders alike after it does. - for image in images() { - for upper in uppers() { - for network in [None, Some(direct_network()), Some(proxy_network(None))] { - let plan = plan_with(image, upper.clone()).with_network(network); - let read: SandboxPlan = - serde_json::from_str(&serde_json::to_string(&plan).unwrap()).unwrap(); - assert_eq!(read, plan); - assert_eq!( - read.kernel_command_line("").unwrap(), - plan.kernel_command_line("").unwrap() - ); - assert_eq!( - read.openvmm_arguments().unwrap(), - plan.openvmm_arguments().unwrap() - ); - assert_eq!(read.proxy_url(), plan.proxy_url()); - } - } - } - let scratch = serde_json::to_value(plan_with(images()[1], uppers()[0].clone())).unwrap(); - assert_eq!( - scratch["upper"], - json!({"kind": "scratch", "path": native("scratch.ext4")}) - ); - assert_eq!(scratch["distro"]["identity"], Value::Null); - assert_eq!(scratch["export"], Value::Null); - assert_eq!(scratch["network"], Value::Null); - let proxy = serde_json::to_value(proxy_network(Some(ProxyScheme::Https))).unwrap(); - assert_eq!( - proxy, - json!({ - "guestNetwork": "10.0.0.2/24", - "egress": "deny", - "allow": {"name": "egress.allow", "rules": []}, - "deny": {"name": "egress.deny", "rules": []}, - "ingress": "deny", - "hostLoopback": "deny", - "hostLoopbackForwards": [], - "proxy": {"port": 3128, "scheme": "https"}, - }) - ); - assert_eq!(PLAN_VERSION, 3); -} - -fn plan_with(image: RootImage, upper: SandboxUpper) -> SandboxPlan { - plan(image, upper, SandboxSettings::default()) -} - -/// Returns the plan of an adapter's own guest of `image` over `upper`. -fn adapter(image: RootImage, upper: SandboxUpper) -> SandboxPlan { - SandboxPlan::adapter(SandboxDistro::new("images/distro.img", image), upper) -} - -#[test] -fn adapter_plans_render_their_root_and_devices_alone() { - // The adapter's own tokens, `nvx_*` ones among them, come first, then the root's. - let user = "console=hvc0 nvx_map=c291cmNl,dGFyZ2V0,ro nvx_control_tty=hvc2 nvx_sandbox=1"; - for image in images() { - for upper in uppers() { - let plan = adapter(image, upper.clone()); - assert_eq!(plan.settings(), None); - assert_eq!(plan.export(), None); - let tokens = plan.layout().tokens().join(" "); - assert_eq!( - plan.kernel_command_line(user).unwrap(), - format!("{user} {tokens}") - ); - assert_eq!(plan.kernel_command_line(" \t").unwrap(), tokens); - // No workload identity and no shares: the block devices alone. - let mut expected = vec![ - "--machine".to_owned(), - "microvm".to_owned(), - "--microvm-sandbox-block".to_owned(), - format!("distro:file:{},ro", native("images/distro.img")), - ]; - if let Some(scratch) = upper.scratch_path() { - expected.extend([ - "--microvm-sandbox-block".to_owned(), - format!("scratch:file:{}", scratch.display()), - ]); - } - assert_eq!(plan.openvmm_arguments().unwrap(), expected); - // Then the network's, as for the sandbox profile. - let network = proxy_network(None); - expected.extend(network.openvmm_arguments().unwrap()); - let plan = plan.with_network(Some(network)); - assert_eq!(plan.openvmm_arguments().unwrap(), expected); - assert_eq!( - plan.proxy_url().as_deref(), - Some("10.0.0.1:3128"), - "{plan:?}" - ); - } - } -} - -#[test] -fn adapter_plans_refuse_what_their_root_and_openvmm_own() { - let plan = adapter(RootImage::gpt(Some(uuid(GPT_GUID))), SandboxUpper::RAM); - for (user, name, owner) in [ - ( - "nvx_layer=distro,0xd0003000,gpt", - "nvx_layer", - "the plan's root", - ), - ("nvx_layer", "nvx_layer", "the plan's root"), - ( - "quiet nvx_overlay_upper=ramfs", - "nvx_overlay_upper", - "the plan's root", - ), - ( - "nvx_scratch=0xd0006000,ext4", - "nvx_scratch", - "the plan's root", - ), - ("virtfs_tag=microvm", "virtfs_tag", "OpenVMM's shares"), - ] { - let error = plan.kernel_command_line(user).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported, "{user}"); - assert_eq!( - error.to_string(), - format!("{name} is owned by {owner}"), - "{user}" - ); - } - assert_eq!( - message(plan.kernel_command_line("quiet\0")), - "kernel command line contains an embedded NUL" - ); - // The guest must read the plan's root from the command line. - assert_eq!( - message(plan.kernel_command_line("virtio_mmio.device=0x1000@0xd0006000:22")), - "the kernel command line of an adapter's plan conflicts with the boot tokens of its root: \ - a RAM upper takes no scratch block device, but the command line adds a virtio-mmio \ - device at 0xd0006000" - ); - // The adapter's command line, a space, the root's tokens, and a NUL fill the budget exactly. - for image in images() { - for upper in uppers() { - let plan = adapter(image, upper); - let base = plan.kernel_command_line("").unwrap(); - let fitting = "x".repeat(SANDBOX_COMMAND_LINE_MAX_SIZE - base.len() - 2); - let command_line = plan.kernel_command_line(&fitting).unwrap(); - assert_eq!(command_line.len() + 1, SANDBOX_COMMAND_LINE_MAX_SIZE); - assert_eq!( - message(plan.kernel_command_line(&format!("{fitting}x"))), - "the kernel command line of an adapter's plan exceeds its 1024-byte x86 budget" - ); - } - } - // The adapter attaches its shares itself. - assert_eq!( - message(plan.clone().with_export(Some(shares(true)))), - "an adapter's plan holds no export plan: the adapter attaches its shares itself" - ); - assert_eq!(plan.clone().with_export(None).unwrap(), plan); -} - -#[test] -fn adapter_plans_refuse_the_settings_of_their_root_that_they_do_not_name() { - // An adapter's plan names none of its root's settings, and the guest reads a setting from the - // command line as the plan reads it back, so the plan refuses a setting that the adapter's - // command line names. - let ram = adapter(images()[3], SandboxUpper::RAM); - let scratch = adapter(images()[1], SandboxUpper::scratch("scratch.ext4")); - let hides = "the kernel command line of an adapter's plan hides the boot tokens of its root"; - for (plan, user, refusal) in [ - (&ram, "nvx_overlay_mode=untrusted-lowers", hides), - (&ram, "nvx_ram_upper_percent=5", hides), - (&ram, "x=\"y nvx_ram_upper_percent=5 z\"", hides), - (&ram, "nvx_first_partition_bound=4096", hides), - (&scratch, "nvx_scratch_journal=forbidden", hides), - ( - &scratch, - "nvx_ram_upper_percent=0", - "the kernel command line of an adapter's plan conflicts with the boot tokens of its \ - root: the nvx_ram_upper_percent kernel parameter must be a decimal number from 1 to \ - 90, without a sign or leading zeros, not '0'", - ), - ] { - assert_eq!(message(plan.kernel_command_line(user)), refusal, "{user}"); - } -} - -#[test] -fn plans_refuse_the_tokens_that_the_guest_init_reads_in_quoted_values() { - // The kernel reads `x` with a quoted value, but the guest's init splits the command line at - // spaces, tabs, and newlines without quoting, and reads the tokens in the value. - let sandbox_plan = plan(images()[0], SandboxUpper::RAM, SandboxSettings::default()); - let adapter_plan = adapter(images()[2], SandboxUpper::RAM); - for (plan, user, refusal) in [ - ( - &sandbox_plan, - "x=\"y nvx_entrypoint=/bin/other\"", - "nvx_entrypoint is owned by the sandbox profile", - ), - ( - &sandbox_plan, - "x=\"y\tvirtfs_tag=other\"", - "virtfs_tag is owned by the sandbox --mount option", - ), - ( - &adapter_plan, - "x=\"y nvx_layer=distro,0xd0003000,gpt\"", - "nvx_layer is owned by the plan's root", - ), - ( - &adapter_plan, - "x=\"y\nvirtfs_tag=microvm\"", - "virtfs_tag is owned by OpenVMM's shares", - ), - ] { - let error = plan.kernel_command_line(user).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported, "{user:?}"); - assert_eq!(error.to_string(), refusal, "{user:?}"); - } -} - -#[test] -fn adapter_plans_persist_without_settings_or_an_export() { - let plan = SandboxPlan::adapter( - SandboxDistro::new("/images/distro.img", RootImage::gpt(Some(uuid(GPT_GUID)))), - SandboxUpper::RAM, - ) - .with_network(Some(proxy_network(Some(ProxyScheme::Http)))); - let form = serde_json::to_value(&plan).unwrap(); - assert_eq!(form["version"], json!(PLAN_VERSION)); - assert_eq!(form["settings"], Value::Null); - assert_eq!(form["export"], Value::Null); - let read: SandboxPlan = serde_json::from_value(form.clone()).unwrap(); - assert_eq!(read, plan); - assert_eq!( - read.kernel_command_line("quiet").unwrap(), - plan.kernel_command_line("quiet").unwrap() - ); - assert_eq!( - read.openvmm_arguments().unwrap(), - plan.openvmm_arguments().unwrap() - ); - // An adapter's plan never holds an export plan, and its settings are required as `null`. - let (_, sandbox) = persisted(); - let mut exporting = form.clone(); - exporting["export"] = sandbox["export"].clone(); - let error = serde_json::from_value::(exporting) - .unwrap_err() - .to_string(); - assert!( - error.contains("an adapter's plan holds no export plan"), - "{error}" - ); - let mut missing = form; - missing.as_object_mut().unwrap().remove("settings"); - let error = serde_json::from_value::(missing) - .unwrap_err() - .to_string(); - assert!(error.contains("missing field `settings`"), "{error}"); -} - -/// Returns the length, in UTF-16 code units, of what an adapter's launch of `plan` with the -/// user's command line `user` and `export`, if any, takes of the Windows command line as it is -/// planned: the plan's arguments, those of the export, and the kernel command line. -fn planned_units(plan: &SandboxPlan, user: &str, export: Option<&ExportPlan>) -> usize { - let mut arguments: Vec = plan - .openvmm_arguments() - .unwrap() - .into_iter() - .map(OsString::from) - .collect(); - arguments.extend( - export - .map(ExportPlan::openvmm_arguments) - .unwrap_or_default() - .into_iter() - .map(OsString::from), - ); - arguments.push("--cmdline".into()); - arguments.push(plan.kernel_command_line(user).unwrap().into()); - let empty = std::ffi::OsStr::new(""); - launch::process_command_line_units(empty, &arguments) - - launch::process_command_line_units(empty, &[]) -} - -#[test] -fn adapter_launches_are_planned_with_room_for_the_rest_of_the_launch() { - use crate::host::launch::PROCESS_COMMAND_LINE_LIMIT; - - let adapter = SandboxPlan::adapter( - SandboxDistro::new("/images/distro.img", RootImage::gpt(Some(uuid(GPT_GUID)))), - SandboxUpper::scratch("/sandboxes/one/scratch.ext4"), - ) - .with_network(Some(proxy_network(Some(ProxyScheme::Http)))); - let user = "nvx_sandbox=1"; - adapter.check_adapter_launch(user, None).unwrap(); - adapter - .check_adapter_launch(user, Some(&shares(false))) - .unwrap(); - // An export whose hidden paths take what the reserve leaves the plan, exactly, and then one - // code unit more. - let hiding = |extra: usize| { - let mut export = shares(true); - let child = &mut export.children[0]; - child.denied = (0..100) - .map(|index| format!("/host/work/{index:03}-{}", "d".repeat(240)).into()) - .collect(); - child - .denied - .push(format!("/host/work/last{}", "d".repeat(extra)).into()); - child.denied_identities = vec![None; child.denied.len()]; - export - }; - let room = PROCESS_COMMAND_LINE_LIMIT - VmmLaunch::RESERVE; - let slack = room - planned_units(&adapter, user, Some(&hiding(0))); - let fitting = hiding(slack); - assert_eq!(planned_units(&adapter, user, Some(&fitting)), room); - adapter.check_adapter_launch(user, Some(&fitting)).unwrap(); - let error = adapter - .check_adapter_launch(user, Some(&hiding(slack + 1))) - .unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported); - let message = error.to_string(); - assert!( - message.contains(&format!("would take {}", room + 1)), - "{message}" - ); - assert!( - message.contains("with 2048 held in reserve for the rest of the launch"), - "{message}" - ); - // The plan of NVX's sandbox profile is not an adapter's, and a kernel command line that the - // plan refuses fails the check as it fails the plan. - let profile = plan(images()[0], SandboxUpper::RAM, SandboxSettings::default()); - let (kind, refused) = { - let error = profile.check_adapter_launch(user, None).unwrap_err(); - (error.kind(), error.to_string()) - }; - assert_eq!(kind, ErrorKind::Unsupported); - assert!(refused.contains("NVX's sandbox profile"), "{refused}"); - let error = adapter.check_adapter_launch("quiet\0", None).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported); - assert!(error.to_string().contains("NUL"), "{error}"); -} - -#[test] -fn persisted_plans_that_break_the_form_or_the_checks_are_refused() { - let (_, form) = persisted(); - let refused = |change: &dyn Fn(&mut Value), expected: &str| { - let mut value = form.clone(); - change(&mut value); - let error = serde_json::from_value::(value) - .unwrap_err() - .to_string(); - assert!(error.contains(expected), "{error} lacks {expected}"); - }; - // Every field is required, nulls included, and no other is accepted. - for pointer in [ - "/version", - "/distro", - "/distro/path", - "/distro/format", - "/distro/identity", - "/upper", - "/upper/kind", - "/settings", - "/settings/entrypoint", - "/settings/args", - "/settings/hostname", - "/settings/workloadIdentity", - "/settings/workloadIdentity/uid", - "/settings/memoryMax", - "/settings/pidsMax", - "/export", - "/export/children", - "/export/children/0/root", - "/export/children/0/deniedIdentities", - "/export/binds", - "/export/binds/0/target", - "/export/binds/1/readOnly", - "/network", - "/network/guestNetwork", - "/network/egress", - "/network/allow", - "/network/allow/name", - "/network/allow/rules", - "/network/allow/rules/0/to", - "/network/allow/rules/0/to/0/except", - "/network/allow/rules/0/ports", - "/network/allow/rules/0/ports/0/protocol", - "/network/deny/rules/0/ports/0/endPort", - "/network/deny", - "/network/ingress", - "/network/hostLoopback", - "/network/hostLoopbackForwards", - "/network/hostLoopbackForwards/0/guestPort", - "/network/proxy", - ] { - let (parent, name) = pointer.rsplit_once('/').unwrap(); - refused( - &|value| { - value - .pointer_mut(parent) - .unwrap() - .as_object_mut() - .unwrap() - .remove(name) - .unwrap(); - }, - "missing field", - ); - } - for pointer in [ - "", - "/distro", - "/upper", - "/settings", - "/settings/workloadIdentity", - "/export", - "/export/children/0", - "/export/binds/0", - "/network", - "/network/allow", - "/network/allow/rules/0", - "/network/allow/rules/0/to/0", - "/network/allow/rules/0/ports/0", - "/network/hostLoopbackForwards/0", - ] { - refused( - &|value| { - value.pointer_mut(pointer).unwrap()["origin"] = json!("elsewhere"); - }, - "unknown field", - ); - } - // A RAM upper has no scratch image. - refused( - &|value| value["upper"]["path"] = json!("scratch.ext4"), - "unknown field `path`", - ); - refused( - &|value| value["upper"] = json!({"kind": "scratch"}), - "missing field `path`", - ); - refused( - &|value| value["upper"] = json!({"kind": "tmpfs"}), - "unknown variant", - ); - refused(&|value| value["upper"] = json!("ram"), "invalid type"); - refused( - &|value| value["distro"]["format"] = json!("ext4"), - "unknown variant", - ); - for version in [1, 2, 4] { - refused( - &|value| value["version"] = json!(version), - &format!("the sandbox plan has version {version}, which this build does not read"), - ); - } - // The forms of versions 1 and 2 held the shares as requested, and that of version 1 had no - // network: neither is read. - refused( - &|value| { - value["version"] = json!(1); - value.as_object_mut().unwrap().remove("network"); - }, - "missing field `network`", - ); - refused( - &|value| { - value["version"] = json!(2); - value["settings"]["mounts"] = json!([]); - }, - "unknown field `mounts`", - ); - for identity in ["", "FEDCBA98-7654-3210-FEDC-BA9876543210", "{fedcba98}"] { - refused( - &|value| value["distro"]["identity"] = json!(identity), - &format!( - "the distro identity of the sandbox plan, {identity:?}, is not a UUID in its \ - canonical text" - ), - ); - } - for limit in ["", "0", "01", "+1", "1 ", "-1", "1_000", "\u{0661}", "0x10"] { - refused( - &|value| value["settings"]["memoryMax"] = json!(limit), - "the memoryMax of the sandbox plan", - ); - refused( - &|value| value["settings"]["pidsMax"] = json!(limit), - "is not a positive integer in decimal text", - ); - } - refused( - &|value| value["settings"]["memoryMax"] = json!(268_435_456), - "invalid type", - ); - // What the plan and its shares check, reading checks too. - refused( - &|value| value["settings"]["memoryMax"] = json!("18446744073709551616"), - "is not a positive integer in decimal text", - ); - refused( - &|value| value["settings"]["pidsMax"] = json!("4194304"), - "--pids-max must be at most 4194303", - ); - refused( - &|value| value["settings"]["workloadIdentity"]["uid"] = json!(0), - "sandbox workload identity must be a non-root UID:GID", - ); - refused( - &|value| value["settings"]["entrypoint"] = json!("bin/sh"), - "sandbox entrypoint must be an absolute path without whitespace, control characters, or quotes", - ); - refused( - &|value| value["export"]["binds"][1]["target"] = json!("/workspace/settings.json"), - "would hide part of that path", - ); - refused( - &|value| value["export"]["binds"][0]["target"] = json!("/etc"), - "its guest target /etc would cover the guest's /etc", - ); - refused( - &|value| value["export"]["owner"] = json!("root"), - "unknown variant `root`", - ); - refused( - &|value| value["export"]["children"] = json!([]), - "the recorded export plan is invalid", - ); - // The network's values, and what its policy checks. - refused( - &|value| value["network"]["egress"] = json!("block"), - "unknown variant", - ); - refused( - &|value| value["network"]["hostLoopback"] = json!(true), - "invalid type", - ); - refused( - &|value| value["network"]["allow"]["rules"][0]["ports"][0]["protocol"] = json!("sctp"), - "unknown variant", - ); - refused( - &|value| value["network"]["allow"]["rules"][0]["ports"][0]["port"] = json!(65536), - "invalid value", - ); - refused( - &|value| value["network"]["hostLoopbackForwards"][0]["protocol"] = json!("icmp"), - "unknown variant", - ); - refused( - &|value| value["network"]["guestNetwork"] = json!("10.0.0.1/24"), - "guest network \"10.0.0.1/24\" must be an IPv4 address with a /1 to /30 prefix that is not \ - its network's network, broadcast, or gateway (first) address", - ); - refused( - &|value| value["network"]["allow"]["rules"][0]["to"][0]["cidr"] = json!("example.com"), - "egress.allow[0].to[0].cidr is not a valid IPv4 or IPv6 CIDR: \"example.com\"", - ); - refused( - &|value| value["network"]["hostLoopback"] = json!("outbound"), - "host-loopback forwards need host-loopback access allowed", - ); - // A proxy is the one way out of its network, and its scheme is http or https. - refused( - &|value| value["network"]["proxy"] = json!({"port": 3128, "scheme": null}), - "a network proxy needs egress denied by default", - ); - let proxy_only = serde_json::to_value(proxy_network(None)).unwrap(); - refused( - &|value| { - value["network"] = proxy_only.clone(); - value["network"]["proxy"]["scheme"] = json!("ftp"); - }, - "network proxy scheme \"ftp\" must be http or https", - ); - refused( - &|value| { - value["network"] = proxy_only.clone(); - value["network"]["proxy"]["port"] = json!(0); - }, - "a network proxy needs a port between 1 and 65535", - ); - refused( - &|value| { - value["network"] = proxy_only.clone(); - value["network"]["proxy"]["port"] = json!(53); - }, - "a network proxy needs a port other than 53", - ); - refused( - &|value| { - value["network"] = proxy_only.clone(); - value["network"]["proxy"] - .as_object_mut() - .unwrap() - .remove("scheme"); - }, - "missing field `scheme`", - ); - refused( - &|value| { - value["network"] = proxy_only.clone(); - value["network"]["proxy"]["origin"] = json!("elsewhere"); - }, - "unknown field", - ); -} - -#[test] -fn plans_render_and_report_their_network() { - let without = exporting( - images()[0], - uppers()[0].clone(), - SandboxSettings::default(), - Some(shares(true)), - ); - let shares = without.openvmm_arguments().unwrap(); - assert!(without.network().is_none()); - assert_eq!(without.proxy_url(), None); - - // The network's options follow the plan's other options. - for network in [direct_network(), proxy_network(None)] { - let plan = without.clone().with_network(Some(network.clone())); - let mut expected = shares.clone(); - expected.extend(network.openvmm_arguments().unwrap()); - assert_eq!(plan.openvmm_arguments().unwrap(), expected); - assert_eq!(plan.network(), Some(&network)); - // The network adds no boot token. - assert_eq!( - plan.kernel_command_line("quiet").unwrap(), - without.kernel_command_line("quiet").unwrap() - ); - } - let direct = without.clone().with_network(Some(direct_network())); - let arguments = direct.openvmm_arguments().unwrap(); - assert_eq!( - arguments[shares.len()..], - [ - "--net", - "10.0.0.2/24", - "--network-profile", - "portable", - "--network-egress", - "allow", - "--network-ingress", - "allow", - "--host-loopback", - "allow", - "--network-egress-allow", - "0.0.0.0/0:tcp:443", - "--network-egress-allow", - "192.0.2.0/25:tcp:8000-8010", - "--network-egress-allow", - "192.0.2.0/25:udp:8000-8010", - "--network-egress-allow", - "::/0:tcp:443", - "--network-egress-deny", - "198.51.100.0/24:udp:53", - "--network-egress-deny", - "2001:db8::/32:icmp", - "--host-loopback-forward", - "tcp:18080:8080", - ] - ); - assert_eq!(direct.proxy_url(), None); - - // A proxy-only network reaches its proxy at the gateway, which its URL names. - for (scheme, url) in [ - (None, "10.0.0.1:3128"), - (Some(ProxyScheme::Http), "http://10.0.0.1:3128"), - (Some(ProxyScheme::Https), "https://10.0.0.1:3128"), - ] { - let network = proxy_network(scheme); - assert_eq!(network.gateway(), std::net::Ipv4Addr::new(10, 0, 0, 1)); - assert_eq!(network.proxy_url().as_deref(), Some(url)); - let plan = without.clone().with_network(Some(network)); - assert_eq!(plan.proxy_url().as_deref(), Some(url)); - assert_eq!( - plan.openvmm_arguments().unwrap()[shares.len()..], - [ - "--net", - "10.0.0.2/24", - "--network-profile", - "portable", - "--network-egress", - "deny", - "--network-ingress", - "deny", - "--host-loopback", - "deny", - "--network-proxy", - "10.0.0.1:3128", - ] - ); - } - let narrow = SandboxNetwork::new( - "192.168.7.6/30", - NetworkPolicy::proxy_only(NetworkProxy::new(8080).with_scheme(ProxyScheme::Http)), - ) - .unwrap(); - assert_eq!( - narrow.proxy_url().as_deref(), - Some("http://192.168.7.5:8080") - ); - - // A policy that lets nothing through attaches no network device. - let isolated = SandboxNetwork::new("10.0.0.2/24", NetworkPolicy::new(Access::Deny)).unwrap(); - assert!(isolated.openvmm_arguments().unwrap().is_empty()); - assert_eq!( - without - .clone() - .with_network(Some(isolated)) - .openvmm_arguments() - .unwrap(), - shares - ); -} - -#[test] -fn networks_check_their_guest_network_and_their_policy() { - for guest_network in [ - "10.0.0.1/24", - "10.0.0.0/24", - "10.0.0.2/31", - "10.0.0.2", - "fd00::2/64", - ] { - let error = - SandboxNetwork::new(guest_network, NetworkPolicy::new(Access::Allow)).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported); - assert_eq!( - error.to_string(), - format!( - "guest network {guest_network:?} must be an IPv4 address with a /1 to /30 prefix \ - that is not its network's network, broadcast, or gateway (first) address" - ) - ); - } - // The policy must lower: each way out beside a proxy is refused before launch. - let mut policy = NetworkPolicy::proxy_only(NetworkProxy::new(3128)); - policy.ingress = Access::Allow; - let error = SandboxNetwork::new("10.0.0.2/24", policy).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported); - assert_eq!(error.to_string(), "a network proxy needs ingress denied"); - let mut policy = NetworkPolicy::new(Access::Allow); - policy.host_loopback = HostLoopback::Allow; - assert_eq!( - message(SandboxNetwork::new("10.0.0.2/30", policy)), - "host-loopback access needs a guest network prefix of /29 or shorter, which leaves an \ - address at which OpenVMM presents host peers to the guest" - ); -} - -#[test] -fn launches_carry_their_network_into_their_plans() { - let directory = tempfile::tempdir().unwrap(); - let layer = directory.path().join("distro.erofs"); - fs::write(&layer, b"layer").unwrap(); - let launch = SandboxLaunch::new( - vec![SandboxLayer::new( - LayerRole::Distro, - layer.to_str().unwrap(), - EROFS_UUID, - )], - SandboxUpper::scratch("scratch.ext4"), - SandboxSettings::default(), - ) - .unwrap(); - assert!(launch.network().is_none()); - let launch = launch.with_network(Some(proxy_network(Some(ProxyScheme::Http)))); - assert_eq!( - launch.network(), - Some(&proxy_network(Some(ProxyScheme::Http))) - ); - assert_eq!(launch.plan().network(), launch.network()); - // The plan of the binding keeps the network. - let binding = DistroBinding::verify(launch.distro()).unwrap(); - let bound = launch.bound_plan(&binding).unwrap(); - assert_eq!(bound.network(), launch.network()); - assert_eq!(bound.proxy_url().as_deref(), Some("http://10.0.0.1:3128")); - assert!(launch.with_network(None).plan().network().is_none()); -} diff --git a/nvx-rs/src/host/sandbox/prepared.rs b/nvx-rs/src/host/sandbox/prepared.rs deleted file mode 100644 index 8924e5e60..000000000 --- a/nvx-rs/src/host/sandbox/prepared.rs +++ /dev/null @@ -1,458 +0,0 @@ -//! The prepared launch of a sandbox: the files that the checks of its plan returned, held open -//! until its VMM has them, the descriptors or handles that the VMM inherits, and the OpenVMM -//! arguments and kernel command line that name them. -//! -//! A [`SandboxPlan`] names its block devices by path, and persists. A launch that gave the VMM -//! those paths would let anything that took a path's place after the check, and before the VMM -//! opened the path, take the image's place. A [`PreparedLaunch`] instead takes the files that the -//! checks returned, the [`CheckedDistro`] of -//! [`DistroBinding::check`](super::distro::DistroBinding::check) and, for an upper on scratch, -//! the [`CheckedScratch`] of [`ScratchBinding::check`](super::scratch::ScratchBinding::check), and -//! hands the VMM exactly those objects: -//! -//! - On Linux, the VMM inherits the checked descriptors, through the allowlist of -//! [`VmmCommand::inherit`](crate::host::supervisor::VmmCommand::inherit), and the OpenVMM -//! arguments name each `/proc/self/fd/N`, which OpenVMM opens: the link reaches the checked file -//! whatever later takes its path. No other descriptor of the caller reaches the VMM. A checked -//! file on a standard stream's descriptor, which no spawn can pass under its number, as when the -//! caller had closed that stream, first moves above the standard streams, with its open file -//! description. -//! - On Windows, the arguments name each checked file by its final path, the absolute `\\?\` path -//! that the file's handle gives, which resolves every link and junction that the plan's path -//! went through, and does not depend on the VMM's directory. The checked files hold that path in -//! place until they are closed: the distro layer's file denies other opens for writing or -//! deleting the layer, the scratch image's guard denies deleting or renaming it, and NTFS renames -//! no directory while a file below it is open. OpenVMM's read-only open of the layer, and its -//! open of the scratch image for reading and writing, share them. The VMM also inherits their -//! handles, and the claim's handle of the scratch image, though no argument names a handle, so -//! that its copies hold the paths, and the claim, from its start, even if the caller ends before -//! the VMM has opened the images, as when it crashes: a replacement at their paths then fails -//! until the VMM exits, and never reaches it unchecked. A final path that OpenVMM could not take -//! as the path of a raw image, with a comma, semicolon, unbalanced brackets, or a container's -//! extension, which a link on the way, the directory of a relative path, or the case of a name -//! on disk can bring, fails the preparation. -//! - Other hosts cannot check files, so no launch is prepared there. -//! -//! Images are raw: OpenVMM attaches a file's bytes as they are, and the launch never converts an -//! image or reads its footer, as [`SandboxDistro`](super::SandboxDistro) says. -//! -//! The paths of descriptors exist only in the launch: [`PreparedLaunch::openvmm_arguments`] -//! renders them, and the plan, which callers persist, keeps the real paths. The caller adds its own -//! arguments, such as its boot console, CPU profile, and guest control, or, for an adapter's plan, -//! has [`PreparedLaunch::adapter_arguments`] render the launch's complete arguments, which it holds -//! to the limit of a Windows command line, starts the VMM with [`PreparedLaunch::inherited`] as -//! its allowlist, and holds the prepared launch until the guest reports authenticated Ready, when -//! [`PreparedLaunch::release`] closes the checked files, which releases the guards, or until it has -//! torn down a launch that failed, when dropping the prepared launch closes them. The VMM keeps its -//! own opens of the images, and the claim of the scratch image, which it inherited. On Windows, the -//! release also closes the VMM's copies of the checked files, through the handle of the VMM that -//! its launch holds; a prepared launch that is dropped without one, as when its caller ends before -//! the guest is ready, leaves them to the VMM until it exits. -//! -//! A plan never names a root identity for the workload: [`SandboxPlan::new`] refuses a zero UID -//! or GID, so the VMM's `--microvm-workload-identity` never encodes root, and an adapter's plan, -//! [`SandboxPlan::adapter`], names no workload identity, which the adapter's own guest chooses. - -use std::ffi::OsString; -use std::fs::File; -use std::path::Path; - -use super::distro::CheckedDistro; -#[cfg(any(target_os = "linux", windows))] -use super::mount::os_error; -use super::plan::{self, BlockRole, Delivery, SandboxPlan}; -use super::refused; -use super::scratch::CheckedScratch; -#[cfg(windows)] -use super::{reject_container_path, reject_disk_path}; -#[cfg(windows)] -use crate::error::Error; -use crate::error::Result; -use crate::host::filesystem::ExportPlan; -use crate::host::launch::{VmmLaunch, fit_process_command_line}; -use crate::host::platform::inherit::Inherited; -use crate::host::process::Process; -#[cfg(windows)] -#[cfg(windows)] -use crate::host::seal; -#[cfg(target_os = "linux")] -use crate::host::sys; - -/// A launch that holds the checked files of its plan, and the OpenVMM arguments and kernel -/// command line that hand them to the VMM. -/// -/// [`new`](Self::new) prepares it from a plan and the files that the checks of its block devices -/// returned. The caller then starts the VMM with [`openvmm_arguments`](Self::openvmm_arguments), -/// its own arguments, and [`kernel_command_line`](Self::kernel_command_line), with -/// [`inherited`](Self::inherited) as the allowlist of its -/// [`VmmCommand`](crate::host::supervisor::VmmCommand), and [releases](Self::release) the -/// prepared launch once the guest reports authenticated Ready, or drops it once a launch that -/// failed is torn down: -/// -/// ```no_run -/// use std::ffi::OsString; -/// use std::fs::File; -/// use std::time::Instant; -/// -/// use nvx::host::sandbox::distro::{DISTRO_POLICY_REVISION, DistroBinding}; -/// use nvx::host::sandbox::scratch::{CLAIM_RELEASE_WAIT, ScratchBinding}; -/// use nvx::host::sandbox::{PreparedLaunch, SandboxLaunch}; -/// use nvx::host::supervisor::{self, VmmCommand}; -/// -/// # fn launch( -/// # launch: &SandboxLaunch, -/// # distro_binding: &DistroBinding, -/// # scratch_binding: Option<&ScratchBinding>, -/// # capability: &[u8; nvx::protocol::CAPABILITY_LEN], -/// # ) -> Result<(), Box> { -/// // Check the bindings that the caller recorded when it staged the images: from their metadata -/// // alone, and claiming the scratch image of an upper on scratch for this sandbox. -/// let distro = distro_binding.check(launch.distro(), DISTRO_POLICY_REVISION)?; -/// let scratch = match (launch.scratch(), scratch_binding) { -/// (Some(path), Some(binding)) => Some(binding.check(path, Instant::now() + CLAIM_RELEASE_WAIT)?), -/// _ => None, -/// }; -/// let prepared = PreparedLaunch::new(launch.bound_plan(distro_binding)?, distro, scratch, "")?; -/// -/// let mut arguments: Vec = -/// prepared.openvmm_arguments().iter().map(OsString::from).collect(); -/// arguments.extend(["--cmdline".into(), prepared.kernel_command_line().into()]); -/// // ... the caller's own arguments: hypervisor, memory, kernel, initrd, and guest control ... -/// let mut command = VmmCommand::new("openvmm", arguments, "."); -/// command.inherit = prepared.inherited(); -/// // A log file of its own, created anew, as `supervisor::spawn` asks of each VMM on Windows. -/// let _ = std::fs::remove_file("openvmm.log"); -/// let launched = supervisor::spawn(&command, capability, File::create_new("openvmm.log")?)?; -/// // ... record the VMM, and wait for the guest's authenticated Ready ... -/// prepared.release(&launched.process()?)?; -/// launched.detach(); -/// # Ok(()) -/// # } -/// ``` -#[derive(Debug)] -pub struct PreparedLaunch { - /// The plan, which names the images by their real paths. - plan: SandboxPlan, - /// The checked distro layer. - distro: CheckedDistro, - /// The checked scratch image, for an upper on scratch. - scratch: Option, - /// The OpenVMM arguments of the plan, which name the checked files. - openvmm_arguments: Vec, - /// The kernel command line of the plan. - kernel_command_line: String, -} - -impl PreparedLaunch { - /// Prepares the launch of `plan` with the checked files of its block devices: `distro`, which - /// [`DistroBinding::check`](super::distro::DistroBinding::check) returned for the plan's - /// image, and `scratch`, which [`ScratchBinding::check`](super::scratch::ScratchBinding::check) - /// returned for the scratch image of an upper on scratch, or `None` for an upper in RAM. It - /// renders the plan's OpenVMM arguments, naming the checked files, and its kernel command line - /// with `user_command_line`, as [`SandboxPlan::kernel_command_line`] does. - /// - /// The checked files must be those of the plan: the distro layer checked at the path of the - /// plan's image, with the format and identity of its image, and the scratch image checked at - /// the path of its upper. Fails with - /// [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) for files that are not, and as - /// [`SandboxPlan::openvmm_arguments`] and [`SandboxPlan::kernel_command_line`] fail; on - /// Windows, also with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) for a checked - /// file whose final path has a comma, semicolon, or unbalanced brackets, or ends in `.vhd`, - /// `.vhdx`, `.vmgs`, or `.iso`, which OpenVMM could not attach as a raw image; and with - /// [`ErrorKind::Io`](crate::ErrorKind::Io) if a checked file cannot be moved above the - /// standard streams on Linux, or its final path is unavailable on Windows. It never opens, - /// reads, or maps a file. - pub fn new( - plan: SandboxPlan, - distro: CheckedDistro, - scratch: Option, - user_command_line: &str, - ) -> Result { - Self::prepare( - plan, - distro, - scratch, - user_command_line, - Delivery::MappingTable, - ) - } - - /// Implements [`new`](Self::new), with the shares as `delivery` delivers them. - fn prepare( - plan: SandboxPlan, - distro: CheckedDistro, - scratch: Option, - user_command_line: &str, - delivery: Delivery, - ) -> Result { - let planned = plan.distro().path(); - if !same_path(distro.path(), planned) { - return Err(refused(format!( - "the checked distro layer {} is not the plan's image {}", - distro.path().display(), - planned.display() - ))); - } - let (format, identity) = distro.bound(); - if plan::image_of(format, identity)? != plan.distro().image() { - return Err(refused(format!( - "the plan's image is not the {format} layer with identity {identity} that the \ - distro binding of {} names", - planned.display() - ))); - } - match (plan.upper().scratch_path(), &scratch) { - (Some(planned), Some(checked)) if !same_path(checked.path(), planned) => { - return Err(refused(format!( - "the checked scratch image {} is not the plan's scratch image {}", - checked.path().display(), - planned.display() - ))); - } - (Some(planned), None) => { - return Err(refused(format!( - "the plan's upper is on the scratch image {}, which was not checked", - planned.display() - ))); - } - (None, Some(checked)) => { - return Err(refused(format!( - "the plan's upper is in RAM, which takes no scratch image, not {}", - checked.path().display() - ))); - } - _ => {} - } - #[cfg(target_os = "linux")] - let (distro, scratch) = above_standard_streams(distro, scratch)?; - let distro_block = checked_path(distro.file(), distro.path())?; - let scratch_block = match &scratch { - Some(scratch) => Some(checked_path(scratch.file(), scratch.path())?), - None => None, - }; - let openvmm_arguments = plan.render_arguments( - &|role, _| match (role, &scratch_block) { - (BlockRole::Distro, _) => distro_block.clone(), - (BlockRole::Scratch, Some(block)) => block.clone(), - (BlockRole::Scratch, None) => { - unreachable!("a plan with a scratch block has its checked scratch image") - } - }, - delivery, - )?; - let kernel_command_line = plan.command_line(user_command_line, delivery)?; - Ok(Self { - plan, - distro, - scratch, - openvmm_arguments, - kernel_command_line, - }) - } - - /// Returns the plan, which names the images by their real paths. - pub fn plan(&self) -> &SandboxPlan { - &self.plan - } - - /// Returns the checked distro layer. - pub fn distro(&self) -> &CheckedDistro { - &self.distro - } - - /// Returns the checked scratch image, or `None` for an upper in RAM. - pub fn scratch(&self) -> Option<&CheckedScratch> { - self.scratch.as_ref() - } - - /// Returns the OpenVMM arguments of the plan, as [`SandboxPlan::openvmm_arguments`] renders - /// them, except that each block device names its checked file: on Linux as `/proc/self/fd/N`, - /// which [`inherited`](Self::inherited) passes the VMM under the number `N`, and on Windows by - /// its final path. - /// - /// They hold for this launch only: never persist them. - pub fn openvmm_arguments(&self) -> &[String] { - &self.openvmm_arguments - } - - /// Returns the kernel command line of the plan, with the user's command line given to - /// [`new`](Self::new), as [`SandboxPlan::kernel_command_line`] renders it. - pub fn kernel_command_line(&self) -> &str { - &self.kernel_command_line - } - - /// Returns OpenVMM's complete arguments for this launch of an adapter's plan, - /// [`SandboxPlan::adapter`], with the inputs of `launch`, and with `export`, the export of the - /// host paths that the adapter shares with its guest's root, if any: the plan's - /// [arguments](Self::openvmm_arguments), which name the checked files, then those that export - /// the shares, which [`ExportPlan::openvmm_arguments`] renders, since an adapter's plan holds - /// no export, then those of the launch's own inputs, which [`VmmLaunch::arguments`] renders - /// with the plan's [kernel command line](Self::kernel_command_line). OpenVMM appends the boot - /// tokens with which the guest mounts the export to the kernel command line. - /// - /// Unlike those of NVX's sandbox profile, they name no workload identity, which an adapter's - /// plan does not hold, and no managed lifecycle: OpenVMM only checks the arguments of the - /// managed lifecycle and labels its outcome report with it, and only NVX's init agent, which - /// an adapter's guest replaces, reads its boot token. A caller that adds arguments of its own, - /// such as a CPU profile, checks the command line with them again, with - /// [`fit_process_command_line`]. - /// - /// The arguments name the checked files of this launch alone: never persist them. - /// - /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) for the plan of NVX's - /// sandbox profile, which its own start launches with the managed lifecycle, for a launch of - /// no memory, and if the command line that runs the launch's OpenVMM executable with the - /// arguments would not fit - /// [`PROCESS_COMMAND_LINE_LIMIT`](crate::host::launch::PROCESS_COMMAND_LINE_LIMIT), as - /// [`fit_process_command_line`] checks it without a reserve: Windows would not start - /// OpenVMM with it, and every host holds a launch to it. - pub fn adapter_arguments( - &self, - launch: &VmmLaunch<'_>, - export: Option<&ExportPlan>, - ) -> Result> { - self.plan.require_adapter()?; - if launch.memory_mib == 0 { - return Err(refused("the memory of a launch must be positive")); - } - let mut arguments: Vec = - self.openvmm_arguments.iter().map(OsString::from).collect(); - if let Some(export) = export { - arguments.extend(export.openvmm_arguments().into_iter().map(OsString::from)); - } - arguments.extend(launch.arguments(&self.kernel_command_line)); - fit_process_command_line(launch.openvmm.as_os_str(), &arguments, 0)?; - Ok(arguments) - } - - /// Returns the descriptors or handles that the VMM inherits, for the allowlist of its - /// [`VmmCommand::inherit`](crate::host::supervisor::VmmCommand::inherit): those of the checked - /// distro layer and of the checked scratch image, if any, which on Linux - /// [`openvmm_arguments`](Self::openvmm_arguments) names, and on Windows, where the VMM opens the - /// checked files by their final paths, hold those paths for the VMM, with the claim's handle of - /// the scratch image, which no argument names; elsewhere none. - pub fn inherited(&self) -> Vec> { - #[cfg(any(target_os = "linux", windows))] - { - let mut inherited = vec![Inherited::new(self.distro.file())]; - if let Some(scratch) = &self.scratch { - inherited.push(Inherited::new(scratch.file())); - #[cfg(windows)] - inherited.push(Inherited::new(scratch.claim())); - } - inherited - } - #[cfg(not(any(target_os = "linux", windows)))] - Vec::new() - } - - /// Releases the checked files once the VMM has opened the images, when the guest reports - /// authenticated Ready, which releases their guards: on Windows, it first closes the VMM's - /// copies of the checked files, those of the distro layer and of the scratch image's guard, - /// then, on every host, the checked files themselves, as dropping the prepared launch does. The - /// VMM keeps its own opens of the images, and the scratch image's claim, which it inherited. - /// - /// `vmm` must be the VMM that inherited [`inherited`](Self::inherited), held as its launch - /// holds it: through [`Launched::process`](crate::host::supervisor::Launched::process). A - /// VMM that has exited has closed its copies. A launch that failed needs no release: dropping - /// the prepared launch once the launch is torn down closes the checked files, and the VMM's - /// copies went with it. A prepared launch that is dropped without a release, as when its - /// caller ends before the guest is ready, leaves the VMM its copies until it exits. - /// - /// Fails on Windows with [`ErrorKind::Io`](crate::ErrorKind::Io) if the VMM's copy of a - /// checked file cannot be closed: for a `vmm` that did not inherit it, which keeps all of its - /// handles, or one held without the right to close them, as [`Process::open`] holds a process. - /// The checked files are closed whatever the outcome, and the VMM keeps the copies that remain - /// until it exits. - pub fn release(self, vmm: &Process) -> Result<()> { - #[cfg(windows)] - { - use std::os::windows::io::AsHandle; - - let mut checked = vec![(self.distro.file(), self.distro.path())]; - checked.extend( - self.scratch - .as_ref() - .map(|scratch| (scratch.file(), scratch.path())), - ); - for (file, path) in checked { - vmm.close_inherited(file.as_handle()).map_err(|error| { - Error::io(format!( - "cannot release the guard of {}, which VMM process {} inherited", - path.display(), - vmm.reference().pid - )) - .with_source(error) - })?; - } - } - #[cfg(not(windows))] - let _ = vmm; - drop(self); - Ok(()) - } -} - -/// Returns whether `checked` and `planned`, two paths spelled with their components alone, as -/// [`Path::components`] reads them, are the same text. -fn same_path(checked: &Path, planned: &Path) -> bool { - checked.as_os_str() == planned.as_os_str() -} - -/// Moves the checked files that are on a standard stream's descriptor, under whose number no spawn -/// passes them to the VMM, above the standard streams, as when the caller had closed that stream -/// before the checks opened them. A moved file keeps its open file description, and with it the -/// claim of a scratch image. -#[cfg(target_os = "linux")] -fn above_standard_streams( - mut distro: CheckedDistro, - mut scratch: Option, -) -> Result<(CheckedDistro, Option)> { - sys::move_above_standard_streams(distro.file_mut()) - .map_err(|error| os_error(&error, Some(distro.path())))?; - if let Some(scratch) = &mut scratch { - sys::move_above_standard_streams(scratch.file_mut()) - .map_err(|error| os_error(&error, Some(scratch.path())))?; - } - Ok((distro, scratch)) -} - -/// Returns how the OpenVMM arguments name the checked `file`, which a check opened at `path`: on -/// Linux, as `/proc/self/fd/N`; on Windows, by its final path, which resolves no link, and which -/// holds whatever the VMM's directory, refusing one that OpenVMM could not take as the path of a -/// raw image, as [`SandboxPlan::validated`] refuses the plan's paths; elsewhere, as `path`. -fn checked_path(file: &File, path: &Path) -> Result { - #[cfg(target_os = "linux")] - { - let _ = path; - Ok(Inherited::new(file).proc_path().display().to_string()) - } - #[cfg(windows)] - { - let resolved = seal::final_path(file) - .map_err(|error| os_error(&error, Some(path)))? - .into_os_string() - .into_string() - .map_err(|_| { - refused(format!( - "the final path of {} is not valid Unicode", - path.display() - )) - })?; - // OpenVMM reads the delimiters and the extension of the path that it is given, which a - // link on the way, the directory of a relative path, or the case of a name on disk can - // make differ from the plan's. - let shown = format!("{} resolves to {resolved}", path.display()); - reject_disk_path(Path::new(&resolved), &shown)?; - reject_container_path(Path::new(&resolved), &shown)?; - Ok(resolved) - } - #[cfg(not(any(target_os = "linux", windows)))] - { - let _ = file; - Ok(path.to_string_lossy().into_owned()) - } -} - -#[cfg(all(test, any(target_os = "linux", windows)))] -mod tests; diff --git a/nvx-rs/src/host/sandbox/prepared/tests.rs b/nvx-rs/src/host/sandbox/prepared/tests.rs deleted file mode 100644 index b8ac2e32c..000000000 --- a/nvx-rs/src/host/sandbox/prepared/tests.rs +++ /dev/null @@ -1,906 +0,0 @@ -//! Tests of prepared launches: the checked files that they hold, the OpenVMM arguments that name -//! them, the descriptors that the VMM inherits, and the path from staging to launch, which reads -//! no content. - -use std::fmt; -use std::fs; -use std::io::Read; -use std::path::PathBuf; -use std::time::Instant; - -use sha2::{Digest, Sha256}; - -use super::*; -use crate::ErrorKind; -use crate::host::paths; -use crate::host::sandbox::distro::{self, DISTRO_POLICY_REVISION, DistroBinding, PolicyVerdict}; -use crate::host::sandbox::scratch::{self, ScratchBinding}; -use crate::host::sandbox::{ - LayerRole, SandboxDistro, SandboxLaunch, SandboxLayer, SandboxSettings, SandboxUpper, -}; -use crate::host::seal::FileSeal; -use crate::host::seal::tests::open_metadata_only; -#[cfg(windows)] -use crate::host::tests::junction; -use crate::protocol::root::RootImage; -use crate::uuid::Uuid; - -/// The UUID of the staged layers. -const UUID: &str = "11111111-1111-1111-1111-111111111111"; -/// Another UUID. -const OTHER_UUID: &str = "22222222-2222-2222-2222-222222222222"; -/// The user's kernel command line of the launches. -const USER_COMMAND_LINE: &str = "console=ttyS0"; - -/// The access of a host that lets distro bindings open files and read their metadata, but fails -/// every read and mapping of their content. -const METADATA_ONLY: distro::Access<'static> = distro::Access { - open: &open_metadata_only, - seal: &FileSeal::of, -}; - -/// The access of a host that lets scratch bindings open the image for writing alone, so that no -/// read or mapping of its content can succeed through what they open: on Linux an open for writing -/// alone, and on Windows the access of binds and checks, which never ask to read. -#[cfg(target_os = "linux")] -const WRITE_ONLY: scratch::Access<'static> = scratch::Access { - open: &open_write_only, -}; -/// The access of a host that lets scratch bindings open the image for writing alone, so that no -/// read or mapping of its content can succeed through what they open: on Linux an open for writing -/// alone, and on Windows the access of binds and checks, which never ask to read. -#[cfg(windows)] -const WRITE_ONLY: scratch::Access<'static> = scratch::Access::WRITABLE; - -/// Opens `path` for writing alone. -#[cfg(target_os = "linux")] -fn open_write_only(path: &Path) -> std::io::Result { - fs::OpenOptions::new().write(true).open(path) -} - -/// A distro layer with its manifest, and scratch images, staged in a temporary directory. -struct Staged { - /// The temporary directory. - directory: tempfile::TempDir, -} - -impl Staged { - /// Stages a distro layer with a manifest that binds it, another copy of the layer, and two - /// scratch images. - fn new() -> Self { - let staged = Self { - directory: tempfile::tempdir().unwrap(), - }; - let layer = b"distro layer\n"; - fs::write(staged.path("distro.erofs"), layer).unwrap(); - fs::write(staged.path("other.erofs"), layer).unwrap(); - let digest: String = Sha256::digest(layer) - .iter() - .map(|byte| format!("{byte:02x}")) - .collect(); - fs::write( - staged.path("distro.erofs.manifest.json"), - format!( - "{{\"format\": 1, \"artifact\": \"distro.erofs\", \"artifact_sha256\": \ - \"{digest}\", \"packages\": [{{\"name\": \"bash\"}}]}}" - ), - ) - .unwrap(); - fs::write(staged.path("scratch.ext4"), b"scratch image\n").unwrap(); - fs::write(staged.path("other.ext4"), b"scratch image\n").unwrap(); - staged - } - - /// Returns the path of the file `name` of the directory. - fn path(&self, name: &str) -> PathBuf { - self.directory.path().join(name) - } - - /// Returns the text of the path of the file `name`, normalized. - fn text(&self, name: &str) -> String { - let path = paths::normalized(&self.path(name)); - path.to_str().unwrap().to_owned() - } - - /// Returns the distro layer `name`. - fn layer(&self, name: &str) -> SandboxLayer { - SandboxLayer::new(LayerRole::Distro, &self.text(name), UUID) - } - - /// Returns the launch of the layer `distro.erofs` over the image `scratch.ext4`. - fn launch(&self) -> SandboxLaunch { - SandboxLaunch::new( - vec![self.layer("distro.erofs")], - SandboxUpper::scratch(&self.text("scratch.ext4")), - SandboxSettings::default(), - ) - .unwrap() - } - - /// Verifies the layer `name`, checks it, and returns its binding and the checked layer. - fn checked_distro(&self, name: &str) -> (DistroBinding, CheckedDistro) { - let layer = self.layer(name); - let binding = DistroBinding::verify(&layer).unwrap(); - let checked = binding.check(&layer, DISTRO_POLICY_REVISION).unwrap(); - (binding, checked) - } - - /// Binds the scratch image `name`, and returns it checked and claimed. Like a launch, it waits - /// [`CLAIM_RELEASE_WAIT`](scratch::CLAIM_RELEASE_WAIT) for a claim that was released: on - /// Linux, a process that another test spawns holds a copy of the claim until it executes its - /// program. - fn checked_scratch(&self, name: &str) -> CheckedScratch { - let path = self.path(name); - ScratchBinding::bind(&path) - .unwrap() - .check(&path, Instant::now() + scratch::CLAIM_RELEASE_WAIT) - .unwrap() - } -} - -/// Returns the kind and the message of the error of `result`. -fn failure(result: Result) -> (ErrorKind, String) { - let error = result.unwrap_err(); - (error.kind(), error.to_string()) -} - -/// Returns the value of each `option` of `arguments`, in order. -fn values<'a>(arguments: &'a [String], option: &str) -> Vec<&'a str> { - arguments - .windows(2) - .filter(|pair| pair[0] == option) - .map(|pair| pair[1].as_str()) - .collect() -} - -/// Returns the number of the descriptor of `file`. -#[cfg(target_os = "linux")] -fn number(file: &fs::File) -> i32 { - use std::os::fd::AsRawFd; - - file.as_raw_fd() -} - -/// Returns the final path of the staged file `name`, by which a prepared launch names it on -/// Windows. -#[cfg(windows)] -fn final_path(staged: &Staged, name: &str) -> String { - fs::canonicalize(staged.path(name)) - .unwrap() - .to_str() - .unwrap() - .to_owned() -} - -/// Returns the value of the handle of `object`. -#[cfg(windows)] -fn raw(object: &impl std::os::windows::io::AsHandle) -> usize { - use std::os::windows::io::AsRawHandle; - - object.as_handle().as_raw_handle() as usize -} - -#[test] -fn prepared_launches_hand_the_vmm_the_checked_files() { - let staged = Staged::new(); - let launch = staged.launch(); - let (binding, distro) = staged.checked_distro("distro.erofs"); - let scratch = staged.checked_scratch("scratch.ext4"); - let plan = launch.bound_plan(&binding).unwrap(); - let rendered = plan.openvmm_arguments().unwrap(); - #[cfg(target_os = "linux")] - let (distro_number, scratch_number) = (number(distro.file()), number(scratch.file())); - let prepared = - PreparedLaunch::new(plan.clone(), distro, Some(scratch), USER_COMMAND_LINE).unwrap(); - assert_eq!(prepared.plan(), &plan); - assert_eq!( - prepared.kernel_command_line(), - plan.kernel_command_line(USER_COMMAND_LINE).unwrap() - ); - assert_eq!( - prepared.distro().path().to_str().unwrap(), - staged.text("distro.erofs") - ); - assert_eq!( - prepared.scratch().unwrap().path().to_str().unwrap(), - staged.text("scratch.ext4") - ); - - // On Linux, the arguments name the checked files as descriptors that the VMM inherits, which - // reach them; on Windows, by their final paths, which the checked files hold in place. - #[cfg(target_os = "linux")] - { - use std::os::fd::{AsFd, AsRawFd}; - - let expected: Vec = rendered - .iter() - .map(|argument| { - argument - .replace( - &format!("distro:file:{},ro", staged.text("distro.erofs")), - &format!("distro:file:/proc/self/fd/{distro_number},ro"), - ) - .replace( - &format!("scratch:file:{}", staged.text("scratch.ext4")), - &format!("scratch:file:/proc/self/fd/{scratch_number}"), - ) - }) - .collect(); - assert_ne!(expected, rendered); - assert_eq!(prepared.openvmm_arguments(), expected); - let inherited: Vec = prepared - .inherited() - .iter() - .map(|inherited| inherited.as_fd().as_raw_fd()) - .collect(); - assert_eq!(inherited, [distro_number, scratch_number]); - for (number, name) in [ - (distro_number, "distro.erofs"), - (scratch_number, "scratch.ext4"), - ] { - assert_eq!( - fs::read_link(format!("/proc/self/fd/{number}")).unwrap(), - fs::canonicalize(staged.path(name)).unwrap() - ); - } - } - #[cfg(windows)] - { - let expected: Vec = rendered - .iter() - .map(|argument| { - argument - .replace( - &format!("distro:file:{},ro", staged.text("distro.erofs")), - &format!("distro:file:{},ro", final_path(&staged, "distro.erofs")), - ) - .replace( - &format!("scratch:file:{}", staged.text("scratch.ext4")), - &format!("scratch:file:{}", final_path(&staged, "scratch.ext4")), - ) - }) - .collect(); - assert_ne!(expected, rendered); - assert_eq!(prepared.openvmm_arguments(), expected); - // The VMM inherits the checked files, which hold their final paths for it, and the - // claim's handle of the scratch image, which no argument names, and which is not its - // guard. - let inherited: Vec = prepared.inherited().iter().map(raw).collect(); - let scratch = prepared.scratch().unwrap(); - assert_eq!( - inherited, - [ - raw(prepared.distro().file()), - raw(scratch.file()), - raw(scratch.claim()) - ] - ); - assert_ne!(raw(scratch.claim()), raw(scratch.file())); - } - - // The plan, which callers persist, keeps the real paths. - let persisted = serde_json::to_value(prepared.plan()).unwrap(); - assert_eq!(persisted["distro"]["path"], staged.text("distro.erofs")); - assert_eq!(persisted["upper"]["path"], staged.text("scratch.ext4")); -} - -#[test] -fn an_upper_in_ram_takes_no_scratch_image() { - let staged = Staged::new(); - let (_, distro) = staged.checked_distro("distro.erofs"); - #[cfg(target_os = "linux")] - let distro_number = number(distro.file()); - let plan = SandboxPlan::new( - SandboxDistro::new( - &staged.text("distro.erofs"), - RootImage::erofs(Uuid::parse(UUID)), - ), - SandboxUpper::RAM, - SandboxSettings::default(), - ) - .unwrap(); - let prepared = PreparedLaunch::new(plan.clone(), distro, None, "").unwrap(); - assert!(prepared.scratch().is_none()); - // The VMM inherits the checked layer alone. - assert_eq!(prepared.inherited().len(), 1); - let blocks = values(prepared.openvmm_arguments(), "--microvm-sandbox-block"); - #[cfg(target_os = "linux")] - assert_eq!( - blocks, - [format!("distro:file:/proc/self/fd/{distro_number},ro")] - ); - #[cfg(windows)] - { - assert_eq!( - blocks, - [format!( - "distro:file:{},ro", - final_path(&staged, "distro.erofs") - )] - ); - let inherited: Vec = prepared.inherited().iter().map(raw).collect(); - assert_eq!(inherited, [raw(prepared.distro().file())]); - } - assert_eq!( - prepared.kernel_command_line(), - plan.kernel_command_line("").unwrap() - ); -} - -#[test] -fn gpt_disks_are_prepared_from_their_bindings() { - let staged = Staged::new(); - fs::write(staged.path("distro.img"), b"gpt disk\n").unwrap(); - let path = staged.text("distro.img"); - let layer = SandboxLayer::gpt(LayerRole::Distro, &path, UUID); - let binding = DistroBinding::import(&layer, PolicyVerdict::ACCEPTED).unwrap(); - let distro = binding.check(&layer, DISTRO_POLICY_REVISION).unwrap(); - #[cfg(target_os = "linux")] - let distro_number = number(distro.file()); - let plan = SandboxPlan::new( - SandboxDistro::new(&path, RootImage::gpt(Uuid::parse(UUID))), - SandboxUpper::RAM, - SandboxSettings::default(), - ) - .unwrap(); - let prepared = PreparedLaunch::new(plan.clone(), distro, None, "").unwrap(); - let token = format!("nvx_layer=distro,0xd0003000,gpt:{UUID}"); - assert!( - prepared - .kernel_command_line() - .split(' ') - .any(|t| t == token), - "{}", - prepared.kernel_command_line() - ); - let blocks = values(prepared.openvmm_arguments(), "--microvm-sandbox-block"); - #[cfg(target_os = "linux")] - assert_eq!( - blocks, - [format!("distro:file:/proc/self/fd/{distro_number},ro")] - ); - #[cfg(windows)] - assert_eq!( - blocks, - [format!( - "distro:file:{},ro", - final_path(&staged, "distro.img") - )] - ); - drop(prepared); - - // The disk's binding prepares no plan of an EROFS image at its path. - let distro = binding.check(&layer, DISTRO_POLICY_REVISION).unwrap(); - let erofs = plan.with_image(RootImage::erofs(Uuid::parse(UUID))); - assert_eq!( - failure(PreparedLaunch::new(erofs, distro, None, "")), - ( - ErrorKind::Unsupported, - format!( - "the plan's image is not the gpt layer with identity {UUID} that the distro \ - binding of {path} names" - ) - ) - ); -} - -#[test] -fn prepared_launches_take_only_the_checked_files_of_their_plan() { - let staged = Staged::new(); - let launch = staged.launch(); - let distro = staged.text("distro.erofs"); - let scratch = staged.text("scratch.ext4"); - let refused = |message: String| (ErrorKind::Unsupported, message); - let plan = |binding: &DistroBinding| launch.bound_plan(binding).unwrap(); - - // Another layer than the plan's image, even with the same content and identity. - let (binding, _) = staged.checked_distro("distro.erofs"); - let (_, other) = staged.checked_distro("other.erofs"); - assert_eq!( - failure(PreparedLaunch::new( - plan(&binding), - other, - Some(staged.checked_scratch("scratch.ext4")), - "" - )), - refused(format!( - "the checked distro layer {} is not the plan's image {distro}", - staged.text("other.erofs") - )) - ); - - // A layer whose binding names another identity than the plan's image. - let (binding, checked) = staged.checked_distro("distro.erofs"); - let other_image = launch - .plan() - .with_image(RootImage::erofs(Uuid::parse(OTHER_UUID))); - assert_eq!( - failure(PreparedLaunch::new( - other_image, - checked, - Some(staged.checked_scratch("scratch.ext4")), - "" - )), - refused(format!( - "the plan's image is not the erofs layer with identity {UUID} that the distro binding \ - of {distro} names" - )) - ); - - // Another scratch image than the plan's, none for a plan on scratch, and one for an upper in - // RAM. - let (_, checked) = staged.checked_distro("distro.erofs"); - assert_eq!( - failure(PreparedLaunch::new( - plan(&binding), - checked, - Some(staged.checked_scratch("other.ext4")), - "" - )), - refused(format!( - "the checked scratch image {} is not the plan's scratch image {scratch}", - staged.text("other.ext4") - )) - ); - let (_, checked) = staged.checked_distro("distro.erofs"); - assert_eq!( - failure(PreparedLaunch::new(plan(&binding), checked, None, "")), - refused(format!( - "the plan's upper is on the scratch image {scratch}, which was not checked" - )) - ); - let (_, checked) = staged.checked_distro("distro.erofs"); - let ram = SandboxPlan::new( - SandboxDistro::new(&distro, RootImage::erofs(Uuid::parse(UUID))), - SandboxUpper::RAM, - SandboxSettings::default(), - ) - .unwrap(); - assert_eq!( - failure(PreparedLaunch::new( - ram, - checked, - Some(staged.checked_scratch("scratch.ext4")), - "" - )), - refused(format!( - "the plan's upper is in RAM, which takes no scratch image, not {scratch}" - )) - ); - - // A user's command line that the plan refuses fails the preparation. - let (_, checked) = staged.checked_distro("distro.erofs"); - assert_eq!( - failure(PreparedLaunch::new( - plan(&binding), - checked, - Some(staged.checked_scratch("scratch.ext4")), - "nvx_sandbox=0" - )), - refused("nvx_sandbox is owned by the sandbox profile".to_owned()) - ); -} - -#[cfg(windows)] -#[test] -fn prepared_launches_refuse_final_paths_that_openvmm_cannot_take() { - let staged = Staged::new(); - let refused = |message: String| (ErrorKind::Unsupported, message); - let delimited = |planned: &str, name: &str| { - refused(format!( - "disk paths containing commas, semicolons, or unbalanced brackets are unsupported: \ - {} resolves to {}", - staged.text(planned), - final_path(&staged, name) - )) - }; - - for (link, target) in [ - ("layers", "real;layers"), - ("unclosed-layers", "real[layers"), - ("unopened-layers", "real]layers"), - ("reversed-layers", "real][layers"), - ] { - let layers = staged.path(target); - fs::create_dir(&layers).unwrap(); - for name in ["distro.erofs", "distro.erofs.manifest.json"] { - fs::copy(staged.path(name), layers.join(name)).unwrap(); - } - junction(&staged.path(link), &layers); - let planned = format!("{link}\\distro.erofs"); - let resolved = format!("{target}\\distro.erofs"); - let layer = staged.layer(&planned); - let binding = DistroBinding::verify(&layer).unwrap(); - let checked = binding.check(&layer, DISTRO_POLICY_REVISION).unwrap(); - let launch = SandboxLaunch::new( - vec![layer], - SandboxUpper::scratch(&staged.text("scratch.ext4")), - SandboxSettings::default(), - ) - .unwrap(); - assert_eq!( - failure(PreparedLaunch::new( - launch.bound_plan(&binding).unwrap(), - checked, - Some(staged.checked_scratch("scratch.ext4")), - "" - )), - delimited(&planned, &resolved) - ); - } - - let plan_on = |scratch: &str| { - let (binding, checked) = staged.checked_distro("distro.erofs"); - let launch = SandboxLaunch::new( - vec![staged.layer("distro.erofs")], - SandboxUpper::scratch(&staged.text(scratch)), - SandboxSettings::default(), - ) - .unwrap(); - (launch.bound_plan(&binding).unwrap(), checked) - }; - for (link, target) in [ - ("images", "real,images"), - ("unclosed-images", "real[images"), - ("unopened-images", "real]images"), - ("reversed-images", "real][images"), - ] { - let images = staged.path(target); - fs::create_dir(&images).unwrap(); - fs::copy(staged.path("scratch.ext4"), images.join("scratch.ext4")).unwrap(); - junction(&staged.path(link), &images); - let planned = format!("{link}\\scratch.ext4"); - let resolved = format!("{target}\\scratch.ext4"); - let (plan, checked) = plan_on(&planned); - assert_eq!( - failure(PreparedLaunch::new( - plan, - checked, - Some(staged.checked_scratch(&planned)), - "" - )), - delimited(&planned, &resolved) - ); - } - - // A scratch image whose name on disk ends in `.vhd`, which the plan's path names in capitals: - // OpenVMM, comparing extensions case-sensitively, would attach the plan's path as a raw - // image, but would open the final path as a VHD. - fs::write(staged.path("image.vhd"), b"scratch image\n").unwrap(); - let (plan, checked) = plan_on("IMAGE.VHD"); - assert_eq!( - failure(PreparedLaunch::new( - plan, - checked, - Some(staged.checked_scratch("IMAGE.VHD")), - "" - )), - refused(format!( - "disk paths ending in .vhd are unsupported, since OpenVMM would not attach them as \ - raw images: {} resolves to {}", - staged.text("IMAGE.VHD"), - final_path(&staged, "image.vhd") - )) - ); -} - -#[cfg(windows)] -#[test] -fn prepared_launches_accept_balanced_brackets_in_final_paths() { - let staged = Staged::new(); - let images = staged.path("real[[images]]"); - fs::create_dir(&images).unwrap(); - for name in ["distro.erofs", "distro.erofs.manifest.json", "scratch.ext4"] { - fs::copy(staged.path(name), images.join(name)).unwrap(); - } - junction(&staged.path("images"), &images); - let (binding, checked) = staged.checked_distro("images\\distro.erofs"); - let launch = SandboxLaunch::new( - vec![staged.layer("images\\distro.erofs")], - SandboxUpper::scratch(&staged.text("images\\scratch.ext4")), - SandboxSettings::default(), - ) - .unwrap(); - let prepared = PreparedLaunch::new( - launch.bound_plan(&binding).unwrap(), - checked, - Some(staged.checked_scratch("images\\scratch.ext4")), - "", - ) - .unwrap(); - assert_eq!( - values(prepared.openvmm_arguments(), "--microvm-sandbox-block"), - [ - format!( - "distro:file:{},ro", - final_path(&staged, "real[[images]]\\distro.erofs") - ), - format!( - "scratch:file:{}", - final_path(&staged, "real[[images]]\\scratch.ext4") - ), - ] - ); -} - -#[cfg(windows)] -#[test] -fn a_release_through_a_process_that_inherited_nothing_closes_none_of_its_handles() { - use std::ffi::OsString; - - use crate::host::supervisor::{self, VmmCommand}; - use crate::protocol::CAPABILITY_LEN; - - let staged = Staged::new(); - let launch = staged.launch(); - let (binding, distro) = staged.checked_distro("distro.erofs"); - let prepared = PreparedLaunch::new( - launch.bound_plan(&binding).unwrap(), - distro, - Some(staged.checked_scratch("scratch.ext4")), - "", - ) - .unwrap(); - // A process that runs until it is ended, and that inherited none of the prepared launch's - // handles, whose own handles the release must leave alone. - let system = std::env::var_os("SystemRoot").unwrap(); - let command = VmmCommand::new( - Path::new(&system).join("System32").join("PING.EXE"), - ["-n", "60", "127.0.0.1"].map(OsString::from).to_vec(), - staged.directory.path(), - ); - let log = fs::File::create(staged.path("other.log")).unwrap(); - let other = supervisor::spawn(&command, &[1; CAPABILITY_LEN], log).unwrap(); - let process = other.process().unwrap(); - let released = failure(prepared.release(&process)); - other.kill_and_wait(); - assert_eq!( - released, - ( - ErrorKind::Io, - format!( - "cannot release the guard of {}, which VMM process {} inherited", - staged.text("distro.erofs"), - process.reference().pid - ) - ) - ); - // The release closed the checked files nonetheless, which no longer hold the images. - for name in ["distro.erofs", "scratch.ext4"] { - fs::rename(staged.path(name), staged.path("moved")).unwrap(); - fs::rename(staged.path("moved"), staged.path(name)).unwrap(); - } -} - -#[test] -fn staging_checking_and_preparing_a_launch_read_no_content() { - let staged = Staged::new(); - let launch = staged.launch(); - let scratch = PathBuf::from(staged.text("scratch.ext4")); - - // Staging: a caller that verified the layer imports its binding, with its manifest, and binds - // the scratch image. Then the checks and the preparation of the launch. On this host, every - // read and mapping of content through what they open fails. - let binding = - distro::import_with(launch.distro(), PolicyVerdict::ACCEPTED, METADATA_ONLY).unwrap(); - assert!(binding.manifest().is_some()); - let scratch_binding = scratch::bind_with(&scratch, WRITE_ONLY).unwrap(); - let checked = binding - .check_with(launch.distro(), DISTRO_POLICY_REVISION, METADATA_ONLY) - .unwrap(); - let checked_scratch = scratch_binding - .check_with(&scratch, Instant::now(), WRITE_ONLY) - .unwrap(); - let prepared = PreparedLaunch::new( - launch.bound_plan(&binding).unwrap(), - checked, - Some(checked_scratch), - USER_COMMAND_LINE, - ) - .unwrap(); - let mut byte = [0]; - assert!(prepared.distro().file().read(&mut byte).is_err()); - assert!(prepared.scratch().unwrap().file().read(&mut byte).is_err()); - - // Verification, the positive control, reads content: it fails on that host... - let (kind, message) = failure(distro::verify_with(launch.distro(), METADATA_ONLY)); - assert_eq!(kind, ErrorKind::Unsupported); - assert!( - message.starts_with(&format!( - "invalid sandbox distro manifest: {:?}: ", - staged.path("distro.erofs.manifest.json") - )), - "{message}" - ); - // ...and with reads allowed, binds the layer as the staging caller did. - assert_eq!(DistroBinding::verify(launch.distro()).unwrap(), binding); -} - -#[test] -fn no_plan_encodes_root_as_the_workload_identity() { - // The VMM receives the plan's identity, which is never root's: a root profile is the - // adapter's choice, which it makes otherwise. - let staged = Staged::new(); - for identity in [(0, 0), (0, 1000), (1000, 0)] { - let settings = SandboxSettings { - workload_identity: identity, - ..SandboxSettings::default() - }; - let (kind, message) = failure(SandboxPlan::new( - SandboxDistro::new( - &staged.text("distro.erofs"), - RootImage::erofs(Uuid::parse(UUID)), - ), - SandboxUpper::RAM, - settings, - )); - assert_eq!(kind, ErrorKind::Unsupported, "{message}"); - } - let (_, distro) = staged.checked_distro("distro.erofs"); - let prepared = PreparedLaunch::new( - staged.launch().plan().clone(), - distro, - Some(staged.checked_scratch("scratch.ext4")), - "", - ) - .unwrap(); - assert_eq!( - values(prepared.openvmm_arguments(), "--microvm-workload-identity"), - ["65534:65534"] - ); - - // An adapter's plan names no workload identity: its own guest chooses one, root's too. - fs::write(staged.path("distro.img"), b"gpt disk\n").unwrap(); - let path = staged.text("distro.img"); - let layer = SandboxLayer::gpt(LayerRole::Distro, &path, UUID); - let binding = DistroBinding::import(&layer, PolicyVerdict::ACCEPTED).unwrap(); - let plan = SandboxPlan::adapter( - SandboxDistro::new(&path, RootImage::gpt(Uuid::parse(UUID))), - SandboxUpper::RAM, - ); - let user = "console=hvc0 nvx_control_tty=hvc2"; - let prepared = PreparedLaunch::new( - plan.clone(), - binding.check(&layer, DISTRO_POLICY_REVISION).unwrap(), - None, - user, - ) - .unwrap(); - assert!(values(prepared.openvmm_arguments(), "--microvm-workload-identity").is_empty()); - assert_eq!( - prepared.kernel_command_line(), - plan.kernel_command_line(user).unwrap() - ); -} - -#[test] -fn an_adapter_s_launch_renders_complete_arguments_that_fit_a_command_line() { - use crate::Hypervisor; - use crate::host::launch::{PROCESS_COMMAND_LINE_LIMIT, process_command_line_units}; - use crate::host::sandbox::tests::exported; - use crate::host::sandbox::{MountAccess, MountKind}; - - let staged = Staged::new(); - let (_, distro) = staged.checked_distro("distro.erofs"); - let plan = SandboxPlan::adapter( - SandboxDistro::new( - &staged.text("distro.erofs"), - RootImage::erofs(Uuid::parse(UUID)), - ), - SandboxUpper::RAM, - ); - let user = "nvx_sandbox=1"; - let prepared = PreparedLaunch::new(plan.clone(), distro, None, user).unwrap(); - let launch = VmmLaunch { - openvmm: Path::new("/o/openvmm"), - hypervisor: Hypervisor::Whp, - memory_mib: 384, - kernel: Path::new("/k/vmlinux"), - initrd: Path::new("/k/initramfs.cpio.gz"), - endpoint: "/s/control.sock", - report: Path::new("/s/outcome.json"), - }; - let command_line = plan.kernel_command_line(user).unwrap(); - let own = [ - "--single-process", - "--hypervisor", - "whp", - "--memory", - "384M", - "--kernel", - "/k/vmlinux", - "--initrd", - "/k/initramfs.cpio.gz", - "--cmdline", - command_line.as_str(), - "--virtio-console", - "none", - "--microvm-control-console", - "listen=/s/control.sock", - "--microvm-control-auth-stdin", - "--microvm-report", - "/s/outcome.json", - ] - .map(OsString::from); - // The plan's arguments, which name the checked files, then the launch's own inputs. - let devices: Vec = prepared - .openvmm_arguments() - .iter() - .map(OsString::from) - .collect(); - let arguments = prepared.adapter_arguments(&launch, None).unwrap(); - assert_eq!(arguments, [devices.clone(), own.to_vec()].concat()); - // Without a workload identity or a lifecycle, which only NVX's sandbox profile names. - for option in ["--microvm-workload-identity", "--microvm-lifecycle"] { - assert!( - !arguments.iter().any(|argument| argument == option), - "{option}" - ); - } - // The export of the adapter's shares between the two, as OpenVMM exports it. - let export = exported(&[( - "/workspace", - "/host/work", - MountAccess::ReadWrite, - MountKind::Directory, - )]); - let shared: Vec = export - .openvmm_arguments() - .into_iter() - .map(OsString::from) - .collect(); - assert!( - shared - .iter() - .any(|argument| argument == "--mount-aggregate") - ); - let arguments = prepared.adapter_arguments(&launch, Some(&export)).unwrap(); - assert_eq!(arguments, [devices, shared, own.to_vec()].concat()); - - // A command line of the limit fits, and one more code unit does not. - let base = process_command_line_units(launch.openvmm.as_os_str(), &arguments); - let program = |extra: usize| PathBuf::from(format!("/o/openvmm{}", "o".repeat(extra))); - let fitting = program(PROCESS_COMMAND_LINE_LIMIT - base); - let at_limit = VmmLaunch { - openvmm: &fitting, - ..launch - }; - let arguments = prepared - .adapter_arguments(&at_limit, Some(&export)) - .unwrap(); - assert_eq!( - process_command_line_units(fitting.as_os_str(), &arguments), - PROCESS_COMMAND_LINE_LIMIT - ); - let longer = program(PROCESS_COMMAND_LINE_LIMIT - base + 1); - let past_limit = VmmLaunch { - openvmm: &longer, - ..launch - }; - let (kind, message) = failure(prepared.adapter_arguments(&past_limit, Some(&export))); - assert_eq!(kind, ErrorKind::Unsupported, "{message}"); - assert!( - message.contains("a Windows process command line allows; map"), - "{message}" - ); - - // A launch of no memory is refused, as is the plan of NVX's sandbox profile. - let empty = VmmLaunch { - memory_mib: 0, - ..launch - }; - let (kind, message) = failure(prepared.adapter_arguments(&empty, None)); - assert_eq!(kind, ErrorKind::Unsupported, "{message}"); - assert!(message.contains("memory"), "{message}"); - let (_, distro) = staged.checked_distro("distro.erofs"); - let profile = PreparedLaunch::new( - staged.launch().plan().clone(), - distro, - Some(staged.checked_scratch("scratch.ext4")), - "", - ) - .unwrap(); - let (kind, message) = failure(profile.adapter_arguments(&launch, None)); - assert_eq!(kind, ErrorKind::Unsupported, "{message}"); - assert!(message.contains("NVX's sandbox profile"), "{message}"); -} diff --git a/nvx-rs/src/host/sandbox/scratch.rs b/nvx-rs/src/host/sandbox/scratch.rs index 87d985625..0c182328b 100644 --- a/nvx-rs/src/host/sandbox/scratch.rs +++ b/nvx-rs/src/host/sandbox/scratch.rs @@ -11,10 +11,9 @@ //! [`ScratchBinding::check`] checks it before a launch. Neither reads or maps the image's content. //! The check opens the image, compares it with the binding, claims it, compares it again, since //! the launch whose claim it waited for may have changed it, and returns it open, as a -//! [`CheckedScratch`], which a [`PreparedLaunch`](super::PreparedLaunch) hands to the VMM: a -//! replacement, or a change of kind or length, fails the check. On Linux, the file ID is the inode -//! number, which a file that is created once the image is removed can take again: a replacement -//! made that way, with the image's length, passes the check. +//! [`CheckedScratch`]: a replacement, or a change of kind or length, fails the check. On Linux, +//! the file ID is the inode number, which a file that is created once the image is removed can +//! take again: a replacement made that way, with the image's length, passes the check. //! //! # Claims //! @@ -25,25 +24,22 @@ //! [`ErrorCode::ScratchClaimed`]. //! //! - On Linux, the check opens the image for reading and writing and takes an exclusive `flock` -//! lock of that open file description. The VMM inherits the descriptor, which it opens again as -//! `/proc/self/fd/N`, and keeps it until it exits, so the claim lasts until both the caller and -//! the VMM have closed it. A process that another thread of the caller spawns while the -//! descriptor is open holds a copy of it from its fork until it executes its program, and with -//! it the claim, for that moment: a check therefore waits for a claim that should be free, for -//! [`CLAIM_RELEASE_WAIT`] for example, rather than fail at once. +//! lock of that open file description. A VMM that inherits the descriptor keeps it until it +//! exits, so the claim lasts until both the caller and the VMM have closed it. A process that +//! another thread of the caller spawns while the descriptor is open holds a copy of it from its +//! fork until it executes its program, and with it the claim, for that moment: a check +//! therefore waits for a claim that should be free, for [`CLAIM_RELEASE_WAIT`] for example, +//! rather than fail at once. //! - On Windows, the check's handle is a guard that holds no access to the image's content: it //! shares reading and writing, which OpenVMM's open of the image needs, and denies deleting and //! renaming it, so that the image's path names the checked image until the guard is released. //! Two guards exclude each other, since each asks for the delete access that the other denies. //! The check then makes sure that no other process has the image open for writing, as the VMM //! of another sandbox does: an open that denies writing succeeds only then. It then opens the -//! image for writing, sharing every access, as the claim's handle. The VMM inherits the guard -//! and the claim's handle, and uses neither: so it holds the claim, and its copy of the guard the -//! image's path, from its start, even if the caller ends before the VMM has opened the image, -//! which closes the caller's own. Once the VMM has opened the image, at authenticated Ready, the -//! caller releases the guard, its own and the VMM's copy, through -//! [`PreparedLaunch::release`](super::PreparedLaunch::release), and the VMM's handles, open for -//! writing, keep the claim. +//! image for writing, sharing every access, as the claim's handle. A VMM that inherits the +//! claim's handle holds the claim from its start, even if the caller ends before the VMM has +//! opened the image, which closes the caller's own; once the VMM has opened the image, its +//! handles, open for writing, keep the claim. //! //! Claims exclude the launches of NVX: a process that writes the image without claiming it, which //! on Linux takes no `flock` lock and on Windows does not keep the image open, is not kept out. @@ -128,15 +124,12 @@ pub struct ScratchBinding { /// A scratch image that [`ScratchBinding::check`] found unchanged and claimed, held open. /// /// On Linux, the file is the image, open for reading and writing, whose open file description -/// holds the claim, and which the VMM inherits. On Windows, it is the image's guard, which holds -/// no access to its content and denies deleting and renaming it, so that the image's path names -/// the checked image while the guard is held, and the checked scratch also holds the claim's -/// handle, open for writing; the VMM inherits both. Dropping it closes them, which releases the -/// claim, and on Windows the guard, unless a VMM that inherited them still holds them; see -/// [Claims](self#claims). -/// -/// A [`PreparedLaunch`](super::PreparedLaunch) takes it, with the path at which the check opened -/// the image, and hands the image to the VMM. +/// holds the claim, which a VMM that inherits the file keeps. On Windows, it is the image's guard, +/// which holds no access to its content and denies deleting and renaming it, so that the image's +/// path names the checked image while the guard is held, and the checked scratch also holds the +/// claim's handle, open for writing, which a VMM that inherits it keeps. Dropping it closes them, +/// which releases the claim, and on Windows the guard, unless a VMM that inherited them still +/// holds them; see [Claims](self#claims). #[derive(Debug)] pub struct CheckedScratch { /// The checked image, or on Windows its guard. @@ -144,6 +137,13 @@ pub struct CheckedScratch { /// The claim's handle on Windows: the image, open for writing and sharing every access, /// which the VMM inherits so that it holds the claim from its start. #[cfg(windows)] + #[cfg_attr( + not(feature = "cli"), + allow( + dead_code, + reason = "it holds the claim, which only the CLI hands to the VMMs that it starts" + ) + )] claim: File, /// The path at which the check opened the image. path: PathBuf, @@ -187,16 +187,10 @@ impl CheckedScratch { } /// Returns the claim's handle, which the VMM inherits on Windows. - #[cfg(windows)] + #[cfg(all(windows, feature = "cli"))] pub(crate) fn claim(&self) -> &File { &self.claim } - - /// Returns the checked image, for a prepared launch to move. - #[cfg(target_os = "linux")] - pub(crate) fn file_mut(&mut self) -> &mut File { - &mut self.file - } } impl ScratchBinding { diff --git a/nvx-rs/src/host/sandbox/scratch/tests.rs b/nvx-rs/src/host/sandbox/scratch/tests.rs index f7ce827d4..611110a12 100644 --- a/nvx-rs/src/host/sandbox/scratch/tests.rs +++ b/nvx-rs/src/host/sandbox/scratch/tests.rs @@ -259,8 +259,8 @@ fn checks_claim_the_image_without_reading_it() { assert!(!inheritable(checked.file())); #[cfg(windows)] { - assert!(checked.claim().read(&mut byte).is_err()); - assert!(!inheritable(checked.claim())); + assert!((&checked.claim).read(&mut byte).is_err()); + assert!(!inheritable(&checked.claim)); } // A second check finds the image claimed until its deadline... diff --git a/nvx-rs/src/host/sandbox/tests.rs b/nvx-rs/src/host/sandbox/tests.rs index 0f0e920d0..18165773e 100644 --- a/nvx-rs/src/host/sandbox/tests.rs +++ b/nvx-rs/src/host/sandbox/tests.rs @@ -1,22 +1,16 @@ -//! Launch-profile contracts and validation order. +//! The checks of the sandbox profile: workload identities, share sets, and the host paths of +//! shares. use std::fs; use std::path::PathBuf; use super::*; use crate::ErrorKind; -use crate::host::filesystem::{Bind, ExportChild, FileIdentity, Owner}; +use crate::host::filesystem::ExportPlan; use crate::host::paths; -use crate::protocol::root::{RootLayout, RootUpper}; mod specification; -const DISTRO: &str = "distro,distro.erofs,11111111-1111-1111-1111-111111111111"; - -fn distro() -> SandboxLayer { - SandboxLayer::parse(DISTRO).unwrap() -} - /// Returns the access policy of a share that hides `denied`, exposes `allowed` inside them again, /// and lets the workload modify only `writable`. The share tests use it too. pub(super) fn policy(denied: &[&str], allowed: &[&str], writable: &[&str]) -> MountPolicy { @@ -38,15 +32,6 @@ fn mount(value: &str, denied: &[&str]) -> SandboxMount { .unwrap() } -fn with_export(export: Option) -> Result { - SandboxLaunch::new( - vec![distro()], - SandboxUpper::scratch("scratch.ext4"), - SandboxSettings::default(), - )? - .with_export(export) -} - /// Returns `path`, an absolute POSIX path, as an absolute host path: on Windows, on drive `C:`. pub(super) fn host(path: &str) -> PathBuf { if cfg!(windows) { @@ -57,74 +42,11 @@ pub(super) fn host(path: &str) -> PathBuf { } /// Returns the text of [`host`] of `path`. +#[cfg(windows)] pub(super) fn host_text(path: &str) -> String { host(path).to_str().unwrap().to_owned() } -/// Returns the export plan that maps each of `shares`, a guest target, an absolute POSIX host path -/// that [`host`] places, an access mode, and a kind, at the root of a child of its own, in order, -/// without access-policy paths, as [`mount::plan_mounts`] plans shares. -pub(super) fn exported(shares: &[(&str, &str, MountAccess, MountKind)]) -> ExportPlan { - let children = shares - .iter() - .map(|&(_, root, access, kind)| ExportChild { - root: host(root), - file: kind == MountKind::File, - writable: access == MountAccess::ReadWrite, - denied: Vec::new(), - denied_identities: Vec::new(), - allowed: Vec::new(), - write: Vec::new(), - }) - .collect(); - let mut binds: Vec = shares - .iter() - .enumerate() - .map(|(child, &(target, _, access, _))| Bind { - child, - source: String::new(), - target: target.to_owned(), - read_only: access == MountAccess::ReadOnly, - identity: None, - }) - .collect(); - // Parents bind before children, as the planner orders them. - binds.sort_by(|left, right| { - (left.target.matches('/').count(), &left.target) - .cmp(&(right.target.matches('/').count(), &right.target)) - }); - ExportPlan { - children, - binds, - owner: Owner::Vmm, - } -} - -/// Hides `paths`, absolute POSIX paths that [`host`] places, in the child `index` of `plan`, and -/// pins with made-up identities those that a workload could move, as the planner does: those -/// strictly inside a writable child without write paths, or strictly inside one of its write -/// paths, which callers set first. -pub(super) fn deny(plan: &mut ExportPlan, index: usize, paths: &[&str]) { - let child = &mut plan.children[index]; - child.denied = paths.iter().map(|path| host(path)).collect(); - child.denied_identities = (1..) - .zip(&child.denied) - .map(|(number, path)| { - let moves = child.writable - && *path != child.root - && (child.write.is_empty() - || child - .write - .iter() - .any(|write| path != write && path.starts_with(write))); - moves.then_some(FileIdentity { - device: 1, - index: number, - }) - }) - .collect(); -} - fn message(result: Result) -> String { result.unwrap_err().to_string() } @@ -138,201 +60,6 @@ pub(super) fn native(path: &str) -> String { .to_owned() } -#[test] -fn launch_contract_builds_agent_command_line() { - let launch = SandboxLaunch::new( - vec![distro()], - SandboxUpper::scratch("scratch.ext4"), - SandboxSettings { - entrypoint: "/bin/workload".to_owned(), - args: vec!["--serve".to_owned()], - hostname: "example".to_owned(), - workload_identity: (1000, 1001), - memory_max: Some(268_435_456), - pids_max: Some(64), - }, - ) - .unwrap(); - let blocks = launch.plan().blocks(); - assert_eq!( - blocks, - [ - (BlockRole::Distro, Path::new("distro.erofs")), - (BlockRole::Scratch, Path::new("scratch.ext4")), - ] - ); - assert_eq!( - launch.plan().layout(), - RootLayout { - image: RootImage::erofs(Uuid::parse("11111111-1111-1111-1111-111111111111")), - upper: RootUpper::Scratch, - overlay: None, - journal: None, - gpt_first_partition_bound: None, - max_memory_percent: None, - } - ); - assert_eq!( - launch.plan().kernel_command_line("quiet").unwrap(), - "quiet nvx_sandbox=1 \ - nvx_layer=distro,0xd0003000,erofs:11111111-1111-1111-1111-111111111111 \ - nvx_scratch=0xd0006000,ext4 \ - nvx_entrypoint=/bin/workload nvx_hostname=example \ - nvx_arg=--serve nvx_memory_max=268435456 nvx_pids_max=65" - ); - assert_eq!( - launch.plan().openvmm_arguments().unwrap(), - [ - "--machine", - "microvm", - "--microvm-sandbox-block", - "distro:file:distro.erofs,ro", - "--microvm-sandbox-block", - "scratch:file:scratch.ext4", - "--microvm-workload-identity", - "1000:1001", - ] - ); -} - -#[test] -fn launch_contract_rejects_duplicates_and_reserved_tokens() { - let error = SandboxLaunch::new( - vec![distro(), distro()], - SandboxUpper::scratch("scratch.ext4"), - SandboxSettings::default(), - ) - .unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported); - assert_eq!(error.to_string(), "duplicate sandbox layer role(s): distro"); - let launch = with_export(None).unwrap(); - let plan = launch.plan(); - let renderings: [fn(&SandboxPlan, &str) -> Result; 1] = - [SandboxPlan::kernel_command_line]; - for render in renderings { - assert_eq!( - message(render(plan, "nvx_sandbox=0")), - "nvx_sandbox is owned by the sandbox profile" - ); - assert_eq!( - message(render(plan, r"foo=bar\ nvx_memory_max=max")), - "nvx_memory_max is owned by the sandbox profile" - ); - assert_eq!( - message(render(plan, "a\0b")), - "kernel command line contains an embedded NUL" - ); - } - let long = "x".repeat(SANDBOX_COMMAND_LINE_MAX_SIZE); - assert_eq!( - message(plan.kernel_command_line(&long)), - "sandbox kernel command line exceeds its 1024-byte x86 budget; use shorter arguments and \ - kernel parameters" - ); - assert_eq!( - message(parse_workload_identity("0:0")), - "--workload-user UID must be between 1 and 4294967294" - ); -} - -#[test] -fn launch_settings_are_checked_in_order() { - let settings = |change: fn(&mut SandboxSettings)| { - let mut settings = SandboxSettings::default(); - change(&mut settings); - message(SandboxLaunch::new( - vec![distro()], - SandboxUpper::scratch("s.ext4"), - settings, - )) - }; - assert_eq!( - message(SandboxLaunch::new( - Vec::new(), - SandboxUpper::scratch("s.ext4"), - SandboxSettings::default() - )), - "a sandbox requires exactly one read-only distro layer" - ); - assert_eq!( - message(SandboxLaunch::new( - vec![distro(); 4], - SandboxUpper::scratch("s.ext4"), - SandboxSettings::default() - )), - "duplicate sandbox layer role(s): distro" - ); - // The entrypoint is checked before the hostname. - for entrypoint in ["bin/sh", "/bin/s h", "/bin/sh\x1c"] { - let settings = SandboxSettings { - entrypoint: entrypoint.to_owned(), - hostname: "Invalid".to_owned(), - ..SandboxSettings::default() - }; - assert_eq!( - message(SandboxLaunch::new( - vec![distro()], - SandboxUpper::scratch("s.ext4"), - settings - )), - "sandbox entrypoint must be an absolute path without whitespace, control characters, or quotes" - ); - } - assert_eq!( - settings(|settings| settings.args = vec![String::new()]), - "sandbox arguments must be nonempty and contain no whitespace, control characters, or quotes" - ); - assert_eq!( - settings(|settings| settings.args = vec!["a\u{3000}b".to_owned()]), - "sandbox arguments must be nonempty and contain no whitespace, control characters, or quotes" - ); - for hostname in ["", "-a", "a-", "A", "a.b", "a\n"] { - let launch = SandboxLaunch::new( - vec![distro()], - SandboxUpper::scratch("s.ext4"), - SandboxSettings { - hostname: hostname.to_owned(), - ..SandboxSettings::default() - }, - ); - assert_eq!( - message(launch), - "sandbox hostname must be a lowercase RFC 1123 label up to 63 characters", - "{hostname:?}" - ); - } - assert_eq!( - settings(|settings| settings.workload_identity = (1000, 0)), - "sandbox workload identity must be a non-root UID:GID" - ); - assert_eq!( - settings(|settings| settings.memory_max = Some(0)), - "--memory-max must be positive" - ); - assert_eq!( - settings(|settings| settings.pids_max = Some(0)), - "--pids-max must be positive" - ); -} - -#[test] -fn launch_rejects_limits_the_guest_cgroup_cannot_store() { - let settings = |change: fn(&mut SandboxSettings)| { - let mut settings = SandboxSettings::default(); - change(&mut settings); - message(SandboxLaunch::new( - vec![distro()], - SandboxUpper::scratch("s.ext4"), - settings, - )) - }; - assert_eq!(parse_decimal::("18446744073709551616"), None); - assert_eq!( - settings(|settings| settings.pids_max = Some(4_194_304)), - "--pids-max must be at most 4194303" - ); -} - #[test] fn workload_identities_are_ascii_decimal() { assert_eq!(parse_workload_identity("1000:1001").unwrap(), (1000, 1001)); @@ -381,207 +108,6 @@ fn workload_identities_are_ascii_decimal() { ); } -#[test] -fn launch_contract_attaches_mount_and_reserves_bootstrap_tokens() { - let mut plan = exported(&[( - "/workspace", - "/host/share", - MountAccess::ReadWrite, - MountKind::Directory, - )]); - deny(&mut plan, 0, &["/host/share/secrets"]); - let launch = with_export(Some(plan)).unwrap(); - let arguments = launch.plan().openvmm_arguments().unwrap(); - let start = arguments - .iter() - .position(|argument| argument == "--mount-aggregate") - .unwrap(); - assert_eq!( - arguments[start..], - [ - "--mount-aggregate".to_owned(), - "/run/nvx/hostfs/root".to_owned(), - "--mount-child".to_owned(), - format!("0,{},rw", host_text("/host/share")), - "--mount-deny".to_owned(), - host_text("/host/share/secrets"), - ] - ); - assert!( - !launch - .plan() - .kernel_command_line("quiet") - .unwrap() - .contains("virtfs_") - ); - for (token, name) in [ - ("virtfs_dir=/other", "virtfs_dir"), - ("virtfs_tag=other", "virtfs_tag"), - ("virtfs_mode=rw", "virtfs_mode"), - ] { - assert_eq!( - message(launch.plan().kernel_command_line(token)), - format!("{name} is owned by the sandbox --mount option") - ); - } - let base = launch.plan().kernel_command_line("").unwrap(); - let fragment = mount::export_command_line_fragment(launch.export().unwrap()); - assert_eq!( - fragment, - " virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=1" - ); - let fitting = "x".repeat(SANDBOX_COMMAND_LINE_MAX_SIZE - base.len() - fragment.len() - 2); - launch.plan().kernel_command_line(&fitting).unwrap(); - assert!( - message(launch.plan().kernel_command_line(&format!("{fitting}x"))).contains("1024-byte") - ); -} - -#[test] -fn launch_contract_attaches_several_shares_as_an_aggregate() { - let shares = [ - ( - "/workspace", - "/host/work", - MountAccess::ReadWrite, - MountKind::Directory, - ), - ( - "/opt/hostedtoolcache", - "/host/tools", - MountAccess::ReadOnly, - MountKind::Directory, - ), - ( - "/srv/cache", - "/host/cache", - MountAccess::ReadOnly, - MountKind::Directory, - ), - ]; - let mut plan = exported(&shares); - deny(&mut plan, 0, &["/host/work/secrets"]); - deny(&mut plan, 1, &["/host/tools/credentials"]); - let launch = with_export(Some(plan.clone())).unwrap(); - let names: Vec = shares - .iter() - .enumerate() - .map(|(index, _)| index.to_string()) - .collect(); - let arguments = launch.plan().openvmm_arguments().unwrap(); - // Several shares are the children of one aggregate, and OpenVMM attributes each denied - // path, which is absolute, to the child that contains it. - assert!(!arguments.iter().any(|argument| argument == "--mount")); - let start = arguments - .iter() - .position(|argument| argument == "--mount-aggregate") - .unwrap(); - assert_eq!( - arguments[start..], - [ - "--mount-aggregate".to_owned(), - "/run/nvx/hostfs/root".to_owned(), - "--mount-child".to_owned(), - format!("{},{},rw", names[0], host_text("/host/work")), - "--mount-child".to_owned(), - format!("{},{},ro", names[1], host_text("/host/tools")), - "--mount-child".to_owned(), - format!("{},{},ro", names[2], host_text("/host/cache")), - "--mount-deny".to_owned(), - host_text("/host/work/secrets"), - "--mount-deny".to_owned(), - host_text("/host/tools/credentials"), - ] - ); - let fragment = mount::export_command_line_fragment(&plan); - assert_eq!( - fragment, - " virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=rw virtfs_aggregate=1" - ); - // The guest receives each bind in the mapping table, not as boot tokens. - let command_line = launch.plan().kernel_command_line("").unwrap(); - assert!(!command_line.contains("virtfs_")); - assert!(command_line.ends_with(" nvx_maps=3"), "{command_line}"); - assert!(!command_line.contains("nvx_share=")); - assert_eq!( - launch - .plan() - .mapping_table() - .iter() - .map(|map| map.target.as_str()) - .collect::>(), - ["/workspace", "/opt/hostedtoolcache", "/srv/cache"] - ); - assert_eq!( - mount::export_command_line_fragment(&exported(&shares[1..])), - " virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=ro virtfs_aggregate=1" - ); - assert_eq!( - message(launch.plan().kernel_command_line("nvx_share=0,/etc,rw")), - "nvx_share is owned by the sandbox profile" - ); - // Every share's tokens count toward the x86 budget. - let base = with_export(None) - .unwrap() - .plan() - .kernel_command_line("") - .unwrap(); - let tokens = command_line.len() - base.len(); - let fitting = - "x".repeat(SANDBOX_COMMAND_LINE_MAX_SIZE - base.len() - tokens - fragment.len() - 2); - launch.plan().kernel_command_line(&fitting).unwrap(); - assert!( - message(launch.plan().kernel_command_line(&format!("{fitting}x"))).contains("1024-byte") - ); - - let caller = ExportPlan { - owner: Owner::Caller, - ..exported(&shares[1..]) - }; - let arguments = mount::boot_shares(&caller).unwrap().arguments; - assert_eq!( - arguments - .iter() - .filter(|argument| *argument == "--mount-owner") - .count(), - 1 - ); - assert_eq!( - arguments[arguments.len() - 2..], - ["--mount-owner", "caller"] - ); -} - -#[test] -fn launch_bounds_shares_by_the_kernel_command_line() { - let launch = |count: usize| { - let shares: Vec<(String, String)> = (0..count) - .map(|index| (format!("/share/{index:02}"), format!("/host{index}"))) - .collect(); - let shares: Vec<_> = shares - .iter() - .map(|(target, root)| { - ( - target.as_str(), - root.as_str(), - MountAccess::ReadOnly, - MountKind::Directory, - ) - }) - .collect(); - with_export(Some(exported(&shares))).unwrap() - }; - // The managed lifecycle announces only their number, and sends them once the guest is ready. - assert!( - launch(20) - .plan() - .kernel_command_line("") - .unwrap() - .ends_with(" nvx_maps=20") - ); - assert_eq!(launch(20).plan().mapping_table().len(), 20); -} - #[test] fn launch_rejects_invalid_share_sets() { let owned = |value: &str, owner: MountOwner| { @@ -617,103 +143,7 @@ fn launch_rejects_invalid_share_sets() { } #[test] -fn launch_contract_attaches_a_file_as_an_aggregate_child() { - let settings = ( - "/etc/app/settings.json", - "/host/settings.json", - MountAccess::ReadOnly, - MountKind::File, - ); - let launch = with_export(Some(exported(&[settings]))).unwrap(); - // Even alone, a file is the child of an aggregate, because the guest mounts the device's - // root, which is a directory. - let arguments = launch.plan().openvmm_arguments().unwrap(); - assert!(!arguments.iter().any(|argument| argument == "--mount")); - let start = arguments - .iter() - .position(|argument| argument == "--mount-aggregate") - .unwrap(); - assert_eq!( - arguments[start..], - [ - "--mount-aggregate".to_owned(), - "/run/nvx/hostfs/root".to_owned(), - "--mount-child".to_owned(), - format!("0,{},ro,file", host_text("/host/settings.json")), - ] - ); - assert_eq!( - mount::export_command_line_fragment(launch.export().unwrap()), - " virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=ro virtfs_aggregate=1" - ); - assert!( - launch - .plan() - .kernel_command_line("") - .unwrap() - .ends_with(" nvx_maps=1") - ); - let mappings = launch.plan().mapping_table(); - assert_eq!(mappings[0].target, "/etc/app/settings.json"); - assert_eq!(mappings[0].source, "0"); - assert!(launch.export().unwrap().children[0].file); - - // Files and directories share one aggregate, and only a file's token is marked. - let shares = [ - ( - "/workspace", - "/host/work", - MountAccess::ReadWrite, - MountKind::Directory, - ), - settings, - ( - "/results/output.txt", - "/host/output.txt", - MountAccess::ReadWrite, - MountKind::File, - ), - ]; - let mut plan = exported(&shares); - deny(&mut plan, 0, &["/host/work/secrets"]); - let mixed = with_export(Some(plan)).unwrap(); - assert!( - mixed - .plan() - .kernel_command_line("") - .unwrap() - .ends_with(" nvx_maps=3") - ); - assert_eq!( - mixed - .plan() - .mapping_table() - .iter() - .map(|map| map.target.as_str()) - .collect::>(), - [ - "/workspace", - "/results/output.txt", - "/etc/app/settings.json" - ] - ); - assert_eq!( - mixed - .export() - .unwrap() - .children - .iter() - .map(|child| child.file) - .collect::>(), - [false, true, true] - ); - let arguments = mixed.plan().openvmm_arguments().unwrap(); - let denied = arguments - .iter() - .position(|argument| argument == "--mount-deny") - .unwrap(); - assert_eq!(arguments[denied + 1], host_text("/host/work/secrets")); - +fn file_shares_take_no_policy_paths_or_reserved_targets() { // A file's mode applies to the whole file, so no policy path names one. for policy in [ policy(&["x"], &[], &[]), @@ -778,7 +208,7 @@ fn launch_contract_attaches_a_file_as_an_aggregate_child() { } } -/// A sandbox's distro layer and scratch image in a temporary directory. +/// The host directories and files of shares in a temporary directory. struct Fixture { /// The temporary directory. directory: tempfile::TempDir, @@ -787,8 +217,6 @@ struct Fixture { impl Fixture { fn new() -> Self { let directory = tempfile::tempdir().unwrap(); - fs::write(directory.path().join("distro.erofs"), b"layer").unwrap(); - fs::write(directory.path().join("scratch.ext4"), b"scratch").unwrap(); Self { directory } } @@ -798,24 +226,22 @@ impl Fixture { path.to_str().unwrap().to_owned() } - fn launch(&self) -> SandboxLaunch { - SandboxLaunch::new( - vec![SandboxLayer::new( - LayerRole::Distro, - &self.path("distro.erofs"), - "11111111-1111-1111-1111-111111111111", - )], - SandboxUpper::scratch(&self.path("scratch.ext4")), - SandboxSettings::default(), - ) - .unwrap() + /// Returns the export plan of `mounts`, which [`mount::plan_mounts`] checks against the host. + fn planned(&self, mounts: Vec) -> Result { + Ok(mount::plan_mounts(&mounts)?.expect("shares have an export plan")) } +} - /// Returns the launch with the export plan of `mounts`, which [`mount::plan_mounts`] checks - /// against the host. - fn planned(&self, mounts: Vec) -> Result { - self.launch().with_export(mount::plan_mounts(&mounts)?) - } +/// Checks `export` against the host again, as a start of a provisioned sandbox does: its +/// directories and files must still not overlap, and it must pass [`ExportPlan::verify`]. +fn validated(export: &ExportPlan) -> Result<()> { + validated_with(export, &mount::HostQueries::native()) +} + +/// Implements [`validated`] with the given host queries. +fn validated_with(export: &ExportPlan, host: &mount::HostQueries<'_>) -> Result<()> { + mount::check_distinct_export_roots(export, host)?; + export.verify() } fn parse(value: &str, policy: MountPolicy) -> SandboxMount { @@ -900,7 +326,7 @@ fn launch_validation_rejects_overlapping_or_foreign_host_paths() { .unwrap_err(); assert!(error.to_string().contains("not inside"), "{error}"); } - let validated = fixture + let export = fixture .planned(vec![ parse( &format!("/workspace,{work},rw"), @@ -912,8 +338,8 @@ fn launch_validation_rejects_overlapping_or_foreign_host_paths() { ), ]) .unwrap(); - validated.validated().unwrap(); - assert_eq!(validated.export().unwrap().children.len(), 2); + validated(&export).unwrap(); + assert_eq!(export.children.len(), 2); // One share's policy paths must lie in it too. let error = fixture .planned(vec![parse( @@ -1001,18 +427,18 @@ fn launch_validation_rejects_shares_that_came_to_overlap() { ) .unwrap() }; - let launch = fixture + let export = fixture .planned(vec![ file("/srv/settings.json", "settings.json"), file("/srv/token", "token"), ]) .unwrap(); - launch.validated().unwrap(); + validated(&export).unwrap(); // A hard link that replaced one shared file with the other after provisioning would export // one file twice. fs::remove_file(fixture.path("token")).unwrap(); fs::hard_link(fixture.path("settings.json"), fixture.path("token")).unwrap(); - let error = launch.validated().unwrap_err(); + let error = validated(&export).unwrap_err(); assert_eq!(error.kind(), ErrorKind::Unsupported); assert!( error @@ -1025,7 +451,7 @@ fn launch_validation_rejects_shares_that_came_to_overlap() { // after provisioning would let one share reach the other's files under its access mode. fs::create_dir_all(fixture.path("tools")).unwrap(); fs::create_dir_all(fixture.path("work")).unwrap(); - let launch = fixture + let export = fixture .planned(vec![ parse( &format!("/workspace,{},rw", fixture.path("work")), @@ -1037,10 +463,8 @@ fn launch_validation_rejects_shares_that_came_to_overlap() { ), ]) .unwrap(); - launch.validated().unwrap(); - let roots: Vec = launch - .export() - .unwrap() + validated(&export).unwrap(); + let roots: Vec = export .children .iter() .map(|child| child.root.clone()) @@ -1056,7 +480,7 @@ fn launch_validation_rejects_shares_that_came_to_overlap() { identity: &bound, ..mount::HostQueries::native() }; - let error = launch.plan().validated_with(&queries).unwrap_err(); + let error = validated_with(&export, &queries).unwrap_err(); assert_eq!(error.kind(), ErrorKind::Unsupported); assert!( error @@ -1089,15 +513,15 @@ fn launch_validation_requires_plain_mount_directory() { assert_eq!(launch().unwrap_err().to_string(), expected); fs::remove_file(&share).unwrap(); fs::create_dir(&share).unwrap(); - let validated = launch().unwrap(); - validated.validated().unwrap(); + let export = launch().unwrap(); + validated(&export).unwrap(); let canonical = fs::canonicalize(&share).unwrap(); // The planner leaves out Windows' verbatim prefix. let canonical = canonical .to_str() .and_then(|text| text.strip_prefix(r"\\?\")) .map_or(canonical.clone(), PathBuf::from); - assert_eq!(validated.export().unwrap().children[0].root, canonical); + assert_eq!(export.children[0].root, canonical); } #[test] @@ -1164,73 +588,11 @@ fn launch_validation_requires_the_kind_of_each_share() { } #[test] -fn launch_contract_rejects_disk_option_delimiters_and_unbalanced_brackets() { - let fixture = Fixture::new(); - for name in [ - "layer;create=1G", - "layer[1.erofs", - "layer]1.erofs", - "layer][1.erofs", - ] { - let layer = fixture.path(name); - fs::write(&layer, b"").unwrap(); - let launch = SandboxLaunch::new( - vec![SandboxLayer::new( - LayerRole::Distro, - &layer, - "11111111-1111-1111-1111-111111111111", - )], - SandboxUpper::scratch(&fixture.path("scratch.ext4")), - SandboxSettings::default(), - ) - .unwrap(); - let error = launch.validated().unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported); - assert_eq!( - error.to_string(), - format!( - "disk paths containing commas, semicolons, or unbalanced brackets are unsupported: \ - {layer}" - ) - ); - } -} - -#[test] -fn launch_validation_requires_the_layers_and_the_scratch_image() { - let fixture = Fixture::new(); - let missing = SandboxLaunch::new( - vec![SandboxLayer::new( - LayerRole::Distro, - &fixture.path("missing.erofs"), - "11111111-1111-1111-1111-111111111111", - )], - SandboxUpper::scratch(&fixture.path("scratch.ext4")), - SandboxSettings::default(), - ) - .unwrap(); - let error = missing.validated().unwrap_err(); +fn missing_files_are_named_with_their_components_alone() { + let error = require_file(Path::new("missing//file"), "OpenVMM release binary").unwrap_err(); assert_eq!(error.kind(), ErrorKind::Unavailable); assert_eq!( error.to_string(), - format!( - "distro EROFS layer not found: {}", - fixture.path("missing.erofs") - ) - ); - let scratch = fixture.launch(); - fs::remove_file(fixture.path("scratch.ext4")).unwrap(); - assert_eq!( - scratch.validated().unwrap_err().to_string(), - format!( - "ext4 scratch image not found: {}", - fixture.path("scratch.ext4") - ) - ); - assert_eq!( - require_file(Path::new("missing//file"), "OpenVMM release binary") - .unwrap_err() - .to_string(), format!( "OpenVMM release binary not found: {}", native("missing/file") diff --git a/nvx-rs/src/host/sandbox/tests/specification.rs b/nvx-rs/src/host/sandbox/tests/specification.rs index 5cd9345a3..746221464 100644 --- a/nvx-rs/src/host/sandbox/tests/specification.rs +++ b/nvx-rs/src/host/sandbox/tests/specification.rs @@ -1,29 +1,21 @@ -//! Specifications of the documented launch options and the contracts with OpenVMM and the guest. +//! Specifications of the documented options of the sandbox profile: workload identities, layers, +//! and shares. //! These tests deliberately do not specify Python's input extensions or error formatting. use std::fs; use std::path::PathBuf; use proptest::prelude::*; -use sha2::{Digest, Sha256}; -use super::mount::{aggregate_child_name, parse_mount_value, validate_mount_target}; -use super::tests::{exported, host_text}; +use super::distro::DistroFormat; +use super::mount::{parse_mount_value, validate_mount_target}; use super::*; use crate::ErrorKind; use crate::protocol::cmdline::tests::model; +use crate::uuid::Uuid; const UUID: &str = "11111111-2222-3333-4444-555555555555"; -fn launch(settings: SandboxSettings) -> SandboxLaunch { - SandboxLaunch::new( - vec![SandboxLayer::new(LayerRole::Distro, "distro.erofs", UUID)], - SandboxUpper::scratch("scratch.ext4"), - settings, - ) - .unwrap() -} - fn refused(result: Result) { assert_eq!(result.unwrap_err().kind(), ErrorKind::Unsupported); } @@ -67,20 +59,6 @@ fn layer_values_are_role_path_uuid() { assert_eq!(layer.path(), Path::new("distro.erofs")); assert_eq!(layer.format(), DistroFormat::Erofs); assert_eq!(layer.uuid(), UUID); - let launch = SandboxLaunch::new( - vec![layer], - SandboxUpper::scratch("scratch.ext4"), - SandboxSettings::default(), - ) - .unwrap(); - assert!( - launch - .plan() - .kernel_command_line("") - .unwrap() - .split_ascii_whitespace() - .any(|token| token == format!("nvx_layer=distro,0xd0003000,erofs:{UUID}")) - ); for role in ["runtime", "custom", "Distro", "", "unknown"] { refused(SandboxLayer::parse(&format!("{role},distro.erofs,{UUID}"))); } @@ -94,16 +72,6 @@ fn layer_values_are_role_path_uuid() { ] { refused(SandboxLayer::parse(value)); } - refused(SandboxLaunch::new( - Vec::new(), - SandboxUpper::scratch("scratch.ext4"), - SandboxSettings::default(), - )); - refused(SandboxLaunch::new( - vec![launch.distro().clone(); 2], - SandboxUpper::scratch("scratch.ext4"), - SandboxSettings::default(), - )); } #[test] @@ -195,256 +163,6 @@ fn mount_targets_are_canonical_and_byte_bounded() { refused(validate_mount_target(&format!("{multibyte}\u{e9}"))); } -#[test] -fn aggregate_child_name_known_answer() { - assert_eq!( - aggregate_child_name(0, "/workspace"), - "0-c52ddf65534b7b46035084358ab7902b", - ); - assert_eq!( - aggregate_child_name(17, "/workspace"), - "17-c52ddf65534b7b46035084358ab7902b", - ); -} - -#[test] -fn launch_arguments_follow_the_openvmm_contract() { - for format in [DistroFormat::Erofs, DistroFormat::Gpt] { - for identity in [(1, 2), (65534, 65534), (u32::MAX, u32::MAX)] { - let layer = match format { - DistroFormat::Erofs => SandboxLayer::new(LayerRole::Distro, "distro.img", UUID), - DistroFormat::Gpt => SandboxLayer::gpt(LayerRole::Distro, "distro.img", UUID), - }; - let launch = SandboxLaunch::new( - vec![layer], - SandboxUpper::scratch("scratch.ext4"), - SandboxSettings { - workload_identity: identity, - ..SandboxSettings::default() - }, - ) - .unwrap(); - let expected = [ - "--machine".to_owned(), - "microvm".to_owned(), - "--microvm-sandbox-block".to_owned(), - "distro:file:distro.img,ro".to_owned(), - "--microvm-sandbox-block".to_owned(), - "scratch:file:scratch.ext4".to_owned(), - "--microvm-workload-identity".to_owned(), - format!("{}:{}", identity.0, identity.1), - ]; - assert_eq!(launch.plan().openvmm_arguments().unwrap(), expected); - assert_eq!( - launch.plan().kernel_command_line("quiet").unwrap(), - format!( - "quiet nvx_sandbox=1 nvx_layer=distro,0xd0003000,{format}:{UUID} \ - nvx_scratch=0xd0006000,ext4 nvx_entrypoint=/bin/sh nvx_hostname=nvx-sandbox" - ), - ); - } - } -} - -#[test] -fn accepted_launch_shapes_keep_their_arguments_and_kernel_tokens() { - let base = "nvx_sandbox=1 \ - nvx_layer=distro,0xd0003000,erofs:11111111-1111-1111-1111-111111111111 \ - nvx_scratch=0xd0006000,ext4 nvx_entrypoint=/bin/sh nvx_hostname=nvx-sandbox"; - let cases = [ - (Vec::new(), String::new(), Vec::new()), - ( - vec![( - "/workspace", - "/host", - MountAccess::ReadWrite, - MountKind::Directory, - )], - " nvx_maps=1".to_owned(), - vec![ - "--mount-aggregate".to_owned(), - "/run/nvx/hostfs/root".to_owned(), - "--mount-child".to_owned(), - format!("0,{},rw", host_text("/host")), - ], - ), - ( - vec![ - ( - "/workspace", - "/host", - MountAccess::ReadWrite, - MountKind::Directory, - ), - ( - "/srv/cache", - "/cache", - MountAccess::ReadOnly, - MountKind::Directory, - ), - ], - " nvx_maps=2".to_owned(), - vec![ - "--mount-aggregate".to_owned(), - "/run/nvx/hostfs/root".to_owned(), - "--mount-child".to_owned(), - format!("0,{},rw", host_text("/host")), - "--mount-child".to_owned(), - format!("1,{},ro", host_text("/cache")), - ], - ), - ( - vec![( - "/etc/app/settings.json", - "/settings.json", - MountAccess::ReadOnly, - MountKind::File, - )], - " nvx_maps=1".to_owned(), - vec![ - "--mount-aggregate".to_owned(), - "/run/nvx/hostfs/root".to_owned(), - "--mount-child".to_owned(), - format!("0,{},ro,file", host_text("/settings.json")), - ], - ), - ]; - for (shares, suffix, tail) in cases { - let fragment = if shares.is_empty() { - String::new() - } else { - let mode = if shares.iter().any(|share| share.2 == MountAccess::ReadWrite) { - "rw" - } else { - "ro" - }; - format!( - " virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode={mode} virtfs_aggregate=1" - ) - }; - let padding = 1024 - base.len() - suffix.len() - fragment.len() - 2; - let launch = SandboxLaunch::new( - vec![SandboxLayer::new( - LayerRole::Distro, - "layer.erofs", - "11111111-1111-1111-1111-111111111111", - )], - SandboxUpper::scratch("scratch.ext4"), - SandboxSettings::default(), - ) - .unwrap() - .with_export((!shares.is_empty()).then(|| exported(&shares))) - .unwrap(); - let mut expected = [ - "--machine", - "microvm", - "--microvm-sandbox-block", - "distro:file:layer.erofs,ro", - "--microvm-sandbox-block", - "scratch:file:scratch.ext4", - "--microvm-workload-identity", - "65534:65534", - ] - .map(str::to_owned) - .to_vec(); - expected.extend(tail); - assert_eq!(launch.plan().openvmm_arguments().unwrap(), expected); - assert_eq!( - launch - .plan() - .kernel_command_line(&"x".repeat(padding)) - .unwrap(), - format!("{} {base}{suffix}", "x".repeat(padding)), - ); - refused(launch.plan().kernel_command_line(&"x".repeat(padding + 1))); - } -} - -#[test] -fn shares_follow_the_openvmm_and_guest_contracts() { - for (access, mode) in [ - (MountAccess::ReadOnly, "ro"), - (MountAccess::ReadWrite, "rw"), - ] { - for kind in [MountKind::Directory, MountKind::File] { - let export = exported(&[("/workspace", "/host/share", access, kind)]); - let launch = launch(SandboxSettings::default()) - .with_export(Some(export)) - .unwrap(); - let arguments = launch.plan().openvmm_arguments().unwrap(); - let command_line = launch.plan().kernel_command_line("").unwrap(); - let file = if kind == MountKind::File { ",file" } else { "" }; - assert_eq!( - &arguments[8..], - [ - "--mount-aggregate", - "/run/nvx/hostfs/root", - "--mount-child", - &format!("0,{},{mode}{file}", host_text("/host/share")), - ], - ); - assert!(!command_line.contains("nvx_share=")); - let maps = launch.plan().mapping_table(); - assert_eq!(maps[0].target, "/workspace"); - assert_eq!(maps[0].read_only, access == MountAccess::ReadOnly); - assert_eq!(maps[0].source, "0"); - assert_eq!( - launch.export().unwrap().children[0].file, - kind == MountKind::File - ); - assert!( - launch - .plan() - .kernel_command_line("") - .unwrap() - .ends_with(" nvx_maps=1") - ); - } - } -} - -#[test] -fn launch_settings_follow_the_guest_contract() { - let settings = SandboxSettings { - entrypoint: "/usr/bin/workload".to_owned(), - args: vec!["first".to_owned(), "--second=value".to_owned()], - hostname: "workload-1".to_owned(), - workload_identity: (1000, 1001), - memory_max: Some(u64::MAX), - pids_max: Some(4_194_303), - }; - assert_eq!( - launch(settings).plan().kernel_command_line("").unwrap(), - format!( - "nvx_sandbox=1 nvx_layer=distro,0xd0003000,erofs:{UUID} \ - nvx_scratch=0xd0006000,ext4 nvx_entrypoint=/usr/bin/workload \ - nvx_hostname=workload-1 nvx_arg=first nvx_arg=--second=value \ - nvx_memory_max=18446744073709551615 nvx_pids_max=4194304" - ), - ); - for (memory, pids) in [(0_u32, 1_u32), (1, 0), (1, 4_194_304)] { - refused(SandboxLaunch::new( - vec![SandboxLayer::new(LayerRole::Distro, "distro.erofs", UUID)], - SandboxUpper::scratch("scratch.ext4"), - SandboxSettings { - memory_max: Some(u64::from(memory)), - pids_max: Some(pids), - ..SandboxSettings::default() - }, - )); - } - for entrypoint in ["bin/sh", "/bin/a b", "/bin/a\"b"] { - refused(SandboxLaunch::new( - vec![SandboxLayer::new(LayerRole::Distro, "distro.erofs", UUID)], - SandboxUpper::scratch("scratch.ext4"), - SandboxSettings { - entrypoint: entrypoint.to_owned(), - ..SandboxSettings::default() - }, - )); - } -} - #[cfg(windows)] #[test] fn windows_drive_relative_and_unc_paths() { @@ -533,109 +251,17 @@ fn kernel_token_values_refuse_separators() { '\0', '\t', '\n', '\u{b}', '\u{c}', '\r', ' ', '"', '\u{a0}', '\u{120}', '\u{3000}', '\u{1c}', ] { - for settings in [ - SandboxSettings { - entrypoint: format!("/bin/a{separator}b"), - ..SandboxSettings::default() - }, - SandboxSettings { - args: vec![format!("a{separator}b")], - ..SandboxSettings::default() - }, - ] { - refused(SandboxLaunch::new( - vec![SandboxLayer::new(LayerRole::Distro, "distro.erofs", UUID)], - SandboxUpper::scratch("scratch.ext4"), - settings, - )); - } refused(validate_mount_target(&format!("/workspace/a{separator}b"))); } } -#[test] -fn caller_parameters_follow_linux_and_guest_rules() { - let launch = launch(SandboxSettings { - entrypoint: "/bin/workload".to_owned(), - memory_max: Some(268_435_456), - pids_max: Some(64), - ..SandboxSettings::default() - }); - let adapter = SandboxPlan::adapter( - SandboxDistro::new("distro.erofs", RootImage::erofs(Uuid::parse(UUID))), - SandboxUpper::RAM, - ); - for user in [ - r#""nvx_layer=other""#, - r#""virtfs_dir=/other""#, - "nvx-layer=other", - "virtfs-dir=/other", - "quiet\u{120}nvx_layer=other", - "quiet\u{a0}virtfs_dir=/other", - "\u{2003}nvx_layer=other", - "\u{2003}virtfs_dir=/other", - r#"a="nvx_layer=other virtfs_dir=/other""#, - r#"extra="ignored virtfs_dir=/other ignored""#, - "--", - "quiet -- ignored", - "value=\"unterminated", - ] { - refused(launch.plan().kernel_command_line(user)); - refused(adapter.kernel_command_line(user)); - } - for user in [ - "\u{2003}nvx_entrypoint=/bin/other", - r#"extra="ignored nvx_entrypoint=/bin/other ignored""#, - r#"extra="ignored nvx_memory_max=max ignored""#, - r#"extra="ignored nvx_pids_max=0 ignored""#, - ] { - refused(launch.plan().kernel_command_line(user)); - } - for user in [ - r#"a="b c" d"#, - r#"a="ordinary=value other=value""#, - r#""ordinary=value with spaces""#, - ] { - for command_line in [ - launch.plan().kernel_command_line(user).unwrap(), - adapter.kernel_command_line(user).unwrap(), - ] { - assert!(command_line.starts_with(&format!("{user} "))); - } - } - assert!(adapter.kernel_command_line("nvx_custom=adapter").is_ok()); -} - #[test] fn the_reproduction_of_microsoft_nvx_723_is_refused() { // `à`, U+00E0, is C3 A0 in UTF-8, and the kernel takes the byte 0xa0 for whitespace; it also // drops the quote that starts a parameter. - let launch = launch(SandboxSettings::default()); - for user in [ - "quiet\u{e0}nvx_entrypoint=/bin/other", - r#""nvx_entrypoint=/bin/other""#, - ] { - refused(launch.plan().kernel_command_line(user)); - } - refused(SandboxLaunch::new( - vec![SandboxLayer::new(LayerRole::Distro, "distro.erofs", UUID)], - SandboxUpper::scratch("scratch.ext4"), - SandboxSettings { - args: vec!["x\u{e0}tsc=unstable".to_owned()], - ..SandboxSettings::default() - }, - )); for target in ["/data/\u{e0}x", r#"/data/a"b"#] { refused(validate_mount_target(target)); } - // The kernel honors quoted values, so a caller's command line may hold them. - for user in [r#"a="b c" d"#, r#"quiet a="b c""#] { - let command_line = launch.plan().kernel_command_line(user).unwrap(); - assert!( - command_line.starts_with(&format!("{user} ")), - "{command_line}" - ); - } } /// Returns a strategy for a value that becomes part of a kernel token, such as an argument: @@ -665,14 +291,10 @@ proptest! { } #[test] - fn canonical_layer_uuids_reach_the_guest(bytes in any::<[u8; 16]>()) { + fn canonical_layer_uuids_round_trip(bytes in any::<[u8; 16]>()) { let uuid = Uuid::from_bytes(bytes).to_string(); let layer = SandboxLayer::parse(&format!("distro,distro.erofs,{uuid}")).unwrap(); prop_assert_eq!(layer.uuid(), &uuid); - let launch = SandboxLaunch::new(vec![layer], SandboxUpper::scratch("scratch.ext4"), SandboxSettings::default()).unwrap(); - let command_line = launch.plan().kernel_command_line("").unwrap(); - let expected = format!("nvx_layer=distro,0xd0003000,erofs:{uuid}"); - prop_assert!(command_line.split_ascii_whitespace().any(|token| token == expected)); } #[test] @@ -698,16 +320,6 @@ proptest! { } } - #[test] - fn aggregate_child_names_follow_the_restore_contract( - index in any::(), - target in any::(), - ) { - let digest = format!("{:x}", Sha256::digest(target.as_bytes())); - let expected = format!("{index}-{}", &digest[..32]); - prop_assert_eq!(aggregate_child_name(index, &target), expected); - } - #[test] fn host_paths_become_absolute(components in prop::collection::vec("dir-[a-z]{1,12}", 1..5)) { let cwd = std::env::current_dir().unwrap(); @@ -727,137 +339,17 @@ proptest! { prop_assert_eq!(absolute.absolute().unwrap(), absolute); } + /// A share's target is one value for the guest: a target in which the kernel would end a + /// parameter, with whitespace, a quote, or a NUL, is refused. #[test] - fn kernel_command_line_stays_within_1024_bytes( - argument in "[a-z0-9]{1,40}", - user_bytes in 0_usize..1200, - share_count in 0_usize..4, - writable in any::(), - ) { - let access = if writable { MountAccess::ReadWrite } else { MountAccess::ReadOnly }; - let targets: Vec = (0..share_count).map(|index| format!("/share-{index}")).collect(); - let roots: Vec = (0..share_count).map(|index| format!("/host-{index}")).collect(); - let shares: Vec<_> = targets.iter().zip(&roots) - .map(|(target, root)| (target.as_str(), root.as_str(), access, MountKind::Directory)).collect(); - let settings = SandboxSettings { args: vec![argument.clone()], ..SandboxSettings::default() }; - let launch = launch(settings).with_export( - (!shares.is_empty()).then(|| exported(&shares)), - ).unwrap(); - let user = "x".repeat(user_bytes); - { - let render = |user: &str| launch.plan().kernel_command_line(user); - let mut tokens = vec![ - "nvx_sandbox=1".to_owned(), - format!("nvx_layer=distro,0xd0003000,erofs:{UUID}"), - "nvx_scratch=0xd0006000,ext4".to_owned(), - "nvx_entrypoint=/bin/sh".to_owned(), - "nvx_hostname=nvx-sandbox".to_owned(), - format!("nvx_arg={argument}"), - ]; - if share_count != 0 { - tokens.push(format!("nvx_maps={share_count}")); - } - let base = tokens.join(" "); - prop_assert_eq!(render("").unwrap(), base.clone()); - let fragment = if share_count == 0 { - String::new() - } else { - " virtfs_dir=/run/nvx/hostfs/root virtfs_tag=microvm virtfs_mode=ro virtfs_aggregate=1".to_owned() - }; - let bytes = base.len() + user_bytes + usize::from(user_bytes != 0) + fragment.len() + 1; - match render(&user) { - Ok(command_line) => { - prop_assert!(bytes <= 1024, "{bytes}"); - let expected = if user.is_empty() { base.clone() } else { format!("{user} {base}") }; - prop_assert_eq!(command_line, expected); - } - Err(error) => { - prop_assert!(bytes > 1024, "{bytes}: {error}"); - prop_assert_eq!(error.kind(), ErrorKind::Unsupported); - } - } - let fitting = 1024 - base.len() - fragment.len() - 2; - let boundary = "x".repeat(fitting); - prop_assert_eq!(render(&boundary).unwrap().len() + fragment.len() + 1, 1024); - refused(render(&format!("{boundary}x"))); - } - } - - #[test] - fn reserved_kernel_tokens_are_always_refused( - name in "[a-z][a-z0-9_]{0,24}", - value in "[a-z0-9]{0,24}", - separator in prop::sample::select(vec![" ", "\t", "\n", "\r", "\u{b}", "\u{c}", "\u{a0}", "\u{120}"]), - quoted in any::(), - nested in any::(), - leading_unicode in any::(), - ) { - let launch = launch(SandboxSettings::default()); - for prefix in ["nvx_", "virtfs_", "nvx-", "virtfs-"] { - let token = format!("{prefix}{name}={value}"); - let token = if nested { - format!("extra=\"ignored {token} ignored\"") - } else if quoted { - format!("\"{token}\"") - } else { - token - }; - let user = if leading_unicode { - format!("\u{2003}{token}") - } else { - format!("quiet{separator}{token}") - }; - refused(launch.plan().kernel_command_line(&user)); - } - } - - /// A value that becomes one kernel token, an entrypoint, an argument, or a share's target, is - /// one parameter for the kernel: a value in which the kernel would end a parameter, with - /// whitespace, a quote, or a NUL, is refused, and the kernel reads every token of the command - /// lines of a launch whose values are accepted as one parameter, whole. - #[test] - fn accepted_values_stay_one_kernel_parameter( - entrypoint in token_value(), - argument in token_value(), - first in token_value(), - second in token_value(), - ) { + fn accepted_values_stay_one_kernel_parameter(first in token_value(), second in token_value()) { let splits = |value: &str| { value.bytes().any(|byte| matches!(byte, b'\0' | b'"') || model::isspace(byte)) }; - let entrypoint = format!("/{entrypoint}"); - let targets = [format!("/share-0/{first}"), format!("/share-1/{second}")]; - let launch = SandboxLaunch::new( - vec![SandboxLayer::new(LayerRole::Distro, "distro.erofs", UUID)], - SandboxUpper::scratch("scratch.ext4"), - SandboxSettings { - entrypoint: entrypoint.clone(), - args: vec![argument.clone()], - ..SandboxSettings::default() - }, - ); - if splits(&entrypoint) || splits(&argument) { - prop_assert!(launch.is_err(), "{:?} {:?}", entrypoint, argument); - } - for target in &targets { - if splits(target) { - prop_assert!(validate_mount_target(target).is_err(), "{:?}", target); + for target in [format!("/share-0/{first}"), format!("/share-1/{second}")] { + if splits(&target) { + prop_assert!(validate_mount_target(&target).is_err(), "{:?}", target); } } - let accepted = targets.iter().all(|target| validate_mount_target(target).is_ok()); - let (Ok(launch), true) = (launch, accepted) else { - return Ok(()); - }; - let shares: Vec<_> = targets - .iter() - .zip(["/host-0", "/host-1"]) - .map(|(target, root)| (target.as_str(), root, MountAccess::ReadWrite, MountKind::Directory)) - .collect(); - let launch = launch.with_export(Some(exported(&shares))).unwrap(); - let command_line = launch.plan().kernel_command_line("quiet").unwrap(); - let reading = model::read(command_line.as_bytes()); - prop_assert_eq!(&reading.rest, &None); - let tokens: Vec<&[u8]> = command_line.split(' ').map(str::as_bytes).collect(); - prop_assert_eq!(reading.lines(), tokens); } } diff --git a/nvx-rs/src/host/seal.rs b/nvx-rs/src/host/seal.rs index f05f60bb9..3bc05e470 100644 --- a/nvx-rs/src/host/seal.rs +++ b/nvx-rs/src/host/seal.rs @@ -229,14 +229,6 @@ pub(crate) fn open_read(path: &Path) -> io::Result { imp::open_read(path) } -/// Returns the final path of the open `file`, as `GetFinalPathNameByHandleW` gives it: the -/// absolute path, in its `\\?\` form, that names the file with every link and junction on the way -/// to it resolved. -#[cfg(windows)] -pub(crate) fn final_path(file: &File) -> io::Result { - imp::final_path(file) -} - #[cfg(target_os = "linux")] /// The Linux seal, from `statx` or `fstat` of the open file. mod imp { diff --git a/nvx-rs/src/host/seal/windows.rs b/nvx-rs/src/host/seal/windows.rs index dd9d25d62..ea0af754f 100644 --- a/nvx-rs/src/host/seal/windows.rs +++ b/nvx-rs/src/host/seal/windows.rs @@ -1,20 +1,18 @@ //! The Windows seal, from the file information of the open handle, and the opens of sealed files. -use std::ffi::{OsString, c_void}; +use std::ffi::c_void; use std::fs::{File, OpenOptions}; use std::io; use std::mem; -use std::os::windows::ffi::OsStringExt; use std::os::windows::fs::OpenOptionsExt; use std::os::windows::io::AsRawHandle; -use std::path::{Path, PathBuf}; +use std::path::Path; use windows_sys::Win32::Foundation::{FILETIME, GetLastError, HANDLE}; use windows_sys::Win32::Storage::FileSystem::{ BY_HANDLE_FILE_INFORMATION, FILE_ATTRIBUTE_DIRECTORY, FILE_FLAG_BACKUP_SEMANTICS, FILE_ID_INFO, - FILE_SHARE_READ, FILE_TYPE_DISK, FILE_TYPE_UNKNOWN, FileIdInfo, GETFINALPATHNAMEBYHANDLE_FLAGS, - GetFileInformationByHandle, GetFileInformationByHandleEx, GetFileType, - GetFinalPathNameByHandleW, VOLUME_NAME_DOS, + FILE_SHARE_READ, FILE_TYPE_DISK, FILE_TYPE_UNKNOWN, FileIdInfo, GetFileInformationByHandle, + GetFileInformationByHandleEx, GetFileType, }; use super::{FileId, FileKind, FileSeal, SEAL_VERSION, SealScheme, SealTime}; @@ -109,33 +107,5 @@ pub(super) fn open_read(path: &Path) -> io::Result { File::open(path) } -/// See [`super::final_path`]. -pub(super) fn final_path(file: &File) -> io::Result { - final_path_by(file, VOLUME_NAME_DOS) -} - -/// Returns the final path of the open `file`, which names its volume as `volume` asks. -fn final_path_by(file: &File, volume: GETFINALPATHNAMEBYHANDLE_FLAGS) -> io::Result { - let handle = file.as_raw_handle() as HANDLE; - let mut buffer = vec![0u16; 260]; - loop { - // SAFETY: the handle is open for as long as `file` is borrowed, and the buffer holds - // `buffer.len()` units. - let length = unsafe { - GetFinalPathNameByHandleW(handle, buffer.as_mut_ptr(), buffer.len() as u32, volume) - } as usize; - if length == 0 { - return Err(io::Error::last_os_error()); - } - // The path fits, and the length leaves out the NUL that follows it; or the buffer is too - // small, and the length is what the path needs, with its NUL. - if length < buffer.len() { - buffer.truncate(length); - return Ok(PathBuf::from(OsString::from_wide(&buffer))); - } - buffer.resize(length, 0); - } -} - #[cfg(test)] mod tests; diff --git a/nvx-rs/src/host/seal/windows/tests.rs b/nvx-rs/src/host/seal/windows/tests.rs index 02045e690..8f5b4d508 100644 --- a/nvx-rs/src/host/seal/windows/tests.rs +++ b/nvx-rs/src/host/seal/windows/tests.rs @@ -38,26 +38,3 @@ fn file_times_count_from_the_unix_epoch() { } ); } - -#[test] -fn final_paths_hold_no_nul_whatever_their_length() { - use std::os::windows::ffi::OsStrExt; - - // A path shorter than the first buffer, and one longer, which takes a second call with - // the length, terminator included, that the first one asked for. - let directory = tempfile::tempdir().unwrap(); - let mut deep = directory.path().to_owned(); - while deep.as_os_str().len() < 300 { - deep.push("a-directory-that-lengthens-the-final-path"); - } - std::fs::create_dir_all(&deep).unwrap(); - for path in [directory.path().join("short"), deep.join("long")] { - std::fs::write(&path, b"").unwrap(); - let file = File::open(&path).unwrap(); - let found = final_path(&file).unwrap(); - assert!(!found.as_os_str().encode_wide().any(|unit| unit == 0)); - assert_eq!(found, std::fs::canonicalize(&path).unwrap()); - } - let file = File::open(deep.join("long")).unwrap(); - assert!(final_path(&file).unwrap().as_os_str().len() > 260); -} diff --git a/nvx-rs/src/host/session/tests/managed_lifecycle.rs b/nvx-rs/src/host/session/tests/managed_lifecycle.rs index be185de6c..c0f0c67a4 100644 --- a/nvx-rs/src/host/session/tests/managed_lifecycle.rs +++ b/nvx-rs/src/host/session/tests/managed_lifecycle.rs @@ -511,7 +511,7 @@ fn a_managed_guest_runs_request_scoped_workloads_until_it_stops() { let mut capability = [0; CAPABILITY_LEN]; getrandom::fill(&mut capability).unwrap(); // OpenVMM reads its capability as soon as it starts, so its input already holds the - // capability and its end, as `nvx::host::supervisor::spawn` prepares it. + // capability and its end, as `nvx::host::vm::spawn` prepares it. let (input, mut writer) = io::pipe().unwrap(); writer.write_all(&capability).unwrap(); drop(writer); diff --git a/nvx-rs/src/host/snapshot.rs b/nvx-rs/src/host/snapshot.rs index 5c8328efa..7f206a927 100644 --- a/nvx-rs/src/host/snapshot.rs +++ b/nvx-rs/src/host/snapshot.rs @@ -7,18 +7,17 @@ //! [`finish_capture`] validates an exited capture and writes its key and informational //! description without publishing or removing any VM-directory entry. //! -//! [`SnapshotKey`] derives a compatibility key from a capture's class and manifest. +//! [`SnapshotKey`] derives a compatibility key from a capture's context and manifest. //! [`SnapshotManifest::read`] reads the manifest that OpenVMM wrote, and //! [`TimeContract::eligibility`] checks its restore window at a [`HostTime`]. //! [`Snapshot::accepts`] checks a requested VM against that verified metadata. //! //! # Canonical encoding //! -//! A key and a class each have one canonical encoding, which [`SnapshotKey::to_bytes`] and -//! [`CaptureClass::to_bytes`] write. A key's [`KeyIdentity`] digests its compatibility prefix, -//! excluding the appended capture-time anchors and identities; a [`ClassIdentity`] digests the -//! complete class encoding. Both use SHA-256 after a domain of their own. Each encoding starts -//! with its version, and each value in it has exactly one encoding: +//! A key has one canonical encoding, which [`SnapshotKey::to_bytes`] writes. Its [`KeyIdentity`] +//! digests its compatibility prefix, excluding the appended capture-time anchors and identities, +//! with SHA-256 after a domain of its own. The encoding starts with its version, and each value in +//! it has exactly one encoding: //! //! - An integer is its fixed number of little-endian bytes: one for a `u8`, two for a `u16`, four //! for a `u32`, and eight for a `u64`. @@ -30,7 +29,7 @@ //! - A list is its count, a `u32`, then its items in order. //! //! [`SnapshotKey::from_bytes`] refuses any other encoding, and bytes left after it. The vectors -//! in `tests/vectors/snapshot-key-v1.txt` fix version 1 of a key's encoding and of a class's. +//! in `tests/vectors/snapshot-key-v1.txt` fix version 1 of a key's encoding. mod canonical; mod capture; @@ -61,7 +60,7 @@ use crate::{Error, Hypervisor, Result}; pub(super) use self::capture::check_snapshot; pub use self::capture::{CapturedSnapshot, finish_capture}; pub use self::eligibility::{Downtime, DowntimeSource, Eligibility, HostTime, MAX_DOWNTIME_NS}; -pub use self::key::{CaptureClass, ClassIdentity, Invariants, KeyIdentity, SnapshotKey}; +pub use self::key::{Invariants, KeyIdentity, SnapshotKey}; pub use self::manifest::{ BlockGeometry, HostClock, MANIFEST_NAME, MAX_MANIFEST_LEN, SnapshotManifest, TimeContract, }; diff --git a/nvx-rs/src/host/snapshot/canonical.rs b/nvx-rs/src/host/snapshot/canonical.rs index c300c8cd4..92723c35c 100644 --- a/nvx-rs/src/host/snapshot/canonical.rs +++ b/nvx-rs/src/host/snapshot/canonical.rs @@ -1,5 +1,5 @@ -//! The canonical encoding of a snapshot key and of a capture class. The key's compatibility -//! prefix, and the complete class encoding, are the bytes whose SHA-256 digest is their identity. +//! The canonical encoding of a snapshot key. The key's compatibility prefix is the bytes whose +//! SHA-256 digest is its identity. //! //! Every value has exactly one encoding, and the reader refuses any other: //! diff --git a/nvx-rs/src/host/snapshot/capture/tests.rs b/nvx-rs/src/host/snapshot/capture/tests.rs index 93b534f66..5f076944a 100644 --- a/nvx-rs/src/host/snapshot/capture/tests.rs +++ b/nvx-rs/src/host/snapshot/capture/tests.rs @@ -10,10 +10,10 @@ use serde_json::Value; use super::*; use crate::Hypervisor; -use crate::host::config::{Capacity, CaptureConfig, ConsoleAttachment, DiskCapture, NicConfig}; +use crate::host::config::{DiskCapture, NicConfig}; use crate::host::process::{BootId, Opened, Process, ProcessRef}; use crate::host::snapshot::Snapshot; -use crate::host::snapshot::tests::{MIB, ManifestSpec, SnapshotFixture, class}; +use crate::host::snapshot::tests::{ManifestSpec, SnapshotFixture, context}; use crate::host::vm::create_private_dir; /// A producer kept alive through its stdin until its parent records its real identity. @@ -81,29 +81,7 @@ impl Fixture { fs::remove_file(snapshot.dir.join(name)).unwrap(); } let dir = VmDir::open(&snapshot.dir).unwrap(); - let class = class(); - let context = CaptureContext { - capture: CaptureConfig { - disks: DiskCapture::Fresh { - image: class.distro, - scratch: class.scratch, - }, - processors: Capacity { - boot: u64::from(class.boot_processors), - capacity: u64::from(class.processors), - }, - memory: Capacity { - boot: class.memory / MIB, - capacity: class.memory_capacity / MIB, - }, - invariants: class.invariants, - }, - nic: None, - console: ConsoleAttachment::Disconnected, - runtime: class.runtime, - hypervisor: class.hypervisor, - cpu_profile: Some(class.cpu_profile), - }; + let context = context(); let fixture = Self { _snapshot: snapshot, dir, diff --git a/nvx-rs/src/host/snapshot/key.rs b/nvx-rs/src/host/snapshot/key.rs index 0aeb8fbf0..bfc3d5406 100644 --- a/nvx-rs/src/host/snapshot/key.rs +++ b/nvx-rs/src/host/snapshot/key.rs @@ -1,5 +1,4 @@ -//! The compatibility key of a snapshot, [`SnapshotKey`], and the [`CaptureClass`] of the capture -//! that makes one. +//! The compatibility key of a snapshot, [`SnapshotKey`]. use std::fmt; use std::net::{Ipv4Addr, Ipv6Addr}; @@ -12,8 +11,8 @@ use super::manifest::{ AddressSpace, BlockGeometry, CPUID_ENTRY_LEN, CpuProfile, Device, DeviceRange, FILESYSTEM_SLOT_VERSION, HostClock, Interrupts, Ipv6Identity, MAX_COMMAND_LINE_LEN, MAX_CPUID_ENTRIES, MAX_DEVICE_RANGES, MAX_DEVICE_WORDS, MAX_DEVICES, MAX_MEMORY_RANGES, - MAX_PROCESSORS, MAX_TEXT_LEN, Machine, Memory, MemoryRange, Nic, PROCESSOR_COUNTS, - PolicyIdentity, Rates, SnapshotManifest, TimeContract, Topology, parse_profile_id, + MAX_PROCESSORS, MAX_TEXT_LEN, Machine, Memory, MemoryRange, Nic, PolicyIdentity, Rates, + SnapshotManifest, TimeContract, Topology, parse_profile_id, }; use crate::activation; use crate::contract; @@ -22,7 +21,6 @@ use crate::host::config::{DiskCapture, check_setting}; use crate::host::platform::parse_hex32; use crate::host::process::BootId; use crate::host::runtime::{ArtifactIdentity, MAX_IDENTITY_LEN, RuntimeIdentity}; -use crate::host::sandbox::SandboxNetwork; use crate::host::sandbox::mount::MOUNT_TAG; use crate::hypervisor::Hypervisor; @@ -31,10 +29,6 @@ pub use crate::host::config::Invariants; /// The domain of a key's identity: the bytes that precede its canonical encoding in the digest. const KEY_DOMAIN: &[u8] = b"nvx snapshot key\0"; -/// The domain of a capture class's identity: the bytes that precede its canonical encoding in the -/// digest. -const CLASS_DOMAIN: &[u8] = b"nvx capture class\0"; - /// The generation of a CPU profile that a host derives, rather than a built-in one. const HOST_GENERATION: &str = "host"; @@ -215,16 +209,6 @@ identity!( "a template key's identity" ); -identity!( - /// The identity of a [`CaptureClass`]: the SHA-256 digest of its canonical encoding, after the - /// domain `nvx capture class` and a NUL. - /// - /// Its text form, which `Display` writes and `FromStr` parses, is 64 lowercase hexadecimal - /// digits; parsing also accepts uppercase ones. - ClassIdentity, - "a capture class's identity" -); - /// The canonical encoding of a template's invariant settings, which its key holds. impl Invariants { /// Writes the settings: a list of names and values. @@ -277,162 +261,12 @@ pub(super) fn cpu_profile_matches(profile: &CpuProfile, requested: &str) -> bool } } -/// Returns the guest's network on `nic`, its IPv4 address and prefix length, in the canonical form -/// of a [`SandboxNetwork`]. +/// Returns the guest's network on `nic`, its IPv4 address and prefix length, in its canonical +/// form. fn guest_network_of(nic: &Nic) -> String { format!("{}/{}", nic.ipv4, nic.prefix_length) } -/// The invariant inputs of a template's capture, known before it: what a template's adapter -/// requests, and the class of the templates that the capture makes. -/// -/// A capture whose manifest matches its class makes a template whose [`SnapshotKey`] combines -/// them: [`SnapshotKey::new`] checks that the machine of the manifest is the class's, and takes -/// the rest of the key from it. Captures of one class have one compatibility identity unless -/// something that the class does not fix changes, such as the host's CPU, which its effective -/// CPUID and its rates follow. Each key also retains its own capture's time contract. -/// -/// Its [`identity`](Self::identity) names the class: the SHA-256 digest of its canonical encoding, -/// which holds every field, and the versions of the contracts that the key holds. -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct CaptureClass { - /// The runtime: the identities of OpenVMM, the guest kernel, and the guest initramfs, and the - /// versions of the boot and control contracts. - pub runtime: RuntimeIdentity, - /// The build of the guest's agent, which the adapter supplies. - pub guest_build: ArtifactIdentity, - /// The backend. - pub hypervisor: Hypervisor, - /// The CPU profile that the capture requests, as OpenVMM's `--cpu-profile` takes it: `auto`, - /// for the built-in profile of the host's CPU, `host`, for a profile that the host derives, or - /// a profile ID, `VENDOR.GENERATION.vREVISION`. - pub cpu_profile: String, - /// The processor capacity: 1, 2, 4, or 8. - pub processors: u32, - /// The processors online at boot: 1, 2, 4, or 8, at most [`processors`](Self::processors). - pub boot_processors: u32, - /// The RAM at boot, in bytes. - pub memory: u64, - /// The RAM capacity, in bytes, to which a restore may expand the RAM: at least - /// [`memory`](Self::memory). - pub memory_capacity: u64, - /// The placeholder `distro` that the capture attaches: its length is the capacity of every - /// clone's image. - pub distro: BlockGeometry, - /// The dormant scratch device. - pub scratch: BlockGeometry, - /// The network: `None` or a network that OpenVMM attaches no device for, as - /// [`SandboxNetwork::openvmm_arguments`] lowers it, for a template without a NIC, and - /// otherwise a network at the address of the template's fixed NIC. Of the network, the class - /// takes only what the template captures: its NIC, at the guest's address, under the - /// complete deny-all policy, as [`SandboxNetwork::template_arguments`] lowers it, whatever - /// the network's own policy, which each clone's restore replaces with its own. So the classes - /// of two networks at one address, whose policies both attach a NIC, are one. - pub network: Option, - /// The invariant settings beyond the machine. - pub invariants: Invariants, -} - -impl CaptureClass { - /// Version of the canonical encoding of a class, its first two bytes. - pub const VERSION: u16 = 1; - - /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) unless every field - /// has the form that its documentation gives, the runtime identity passes - /// [`RuntimeIdentity::check`], and the network lowers to OpenVMM's arguments. - pub fn check(&self) -> Result<()> { - self.runtime.check()?; - if let Some(problem) = cpu_profile_problem(&self.cpu_profile) { - return Err(Error::unsupported(problem)); - } - if !PROCESSOR_COUNTS.contains(&self.processors) - || !PROCESSOR_COUNTS.contains(&self.boot_processors) - || self.boot_processors > self.processors - { - return Err(Error::unsupported(format!( - "a capture of {} processors with {} online at boot needs 1, 2, 4, or 8 of each, \ - and no more online than in all", - self.processors, self.boot_processors - ))); - } - if self.memory == 0 || self.memory_capacity < self.memory { - return Err(Error::unsupported(format!( - "a capture of {} bytes of RAM needs a nonzero RAM and a capacity of at least it, \ - not {}", - self.memory, self.memory_capacity - ))); - } - for (role, geometry) in [("distro", &self.distro), ("scratch", &self.scratch)] { - if let Some(problem) = geometry.problem() { - return Err(Error::unsupported(format!( - "the {role} device of a capture is invalid: its {problem}" - ))); - } - } - self.network_arguments()?; - Ok(()) - } - - /// Returns OpenVMM's arguments of the template's NIC, as - /// [`SandboxNetwork::template_arguments`] lowers the network, none without a NIC. - fn network_arguments(&self) -> Result> { - match &self.network { - None => Ok(Vec::new()), - Some(network) => network.template_arguments(), - } - } - - /// Returns the canonical encoding of the class, which its [`identity`](Self::identity) - /// digests. - /// - /// Version 1 holds, in order: [`VERSION`](Self::VERSION) as a `u16`; the runtime identity's - /// OpenVMM, kernel, and initramfs identities as texts, its boot contract version as a `u64`, - /// its control contract revision as a text, and its control session version as a `u64`; the - /// guest build as a text; the snapshot format, device-state schema, time ABI, restore packet, - /// and activation contract versions as `u64`s; the backend, as OpenVMM spells it, and the - /// CPU profile, as texts; the processors and those online at boot as `u32`s; the RAM and its - /// capacity as `u64`s; the `distro` and scratch geometries, each a `u64` length and two - /// `u32` block sizes; the network, as the list of the texts of OpenVMM's arguments of the - /// template's NIC, which [`SandboxNetwork::template_arguments`] lowers it to, empty without a - /// NIC; and the invariant settings, as a list of names and values, each a text, in the order - /// of their names. The [module](super) describes the encoding of each kind of value. - /// - /// Fails as [`check`](Self::check) does. - pub fn to_bytes(&self) -> Result> { - self.encode(&Contracts::CURRENT) - } - - /// Returns the canonical encoding of the class, with the contract versions `contracts`. - fn encode(&self, contracts: &Contracts) -> Result> { - self.check()?; - let mut writer = Writer::default(); - writer.u16(Self::VERSION); - write_runtime(&mut writer, &self.runtime); - writer.text(self.guest_build.as_str()); - contracts.write(&mut writer); - writer.text(self.hypervisor.as_str()); - writer.text(&self.cpu_profile); - writer.u32(self.processors); - writer.u32(self.boot_processors); - writer.u64(self.memory); - writer.u64(self.memory_capacity); - write_geometry(&mut writer, &self.distro); - write_geometry(&mut writer, &self.scratch); - writer.list(&self.network_arguments()?, |writer, argument| { - writer.text(argument) - }); - self.invariants.write(&mut writer); - Ok(writer.finish()) - } - - /// Returns the identity of the class. - /// - /// Fails as [`check`](Self::check) does. - pub fn identity(&self) -> Result { - Ok(ClassIdentity(digest(CLASS_DOMAIN, &self.to_bytes()?))) - } -} - /// The compatibility key of a template: a canonical, versioned projection of the small metadata /// that decides which clones a template can serve, and on which hosts. /// @@ -467,7 +301,7 @@ impl CaptureClass { /// clones of every network policy at its guest's address. /// /// The key comes from metadata alone: the artifacts' identities that their install metadata -/// records, the class's inputs, and the capture's manifest. It never reads or digests a binary, +/// records, the capture's inputs, and its manifest. It never reads or digests a binary, /// an image, the RAM, or the saved state. A bundle that changes any of its artifacts therefore /// makes templates of a new key: the templates of the old one are not recertified. /// @@ -490,7 +324,7 @@ pub struct SnapshotKey { time: TimeContract, } -/// The checked inputs shared by a legacy capture class and a caller-owned capture context. +/// The checked inputs of a capture, which its context holds. struct CaptureInputs<'a> { /// The runtime's artifact and contract identities. runtime: &'a RuntimeIdentity, @@ -530,54 +364,6 @@ impl SnapshotKey { /// [`MAX_MANIFEST_LEN`](super::MAX_MANIFEST_LEN) bytes can describe takes, with room to spare. pub const MAX_LEN: usize = 256 * 1024; - /// Returns the key of the template that a capture of `class` made, whose manifest is - /// `manifest`. - /// - /// The manifest's machine must be the class's: the backend, the processors and those online - /// at boot, the RAM and its capacity, the `distro` and scratch geometries, the presence of a - /// NIC, and the NIC's address and prefix, and a CPU profile that the class's request selects. - /// A manifest whose machine has no processors online at boot activates all of them at boot, - /// which a class whose processors are all online at boot requests. The key takes the class's - /// runtime and invariant settings, and this crate's versions of the other - /// contracts. - /// - /// The key also takes the identity of the network policy that OpenVMM enforced on the NIC, - /// which this check leaves to OpenVMM: OpenVMM derives it from the arguments of the - /// template's NIC, under the complete deny-all policy, as - /// [`SandboxNetwork::template_arguments`] lowers the class's network, by an encoding of its - /// own, which this crate does not reproduce. A capture that enforced another policy makes a - /// key of its own, which no restore serves, since OpenVMM lets a restore replace the complete - /// deny-all policy alone. - /// - /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) as - /// [`CaptureClass::check`] does, and with [`ErrorKind::Io`](crate::ErrorKind::Io) for a - /// manifest whose machine is not the class's, as a capture that did not do what its class - /// requested. - pub fn new(class: &CaptureClass, manifest: &SnapshotManifest) -> Result { - class.check()?; - let attached = !class.network_arguments()?.is_empty(); - Self::from_inputs( - CaptureInputs { - runtime: &class.runtime, - hypervisor: class.hypervisor, - cpu_profile: &class.cpu_profile, - processors: u64::from(class.processors), - boot_processors: u64::from(class.boot_processors), - memory: class.memory, - memory_capacity: class.memory_capacity, - distro: &class.distro, - scratch: &class.scratch, - guest_network: class - .network - .as_ref() - .filter(|_| attached) - .map(|network| network.guest_network().to_owned()), - invariants: &class.invariants, - }, - manifest, - ) - } - /// Derives the key from the checked creation context and OpenVMM's validated manifest. pub(super) fn from_capture( context: &CaptureContext, @@ -677,7 +463,7 @@ impl SnapshotKey { } } let requested_network = class.guest_network; - // A network that `SandboxNetwork` accepts has the canonical form that this writes. + // An `Ipv4Cidr` displays a network in the canonical form that this writes. let found_network = machine.nic.as_ref().map(guest_network_of); if found_network != requested_network { let describe = |network: Option| network.unwrap_or_else(|| "none".to_owned()); @@ -797,8 +583,8 @@ impl SnapshotKey { } /// Returns the guest's network on the template's NIC, its IPv4 address and prefix length, in - /// the canonical form of a [`SandboxNetwork`], such as `10.0.0.2/24`, which OpenVMM restores - /// from the snapshot with the NIC, or `None` for a template without a NIC. The template serves + /// its canonical form, such as `10.0.0.2/24`, which OpenVMM restores from the snapshot with the + /// NIC, or `None` for a template without a NIC. The template serves /// the clones of every network policy at that address. pub fn guest_network(&self) -> Option { self.machine.nic.as_ref().map(guest_network_of) diff --git a/nvx-rs/src/host/snapshot/key/tests.rs b/nvx-rs/src/host/snapshot/key/tests.rs index 8e416fd9e..8e6540cb7 100644 --- a/nvx-rs/src/host/snapshot/key/tests.rs +++ b/nvx-rs/src/host/snapshot/key/tests.rs @@ -1,12 +1,12 @@ -//! Tests of [`SnapshotKey`] and [`CaptureClass`]: what a key holds, what it leaves out, and its -//! canonical encoding. +//! Tests of [`SnapshotKey`]: what a key holds, what it leaves out, and its canonical encoding. use std::collections::HashSet; use super::*; use crate::ErrorCode; -use crate::host::network::{Access, EgressRule, HostLoopback, NetworkPolicy, NetworkProxy}; -use crate::host::snapshot::tests::{self as support, MIB, ManifestSpec, artifact, network_class}; +use crate::host::config::NicConfig; +use crate::host::snapshot::tests::{self as support, MIB, ManifestSpec, artifact, network_context}; +use crate::net::Ipv4Cidr; /// Returns the manifest that `spec` encodes. fn manifest(spec: &ManifestSpec) -> SnapshotManifest { @@ -28,31 +28,46 @@ fn capture_time_len(time: &TimeContract) -> usize { } } -/// Returns the key of the template that a capture of `class` made, whose manifest `spec` +/// Returns the key of the template that the capture of `context` made, whose manifest `spec` /// encodes. -fn key(class: &CaptureClass, spec: &ManifestSpec) -> SnapshotKey { - SnapshotKey::new(class, &manifest(spec)).unwrap() +fn key(context: &CaptureContext, spec: &ManifestSpec) -> SnapshotKey { + SnapshotKey::from_capture(context, &manifest(spec)).unwrap() } /// Returns the key of the template with a NIC of the tests. fn network_key() -> SnapshotKey { - key(&network_class(), &ManifestSpec::with_network()) + key(&network_context(), &ManifestSpec::with_network()) } -/// A change to the class and the manifest of the template with a NIC of the tests. -type Change = fn(&mut CaptureClass, &mut ManifestSpec); +/// A change to the context and the manifest of the capture of the template with a NIC of the +/// tests. +type Change = fn(&mut CaptureContext, &mut ManifestSpec); /// Returns the identity of the key of the template with a NIC of the tests once `change` changes /// it. fn identity_after(change: Change) -> KeyIdentity { - let (mut class, mut spec) = (network_class(), ManifestSpec::with_network()); - change(&mut class, &mut spec); - key(&class, &spec).identity() + let (mut context, mut spec) = (network_context(), ManifestSpec::with_network()); + change(&mut context, &mut spec); + key(&context, &spec).identity() +} + +/// Returns the geometries of the placeholder image and of the dormant scratch that the capture of +/// `context` attaches. +fn disks(context: &mut CaptureContext) -> (&mut BlockGeometry, &mut BlockGeometry) { + let DiskCapture::Fresh { image, scratch } = &mut context.capture.disks; + (image, scratch) +} + +/// Returns the NIC at `network`, an IPv4 address and prefix length. +fn nic(network: &str) -> Option { + Some(NicConfig { + guest_network: Ipv4Cidr::parse(network).unwrap(), + }) } /// Changes the RAM of a capture to `memory` bytes, below 3 GiB, of the same capacity. -fn set_memory(class: &mut CaptureClass, spec: &mut ManifestSpec, memory: u64) { - class.memory = memory; +fn set_memory(context: &mut CaptureContext, spec: &mut ManifestSpec, memory: u64) { + context.capture.memory.boot = memory / MIB; spec.memory_size = memory; let contract = spec.contract(); contract.memory_ranges = vec![(0, memory, 0)]; @@ -63,21 +78,27 @@ fn set_memory(class: &mut CaptureClass, spec: &mut ManifestSpec, memory: u64) { fn each_part_of_the_key_changes_it() { let changes: &[(&str, Change)] = &[ // Bundle and contracts. - ("OpenVMM", |class, _| class.runtime.openvmm = artifact('d')), - ("kernel", |class, _| class.runtime.kernel = artifact('d')), - ("initramfs", |class, _| { - class.runtime.initramfs = artifact('d') + ("OpenVMM", |context, _| { + context.runtime.openvmm = artifact('d') + }), + ("kernel", |context, _| { + context.runtime.kernel = artifact('d') }), - ("boot contract", |class, _| class.runtime.boot_contract += 1), - ("control contract", |class, _| { - class.runtime.control_contract = "other-revision".to_owned(); + ("initramfs", |context, _| { + context.runtime.initramfs = artifact('d') }), - ("control session", |class, _| { - class.runtime.control_session += 1 + ("boot contract", |context, _| { + context.runtime.boot_contract += 1 + }), + ("control contract", |context, _| { + context.runtime.control_contract = "other-revision".to_owned(); + }), + ("control session", |context, _| { + context.runtime.control_session += 1 }), // CPU and time. - ("backend", |class, spec| { - class.hypervisor = Hypervisor::Mshv; + ("backend", |context, spec| { + context.hypervisor = Hypervisor::Mshv; spec.contract().source_hypervisor = "mshv".to_owned(); }), ("CPU vendor", |_, spec| { @@ -104,18 +125,18 @@ fn each_part_of_the_key_changes_it() { ("LAPIC timer rate", |_, spec| spec.time().apic_hz += 1), ("rate policy", |_, spec| spec.time().tolerance_ppm += 1), // Resources. - ("processors", |class, spec| { - class.processors = 4; - class.boot_processors = 4; + ("processors", |context, spec| { + context.capture.processors.capacity = 4; + context.capture.processors.boot = 4; spec.vp_count = 4; spec.contract().topology = (1, 1, 4, 1, vec![0, 1, 2, 3]); }), ("processors online at boot", |_, spec| { spec.contract().boot_online_vp_count = 2; }), - ("RAM", |class, spec| set_memory(class, spec, 640 * MIB)), - ("RAM capacity", |class, spec| { - class.memory_capacity = 1024 * MIB; + ("RAM", |context, spec| set_memory(context, spec, 640 * MIB)), + ("RAM capacity", |context, spec| { + context.capture.memory.capacity = 1024; let contract = spec.contract(); contract.memory_capacity = 1024 * MIB; contract.expansion_ranges = vec![(512 * MIB, 512 * MIB)]; @@ -136,21 +157,22 @@ fn each_part_of_the_key_changes_it() { spec.contract().command_line.push_str(" quiet"); }), // Disks. - ("distro capacity", |class, spec| { - class.distro.length *= 2; + ("distro capacity", |context, spec| { + disks(context).0.length *= 2; spec.contract().blocks[0].length *= 2; }), - ("distro block sizes", |class, spec| { - class.distro.physical_block_size = 512; + ("distro block sizes", |context, spec| { + disks(context).0.physical_block_size = 512; spec.contract().blocks[0].physical_block_size = 512; }), - ("scratch length", |class, spec| { - class.scratch.length *= 2; + ("scratch length", |context, spec| { + disks(context).1.length *= 2; spec.contract().blocks[1].length *= 2; }), - ("scratch geometry", |class, spec| { - class.scratch.logical_block_size = 4096; - class.scratch.physical_block_size = 4096; + ("scratch geometry", |context, spec| { + let (_, requested) = disks(context); + requested.logical_block_size = 4096; + requested.physical_block_size = 4096; let scratch = &mut spec.contract().blocks[1]; scratch.logical_block_size = 4096; scratch.physical_block_size = 4096; @@ -203,8 +225,8 @@ fn each_part_of_the_key_changes_it() { ("interrupt-status page", |_, spec| { spec.contract().status_page.0 += 0x1000; }), - ("NIC presence", |class, spec| { - *class = support::class(); + ("NIC presence", |context, spec| { + *context = support::context(); *spec = ManifestSpec::template(); }), ("network profile", |_, spec| { @@ -212,16 +234,12 @@ fn each_part_of_the_key_changes_it() { }), ("guest MAC", |_, spec| spec.network().guest_mac[5] = 3), ("gateway MAC", |_, spec| spec.network().gateway_mac[5] = 3), - ("IPv4 address", |class, spec| { - class.network = Some( - SandboxNetwork::new("10.0.0.3/24", NetworkPolicy::new(Access::Allow)).unwrap(), - ); + ("IPv4 address", |context, spec| { + context.nic = nic("10.0.0.3/24"); spec.network().ipv4 = [10, 0, 0, 3]; }), - ("IPv4 prefix", |class, spec| { - class.network = Some( - SandboxNetwork::new("10.0.0.2/16", NetworkPolicy::new(Access::Allow)).unwrap(), - ); + ("IPv4 prefix", |context, spec| { + context.nic = nic("10.0.0.2/16"); spec.network().prefix_length = 16; }), ("IPv4 gateway", |_, spec| { @@ -252,8 +270,8 @@ fn each_part_of_the_key_changes_it() { ("policy encoding", |_, spec| { spec.network().policy_encoding_version = 3; }), - ("invariant setting", |class, _| { - class.invariants = Invariants::new() + ("invariant setting", |context, _| { + context.capture.invariants = Invariants::new() .with("isolation.seccomp", "strict") .unwrap(); }), @@ -267,11 +285,11 @@ fn each_part_of_the_key_changes_it() { ); } // Two values of one setting make two keys. - let mut class = network_class(); - class.invariants = Invariants::new() + let mut context = network_context(); + context.capture.invariants = Invariants::new() .with("isolation.seccomp", "relaxed") .unwrap(); - assert!(identities.insert(key(&class, &ManifestSpec::with_network()).identity())); + assert!(identities.insert(key(&context, &ManifestSpec::with_network()).identity())); } #[test] @@ -329,9 +347,9 @@ fn a_generations_own_metadata_leaves_the_key_as_it_is() { spec.cpu().document = b"another document".to_vec(); }), ("capture CPU signature", |_, spec| spec.cpu().signature += 1), - // The class requests a profile, which the key names as OpenVMM selected it. - ("requested CPU profile", |class, _| { - class.cpu_profile = "intel.icelake-sp.v1".to_owned(); + // The capture requests a profile, which the key names as OpenVMM selected it. + ("requested CPU profile", |context, _| { + context.cpu_profile = Some("intel.icelake-sp.v1".to_owned()); }), ]; let base = network_key(); @@ -344,22 +362,6 @@ fn a_generations_own_metadata_leaves_the_key_as_it_is() { } } -#[test] -fn the_guest_build_is_not_part_of_a_snapshot_key() { - let class = network_class(); - let spec = ManifestSpec::with_network(); - let made = key(&class, &spec); - let other = CaptureClass { - guest_build: ArtifactIdentity::new("guest-build-2").unwrap(), - ..class.clone() - }; - assert_ne!(class.identity().unwrap(), other.identity().unwrap()); - let changed = key(&other, &spec); - assert_eq!(changed, made); - assert_eq!(changed.to_bytes(), made.to_bytes()); - assert_eq!(changed.identity(), made.identity()); -} - #[test] fn complete_capture_time_round_trips_without_changing_compatibility_identity() { let base = network_key(); @@ -375,7 +377,7 @@ fn complete_capture_time_round_trips_without_changing_compatibility_identity() { time.lineage = u32::MAX; time.cut_to_anchor_ns = cut; let manifest = manifest(&spec); - let key = SnapshotKey::new(&network_class(), &manifest).unwrap(); + let key = SnapshotKey::from_capture(&network_context(), &manifest).unwrap(); assert_eq!(key.time(), manifest.time()); assert_ne!(key.to_bytes(), base.to_bytes()); assert_eq!(key.identity(), base.identity()); @@ -407,8 +409,8 @@ fn canonical_keys_require_capture_time_and_a_known_host_clock() { #[test] fn a_key_holds_what_its_capture_made() { let made = network_key(); - let class = network_class(); - assert_eq!(made.runtime(), &class.runtime); + let mut context = network_context(); + assert_eq!(made.runtime(), &context.runtime); assert_eq!(made.hypervisor(), Hypervisor::Kvm); assert_eq!(made.cpu_profile(), "intel.icelake-sp.v1"); assert_eq!((made.processors(), made.boot_processors()), (2, 0)); @@ -424,23 +426,21 @@ fn a_key_holds_what_its_capture_made() { made.command_line(), ManifestSpec::with_network().contract().command_line ); - assert_eq!( - (made.distro(), made.scratch()), - (class.distro, class.scratch) - ); + let (image, scratch) = disks(&mut context); + assert_eq!((made.distro(), made.scratch()), (*image, *scratch)); assert!(made.has_network()); assert_eq!(made.guest_network().as_deref(), Some("10.0.0.2/24")); - let template = key(&support::class(), &ManifestSpec::template()); + let template = key(&support::context(), &ManifestSpec::template()); assert!(!template.has_network()); assert_eq!(template.guest_network(), None); assert_eq!(made.filesystem_tag(), "microvm"); assert_eq!(made.invariants(), &Invariants::new()); } -/// Asserts that the capture of `class` whose manifest is `spec` does not match its class, with -/// a message that holds `expected`. -fn assert_mismatched(class: &CaptureClass, spec: &ManifestSpec, expected: &str) { - let error = SnapshotKey::new(class, &manifest(spec)).unwrap_err(); +/// Asserts that the capture of `context` whose manifest is `spec` does not match what `context` +/// requests, with a message that holds `expected`. +fn assert_mismatched(context: &CaptureContext, spec: &ManifestSpec, expected: &str) { + let error = SnapshotKey::from_capture(context, &manifest(spec)).unwrap_err(); assert_eq!(error.code(), ErrorCode::Io, "{error}"); assert!( error @@ -455,38 +455,38 @@ fn assert_mismatched(class: &CaptureClass, spec: &ManifestSpec, expected: &str) #[test] fn a_capture_that_does_not_match_its_class_is_refused() { let (template, with_network) = (ManifestSpec::template(), ManifestSpec::with_network()); - let base = support::class(); - let mut class = base.clone(); - class.hypervisor = Hypervisor::Mshv; + let base = support::context(); + let mut context = base.clone(); + context.hypervisor = Hypervisor::Mshv; assert_mismatched( - &class, + &context, &template, "its backend is kvm, but the class requests mshv", ); for requested in ["amd.genoa.v1", "intel.icelake-sp.v2", "host"] { - class = base.clone(); - class.cpu_profile = requested.to_owned(); + context = base.clone(); + context.cpu_profile = Some(requested.to_owned()); assert_mismatched( - &class, + &context, &template, &format!("its CPU profile is intel.icelake-sp.v1, but the class requests {requested}"), ); } - class = base.clone(); - class.processors = 4; - class.boot_processors = 4; + context = base.clone(); + context.capture.processors.capacity = 4; + context.capture.processors.boot = 4; assert_mismatched( - &class, + &context, &template, "it has 2 processors, 2 online at boot, but the class requests 4 processors, 4 online at \ boot", ); - // A capture whose processors are all online at boot does not serve a class that keeps some + // A capture whose processors are all online at boot does not serve a request that keeps some // offline. - class = base.clone(); - class.boot_processors = 1; + context = base.clone(); + context.capture.processors.boot = 1; assert_mismatched( - &class, + &context, &template, "it has 2 processors, 2 online at boot, but the class requests 2 processors, 1 online", ); @@ -497,44 +497,38 @@ fn a_capture_that_does_not_match_its_class_is_refused() { &spec, "it has 2 processors, 1 online at boot, but the class requests 2 processors, 2 online", ); - assert!(SnapshotKey::new(&class, &manifest(&spec)).is_ok()); - class = base.clone(); - class.memory = 1024 * MIB; + assert!(SnapshotKey::from_capture(&context, &manifest(&spec)).is_ok()); + context = base.clone(); + context.capture.memory.boot = 1024; assert_mismatched( - &class, + &context, &template, "it has 536870912 bytes of RAM, of a capacity of 2147483648, but the class requests \ 1073741824 bytes of RAM", ); - class = base.clone(); - class.memory_capacity = 4096 * MIB; - assert_mismatched(&class, &template, "of a capacity of 4294967296"); - class = base.clone(); - class.distro.length *= 2; - assert_mismatched(&class, &template, "its distro geometry is"); - class = base.clone(); - class.scratch.physical_block_size = 4096; - assert_mismatched(&class, &template, "its scratch geometry is"); + context = base.clone(); + context.capture.memory.capacity = 4096; + assert_mismatched(&context, &template, "of a capacity of 4294967296"); + context = base.clone(); + disks(&mut context).0.length *= 2; + assert_mismatched(&context, &template, "its distro geometry is"); + context = base.clone(); + disks(&mut context).1.physical_block_size = 4096; + assert_mismatched(&context, &template, "its scratch geometry is"); assert_mismatched( &base, &with_network, "its NIC network is 10.0.0.2/24, but the class requests none", ); assert_mismatched( - &network_class(), + &network_context(), &template, "its NIC network is none, but the class requests 10.0.0.2/24", ); - // A network whose policy lets nothing through attaches no NIC. - class = base.clone(); - class.network = - Some(SandboxNetwork::new("10.0.0.2/24", NetworkPolicy::new(Access::Deny)).unwrap()); - assert_mismatched(&class, &with_network, "but the class requests none"); - assert_eq!(key(&class, &template), key(&base, &template)); - class.network = - Some(SandboxNetwork::new("10.0.0.3/24", NetworkPolicy::new(Access::Allow)).unwrap()); + context = base.clone(); + context.nic = nic("10.0.0.3/24"); assert_mismatched( - &class, + &context, &with_network, "its NIC network is 10.0.0.2/24, but the class requests 10.0.0.3/24", ); @@ -544,166 +538,79 @@ fn a_capture_that_does_not_match_its_class_is_refused() { fn a_class_selects_a_profile_as_openvmm_does() { let mut spec = ManifestSpec::template(); spec.cpu().id = "intel.host.v1".to_owned(); - let mut class = support::class(); + let mut context = support::context(); // `auto` falls back to a profile that the host derives when no built-in profile serves the // host's CPU. for requested in ["auto", "host", "intel.host.v1"] { - class.cpu_profile = requested.to_owned(); - assert_eq!(key(&class, &spec).cpu_profile(), "intel.host.v1"); + context.cpu_profile = Some(requested.to_owned()); + assert_eq!(key(&context, &spec).cpu_profile(), "intel.host.v1"); } - class.cpu_profile = "intel.icelake-sp.v1".to_owned(); - assert_mismatched(&class, &spec, "its CPU profile is intel.host.v1"); + context.cpu_profile = Some("intel.icelake-sp.v1".to_owned()); + assert_mismatched(&context, &spec, "its CPU profile is intel.host.v1"); } #[test] -fn invalid_classes_are_refused() { - type ClassChange = fn(&mut CaptureClass); - let cases: &[(ClassChange, &str)] = &[ +fn captures_of_invalid_contexts_are_refused() { + type ContextChange = fn(&mut CaptureContext); + let cases: &[(ContextChange, &str)] = &[ ( - |class| class.runtime.control_contract = "two words".to_owned(), + |context| context.runtime.control_contract = "two words".to_owned(), "control contract revision \"two words\" must hold 1 to 128 printable ASCII \ characters and no space", ), ( - |class| class.cpu_profile = "fastest".to_owned(), + |context| context.cpu_profile = Some("fastest".to_owned()), "CPU profile \"fastest\" must be \"auto\", \"host\", or a profile ID", ), ( - |class| class.processors = 3, + |context| context.capture.processors.capacity = 3, "a capture of 3 processors with 2 online at boot needs 1, 2, 4, or 8 of each", ), ( - |class| class.boot_processors = 0, + |context| context.capture.processors.boot = 0, "a capture of 2 processors with 0 online at boot", ), ( - |class| class.boot_processors = 4, - "a capture of 2 processors with 4 online at boot", + |context| context.capture.processors.boot = 4, + "a capture of 2 processors can't have 4 online at boot", ), ( - |class| class.memory = 0, - "a capture of 0 bytes of RAM needs a nonzero RAM", + |context| context.capture.memory.boot = 0, + "a capture needs memory at boot", ), ( - |class| class.memory_capacity = class.memory - 1, - "needs a nonzero RAM and a capacity of at least it", + |context| context.capture.memory.capacity = context.capture.memory.boot - 128, + "is below its 512 MiB at boot", ), ( - |class| class.distro.logical_block_size = 256, - "the distro device of a capture is invalid: its logical block size 256 is not a \ - power of two of at least 512", + |context| disks(context).0.logical_block_size = 256, + "the image disk of a capture is invalid: its logical block size 256 is not a power \ + of two of at least 512", ), ( - |class| class.scratch.length = 0, - "the scratch device of a capture is invalid: its length 0", + |context| disks(context).1.length = 0, + "the scratch disk of a capture is invalid: its length 0", ), ]; for (change, expected) in cases { - let mut class = network_class(); - change(&mut class); - for error in [ - class.check().unwrap_err(), - class.to_bytes().unwrap_err(), - class.identity().unwrap_err(), - SnapshotKey::new(&class, &manifest(&ManifestSpec::with_network())).unwrap_err(), - ] { - assert_eq!(error.code(), ErrorCode::Unsupported, "{error}"); - assert!( - error.message().contains(expected), - "{:?} lacks {expected:?}", - error.message() - ); - } - } -} - -#[test] -fn each_field_of_a_class_changes_its_identity() { - type ClassChange = fn(&mut CaptureClass); - let changes: &[ClassChange] = &[ - |class| class.runtime.openvmm = artifact('d'), - |class| class.runtime.kernel = artifact('d'), - |class| class.runtime.initramfs = artifact('d'), - |class| class.runtime.boot_contract += 1, - |class| class.runtime.control_contract = "other".to_owned(), - |class| class.runtime.control_session += 1, - |class| class.guest_build = artifact('e'), - |class| class.hypervisor = Hypervisor::Whp, - |class| class.cpu_profile = "host".to_owned(), - |class| class.processors = 4, - |class| class.boot_processors = 1, - |class| class.memory = 1024 * MIB, - |class| class.memory_capacity = 4096 * MIB, - |class| class.distro.length *= 2, - |class| class.distro.physical_block_size = 512, - |class| class.scratch.length *= 2, - |class| { - class.scratch.logical_block_size = 4096; - class.scratch.physical_block_size = 4096; - }, - |class| class.network = None, - |class| { - let policy = NetworkPolicy::new(Access::Allow); - class.network = Some(SandboxNetwork::new("10.0.0.3/24", policy).unwrap()); - }, - |class| { - class.invariants = Invariants::new() - .with("isolation.seccomp", "strict") - .unwrap(); - }, - ]; - let base = network_class().identity().unwrap(); - let mut identities = HashSet::from([base]); - for (index, change) in changes.iter().enumerate() { - let mut class = network_class(); - change(&mut class); + let mut context = network_context(); + change(&mut context); + let error = SnapshotKey::from_capture(&context, &manifest(&ManifestSpec::with_network())) + .unwrap_err(); + assert_eq!(error.code(), ErrorCode::Unsupported, "{error}"); assert!( - identities.insert(class.identity().unwrap()), - "change {index} leaves the class's identity as it is" + error.message().contains(expected), + "{:?} lacks {expected:?}", + error.message() ); } - // A network that attaches no NIC is no network. - let mut denied = network_class(); - denied.network = - Some(SandboxNetwork::new("10.0.0.2/24", NetworkPolicy::new(Access::Deny)).unwrap()); - let mut none = network_class(); - none.network = None; - assert_eq!(denied.identity().unwrap(), none.identity().unwrap()); - // The template captures its NIC under the complete deny-all policy, which each clone's restore - // replaces, so the policy of a network that attaches one is no part of its class. - for policy in [ - NetworkPolicy::new(Access::Allow).with_deny(EgressRule::to("192.0.2.0/24")), - NetworkPolicy::new(Access::Deny).with_allow(EgressRule::to("192.0.2.0/24")), - NetworkPolicy { - ingress: Access::Allow, - ..NetworkPolicy::new(Access::Deny) - }, - NetworkPolicy { - host_loopback: HostLoopback::Allow, - ..NetworkPolicy::new(Access::Deny) - }, - NetworkPolicy::proxy_only(NetworkProxy::new(3128)), - ] { - let mut class = network_class(); - class.network = Some(SandboxNetwork::new("10.0.0.2/24", policy).unwrap()); - assert_eq!(class.identity().unwrap(), base, "{:?}", class.network); - } - // The identity digests the canonical encoding after the class's domain. - let class = network_class(); - let mut digest = Sha256::new(); - digest.update(b"nvx capture class\0"); - digest.update(class.to_bytes().unwrap()); - assert_eq!( - class.identity().unwrap().as_bytes(), - &<[u8; 32]>::from(digest.finalize()) - ); } #[test] fn keys_read_back_from_their_canonical_encoding() { for made in [ network_key(), - key(&support::class(), &ManifestSpec::template()), + key(&support::context(), &ManifestSpec::template()), ] { let bytes = made.to_bytes(); assert_eq!(&bytes[..2], &[1, 0]); @@ -722,14 +629,13 @@ fn keys_read_back_from_their_canonical_encoding() { #[test] fn keys_with_the_removed_guest_build_field_are_refused() { - let class = network_class(); let made = network_key(); let mut prefix = Writer::default(); prefix.u16(SnapshotKey::VERSION); write_runtime(&mut prefix, made.runtime()); let offset = prefix.finish().len(); let mut field = Writer::default(); - field.text(class.guest_build.as_str()); + field.text("guest-build-1"); let mut old = made.to_bytes(); old.splice(offset..offset, field.finish()); assert_eq!( @@ -813,7 +719,7 @@ fn a_key_that_holds_its_network_s_lowering_after_its_invariants_is_refused() { vec!["--net", "10.0.0.2/24", "--network-profile", "portable"], ), ( - key(&support::class(), &ManifestSpec::template()), + key(&support::context(), &ManifestSpec::template()), Vec::new(), ), ] { @@ -879,13 +785,13 @@ fn keys_of_machines_that_no_manifest_describes_are_refused() { ); } // Invariant settings out of the order of their names, which a key never writes. - let mut class = network_class(); - class.invariants = Invariants::new() + let mut context = network_context(); + context.capture.invariants = Invariants::new() .with("a", "1") .unwrap() .with("b", "2") .unwrap(); - let made = key(&class, &ManifestSpec::with_network()); + let made = key(&context, &ManifestSpec::with_network()); let mut bytes = made.to_bytes(); let end = bytes.len() - capture_time_len(made.time()); let tail = [ @@ -937,17 +843,9 @@ fn identities_have_a_text_form() { format!("{invalid:?} is not a template key's identity, which is 64 hexadecimal digits") ); } - let class = network_class().identity().unwrap(); - assert_eq!(class.to_string().parse::().unwrap(), class); - assert!( - "x".parse::() - .unwrap_err() - .message() - .contains("is not a capture class's identity") - ); } -/// The conformance vectors of version 1 of the canonical encodings of keys and classes. +/// The conformance vectors of version 1 of the canonical encoding of keys. const VECTORS: &str = include_str!("../../../../tests/vectors/snapshot-key-v1.txt"); /// The versions of the contracts that the vectors hold. @@ -979,23 +877,21 @@ fn vectors(kind: &str) -> Vec<(&'static str, Vec, Vec<&'static str>)> { .collect() } -/// Returns the class of the capture that the vectors call `name`, with the runtime that they +/// Returns the context of the capture that the vectors call `name`, with the runtime that they /// hold. -fn vector_class(name: &str) -> CaptureClass { - let mut class = if name == "NETWORK" { - let class = network_class(); - CaptureClass { - invariants: Invariants::new() - .with("isolation.seccomp", "strict") - .unwrap() - .with("limits.pids", "512") - .unwrap(), - ..class - } +fn vector_context(name: &str) -> CaptureContext { + let mut context = if name == "NETWORK" { + let mut context = network_context(); + context.capture.invariants = Invariants::new() + .with("isolation.seccomp", "strict") + .unwrap() + .with("limits.pids", "512") + .unwrap(); + context } else { - support::class() + support::context() }; - class.runtime = RuntimeIdentity { + context.runtime = RuntimeIdentity { openvmm: artifact('a'), kernel: artifact('b'), initramfs: artifact('c'), @@ -1003,7 +899,7 @@ fn vector_class(name: &str) -> CaptureClass { control_contract: "nvx-microvm-v2-control-v2".to_owned(), control_session: 1, }; - class + context } /// Returns the manifest of the capture that the vectors call `name`, as microVM ABI 2 captured it: @@ -1070,8 +966,9 @@ fn the_conformance_vectors_decode_and_encode() { if name != "MINIMAL" { // The key that this crate derives from the capture is the vector's, but for the // versions of the contracts, which are this crate's. - let derived = SnapshotKey::new(&vector_class(name), &manifest(&vector_spec(name))) - .unwrap_or_else(|error| panic!("{name}: {error}")); + let derived = + SnapshotKey::from_capture(&vector_context(name), &manifest(&vector_spec(name))) + .unwrap_or_else(|error| panic!("{name}: {error}")); let derived = SnapshotKey { contracts: VECTOR_CONTRACTS, ..derived @@ -1079,18 +976,6 @@ fn the_conformance_vectors_decode_and_encode() { assert_eq!(derived, key, "{name}"); } } - let classes = vectors("CLASS"); - let names: Vec<&str> = classes.iter().map(|(name, ..)| *name).collect(); - assert_eq!(names, ["TEMPLATE", "NETWORK"]); - for (name, bytes, values) in classes { - let class = vector_class(name); - assert_eq!(class.encode(&VECTOR_CONTRACTS).unwrap(), bytes, "{name}"); - assert_eq!( - values, - [ClassIdentity(digest(CLASS_DOMAIN, &bytes)).to_string()], - "{name}" - ); - } let versions = vectors("INVALID_KEY_VERSION"); assert_eq!(versions.len(), 3); for (name, bytes, _) in versions { diff --git a/nvx-rs/src/host/snapshot/tests.rs b/nvx-rs/src/host/snapshot/tests.rs index a1fada3ca..82d73f364 100644 --- a/nvx-rs/src/host/snapshot/tests.rs +++ b/nvx-rs/src/host/snapshot/tests.rs @@ -1,4 +1,4 @@ -//! Builders of the manifests and capture classes of snapshots, for their tests. +//! Builders of the manifests and capture contexts of snapshots, for their tests. //! //! [`Proto`] writes the protobuf wire format as OpenVMM's `mesh` does, independently of the reader //! that the tests check, and [`ManifestSpec`] holds each field of a manifest, so that a test @@ -10,18 +10,18 @@ use std::path::{Path, PathBuf}; use sha2::{Digest, Sha256}; use super::{ - BlockGeometry, CaptureClass, DESCRIPTION_FILE, Invariants, KEY_FILE, MANIFEST_NAME, + BlockGeometry, CaptureContext, DESCRIPTION_FILE, Invariants, KEY_FILE, MANIFEST_NAME, OPENVMM_DIR, Snapshot, SnapshotKey, SnapshotManifest, }; use crate::ErrorKind; use crate::activation::{Journal, MachineIdMode, OverlayMode, View}; use crate::host::config::{ - ConsoleAttachment, LayeredRoot, NetworkConfig, NicConfig, RootConfig, UpperConfig, VmConfig, + Capacity, CaptureConfig, ConsoleAttachment, DiskCapture, LayeredRoot, NetworkConfig, NicConfig, + RootConfig, UpperConfig, VmConfig, }; use crate::host::network::{Access, NetworkPolicy}; use crate::host::process::BootId; use crate::host::runtime::{ArtifactIdentity, LaunchOptions, Runtime, RuntimeIdentity}; -use crate::host::sandbox::SandboxNetwork; use crate::hypervisor::Hypervisor; use crate::net::Ipv4Cidr; use crate::protocol::root::RootImage; @@ -965,39 +965,47 @@ pub(in crate::host) fn artifact(digit: char) -> ArtifactIdentity { ArtifactIdentity::sha256(&digit.to_string().repeat(64)).unwrap() } -/// Returns the class of the capture of [`ManifestSpec::template`]. -pub(in crate::host) fn class() -> CaptureClass { - CaptureClass { +/// Returns the checked inputs of the capture of [`ManifestSpec::template`]. +pub(in crate::host) fn context() -> CaptureContext { + CaptureContext { + capture: CaptureConfig { + disks: DiskCapture::Fresh { + image: BlockGeometry { + length: 1024 * MIB, + logical_block_size: 512, + physical_block_size: 4096, + }, + scratch: BlockGeometry { + length: 64 * MIB, + logical_block_size: 512, + physical_block_size: 512, + }, + }, + processors: Capacity { + boot: 2, + capacity: 2, + }, + memory: Capacity { + boot: 512, + capacity: 2048, + }, + invariants: Invariants::new(), + }, + nic: None, + console: ConsoleAttachment::Disconnected, runtime: RuntimeIdentity::new(artifact('a'), artifact('b'), artifact('c')), - guest_build: ArtifactIdentity::new("guest-build-1").unwrap(), hypervisor: Hypervisor::Kvm, - cpu_profile: "auto".to_owned(), - processors: 2, - boot_processors: 2, - memory: 512 * MIB, - memory_capacity: 2048 * MIB, - distro: BlockGeometry { - length: 1024 * MIB, - logical_block_size: 512, - physical_block_size: 4096, - }, - scratch: BlockGeometry { - length: 64 * MIB, - logical_block_size: 512, - physical_block_size: 512, - }, - network: None, - invariants: Invariants::new(), + cpu_profile: Some("auto".to_owned()), } } -/// Returns the class of the capture of [`ManifestSpec::with_network`]. -pub(in crate::host) fn network_class() -> CaptureClass { - CaptureClass { - network: Some( - SandboxNetwork::new("10.0.0.2/24", NetworkPolicy::new(Access::Allow)).unwrap(), - ), - ..class() +/// Returns the checked inputs of the capture of [`ManifestSpec::with_network`]. +pub(in crate::host) fn network_context() -> CaptureContext { + CaptureContext { + nic: Some(NicConfig { + guest_network: Ipv4Cidr::parse("10.0.0.2/24").unwrap(), + }), + ..context() } } @@ -1020,7 +1028,7 @@ impl SnapshotFixture { crate::host::vm::create_private_dir(&openvmm).unwrap(); let spec = ManifestSpec::template(); let manifest = SnapshotManifest::parse(&spec.encode()).unwrap(); - let key = SnapshotKey::new(&class(), &manifest).unwrap(); + let key = SnapshotKey::from_capture(&context(), &manifest).unwrap(); write_private(&dir.join(KEY_FILE), &key.to_bytes()).unwrap(); write_private(&dir.join(DESCRIPTION_FILE), b"not JSON").unwrap(); write_private(&openvmm.join(MANIFEST_NAME), &spec.encode()).unwrap(); @@ -1197,7 +1205,7 @@ fn snapshot_open_refuses_a_key_from_another_manifest_machine() { let mut spec = fixture.spec.clone(); mutate(&mut spec); let manifest = SnapshotManifest::parse(&spec.encode()).unwrap(); - let key = SnapshotKey::new(&class(), &manifest).unwrap(); + let key = SnapshotKey::from_capture(&context(), &manifest).unwrap(); write_private(&fixture.dir.join(KEY_FILE), &key.to_bytes()).unwrap(); refused(&fixture.dir); } @@ -1234,7 +1242,7 @@ fn snapshot_open_refuses_a_compatible_key_with_another_capture_time() { let mut spec = fixture.spec.clone(); mutate(&mut spec); let manifest = SnapshotManifest::parse(&spec.encode()).unwrap(); - let key = SnapshotKey::new(&class(), &manifest).unwrap(); + let key = SnapshotKey::from_capture(&context(), &manifest).unwrap(); assert_eq!(key.identity(), fixture.key.identity()); write_private(&fixture.dir.join(KEY_FILE), &key.to_bytes()).unwrap(); refused(&fixture.dir); @@ -1524,7 +1532,7 @@ impl RestoreFixture { openvmm: snapshot.root.path().join("unread-openvmm"), kernel: snapshot.root.path().join("unread-kernel"), initramfs: snapshot.root.path().join("unread-initramfs"), - identity: class().runtime, + identity: context().runtime, hypervisor: Hypervisor::Kvm, cpu_profile: Some("intel.icelake-sp.v1".to_owned()), windows_breakaway: false, @@ -1549,15 +1557,20 @@ impl RestoreFixture { fn write_metadata(&mut self) { let manifest = SnapshotManifest::parse(&self.snapshot.spec.encode()).unwrap(); - let mut class = class(); - class.processors = self.snapshot.spec.vp_count; - class.boot_processors = match manifest.machine.boot_processors { - 0 => class.processors, - count => count, + let mut context = context(); + let processors = self.snapshot.spec.vp_count; + context.capture.processors = Capacity { + boot: u64::from(match manifest.machine.boot_processors { + 0 => processors, + count => count, + }), + capacity: u64::from(processors), + }; + context.capture.disks = DiskCapture::Fresh { + image: manifest.machine.distro, + scratch: manifest.machine.scratch, }; - class.distro = manifest.machine.distro; - class.scratch = manifest.machine.scratch; - class.network = self + context.nic = self .snapshot .spec .contract @@ -1565,18 +1578,15 @@ impl RestoreFixture { .unwrap() .network .as_ref() - .map(|network| { - SandboxNetwork::new( - &format!( - "{}/{}", - std::net::Ipv4Addr::from(network.ipv4), - network.prefix_length - ), - NetworkPolicy::new(Access::Allow), - ) - .unwrap() + .map(|network| NicConfig { + guest_network: Ipv4Cidr::parse(&format!( + "{}/{}", + std::net::Ipv4Addr::from(network.ipv4), + network.prefix_length + )) + .unwrap(), }); - self.snapshot.key = SnapshotKey::new(&class, &manifest).unwrap(); + self.snapshot.key = SnapshotKey::from_capture(&context, &manifest).unwrap(); crate::host::platform::write_private( &self.snapshot.dir.join(KEY_FILE), &self.snapshot.key.to_bytes(), diff --git a/nvx-rs/src/host/spawn.rs b/nvx-rs/src/host/spawn.rs index 2cee04fa5..802152b84 100644 --- a/nvx-rs/src/host/spawn.rs +++ b/nvx-rs/src/host/spawn.rs @@ -18,52 +18,39 @@ use crate::host::platform; use crate::host::platform::inherit::Inherited; #[cfg(not(windows))] use crate::host::process::POLL_INTERVAL; -use crate::host::process::{BootId, KILL_TIMEOUT, Process, ProcessRef}; +use crate::host::process::{BootId, Process, ProcessRef}; use crate::protocol::CAPABILITY_LEN; -/// How [`spawn`] or [`spawn_vm`] starts a VMM: its executable, arguments, and working directory, +/// How [`spawn_vm_with_mode`] starts a VMM: its executable, arguments, and working directory, /// whether it leaves the caller's job object on Windows, and the descriptors or handles of the /// caller that it inherits besides its standard streams. #[derive(Debug, Clone)] -pub struct VmmCommand<'a> { +pub(crate) struct VmmCommand<'a> { /// The VMM's executable. - pub program: PathBuf, + pub(crate) program: PathBuf, /// The VMM's arguments, after the name of its executable. - pub arguments: Vec, + pub(crate) arguments: Vec, /// The VMM's working directory. /// /// Windows permits at most 258 UTF-16 code units in its absolute path, including any verbatim /// prefix and trailing separator. It may differ from the directories of the files that the /// VMM uses, whose paths can be longer. - pub working_dir: PathBuf, + pub(crate) working_dir: PathBuf, /// Whether the VMM starts outside the caller's job object on Windows, which must permit /// breakaway, so that the VMM survives the job. Other hosts ignore it. - pub breakaway_from_job: bool, + #[cfg_attr( + not(windows), + allow(dead_code, reason = "only Windows has job objects to break away from") + )] + pub(crate) breakaway_from_job: bool, /// The descriptors or handles of the caller that the VMM inherits under their numbers in the /// caller, besides its standard streams. One that the list names twice is inherited once. - pub inherit: Vec>, + pub(crate) inherit: Vec>, } -impl VmmCommand<'_> { - /// Returns the command that starts `program` with `arguments` in `working_dir`, inside the - /// caller's job object, and with nothing of the caller's besides its standard streams. - pub fn new( - program: impl Into, - arguments: Vec, - working_dir: impl Into, - ) -> Self { - Self { - program: program.into(), - arguments, - working_dir: working_dir.into(), - breakaway_from_job: false, - inherit: Vec::new(), - } - } -} - -/// Starts the VMM that `command` describes, detached from the caller, with `capability` on its -/// standard input and `log` as its standard output and error. +/// Starts the VMM that `command` describes, as [`vm::spawn`](crate::host::vm::spawn) starts +/// OpenVMM: detached from the caller unless `foreground`, with `capability` on its standard input +/// and `log` as its standard output and error. /// /// OpenVMM reads its capability from standard input as soon as it starts, so the capability is /// written into a pipe whose write end is closed before the spawn: it never appears in arguments, @@ -76,59 +63,32 @@ impl VmmCommand<'_> { /// gives them whatever the caller's umask, outside directories with a default ACL, which Linux /// applies instead of the umask; on Windows, without a console window, and outside the caller's /// job object when `command` asks for it, in a job object of its own that is named after `log`, -/// which limits no resource but lets no process that the VMM starts break away from it, unless -/// `log` is not a file of a disk volume, such as `NUL` or a pipe, or what takes that name is not -/// the user's job. A process that the caller spawns while the VMM is created, which can inherit -/// the log as [`Inherited`] describes, never joins the VMM's job, so it holds the log's claim only -/// if the VMM has none. +/// which limits no resource but lets no process that the VMM starts break away from it. A process +/// that the caller spawns while the VMM is created, which can inherit the log as [`Inherited`] +/// describes, never joins the VMM's job. A `foreground` VMM, which the CLI's one-shot runs start, +/// stays in the caller's session and console instead. /// /// Record the VMM's identity right after the spawn, through [`Launched::process`], if the caller -/// recovers its launches. Fails with [`io::ErrorKind::InvalidInput`] on -/// Linux for a descriptor of a standard stream, 0 to 2, or one that is not close-on-exec, and on -/// Windows for a working directory whose absolute path exceeds 258 UTF-16 code units, including -/// any verbatim prefix and trailing separator, or for a batch file, which Windows would run -/// through `cmd.exe` with arguments that it reads as shell syntax, and with -/// [`io::ErrorKind::AlreadyExists`] on Windows -/// if a job of the user takes -/// the name of the job of `log` already, as described below; and with -/// [`io::ErrorKind::Unsupported`] for any descriptor or handle on hosts other than Linux and -/// Windows. +/// recovers its launches. Fails, before it creates any child, with +/// [`io::ErrorKind::InvalidInput`] on Linux for a descriptor of a standard stream, 0 to 2, or one +/// that is not close-on-exec, and on Windows for a working directory whose absolute path exceeds +/// 258 UTF-16 code units, including any verbatim prefix and trailing separator, or for a batch +/// file, which Windows would run through `cmd.exe` with arguments that it reads as shell syntax; +/// on Windows with [`io::ErrorKind::PermissionDenied`] for a log that is not a file of a disk +/// volume, such as `NUL` or a pipe, or for an untrusted object or object-manager link under the +/// name of its job, and with [`io::ErrorKind::AlreadyExists`] if a job of the user takes that name +/// already, as described below; and with [`io::ErrorKind::Unsupported`] for any descriptor or +/// handle on hosts other than Linux and Windows. /// /// On Windows, give each VMM a log file of its own, created anew, as -/// [`vm::create`](crate::host::vm::create) does. The -/// job of `log` is named after the file's identity, which the file keeps when it is truncated, as +/// [`vm::create`](crate::host::vm::create) does. The job of `log` is named after the file's +/// identity, which the file keeps when it is truncated, as /// [`File::create`](std::fs::File::create) truncates an existing file, and its name stays taken /// while another spawn with the file holds the job, or while a process that inherited the job's /// handle in the window of such a spawn still runs, as one spawned while an earlier VMM of the -/// file started may: a spawn with the file fails until then. -pub fn spawn( - command: &VmmCommand<'_>, - capability: &[u8; CAPABILITY_LEN], - log: File, -) -> io::Result { - spawn_inner(command, capability, log, false, false) -} - -/// Starts a VMM with the strict launch contract of [`crate::host::vm::VmmCommand`]. -/// -/// Uses [`spawn`]'s standard streams, inheritance allowlist, detachment, and process lifetime. -/// Unlike [`spawn`], on Windows this requires a trusted log-named job: a log that is not a file -/// of a disk volume, or an untrusted object or object-manager link under its job name, fails with -/// [`io::ErrorKind::PermissionDenied`] before any child is created. A job of the user already -/// holding that name fails with [`io::ErrorKind::AlreadyExists`], as in [`spawn`]. -/// -/// On other hosts this has the same behavior and errors as [`spawn`]. The caller retains its -/// owned inherited resources until the spawn returns, and keeps or ends the returned child as +/// file started may: a spawn with the file fails until then. The caller retains its owned +/// inherited resources until the spawn returns, and keeps or ends the returned child as /// [`Launched`] describes. -pub fn spawn_vm( - command: &VmmCommand<'_>, - capability: &[u8; CAPABILITY_LEN], - log: File, -) -> io::Result { - spawn_vm_with_mode(command, capability, log, false) -} - -/// Shares the strict VM spawn with the CLI's private foreground mode. pub(crate) fn spawn_vm_with_mode( command: &VmmCommand<'_>, capability: &[u8; CAPABILITY_LEN], @@ -209,15 +169,15 @@ fn spawn_inner( } } -/// A VMM that [`spawn`] or [`spawn_vm`] started, which the caller has not released yet: on Unix, +/// A VMM that [`spawn_vm_with_mode`] started, which the caller has not released yet: on Unix, /// its child process; on Windows, its process handle. On Unix the child's ID names no other /// process until [`wait_timeout`](Self::wait_timeout) reaps it; hold it with /// [`process`](Self::process) before waiting to keep a verified handle after it exits. /// /// Hold it with [`process`](Self::process). Then release it with [`detach`](Self::detach), or end it /// with [`kill_and_confirm_exit`](Self::kill_and_confirm_exit); the [`Process`] stays valid -/// either way. [`kill_and_wait`](Self::kill_and_wait) provides legacy, time-bounded cleanup. -pub struct Launched { +/// either way. +pub(crate) struct Launched { #[cfg(not(windows))] /// Child process retained until it is detached or terminated. child: Child, @@ -240,7 +200,7 @@ impl fmt::Debug for Launched { impl Launched { /// Returns the process ID. - pub fn id(&self) -> u32 { + pub(crate) fn id(&self) -> u32 { #[cfg(not(windows))] return self.child.id(); #[cfg(windows)] @@ -252,13 +212,13 @@ impl Launched { /// The caller has not reaped the process, so its ID and start time still identify it, and /// the reference of the `Process` is the one to record: a successor of the caller can find /// the process again with [`Process::open`]. The `Process` stays valid after - /// [`detach`](Self::detach), [`kill_and_confirm_exit`](Self::kill_and_confirm_exit), or - /// [`kill_and_wait`](Self::kill_and_wait) releases this `Launched`. A process that exited - /// already keeps its identity, and the `Process` reports that it exited. + /// [`detach`](Self::detach) or [`kill_and_confirm_exit`](Self::kill_and_confirm_exit) + /// releases this `Launched`. A process that exited already keeps its identity, and the + /// `Process` reports that it exited. /// /// Fails if the identity of the process or of the current boot cannot be read, and with /// [`io::ErrorKind::Unsupported`] on hosts other than Linux and Windows. - pub fn process(&self) -> io::Result { + pub(crate) fn process(&self) -> io::Result { #[cfg(windows)] let (handle, start_time) = self.process.pin()?; #[cfg(not(windows))] @@ -284,7 +244,7 @@ impl Launched { /// zombie while this process lives. If that thread cannot start, the child becomes a zombie /// after it exits; liveness checks treat zombies without threads as exited, so only the /// process table entry leaks. - pub fn detach(self) { + pub(crate) fn detach(self) { #[cfg(not(windows))] { let mut child = self.child; @@ -298,44 +258,15 @@ impl Launched { drop(self); } - /// Ends the process through its handle, which cannot race with the reuse of its ID, and - /// returns whether it exited within ten seconds. A process that did not is released as - /// [`detach`](Self::detach) releases it. - pub fn kill_and_wait(self) -> bool { - #[cfg(windows)] - return self.process.terminate(KILL_TIMEOUT); - #[cfg(not(windows))] - { - let mut child = self.child; - if child.kill().is_ok() { - let deadline = Instant::now() + KILL_TIMEOUT; - loop { - match child.try_wait() { - Ok(Some(_)) => return true, - Ok(None) if Instant::now() < deadline => thread::sleep(POLL_INTERVAL), - _ => break, - } - } - } - Self { - child, - reaped: self.reaped, - } - .detach(); - false - } - } - /// Terminates this owned child and waits without a deadline for proof of its exit. /// - /// Unlike [`kill_and_wait`](Self::kill_and_wait), this has no timeout and never hands off a - /// child to a background reaper. On Unix a successful call reaps the child. Even if - /// termination fails, it waits for the child's exit; a child that exited already succeeds. - /// Use this when setup fails after [`spawn_vm`] created a child, including when - /// [`process`](Self::process) cannot verify its identity. + /// It has no timeout and never hands off a child to a background reaper. On Unix a successful + /// call reaps the child. Even if termination fails, it waits for the child's exit; a child + /// that exited already succeeds. Use this when setup fails after [`spawn_vm_with_mode`] + /// created a child, including when [`process`](Self::process) cannot verify its identity. /// /// Fails if waiting cannot confirm exit, retaining both failures if termination also failed. - pub fn kill_and_confirm_exit(self) -> io::Result<()> { + pub(crate) fn kill_and_confirm_exit(self) -> io::Result<()> { #[cfg(windows)] return self.process.terminate_and_wait(); #[cfg(not(windows))] @@ -358,7 +289,14 @@ impl Launched { /// /// On Unix a successful wait reaps the child; call [`process`](Self::process) before it to /// retain a verified process handle. Later waits return the same status. - pub fn wait_timeout(&mut self, timeout: Duration) -> io::Result> { + #[cfg_attr( + not(feature = "cli"), + allow( + dead_code, + reason = "only the CLI's one-shot runs wait for the status of the VMMs that they start" + ) + )] + pub(crate) fn wait_timeout(&mut self, timeout: Duration) -> io::Result> { #[cfg(windows)] return Ok(self.process.wait_timeout(timeout)?.map(i64::from)); #[cfg(not(windows))] @@ -380,6 +318,13 @@ impl Launched { /// Reaps the child if it has exited, and returns its status if it has. #[cfg(not(windows))] + #[cfg_attr( + not(feature = "cli"), + allow( + dead_code, + reason = "only the CLI's one-shot runs wait for the status of the VMMs that they start" + ) + )] fn try_reap(&mut self) -> io::Result> { let status = self.child.try_wait()?; self.reaped |= status.is_some(); @@ -389,6 +334,13 @@ impl Launched { /// Returns `Popen.returncode` for `status`: the exit code, or the negated signal number. #[cfg(not(windows))] +#[cfg_attr( + not(feature = "cli"), + allow( + dead_code, + reason = "only the CLI's one-shot runs wait for the status of the VMMs that they start" + ) +)] fn status_value(status: std::process::ExitStatus) -> i64 { #[cfg(unix)] use std::os::unix::process::ExitStatusExt; @@ -405,5 +357,5 @@ fn status_value(status: std::process::ExitStatus) -> i64 { } } -#[cfg(all(test, target_os = "linux"))] +#[cfg(all(test, any(target_os = "linux", windows)))] mod tests; diff --git a/nvx-rs/src/host/spawn/tests.rs b/nvx-rs/src/host/spawn/tests.rs index 02ca109b3..268b72ba1 100644 --- a/nvx-rs/src/host/spawn/tests.rs +++ b/nvx-rs/src/host/spawn/tests.rs @@ -1,228 +1,942 @@ -use std::fs; -use std::os::fd::{AsFd, AsRawFd, BorrowedFd, RawFd}; -use std::path::Path; +//! Tests of the spawn of a VMM: the descriptors or handles that a VMM inherits while other +//! threads spawn, its working directory on Windows, the detachment or job that keeps it running +//! or ends it, and the waits and cleanup of the child that the spawn returns. +//! +//! The processes are this test executable, run again as one of two fixtures. A VMM fixture reads +//! the capability that the spawn passes on its standard input, as OpenVMM does, reports its +//! descriptors on Linux, writes a line through each descriptor or handle that its arguments name +//! as inherited, reports that it started, and runs until a file named `exit` appears in its +//! working directory, or, when the file `exit-status` names a status, until a file named `go` +//! appears, when it exits with that status. A launcher fixture spawns a VMM fixture, prints the +//! VMM's reference as an orchestrator would record it, and runs until its standard input closes. +//! A launcher that the test kills, or whose job object the test closes, stands for an +//! orchestrator that crashed and restarts within the same boot: the test then finds the VMM +//! again, as the restarted orchestrator would. -use nix::fcntl::{FcntlArg, FdFlag, fcntl}; +use std::env; +use std::ffi::OsString; +use std::fs::{self, File}; +use std::io::{self, BufRead, BufReader, PipeReader, PipeWriter, Read, Write}; +use std::mem::ManuallyDrop; +use std::path::{Path, PathBuf}; +use std::process::{Child, ChildStdin, Command, ExitStatus, Stdio}; +use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering}; +use std::sync::{Mutex, MutexGuard, PoisonError, mpsc}; +use std::thread; +use std::time::{Duration, Instant}; use super::*; -use crate::host::sys; +use crate::host::process::{Opened, WaitMethod}; -/// Kills and reaps a child process when dropped, even when a test fails. -struct Reaped(Child); +/// The spawn's descriptors and umask on Linux. +#[cfg(target_os = "linux")] +mod linux; -impl Drop for Reaped { - fn drop(&mut self) { - let _ = self.0.kill(); - let _ = self.0.wait(); +/// The VMM fixture of this test executable. +const VMM_FIXTURE: &str = "host::spawn::tests::vmm_fixture"; +/// The launcher fixture of this test executable. +const LAUNCHER_FIXTURE: &str = "host::spawn::tests::launcher_fixture"; +/// File whose presence in its working directory marks a VMM fixture. Without it, the fixture +/// returns at once, as it does when `cargo test -- --ignored` runs it. +const VMM_MARKER: &str = "vmm-fixture"; +/// File whose appearance in its working directory ends a VMM fixture. +const EXIT_MARKER: &str = "exit"; +/// File that names the status with which a VMM fixture exits once [`GO_MARKER`] appears. +const EXIT_STATUS: &str = "exit-status"; +/// File whose appearance in its working directory lets a VMM fixture exit with [`EXIT_STATUS`]. +const GO_MARKER: &str = "go"; +/// File that a VMM fixture writes once it has started. +const STARTED_REPORT: &str = "started"; +/// File in which a VMM fixture reports its descriptors on Linux, a line for each: its number and +/// the object that it holds. +#[cfg(target_os = "linux")] +const DESCRIPTORS_REPORT: &str = "descriptors"; +/// Prefix of an argument of a VMM fixture that names, in decimal, a descriptor or handle that the +/// fixture inherited, and through which it writes a line. Libtest takes the argument for a test +/// name that matches no test. +const INHERITED_PREFIX: &str = "inherited="; +/// Environment variable that marks a launcher fixture, and names the working directory of its +/// VMM. +const LAUNCHER_DIR: &str = "NVX_SPAWN_TEST_VMM_DIR"; +/// Environment variable that asks a launcher fixture to start its VMM outside its job object. +const LAUNCHER_BREAKAWAY: &str = "NVX_SPAWN_TEST_BREAKAWAY"; +/// Marker before the reference that a launcher fixture prints. Libtest's own output, the name of +/// the fixture, precedes it on its line. +const REFERENCE_PREFIX: &str = "vmm-reference "; +/// The capability of every VMM of these tests. +const CAPABILITY: [u8; CAPABILITY_LEN] = [0x5a; CAPABILITY_LEN]; +/// Longest wait for a process that is expected to exit, or for a fixture to answer. +const TIMEOUT: Duration = Duration::from_secs(60); +/// Longest life of a VMM fixture that nothing ends, so that a failed test leaves no process. +const VMM_LIFETIME: Duration = Duration::from_secs(180); +/// A wait that a running process outlasts. +const SHORT_WAIT: Duration = Duration::from_millis(100); + +/// Serializes the tests that spawn VMM fixtures. On Windows, a process that one test spawns while +/// another test's handles are inheritable for a spawn of its own, as the log of a launch is, +/// could hold them, and so the claim of that log, for a while. +static SPAWNS: Mutex<()> = Mutex::new(()); + +/// Takes [`SPAWNS`]. +fn serialized() -> MutexGuard<'static, ()> { + SPAWNS.lock().unwrap_or_else(PoisonError::into_inner) +} + +/// The VMM fixture: reads its capability as OpenVMM does, reports its descriptors on Linux, +/// writes a line through each descriptor or handle that an argument names as inherited, reports +/// that it started, then runs until a file named [`EXIT_MARKER`] appears in its working +/// directory, or exits with the status that [`EXIT_STATUS`] names once [`GO_MARKER`] appears. +#[test] +#[ignore = "fixture that the spawn tests run as a VMM"] +fn vmm_fixture() { + let directory = env::current_dir().unwrap(); + if !directory.join(VMM_MARKER).exists() { + return; + } + let mut capability = [0; CAPABILITY_LEN]; + io::stdin().read_exact(&mut capability).unwrap(); + assert_eq!(capability, CAPABILITY); + #[cfg(target_os = "linux")] + report_descriptors(&directory); + for argument in env::args() { + if let Some(number) = argument.strip_prefix(INHERITED_PREFIX) { + // A VMM keeps what it inherits open. + let mut inherited = ManuallyDrop::new(inherited_file(number.parse().unwrap())); + writeln!(inherited, "inherited {number}").unwrap(); + } + } + write_whole(&directory.join(STARTED_REPORT), ""); + let status = fs::read_to_string(directory.join(EXIT_STATUS)).ok(); + let started = Instant::now(); + while !directory.join(EXIT_MARKER).exists() && started.elapsed() < VMM_LIFETIME { + if let Some(status) = &status + && directory.join(GO_MARKER).exists() + { + std::process::exit(status.parse().unwrap()); + } + thread::sleep(Duration::from_millis(5)); } } -/// Returns the object that the descriptor `descriptor` of process `pid` holds, as the target -/// of its link in `/proc`, or `None` if the process has no such descriptor. -fn object(pid: u32, descriptor: RawFd) -> Option { - fs::read_link(format!("/proc/{pid}/fd/{descriptor}")).ok() +/// Writes `text` to the file at `path`, which appears whole. +fn write_whole(path: &Path, text: &str) { + let temporary = path.with_extension("tmp"); + fs::write(&temporary, text).unwrap(); + fs::rename(&temporary, path).unwrap(); } -/// Returns whether process `pid` holds a descriptor of `object`, under any number. -fn holds(pid: u32, object: &Path) -> bool { - fs::read_dir(format!("/proc/{pid}/fd")) +/// Writes the descriptors of this VMM fixture, but for the one that lists them, to the file +/// [`DESCRIPTORS_REPORT`] in `directory`, which appears whole. +#[cfg(target_os = "linux")] +fn report_descriptors(directory: &Path) { + // The descriptor that lists the others holds this directory. + let listing = PathBuf::from(format!("/proc/{}/fd", std::process::id())); + let mut descriptors: Vec<(usize, PathBuf)> = fs::read_dir("/proc/self/fd") .unwrap() - .any(|entry| fs::read_link(entry.unwrap().path()).is_ok_and(|target| target == object)) -} - -/// Returns whether the descriptor `descriptor` of this process is close-on-exec. -fn close_on_exec(descriptor: BorrowedFd<'_>) -> bool { - FdFlag::from_bits_retain(fcntl(descriptor, FcntlArg::F_GETFD).unwrap()) - .contains(FdFlag::FD_CLOEXEC) -} - -/// Spawns a long-running child with `spawn` while this process holds an inheritable pipe -/// writer, as a descriptor that the caller's parent or native code left without close-on-exec -/// would be. Returns whether the child holds a copy, as its own descriptor table shows. Unlike -/// the pipe reaching end-of-file, that cannot depend on the children of concurrent tests, -/// which may inherit the writer too. -fn child_inherits_writer(spawn: impl FnOnce() -> Child) -> bool { - let (pipe, child) = { - let (_, writer) = io::pipe().unwrap(); - sys::tests::clear_close_on_exec(&writer).unwrap(); - let pipe = object(std::process::id(), writer.as_raw_fd()).unwrap(); - (pipe, Reaped(spawn())) - }; - // `spawn` returns once the child has executed its program, which closed the child's - // close-on-exec copies, so a copy that remains was inherited. - holds(child.0.id(), &pipe) + .filter_map(|entry| { + let entry = entry.unwrap(); + let number = entry.file_name().to_str().unwrap().parse().unwrap(); + let object = fs::read_link(entry.path()).ok()?; + (object != listing).then_some((number, object)) + }) + .collect(); + descriptors.sort(); + let report: String = descriptors + .iter() + .map(|(number, object)| format!("{number} {}\n", object.display())) + .collect(); + write_whole(&directory.join(DESCRIPTORS_REPORT), &report); } -/// Returns a command that runs for a minute unless it is killed. -fn sleeper() -> Command { - let mut command = Command::new("sleep"); - command.arg("60").stdin(Stdio::null()); - command +/// Takes the descriptor `number`, which this VMM fixture inherited. +#[cfg(target_os = "linux")] +#[allow(unsafe_code)] +fn inherited_file(number: usize) -> File { + use std::os::fd::FromRawFd; + + // SAFETY: the test named a descriptor that this process inherited under that number, and + // that nothing else in this process owns. + unsafe { File::from_raw_fd(i32::try_from(number).unwrap()) } } -/// Starts `sleep 60` as a VMM in `directory`, which also inherits `inherit`. -fn spawn_sleeper(directory: &Path, inherit: Vec>) -> io::Result { - let log = File::create(directory.join("vmm.log"))?; - let mut command = VmmCommand::new("sleep", vec![OsString::from("60")], directory); - command.inherit = inherit; - spawn(&command, &[1; CAPABILITY_LEN], log) +/// Takes the handle `number`, which this VMM fixture inherited. +#[cfg(windows)] +#[allow(unsafe_code)] +fn inherited_file(number: usize) -> File { + use std::os::windows::io::{FromRawHandle, RawHandle}; + + // SAFETY: the test named a handle that this process inherited under that value, and that + // nothing else in this process owns. + unsafe { File::from_raw_handle(number as RawHandle) } } +/// The launcher fixture: when the test writes a line to its standard input, spawns a VMM fixture +/// in the directory that [`LAUNCHER_DIR`] names, prints its reference, releases it, and runs +/// until its standard input closes. #[test] -fn vmms_inherit_no_descriptor_beyond_their_standard_streams() { - // A plain child keeps the writer, which shows that the check detects a leak. - let plain = || sleeper().spawn().unwrap(); - assert!(child_inherits_writer(plain)); - - let directory = tempfile::tempdir().unwrap(); - let vmm = || spawn_sleeper(directory.path(), Vec::new()).unwrap().child; - assert!(!child_inherits_writer(vmm)); - - // Kernels without CLOSE_RANGE_CLOEXEC take the /proc/self/fd fallback. - let fallback = || { - let mut command = sleeper(); - sys::tests::mark_listed_descriptors_on_spawn(&mut command); - command.spawn().unwrap() +#[ignore = "fixture that the spawn tests run as a launcher"] +fn launcher_fixture() { + let Some(directory) = env::var_os(LAUNCHER_DIR) else { + return; }; - assert!(!child_inherits_writer(fallback)); + let mut input = io::stdin().lock(); + input.read_line(&mut String::new()).unwrap(); + let breakaway = env::var_os(LAUNCHER_BREAKAWAY).is_some(); + let launched = spawn_vmm(Path::new(&directory), breakaway); + print_reference(launched.process().unwrap().reference()); + launched.detach(); + io::copy(&mut input, &mut io::sink()).unwrap(); } -#[test] -fn vmms_inherit_the_allowed_descriptors_under_their_numbers() { - let directory = tempfile::tempdir().unwrap(); - let (_reader, pipe) = io::pipe().unwrap(); - let file = File::create(directory.path().join("distro")).unwrap(); - // A descriptor that the list names twice is inherited once. - let inherit = vec![ - Inherited::new(&pipe), - Inherited::new(&file), - Inherited::new(&pipe), - ]; - let vmm = Reaped(spawn_sleeper(directory.path(), inherit).unwrap().child); - let (pid, this) = (vmm.0.id(), std::process::id()); - for descriptor in [pipe.as_raw_fd(), file.as_raw_fd()] { - let inherited = object(pid, descriptor); - assert!( - inherited.is_some(), - "descriptor {descriptor} was not inherited" - ); - assert_eq!(inherited, object(this, descriptor)); - } - // Only the VMM's process cleared their close-on-exec flags. - assert!(close_on_exec(pipe.as_fd())); - assert!(close_on_exec(file.as_fd())); - assert_eq!( - Inherited::new(&file).proc_path(), - PathBuf::from(format!("/proc/self/fd/{}", file.as_raw_fd())) +/// Prints `reference` as JSON, after [`REFERENCE_PREFIX`]. +fn print_reference(reference: &ProcessRef) { + println!( + "{REFERENCE_PREFIX}{}", + serde_json::to_string(reference).unwrap() ); } +/// Returns the arguments that run only the fixture `name` of this test executable. +/// +/// The fixture runs on libtest's main thread, whatever the host's processors or the inherited +/// `RUST_TEST_THREADS`, so libtest always prints its name before the fixture's own output. +fn fixture_arguments(name: &str) -> Vec { + [ + "--exact", + name, + "--ignored", + "--nocapture", + "--test-threads=1", + ] + .map(OsString::from) + .to_vec() +} + +/// Returns the command of a VMM fixture in `directory`, which it marks, outside the caller's job +/// object if `breakaway`. +fn vmm_command(directory: &Path, breakaway: bool) -> VmmCommand<'static> { + File::create(directory.join(VMM_MARKER)).unwrap(); + VmmCommand { + program: env::current_exe().unwrap(), + arguments: fixture_arguments(VMM_FIXTURE), + working_dir: directory.to_path_buf(), + breakaway_from_job: breakaway, + inherit: Vec::new(), + } +} + +/// Spawns a VMM fixture in `directory` as OpenVMM is spawned, outside the caller's job object if +/// `breakaway`. +fn spawn_vmm(directory: &Path, breakaway: bool) -> Launched { + let log = File::create(directory.join("vmm.log")).unwrap(); + spawn_vm_with_mode(&vmm_command(directory, breakaway), &CAPABILITY, log, false).unwrap() +} + +/// The working directory of a VMM fixture. Dropping it ends the fixture, or a VMM that ignores +/// the exit marker, and waits for it to exit, so that a failed test leaves no process behind. +struct VmmDirectory { + /// The reference of the VMM, once the test knows it. + reference: Option, + /// The launched VMM, if the test holds it. + launched: Option, + /// The directory, which is removed after the VMM exits. + directory: tempfile::TempDir, +} + +impl VmmDirectory { + /// Creates an empty directory. + fn new() -> Self { + Self { + reference: None, + launched: None, + directory: tempfile::tempdir().unwrap(), + } + } + + /// Returns the path of the directory. + fn path(&self) -> &Path { + self.directory.path() + } + + /// Records the reference of the VMM that runs in the directory. + fn record(&mut self, reference: &ProcessRef) { + self.reference = Some(reference.clone()); + } + + /// Asks the VMM to exit. + fn end(&self) { + File::create(self.path().join(EXIT_MARKER)).unwrap(); + } + + /// Waits for the VMM's report `name`, and returns it. + fn report(&self, name: &str) -> String { + let path = self.path().join(name); + let deadline = Instant::now() + TIMEOUT; + while !path.exists() { + assert!(Instant::now() < deadline, "the VMM did not report {name}"); + thread::sleep(Duration::from_millis(10)); + } + fs::read_to_string(path).unwrap() + } +} + +impl Drop for VmmDirectory { + fn drop(&mut self) { + let _ = File::create(self.path().join(EXIT_MARKER)); + if let Some(reference) = &self.reference + && let Ok(Opened::Running(process)) = Process::open(reference) + && !process + .wait(Instant::now() + TIMEOUT) + .is_ok_and(|exited| exited.is_some()) + { + let _ = process.kill(); + let _ = process.wait(Instant::now() + TIMEOUT); + } + if let Some(launched) = self.launched.take() { + if let Ok(process) = launched.process() { + let _ = process.wait(Instant::now() + TIMEOUT); + } + let _ = launched.kill_and_confirm_exit(); + } + } +} + +/// A launcher fixture, which the test drives through its standard streams. +struct Launcher { + /// The fixture's process. + child: Child, + /// Its standard input, which the test closes to end it. + input: Option, +} + +impl Launcher { + /// Starts a launcher whose VMM runs in `directory`, outside the launcher's job object if + /// `breakaway`. + fn start(directory: &Path, breakaway: bool) -> Self { + let mut command = Command::new(env::current_exe().unwrap()); + command + .args(fixture_arguments(LAUNCHER_FIXTURE)) + .env(LAUNCHER_DIR, directory); + if breakaway { + command.env(LAUNCHER_BREAKAWAY, "1"); + } + command.stdin(Stdio::piped()).stdout(Stdio::piped()); + let mut child = crate::host::tests::spawn_locked(&mut command).unwrap(); + let input = child.stdin.take(); + Self { child, input } + } + + /// Asks the fixture to start its VMM, and returns the reference that it prints, as the + /// fixture would record it: as JSON. + fn launch(&mut self) -> String { + let input = self.input.as_mut().unwrap(); + input.write_all(b"launch\n").unwrap(); + input.flush().unwrap(); + let output = self.child.stdout.take().unwrap(); + let (sender, receiver) = mpsc::channel(); + thread::spawn(move || { + let mut lines = BufReader::new(output).lines(); + let reference = lines.by_ref().find_map(|line| { + let line = line.ok()?; + Some(line.split_once(REFERENCE_PREFIX)?.1.to_owned()) + }); + let _ = sender.send(reference); + // Read the rest, so that the fixture can still write its own output. + lines.for_each(drop); + }); + receiver + .recv_timeout(TIMEOUT) + .expect("the fixture printed no reference in time") + .expect("the fixture exited before it printed a reference") + } + + /// Asks the fixture to start its VMM, and returns the reference that it prints. + #[cfg(windows)] + fn launch_reference(&mut self) -> ProcessRef { + serde_json::from_str(&self.launch()).unwrap() + } + + /// Waits for the fixture to exit, and returns its status. + fn wait(&mut self) -> ExitStatus { + let deadline = Instant::now() + TIMEOUT; + loop { + if let Some(status) = self.child.try_wait().unwrap() { + return status; + } + assert!(Instant::now() < deadline, "the fixture did not exit"); + thread::sleep(Duration::from_millis(10)); + } + } +} + +impl Drop for Launcher { + fn drop(&mut self) { + let _ = self.child.kill(); + let _ = self.child.wait(); + } +} + +/// Opens the process that `reference` names, which must still run. +fn open_running(reference: &ProcessRef) -> Process { + let Opened::Running(process) = Process::open(reference).unwrap() else { + panic!("{reference:?} does not run"); + }; + process +} + +/// Checks that the process that `process` holds has exited, and that no process matches its +/// reference. +fn assert_ended(process: &Process) { + assert!(process.has_exited().unwrap()); + assert!(matches!( + Process::open(process.reference()).unwrap(), + Opened::Exited(_) + )); +} + #[test] -fn standard_streams_cannot_be_inherited() { - let directory = tempfile::tempdir().unwrap(); - for stream in [ - io::stdin().as_fd(), - io::stdout().as_fd(), - io::stderr().as_fd(), +#[cfg(windows)] +fn working_directory_limits_include_trailing_separators() { + use std::os::windows::ffi::OsStrExt; + + let _spawns = serialized(); + let root = tempfile::tempdir().unwrap(); + for (units, trailing, accepted) in [ + (258, false, true), + (259, false, false), + (257, true, true), + (258, true, false), ] { - let error = spawn_sleeper(directory.path(), vec![Inherited::from(stream)]).unwrap_err(); - assert_eq!(error.kind(), io::ErrorKind::InvalidInput); + let name = "w".repeat(units - root.path().as_os_str().encode_wide().count() - 1); + let directory = root.path().join(name); + fs::create_dir_all(&directory).unwrap(); + let working = if trailing { + directory.join("") + } else { + directory.clone() + }; + let log = File::create(directory.join(format!("vmm-{units}-{trailing}.log"))).unwrap(); + let launched = spawn_vm_with_mode(&vmm_command(&working, false), &CAPABILITY, log, false); + if accepted { + launched.unwrap().kill_and_confirm_exit().unwrap(); + } else { + let error = launched + .map(|launched| launched.kill_and_confirm_exit().unwrap()) + .unwrap_err(); + assert_eq!(error.kind(), io::ErrorKind::InvalidInput); + assert!(error.to_string().contains("258")); + } } } #[test] -fn descriptors_that_are_not_close_on_exec_are_refused() { - let directory = tempfile::tempdir().unwrap(); - // As a descriptor that the caller's parent or native code left without close-on-exec, - // which any process that the caller spawns would inherit, and not only the VMM. - let (_reader, writer) = io::pipe().unwrap(); - sys::tests::clear_close_on_exec(&writer).unwrap(); - let error = spawn_sleeper(directory.path(), vec![Inherited::new(&writer)]).unwrap_err(); - assert_eq!(error.kind(), io::ErrorKind::InvalidInput); - assert!(error.to_string().contains("close-on-exec"), "{error}"); - // The refusal, which comes before the spawn, leaves the descriptor as it was. - assert!(!close_on_exec(writer.as_fd())); -} - -/// Returns the umask that the status file of a process or a thread at `status`, in procfs, -/// tells. -fn umask_in(status: &str) -> u32 { - fs::read_to_string(status) - .unwrap() - .lines() - .find_map(|line| line.strip_prefix("Umask:")) - .and_then(|umask| u32::from_str_radix(umask.trim(), 8).ok()) - .unwrap() +fn a_restarted_launcher_finds_its_vmm_again_and_ends_it() { + let _spawns = serialized(); + let mut vmm = VmmDirectory::new(); + // On Windows, the launcher runs in a job object that terminates it when the job closes, and + // its VMM breaks away from the job, as an orchestrator's VMMs do to survive it. + #[cfg(windows)] + let job = job::Job::new(true); + let mut launcher = Launcher::start(vmm.path(), cfg!(windows)); + #[cfg(windows)] + job.assign(&launcher.child); + let recorded = launcher.launch(); + // The launcher crashes. + #[cfg(windows)] + drop(job); + #[cfg(not(windows))] + launcher.child.kill().unwrap(); + launcher.wait(); + + // Its successor reads the recorded reference back, and finds the VMM, which survived. + let reference: ProcessRef = serde_json::from_str(&recorded).unwrap(); + vmm.record(&reference); + let process = open_running(&reference); + assert_eq!(process.wait_method(), WaitMethod::Event); + assert!(process.wait(Instant::now() + SHORT_WAIT).unwrap().is_none()); + let started = Instant::now(); + process.kill().unwrap(); + assert!(process.wait(started + TIMEOUT).unwrap().is_some()); + assert_ended(&process); } -/// Asserts that the process that `spawn` starts in a directory of its own, from a thread whose -/// umask no other thread shares, has the thread's umask without the owner's bits, and that -/// the thread keeps its own, for the usual umasks, which keep the owner's bits, and for umasks -/// that clear some of them. -fn assert_spawns_clear_the_owners_bits(spawn: impl Fn(&Path) -> Child + Sync) { - use nix::sys::stat::{Mode, umask}; +/// Sets its flag when dropped, also when its thread panics. +struct SetOnDrop<'a>(&'a AtomicBool); - for mask in [ - 0o002, 0o022, 0o027, 0o077, 0o100, 0o200, 0o400, 0o477, 0o777, - ] { - let directory = tempfile::tempdir().unwrap(); - let (child, caller) = thread::scope(|scope| { - scope - .spawn(|| { - nix::sched::unshare(nix::sched::CloneFlags::CLONE_FS).unwrap(); - umask(Mode::from_bits_truncate(mask)); - let child = Reaped(spawn(directory.path())); - (child, umask_in("/proc/thread-self/status")) - }) - .join() - .unwrap_or_else(|panic| std::panic::resume_unwind(panic)) - }); - assert_eq!( - caller, mask, - "the caller's umask {mask:03o} became {caller:03o}" +impl Drop for SetOnDrop<'_> { + fn drop(&mut self) { + self.0.store(true, Ordering::SeqCst); + } +} + +#[test] +fn vmms_inherit_only_their_streams_and_allowed_descriptors_while_others_spawn() { + let _spawns = serialized(); + /// The VMMs that the test launches. + const VMMS: usize = 6; + let (readers, writers): (Vec, Vec) = + (0..VMMS).map(|_| io::pipe().unwrap()).unzip(); + // Only the VMM that inherits one of these pipes may hold it. + #[cfg(target_os = "linux")] + let forbidden: Vec = writers + .iter() + .map(|writer| this_object(number_of(writer))) + .collect(); + #[cfg(windows)] + let forbidden: Vec = Vec::new(); + let done = AtomicBool::new(false); + let rounds = AtomicUsize::new(0); + let others = Mutex::new(Vec::new()); + let leaks = thread::scope(|scope| { + let stop_spawner = SetOnDrop(&done); + let spawner = scope.spawn(|| spawn_others(&done, &rounds, &others, &forbidden)); + // The other thread spawns throughout the launches of the VMMs. + let deadline = Instant::now() + TIMEOUT; + while rounds.load(Ordering::SeqCst) == 0 { + assert!( + Instant::now() < deadline && !spawner.is_finished(), + "the other thread spawned nothing" + ); + thread::sleep(Duration::from_millis(10)); + } + for (reader, writer) in readers.into_iter().zip(&writers) { + check_vmm(reader, writer, &others); + } + drop(stop_spawner); + spawner.join().unwrap() + }); + assert!(rounds.load(Ordering::SeqCst) > 0); + assert!(leaks.is_empty(), "{leaks:?}"); +} + +/// Spawns processes until `done`, as other threads of an orchestrator do, and counts its rounds +/// in `rounds`. Each round spawns a VMM fixture through the spawn of a VMM, which inherits a pipe +/// of its own, and a process through `std::process`, which inherits another pipe, which the round +/// makes inheritable for the spawn, as code that spawns without an allowlist does. +/// +/// It lists the numbers of those pipes in `others`, and keeps the pipes open until it returns. +/// It returns the processes that it spawned through `std::process` that held a descriptor of any +/// of the objects `forbidden`. +fn spawn_others( + done: &AtomicBool, + rounds: &AtomicUsize, + others: &Mutex>, + forbidden: &[PathBuf], +) -> Vec { + // Without the VMM marker, the fixtures that it spawns return at once. + let directory = tempfile::tempdir().unwrap(); + let mut pipes = Vec::new(); + let mut leaks = Vec::new(); + while !done.load(Ordering::SeqCst) { + let (_, allowed) = io::pipe().unwrap(); + let (_, naive) = io::pipe().unwrap(); + others + .lock() + .unwrap() + .extend([number_of(&allowed), number_of(&naive)]); + + let command = VmmCommand { + program: env::current_exe().unwrap(), + arguments: fixture_arguments(VMM_FIXTURE), + working_dir: directory.path().to_path_buf(), + breakaway_from_job: false, + inherit: vec![Inherited::new(&allowed)], + }; + let round = rounds.load(Ordering::SeqCst); + let log = File::create(directory.path().join(format!("{round}.log"))).unwrap(); + spawn_vm_with_mode(&command, &CAPABILITY, log, false) + .unwrap() + .kill_and_confirm_exit() + .unwrap(); + + set_inheritable(&naive, true); + let child = sleeper().spawn(); + set_inheritable(&naive, false); + let mut child = child.unwrap(); + leaks.extend( + held_objects(child.id(), forbidden) + .into_iter() + .map(|object| format!("process {} held {}", child.id(), object.display())), ); - let pid = child.0.id(); - let inherited = umask_in(&format!("/proc/{pid}/status")); - assert_eq!( - inherited, - mask & 0o077, - "the caller's umask {mask:03o} gave the process {inherited:03o}" + let _ = child.kill(); + child.wait().unwrap(); + pipes.extend([allowed, naive]); + rounds.fetch_add(1, Ordering::SeqCst); + } + leaks +} + +/// Spawns a VMM fixture that inherits `writer` while another thread spawns processes whose pipes +/// `others` lists, and checks that the VMM holds its standard streams and `writer`, under its +/// number, and nothing else of this process's. +fn check_vmm(reader: PipeReader, writer: &PipeWriter, others: &Mutex>) { + let mut vmm = VmmDirectory::new(); + let number = number_of(writer); + let mut command = vmm_command(vmm.path(), false); + command + .arguments + .push(format!("{INHERITED_PREFIX}{number}").into()); + command.inherit.push(Inherited::new(writer)); + let log = File::create(vmm.path().join("vmm.log")).unwrap(); + let launched = spawn_vm_with_mode(&command, &CAPABILITY, log, false).unwrap(); + let process = launched.process().unwrap(); + vmm.record(process.reference()); + // The VMM writes through its pipe, under the number that its arguments name. + assert_eq!(read_line(reader), format!("inherited {number}\n")); + #[cfg(target_os = "linux")] + { + let _ = others; + let reported = reported_descriptors(vmm.path()); + let numbers: Vec = reported.iter().map(|(number, _)| *number).collect(); + assert_eq!(numbers, [0, 1, 2, number], "{reported:?}"); + let log = fs::canonicalize(vmm.path().join("vmm.log")).unwrap(); + assert!( + reported[0].1.to_string_lossy().starts_with("pipe:"), + "{reported:?}" ); + assert_eq!(reported[1].1, log); + assert_eq!(reported[2].1, log); + assert_eq!(reported[3].1, this_object(number)); } + #[cfg(windows)] + { + let pid = launched.id(); + assert!(holds(pid, number), "the VMM did not inherit its pipe"); + for &other in others.lock().unwrap().iter() { + assert!( + !holds(pid, other), + "the VMM inherited handle {other} of another spawn" + ); + } + } + vmm.end(); + assert!(process.wait(Instant::now() + TIMEOUT).unwrap().is_some()); + launched.kill_and_confirm_exit().unwrap(); } -#[test] -fn vmms_start_with_the_callers_umask_without_the_owners_bits() { - assert_spawns_clear_the_owners_bits(|directory| { - spawn_sleeper(directory, Vec::new()).unwrap().child +/// Reads a line from `reader` within [`TIMEOUT`]. +fn read_line(reader: PipeReader) -> String { + let (sender, receiver) = mpsc::channel(); + thread::spawn(move || { + let mut line = String::new(); + let _ = BufReader::new(reader).read_line(&mut line); + let _ = sender.send(line); }); + receiver + .recv_timeout(TIMEOUT) + .expect("the VMM wrote no line in time") +} + +/// Returns the number of the descriptor of `pipe`, which a VMM inherits under that number. +#[cfg(target_os = "linux")] +fn number_of(pipe: &PipeWriter) -> usize { + use std::os::fd::AsRawFd; + + usize::try_from(pipe.as_raw_fd()).unwrap() +} + +/// Returns the value of the handle of `pipe`, which a VMM inherits under that value. +#[cfg(windows)] +fn number_of(pipe: &PipeWriter) -> usize { + use std::os::windows::io::AsRawHandle; + + pipe.as_raw_handle() as usize +} + +/// Returns the object that the descriptor `number` of this process holds, as its link in `/proc` +/// names it. +#[cfg(target_os = "linux")] +fn this_object(number: usize) -> PathBuf { + fs::read_link(format!("/proc/self/fd/{number}")).unwrap() +} + +/// Waits for the VMM fixture in `directory` to report its descriptors, and returns them, with the +/// objects that they hold. +#[cfg(target_os = "linux")] +fn reported_descriptors(directory: &Path) -> Vec<(usize, PathBuf)> { + let path = directory.join(DESCRIPTORS_REPORT); + let deadline = Instant::now() + TIMEOUT; + let report = loop { + match fs::read_to_string(&path) { + Ok(report) => break report, + Err(error) if error.kind() == io::ErrorKind::NotFound && Instant::now() < deadline => { + thread::sleep(Duration::from_millis(10)); + } + Err(error) => panic!("the VMM reported no descriptors: {error}"), + } + }; + report + .lines() + .map(|line| { + let (number, object) = line.split_once(' ').unwrap(); + (number.parse().unwrap(), PathBuf::from(object)) + }) + .collect() +} + +/// Returns the objects of `forbidden` that process `pid` holds a descriptor of, once its exec +/// has closed the descriptors that it did not inherit. +/// +/// The kernel wakes the parent of a `vfork`, as `posix_spawn` makes one, before the child's exec +/// closes its close-on-exec descriptors, so for an instant after the spawn returns, the child's +/// table still lists every descriptor of the parent. A descriptor that the child inherited stays, +/// so this looks again until the objects are gone or a second has passed. +#[cfg(target_os = "linux")] +fn held_objects(pid: u32, forbidden: &[PathBuf]) -> Vec { + let deadline = Instant::now() + Duration::from_secs(1); + loop { + let held: Vec = fs::read_dir(format!("/proc/{pid}/fd")) + .unwrap() + .filter_map(|entry| fs::read_link(entry.unwrap().path()).ok()) + .filter(|object| forbidden.contains(object)) + .collect(); + if held.is_empty() || Instant::now() >= deadline { + return held; + } + thread::sleep(Duration::from_millis(10)); + } +} + +/// Returns the objects of `forbidden` that process `pid` holds a handle of: none, since a process +/// that another thread creates while a VMM is created can inherit the VMM's handles on Windows, as +/// [`Inherited`] documents, so no object is forbidden there. +#[cfg(windows)] +fn held_objects(_pid: u32, forbidden: &[PathBuf]) -> Vec { + assert!(forbidden.is_empty()); + Vec::new() +} + +/// Returns whether the handle `value` of process `pid` refers to the object of this process's +/// handle `value`, as a handle that the process inherited from this one does. +#[cfg(windows)] +#[allow(unsafe_code)] +fn holds(pid: u32, value: usize) -> bool { + use std::os::windows::io::{AsRawHandle, FromRawHandle, OwnedHandle}; + use std::ptr; + + use windows_sys::Win32::Foundation::{ + CompareObjectHandles, DUPLICATE_SAME_ACCESS, DuplicateHandle, HANDLE, + }; + use windows_sys::Win32::System::Threading::{ + GetCurrentProcess, OpenProcess, PROCESS_DUP_HANDLE, + }; + + let ours = value as HANDLE; + // SAFETY: OpenProcess has no memory-safety preconditions. + let process = unsafe { OpenProcess(PROCESS_DUP_HANDLE, 0, pid) }; + assert!( + !process.is_null(), + "cannot open process {pid}: {}", + io::Error::last_os_error() + ); + // SAFETY: the handle is new, and nothing else owns it. + let process = unsafe { OwnedHandle::from_raw_handle(process) }; + let mut copy = ptr::null_mut(); + // SAFETY: the process handle may duplicate the process's handles, the value is looked up in + // that process only, and the output pointer is valid. + let duplicated = unsafe { + DuplicateHandle( + process.as_raw_handle(), + ours, + GetCurrentProcess(), + &mut copy, + 0, + 0, + DUPLICATE_SAME_ACCESS, + ) + } != 0; + // A value that names no handle of the process names none of ours. + if !duplicated { + return false; + } + // SAFETY: the copy is new, and nothing else owns it. + let copy = unsafe { OwnedHandle::from_raw_handle(copy) }; + // SAFETY: both handles are open. + unsafe { CompareObjectHandles(copy.as_raw_handle(), ours) != 0 } } -/// The one-shot runs of the CLI start OpenVMM in the caller's session, as `inherit_only` makes -/// a command start, with the umask that a detached VMM starts with. -#[cfg(feature = "cli")] +/// Makes `pipe` inheritable, or not, as code that spawns without an allowlist does around its +/// spawns. +#[cfg(target_os = "linux")] +fn set_inheritable(pipe: &PipeWriter, inheritable: bool) { + use nix::fcntl::{FcntlArg, FdFlag, fcntl}; + + let flags = if inheritable { + FdFlag::empty() + } else { + FdFlag::FD_CLOEXEC + }; + fcntl(pipe, FcntlArg::F_SETFD(flags)).unwrap(); +} + +/// Makes `pipe` inheritable, or not, as code that spawns without an allowlist does around its +/// spawns. +#[cfg(windows)] +#[allow(unsafe_code)] +fn set_inheritable(pipe: &PipeWriter, inheritable: bool) { + use std::os::windows::io::AsRawHandle; + + use windows_sys::Win32::Foundation::{HANDLE, HANDLE_FLAG_INHERIT, SetHandleInformation}; + + let flags = if inheritable { HANDLE_FLAG_INHERIT } else { 0 }; + // SAFETY: the handle is open. + let set = + unsafe { SetHandleInformation(pipe.as_raw_handle() as HANDLE, HANDLE_FLAG_INHERIT, flags) }; + assert_ne!(set, 0, "{}", io::Error::last_os_error()); +} + +/// Returns a command that runs for a few seconds, without standard streams. +fn sleeper() -> Command { + let mut command = if cfg!(windows) { + let mut command = Command::new("ping"); + command.args(["-n", "5", "127.0.0.1"]); + command + } else { + let mut command = Command::new("sleep"); + command.arg("5"); + command + }; + command + .stdin(Stdio::null()) + .stdout(Stdio::null()) + .stderr(Stdio::null()); + command +} + +#[cfg(windows)] #[test] -fn one_shot_vmms_start_with_the_callers_umask_without_the_owners_bits() { - assert_spawns_clear_the_owners_bits(|_| { - let mut command = sleeper(); - platform::inherit_only(&mut command, &[]).unwrap(); - command.spawn().unwrap() - }); +fn a_job_that_ends_its_vmm_wakes_the_waiters() { + let _spawns = serialized(); + let mut vmm = VmmDirectory::new(); + let job = job::Job::new(false); + let mut launcher = Launcher::start(vmm.path(), false); + job.assign(&launcher.child); + let reference = launcher.launch_reference(); + vmm.record(&reference); + // The VMM did not break away, so the job holds it too, and closing the job ends it. + let process = open_running(&reference); + assert!(process.wait(Instant::now() + SHORT_WAIT).unwrap().is_none()); + let started = Instant::now(); + drop(job); + assert!(process.wait(started + TIMEOUT).unwrap().is_some()); + assert_ended(&process); + launcher.wait(); +} + +#[test] +fn launched_waits_return_and_retain_the_actual_exit_status() { + let _spawns = serialized(); + for status in [0, 17] { + let mut vmm = VmmDirectory::new(); + fs::write(vmm.path().join(EXIT_STATUS), status.to_string()).unwrap(); + let log = File::create(vmm.path().join("vmm.log")).unwrap(); + let command = vmm_command(vmm.path(), false); + vmm.launched = Some(spawn_vm_with_mode(&command, &CAPABILITY, log, false).unwrap()); + assert_eq!( + vmm.launched + .as_mut() + .unwrap() + .wait_timeout(Duration::ZERO) + .unwrap(), + None + ); + File::create(vmm.path().join(GO_MARKER)).unwrap(); + let launched = vmm.launched.as_mut().unwrap(); + assert_eq!(launched.wait_timeout(TIMEOUT).unwrap(), Some(status)); + assert_eq!(launched.wait_timeout(Duration::ZERO).unwrap(), Some(status)); + vmm.launched + .take() + .unwrap() + .kill_and_confirm_exit() + .unwrap(); + } } #[test] -fn spawns_fail_with_the_error_of_a_refused_umask() { - // As under a seccomp policy that refuses `umask`: the spawn fails with the call's error, - // rather than return a process that ended before it could run its program. The filter - // refuses the call with `EDOM`, which no other step of the spawn returns, so a failure to - // install the filter cannot pass for the refusal. - let mut command = sleeper(); - sys::tests::refuse_umask_on_spawn(&mut command, nix::libc::EDOM); - platform::detach(&mut command, &[]).unwrap(); - let error = command - .spawn() - .map(Reaped) - .err() - .expect("the spawn succeeded"); - assert_eq!(error.raw_os_error(), Some(nix::libc::EDOM), "{error}"); +fn strict_cleanup_terminates_and_confirms_a_running_child() { + let _spawns = serialized(); + let mut vmm = VmmDirectory::new(); + let log = File::create(vmm.path().join("vmm.log")).unwrap(); + let command = vmm_command(vmm.path(), false); + vmm.launched = Some(spawn_vm_with_mode(&command, &CAPABILITY, log, false).unwrap()); + vmm.report(STARTED_REPORT); + let process = vmm.launched.as_ref().unwrap().process().unwrap(); + assert!(!process.has_exited().unwrap()); + vmm.launched + .take() + .unwrap() + .kill_and_confirm_exit() + .unwrap(); + assert!(process.has_exited().unwrap()); +} + +/// Job objects, which Windows orchestrators run in. +#[cfg(windows)] +#[allow(unsafe_code)] +mod job { + use std::os::windows::io::{AsRawHandle, FromRawHandle, OwnedHandle}; + use std::process::Child; + use std::{io, mem, ptr}; + + use windows_sys::Win32::System::JobObjects::{ + AssignProcessToJobObject, CreateJobObjectW, JOB_OBJECT_LIMIT_BREAKAWAY_OK, + JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE, JOBOBJECT_EXTENDED_LIMIT_INFORMATION, + JobObjectExtendedLimitInformation, SetInformationJobObject, + }; + + /// A job object that terminates its processes when the test closes it. + pub(super) struct Job(OwnedHandle); + + impl Job { + /// Creates a job that terminates its processes when it closes, and that lets them start + /// processes outside it if `breakaway`. + pub(super) fn new(breakaway: bool) -> Self { + // SAFETY: creates an unnamed job object with default security. + let handle = unsafe { CreateJobObjectW(ptr::null(), ptr::null()) }; + assert!( + !handle.is_null(), + "cannot create a job object: {}", + io::Error::last_os_error() + ); + // SAFETY: the handle is new, and nothing else owns it. + let job = Self(unsafe { OwnedHandle::from_raw_handle(handle) }); + // SAFETY: the structure is plain data for which all-zero bytes are valid. + let mut limits: JOBOBJECT_EXTENDED_LIMIT_INFORMATION = unsafe { mem::zeroed() }; + limits.BasicLimitInformation.LimitFlags = JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE + | if breakaway { + JOB_OBJECT_LIMIT_BREAKAWAY_OK + } else { + 0 + }; + // SAFETY: the handle is a job object, and the information has the size given. + let set = unsafe { + SetInformationJobObject( + handle, + JobObjectExtendedLimitInformation, + (&raw const limits).cast(), + mem::size_of_val(&limits) as u32, + ) + }; + assert_ne!( + set, + 0, + "cannot set the limits of a job object: {}", + io::Error::last_os_error() + ); + job + } + + /// Places `child` in the job, before it starts any process of its own. + pub(super) fn assign(&self, child: &Child) { + // SAFETY: both handles are open for the duration of the call. + let assigned = + unsafe { AssignProcessToJobObject(self.0.as_raw_handle(), child.as_raw_handle()) }; + assert_ne!( + assigned, + 0, + "cannot place the launcher in a job object: {}", + io::Error::last_os_error() + ); + } + } } diff --git a/nvx-rs/src/host/spawn/tests/linux.rs b/nvx-rs/src/host/spawn/tests/linux.rs new file mode 100644 index 000000000..25998282b --- /dev/null +++ b/nvx-rs/src/host/spawn/tests/linux.rs @@ -0,0 +1,231 @@ +//! The spawn's descriptors and umask on Linux. + +use std::fs; +use std::os::fd::{AsFd, AsRawFd, BorrowedFd, RawFd}; +use std::path::Path; + +use nix::fcntl::{FcntlArg, FdFlag, fcntl}; + +use super::*; +use crate::host::sys; + +/// Kills and reaps a child process when dropped, even when a test fails. +struct Reaped(Child); + +impl Drop for Reaped { + fn drop(&mut self) { + let _ = self.0.kill(); + let _ = self.0.wait(); + } +} + +/// Returns the object that the descriptor `descriptor` of process `pid` holds, as the target +/// of its link in `/proc`, or `None` if the process has no such descriptor. +fn object(pid: u32, descriptor: RawFd) -> Option { + fs::read_link(format!("/proc/{pid}/fd/{descriptor}")).ok() +} + +/// Returns whether process `pid` holds a descriptor of `object`, under any number. +fn holds(pid: u32, object: &Path) -> bool { + fs::read_dir(format!("/proc/{pid}/fd")) + .unwrap() + .any(|entry| fs::read_link(entry.unwrap().path()).is_ok_and(|target| target == object)) +} + +/// Returns whether the descriptor `descriptor` of this process is close-on-exec. +fn close_on_exec(descriptor: BorrowedFd<'_>) -> bool { + FdFlag::from_bits_retain(fcntl(descriptor, FcntlArg::F_GETFD).unwrap()) + .contains(FdFlag::FD_CLOEXEC) +} + +/// Spawns a long-running child with `spawn` while this process holds an inheritable pipe +/// writer, as a descriptor that the caller's parent or native code left without close-on-exec +/// would be. Returns whether the child holds a copy, as its own descriptor table shows. Unlike +/// the pipe reaching end-of-file, that cannot depend on the children of concurrent tests, +/// which may inherit the writer too. +fn child_inherits_writer(spawn: impl FnOnce() -> Child) -> bool { + let (pipe, child) = { + let (_, writer) = io::pipe().unwrap(); + sys::tests::clear_close_on_exec(&writer).unwrap(); + let pipe = object(std::process::id(), writer.as_raw_fd()).unwrap(); + (pipe, Reaped(spawn())) + }; + // `spawn` returns once the child has executed its program, which closed the child's + // close-on-exec copies, so a copy that remains was inherited. + holds(child.0.id(), &pipe) +} + +/// Returns a command that runs for a minute unless it is killed. +fn sleeper() -> Command { + let mut command = Command::new("sleep"); + command.arg("60").stdin(Stdio::null()); + command +} + +/// Starts `sleep 60` as a VMM in `directory`, which also inherits `inherit`. +fn spawn_sleeper(directory: &Path, inherit: Vec>) -> io::Result { + let log = File::create(directory.join("vmm.log"))?; + let command = VmmCommand { + program: PathBuf::from("sleep"), + arguments: vec![OsString::from("60")], + working_dir: directory.to_path_buf(), + breakaway_from_job: false, + inherit, + }; + spawn_vm_with_mode(&command, &[1; CAPABILITY_LEN], log, false) +} + +#[test] +fn vmms_inherit_no_descriptor_beyond_their_standard_streams() { + // A plain child keeps the writer, which shows that the check detects a leak. + let plain = || sleeper().spawn().unwrap(); + assert!(child_inherits_writer(plain)); + + let directory = tempfile::tempdir().unwrap(); + let vmm = || spawn_sleeper(directory.path(), Vec::new()).unwrap().child; + assert!(!child_inherits_writer(vmm)); + + // Kernels without CLOSE_RANGE_CLOEXEC take the /proc/self/fd fallback. + let fallback = || { + let mut command = sleeper(); + sys::tests::mark_listed_descriptors_on_spawn(&mut command); + command.spawn().unwrap() + }; + assert!(!child_inherits_writer(fallback)); +} + +#[test] +fn vmms_inherit_the_allowed_descriptors_under_their_numbers() { + let directory = tempfile::tempdir().unwrap(); + let (_reader, pipe) = io::pipe().unwrap(); + let file = File::create(directory.path().join("distro")).unwrap(); + // A descriptor that the list names twice is inherited once. + let inherit = vec![ + Inherited::new(&pipe), + Inherited::new(&file), + Inherited::new(&pipe), + ]; + let vmm = Reaped(spawn_sleeper(directory.path(), inherit).unwrap().child); + let (pid, this) = (vmm.0.id(), std::process::id()); + for descriptor in [pipe.as_raw_fd(), file.as_raw_fd()] { + let inherited = object(pid, descriptor); + assert!( + inherited.is_some(), + "descriptor {descriptor} was not inherited" + ); + assert_eq!(inherited, object(this, descriptor)); + } + // Only the VMM's process cleared their close-on-exec flags. + assert!(close_on_exec(pipe.as_fd())); + assert!(close_on_exec(file.as_fd())); +} + +#[test] +fn standard_streams_cannot_be_inherited() { + let directory = tempfile::tempdir().unwrap(); + for stream in [ + io::stdin().as_fd(), + io::stdout().as_fd(), + io::stderr().as_fd(), + ] { + let error = spawn_sleeper(directory.path(), vec![Inherited::from(stream)]).unwrap_err(); + assert_eq!(error.kind(), io::ErrorKind::InvalidInput); + } +} + +#[test] +fn descriptors_that_are_not_close_on_exec_are_refused() { + let directory = tempfile::tempdir().unwrap(); + // As a descriptor that the caller's parent or native code left without close-on-exec, + // which any process that the caller spawns would inherit, and not only the VMM. + let (_reader, writer) = io::pipe().unwrap(); + sys::tests::clear_close_on_exec(&writer).unwrap(); + let error = spawn_sleeper(directory.path(), vec![Inherited::new(&writer)]).unwrap_err(); + assert_eq!(error.kind(), io::ErrorKind::InvalidInput); + assert!(error.to_string().contains("close-on-exec"), "{error}"); + // The refusal, which comes before the spawn, leaves the descriptor as it was. + assert!(!close_on_exec(writer.as_fd())); +} + +/// Returns the umask that the status file of a process or a thread at `status`, in procfs, +/// tells. +fn umask_in(status: &str) -> u32 { + fs::read_to_string(status) + .unwrap() + .lines() + .find_map(|line| line.strip_prefix("Umask:")) + .and_then(|umask| u32::from_str_radix(umask.trim(), 8).ok()) + .unwrap() +} + +/// Asserts that the process that `spawn` starts in a directory of its own, from a thread whose +/// umask no other thread shares, has the thread's umask without the owner's bits, and that +/// the thread keeps its own, for the usual umasks, which keep the owner's bits, and for umasks +/// that clear some of them. +fn assert_spawns_clear_the_owners_bits(spawn: impl Fn(&Path) -> Child + Sync) { + use nix::sys::stat::{Mode, umask}; + + for mask in [ + 0o002, 0o022, 0o027, 0o077, 0o100, 0o200, 0o400, 0o477, 0o777, + ] { + let directory = tempfile::tempdir().unwrap(); + let (child, caller) = thread::scope(|scope| { + scope + .spawn(|| { + nix::sched::unshare(nix::sched::CloneFlags::CLONE_FS).unwrap(); + umask(Mode::from_bits_truncate(mask)); + let child = Reaped(spawn(directory.path())); + (child, umask_in("/proc/thread-self/status")) + }) + .join() + .unwrap_or_else(|panic| std::panic::resume_unwind(panic)) + }); + assert_eq!( + caller, mask, + "the caller's umask {mask:03o} became {caller:03o}" + ); + let pid = child.0.id(); + let inherited = umask_in(&format!("/proc/{pid}/status")); + assert_eq!( + inherited, + mask & 0o077, + "the caller's umask {mask:03o} gave the process {inherited:03o}" + ); + } +} + +#[test] +fn vmms_start_with_the_callers_umask_without_the_owners_bits() { + assert_spawns_clear_the_owners_bits(|directory| { + spawn_sleeper(directory, Vec::new()).unwrap().child + }); +} + +/// The one-shot runs of the CLI start OpenVMM in the caller's session, as `inherit_only` makes +/// a command start, with the umask that a detached VMM starts with. +#[cfg(feature = "cli")] +#[test] +fn one_shot_vmms_start_with_the_callers_umask_without_the_owners_bits() { + assert_spawns_clear_the_owners_bits(|_| { + let mut command = sleeper(); + platform::inherit_only(&mut command, &[]).unwrap(); + command.spawn().unwrap() + }); +} + +#[test] +fn spawns_fail_with_the_error_of_a_refused_umask() { + // As under a seccomp policy that refuses `umask`: the spawn fails with the call's error, + // rather than return a process that ended before it could run its program. The filter + // refuses the call with `EDOM`, which no other step of the spawn returns, so a failure to + // install the filter cannot pass for the refusal. + let mut command = sleeper(); + sys::tests::refuse_umask_on_spawn(&mut command, nix::libc::EDOM); + platform::detach(&mut command, &[]).unwrap(); + let error = command + .spawn() + .map(Reaped) + .err() + .expect("the spawn succeeded"); + assert_eq!(error.raw_os_error(), Some(nix::libc::EDOM), "{error}"); +} diff --git a/nvx-rs/src/host/supervisor/mod.rs b/nvx-rs/src/host/supervisor/mod.rs deleted file mode 100644 index 0ee278b5c..000000000 --- a/nvx-rs/src/host/supervisor/mod.rs +++ /dev/null @@ -1,34 +0,0 @@ -//! The transitional public path of the low-level spawn of a VMM, which code that stays still uses -//! until SL6.3 (#1122): [`spawn`] and [`spawn_vm`], which start a VMM detached, the [`VmmCommand`] -//! that they start and the [`Launched`] VMM that they return, [`Inherited`], which allows the VMM -//! a descriptor (Linux) or handle (Windows) of the caller's, and [`spawn_command`], with which a -//! caller that spawns VMMs spawns its other processes. -//! -//! The module has no code of its own, and supervises nothing: it re-exports these items from the -//! crate's private spawn and platform modules. [`host::vm`](crate::host::vm) creates, spawns, -//! reattaches to, and stops a VM in a private directory that its caller owns. SL6.3 deletes -//! `spawn_command` and the checked-file handoff of -//! [`PreparedLaunch`](crate::host::sandbox::PreparedLaunch), and settles the launcher of the -//! `external-launch` example: the last users of this path, besides the tests of the spawn in -//! `tests/supervisor.rs`. -//! -//! # Inheritance -//! -//! The VMM receives its capability on its standard input, its log as its standard output and -//! error, and the descriptors (Linux) or handles (Windows) that the caller names in -//! [`VmmCommand::inherit`], under the numbers that they have in the caller, and nothing else of -//! the caller's. On Linux, every other descriptor stays close-on-exec, and only the VMM's own -//! process clears the flag of the allowed ones, so a process that another thread spawns meanwhile -//! inherits none of them. On Windows, the allowed handles join the explicit list of handles that -//! the VMM inherits, and stay inheritable only while it is created; see [`Inherited`]. -//! -//! On Windows, the VMM's standard handles are inheritable while it is created too, and a process -//! that another thread of the caller creates in that window without a list of handles, as -//! `std::process::Command` creates one, inherits them all, and holds them until it exits: it could -//! read the VMM's capability before the VMM does, and it holds the VMM's log, and the handles that -//! the VMM is allowed. Each spawn of a VMM holds the crate's spawn lock throughout that window, so -//! a caller that launches VMMs spawns its other processes with [`spawn_command`], which takes the -//! same lock, and keeps them out of it. On Linux, the lock holds nothing. - -pub use super::platform::inherit::{Inherited, spawn_command}; -pub use super::spawn::{Launched, VmmCommand, spawn, spawn_vm}; diff --git a/nvx-rs/src/host/sys/mod.rs b/nvx-rs/src/host/sys/mod.rs index e9e87bba9..dd3741aa4 100644 --- a/nvx-rs/src/host/sys/mod.rs +++ b/nvx-rs/src/host/sys/mod.rs @@ -21,7 +21,6 @@ mod socket; #[cfg(target_os = "linux")] pub(crate) use nix::errno::Errno; -use nix::unistd::{getegid, geteuid}; #[cfg(all(feature = "cli", target_os = "linux"))] pub(crate) use fs::rename_no_replace; @@ -30,18 +29,11 @@ pub(crate) use interrupt::{interrupted, watch_interrupts, write_interruptibly}; #[cfg(all(feature = "cli", target_os = "linux"))] pub(crate) use process::inherit_only; #[cfg(target_os = "linux")] -pub(crate) use process::{ - LAST_STANDARD_STREAM, PidFd, detach, move_above_standard_streams, process_exists, -}; +pub(crate) use process::{LAST_STANDARD_STREAM, PidFd, detach, process_exists}; #[cfg(target_os = "linux")] pub(crate) use socket::{ connect_nonblocking, peer_pid, send_nonblocking, unread_by_peer, wait_readable, wait_writable, }; -/// Returns the effective user and group IDs of this process. -pub(crate) fn effective_ids() -> (u32, u32) { - (geteuid().as_raw(), getegid().as_raw()) -} - #[cfg(test)] pub(crate) mod tests; diff --git a/nvx-rs/src/host/sys/process.rs b/nvx-rs/src/host/sys/process.rs index 05cf15e2f..a2782e4c9 100644 --- a/nvx-rs/src/host/sys/process.rs +++ b/nvx-rs/src/host/sys/process.rs @@ -1,6 +1,5 @@ //! Process descriptors, and the hook that detaches a spawned process from its caller. -use std::fs::File; use std::io; use std::os::fd::{AsFd, AsRawFd, BorrowedFd, FromRawFd, OwnedFd, RawFd}; use std::os::unix::process::CommandExt; @@ -25,23 +24,6 @@ use super::Errno; /// the descriptors that a detached spawn marks close-on-exec are the ones above it. pub(crate) const LAST_STANDARD_STREAM: RawFd = libc::STDERR_FILENO; -/// Moves `file` to a descriptor above [`LAST_STANDARD_STREAM`], under which a spawn can pass it to -/// its process, if it is not there already: as when the caller had closed a standard stream, whose -/// descriptor an open then took. -/// -/// The new descriptor is a close-on-exec copy of the old one, which is then closed. It shares the -/// file's open file description, and with it the file's offset and `flock` locks. -pub(crate) fn move_above_standard_streams(file: &mut File) -> io::Result<()> { - if file.as_raw_fd() > LAST_STANDARD_STREAM { - return Ok(()); - } - let copy = fcntl(&*file, FcntlArg::F_DUPFD_CLOEXEC(LAST_STANDARD_STREAM + 1))?; - // SAFETY: fcntl returned a fresh descriptor, which nothing else owns. Replacing the file - // closes the old descriptor. - *file = File::from(unsafe { OwnedFd::from_raw_fd(copy) }); - Ok(()) -} - /// A pidfd: an owned descriptor that refers to one process. Signals sent through it reach that /// process or fail, even once another process reuses its ID. pub(crate) struct PidFd(OwnedFd); diff --git a/nvx-rs/src/protocol/app.rs b/nvx-rs/src/protocol/app.rs index da56b199e..66b1a0d29 100644 --- a/nvx-rs/src/protocol/app.rs +++ b/nvx-rs/src/protocol/app.rs @@ -23,26 +23,22 @@ use crate::activation::ActivationRecord; ))] use self::workload::WorkloadIdentity; -/// Shared identity data; host-side selection remains in the host module. +/// Shared identity data. #[cfg(any( - feature = "host", + feature = "cli", all(feature = "guest", target_os = "linux", target_arch = "x86_64") ))] pub(crate) mod workload { /// The identity that workloads run as: a non-root user and group ID. #[derive(Debug, Clone, Copy, PartialEq, Eq)] - #[cfg_attr( - not(feature = "host"), - expect(unreachable_pub, reason = "public host re-export when host is enabled") - )] - pub struct WorkloadIdentity { + pub(crate) struct WorkloadIdentity { /// User ID. - pub uid: u32, + pub(crate) uid: u32, /// Primary group ID. - pub gid: u32, + pub(crate) gid: u32, /// Whether the guest creates an account for the IDs if its image lacks a usable one, /// replacing an image account for the UID whose primary group or home does not fit. - pub create_account: bool, + pub(crate) create_account: bool, } } diff --git a/nvx-rs/src/protocol/boot.rs b/nvx-rs/src/protocol/boot.rs index eaeda6ac9..dc2798386 100644 --- a/nvx-rs/src/protocol/boot.rs +++ b/nvx-rs/src/protocol/boot.rs @@ -11,10 +11,6 @@ pub const MAPS_TOKEN: &str = "nvx_maps="; /// The name of the [`MAPS_TOKEN`] token: its text up to the `=`. const MAPS_NAME: &[u8] = MAPS_TOKEN.as_bytes().split_at(MAPS_TOKEN.len() - 1).0; -/// Token with which the guest creates an account for the workload identity when its image lacks -/// a usable one. -pub const WORKLOAD_ACCOUNT_TOKEN: &str = "nvx_workload_account=create"; - /// Parses the `nvx_maps=COUNT` token from a Linux kernel command line, whose tokens it reads as /// the guest's init splits them, with [`cmdline::init_tokens`]. /// diff --git a/nvx-rs/src/protocol/mod.rs b/nvx-rs/src/protocol/mod.rs index a89f0ffe2..de401d1f1 100644 --- a/nvx-rs/src/protocol/mod.rs +++ b/nvx-rs/src/protocol/mod.rs @@ -86,7 +86,7 @@ pub(crate) use app::encode_activate_payload; #[cfg(feature = "cli")] pub(crate) use app::encode_capture_payload; #[cfg(any( - feature = "host", + feature = "cli", all(feature = "guest", target_os = "linux", target_arch = "x86_64") ))] pub(crate) use app::workload; @@ -107,7 +107,7 @@ pub(crate) use app::{ pub(crate) use app::{BUSY, CWD_FAILED, INVALID_REQUEST, LAUNCH_FAILED, MAPPINGS_INCOMPLETE}; #[cfg(all(feature = "guest", target_os = "linux", target_arch = "x86_64"))] pub(crate) use app::{decode_activate_payload, decode_capture_payload}; -pub use boot::{MAPS_TOKEN, WORKLOAD_ACCOUNT_TOKEN, parse_host_mapping_count}; +pub use boot::{MAPS_TOKEN, parse_host_mapping_count}; #[cfg(any( feature = "cli", feature = "testing", diff --git a/nvx-rs/tests/checked_handoff.rs b/nvx-rs/tests/checked_handoff.rs deleted file mode 100644 index 4f7f7f03c..000000000 --- a/nvx-rs/tests/checked_handoff.rs +++ /dev/null @@ -1,1085 +0,0 @@ -//! The checked handoff of a prepared launch, through the public API of `nvx::host::sandbox` and -//! `nvx::host::supervisor`, as an orchestrator uses them: it stages and binds a distro layer and a -//! scratch image, checks them, prepares the launch, and spawns the VMM with the prepared launch's -//! arguments and allowlist. -//! -//! The VMM is this test executable, run again as a fixture. It reads the capability that -//! `nvx::host::supervisor::spawn` passes on its standard input, as OpenVMM does, reports its -//! descriptors on Linux and that it started, and, once a file named `go` appears in its working -//! directory, opens each block device that an argument names as OpenVMM opens it, a layer -//! read-only and the scratch image for reading and writing, reads each whole, writes the scratch -//! image, and reports what each held. It holds them until a file named `exit` appears in its -//! working directory, which ends it before `go` too. On Linux, the executable also runs as an -//! orchestrator fixture that closed its standard input before it checked the images. -#![cfg(all(feature = "host", any(target_os = "linux", windows)))] - -use std::env; -use std::ffi::OsString; -use std::fs::{self, File, OpenOptions}; -use std::io::{self, Read, Write}; -use std::path::{Path, PathBuf}; -use std::process::{Command, Stdio}; -use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering}; -use std::thread; -use std::time::{Duration, Instant}; - -use nvx::host::sandbox::distro::{DISTRO_POLICY_REVISION, DistroBinding}; -use nvx::host::sandbox::scratch::{CLAIM_RELEASE_WAIT, ScratchBinding}; -use nvx::host::sandbox::{ - LayerRole, PreparedLaunch, SandboxLaunch, SandboxLayer, SandboxSettings, SandboxUpper, -}; -use nvx::host::supervisor::{self, Launched, VmmCommand}; -use nvx::protocol::CAPABILITY_LEN; - -/// File whose presence in its working directory marks a VMM fixture. Without it, the fixture -/// returns at once, as it does when `cargo test -- --ignored` runs it. -const VMM_MARKER: &str = "vmm-fixture"; -/// File whose appearance in its working directory lets a VMM fixture open its block devices. -const GO_MARKER: &str = "go"; -/// File that a VMM fixture writes once it has started, after its report of its descriptors. -const STARTED_REPORT: &str = "started"; -/// File whose appearance in its working directory ends a VMM fixture. -const EXIT_MARKER: &str = "exit"; -/// File in which a VMM fixture reports what its block devices held, a line for each: its role -/// and its content, escaped. -const BLOCKS_REPORT: &str = "blocks"; -/// File in which a VMM fixture reports its descriptors on Linux, a line for each: its number and -/// the object that it holds. -#[cfg(target_os = "linux")] -const DESCRIPTORS_REPORT: &str = "descriptors"; -/// Prefix of an argument of a VMM fixture that holds the value of a `--microvm-sandbox-block` -/// option of the prepared launch. Libtest takes the argument for a test name that matches no -/// test. -const BLOCK_PREFIX: &str = "block="; -/// File whose presence in its working directory marks a launcher fixture. Without it, the fixture -/// returns at once. -#[cfg(target_os = "linux")] -const LAUNCHER_MARKER: &str = "launcher-fixture"; -/// File that a launcher fixture writes once the VMM that it launched found the checked images. -#[cfg(target_os = "linux")] -const LAUNCHED_REPORT: &str = "launched"; -/// The capability of the VMMs of these tests. -const CAPABILITY: [u8; CAPABILITY_LEN] = [0x5a; CAPABILITY_LEN]; -/// Longest wait for a fixture to answer. -const TIMEOUT: Duration = Duration::from_secs(60); -/// Longest life of a VMM fixture that nothing ends, so that a failed test leaves no process. -const VMM_LIFETIME: Duration = Duration::from_secs(180); -/// The UUID of the staged layer. -const UUID: &str = "11111111-1111-1111-1111-111111111111"; -/// The content of the staged distro layer. -const DISTRO: &str = "distro layer\n"; -/// The content of the staged scratch image. -const SCRATCH: &str = "scratch image\n"; - -/// The VMM fixture: reads its capability as OpenVMM does, reports its descriptors on Linux, then, -/// once [`GO_MARKER`] appears, opens and reads each block device that an argument names, writes -/// the scratch image, reports what each held, and runs until [`EXIT_MARKER`] appears. -#[test] -#[ignore = "fixture that the handoff tests run as a VMM"] -fn vmm_fixture() { - let directory = env::current_dir().unwrap(); - if !directory.join(VMM_MARKER).exists() { - return; - } - let mut capability = [0; CAPABILITY_LEN]; - io::stdin().read_exact(&mut capability).unwrap(); - assert_eq!(capability, CAPABILITY); - #[cfg(target_os = "linux")] - report_descriptors(&directory); - write_whole(&directory.join(STARTED_REPORT), ""); - // A test that failed before it let the fixture go ends it. - let (go, exit) = (directory.join(GO_MARKER), directory.join(EXIT_MARKER)); - let deadline = Instant::now() + VMM_LIFETIME; - while !go.exists() { - if exit.exists() || Instant::now() >= deadline { - return; - } - thread::sleep(Duration::from_millis(10)); - } - let mut held = Vec::new(); - let mut report = String::new(); - for argument in env::args() { - let Some(block) = argument.strip_prefix(BLOCK_PREFIX) else { - continue; - }; - let (role, rest) = block.split_once(":file:").unwrap(); - let (path, read_only) = match rest.strip_suffix(",ro") { - Some(path) => (path, true), - None => (rest, false), - }; - let mut file = OpenOptions::new() - .read(true) - .write(!read_only) - .open(path) - .unwrap(); - let mut content = String::new(); - file.read_to_string(&mut content).unwrap(); - if !read_only { - file.write_all(b"written by the VMM\n").unwrap(); - } - report.push_str(&format!("{role} {}\n", content.escape_default())); - held.push(file); - } - write_whole(&directory.join(BLOCKS_REPORT), &report); - wait_for(&directory.join(EXIT_MARKER), VMM_LIFETIME); -} - -/// Writes `text` to the file at `path`, which appears whole. -fn write_whole(path: &Path, text: &str) { - let temporary = path.with_extension("tmp"); - fs::write(&temporary, text).unwrap(); - fs::rename(&temporary, path).unwrap(); -} - -/// Waits until the file at `path` exists, for at most `timeout`, and returns whether it does. -fn wait_for(path: &Path, timeout: Duration) -> bool { - let deadline = Instant::now() + timeout; - while !path.exists() { - if Instant::now() >= deadline { - return false; - } - thread::sleep(Duration::from_millis(10)); - } - true -} - -/// Writes the descriptors of this VMM fixture, but for the one that lists them, to the file -/// [`DESCRIPTORS_REPORT`] in `directory`. -#[cfg(target_os = "linux")] -fn report_descriptors(directory: &Path) { - // The descriptor that lists the others holds this directory. - let listing = PathBuf::from(format!("/proc/{}/fd", std::process::id())); - let mut descriptors: Vec<(usize, PathBuf)> = fs::read_dir("/proc/self/fd") - .unwrap() - .filter_map(|entry| { - let entry = entry.unwrap(); - let number = entry.file_name().to_str().unwrap().parse().unwrap(); - let object = fs::read_link(entry.path()).ok()?; - (object != listing).then_some((number, object)) - }) - .collect(); - descriptors.sort(); - let report: String = descriptors - .iter() - .map(|(number, object)| format!("{number} {}\n", object.display())) - .collect(); - write_whole(&directory.join(DESCRIPTORS_REPORT), &report); -} - -/// Returns the arguments that run only the fixture `name` of this test executable. -fn fixture_arguments(name: &str) -> Vec { - [ - "--exact", - name, - "--ignored", - "--nocapture", - "--test-threads=1", - ] - .map(OsString::from) - .to_vec() -} - -/// A VMM fixture that exits with the requested status once the caller lets it go. -#[test] -#[ignore = "fixture that the exit-status test runs as a VMM"] -fn exit_status_fixture() { - let directory = env::current_dir().unwrap(); - if !directory.join(VMM_MARKER).exists() { - return; - } - let mut capability = [0; CAPABILITY_LEN]; - io::stdin().read_exact(&mut capability).unwrap(); - assert_eq!(capability, CAPABILITY); - let deadline = Instant::now() + VMM_LIFETIME; - while !directory.join(GO_MARKER).exists() { - if directory.join(EXIT_MARKER).exists() || Instant::now() >= deadline { - return; - } - thread::sleep(Duration::from_millis(5)); - } - let status = fs::read_to_string(directory.join("exit-status")).unwrap(); - std::process::exit(status.parse().unwrap()); -} - -#[test] -fn launched_waits_return_and_retain_the_actual_exit_status() { - for status in [0, 17] { - let mut vmm = VmmDirectory::new(); - fs::write(vmm.path().join("exit-status"), status.to_string()).unwrap(); - let command = VmmCommand::new( - env::current_exe().unwrap(), - fixture_arguments("exit_status_fixture"), - vmm.path(), - ); - let log = File::create(vmm.path().join("vmm.log")).unwrap(); - vmm.launched = Some(supervisor::spawn(&command, &CAPABILITY, log).unwrap()); - assert_eq!( - vmm.launched - .as_mut() - .unwrap() - .wait_timeout(Duration::ZERO) - .unwrap(), - None - ); - File::create(vmm.path().join(GO_MARKER)).unwrap(); - let launched = vmm.launched.as_mut().unwrap(); - assert_eq!(launched.wait_timeout(TIMEOUT).unwrap(), Some(status)); - assert_eq!(launched.wait_timeout(Duration::ZERO).unwrap(), Some(status)); - vmm.launched - .take() - .unwrap() - .kill_and_confirm_exit() - .unwrap(); - } -} - -#[test] -fn strict_cleanup_terminates_and_confirms_a_running_child() { - let mut vmm = VmmDirectory::new(); - let command = VmmCommand::new( - env::current_exe().unwrap(), - fixture_arguments("vmm_fixture"), - vmm.path(), - ); - let log = File::create(vmm.path().join("vmm.log")).unwrap(); - vmm.launched = Some(supervisor::spawn_vm(&command, &CAPABILITY, log).unwrap()); - vmm.report(STARTED_REPORT); - let process = vmm.launched.as_ref().unwrap().process().unwrap(); - assert!(!process.has_exited().unwrap()); - vmm.launched - .take() - .unwrap() - .kill_and_confirm_exit() - .unwrap(); - assert!(process.has_exited().unwrap()); -} - -/// Returns the arguments that pass a VMM fixture the `--microvm-sandbox-block` values of -/// `prepared`. -fn block_arguments(prepared: &PreparedLaunch) -> Vec { - prepared - .openvmm_arguments() - .windows(2) - .filter(|pair| pair[0] == "--microvm-sandbox-block") - .map(|pair| OsString::from(format!("{BLOCK_PREFIX}{}", pair[1]))) - .collect() -} - -/// Starts a VMM fixture in `vmm`'s directory with the block devices and the allowlist of -/// `prepared`, and waits until it has started. -fn start_vmm(vmm: &mut VmmDirectory, prepared: &PreparedLaunch) { - let mut arguments = fixture_arguments("vmm_fixture"); - arguments.extend(block_arguments(prepared)); - let mut command = VmmCommand::new(env::current_exe().unwrap(), arguments, vmm.path()); - command.inherit = prepared.inherited(); - let log = File::create(vmm.path().join("vmm.log")).unwrap(); - vmm.launched = Some(supervisor::spawn(&command, &CAPABILITY, log).unwrap()); - vmm.report(STARTED_REPORT); -} - -/// Returns the report of a VMM fixture whose block devices are the staged images. -fn staged_blocks() -> String { - format!( - "distro {}\nscratch {}\n", - DISTRO.escape_default(), - SCRATCH.escape_default() - ) -} - -/// Tries to replace both staged images at their paths, as something that takes a path after the -/// checks would, and fails the test unless it can on Linux, where the VMM opens the checked files -/// whatever has their paths, and cannot on Windows, where the checked files hold their paths. -fn replace_images(staged: &Staged) { - for (path, replacement) in [ - (staged.distro(), "replaced distro\n"), - (staged.scratch(), "replaced scratch\n"), - ] { - let new = path.with_extension("new"); - fs::write(&new, replacement).unwrap(); - let replaced = fs::rename(&new, &path); - assert_eq!( - replaced.is_ok(), - cfg!(target_os = "linux"), - "{}: {replaced:?}", - path.display() - ); - } -} - -/// The working directory of a VMM fixture. Dropping it ends the fixture, and the launched VMM -/// that it holds, so that a failed test leaves no process behind. -struct VmmDirectory { - /// The launched VMM, once the test spawned it. - launched: Option, - /// The directory, which is removed after the VMM exits. - directory: tempfile::TempDir, -} - -impl VmmDirectory { - /// Creates a directory that marks a VMM fixture. - fn new() -> Self { - let directory = tempfile::tempdir().unwrap(); - File::create(directory.path().join(VMM_MARKER)).unwrap(); - Self { - launched: None, - directory, - } - } - - /// Returns the path of the directory. - fn path(&self) -> &Path { - self.directory.path() - } - - /// Waits for the VMM's report `name`, and returns it. - fn report(&self, name: &str) -> String { - let path = self.path().join(name); - assert!(wait_for(&path, TIMEOUT), "the VMM did not report {name}"); - fs::read_to_string(path).unwrap() - } -} - -impl Drop for VmmDirectory { - fn drop(&mut self) { - let _ = File::create(self.path().join(EXIT_MARKER)); - if let Some(launched) = self.launched.take() { - if let Ok(process) = launched.process() { - let _ = process.wait(Instant::now() + TIMEOUT); - } - launched.kill_and_wait(); - } - } -} - -/// A distro layer and a scratch image, staged in a temporary directory. -struct Staged { - /// The temporary directory. - directory: tempfile::TempDir, -} - -impl Staged { - /// Stages a distro layer and a scratch image. - fn new() -> Self { - let staged = Self { - directory: tempfile::tempdir().unwrap(), - }; - fs::write(staged.distro(), DISTRO).unwrap(); - fs::write(staged.scratch(), SCRATCH).unwrap(); - staged - } - - /// Returns the path of the distro layer. - fn distro(&self) -> PathBuf { - self.directory.path().join("distro.erofs") - } - - /// Returns the path of the scratch image. - fn scratch(&self) -> PathBuf { - self.directory.path().join("scratch.ext4") - } - - /// Returns the launch of the distro layer over the scratch image. - fn launch(&self) -> SandboxLaunch { - launch_of(&self.distro(), &self.scratch()) - } -} - -/// Returns the launch of the distro layer at `distro` over the scratch image at `scratch`. -fn launch_of(distro: &Path, scratch: &Path) -> SandboxLaunch { - SandboxLaunch::new( - vec![SandboxLayer::new( - LayerRole::Distro, - distro.to_str().unwrap(), - UUID, - )], - SandboxUpper::scratch(scratch.to_str().unwrap()), - SandboxSettings::default(), - ) - .unwrap() -} - -/// Sets its flag when it is dropped, also while a failure unwinds, so that a thread that runs -/// until the flag is set stops. -struct SetOnDrop<'a>(&'a AtomicBool); - -impl Drop for SetOnDrop<'_> { - fn drop(&mut self) { - self.0.store(true, Ordering::SeqCst); - } -} - -/// What [`spawn_others`] checks that no other process holds: on Linux, the objects of the checked -/// descriptors. -#[cfg(target_os = "linux")] -type Checked = PathBuf; -/// What [`spawn_others`] checks that no other process holds: on Windows, the values of the checked -/// handles, under which a process that inherited one would hold it. -#[cfg(windows)] -type Checked = usize; - -/// Spawns processes until `done`, as other threads of an orchestrator do, through `std::process`, -/// and on Windows under a `SpawnLock`, as `supervisor::spawn_command` spawns them, and counts them -/// in `spawned`. Returns those that held what `checked` names, which no process but the VMM may -/// inherit. -/// -/// On Linux, each process lists its own descriptors once its program runs. The spawn returns -/// before the exec has closed the descriptors that are close-on-exec, so a listing that this -/// process took then could show descriptors that never reach the program. -fn spawn_others(done: &AtomicBool, spawned: &AtomicUsize, checked: &[Checked]) -> Vec { - let mut leaks = Vec::new(); - while !done.load(Ordering::SeqCst) { - #[cfg(target_os = "linux")] - { - let output = Command::new("ls") - .args(["-l", "/proc/self/fd/"]) - .stdin(Stdio::null()) - .stderr(Stdio::null()) - .output() - .unwrap(); - assert!(output.status.success()); - for line in String::from_utf8_lossy(&output.stdout).lines() { - if let Some((_, object)) = line.split_once(" -> ") - && checked.iter().any(|checked| checked.as_os_str() == object) - { - leaks.push(format!("a process held {object}")); - } - } - } - #[cfg(windows)] - { - let mut child = supervisor::spawn_command(&mut sleeper()).unwrap(); - let pid = child.id(); - for &value in checked { - if holds(pid, value, value) { - leaks.push(format!("process {pid} held handle {value:#x}")); - } - } - let _ = child.kill(); - child.wait().unwrap(); - } - spawned.fetch_add(1, Ordering::SeqCst); - } - leaks -} - -/// Returns a command that runs for a few seconds, without standard streams. -#[cfg(windows)] -fn sleeper() -> Command { - let mut command = Command::new("ping"); - command - .args(["-n", "5", "127.0.0.1"]) - .stdin(Stdio::null()) - .stdout(Stdio::null()) - .stderr(Stdio::null()); - command -} - -/// Returns the object that the descriptor `number` of this process holds, as its link in `/proc` -/// names it. -#[cfg(target_os = "linux")] -fn this_object(number: i32) -> PathBuf { - fs::read_link(format!("/proc/self/fd/{number}")).unwrap() -} - -/// Makes `pipe` inheritable, or not, as code that spawns without an allowlist does around its -/// spawns. -#[cfg(target_os = "linux")] -fn set_inheritable(pipe: &io::PipeWriter, inheritable: bool) { - use nix::fcntl::{FcntlArg, FdFlag, fcntl}; - - let flags = if inheritable { - FdFlag::empty() - } else { - FdFlag::FD_CLOEXEC - }; - fcntl(pipe, FcntlArg::F_SETFD(flags)).unwrap(); -} - -/// Makes `pipe` inheritable, or not, as code that spawns without an allowlist does around its -/// spawns. -#[cfg(windows)] -fn set_inheritable(pipe: &io::PipeWriter, inheritable: bool) { - use std::os::windows::io::AsRawHandle; - - use windows_sys::Win32::Foundation::{HANDLE, HANDLE_FLAG_INHERIT, SetHandleInformation}; - - let flags = if inheritable { HANDLE_FLAG_INHERIT } else { 0 }; - // SAFETY: the handle is open. - let set = - unsafe { SetHandleInformation(pipe.as_raw_handle() as HANDLE, HANDLE_FLAG_INHERIT, flags) }; - assert_ne!(set, 0, "{}", io::Error::last_os_error()); -} - -/// Returns whether the handle `value` of process `pid` refers to the object of this process's -/// handle `ours`, as a handle that the process inherited from this one, under the value that it -/// has here, does. -#[cfg(windows)] -fn holds(pid: u32, value: usize, ours: usize) -> bool { - use std::os::windows::io::{AsRawHandle, FromRawHandle, OwnedHandle}; - use std::ptr; - - use windows_sys::Win32::Foundation::{ - CompareObjectHandles, DUPLICATE_SAME_ACCESS, DuplicateHandle, HANDLE, - }; - use windows_sys::Win32::System::Threading::{ - GetCurrentProcess, OpenProcess, PROCESS_DUP_HANDLE, - }; - - // SAFETY: OpenProcess has no memory-safety preconditions. - let process = unsafe { OpenProcess(PROCESS_DUP_HANDLE, 0, pid) }; - assert!( - !process.is_null(), - "cannot open process {pid}: {}", - io::Error::last_os_error() - ); - // SAFETY: the handle is new, and nothing else owns it. - let process = unsafe { OwnedHandle::from_raw_handle(process) }; - let mut copy = ptr::null_mut(); - // SAFETY: the process handle may duplicate the process's handles, the value is looked up in - // that process only, and the output pointer is valid. - let duplicated = unsafe { - DuplicateHandle( - process.as_raw_handle(), - value as HANDLE, - GetCurrentProcess(), - &mut copy, - 0, - 0, - DUPLICATE_SAME_ACCESS, - ) - } != 0; - // A value that names no handle of the process names none of ours. - if !duplicated { - return false; - } - // SAFETY: the copy is new, and nothing else owns it. - let copy = unsafe { OwnedHandle::from_raw_handle(copy) }; - // SAFETY: both handles are open. - unsafe { CompareObjectHandles(copy.as_raw_handle(), ours as HANDLE) != 0 } -} - -#[test] -fn a_prepared_launch_hands_the_vmm_its_checked_images_alone() { - let staged = Staged::new(); - let launch = staged.launch(); - // Staging binds the images, where hashing belongs; the launch checks them from their metadata - // alone, and claims the scratch image. - let binding = DistroBinding::verify(launch.distro()).unwrap(); - let scratch_binding = ScratchBinding::bind(launch.scratch().unwrap()).unwrap(); - let distro = binding - .check(launch.distro(), DISTRO_POLICY_REVISION) - .unwrap(); - let scratch = scratch_binding - .check( - launch.scratch().unwrap(), - Instant::now() + CLAIM_RELEASE_WAIT, - ) - .unwrap(); - let prepared = PreparedLaunch::new( - launch.bound_plan(&binding).unwrap(), - distro, - Some(scratch), - "", - ) - .unwrap(); - - // Something replaces both images between the checks and the VMM's opens: on Linux it can, and - // the VMM opens the checked files anyway; on Windows the checked files keep it from doing so. - replace_images(&staged); - - #[cfg(target_os = "linux")] - let checked: Vec = { - use std::os::fd::{AsFd, AsRawFd}; - - prepared - .inherited() - .iter() - .map(|inherited| this_object(inherited.as_fd().as_raw_fd())) - .collect() - }; - #[cfg(windows)] - let checked: Vec = { - use std::os::windows::io::{AsHandle, AsRawHandle}; - - prepared - .inherited() - .iter() - .map(|inherited| inherited.as_handle().as_raw_handle() as usize) - .collect() - }; - let mut vmm = VmmDirectory::new(); - let mut arguments = fixture_arguments("vmm_fixture"); - let blocks = block_arguments(&prepared); - assert_eq!(blocks.len(), 2); - arguments.extend(blocks); - let mut command = VmmCommand::new(env::current_exe().unwrap(), arguments, vmm.path()); - command.inherit = prepared.inherited(); - // A descriptor or handle that this process leaves inheritable, as code that spawns without an - // allowlist does, while another thread spawns processes. - let (_unrelated_reader, unrelated) = io::pipe().unwrap(); - let done = AtomicBool::new(false); - let spawned = AtomicUsize::new(0); - let leaks = thread::scope(|scope| { - let spawner = scope.spawn(|| spawn_others(&done, &spawned, &checked)); - // Stops the other thread when the VMM has started, or when a failure unwinds the scope, - // which then waits for the thread. - let stop = SetOnDrop(&done); - let deadline = Instant::now() + TIMEOUT; - while spawned.load(Ordering::SeqCst) == 0 { - assert!( - Instant::now() < deadline && !spawner.is_finished(), - "the other thread spawned nothing" - ); - thread::sleep(Duration::from_millis(10)); - } - set_inheritable(&unrelated, true); - let log = File::create(vmm.path().join("vmm.log")).unwrap(); - let launched = supervisor::spawn(&command, &CAPABILITY, log); - set_inheritable(&unrelated, false); - vmm.launched = Some(launched.unwrap()); - // The other thread spawns throughout the VMM's start. - vmm.report(STARTED_REPORT); - drop(stop); - spawner.join().unwrap() - }); - assert!(spawned.load(Ordering::SeqCst) > 0); - assert!(leaks.is_empty(), "{leaks:?}"); - let pid = vmm.launched.as_ref().unwrap().id(); - - // The VMM holds its standard streams and the checked files under their numbers, and nothing - // else of this process's: on Linux, the checked descriptors, which the arguments name; on - // Windows, the handles of the checked files, which hold the images' final paths for it, and - // the claim's handle of the scratch image, though no argument names a handle. - #[cfg(target_os = "linux")] - { - use std::os::fd::{AsFd, AsRawFd}; - - let _ = pid; - let reported = vmm.report(DESCRIPTORS_REPORT); - let mut expected: Vec = prepared - .inherited() - .iter() - .map(|inherited| { - let number = inherited.as_fd().as_raw_fd(); - format!("{number} {}", this_object(number).display()) - }) - .collect(); - let log = fs::canonicalize(vmm.path().join("vmm.log")).unwrap(); - let lines: Vec<&str> = reported.lines().collect(); - assert_eq!(lines.len(), 5, "{reported}"); - assert!(lines[0].starts_with("0 pipe:"), "{reported}"); - assert_eq!(lines[1], format!("1 {}", log.display())); - assert_eq!(lines[2], format!("2 {}", log.display())); - expected.sort(); - let mut inherited: Vec = lines[3..].iter().map(|line| (*line).to_owned()).collect(); - inherited.sort(); - assert_eq!(inherited, expected); - for object in &checked { - assert!( - object.to_string_lossy().ends_with(" (deleted)"), - "the checked file was not replaced at its path: {}", - object.display() - ); - } - } - #[cfg(windows)] - { - use std::os::windows::io::{AsHandle, AsRawHandle}; - - let inherited: Vec = prepared - .inherited() - .iter() - .map(|inherited| inherited.as_handle().as_raw_handle() as usize) - .collect(); - let checked = [ - prepared.distro().file().as_raw_handle() as usize, - prepared.scratch().unwrap().file().as_raw_handle() as usize, - ]; - assert_eq!(inherited.len(), 3); - assert_eq!(inherited[..2], checked); - for &value in &inherited { - assert!( - holds(pid, value, value), - "the VMM does not hold handle {value:#x}" - ); - } - let unrelated = unrelated.as_raw_handle() as usize; - assert!(!inherited.contains(&unrelated)); - assert!( - !holds(pid, unrelated, unrelated), - "the VMM holds handle {unrelated:#x}" - ); - } - - // The VMM opens its block devices as OpenVMM does, and finds the objects that the launch - // checked. - File::create(vmm.path().join(GO_MARKER)).unwrap(); - let report = vmm.report(BLOCKS_REPORT); - assert_eq!(report, staged_blocks()); - - // On Windows, while the launch and the VMM hold the checked files, nothing writes the distro - // layer, and nothing removes or renames either image, while the VMM reads and writes the - // scratch image. Once the guest is ready, the launch releases them, its own and the VMM's - // copies, while the VMM keeps the scratch image's claim, and what the VMM then holds, its own - // opens and the claim's handle, shares renames. - #[cfg(windows)] - let moved = |path: &Path| path.with_extension("moved"); - #[cfg(windows)] - let inherited = { - use std::os::windows::io::{AsHandle, AsRawHandle}; - - assert!( - OpenOptions::new() - .append(true) - .open(staged.distro()) - .is_err() - ); - for path in [staged.distro(), staged.scratch()] { - assert!(fs::rename(&path, moved(&path)).is_err()); - assert!(fs::remove_file(&path).is_err()); - } - // Each value that the VMM inherited, with a handle of this process's own to its object, - // which outlives the release. - prepared - .inherited() - .iter() - .map(|inherited| { - let handle = inherited.as_handle(); - ( - handle.as_raw_handle() as usize, - handle.try_clone_to_owned().unwrap(), - ) - }) - .collect::>() - }; - let process = vmm.launched.as_ref().unwrap().process().unwrap(); - prepared.release(&process).unwrap(); - #[cfg(windows)] - { - use std::os::windows::io::AsRawHandle; - - let held: Vec = inherited - .iter() - .map(|(value, object)| holds(pid, *value, object.as_raw_handle() as usize)) - .collect(); - assert_eq!(held, [false, false, true]); - drop(inherited); - for path in [staged.distro(), staged.scratch()] { - rename_away_and_back(&path); - } - } - - // The VMM wrote the checked scratch image, whatever has its path now. - File::create(vmm.path().join(EXIT_MARKER)).unwrap(); - let launched = vmm.launched.take().unwrap(); - assert!( - launched - .process() - .unwrap() - .wait(Instant::now() + TIMEOUT) - .unwrap() - .is_some() - ); - launched.kill_and_wait(); - let scratch = fs::read_to_string(staged.scratch()).unwrap(); - if cfg!(windows) { - assert_eq!(scratch, format!("{SCRATCH}written by the VMM\n")); - } else { - assert_eq!(scratch, "replaced scratch\n"); - } -} - -#[test] -fn a_vmm_keeps_the_scratch_images_claim_once_its_launcher_is_gone() { - use nvx::ErrorKind; - - let staged = Staged::new(); - let launch = staged.launch(); - let binding = DistroBinding::verify(launch.distro()).unwrap(); - let scratch_binding = ScratchBinding::bind(launch.scratch().unwrap()).unwrap(); - let check_scratch = || { - scratch_binding.check( - launch.scratch().unwrap(), - Instant::now() + CLAIM_RELEASE_WAIT, - ) - }; - let prepared = PreparedLaunch::new( - launch.bound_plan(&binding).unwrap(), - binding - .check(launch.distro(), DISTRO_POLICY_REVISION) - .unwrap(), - Some(check_scratch().unwrap()), - "", - ) - .unwrap(); - let mut vmm = VmmDirectory::new(); - start_vmm(&mut vmm, &prepared); - - // The launcher ends before the VMM has opened the images, as when it crashes, which closes - // the checked files as dropping the prepared launch does. The VMM, which inherited the - // scratch image's claim, keeps it: another sandbox cannot claim the image that the VMM is - // about to write. - drop(prepared); - let claimed = check_scratch().unwrap_err(); - assert_eq!(claimed.kind(), ErrorKind::TimedOut, "{claimed}"); - - // Something then replaces both images at their paths, before the VMM opens them: on Linux it - // can, and on Windows the copies of the checked files that the VMM inherited keep it from - // doing so. Either way, the VMM opens the checked images. On Linux, links that the test keeps - // to them put them back at their paths once the VMM has opened them, for the checks below. - #[cfg(target_os = "linux")] - let kept: Vec<(PathBuf, PathBuf)> = [staged.distro(), staged.scratch()] - .into_iter() - .map(|path| { - let kept = path.with_extension("kept"); - fs::hard_link(&path, &kept).unwrap(); - (kept, path) - }) - .collect(); - replace_images(&staged); - File::create(vmm.path().join(GO_MARKER)).unwrap(); - assert_eq!(vmm.report(BLOCKS_REPORT), staged_blocks()); - #[cfg(target_os = "linux")] - for (kept, path) in kept { - fs::rename(kept, path).unwrap(); - } - - // The claim lasts while the VMM writes the image, which changes its length here, so that a - // binding of the written image tells the claim apart, and it ends with the VMM. - let written = ScratchBinding::bind(launch.scratch().unwrap()).unwrap(); - assert_ne!(written, scratch_binding); - let check_written = || { - written.check( - launch.scratch().unwrap(), - Instant::now() + CLAIM_RELEASE_WAIT, - ) - }; - let claimed = check_written().unwrap_err(); - assert_eq!(claimed.kind(), ErrorKind::TimedOut, "{claimed}"); - File::create(vmm.path().join(EXIT_MARKER)).unwrap(); - let launched = vmm.launched.take().unwrap(); - assert!( - launched - .process() - .unwrap() - .wait(Instant::now() + TIMEOUT) - .unwrap() - .is_some() - ); - launched.kill_and_wait(); - drop(check_written().unwrap()); - - // On Windows, the VMM's copies of the checked files went with it too, so the images can be - // renamed again. - #[cfg(windows)] - for path in [staged.distro(), staged.scratch()] { - rename_away_and_back(&path); - } -} - -/// Renames the file at `path` away and back, which no process that holds it without sharing its -/// deletion allows, as the checked files of a launch hold it on Windows. -/// -/// Waits up to [`TIMEOUT`] for a process outside the tests that has the file open to close it. -/// Once the VMM has exited, no process of the tests holds the file: they spawn their other -/// processes under the spawn lock, as `supervisor::spawn_command` does, so none inherits the -/// checked files of a launch, which are inheritable while its VMM is spawned. -#[cfg(windows)] -fn rename_away_and_back(path: &Path) { - use windows_sys::Win32::Foundation::ERROR_SHARING_VIOLATION; - - let moved = path.with_extension("moved"); - let deadline = Instant::now() + TIMEOUT; - loop { - match fs::rename(path, &moved) { - Ok(()) => break, - Err(error) - if error.raw_os_error() == Some(ERROR_SHARING_VIOLATION as i32) - && Instant::now() < deadline => - { - thread::sleep(Duration::from_millis(10)); - } - Err(error) => panic!("cannot rename {}: {error}", path.display()), - } - } - fs::rename(&moved, path).unwrap(); -} - -/// Makes `link` a junction to the directory `target`, which needs no privilege. -#[cfg(windows)] -fn junction(link: &Path, target: &Path) { - let mut command = Command::new("cmd"); - command - .args(["/C", "mklink", "/J"]) - .arg(link) - .arg(target) - .stdout(Stdio::null()); - let status = supervisor::spawn_command(&mut command) - .unwrap() - .wait() - .unwrap(); - assert!(status.success(), "cannot link {}", link.display()); -} - -#[cfg(windows)] -#[test] -fn a_vmm_finds_the_checked_images_past_replaced_links_from_another_directory() { - // The images, and decoys of them, in the target's temporary directory, which the plan names - // through a junction, relative to this process's directory when the target is below it. - let root = tempfile::tempdir_in(env!("CARGO_TARGET_TMPDIR")).unwrap(); - let real = root.path().join("real"); - let decoy = root.path().join("decoy"); - for (directory, distro, scratch) in [ - (&real, DISTRO, SCRATCH), - (&decoy, "decoy distro\n", "decoy scratch\n"), - ] { - fs::create_dir(directory).unwrap(); - fs::write(directory.join("distro.erofs"), distro).unwrap(); - fs::write(directory.join("scratch.ext4"), scratch).unwrap(); - } - let link = root.path().join("link"); - junction(&link, &real); - let named = link - .strip_prefix(env::current_dir().unwrap()) - .map_or_else(|_| link.clone(), Path::to_path_buf); - let launch = launch_of(&named.join("distro.erofs"), &named.join("scratch.ext4")); - let binding = DistroBinding::verify(launch.distro()).unwrap(); - let scratch_binding = ScratchBinding::bind(launch.scratch().unwrap()).unwrap(); - let distro = binding - .check(launch.distro(), DISTRO_POLICY_REVISION) - .unwrap(); - let scratch = scratch_binding - .check( - launch.scratch().unwrap(), - Instant::now() + CLAIM_RELEASE_WAIT, - ) - .unwrap(); - let prepared = PreparedLaunch::new( - launch.bound_plan(&binding).unwrap(), - distro, - Some(scratch), - "", - ) - .unwrap(); - - // The arguments name the images by their final paths, which go through no link. - let final_path = |name: &str| { - let path = fs::canonicalize(real.join(name)).unwrap(); - path.to_str().unwrap().to_owned() - }; - assert_eq!( - block_arguments(&prepared), - [ - format!( - "{BLOCK_PREFIX}distro:file:{},ro", - final_path("distro.erofs") - ), - format!("{BLOCK_PREFIX}scratch:file:{}", final_path("scratch.ext4")), - ] - .map(OsString::from) - ); - - // Then the link leads to the decoys, and nothing renames the directory of the images, which - // the launch holds. - fs::remove_dir(&link).unwrap(); - junction(&link, &decoy); - assert!(fs::rename(&real, root.path().join("moved")).is_err()); - - // The VMM, which starts in another directory, finds the checked images. - let mut vmm = VmmDirectory::new(); - start_vmm(&mut vmm, &prepared); - File::create(vmm.path().join(GO_MARKER)).unwrap(); - assert_eq!(vmm.report(BLOCKS_REPORT), staged_blocks()); - drop(prepared); - drop(vmm); -} - -/// The launcher fixture: an orchestrator that closes its standard input, as a daemon may, before -/// it checks the scratch image, which the check then opens as descriptor 0, under which no spawn -/// passes it. It prepares the launch, launches a VMM fixture, and writes [`LAUNCHED_REPORT`] once -/// the VMM found the checked images. -#[cfg(target_os = "linux")] -#[test] -#[ignore = "fixture that a handoff test runs as an orchestrator without standard input"] -fn launcher_fixture() { - use std::os::fd::{AsRawFd, FromRawFd, OwnedFd}; - - use nvx::ErrorKind; - - let directory = env::current_dir().unwrap(); - if !directory.join(LAUNCHER_MARKER).exists() { - return; - } - let staged = Staged::new(); - let launch = staged.launch(); - let binding = DistroBinding::verify(launch.distro()).unwrap(); - let scratch_binding = ScratchBinding::bind(launch.scratch().unwrap()).unwrap(); - let distro = binding - .check(launch.distro(), DISTRO_POLICY_REVISION) - .unwrap(); - // SAFETY: descriptor 0, standard input, is open, and nothing in this process reads it. - nix::unistd::close(unsafe { OwnedFd::from_raw_fd(0) }).unwrap(); - let scratch = scratch_binding - .check( - launch.scratch().unwrap(), - Instant::now() + CLAIM_RELEASE_WAIT, - ) - .unwrap(); - assert_eq!(scratch.file().as_raw_fd(), 0); - let prepared = PreparedLaunch::new( - launch.bound_plan(&binding).unwrap(), - distro, - Some(scratch), - "", - ) - .unwrap(); - - // The image moved above the standard streams, descriptor 0 is closed again, and the image's - // claim moved with it. - let moved = prepared.scratch().unwrap().file().as_raw_fd(); - assert!(moved > 2, "{moved}"); - assert!(fs::read_link("/proc/self/fd/0").is_err()); - assert_eq!( - scratch_binding - .check(launch.scratch().unwrap(), Instant::now()) - .unwrap_err() - .kind(), - ErrorKind::TimedOut - ); - - // The VMM inherits the image under its new number, and finds the checked images. - let mut vmm = VmmDirectory::new(); - start_vmm(&mut vmm, &prepared); - let descriptors = vmm.report(DESCRIPTORS_REPORT); - assert!( - descriptors - .lines() - .any(|line| line.starts_with(&format!("{moved} "))), - "{descriptors}" - ); - File::create(vmm.path().join(GO_MARKER)).unwrap(); - assert_eq!(vmm.report(BLOCKS_REPORT), staged_blocks()); - drop(prepared); - drop(vmm); - write_whole(&directory.join(LAUNCHED_REPORT), ""); -} - -#[cfg(target_os = "linux")] -#[test] -fn a_launch_passes_a_checked_file_that_took_a_standard_streams_descriptor() { - let directory = tempfile::tempdir().unwrap(); - File::create(directory.path().join(LAUNCHER_MARKER)).unwrap(); - let output = Command::new(env::current_exe().unwrap()) - .args(fixture_arguments("launcher_fixture")) - .current_dir(directory.path()) - .stdin(Stdio::null()) - .output() - .unwrap(); - assert!( - output.status.success(), - "{}{}", - String::from_utf8_lossy(&output.stdout), - String::from_utf8_lossy(&output.stderr) - ); - assert!(directory.path().join(LAUNCHED_REPORT).exists()); -} diff --git a/nvx-rs/tests/cli/openvmm.rs b/nvx-rs/tests/cli/openvmm.rs index a22e37327..f46b8e3ce 100644 --- a/nvx-rs/tests/cli/openvmm.rs +++ b/nvx-rs/tests/cli/openvmm.rs @@ -14,15 +14,14 @@ use crate::harness::{HYPERVISOR, OPENVMM, Root, is_hex}; /// Caller bytes left by the two-directory fixture, its native bootstrap, and a terminating NUL. pub(crate) fn two_share_command_line_room() -> usize { use nvx::host::filesystem::GUEST_EXPORT; - use nvx::host::sandbox::mount::{ - AGGREGATE_MOUNT_TARGET, MountAccess, aggregate_command_line_fragment, - }; + use nvx::host::sandbox::mount::MOUNT_TAG; let kernel = "nvx_hostname=6e7678 nvx_machine_id=image nvx_maps=2 nvx_mode=one-shot \ nvx_share=0,/a,ro nvx_share=1,/b,rw"; - let fragment = - aggregate_command_line_fragment(&[MountAccess::ReadOnly, MountAccess::ReadWrite]) - .replace(AGGREGATE_MOUNT_TARGET, GUEST_EXPORT); + // OpenVMM appends the tokens of the aggregate, read-write since one share is. + let fragment = format!( + " virtfs_dir={GUEST_EXPORT} virtfs_tag={MOUNT_TAG} virtfs_mode=rw virtfs_aggregate=1" + ); nvx::host::config::COMMAND_LINE_BUDGET - kernel.len() - fragment.len() - 2 } diff --git a/nvx-rs/tests/host_modules.rs b/nvx-rs/tests/host_modules.rs index 02ab8dccc..c18f41f46 100644 --- a/nvx-rs/tests/host_modules.rs +++ b/nvx-rs/tests/host_modules.rs @@ -3301,12 +3301,7 @@ fn the_children_of_host_depend_on_one_another_without_cycles() { #[test] fn the_graph_holds_well_known_dependencies_of_the_sources() { let graph = analyze(&read_sources()).unwrap_or_else(|error| panic!("{error}")); - for (from, to) in [ - ("vm", "spawn"), - ("supervisor", "spawn"), - ("cli", "sandbox"), - ("spawn", "platform"), - ] { + for (from, to) in [("vm", "spawn"), ("cli", "sandbox"), ("spawn", "platform")] { assert!( graph.edges.contains_key(&(from.to_owned(), to.to_owned())), "the graph lacks {from} -> {to}" diff --git a/nvx-rs/tests/process.rs b/nvx-rs/tests/process.rs new file mode 100644 index 000000000..c073bd3e6 --- /dev/null +++ b/nvx-rs/tests/process.rs @@ -0,0 +1,577 @@ +//! Verified process references and event-driven exit waits, through the public API of +//! `nvx::host::process`, as an orchestrator uses them. +//! +//! The processes are this test executable, run again as one of two fixtures, which inherit +//! nothing of the test: a process fixture runs until a file named `exit` appears in its working +//! directory, and a launcher fixture starts a process fixture, prints the process's reference as +//! an orchestrator would record it, and runs until its standard input closes. +#![cfg(all(feature = "host", any(target_os = "linux", windows)))] + +use std::env; +use std::ffi::OsString; +use std::fs::File; +use std::io::{self, BufRead, BufReader, Write}; +use std::path::Path; +use std::process::{Child, ChildStdin, Command, ExitStatus, Stdio}; +use std::sync::{Mutex, MutexGuard, PoisonError, mpsc}; +use std::thread; +use std::time::{Duration, Instant}; + +use nvx::host::platform; +use nvx::host::process::{BootId, Opened, Process, ProcessRef, WaitMethod}; + +/// File whose presence in its working directory marks a process fixture. Without it, the fixture +/// returns at once, as it does when `cargo test -- --ignored` runs it. +const PROCESS_MARKER: &str = "process-fixture"; +/// File whose appearance in its working directory ends a process fixture. +const EXIT_MARKER: &str = "exit"; +/// Environment variable that marks a launcher fixture, and names the working directory of its +/// process. +const LAUNCHER_DIR: &str = "NVX_PROCESS_TEST_DIR"; +/// Marker before the reference that a launcher fixture prints. Libtest's own output, the name of +/// the fixture, precedes it on its line. +const REFERENCE_PREFIX: &str = "process-reference "; +/// Longest wait for a process that is expected to exit, or for a fixture to answer. +const TIMEOUT: Duration = Duration::from_secs(60); +/// Longest life of a process fixture that nothing ends, so that a failed test leaves no process. +const PROCESS_LIFETIME: Duration = Duration::from_secs(180); +/// A wait that a running process outlasts. +const SHORT_WAIT: Duration = Duration::from_millis(100); + +/// Serializes the tests that spawn processes. +static SPAWNS: Mutex<()> = Mutex::new(()); + +/// Takes [`SPAWNS`]. +fn serialized() -> MutexGuard<'static, ()> { + SPAWNS.lock().unwrap_or_else(PoisonError::into_inner) +} + +/// The process fixture: runs until a file named [`EXIT_MARKER`] appears in its working directory. +#[test] +#[ignore = "fixture that the process tests run"] +fn process_fixture() { + let directory = env::current_dir().unwrap(); + if !directory.join(PROCESS_MARKER).exists() { + return; + } + let started = Instant::now(); + while !directory.join(EXIT_MARKER).exists() && started.elapsed() < PROCESS_LIFETIME { + thread::sleep(Duration::from_millis(10)); + } +} + +/// The launcher fixture: when the test writes a line to its standard input, starts a process +/// fixture in the directory that [`LAUNCHER_DIR`] names, prints its reference, releases it, and +/// runs until its standard input closes. +#[test] +#[ignore = "fixture that the process tests run as a launcher"] +fn launcher_fixture() { + let Some(directory) = env::var_os(LAUNCHER_DIR) else { + return; + }; + let mut input = io::stdin().lock(); + input.read_line(&mut String::new()).unwrap(); + let mut child = spawn_process(Path::new(&directory)); + println!( + "{REFERENCE_PREFIX}{}", + serde_json::to_string(&reference_of(&child)).unwrap() + ); + // The launcher releases its process, which it reaps only if it exits first. + thread::spawn(move || child.wait()); + io::copy(&mut input, &mut io::sink()).unwrap(); +} + +/// Returns the arguments that run only the fixture `name` of this test executable. +/// +/// The fixture runs on libtest's main thread, whatever the host's processors or the inherited +/// `RUST_TEST_THREADS`, so libtest always prints its name before the fixture's own output. +fn fixture_arguments(name: &str) -> Vec { + [ + "--exact", + name, + "--ignored", + "--nocapture", + "--test-threads=1", + ] + .map(OsString::from) + .to_vec() +} + +/// Starts a process fixture in `directory`, which it marks, without standard streams. +fn spawn_process(directory: &Path) -> Child { + File::create(directory.join(PROCESS_MARKER)).unwrap(); + Command::new(env::current_exe().unwrap()) + .args(fixture_arguments("process_fixture")) + .current_dir(directory) + .stdin(Stdio::null()) + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .spawn() + .unwrap() +} + +/// Returns the reference of `child`, which has not been reaped, so that its ID still names it. +fn reference_of(child: &Child) -> ProcessRef { + let pid = child.id(); + ProcessRef { + pid, + start_time: platform::process_start_time(pid).unwrap().unwrap(), + boot: BootId::current().unwrap(), + } +} + +/// The working directory of a process fixture. Dropping it ends the fixture, or a process that +/// ignores the exit marker, and waits for it to exit, so that a failed test leaves no process +/// behind. +struct ProcessDirectory { + /// The reference of the process, once the test knows it. + reference: Option, + /// The directory, which is removed after the process exits. + directory: tempfile::TempDir, +} + +impl ProcessDirectory { + /// Creates an empty directory. + fn new() -> Self { + Self { + reference: None, + directory: tempfile::tempdir().unwrap(), + } + } + + /// Returns the path of the directory. + fn path(&self) -> &Path { + self.directory.path() + } + + /// Records the reference of the process that runs in the directory. + fn record(&mut self, reference: &ProcessRef) { + self.reference = Some(reference.clone()); + } + + /// Asks the process to exit. + fn end(&self) { + File::create(self.path().join(EXIT_MARKER)).unwrap(); + } +} + +impl Drop for ProcessDirectory { + fn drop(&mut self) { + let _ = File::create(self.path().join(EXIT_MARKER)); + if let Some(reference) = &self.reference + && let Ok(Opened::Running(process)) = Process::open(reference) + && !process + .wait(Instant::now() + TIMEOUT) + .is_ok_and(|exited| exited.is_some()) + { + let _ = process.kill(); + let _ = process.wait(Instant::now() + TIMEOUT); + } + } +} + +/// A child of the test, which is ended and reaped when dropped. +struct Reaped(Child); + +impl Drop for Reaped { + fn drop(&mut self) { + let _ = self.0.kill(); + let _ = self.0.wait(); + } +} + +/// A launcher fixture, which the test drives through its standard streams. +struct Launcher { + /// The fixture's process. + child: Child, + /// Its standard input, which the test closes to end it. + input: Option, +} + +impl Launcher { + /// Starts a launcher whose process runs in `directory`. + fn start(directory: &Path) -> Self { + let mut child = Command::new(env::current_exe().unwrap()) + .args(fixture_arguments("launcher_fixture")) + .env(LAUNCHER_DIR, directory) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .spawn() + .unwrap(); + let input = child.stdin.take(); + Self { child, input } + } + + /// Asks the fixture to start its process, and returns the reference that it prints. + fn launch_reference(&mut self) -> ProcessRef { + let input = self.input.as_mut().unwrap(); + input.write_all(b"launch\n").unwrap(); + input.flush().unwrap(); + let output = self.child.stdout.take().unwrap(); + let (sender, receiver) = mpsc::channel(); + thread::spawn(move || { + let mut lines = BufReader::new(output).lines(); + let reference = lines.by_ref().find_map(|line| { + let line = line.ok()?; + Some(line.split_once(REFERENCE_PREFIX)?.1.to_owned()) + }); + let _ = sender.send(reference); + // Read the rest, so that the fixture can still write its own output. + lines.for_each(drop); + }); + let reference = receiver + .recv_timeout(TIMEOUT) + .expect("the fixture printed no reference in time") + .expect("the fixture exited before it printed a reference"); + serde_json::from_str(&reference).unwrap() + } + + /// Waits for the fixture to exit, and returns its status. + fn wait(&mut self) -> ExitStatus { + let deadline = Instant::now() + TIMEOUT; + loop { + if let Some(status) = self.child.try_wait().unwrap() { + return status; + } + assert!(Instant::now() < deadline, "the fixture did not exit"); + thread::sleep(Duration::from_millis(10)); + } + } + + /// Closes the fixture's input, which ends it, and waits for it to exit. + fn finish(&mut self) { + drop(self.input.take()); + assert!(self.wait().success()); + } +} + +impl Drop for Launcher { + fn drop(&mut self) { + let _ = self.child.kill(); + let _ = self.child.wait(); + } +} + +/// Opens the process that `reference` names, which must still run. +fn open_running(reference: &ProcessRef) -> Process { + let Opened::Running(process) = Process::open(reference).unwrap() else { + panic!("{reference:?} does not run"); + }; + process +} + +/// Checks that the process that `process` holds has exited, and that no process matches its +/// reference. +fn assert_ended(process: &Process) { + assert!(process.has_exited().unwrap()); + assert!(matches!( + Process::open(process.reference()).unwrap(), + Opened::Exited(_) + )); +} + +#[test] +fn processes_can_be_shared_between_threads() { + /// Compiles only for types that threads can share. + fn shared() {} + shared::(); + shared::(); +} + +#[test] +fn a_child_that_exits_wakes_its_waiter() { + let _spawns = serialized(); + let mut directory = ProcessDirectory::new(); + let mut child = Reaped(spawn_process(directory.path())); + let process = open_running(&reference_of(&child.0)); + directory.record(process.reference()); + assert_eq!(process.reference().pid, child.0.id()); + assert_eq!(process.reference().boot, BootId::current().unwrap()); + assert_eq!(process.wait_method(), WaitMethod::Event); + assert!(!process.has_exited().unwrap()); + assert!(process.wait(Instant::now() + SHORT_WAIT).unwrap().is_none()); + // The reference finds the same process through a handle of its own. + let found = open_running(process.reference()); + assert_eq!(found.reference(), process.reference()); + + let started = Instant::now(); + directory.end(); + assert!(process.wait(started + TIMEOUT).unwrap().is_some()); + assert!(found.wait(started + TIMEOUT).unwrap().is_some()); + assert!(started.elapsed() < TIMEOUT); + assert_ended(&process); + // Ending a process that exited does nothing. + process.kill().unwrap(); + assert!(child.0.wait().unwrap().success()); +} + +#[test] +fn a_process_that_is_not_a_child_wakes_its_waiters() { + let _spawns = serialized(); + let mut directory = ProcessDirectory::new(); + let mut launcher = Launcher::start(directory.path()); + let reference = launcher.launch_reference(); + directory.record(&reference); + // The launcher exits, so its process is no child of this process. + launcher.finish(); + let process = open_running(&reference); + assert_eq!(process.wait_method(), WaitMethod::Event); + assert!(process.wait(Instant::now() + SHORT_WAIT).unwrap().is_none()); + + let started = Instant::now(); + let waiters: Vec = thread::scope(|scope| { + let waiters: Vec<_> = (0..2) + .map(|_| scope.spawn(|| process.wait(started + TIMEOUT).unwrap().is_some())) + .collect(); + directory.end(); + waiters + .into_iter() + .map(|waiter| waiter.join().unwrap()) + .collect() + }); + assert_eq!(waiters, [true, true]); + assert!(started.elapsed() < TIMEOUT); + assert_ended(&process); +} + +#[test] +fn a_reused_process_id_is_never_signaled() { + let _spawns = serialized(); + let mut directory = ProcessDirectory::new(); + let mut child = Reaped(spawn_process(directory.path())); + let process = open_running(&reference_of(&child.0)); + let reference = process.reference().clone(); + directory.record(&reference); + let mut other_boot = *reference.boot.as_bytes(); + other_boot[15] ^= 1; + // A process with the same ID and another start time is not the referenced one, and neither + // is one of another boot: the referenced process exited, and its ID now names another. + for impostor in [ + ProcessRef { + start_time: reference.start_time + 1, + ..reference.clone() + }, + ProcessRef { + start_time: reference.start_time - 1, + ..reference.clone() + }, + ProcessRef { + boot: BootId::from_bytes(other_boot), + ..reference.clone() + }, + ] { + assert!( + matches!(Process::open(&impostor).unwrap(), Opened::Exited(_)), + "{impostor:?} names a running process" + ); + } + + // The process that has the ID still runs. + assert!(!process.has_exited().unwrap()); + open_running(&reference); + directory.end(); + assert!(process.wait(Instant::now() + TIMEOUT).unwrap().is_some()); + assert!(child.0.wait().unwrap().success()); +} + +#[test] +fn references_that_name_no_process_are_never_opened() { + let _spawns = serialized(); + let boot = BootId::current().unwrap(); + let invalid = ProcessRef { + pid: 0, + start_time: 1, + boot, + }; + let error = Process::open(&invalid).unwrap_err(); + assert_eq!(error.kind(), io::ErrorKind::InvalidInput); + // Linux assigns no process ID beyond `i32::MAX`. + #[cfg(target_os = "linux")] + { + let invalid = ProcessRef { + pid: u32::MAX, + ..invalid + }; + let error = Process::open(&invalid).unwrap_err(); + assert_eq!(error.kind(), io::ErrorKind::InvalidInput); + } + + // A process that exited and was reaped: no process has its ID, or another one has it. + let directory = ProcessDirectory::new(); + let mut child = Reaped(spawn_process(directory.path())); + let process = open_running(&reference_of(&child.0)); + directory.end(); + assert!(process.wait(Instant::now() + TIMEOUT).unwrap().is_some()); + assert!(child.0.wait().unwrap().success()); + assert!(matches!( + Process::open(process.reference()).unwrap(), + Opened::Exited(_) + )); + assert!(process.has_exited().unwrap()); +} + +#[cfg(windows)] +#[test] +fn a_process_that_this_user_may_not_query_has_an_unknown_state() { + use windows_sys::Win32::System::Threading::PROCESS_QUERY_LIMITED_INFORMATION; + + let _spawns = serialized(); + let (mut child, reference) = restricted::spawn(0); + if restricted::may_open(&child.0, PROCESS_QUERY_LIMITED_INFORMATION) { + eprintln!("skipping: this user's privileges bypass the security descriptors of processes"); + return; + } + // Its identity cannot be checked, so it may still run. + let error = Process::open(&reference).unwrap_err(); + assert_eq!(error.kind(), io::ErrorKind::PermissionDenied); + assert!( + child.0.try_wait().unwrap().is_none(), + "the process was ended" + ); +} + +#[cfg(windows)] +#[test] +fn a_process_that_this_user_may_not_terminate_is_waited_for_but_never_signaled() { + use windows_sys::Win32::System::Threading::{ + PROCESS_QUERY_LIMITED_INFORMATION, PROCESS_SYNCHRONIZE, PROCESS_TERMINATE, + }; + + let _spawns = serialized(); + let (mut child, reference) = + restricted::spawn(PROCESS_QUERY_LIMITED_INFORMATION | PROCESS_SYNCHRONIZE); + if restricted::may_open(&child.0, PROCESS_TERMINATE) { + eprintln!("skipping: this user's privileges bypass the security descriptors of processes"); + return; + } + let process = open_running(&reference); + assert_eq!(process.wait_method(), WaitMethod::Event); + let error = process.kill().unwrap_err(); + assert_eq!(error.kind(), io::ErrorKind::PermissionDenied); + assert!(!process.has_exited().unwrap()); + assert!( + child.0.try_wait().unwrap().is_none(), + "the process was ended" + ); + + // Its creator ends it, which wakes the waiter, and ending a process that exited does nothing. + child.0.kill().unwrap(); + assert!(process.wait(Instant::now() + TIMEOUT).unwrap().is_some()); + process.kill().unwrap(); + assert_ended(&process); +} + +/// Processes whose security descriptor restricts what this user may do with them. +#[cfg(windows)] +mod restricted { + use std::os::windows::io::AsRawHandle; + use std::process::{Child, Command, Stdio}; + use std::{io, mem, ptr}; + + use nvx::host::platform; + use nvx::host::process::{BootId, ProcessRef}; + use windows_sys::Win32::Foundation::{CloseHandle, ERROR_SUCCESS}; + use windows_sys::Win32::Security::Authorization::{SE_KERNEL_OBJECT, SetSecurityInfo}; + use windows_sys::Win32::Security::{ + ACCESS_ALLOWED_ACE, ACL, ACL_REVISION, AddAccessAllowedAce, CreateWellKnownSid, + DACL_SECURITY_INFORMATION, InitializeAcl, PROTECTED_DACL_SECURITY_INFORMATION, + SECURITY_MAX_SID_SIZE, WinWorldSid, + }; + use windows_sys::Win32::System::Threading::OpenProcess; + + /// A child process, which is ended when dropped, through the handle with full access that + /// its creator holds. + pub(crate) struct Running(pub(crate) Child); + + impl Drop for Running { + fn drop(&mut self) { + let _ = self.0.kill(); + let _ = self.0.wait(); + } + } + + /// Starts a process that runs for a minute unless it is ended, and whose security + /// descriptor then grants everyone, this user included, only `access` to it. Returns it with + /// its reference. + pub(crate) fn spawn(access: u32) -> (Running, ProcessRef) { + let child = Command::new("ping") + .args(["-n", "60", "127.0.0.1"]) + .stdin(Stdio::null()) + .stdout(Stdio::null()) + .spawn() + .unwrap(); + let child = Running(child); + let pid = child.0.id(); + let reference = ProcessRef { + pid, + start_time: platform::process_start_time(pid).unwrap().unwrap(), + boot: BootId::current().unwrap(), + }; + restrict(&child.0, access); + (child, reference) + } + + /// Replaces the DACL of `child` with one that grants everyone `access`, or no access when + /// `access` is 0. + fn restrict(child: &Child, access: u32) { + let mut sid = [0u8; SECURITY_MAX_SID_SIZE as usize]; + let mut sid_size = SECURITY_MAX_SID_SIZE; + // SAFETY: the buffer holds `sid_size` bytes. + let created = unsafe { + CreateWellKnownSid( + WinWorldSid, + ptr::null_mut(), + sid.as_mut_ptr().cast(), + &mut sid_size, + ) + }; + assert_ne!(created, 0, "{}", io::Error::last_os_error()); + // Room for the ACL header, one ACE, and its SID, in 4-byte units, which align the ACL. + let size = mem::size_of::() + + mem::size_of::() + + usize::try_from(sid_size).unwrap(); + let mut buffer = vec![0u32; size.div_ceil(4)]; + let acl: *mut ACL = buffer.as_mut_ptr().cast(); + let length = u32::try_from(buffer.len() * 4).unwrap(); + // SAFETY: the buffer holds `length` bytes, aligned for an ACL. + let initialized = unsafe { InitializeAcl(acl, length, ACL_REVISION) }; + assert_ne!(initialized, 0, "{}", io::Error::last_os_error()); + if access != 0 { + // SAFETY: the ACL has room for the ACE, and the SID is valid. + let added = + unsafe { AddAccessAllowedAce(acl, ACL_REVISION, access, sid.as_mut_ptr().cast()) }; + assert_ne!(added, 0, "{}", io::Error::last_os_error()); + } + // SAFETY: the creator's handle has WRITE_DAC access, and the ACL is valid. + let status = unsafe { + SetSecurityInfo( + child.as_raw_handle(), + SE_KERNEL_OBJECT, + DACL_SECURITY_INFORMATION | PROTECTED_DACL_SECURITY_INFORMATION, + ptr::null_mut(), + ptr::null_mut(), + acl, + ptr::null(), + ) + }; + assert_eq!( + status, + ERROR_SUCCESS, + "cannot restrict a process: {}", + io::Error::from_raw_os_error(status as i32) + ); + } + + /// Returns whether this user may still open `child` with `access`, as a user may whose + /// privileges bypass security descriptors. + pub(crate) fn may_open(child: &Child, access: u32) -> bool { + // SAFETY: OpenProcess has no memory-safety preconditions. + let handle = unsafe { OpenProcess(access, 0, child.id()) }; + if handle.is_null() { + return false; + } + // SAFETY: the handle is open, and nothing else owns it. + unsafe { CloseHandle(handle) }; + true + } +} diff --git a/nvx-rs/tests/supervisor.rs b/nvx-rs/tests/supervisor.rs deleted file mode 100644 index 7cc91fbfa..000000000 --- a/nvx-rs/tests/supervisor.rs +++ /dev/null @@ -1,1159 +0,0 @@ -//! Verified process references, event-driven exit waits, and the spawn of VMM processes, -//! through the public API of `nvx::host::process` and the low-level spawn that -//! `nvx::host::supervisor` exports, as an orchestrator uses them. -//! -//! The processes are this test executable, run again as one of two fixtures. A VMM fixture -//! reads the capability that `nvx::host::supervisor::spawn` passes on its standard input, as -//! OpenVMM does, reports its descriptors on Linux, writes a line through each descriptor or handle -//! that its arguments name as inherited, and runs until a file named `exit` appears in its working -//! directory. A launcher fixture spawns a VMM fixture, prints the VMM's reference as an -//! orchestrator would record it, and runs until its standard input closes. A launcher that the -//! test kills, or whose job object the test closes, stands for an orchestrator that crashed and -//! restarts within the same boot: the test then finds the VMM again, as the restarted -//! orchestrator would. -#![cfg(all(feature = "host", any(target_os = "linux", windows)))] - -use std::env; -use std::ffi::OsString; -use std::fs::{self, File}; -use std::io::{self, BufRead, BufReader, PipeReader, PipeWriter, Read, Write}; -use std::mem::ManuallyDrop; -use std::path::{Path, PathBuf}; -use std::process::{Child, ChildStdin, Command, ExitStatus, Stdio}; -use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering}; -use std::sync::{Mutex, MutexGuard, PoisonError, mpsc}; -use std::thread; -use std::time::{Duration, Instant}; - -use nvx::host::process::{BootId, Opened, Process, ProcessRef, WaitMethod}; -use nvx::host::supervisor::{self, Inherited, Launched, VmmCommand}; -use nvx::protocol::CAPABILITY_LEN; - -/// File whose presence in its working directory marks a VMM fixture. Without it, the fixture -/// returns at once, as it does when `cargo test -- --ignored` runs it. -const VMM_MARKER: &str = "vmm-fixture"; -/// File whose appearance in its working directory ends a VMM fixture. -const EXIT_MARKER: &str = "exit"; -/// File in which a VMM fixture reports its descriptors on Linux, a line for each: its number and -/// the object that it holds. -#[cfg(target_os = "linux")] -const DESCRIPTORS_REPORT: &str = "descriptors"; -/// Prefix of an argument of a VMM fixture that names, in decimal, a descriptor or handle that the -/// fixture inherited, and through which it writes a line. Libtest takes the argument for a test -/// name that matches no test. -const INHERITED_PREFIX: &str = "inherited="; -/// Environment variable that marks a launcher fixture, and names the working directory of its -/// VMM. -const LAUNCHER_DIR: &str = "NVX_SUPERVISOR_TEST_VMM_DIR"; -/// Environment variable that asks a launcher fixture to start its VMM outside its job object. -const LAUNCHER_BREAKAWAY: &str = "NVX_SUPERVISOR_TEST_BREAKAWAY"; -/// Marker before the reference that a launcher fixture prints. Libtest's own output, the name of -/// the fixture, precedes it on its line. -const REFERENCE_PREFIX: &str = "vmm-reference "; -/// The capability of every VMM of these tests. -const CAPABILITY: [u8; CAPABILITY_LEN] = [0x5a; CAPABILITY_LEN]; -/// Longest wait for a process that is expected to exit, or for a fixture to answer. -const TIMEOUT: Duration = Duration::from_secs(60); -/// Longest life of a VMM fixture that nothing ends, so that a failed test leaves no process. -const VMM_LIFETIME: Duration = Duration::from_secs(180); -/// A wait that a running process outlasts. -const SHORT_WAIT: Duration = Duration::from_millis(100); - -/// Serializes the tests that spawn processes. On Windows, a process that one test spawns while -/// another test's handles are inheritable for a spawn of its own, as the log of a launch is, -/// could hold them, and so the claim of that log, for a while. -static SPAWNS: Mutex<()> = Mutex::new(()); - -/// Takes [`SPAWNS`]. -fn serialized() -> MutexGuard<'static, ()> { - SPAWNS.lock().unwrap_or_else(PoisonError::into_inner) -} - -/// The VMM fixture: reads its capability as OpenVMM does, reports its descriptors on Linux, -/// writes a line through each descriptor or handle that an argument names as inherited, then runs -/// until a file named [`EXIT_MARKER`] appears in its working directory. -#[test] -#[ignore = "fixture that the supervision tests run as a VMM"] -fn vmm_fixture() { - let directory = env::current_dir().unwrap(); - if !directory.join(VMM_MARKER).exists() { - return; - } - let mut capability = [0; CAPABILITY_LEN]; - io::stdin().read_exact(&mut capability).unwrap(); - assert_eq!(capability, CAPABILITY); - #[cfg(target_os = "linux")] - report_descriptors(&directory); - for argument in env::args() { - if let Some(number) = argument.strip_prefix(INHERITED_PREFIX) { - // A VMM keeps what it inherits open. - let mut inherited = ManuallyDrop::new(inherited_file(number.parse().unwrap())); - writeln!(inherited, "inherited {number}").unwrap(); - } - } - let started = Instant::now(); - while !directory.join(EXIT_MARKER).exists() && started.elapsed() < VMM_LIFETIME { - thread::sleep(Duration::from_millis(10)); - } -} - -/// Writes the descriptors of this VMM fixture, but for the one that lists them, to the file -/// [`DESCRIPTORS_REPORT`] in `directory`, which appears whole. -#[cfg(target_os = "linux")] -fn report_descriptors(directory: &Path) { - // The descriptor that lists the others holds this directory. - let listing = PathBuf::from(format!("/proc/{}/fd", std::process::id())); - let mut descriptors: Vec<(usize, PathBuf)> = fs::read_dir("/proc/self/fd") - .unwrap() - .filter_map(|entry| { - let entry = entry.unwrap(); - let number = entry.file_name().to_str().unwrap().parse().unwrap(); - let object = fs::read_link(entry.path()).ok()?; - (object != listing).then_some((number, object)) - }) - .collect(); - descriptors.sort(); - let report: String = descriptors - .iter() - .map(|(number, object)| format!("{number} {}\n", object.display())) - .collect(); - let temporary = directory.join(format!("{DESCRIPTORS_REPORT}.tmp")); - fs::write(&temporary, report).unwrap(); - fs::rename(&temporary, directory.join(DESCRIPTORS_REPORT)).unwrap(); -} - -/// Takes the descriptor `number`, which this VMM fixture inherited. -#[cfg(target_os = "linux")] -fn inherited_file(number: usize) -> File { - use std::os::fd::FromRawFd; - - // SAFETY: the test named a descriptor that this process inherited under that number, and - // that nothing else in this process owns. - unsafe { File::from_raw_fd(i32::try_from(number).unwrap()) } -} - -/// Takes the handle `number`, which this VMM fixture inherited. -#[cfg(windows)] -fn inherited_file(number: usize) -> File { - use std::os::windows::io::{FromRawHandle, RawHandle}; - - // SAFETY: the test named a handle that this process inherited under that value, and that - // nothing else in this process owns. - unsafe { File::from_raw_handle(number as RawHandle) } -} - -/// The launcher fixture: when the test writes a line to its standard input, spawns a VMM fixture -/// in the directory that [`LAUNCHER_DIR`] names, prints its reference, releases it, and runs -/// until its standard input closes. -#[test] -#[ignore = "fixture that the supervision tests run as a launcher"] -fn launcher_fixture() { - let Some(directory) = env::var_os(LAUNCHER_DIR) else { - return; - }; - let mut input = io::stdin().lock(); - input.read_line(&mut String::new()).unwrap(); - let breakaway = env::var_os(LAUNCHER_BREAKAWAY).is_some(); - let launched = spawn_vmm(Path::new(&directory), breakaway); - print_reference(launched.process().unwrap().reference()); - launched.detach(); - io::copy(&mut input, &mut io::sink()).unwrap(); -} - -/// Prints `reference` as JSON, after [`REFERENCE_PREFIX`]. -fn print_reference(reference: &ProcessRef) { - println!( - "{REFERENCE_PREFIX}{}", - serde_json::to_string(reference).unwrap() - ); -} - -/// Returns the arguments that run only the fixture `name` of this test executable. -/// -/// The fixture runs on libtest's main thread, whatever the host's processors or the inherited -/// `RUST_TEST_THREADS`, so libtest always prints its name before the fixture's own output. -fn fixture_arguments(name: &str) -> Vec { - [ - "--exact", - name, - "--ignored", - "--nocapture", - "--test-threads=1", - ] - .map(OsString::from) - .to_vec() -} - -/// Returns the command of a VMM fixture in `directory`, which it marks, outside the caller's job -/// object if `breakaway`. -fn vmm_command(directory: &Path, breakaway: bool) -> VmmCommand<'static> { - File::create(directory.join(VMM_MARKER)).unwrap(); - let mut command = VmmCommand::new( - env::current_exe().unwrap(), - fixture_arguments("vmm_fixture"), - directory, - ); - command.breakaway_from_job = breakaway; - command -} - -/// Spawns a VMM fixture in `directory` as OpenVMM is spawned, outside the caller's job object if -/// `breakaway`. -fn spawn_vmm(directory: &Path, breakaway: bool) -> Launched { - let log = File::create(directory.join("vmm.log")).unwrap(); - supervisor::spawn(&vmm_command(directory, breakaway), &CAPABILITY, log).unwrap() -} - -/// The working directory of a VMM fixture. Dropping it ends the fixture, or a VMM that ignores -/// the exit marker, and waits for it to exit, so that a failed test leaves no process behind. -struct VmmDirectory { - /// The reference of the VMM, once the test knows it. - reference: Option, - /// The directory, which is removed after the VMM exits. - directory: tempfile::TempDir, -} - -impl VmmDirectory { - /// Creates an empty directory. - fn new() -> Self { - Self { - reference: None, - directory: tempfile::tempdir().unwrap(), - } - } - - /// Returns the path of the directory. - fn path(&self) -> &Path { - self.directory.path() - } - - /// Records the reference of the VMM that runs in the directory. - fn record(&mut self, reference: &ProcessRef) { - self.reference = Some(reference.clone()); - } - - /// Asks the VMM to exit. - fn end(&self) { - File::create(self.path().join(EXIT_MARKER)).unwrap(); - } -} - -impl Drop for VmmDirectory { - fn drop(&mut self) { - let _ = File::create(self.path().join(EXIT_MARKER)); - if let Some(reference) = &self.reference - && let Ok(Opened::Running(process)) = Process::open(reference) - && !process - .wait(Instant::now() + TIMEOUT) - .is_ok_and(|exited| exited.is_some()) - { - let _ = process.kill(); - let _ = process.wait(Instant::now() + TIMEOUT); - } - } -} - -/// A launcher fixture, which the test drives through its standard streams. -struct Launcher { - /// The fixture's process. - child: Child, - /// Its standard input, which the test closes to end it. - input: Option, -} - -impl Launcher { - /// Starts a launcher whose VMM runs in `directory`, outside the launcher's job object if - /// `breakaway`. - fn start(directory: &Path, breakaway: bool) -> Self { - let mut command = Command::new(env::current_exe().unwrap()); - command - .args(fixture_arguments("launcher_fixture")) - .env(LAUNCHER_DIR, directory); - if breakaway { - command.env(LAUNCHER_BREAKAWAY, "1"); - } - Self::spawn(command) - } - - /// Starts the fixture that `command` runs, with its standard input and output piped. - fn spawn(mut command: Command) -> Self { - command.stdin(Stdio::piped()).stdout(Stdio::piped()); - let mut child = command.spawn().unwrap(); - let input = child.stdin.take(); - Self { child, input } - } - - /// Asks the fixture to start its VMM, and returns the reference that it prints, as the - /// fixture would record it: as JSON. Returns `None` if the fixture exits without printing - /// one. - fn try_launch(&mut self) -> Option { - let input = self.input.as_mut().unwrap(); - input.write_all(b"launch\n").unwrap(); - input.flush().unwrap(); - let output = self.child.stdout.take().unwrap(); - let (sender, receiver) = mpsc::channel(); - thread::spawn(move || { - let mut lines = BufReader::new(output).lines(); - let reference = lines.by_ref().find_map(|line| { - let line = line.ok()?; - Some(line.split_once(REFERENCE_PREFIX)?.1.to_owned()) - }); - let _ = sender.send(reference); - // Read the rest, so that the fixture can still write its own output. - lines.for_each(drop); - }); - receiver - .recv_timeout(TIMEOUT) - .expect("the fixture printed no reference in time") - } - - /// Asks the fixture to start its VMM, and returns the reference that it prints, as the - /// fixture would record it: as JSON. - fn launch(&mut self) -> String { - self.try_launch() - .expect("the fixture exited before it printed a reference") - } - - /// Asks the fixture to start its VMM, and returns the reference that it prints. - fn launch_reference(&mut self) -> ProcessRef { - serde_json::from_str(&self.launch()).unwrap() - } - - /// Waits for the fixture to exit, and returns its status. - fn wait(&mut self) -> ExitStatus { - let deadline = Instant::now() + TIMEOUT; - loop { - if let Some(status) = self.child.try_wait().unwrap() { - return status; - } - assert!(Instant::now() < deadline, "the fixture did not exit"); - thread::sleep(Duration::from_millis(10)); - } - } - - /// Closes the fixture's input, which ends it, and waits for it to exit. - fn finish(&mut self) { - drop(self.input.take()); - assert!(self.wait().success()); - } -} - -impl Drop for Launcher { - fn drop(&mut self) { - let _ = self.child.kill(); - let _ = self.child.wait(); - } -} - -/// Opens the process that `reference` names, which must still run. -fn open_running(reference: &ProcessRef) -> Process { - let Opened::Running(process) = Process::open(reference).unwrap() else { - panic!("{reference:?} does not run"); - }; - process -} - -/// Checks that the process that `process` holds has exited, and that no process matches its -/// reference. -fn assert_ended(process: &Process) { - assert!(process.has_exited().unwrap()); - assert!(matches!( - Process::open(process.reference()).unwrap(), - Opened::Exited(_) - )); -} - -#[test] -fn processes_can_be_shared_between_threads() { - /// Compiles only for types that threads can share. - fn shared() {} - shared::(); - shared::(); -} - -#[test] -#[cfg(windows)] -fn working_directory_limits_include_trailing_separators() { - use std::os::windows::ffi::OsStrExt; - - let _spawns = serialized(); - let root = tempfile::tempdir().unwrap(); - for (units, trailing, accepted) in [ - (258, false, true), - (259, false, false), - (257, true, true), - (258, true, false), - ] { - let name = "w".repeat(units - root.path().as_os_str().encode_wide().count() - 1); - let directory = root.path().join(name); - fs::create_dir_all(&directory).unwrap(); - let working = if trailing { - directory.join("") - } else { - directory.clone() - }; - let log = File::create(directory.join(format!("vmm-{units}-{trailing}.log"))).unwrap(); - let launched = supervisor::spawn(&vmm_command(&working, false), &CAPABILITY, log); - if accepted { - assert!(launched.unwrap().kill_and_wait()); - } else { - let error = launched - .map(|launched| assert!(launched.kill_and_wait())) - .unwrap_err(); - assert_eq!(error.kind(), io::ErrorKind::InvalidInput); - assert!(error.to_string().contains("258")); - } - } -} - -#[test] -fn a_child_that_exits_wakes_its_waiter() { - let _spawns = serialized(); - let mut vmm = VmmDirectory::new(); - let launched = spawn_vmm(vmm.path(), false); - let process = launched.process().unwrap(); - vmm.record(process.reference()); - assert_eq!(process.reference().pid, launched.id()); - assert_eq!(process.reference().boot, BootId::current().unwrap()); - assert_eq!(process.wait_method(), WaitMethod::Event); - assert!(!process.has_exited().unwrap()); - assert!(process.wait(Instant::now() + SHORT_WAIT).unwrap().is_none()); - // The reference finds the same process through a handle of its own. - let found = open_running(process.reference()); - assert_eq!(found.reference(), process.reference()); - - let started = Instant::now(); - vmm.end(); - assert!(process.wait(started + TIMEOUT).unwrap().is_some()); - assert!(found.wait(started + TIMEOUT).unwrap().is_some()); - assert!(started.elapsed() < TIMEOUT); - assert_ended(&process); - // Ending a process that exited does nothing. - process.kill().unwrap(); - assert!(launched.kill_and_wait()); -} - -#[test] -fn a_process_that_is_not_a_child_wakes_its_waiters() { - let _spawns = serialized(); - let mut vmm = VmmDirectory::new(); - let mut launcher = Launcher::start(vmm.path(), false); - let reference = launcher.launch_reference(); - vmm.record(&reference); - // The launcher exits, so its VMM is no child of this process. - launcher.finish(); - let process = open_running(&reference); - assert_eq!(process.wait_method(), WaitMethod::Event); - assert!(process.wait(Instant::now() + SHORT_WAIT).unwrap().is_none()); - - let started = Instant::now(); - let waiters: Vec = thread::scope(|scope| { - let waiters: Vec<_> = (0..2) - .map(|_| scope.spawn(|| process.wait(started + TIMEOUT).unwrap().is_some())) - .collect(); - vmm.end(); - waiters - .into_iter() - .map(|waiter| waiter.join().unwrap()) - .collect() - }); - assert_eq!(waiters, [true, true]); - assert!(started.elapsed() < TIMEOUT); - assert_ended(&process); -} - -#[test] -fn a_restarted_launcher_finds_its_vmm_again_and_ends_it() { - let _spawns = serialized(); - let mut vmm = VmmDirectory::new(); - // On Windows, the launcher runs in a job object that terminates it when the job closes, and - // its VMM breaks away from the job, as an orchestrator's VMMs do to survive it. - #[cfg(windows)] - let job = job::Job::new(true); - let mut launcher = Launcher::start(vmm.path(), cfg!(windows)); - #[cfg(windows)] - job.assign(&launcher.child); - let recorded = launcher.launch(); - // The launcher crashes. - #[cfg(windows)] - drop(job); - #[cfg(not(windows))] - launcher.child.kill().unwrap(); - launcher.wait(); - - // Its successor reads the recorded reference back, and finds the VMM, which survived. - let reference: ProcessRef = serde_json::from_str(&recorded).unwrap(); - vmm.record(&reference); - let process = open_running(&reference); - assert_eq!(process.wait_method(), WaitMethod::Event); - assert!(process.wait(Instant::now() + SHORT_WAIT).unwrap().is_none()); - let started = Instant::now(); - process.kill().unwrap(); - assert!(process.wait(started + TIMEOUT).unwrap().is_some()); - assert_ended(&process); -} - -#[test] -fn a_reused_process_id_is_never_signaled() { - let _spawns = serialized(); - let mut vmm = VmmDirectory::new(); - let launched = spawn_vmm(vmm.path(), false); - let process = launched.process().unwrap(); - let reference = process.reference().clone(); - vmm.record(&reference); - let mut other_boot = *reference.boot.as_bytes(); - other_boot[15] ^= 1; - // A process with the same ID and another start time is not the referenced one, and neither - // is one of another boot: the referenced process exited, and its ID now names another. - for impostor in [ - ProcessRef { - start_time: reference.start_time + 1, - ..reference.clone() - }, - ProcessRef { - start_time: reference.start_time - 1, - ..reference.clone() - }, - ProcessRef { - boot: BootId::from_bytes(other_boot), - ..reference.clone() - }, - ] { - assert!( - matches!(Process::open(&impostor).unwrap(), Opened::Exited(_)), - "{impostor:?} names a running process" - ); - } - - // The process that has the ID still runs. - assert!(!process.has_exited().unwrap()); - open_running(&reference); - vmm.end(); - assert!(process.wait(Instant::now() + TIMEOUT).unwrap().is_some()); - assert!(launched.kill_and_wait()); -} - -#[test] -fn references_that_name_no_process_are_never_opened() { - let _spawns = serialized(); - let boot = BootId::current().unwrap(); - let invalid = ProcessRef { - pid: 0, - start_time: 1, - boot, - }; - let error = Process::open(&invalid).unwrap_err(); - assert_eq!(error.kind(), io::ErrorKind::InvalidInput); - // Linux assigns no process ID beyond `i32::MAX`. - #[cfg(target_os = "linux")] - { - let invalid = ProcessRef { - pid: u32::MAX, - ..invalid - }; - let error = Process::open(&invalid).unwrap_err(); - assert_eq!(error.kind(), io::ErrorKind::InvalidInput); - } - - // A process that exited and was reaped: no process has its ID, or another one has it. - let vmm = VmmDirectory::new(); - let launched = spawn_vmm(vmm.path(), false); - let process = launched.process().unwrap(); - vmm.end(); - assert!(process.wait(Instant::now() + TIMEOUT).unwrap().is_some()); - assert!(launched.kill_and_wait()); - assert!(matches!( - Process::open(process.reference()).unwrap(), - Opened::Exited(_) - )); - assert!(process.has_exited().unwrap()); -} - -/// Sets its flag when dropped, also when its thread panics. -struct SetOnDrop<'a>(&'a AtomicBool); - -impl Drop for SetOnDrop<'_> { - fn drop(&mut self) { - self.0.store(true, Ordering::SeqCst); - } -} - -#[test] -fn vmms_inherit_only_their_streams_and_allowed_descriptors_while_others_spawn() { - let _spawns = serialized(); - /// The VMMs that the test launches. - const VMMS: usize = 6; - let (readers, writers): (Vec, Vec) = - (0..VMMS).map(|_| io::pipe().unwrap()).unzip(); - // Only the VMM that inherits one of these pipes may hold it. - #[cfg(target_os = "linux")] - let forbidden: Vec = writers - .iter() - .map(|writer| this_object(number_of(writer))) - .collect(); - #[cfg(windows)] - let forbidden: Vec = Vec::new(); - let done = AtomicBool::new(false); - let rounds = AtomicUsize::new(0); - let others = Mutex::new(Vec::new()); - let leaks = thread::scope(|scope| { - let stop_spawner = SetOnDrop(&done); - let spawner = scope.spawn(|| spawn_others(&done, &rounds, &others, &forbidden)); - // The other thread spawns throughout the launches of the VMMs. - let deadline = Instant::now() + TIMEOUT; - while rounds.load(Ordering::SeqCst) == 0 { - assert!( - Instant::now() < deadline && !spawner.is_finished(), - "the other thread spawned nothing" - ); - thread::sleep(Duration::from_millis(10)); - } - for (reader, writer) in readers.into_iter().zip(&writers) { - check_vmm(reader, writer, &others); - } - drop(stop_spawner); - spawner.join().unwrap() - }); - assert!(rounds.load(Ordering::SeqCst) > 0); - assert!(leaks.is_empty(), "{leaks:?}"); -} - -/// Spawns processes until `done`, as other threads of an orchestrator do, and counts its rounds -/// in `rounds`. Each round spawns a VMM fixture through the supervisor, which inherits a pipe of -/// its own, and a process through `std::process`, which inherits another pipe, which the round -/// makes inheritable for the spawn, as code that spawns without an allowlist does. -/// -/// It lists the numbers of those pipes in `others`, and keeps the pipes open until it returns. -/// It returns the processes that it spawned through `std::process` that held a descriptor of any -/// of the objects `forbidden`. -fn spawn_others( - done: &AtomicBool, - rounds: &AtomicUsize, - others: &Mutex>, - forbidden: &[PathBuf], -) -> Vec { - // Without the VMM marker, the fixtures that it spawns return at once. - let directory = tempfile::tempdir().unwrap(); - let mut pipes = Vec::new(); - let mut leaks = Vec::new(); - while !done.load(Ordering::SeqCst) { - let (_, allowed) = io::pipe().unwrap(); - let (_, naive) = io::pipe().unwrap(); - others - .lock() - .unwrap() - .extend([number_of(&allowed), number_of(&naive)]); - - let mut command = VmmCommand::new( - env::current_exe().unwrap(), - fixture_arguments("vmm_fixture"), - directory.path(), - ); - command.inherit.push(Inherited::new(&allowed)); - let round = rounds.load(Ordering::SeqCst); - let log = File::create(directory.path().join(format!("{round}.log"))).unwrap(); - assert!( - supervisor::spawn(&command, &CAPABILITY, log) - .unwrap() - .kill_and_wait() - ); - - set_inheritable(&naive, true); - let child = sleeper().spawn(); - set_inheritable(&naive, false); - let mut child = child.unwrap(); - leaks.extend( - held_objects(child.id(), forbidden) - .into_iter() - .map(|object| format!("process {} held {}", child.id(), object.display())), - ); - let _ = child.kill(); - child.wait().unwrap(); - pipes.extend([allowed, naive]); - rounds.fetch_add(1, Ordering::SeqCst); - } - leaks -} - -/// Spawns a VMM fixture that inherits `writer` while another thread spawns processes whose pipes -/// `others` lists, and checks that the VMM holds its standard streams and `writer`, under its -/// number, and nothing else of this process's. -fn check_vmm(reader: PipeReader, writer: &PipeWriter, others: &Mutex>) { - let mut vmm = VmmDirectory::new(); - let number = number_of(writer); - let mut command = vmm_command(vmm.path(), false); - command - .arguments - .push(format!("{INHERITED_PREFIX}{number}").into()); - command.inherit.push(Inherited::new(writer)); - let log = File::create(vmm.path().join("vmm.log")).unwrap(); - let launched = supervisor::spawn(&command, &CAPABILITY, log).unwrap(); - let process = launched.process().unwrap(); - vmm.record(process.reference()); - // The VMM writes through its pipe, under the number that its arguments name. - assert_eq!(read_line(reader), format!("inherited {number}\n")); - #[cfg(target_os = "linux")] - { - let _ = others; - let reported = reported_descriptors(vmm.path()); - let numbers: Vec = reported.iter().map(|(number, _)| *number).collect(); - assert_eq!(numbers, [0, 1, 2, number], "{reported:?}"); - let log = fs::canonicalize(vmm.path().join("vmm.log")).unwrap(); - assert!( - reported[0].1.to_string_lossy().starts_with("pipe:"), - "{reported:?}" - ); - assert_eq!(reported[1].1, log); - assert_eq!(reported[2].1, log); - assert_eq!(reported[3].1, this_object(number)); - } - #[cfg(windows)] - { - let pid = launched.id(); - assert!(holds(pid, number), "the VMM did not inherit its pipe"); - for &other in others.lock().unwrap().iter() { - assert!( - !holds(pid, other), - "the VMM inherited handle {other} of another spawn" - ); - } - } - vmm.end(); - assert!(process.wait(Instant::now() + TIMEOUT).unwrap().is_some()); - assert!(launched.kill_and_wait()); -} - -/// Reads a line from `reader` within [`TIMEOUT`]. -fn read_line(reader: PipeReader) -> String { - let (sender, receiver) = mpsc::channel(); - thread::spawn(move || { - let mut line = String::new(); - let _ = BufReader::new(reader).read_line(&mut line); - let _ = sender.send(line); - }); - receiver - .recv_timeout(TIMEOUT) - .expect("the VMM wrote no line in time") -} - -/// Returns the number of the descriptor of `pipe`, which a VMM inherits under that number. -#[cfg(target_os = "linux")] -fn number_of(pipe: &PipeWriter) -> usize { - use std::os::fd::AsRawFd; - - usize::try_from(pipe.as_raw_fd()).unwrap() -} - -/// Returns the value of the handle of `pipe`, which a VMM inherits under that value. -#[cfg(windows)] -fn number_of(pipe: &PipeWriter) -> usize { - use std::os::windows::io::AsRawHandle; - - pipe.as_raw_handle() as usize -} - -/// Returns the object that the descriptor `number` of this process holds, as its link in `/proc` -/// names it. -#[cfg(target_os = "linux")] -fn this_object(number: usize) -> PathBuf { - fs::read_link(format!("/proc/self/fd/{number}")).unwrap() -} - -/// Waits for the VMM fixture in `directory` to report its descriptors, and returns them, with the -/// objects that they hold. -#[cfg(target_os = "linux")] -fn reported_descriptors(directory: &Path) -> Vec<(usize, PathBuf)> { - let path = directory.join(DESCRIPTORS_REPORT); - let deadline = Instant::now() + TIMEOUT; - let report = loop { - match fs::read_to_string(&path) { - Ok(report) => break report, - Err(error) if error.kind() == io::ErrorKind::NotFound && Instant::now() < deadline => { - thread::sleep(Duration::from_millis(10)); - } - Err(error) => panic!("the VMM reported no descriptors: {error}"), - } - }; - report - .lines() - .map(|line| { - let (number, object) = line.split_once(' ').unwrap(); - (number.parse().unwrap(), PathBuf::from(object)) - }) - .collect() -} - -/// Returns the objects of `forbidden` that process `pid` holds a descriptor of. -#[cfg(target_os = "linux")] -fn held_objects(pid: u32, forbidden: &[PathBuf]) -> Vec { - fs::read_dir(format!("/proc/{pid}/fd")) - .unwrap() - .filter_map(|entry| fs::read_link(entry.unwrap().path()).ok()) - .filter(|object| forbidden.contains(object)) - .collect() -} - -/// Returns the objects of `forbidden` that process `pid` holds a handle of: none, since a process -/// that another thread creates while a VMM is created can inherit the VMM's handles on Windows, as -/// [`Inherited`] documents, so no object is forbidden there. -#[cfg(windows)] -fn held_objects(_pid: u32, forbidden: &[PathBuf]) -> Vec { - assert!(forbidden.is_empty()); - Vec::new() -} - -/// Returns whether the handle `value` of process `pid` refers to the object of this process's -/// handle `value`, as a handle that the process inherited from this one does. -#[cfg(windows)] -fn holds(pid: u32, value: usize) -> bool { - use std::os::windows::io::{FromRawHandle, OwnedHandle}; - use std::ptr; - - use windows_sys::Win32::Foundation::{ - CompareObjectHandles, DUPLICATE_SAME_ACCESS, DuplicateHandle, HANDLE, - }; - use windows_sys::Win32::System::Threading::{ - GetCurrentProcess, OpenProcess, PROCESS_DUP_HANDLE, - }; - - let ours = value as HANDLE; - // SAFETY: OpenProcess has no memory-safety preconditions. - let process = unsafe { OpenProcess(PROCESS_DUP_HANDLE, 0, pid) }; - assert!( - !process.is_null(), - "cannot open process {pid}: {}", - io::Error::last_os_error() - ); - // SAFETY: the handle is new, and nothing else owns it. - let process = unsafe { OwnedHandle::from_raw_handle(process) }; - let mut copy = ptr::null_mut(); - // SAFETY: the process handle may duplicate the process's handles, the value is looked up in - // that process only, and the output pointer is valid. - let duplicated = unsafe { - DuplicateHandle( - std::os::windows::io::AsRawHandle::as_raw_handle(&process), - ours, - GetCurrentProcess(), - &mut copy, - 0, - 0, - DUPLICATE_SAME_ACCESS, - ) - } != 0; - // A value that names no handle of the process names none of ours. - if !duplicated { - return false; - } - // SAFETY: the copy is new, and nothing else owns it. - let copy = unsafe { OwnedHandle::from_raw_handle(copy) }; - // SAFETY: both handles are open. - unsafe { - CompareObjectHandles( - std::os::windows::io::AsRawHandle::as_raw_handle(©), - ours, - ) != 0 - } -} - -/// Makes `pipe` inheritable, or not, as code that spawns without an allowlist does around its -/// spawns. -#[cfg(target_os = "linux")] -fn set_inheritable(pipe: &PipeWriter, inheritable: bool) { - use nix::fcntl::{FcntlArg, FdFlag, fcntl}; - - let flags = if inheritable { - FdFlag::empty() - } else { - FdFlag::FD_CLOEXEC - }; - fcntl(pipe, FcntlArg::F_SETFD(flags)).unwrap(); -} - -/// Makes `pipe` inheritable, or not, as code that spawns without an allowlist does around its -/// spawns. -#[cfg(windows)] -fn set_inheritable(pipe: &PipeWriter, inheritable: bool) { - use std::os::windows::io::AsRawHandle; - - use windows_sys::Win32::Foundation::{HANDLE, HANDLE_FLAG_INHERIT, SetHandleInformation}; - - let flags = if inheritable { HANDLE_FLAG_INHERIT } else { 0 }; - // SAFETY: the handle is open. - let set = - unsafe { SetHandleInformation(pipe.as_raw_handle() as HANDLE, HANDLE_FLAG_INHERIT, flags) }; - assert_ne!(set, 0, "{}", io::Error::last_os_error()); -} - -/// Returns a command that runs for a few seconds, without standard streams. -fn sleeper() -> Command { - let mut command = if cfg!(windows) { - let mut command = Command::new("ping"); - command.args(["-n", "5", "127.0.0.1"]); - command - } else { - let mut command = Command::new("sleep"); - command.arg("5"); - command - }; - command - .stdin(Stdio::null()) - .stdout(Stdio::null()) - .stderr(Stdio::null()); - command -} - -#[cfg(windows)] -#[test] -fn a_job_that_ends_its_vmm_wakes_the_waiters() { - let _spawns = serialized(); - let mut vmm = VmmDirectory::new(); - let job = job::Job::new(false); - let mut launcher = Launcher::start(vmm.path(), false); - job.assign(&launcher.child); - let reference = launcher.launch_reference(); - vmm.record(&reference); - // The VMM did not break away, so the job holds it too, and closing the job ends it. - let process = open_running(&reference); - assert!(process.wait(Instant::now() + SHORT_WAIT).unwrap().is_none()); - let started = Instant::now(); - drop(job); - assert!(process.wait(started + TIMEOUT).unwrap().is_some()); - assert_ended(&process); - launcher.wait(); -} - -#[cfg(windows)] -#[test] -fn a_process_that_this_user_may_not_query_has_an_unknown_state() { - use windows_sys::Win32::System::Threading::PROCESS_QUERY_LIMITED_INFORMATION; - - let _spawns = serialized(); - let (mut child, reference) = restricted::spawn(0); - if restricted::may_open(&child.0, PROCESS_QUERY_LIMITED_INFORMATION) { - eprintln!("skipping: this user's privileges bypass the security descriptors of processes"); - return; - } - // Its identity cannot be checked, so it may still run. - let error = Process::open(&reference).unwrap_err(); - assert_eq!(error.kind(), io::ErrorKind::PermissionDenied); - assert!( - child.0.try_wait().unwrap().is_none(), - "the process was ended" - ); -} - -#[cfg(windows)] -#[test] -fn a_process_that_this_user_may_not_terminate_is_waited_for_but_never_signaled() { - use windows_sys::Win32::System::Threading::{ - PROCESS_QUERY_LIMITED_INFORMATION, PROCESS_SYNCHRONIZE, PROCESS_TERMINATE, - }; - - let _spawns = serialized(); - let (mut child, reference) = - restricted::spawn(PROCESS_QUERY_LIMITED_INFORMATION | PROCESS_SYNCHRONIZE); - if restricted::may_open(&child.0, PROCESS_TERMINATE) { - eprintln!("skipping: this user's privileges bypass the security descriptors of processes"); - return; - } - let process = open_running(&reference); - assert_eq!(process.wait_method(), WaitMethod::Event); - let error = process.kill().unwrap_err(); - assert_eq!(error.kind(), io::ErrorKind::PermissionDenied); - assert!(!process.has_exited().unwrap()); - assert!( - child.0.try_wait().unwrap().is_none(), - "the process was ended" - ); - - // Its creator ends it, which wakes the waiter, and ending a process that exited does nothing. - child.0.kill().unwrap(); - assert!(process.wait(Instant::now() + TIMEOUT).unwrap().is_some()); - process.kill().unwrap(); - assert_ended(&process); -} - -/// Processes whose security descriptor restricts what this user may do with them. -#[cfg(windows)] -mod restricted { - use std::os::windows::io::AsRawHandle; - use std::process::{Child, Command, Stdio}; - use std::{io, mem, ptr}; - - use nvx::host::platform; - use nvx::host::process::{BootId, ProcessRef}; - use windows_sys::Win32::Foundation::{CloseHandle, ERROR_SUCCESS}; - use windows_sys::Win32::Security::Authorization::{SE_KERNEL_OBJECT, SetSecurityInfo}; - use windows_sys::Win32::Security::{ - ACCESS_ALLOWED_ACE, ACL, ACL_REVISION, AddAccessAllowedAce, CreateWellKnownSid, - DACL_SECURITY_INFORMATION, InitializeAcl, PROTECTED_DACL_SECURITY_INFORMATION, - SECURITY_MAX_SID_SIZE, WinWorldSid, - }; - use windows_sys::Win32::System::Threading::OpenProcess; - - /// A child process, which is ended when dropped, through the handle with full access that - /// its creator holds. - pub(crate) struct Running(pub(crate) Child); - - impl Drop for Running { - fn drop(&mut self) { - let _ = self.0.kill(); - let _ = self.0.wait(); - } - } - - /// Starts a process that runs for a minute unless it is ended, and whose security - /// descriptor then grants everyone, this user included, only `access` to it. Returns it with - /// its reference. - pub(crate) fn spawn(access: u32) -> (Running, ProcessRef) { - let child = Command::new("ping") - .args(["-n", "60", "127.0.0.1"]) - .stdin(Stdio::null()) - .stdout(Stdio::null()) - .spawn() - .unwrap(); - let child = Running(child); - let pid = child.0.id(); - let reference = ProcessRef { - pid, - start_time: platform::process_start_time(pid).unwrap().unwrap(), - boot: BootId::current().unwrap(), - }; - restrict(&child.0, access); - (child, reference) - } - - /// Replaces the DACL of `child` with one that grants everyone `access`, or no access when - /// `access` is 0. - fn restrict(child: &Child, access: u32) { - let mut sid = [0u8; SECURITY_MAX_SID_SIZE as usize]; - let mut sid_size = SECURITY_MAX_SID_SIZE; - // SAFETY: the buffer holds `sid_size` bytes. - let created = unsafe { - CreateWellKnownSid( - WinWorldSid, - ptr::null_mut(), - sid.as_mut_ptr().cast(), - &mut sid_size, - ) - }; - assert_ne!(created, 0, "{}", io::Error::last_os_error()); - // Room for the ACL header, one ACE, and its SID, in 4-byte units, which align the ACL. - let size = mem::size_of::() - + mem::size_of::() - + usize::try_from(sid_size).unwrap(); - let mut buffer = vec![0u32; size.div_ceil(4)]; - let acl: *mut ACL = buffer.as_mut_ptr().cast(); - let length = u32::try_from(buffer.len() * 4).unwrap(); - // SAFETY: the buffer holds `length` bytes, aligned for an ACL. - let initialized = unsafe { InitializeAcl(acl, length, ACL_REVISION) }; - assert_ne!(initialized, 0, "{}", io::Error::last_os_error()); - if access != 0 { - // SAFETY: the ACL has room for the ACE, and the SID is valid. - let added = - unsafe { AddAccessAllowedAce(acl, ACL_REVISION, access, sid.as_mut_ptr().cast()) }; - assert_ne!(added, 0, "{}", io::Error::last_os_error()); - } - // SAFETY: the creator's handle has WRITE_DAC access, and the ACL is valid. - let status = unsafe { - SetSecurityInfo( - child.as_raw_handle(), - SE_KERNEL_OBJECT, - DACL_SECURITY_INFORMATION | PROTECTED_DACL_SECURITY_INFORMATION, - ptr::null_mut(), - ptr::null_mut(), - acl, - ptr::null(), - ) - }; - assert_eq!( - status, - ERROR_SUCCESS, - "cannot restrict a process: {}", - io::Error::from_raw_os_error(status as i32) - ); - } - - /// Returns whether this user may still open `child` with `access`, as a user may whose - /// privileges bypass security descriptors. - pub(crate) fn may_open(child: &Child, access: u32) -> bool { - // SAFETY: OpenProcess has no memory-safety preconditions. - let handle = unsafe { OpenProcess(access, 0, child.id()) }; - if handle.is_null() { - return false; - } - // SAFETY: the handle is open, and nothing else owns it. - unsafe { CloseHandle(handle) }; - true - } -} - -/// Job objects, which Windows orchestrators run in. -#[cfg(windows)] -mod job { - use std::os::windows::io::{AsRawHandle, FromRawHandle, OwnedHandle}; - use std::process::Child; - use std::{io, mem, ptr}; - - use windows_sys::Win32::System::JobObjects::{ - AssignProcessToJobObject, CreateJobObjectW, JOB_OBJECT_LIMIT_BREAKAWAY_OK, - JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE, JOBOBJECT_EXTENDED_LIMIT_INFORMATION, - JobObjectExtendedLimitInformation, SetInformationJobObject, - }; - - /// A job object that terminates its processes when the test closes it. - pub(crate) struct Job(OwnedHandle); - - impl Job { - /// Creates a job that terminates its processes when it closes, and that lets them start - /// processes outside it if `breakaway`. - pub(crate) fn new(breakaway: bool) -> Self { - // SAFETY: creates an unnamed job object with default security. - let handle = unsafe { CreateJobObjectW(ptr::null(), ptr::null()) }; - assert!( - !handle.is_null(), - "cannot create a job object: {}", - io::Error::last_os_error() - ); - // SAFETY: the handle is new, and nothing else owns it. - let job = Self(unsafe { OwnedHandle::from_raw_handle(handle) }); - // SAFETY: the structure is plain data for which all-zero bytes are valid. - let mut limits: JOBOBJECT_EXTENDED_LIMIT_INFORMATION = unsafe { mem::zeroed() }; - limits.BasicLimitInformation.LimitFlags = JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE - | if breakaway { - JOB_OBJECT_LIMIT_BREAKAWAY_OK - } else { - 0 - }; - // SAFETY: the handle is a job object, and the information has the size given. - let set = unsafe { - SetInformationJobObject( - handle, - JobObjectExtendedLimitInformation, - (&raw const limits).cast(), - mem::size_of_val(&limits) as u32, - ) - }; - assert_ne!( - set, - 0, - "cannot set the limits of a job object: {}", - io::Error::last_os_error() - ); - job - } - - /// Places `child` in the job, before it starts any process of its own. - pub(crate) fn assign(&self, child: &Child) { - // SAFETY: both handles are open for the duration of the call. - let assigned = - unsafe { AssignProcessToJobObject(self.0.as_raw_handle(), child.as_raw_handle()) }; - assert_ne!( - assigned, - 0, - "cannot place the launcher in a job object: {}", - io::Error::last_os_error() - ); - } - } -} diff --git a/nvx-rs/tests/vectors/README.md b/nvx-rs/tests/vectors/README.md index 547866e3d..a3271f930 100644 --- a/nvx-rs/tests/vectors/README.md +++ b/nvx-rs/tests/vectors/README.md @@ -12,7 +12,7 @@ yet breaks those peers. Only bytes that no codec produced catch such a change. | [`nvxc-v1.txt`](nvxc-v1.txt) | Version 1 of the control protocol: `NVXC` frames, their `EXEC`, `MAPS`, and features payloads, and `NVXS` records of NVX's host and guest | | [`nvxl-v1.txt`](nvxl-v1.txt) | Version 1 of the `NVXL` listener announcement | | [`activation-v2.txt`](activation-v2.txt) | Version 2 of the activation record of a layered root, and what Ready reports about an activation | -| [`snapshot-key-v1.txt`](snapshot-key-v1.txt) | Version 1 of the canonical encodings of a snapshot's compatibility key and of a capture class, with their identities | +| [`snapshot-key-v1.txt`](snapshot-key-v1.txt) | Version 1 of the canonical encoding of a snapshot's compatibility key, with its identity | | [`openvmm/`](openvmm) | OpenVMM's vectors and cases of the `NVXS` session, and the excerpts of its sources that define the kernel command-line tokens of its platform tier and the error of a port forward that it cannot bind, unchanged | [4.4 Host-Guest Control Interface](../../../doc/design/4.4-host-guest-control-interface.md) @@ -49,9 +49,9 @@ A decoder must decode the bytes of each vector to its values, and an encoder must encode the values to exactly its bytes. A kind whose name starts with `INVALID_` has no values: every decoder must refuse the bytes of its vectors. -A key or a class holds too many values for one line, so the value of each -vector of `snapshot-key-v1.txt` is its identity, and the file's header names -the captures whose keys and classes the vectors encode. The tests of +A key holds too many values for one line, so the value of each vector of +`snapshot-key-v1.txt` is its identity, and the file's header names the +captures whose keys the vectors encode. The tests of `nvx::host::snapshot` derive each key from its capture, and check that it is the key that the vector's bytes decode to. @@ -61,22 +61,18 @@ the key that the vector's bytes decode to. its own, such as `nvxc-v2.txt` for the exec v2 of [microsoft/nvx#623](https://github.com/microsoft/nvx/issues/623), and the bytes of an earlier version that it refuses become its rejection vectors. - Version 1 has three exceptions. `MAPS|ROOT_SOURCE` of `nvxc-v1.txt` was the + Version 1 has two exceptions. `MAPS|ROOT_SOURCE` of `nvxc-v1.txt` was the rejection vector `INVALID_MAPS|EMPTY_SOURCE` until an empty source came to name the export's root, for a host whose export is one directory. NVX's hosts, whose exports are aggregates, send none, and a host whose export is one directory sends one only to a guest that deploys with it, so the C agent - of earlier releases, which refuses it, never receives one. And - `CLASS|NETWORK` of - `snapshot-key-v1.txt` came to encode its network in place as the template - captures it, under the complete deny-all policy: the encodings of a key and - of a class have not been released, so no key or class that an earlier build - encoded is supported, since nothing migrates it. The snapshot key also drops - the obsolete guest-build field and uses the `nvx snapshot key` hash domain in - place, and now appends the full - capture-time contract while its compatibility identity still digests only - the preceding compatibility fields: no snapshot outlives a deployment, - and the prior key encoding is not supported. + of earlier releases, which refuses it, never receives one. And the snapshot + key of `snapshot-key-v1.txt` drops the obsolete guest-build field, uses the + `nvx snapshot key` hash domain in place, and appends the full capture-time + contract, while its compatibility identity still digests only the preceding + compatibility fields: its encoding has not been released, and no snapshot + outlives a deployment, so no key that an earlier build encoded is supported, + since nothing migrates it. - The bytes come from the protocol's definition, never from the code that they check, and so do the values of each vector. diff --git a/nvx-rs/tests/vectors/snapshot-key-v1.txt b/nvx-rs/tests/vectors/snapshot-key-v1.txt index 46fef1809..0bd96f4a2 100644 --- a/nvx-rs/tests/vectors/snapshot-key-v1.txt +++ b/nvx-rs/tests/vectors/snapshot-key-v1.txt @@ -1,21 +1,17 @@ -# Conformance vectors of version 1 of the canonical encodings of a snapshot's compatibility key, -# nvx::host::snapshot::SnapshotKey, and of a capture class, nvx::host::snapshot::CaptureClass, -# which the module nvx::host::snapshot describes. README.md describes the line format and the +# Conformance vectors of version 1 of the canonical encoding of a snapshot's compatibility key, +# nvx::host::snapshot::SnapshotKey, which the module nvx::host::snapshot describes. README.md describes the line format and the # rules that the vectors follow. # # KEY|NAME|BYTES|IDENTITY # The canonical persisted key, and its compatibility identity: the SHA-256 digest of # "nvx snapshot key", a NUL byte, and the compatibility prefix before the capture-time fields. # A decoder must decode the full key, and an encoder must reproduce its bytes. -# CLASS|NAME|BYTES|IDENTITY -# The canonical encoding of a class, and its identity: the SHA-256 digest of "nvx capture class", -# a NUL byte, and the encoding. # INVALID_KEY_VERSION|NAME|BYTES # A key of another version of the encoding, which every decoder refuses as unsupported. # INVALID_KEY|NAME|BYTES # Bytes that every decoder of a key must refuse. # -# The keys and classes are those of the captures that tests/fixtures/template/README.md +# The keys are those of the captures that tests/fixtures/template/README.md # describes, as microVM ABI 2 captured them, when every virtio device but the NIC also offered # access-platform (feature bit 33), each console also offered its size (feature bit 0), and the # virtio devices interrupted on ISA IRQs (the NIC on 10, the filesystem on 6, the consoles on 7 @@ -24,7 +20,6 @@ # sha256:c..., boot contract 3, control contract # nvx-microvm-v2-control-v2, control session 1, snapshot format 6, device-state schema 1, time # ABI 1, restore packet 4, and activation contract 1. -# Capture classes alone also hold the guest build guest-build-1. # TEMPLATE, NETWORK and HOST_PROFILE append capture TSC 12345678, UTC 1700000000000000000 ns, # monotonic 123000000000 ns, clock linux-boottime, host and boot IDs of 16 bytes of 01 and 02, # lineage 3 and cut-to-anchor Some(200). MINIMAL uses zero anchors, zero host/boot IDs, lineage 0 @@ -44,13 +39,6 @@ KEY|HOST_PROFILE|0100470000007368613235363a6161616161616161616161616161616161616 # A machine of one processor and one device, and no NIC, which the invalid vectors change. KEY|MINIMAL|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d7632010000000000000006000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d00000000000000000000000000000000000000000000000000000000000e0000006c696e75782d626f6f7474696d6500000000000000000000000000000000000000000000000000000000000000000000000000|9adf5b7b4c5509968914b0cae4664263c768bab0e8e010616b562b0023331d64 -# The class of the capture of TEMPLATE: the CPU profile auto, and 2 processors, all online at -# boot. -CLASS|TEMPLATE|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d040000006175746f02000000020000000000002000000000000000800000000000000040000000000002000000100000000000040000000000020000000200000000000000000000|b53b0169db5d983b31f24bec900cd0b962d26f12480d90803f21ae1d5c3ff334 -# The class of the capture of NETWORK, whose NIC the capture attaches under the complete -# deny-all policy, whatever the policy of the class's network, here one that allows egress. -CLASS|NETWORK|0100470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d763201000000000000000d00000067756573742d6275696c642d3106000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d040000006175746f02000000020000000000002000000000000000800000000000000040000000000002000000100000000000040000000000020000000200000a000000050000002d2d6e65740b00000031302e302e302e322f3234110000002d2d6e6574776f726b2d70726f66696c6508000000706f727461626c65100000002d2d6e6574776f726b2d6567726573730400000064656e79110000002d2d6e6574776f726b2d696e67726573730400000064656e790f0000002d2d686f73742d6c6f6f706261636b0400000064656e79020000001100000069736f6c6174696f6e2e736563636f6d70060000007374726963740b0000006c696d6974732e7069647303000000353132|2f32edae7e9eb532625af600232c68aa78b92e02c8116cb49f857f13ff5fcc7c - # MINIMAL in other versions of the encoding. INVALID_KEY_VERSION|VERSION_0|0000470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d7632010000000000000006000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d00000000000000000000000000000000000000000000000000000000000e0000006c696e75782d626f6f7474696d6500000000000000000000000000000000000000000000000000000000000000000000000000 INVALID_KEY_VERSION|VERSION_2|0200470000007368613235363a61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161470000007368613235363a62626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262470000007368613235363a636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363630300000000000000190000006e76782d6d6963726f766d2d76322d636f6e74726f6c2d7632010000000000000006000000000000000100000000000000010000000000000004000000000000000100000000000000030000006b766d03000000616d640500000067656e6f610100000011111111111111111111111111111111111111111111111111111111111111112c000000000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b01000000010000000100000001000000010000000000000000ca9a3b0000000000e1f505000000000000000001000000000000000000000800000000000000080000000000000008000000000100000000000000000000000000000800000000000000000c000000636f6e736f6c653d687663308c7f813b328f96bdb26c4e97b6f5c874cbd968e8664778addc56a7c81ea55113001000000000000000020000000200000010000000000000000200000002000012000000656467652d7368617265642d737461747573000003000000000000100000000000000100000009000000706172746974696f6e09000000706172746974696f6e09000000706172746974696f6e00000000000000000000000000000000000000000001000000070000006d6963726f766d00000000000000000000000000000000000000000000000000000000000e0000006c696e75782d626f6f7474696d6500000000000000000000000000000000000000000000000000000000000000000000000000 diff --git a/scripts/nvx_tools/microvm_tests.py b/scripts/nvx_tools/microvm_tests.py index 115586abc..ba5144632 100644 --- a/scripts/nvx_tools/microvm_tests.py +++ b/scripts/nvx_tools/microvm_tests.py @@ -444,6 +444,7 @@ class MicrovmVirtioDevice(NamedTuple): SCRATCH_FRESH_VALUE_PREFIX = b"NVX-SCRATCH-FRESH-VALUE-" SCRATCH_FRESH_VALUE_SUFFIX = b"-END" WORKLOAD_IDENTITY_MARKER = b"NVX-WORKLOAD-IDENTITY-OK uid=65534 gid=65534" +OPENVMM_IDENTITY_REFUSAL = b"nvx_identity must name OpenVMM's workload identity" BOOT_MARKER = b"NVX-GUEST-BOOT-OK:" GUEST_BOOT_COMPLETION_MARKER = b"NVX-GUEST-BOOT-CHECK-OK" GUEST_IDENTITY_COMPLETION_MARKER = b"NVX-GUEST-IDENTITY-OK" @@ -1318,15 +1319,23 @@ def run_workload_identity( timeout: float, output_dir: Path, ) -> None: - base = workload_boot_command( - executable, - backend, - kernel, - initrd, - memory_mib, - "quiet loglevel=0 nvx_exec=/sbin/nvx-identity-probe", - ) - accepted = [*base, "--microvm-workload-identity", "65534:65534"] + def command(identity: str, cli_identity: str | None = None) -> list[str]: + # The guest's init reads the identity from the CLI's token, and boots + # only if it names the identity of OpenVMM's option, which OpenVMM + # checks and appends as tokens of its own. + cli_identity = identity if cli_identity is None else cli_identity + tokens = "quiet loglevel=0 nvx_exec=/sbin/nvx-identity-probe" + if cli_identity: + tokens += f" nvx_identity={cli_identity}" + return [ + *workload_boot_command( + executable, backend, kernel, initrd, memory_mib, tokens + ), + "--microvm-workload-identity", + identity, + ] + + accepted = command("65534:65534") with OpenvmmProcess( accepted, output_dir / "workload-identity.log", @@ -1337,7 +1346,7 @@ def run_workload_identity( ): raise RuntimeError("fixed non-root workload identity was not enforced") - unavailable = [*base, "--microvm-workload-identity", "12345:12345"] + unavailable = command("12345:12345") with OpenvmmProcess( unavailable, output_dir / "workload-identity-unavailable.log", @@ -1350,7 +1359,7 @@ def run_workload_identity( ): raise RuntimeError("unavailable workload identity did not fail closed") - root = [*base, "--microvm-workload-identity", "0:0"] + root = command("0:0") with OpenvmmProcess( root, output_dir / "workload-identity-root.log", @@ -1359,6 +1368,29 @@ def run_workload_identity( if result.returncode == 0 or BOOT_MARKER in result.output: raise RuntimeError("root workload identity was not rejected before boot") + # Without the CLI's identity, or with another one, the guest fails closed + # before its workload starts: it neither runs the workload as root nor as + # an identity that OpenVMM's option does not name. + for name, description, refused in ( + ("missing", "no CLI workload identity", command("65534:65534", "")), + ( + "other", + "another CLI workload identity", + command("12345:12345", "65534:65534"), + ), + ): + with OpenvmmProcess( + refused, + output_dir / f"workload-identity-{name}.log", + ) as process: + result = process.wait(timeout) + if ( + result.returncode == 0 + or WORKLOAD_IDENTITY_MARKER in result.output + or OPENVMM_IDENTITY_REFUSAL not in result.output + ): + raise RuntimeError(f"a boot with {description} did not fail closed") + # The ignored nvx-rs test that drives the managed lifecycle of a guest whose # init hands the boot to the managed agent: a unit test of the host's control @@ -1368,6 +1400,10 @@ def run_workload_identity( "host::session::tests::managed_lifecycle::" "a_managed_guest_runs_request_scoped_workloads_until_it_stops" ) +# The CLI's tokens with which the guest's init selects the managed lifecycle +# and the workload identity; OpenVMM's options, which OpenVMM itself checks, +# append tokens that the init ignores. +MANAGED_LIFECYCLE_TOKENS = "nvx_mode=managed nvx_identity=65534:65534" def ignored_nvx_rs_test_command(cargo: str, test: str) -> list[str]: @@ -1451,7 +1487,7 @@ def run_managed_lifecycle( kernel.resolve(), initrd.resolve(), memory_mib, - "quiet loglevel=0", + f"quiet loglevel=0 {MANAGED_LIFECYCLE_TOKENS}", boot_console=False, ) command.extend( @@ -1548,7 +1584,7 @@ def run_managed_lifecycle( kernel, initrd, memory_mib, - "quiet loglevel=0", + f"quiet loglevel=0 {MANAGED_LIFECYCLE_TOKENS}", ) invalid.extend( ( diff --git a/scripts/test_microvm_tests.py b/scripts/test_microvm_tests.py index 5af17c567..03ebf8d5a 100644 --- a/scripts/test_microvm_tests.py +++ b/scripts/test_microvm_tests.py @@ -5760,6 +5760,12 @@ def test_managed_lifecycle_runs_the_nvx_rs_test_and_checks_what_it_leaves(self): ("--microvm-lifecycle", "managed"), ): self.assertEqual(openvmm[openvmm.index(option) + 1], value) + # The guest's init takes the lifecycle and the identity from the CLI's + # tokens, not from those that OpenVMM's options append. + self.assertEqual( + openvmm[openvmm.index("--cmdline") + 1].split()[-2:], + ["nvx_mode=managed", "nvx_identity=65534:65534"], + ) self.assertIn("--microvm-control-auth-stdin", openvmm) self.assertEqual(preserved, self._managed_report()) @@ -7039,20 +7045,30 @@ def _outcome_report( "teardown": {name: True for name in microvm_tests.OUTCOME_TEARDOWN_FIELDS}, } - def test_workload_identity_scenario_checks_enforcement_and_rejection(self): + # The results of the scenario's first three boots, which run the probe as + # its identity, refuse an identity that the guest lacks, and reject root. + WORKLOAD_IDENTITY_CHECKS = ( + openvmm_process.OpenvmmProcessResult( + 0, microvm_tests.WORKLOAD_IDENTITY_MARKER + b"\n" + ), + openvmm_process.OpenvmmProcessResult( + 125, b"configured workload UID is unavailable\n" + ), + openvmm_process.OpenvmmProcessResult( + 2, b"microVM workload UID must be nonzero\n" + ), + ) + WORKLOAD_IDENTITY_REFUSED = openvmm_process.OpenvmmProcessResult( + 125, + b"NVX-WORKLOAD-ERROR: " + microvm_tests.OPENVMM_IDENTITY_REFUSAL + b"\n", + ) + + def _run_workload_identity( + self, results: list[openvmm_process.OpenvmmProcessResult] + ) -> MagicMock: with tempfile.TemporaryDirectory() as temporary: with patch.object(microvm_tests, "OpenvmmProcess") as process: - process.return_value.__enter__.return_value.wait.side_effect = [ - openvmm_process.OpenvmmProcessResult( - 0, microvm_tests.WORKLOAD_IDENTITY_MARKER + b"\n" - ), - openvmm_process.OpenvmmProcessResult( - 125, b"configured workload UID is unavailable\n" - ), - openvmm_process.OpenvmmProcessResult( - 2, b"microVM workload UID must be nonzero\n" - ), - ] + process.return_value.__enter__.return_value.wait.side_effect = results microvm_tests.run_workload_identity( Path("openvmm"), Path("kernel"), @@ -7062,15 +7078,21 @@ def test_workload_identity_scenario_checks_enforcement_and_rejection(self): timeout=40, output_dir=Path(temporary), ) + return process - self.assertEqual(process.call_count, 3) + def test_workload_identity_scenario_checks_enforcement_and_rejection(self): + process = self._run_workload_identity( + [*self.WORKLOAD_IDENTITY_CHECKS, *[self.WORKLOAD_IDENTITY_REFUSED] * 2] + ) + + self.assertEqual(process.call_count, 5) commands = [call.args[0] for call in process.call_args_list] self.assertEqual( [ command[command.index("--microvm-workload-identity") + 1] for command in commands ], - ["65534:65534", "12345:12345", "0:0"], + ["65534:65534", "12345:12345", "0:0", "65534:65534", "12345:12345"], ) self.assertTrue( all( @@ -7079,6 +7101,43 @@ def test_workload_identity_scenario_checks_enforcement_and_rejection(self): for command in commands ) ) + # The guest's init takes the identity from the CLI's token, which + # names the identity of OpenVMM's option, but for the last two boots, + # which must fail closed: one without the CLI's token, and one with + # another identity. + self.assertEqual( + [ + command[command.index("--cmdline") + 1].split()[-1] + for command in commands + ], + [ + "nvx_identity=65534:65534", + "nvx_identity=12345:12345", + "nvx_identity=0:0", + "nvx_exec=/sbin/nvx-identity-probe", + "nvx_identity=65534:65534", + ], + ) + + def test_workload_identity_scenario_fails_a_guest_that_does_not_fail_closed(self): + ran = openvmm_process.OpenvmmProcessResult( + 0, microvm_tests.WORKLOAD_IDENTITY_MARKER + b"\n" + ) + for results, message in ( + ( + [openvmm_process.OpenvmmProcessResult(80, b"")], + "a boot with no CLI workload identity did not fail closed", + ), + ( + [self.WORKLOAD_IDENTITY_REFUSED, ran], + "a boot with another CLI workload identity did not fail closed", + ), + ): + with self.subTest(message=message): + with self.assertRaisesRegex(RuntimeError, message): + self._run_workload_identity( + [*self.WORKLOAD_IDENTITY_CHECKS, *results] + ) def test_structured_outcome_scenario_covers_exit_policy_and_rejection(self): applied = self._outcome_report( diff --git a/scripts/test_nvx_tools.py b/scripts/test_nvx_tools.py index 168fa4c5b..7e87cd565 100644 --- a/scripts/test_nvx_tools.py +++ b/scripts/test_nvx_tools.py @@ -18174,7 +18174,7 @@ def test_tokenless_layered_and_fresh_sandboxes_use_new_init(self): self.assertEqual(result.returncode, 0, result.stderr) self.assertEqual(result.stdout, "new-init <4242>\n") - def test_old_lifecycle_tokens_keep_the_old_layered_init(self): + def test_old_lifecycle_tokens_no_longer_select_the_old_layered_init(self): for lifecycle in ("managed", "one-shot", ""): with self.subTest(lifecycle=lifecycle): result = self._run( @@ -18182,8 +18182,40 @@ def test_old_lifecycle_tokens_keep_the_old_layered_init(self): f"nvx_overlay_upper=ramfs nvx_lifecycle={lifecycle}" ) self.assertEqual(result.returncode, 0, result.stderr) - self.assertTrue(result.stdout.startswith("old-init"), result.stdout) - self.assertNotIn("new-init", result.stdout) + self.assertEqual(result.stdout, "new-init <4242>\n") + + def test_openvmm_workload_identity_needs_the_same_nvx_identity(self): + # OpenVMM's identity tokens no longer name the identity: without the + # same nvx_identity=, a boot of any route fails closed before it + # starts anything. + refused = ( + "NVX-WORKLOAD-ERROR: nvx_identity must name OpenVMM's workload identity\n" + ) + openvmm = "nvx_workload_uid=4242 nvx_workload_gid=4242" + for tokens in ( + f"nvx_exec=/bin/true {openvmm}", + "nvx_exec=/bin/true nvx_workload_uid=4242", + f"nvx_exec=/bin/true nvx_identity=4243:4242 {openvmm}", + f"{openvmm} nvx_identity=4242:4243", + f"nvx_entrypoint=/bin/true {openvmm}", + f"nvx_mode=managed nvx_control_tty=hvc2 {openvmm}", + f"nvx_sandbox=1 nvx_snapshot_tier=platform {openvmm}", + ): + with self.subTest(tokens=tokens): + result = self._run(tokens) + self.assertEqual(result.returncode, 125, result.stdout) + self.assertEqual(result.stdout, "") + self.assertIn(refused, result.stderr) + # The same identity boots on, here to the one-shot workload's check + # of its entrypoint. + result = self._run(f"{openvmm} nvx_identity=4242:4242 nvx_entrypoint=bin/true") + self.assertEqual(result.returncode, 125, result.stdout) + self.assertIn("hostmount\n", result.stdout) + self.assertNotIn(refused, result.stderr) + self.assertIn( + "NVX-WORKLOAD-ERROR: nvx_entrypoint must be an absolute path\n", + result.stderr, + ) def test_exec_override_cannot_bypass_the_new_sandbox_init(self): result = self._run( @@ -18191,11 +18223,12 @@ def test_exec_override_cannot_bypass_the_new_sandbox_init(self): ) self.assertEqual(result.returncode, 0, result.stderr) self.assertEqual(result.stdout, "new-init <4242>\n") - for tokens in ("", "nvx_sandbox=1 nvx_lifecycle=managed"): - with self.subTest(tokens=tokens): - result = self._run(f"{tokens} nvx_exec=/bin/true") - self.assertEqual(result.returncode, 0, result.stderr) - self.assertEqual(result.stdout, "") + result = self._run("nvx_sandbox=1 nvx_lifecycle=managed nvx_exec=/bin/true") + self.assertEqual(result.returncode, 0, result.stderr) + self.assertEqual(result.stdout, "new-init <4242>\n") + result = self._run("nvx_exec=/bin/true") + self.assertEqual(result.returncode, 0, result.stderr) + self.assertEqual(result.stdout, "") def test_blockless_platform_and_snapshot_vms_keep_their_routes(self): for tokens in ("", "nvx_snapshot_tier=platform", "shellsnap snap"): @@ -18250,8 +18283,9 @@ def test_missing_helpers_or_selected_init_fail_without_fallback(self): self.assertIn("selected guest cannot act as sandbox control", result.stderr) self.assertNotIn("old-init", result.stdout) result = self._run("nvx_sandbox=1 nvx_lifecycle=managed") - self.assertEqual(result.returncode, 0, result.stderr) - self.assertTrue(result.stdout.startswith("old-init"), result.stdout) + self.assertEqual(result.returncode, 125, result.stdout) + self.assertIn("selected guest cannot act as sandbox control", result.stderr) + self.assertNotIn("old-init", result.stdout) class GuestInitWorkloadIdentityTests(unittest.TestCase):