host: hand owned cold-launch commands to callers

nvx-rs/Cargo.toml

@@ -132,6 +132,11 @@ path = "src/bin/nvx-rs.rs"
 required-features = ["cli"]
 doc = false
 
+[[example]]
+name = "external-launch"
+path = "examples/external-launch.rs"
+required-features = ["host"]
+
 # The guest binaries. Their unit tests live in the library, so the binaries have no test harness.
 [[bin]]
 name = "nvx-console-pending"

nvx-rs/README.md

@@ -124,7 +124,7 @@ changes only when the module's purpose does.
 | [`nvx::Hypervisor`](#nvxhypervisor) | KVM, MSHV, or WHP, with its serde form behind the `serde` feature |
 | [`nvx::exec`](#nvxexec) | The execution engine: the streams, credit, and outcome of an execution's two sides, without I/O, for any framing |
 | [`nvx::host::runtime`](#nvxhostruntime) | `Runtime`: what every launch shares, the runtime's files and their identity, the hypervisor and its probe, and the launch options |
-| [`nvx::host::vm`](#nvxhostvm) | A private VM directory, cold and capture preparation, verified detached launch, confirmed stop, and caller-saved reattachment |
+| [`nvx::host::vm`](#nvxhostvm) | A private VM directory, cold and capture preparation, owned cold-launch command, verified detached launch, confirmed stop, and caller-saved reattachment |
 | [`nvx::host::snapshot`](#nvxhostsnapshot) | Strict private snapshot directories, verified metadata, VM compatibility, captured NIC addresses, and time eligibility |
 | [`nvx::host::config`](#nvxhostconfig) | `VmConfig` and `CaptureConfig`: pure configuration, cold-boot kernel/OpenVMM arguments, portable length checks, and capture identity |
 | [`nvx::host::restore`](#nvxhostrestore) | A caller-driven restore to Ready without I/O or a clock, and its peer-verified restore-ready listener |
@@ -540,7 +540,7 @@ these names before creating any output, including the log.
 
 `Prepared::capability()` and `outputs()` return what the caller may journal
 before a spawn. `log()` returns the same opened log, without reopening its
-path. `inherit(file)` moves an already opened resource into the preparation's
+path. `inherit(resource)` moves an already opened resource into the preparation's
 inheritance allowlist and keeps it open. `arguments()` returns the whole
 configuration's OpenVMM arguments. Creation makes caller-relative runtime
 and block-image paths absolute without resolving links or `..` components,
@@ -569,6 +569,27 @@ blockless root or memory that cannot fit the record's byte count fails with
 `ErrorKind::Unsupported`. These projections do not enable the restore caller;
 `create` still accepts cold launches only.
 
+`Prepared::into_command()` moves the complete launch into `VmmCommand`: its
+program, arguments, working directory, capability, outputs, inherited resources,
+opened log and Windows breakaway setting. It neither reopens nor duplicates
+resources, renders again nor rediscovers paths. `InheritedHandle` owns its
+resource, converts from and back to `OwnedFd` on Unix or `OwnedHandle` on Windows,
+and also accepts an opened `File`. The command documents the caller's spawn
+contract, including standard streams, allowlisted inheritance, Linux detachment
+and umask, Windows log job and breakaway, and cleanup of a failed child.
+Read the resolved NIC or capture context, or build an activation record,
+before consuming `Prepared` into the command.
+
+The compiled [external-launch example](examples/external-launch.rs) consumes
+this command, uses `supervisor::spawn_vm` without a sandbox store, checks the
+cold guest's boot marker and shell response, then asks it to exit cleanly.
+It requires the VMM's actual exit status 0 and a successful
+outcome with complete teardown. Build with
+`cargo build --locked --example external-launch` and supply the backend, OpenVMM,
+kernel, initramfs, new VM directory, and the three artifact SHA-256 digests from
+their installation metadata, in that order. Artifact paths may be relative to
+the caller's directory.
+
 `vm::spawn(prepared)` consumes that preparation and returns `Launched` with
 its verified `process`, the same `outputs`, and the same `capability`. The log
 and inherited resources are passed as already opened objects, never reopened
@@ -1094,6 +1115,15 @@ after, and returns a `Starting` launch that the caller commits once the VMM
 serves its control endpoint, or abandons. A launch that fails never leaves an
 unrecorded VMM running.
 
+`spawn_vm` starts a caller-owned command without a store, with the same strict
+Windows log-job requirement as `vm::spawn`: an untrusted job name or a log that
+cannot name a job is refused before child creation. The legacy `spawn` keeps
+its fallback without a job for those logs and names. Other spawn behavior is
+shared. If setup fails after a child is created, including failed process
+verification, `Launched::kill_and_confirm_exit` waits without a deadline for
+its exit and reaps it on Unix. The caller reports any cleanup error; the
+legacy `kill_and_wait` remains time-bounded and may release a running child.
+
 `VmmCommand::inherit` lists the descriptors (Linux) or handles (Windows) that
 the VMM inherits under their numbers, besides its standard streams, and it
 inherits nothing else of the caller's: on Linux, they must be close-on-exec, a

nvx-rs/examples/external-launch.rs

@@ -0,0 +1,176 @@
+//! A caller-owned cold launch, using the strict low-level spawn primitive without a store.
+//!
+//! Run the compiled example with a backend, OpenVMM, kernel, initramfs, new VM directory, and
+//! the three artifact SHA-256 digests from their installation metadata, in that order.
+
+use std::error::Error;
+use std::io::{self, Write};
+use std::path::Path;
+use std::time::{Duration, Instant};
+
+use nvx::Hypervisor;
+use nvx::activation::{MachineIdMode, View};
+use nvx::host::config::{Boot, ConsoleAttachment, RootConfig, VmConfig};
+use nvx::host::platform::Transport;
+use nvx::host::runtime::{ArtifactIdentity, LaunchOptions, Runtime, RuntimeIdentity};
+use nvx::host::supervisor::{self, Inherited};
+use nvx::host::{artifacts, report, session, vm};
+
+fn main() -> Result<(), Box<dyn Error>> {
+    let args: Vec<_> = std::env::args_os().collect();
+    if args.len() != 9 {
+        return Err(io::Error::new(
+            io::ErrorKind::InvalidInput,
+            "usage: external-launch BACKEND OPENVMM KERNEL INITRAMFS VM-DIR \
+             OPENVMM-SHA256 KERNEL-SHA256 INITRAMFS-SHA256",
+        )
+        .into());
+    }
+    let hypervisor = match args[1].to_str() {
+        Some("kvm") => Hypervisor::Kvm,
+        Some("mshv") => Hypervisor::Mshv,
+        Some("whp") => Hypervisor::Whp,
+        _ => return Err(io::Error::new(io::ErrorKind::InvalidInput, "invalid backend").into()),
+    };
+    nvx::host::runtime::probe_hypervisor(hypervisor)?;
+    let identity = |index: usize| {
+        ArtifactIdentity::sha256(args[index].to_str().ok_or_else(|| {
+            io::Error::new(
+                io::ErrorKind::InvalidInput,
+                "artifact digest is not Unicode",
+            )
+        })?)
+        .map_err(Box::<dyn Error>::from)
+    };
+    let runtime = Runtime {
+        openvmm: artifacts::absolute(Path::new(&args[2]))?,
+        kernel: artifacts::absolute(Path::new(&args[3]))?,
+        initramfs: artifacts::absolute(Path::new(&args[4]))?,
+        identity: RuntimeIdentity::new(identity(6)?, identity(7)?, identity(8)?),
+        hypervisor,
+        cpu_profile: Some("auto".to_owned()),
+        windows_breakaway: false,
+        options: LaunchOptions::default(),
+    };
+    let config = VmConfig {
+        processors: 1,
+        memory_mib: 256,
+        root: RootConfig::Blockless,
+        view: View {
+            hostname: b"external".to_vec(),
+            machine_id: MachineIdMode::Image,
+        },
+        network: None,
+        filesystem: None,
+        console: ConsoleAttachment::Listener,
+        boot: Some(Boot {
+            command_line: "quiet loglevel=0".to_owned(),
+            capture: None,
+        }),
+    };
+    vm::create_private_dir(Path::new(&args[5]))?;
+    let dir = vm::VmDir::open(Path::new(&args[5]))?;
+    let vm::VmmCommand {
+        program,
+        arguments,
+        working_dir,
+        capability,
+        outputs,
+        inherit,
+        log,
+        breakaway_from_job,
+    } = vm::create(&runtime, &config, None, &dir)?.into_command();
+    let command = supervisor::VmmCommand {
+        program,
+        arguments,
+        working_dir,
+        breakaway_from_job,
+        inherit: inherit.iter().map(Inherited::new).collect(),
+    };
+    let mut launched = supervisor::spawn_vm(&command, capability.as_bytes(), log)?;
+    let result = (|| -> Result<(), Box<dyn Error>> {
+        let process = launched.process()?;
+        let deadline = Instant::now() + Duration::from_secs(60);
+        let console = outputs.console.as_deref().ok_or_else(|| {
+            io::Error::new(io::ErrorKind::InvalidInput, "no prepared boot console")
+        })?;
+        let mut console = session::connect(
+            console,
+            process.reference().pid,
+            process.reference().start_time,
+            deadline,
+        )?;
+        read_until(console.as_mut(), b"NVX-GUEST-BOOT-OK:", deadline)?;
+        console.write_all(
+            b"printf 'NVX-EXTERNAL-LAUNCH-OK\\n'; /sbin/nvx-exit 0\n",
+            Some(deadline.saturating_duration_since(Instant::now())),
+        )?;
+        read_until(console.as_mut(), b"\nNVX-EXTERNAL-LAUNCH-OK", deadline)?;
+        if process.wait(deadline)?.is_none() {
+            return Err(io::Error::new(io::ErrorKind::TimedOut, "VM did not exit").into());
+        }
+        match launched.wait_timeout(Duration::ZERO)? {
+            Some(0) => {}
+            Some(status) => {
+                return Err(io::Error::other(format!("VMM exited with status {status}")).into());
+            }
+            None => return Err(io::Error::other("VMM exit status is unavailable").into()),
+        }
+        let outcome = report::read(&outputs.outcome)?
+            .ok_or_else(|| io::Error::other("missing or invalid VM outcome"))?;
+        if outcome.category.as_deref() != Some("success")
+            || outcome.status_code != Some(0)
+            || outcome.teardown_complete != Some(true)
+        {
+            return Err(
+                io::Error::other(format!("VM did not shut down cleanly: {outcome:?}")).into(),
+            );
+        }
+        Ok(())
+    })();
+    if let Err(error) = result {
+        if let Err(cleanup) = launched.kill_and_confirm_exit() {
+            return Err(io::Error::other(format!(
+                "{error}; could not confirm exit of the launched VM: {cleanup}"
+            ))
+            .into());
+        }
+        return Err(error);
+    }
+    launched.detach();
+    println!(
+        "external cold launch completed; outcome={}",
+        outputs.outcome.display()
+    );
+    Ok(())
+}
+
+/// Prints console output through a marker, retaining only enough bytes to match across reads.
+fn read_until(console: &mut dyn Transport, marker: &[u8], deadline: Instant) -> io::Result<()> {
+    let mut pending = Vec::new();
+    let mut buffer = [0; 4096];
+    loop {
+        let remaining = deadline.saturating_duration_since(Instant::now());
+        if remaining.is_zero() {
+            return Err(io::Error::new(
+                io::ErrorKind::TimedOut,
+                "console marker not received",
+            ));
+        }
+        let count = console.read(&mut buffer, Some(remaining))?;
+        if count == 0 {
+            return Err(io::Error::new(
+                io::ErrorKind::UnexpectedEof,
+                "boot console closed",
+            ));
+        }
+        io::stdout().write_all(&buffer[..count])?;
+        pending.extend_from_slice(&buffer[..count]);
+        if pending.windows(marker.len()).any(|bytes| bytes == marker) {
+            return Ok(());
+        }
+        if pending.len() > marker.len() {
+            pending.drain(..pending.len() - marker.len());
+        }
+    }
+}

nvx-rs/public-api/linux.txt

@@ -3274,8 +3274,10 @@ method nvx::host::supervisor::Inherited::proc_path
 struct nvx::host::supervisor::Launched
 method nvx::host::supervisor::Launched::detach
 method nvx::host::supervisor::Launched::id
+method nvx::host::supervisor::Launched::kill_and_confirm_exit
 method nvx::host::supervisor::Launched::kill_and_wait
 method nvx::host::supervisor::Launched::process
+method nvx::host::supervisor::Launched::wait_timeout
 enum nvx::host::supervisor::Reattached
 variant nvx::host::supervisor::Reattached::Exited
 field nvx::host::supervisor::Reattached::Exited::record
@@ -3322,6 +3324,7 @@ fn nvx::host::supervisor::lock_spawns
 fn nvx::host::supervisor::reattach
 fn nvx::host::supervisor::spawn
 fn nvx::host::supervisor::spawn_command
+fn nvx::host::supervisor::spawn_vm
 mod nvx::host::template
 struct nvx::host::template::Acquired
 field nvx::host::template::Acquired::lease
@@ -3766,6 +3769,7 @@ fn nvx::host::transport::connect_verified
 mod nvx::host::vm
 struct nvx::host::vm::Capability
 method nvx::host::vm::Capability::as_bytes
+struct nvx::host::vm::InheritedHandle
 struct nvx::host::vm::Launched
 field nvx::host::vm::Launched::capability
 field nvx::host::vm::Launched::outputs
@@ -3776,6 +3780,7 @@ method nvx::host::vm::Prepared::arguments
 method nvx::host::vm::Prepared::capability
 method nvx::host::vm::Prepared::capture_context
 method nvx::host::vm::Prepared::inherit
+method nvx::host::vm::Prepared::into_command
 method nvx::host::vm::Prepared::log
 method nvx::host::vm::Prepared::nic
 method nvx::host::vm::Prepared::outputs
@@ -3800,6 +3805,15 @@ field nvx::host::vm::VmOutputs::console
 field nvx::host::vm::VmOutputs::endpoint
 field nvx::host::vm::VmOutputs::log
 field nvx::host::vm::VmOutputs::outcome
+struct nvx::host::vm::VmmCommand
+field nvx::host::vm::VmmCommand::arguments
+field nvx::host::vm::VmmCommand::breakaway_from_job
+field nvx::host::vm::VmmCommand::capability
+field nvx::host::vm::VmmCommand::inherit
+field nvx::host::vm::VmmCommand::log
+field nvx::host::vm::VmmCommand::outputs
+field nvx::host::vm::VmmCommand::program
+field nvx::host::vm::VmmCommand::working_dir
 fn nvx::host::vm::create
 fn nvx::host::vm::create_private_dir
 fn nvx::host::vm::reattach

nvx-rs/public-api/windows.txt

@@ -1859,8 +1859,10 @@ method nvx::host::supervisor::Inherited::new
 struct nvx::host::supervisor::Launched
 method nvx::host::supervisor::Launched::detach
 method nvx::host::supervisor::Launched::id
+method nvx::host::supervisor::Launched::kill_and_confirm_exit
 method nvx::host::supervisor::Launched::kill_and_wait
 method nvx::host::supervisor::Launched::process
+method nvx::host::supervisor::Launched::wait_timeout
 enum nvx::host::supervisor::Reattached
 variant nvx::host::supervisor::Reattached::Exited
 field nvx::host::supervisor::Reattached::Exited::record
@@ -1907,6 +1909,7 @@ fn nvx::host::supervisor::lock_spawns
 fn nvx::host::supervisor::reattach
 fn nvx::host::supervisor::spawn
 fn nvx::host::supervisor::spawn_command
+fn nvx::host::supervisor::spawn_vm
 mod nvx::host::template
 struct nvx::host::template::Acquired
 field nvx::host::template::Acquired::lease
@@ -2351,6 +2354,7 @@ fn nvx::host::transport::connect_verified
 mod nvx::host::vm
 struct nvx::host::vm::Capability
 method nvx::host::vm::Capability::as_bytes
+struct nvx::host::vm::InheritedHandle
 struct nvx::host::vm::Launched
 field nvx::host::vm::Launched::capability
 field nvx::host::vm::Launched::outputs
@@ -2361,6 +2365,7 @@ method nvx::host::vm::Prepared::arguments
 method nvx::host::vm::Prepared::capability
 method nvx::host::vm::Prepared::capture_context
 method nvx::host::vm::Prepared::inherit
+method nvx::host::vm::Prepared::into_command
 method nvx::host::vm::Prepared::log
 method nvx::host::vm::Prepared::nic
 method nvx::host::vm::Prepared::outputs
@@ -2385,6 +2390,15 @@ field nvx::host::vm::VmOutputs::console
 field nvx::host::vm::VmOutputs::endpoint
 field nvx::host::vm::VmOutputs::log
 field nvx::host::vm::VmOutputs::outcome
+struct nvx::host::vm::VmmCommand
+field nvx::host::vm::VmmCommand::arguments
+field nvx::host::vm::VmmCommand::breakaway_from_job
+field nvx::host::vm::VmmCommand::capability
+field nvx::host::vm::VmmCommand::inherit
+field nvx::host::vm::VmmCommand::log
+field nvx::host::vm::VmmCommand::outputs
+field nvx::host::vm::VmmCommand::program
+field nvx::host::vm::VmmCommand::working_dir
 fn nvx::host::vm::create
 fn nvx::host::vm::create_private_dir
 fn nvx::host::vm::reattach

nvx-rs/src/host/platform/windows/spawn.rs

@@ -11,26 +11,27 @@ use std::{mem, ptr};
 
 #[cfg(feature = "cli")]
 use windows_sys::Win32::Foundation::{
-    DUPLICATE_SAME_ACCESS, DuplicateHandle, INVALID_HANDLE_VALUE, WAIT_TIMEOUT,
+    DUPLICATE_SAME_ACCESS, DuplicateHandle, INVALID_HANDLE_VALUE,
 };
 use windows_sys::Win32::Foundation::{
     GetHandleInformation, HANDLE, HANDLE_FLAG_INHERIT, SetHandleInformation, WAIT_OBJECT_0,
+    WAIT_TIMEOUT,
 };
 #[cfg(feature = "cli")]
 use windows_sys::Win32::System::Console::{
     GetStdHandle, STD_ERROR_HANDLE, STD_HANDLE, STD_INPUT_HANDLE, STD_OUTPUT_HANDLE,
 };
+#[cfg(feature = "cli")]
+use windows_sys::Win32::System::Threading::GetCurrentProcess;
 use windows_sys::Win32::System::Threading::{
     CREATE_BREAKAWAY_FROM_JOB, CREATE_NEW_PROCESS_GROUP, CREATE_NO_WINDOW,
     CREATE_UNICODE_ENVIRONMENT, CreateProcessW, DeleteProcThreadAttributeList,
-    EXTENDED_STARTUPINFO_PRESENT, INFINITE, InitializeProcThreadAttributeList,
+    EXTENDED_STARTUPINFO_PRESENT, GetExitCodeProcess, INFINITE, InitializeProcThreadAttributeList,
     LPPROC_THREAD_ATTRIBUTE_LIST, PROC_THREAD_ATTRIBUTE_HANDLE_LIST,
     PROC_THREAD_ATTRIBUTE_JOB_LIST, PROCESS_CREATION_FLAGS, PROCESS_INFORMATION,
     STARTF_USESTDHANDLES, STARTUPINFOEXW, TerminateProcess, UpdateProcThreadAttribute,
     WaitForSingleObject,
 };
-#[cfg(feature = "cli")]
-use windows_sys::Win32::System::Threading::{GetCurrentProcess, GetExitCodeProcess};
 
 #[cfg(feature = "cli")]
 use super::process::refused_termination;
@@ -99,7 +100,6 @@ impl LaunchedProcess {
 
     /// Waits up to `timeout` for the process to exit, and returns its exit status if it has,
     /// as `Popen.wait` reads it with `GetExitCodeProcess`.
-    #[cfg(feature = "cli")]
     pub(crate) fn wait_timeout(&self, timeout: Duration) -> io::Result<Option<u32>> {
         let milliseconds = u32::try_from(timeout.as_millis()).unwrap_or(INFINITE - 1);
         // SAFETY: CreateProcessW returned the handle with full access.