integration: compose qualified CLI liveness and VM reattach inputs

doc/usage.md

@@ -110,6 +110,7 @@ each code for callers of the `nvx` library.
 | `template-ineligible` | 1 | The restore window of a template's generation, which the time contract of its capture sets, excludes the host's time, so no clone may restore it now. |
 | `template-tombstoned` | 1 | A tombstone quarantines the template's generation, which takes no new clones. |
 | `exited` | 1 | OpenVMM exited, as when a start fails, whose message names the category and status of OpenVMM's outcome report when it has one. |
+| `still-running` | 1 | A stop could not confirm the VM process's exit, so the caller keeps its verified process. |
 | `exec-refused` | 1 | The guest agent refused the workload, as for a working directory that it cannot use. |
 | `endpoint-closed` | 1 | OpenVMM closed the control endpoint, as it does when the VM shuts down. |
 | `protocol` | 1 | The control session failed otherwise, as when the guest breaks the protocol or the session is reset. |

nvx-rs/README.md

@@ -124,7 +124,7 @@ changes only when the module's purpose does.
 | [`nvx::Hypervisor`](#nvxhypervisor) | KVM, MSHV, or WHP, with its serde form behind the `serde` feature |
 | [`nvx::exec`](#nvxexec) | The execution engine: the streams, credit, and outcome of an execution's two sides, without I/O, for any framing |
 | [`nvx::host::runtime`](#nvxhostruntime) | `Runtime`: what every launch shares, the runtime's files and their identity, the hypervisor and its probe, and the launch options |
-| [`nvx::host::vm`](#nvxhostvm) | A private VM directory, cold preparation, verified detached launch, and caller-saved reattachment |
+| [`nvx::host::vm`](#nvxhostvm) | A private VM directory, cold preparation, verified detached launch, confirmed stop, and caller-saved reattachment |
 | [`nvx::host::snapshot`](#nvxhostsnapshot) | Snapshot compatibility keys, strict manifests, restore-time eligibility, and the opaque input of a future VM restore |
 | [`nvx::host::config`](#nvxhostconfig) | `VmConfig` and `CaptureConfig`: pure configuration, cold-boot kernel/OpenVMM arguments, portable length checks, and capture identity |
 | [`nvx::host::restore`](#nvxhostrestore) | A caller-driven restore to Ready without I/O or a clock, and its peer-verified restore-ready listener |
@@ -500,6 +500,17 @@ or reading runtime files.
 `Reattached` holds both the process and authenticated connection. Failures
 retain their original kinds and sources and leave the saved values unchanged.
 
+`stop(process, outputs, graceful, grace, deadline)` waits through the verified
+process, first after the caller's graceful request, then after forced termination
+if needed. It returns `Stopped` only after confirmed exit. `self_exited` records
+confirmation before a forced attempt, and `forced` records that attempt, not
+proof from kill success. An unconfirmed exit is `StillRunning`, and the caller
+keeps the process. The callback receives the bounded grace deadline and must
+respect it; its boolean result is never exit proof.
+
+`Stopped::report` retains the outcome-report read's result separately, so report
+failure never hides confirmed exit.
+
 `vm::create(&runtime, &config, None, &dir)` creates an inspectable cold
 `Prepared` launch without starting a process or binding a listener. It requires
 `config.boot`, refuses a snapshot input, rechecks the directory's privacy, and
@@ -959,13 +970,15 @@ compatibility wrapper.
 Verified process references. A `ProcessRef` names one process by its ID, its
 start time, and the `BootId` of the host boot during which it started.
 
-`Process::open` checks a reference and holds its process through a pidfd on
-Linux or a process handle on Windows, whether or not the caller started it, so
-that waits sleep until the process exits and termination never reaches a process
-that reuses its ID. A reference from an earlier boot names an exited process,
-and a process whose identity this user cannot check, such as another user's,
-makes `Process::open` fail rather than report an exit. Linux hosts without
-pidfds poll the reference instead, and cannot terminate the process.
+`Process::open` returns `Opened::Running(Process)` only after checking the
+recorded boot, ID, and start time. It holds the process through a pidfd on
+Linux or a process handle on Windows, whether or not the caller started it,
+so that waits and termination never reach a process that reuses its ID.
+`Opened::Exited(Exited)` proves that the recorded boot ended or no running
+process matches both its ID and start time. A process whose identity this
+user cannot check makes `Process::open` fail rather than report an exit.
+Linux hosts without pidfds poll the reference instead, and cannot terminate
+the process.
 
 On Linux and Windows, `Process::exit_handle()` returns
 `Result<Option<ExitHandle>>`: an independently owned duplicate of the pidfd
@@ -976,6 +989,10 @@ On Windows, the duplicate retains the original handle's access rights.
 Duplication failures have kind `Io` and retain their original I/O source.
 No watcher thread, future or callback is created by the library.
 
+`Process::wait(deadline)` returns `Some(Exited)` only after verified exit, or
+`None` when the deadline passes without confirmation. `Exited` proves exit,
+not an exit status, and cannot be made by a caller.
+
 `Process::bytes_read` reports how many bytes the process has read through the
 operating system's read calls, whether the host's cache served them or not:
 `rchar` of `/proc/<pid>/io` on Linux, which it reads only while the process
@@ -1422,6 +1439,11 @@ The authenticated control session, which carries NVX's control protocol, `NVXC`,
 one frame per DATA record of a `HostConnection`; Linux and Windows primitives;
 and OpenVMM's outcome report.
 
+`report::read` returns `Ok(None)` for a missing report, and an `Io` error for an
+unreadable or malformed report, retaining the underlying error. It keeps today's
+summary of the outcome category, status code and teardown steps, not free-form
+fields.
+
 A `Transport` to an OpenVMM control endpoint reads and writes with deadlines,
 and tells, through `Transport::unread`, what the endpoint has not read yet of
 what it wrote, where the platform tells: on Linux, what `SIOCOUTQ` reports of
@@ -3055,6 +3077,7 @@ optional cause.
 | `Unavailable` | A missing or unusable artifact, hypervisor, or release, a guest image without a required control feature, or an unsupported host |
 | `Unsupported` | A request that NVX refuses: a value that it does not accept, such as a malformed or out-of-range one, options that it cannot combine, or a request that it cannot represent or enforce, such as a policy beyond a limit of OpenVMM or the guest agent |
 | `Exited` | The VM's OpenVMM process exited |
+| `StillRunning` | A stop could not confirm the VM process's exit, so the caller keeps its verified process |
 | `Io` | A host I/O failure, or state that NVX recorded on the host and cannot read |
 | `NotFound` | State that an operation names and that does not exist on the host, such as the store of a sandbox that was never created or that a removal deleted |
 | `Protocol` | A control session that failed, or a request that the guest agent refused |
@@ -3087,6 +3110,7 @@ usage errors, which no library error has.
 | `unavailable` | `Unavailable` | A dependency is missing or unusable, other than as a finer code says: install metadata, the hypervisor, a release, a distro layer or scratch image that the request names, or a host that NVX does not support |
 | `unsupported` | `Unsupported` | NVX refuses the request: options that it cannot combine, a value that it does not accept, or a policy that it cannot represent or enforce. The message names the option or field |
 | `exited` | `Exited` | OpenVMM exited, as when a start fails, whose message names the category and status of OpenVMM's outcome report when it has one |
+| `still-running` | `StillRunning` | A stop could not confirm the VM process's exit, so the caller keeps its verified process |
 | `io` | `Io` | A host I/O operation failed, or state that NVX recorded on the host is unreadable or malformed |
 | `not-found` | `NotFound` | Other state that the operation names does not exist on the host |
 | `protocol` | `Protocol` | The control session failed otherwise: the peer violated the protocol, reset the session, or failed authentication |

nvx-rs/public-api/linux.txt

@@ -58,6 +58,7 @@ variant nvx::ErrorCode::SandboxNotRunning
 variant nvx::ErrorCode::SandboxProvisioned
 variant nvx::ErrorCode::SandboxRunning
 variant nvx::ErrorCode::ScratchClaimed
+variant nvx::ErrorCode::StillRunning
 variant nvx::ErrorCode::TemplateIneligible
 variant nvx::ErrorCode::TemplateTombstoned
 variant nvx::ErrorCode::TimedOut
@@ -73,6 +74,7 @@ variant nvx::ErrorKind::NotFound
 variant nvx::ErrorKind::Other
 variant nvx::ErrorKind::Protocol
 variant nvx::ErrorKind::RuntimeMismatch
+variant nvx::ErrorKind::StillRunning
 variant nvx::ErrorKind::TimedOut
 variant nvx::ErrorKind::Unavailable
 variant nvx::ErrorKind::Unsupported
@@ -2445,6 +2447,12 @@ method nvx::host::process::BootId::as_bytes
 method nvx::host::process::BootId::current
 method nvx::host::process::BootId::from_bytes
 type nvx::host::process::ExitHandle
+struct nvx::host::process::Exited
+enum nvx::host::process::Opened
+variant nvx::host::process::Opened::Exited
+field nvx::host::process::Opened::Exited::0
+variant nvx::host::process::Opened::Running
+field nvx::host::process::Opened::Running::0
 struct nvx::host::process::Process
 method nvx::host::process::Process::bytes_read
 method nvx::host::process::Process::exit_handle
@@ -3768,6 +3776,10 @@ field nvx::host::vm::SavedVm::endpoint
 field nvx::host::vm::SavedVm::process
 field nvx::host::vm::SavedVm::runtime
 field nvx::host::vm::SavedVm::vm_dir
+struct nvx::host::vm::Stopped
+field nvx::host::vm::Stopped::forced
+field nvx::host::vm::Stopped::report
+field nvx::host::vm::Stopped::self_exited
 struct nvx::host::vm::VmDir
 method nvx::host::vm::VmDir::open
 method nvx::host::vm::VmDir::path
@@ -3780,6 +3792,7 @@ fn nvx::host::vm::create
 fn nvx::host::vm::create_private_dir
 fn nvx::host::vm::reattach
 fn nvx::host::vm::spawn
+fn nvx::host::vm::stop
 fn nvx::host::vm::verify
 mod nvx::net
 struct nvx::net::Cidr

nvx-rs/public-api/windows.txt

@@ -58,6 +58,7 @@ variant nvx::ErrorCode::SandboxNotRunning
 variant nvx::ErrorCode::SandboxProvisioned
 variant nvx::ErrorCode::SandboxRunning
 variant nvx::ErrorCode::ScratchClaimed
+variant nvx::ErrorCode::StillRunning
 variant nvx::ErrorCode::TemplateIneligible
 variant nvx::ErrorCode::TemplateTombstoned
 variant nvx::ErrorCode::TimedOut
@@ -73,6 +74,7 @@ variant nvx::ErrorKind::NotFound
 variant nvx::ErrorKind::Other
 variant nvx::ErrorKind::Protocol
 variant nvx::ErrorKind::RuntimeMismatch
+variant nvx::ErrorKind::StillRunning
 variant nvx::ErrorKind::TimedOut
 variant nvx::ErrorKind::Unavailable
 variant nvx::ErrorKind::Unsupported
@@ -1031,6 +1033,12 @@ method nvx::host::process::BootId::as_bytes
 method nvx::host::process::BootId::current
 method nvx::host::process::BootId::from_bytes
 type nvx::host::process::ExitHandle
+struct nvx::host::process::Exited
+enum nvx::host::process::Opened
+variant nvx::host::process::Opened::Exited
+field nvx::host::process::Opened::Exited::0
+variant nvx::host::process::Opened::Running
+field nvx::host::process::Opened::Running::0
 struct nvx::host::process::Process
 method nvx::host::process::Process::bytes_read
 method nvx::host::process::Process::exit_handle
@@ -2353,6 +2361,10 @@ field nvx::host::vm::SavedVm::endpoint
 field nvx::host::vm::SavedVm::process
 field nvx::host::vm::SavedVm::runtime
 field nvx::host::vm::SavedVm::vm_dir
+struct nvx::host::vm::Stopped
+field nvx::host::vm::Stopped::forced
+field nvx::host::vm::Stopped::report
+field nvx::host::vm::Stopped::self_exited
 struct nvx::host::vm::VmDir
 method nvx::host::vm::VmDir::open
 method nvx::host::vm::VmDir::path
@@ -2365,6 +2377,7 @@ fn nvx::host::vm::create
 fn nvx::host::vm::create_private_dir
 fn nvx::host::vm::reattach
 fn nvx::host::vm::spawn
+fn nvx::host::vm::stop
 fn nvx::host::vm::verify
 mod nvx::net
 struct nvx::net::Cidr

nvx-rs/src/error.rs

@@ -19,6 +19,8 @@ pub enum ErrorKind {
     Unsupported,
     /// The OpenVMM process of a VM exited.
     Exited,
+    /// The exit of a VM's process could not be confirmed; it may still run.
+    StillRunning,
     /// A host I/O operation failed, or state that NVX recorded on the host is unreadable or
     /// malformed.
     Io,
@@ -44,6 +46,7 @@ impl ErrorKind {
             Self::Unavailable => "unavailable",
             Self::Unsupported => "unsupported",
             Self::Exited => "exited",
+            Self::StillRunning => "still-running",
             Self::Io => "io",
             Self::NotFound => "not-found",
             Self::Protocol => "protocol",
@@ -88,6 +91,8 @@ pub enum ErrorCode {
     Unsupported,
     /// `exited`: the OpenVMM process of a VM exited, as when a start fails.
     Exited,
+    /// `still-running`: the exit of a VM's process could not be confirmed.
+    StillRunning,
     /// `io`: a host I/O operation failed, or state that NVX recorded on the host is unreadable or
     /// malformed.
     Io,
@@ -170,6 +175,7 @@ impl ErrorCode {
         Self::Unavailable,
         Self::Unsupported,
         Self::Exited,
+        Self::StillRunning,
         Self::Io,
         Self::NotFound,
         Self::Protocol,
@@ -205,6 +211,7 @@ impl ErrorCode {
             Self::Unavailable => ErrorKind::Unavailable.as_str(),
             Self::Unsupported => ErrorKind::Unsupported.as_str(),
             Self::Exited => ErrorKind::Exited.as_str(),
+            Self::StillRunning => ErrorKind::StillRunning.as_str(),
             Self::Io => ErrorKind::Io.as_str(),
             Self::NotFound => ErrorKind::NotFound.as_str(),
             Self::Protocol => ErrorKind::Protocol.as_str(),
@@ -248,6 +255,7 @@ impl ErrorCode {
             | Self::ArgumentMissing
             | Self::ArgumentInvalid => ErrorKind::Unsupported,
             Self::Exited | Self::SandboxExited => ErrorKind::Exited,
+            Self::StillRunning => ErrorKind::StillRunning,
             Self::Io | Self::FormatUnsupported => ErrorKind::Io,
             Self::NotFound | Self::SandboxNotProvisioned => ErrorKind::NotFound,
             Self::Protocol | Self::ExecRefused | Self::EndpointClosed => ErrorKind::Protocol,
@@ -278,6 +286,7 @@ impl From<ErrorKind> for ErrorCode {
             ErrorKind::Unavailable => Self::Unavailable,
             ErrorKind::Unsupported => Self::Unsupported,
             ErrorKind::Exited => Self::Exited,
+            ErrorKind::StillRunning => Self::StillRunning,
             ErrorKind::Io => Self::Io,
             ErrorKind::NotFound => Self::NotFound,
             ErrorKind::Protocol => Self::Protocol,

nvx-rs/src/error/tests.rs

@@ -52,6 +52,7 @@ fn every_kind_has_a_code_of_its_own_spelling() {
         ErrorKind::Unavailable,
         ErrorKind::Unsupported,
         ErrorKind::Exited,
+        ErrorKind::StillRunning,
         ErrorKind::Io,
         ErrorKind::NotFound,
         ErrorKind::Protocol,

nvx-rs/src/host/cli/store/files.rs

@@ -0,0 +1,51 @@
+//! Private, regular, non-link files, atomically replaced and synced under the state lock.
+
+use std::fs;
+use std::io;
+
+use super::StateFile;
+use super::locked::LockedState;
+use crate::host::platform;
+use crate::host::store::{files, regular};
+use crate::{Error, Result};
+
+/// Bounds a fixed record's paths and identities without reading an unbounded corrupt file.
+const MAX_RECORD_BYTES: u64 = 1024 * 1024;
+
+pub(super) fn read(state: &LockedState, name: StateFile) -> Result<Option<Vec<u8>>> {
+    let path = state.path(name);
+    let failed = |error| Error::io(format!("cannot read {}", path.display())).with_source(error);
+    let file = match regular::open_regular(&path) {
+        Ok(file) => file,
+        Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(None),
+        Err(error) => return Err(failed(error)),
+    };
+    if let Some(problem) = platform::private_file_problem(&file).map_err(failed)? {
+        return Err(Error::io(format!(
+            "the sandbox file {} {problem}",
+            path.display()
+        )));
+    }
+    regular::read_at_most(file, MAX_RECORD_BYTES, "a sandbox record")
+        .map(Some)
+        .map_err(failed)
+}
+
+pub(super) fn replace(state: &LockedState, name: StateFile, bytes: &[u8]) -> Result<()> {
+    if bytes.len() as u64 > MAX_RECORD_BYTES {
+        return Err(Error::io("the sandbox record exceeds its size bound"));
+    }
+    // Refuse an existing unsafe entry rather than replacing it or following its target.
+    read(state, name)?;
+    files::write_private(&state.path(name), bytes)?;
+    files::sync_dir(state.dir())
+}
+
+pub(super) fn remove(state: &LockedState, name: StateFile) -> Result<()> {
+    read(state, name)?;
+    let path = state.path(name);
+    fs::remove_file(&path).map_err(|error| {
+        Error::io(format!("cannot remove {}", path.display())).with_source(error)
+    })?;
+    files::sync_dir(state.dir())
+}

nvx-rs/src/host/cli/store/launch.rs

@@ -0,0 +1,80 @@
+//! Locked launch transitions, without spawning, recovery, or process termination.
+
+use super::StateFile;
+use super::files;
+use super::locked::LockedState;
+use super::record::{LaunchJournal, RuntimeRecord};
+use crate::host::process::ProcessRef;
+use crate::host::runtime::RuntimeIdentity;
+use crate::host::vm::{Prepared, SavedVm};
+use crate::{Error, Result};
+
+/// A launch whose lock stays borrowed until commit or until its journal is left behind.
+#[derive(Debug)]
+pub(crate) struct Launch<'a> {
+    locked: &'a mut LockedState,
+    journal: LaunchJournal,
+}
+
+impl LockedState {
+    /// Durably publishes the capability and complete prepared journal before the caller spawns.
+    pub(crate) fn begin_launch(
+        &mut self,
+        prepared: &Prepared,
+        runtime: &RuntimeIdentity,
+    ) -> Result<Launch<'_>> {
+        if self.journal()?.is_some() || self.runtime()?.is_some() {
+            return Err(Error::other(
+                "the sandbox already has a runtime record or an interrupted launch",
+            ));
+        }
+        let journal = LaunchJournal::prepared(self, prepared, runtime)?;
+        files::replace(self, StateFile::Capability, &journal.capability)?;
+        self.write_record(StateFile::Journal, &journal)?;
+        Ok(Launch {
+            locked: self,
+            journal,
+        })
+    }
+}
+
+impl Launch<'_> {
+    /// Records only the caller's spawned process, durably, before changing the in-memory state.
+    pub(crate) fn record_process(&mut self, process: &ProcessRef) -> Result<()> {
+        if self.journal.process.is_some() {
+            return Err(Error::other("the launch has recorded its process already"));
+        }
+        let journal = LaunchJournal {
+            process: Some(process.clone()),
+            ..self.journal.clone()
+        };
+        self.locked.write_record(StateFile::Journal, &journal)?;
+        self.journal = journal;
+        Ok(())
+    }
+
+    /// Publishes the exact saved VM, only after its matching process was journaled.
+    ///
+    /// The runtime record is the commit. A failed journal removal keeps that committed record
+    /// and reports the failure; it never rolls back the commit or signals the VM.
+    pub(crate) fn commit(self, saved: &SavedVm) -> Result<()> {
+        if self.journal.process.as_ref() != Some(&saved.process)
+            || self.journal.vm_dir != saved.vm_dir.path()
+            || self.journal.endpoint != saved.endpoint
+            || self.journal.capability != saved.capability
+            || self.journal.runtime != saved.runtime
+        {
+            return Err(Error::other(
+                "the saved VM does not match the launch and its journaled process",
+            ));
+        }
+        self.locked
+            .write_record(StateFile::Runtime, &RuntimeRecord::from(saved))?;
+        files::remove(self.locked, StateFile::Journal).map_err(|error| {
+            Error::io(format!(
+                "the VM runtime was committed but its launch journal could not be removed: {error}"
+            ))
+            .with_source(error)
+        })
+    }
+}