Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 34 additions & 1 deletion nvx-rs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,7 @@ changes only when the module's purpose does.
| [`nvx::guest::exec`](#nvxguestexec) | `Execution`: drives one program as an execution, for any guest init or agent |
| [`nvx::guest::exec::binding`](#nvxguestexecbinding) | `Binding`: an `nvx::exec::Runner` bound to an `Execution`, the glue that a guest adapter would otherwise write |
| [`nvx::guest::time`](#nvxguesttime) | x86_64: the steps of NVX time ABI v1 as typed calls in the caller's process, for any guest init or agent |
| [`nvx::guest::snapshot`](#nvxguestsnapshot) | x86_64: bounded Fresh capture, staged failure dispositions, and MMIO device mount checks, without execution control |
| [`nvx::guest::activation`](#nvxguestactivation) | x86_64: fence-free activation, with the retained template capture and clone repair interface |
| [`nvx::guest::reseed`](#nvxguestreseed) | A restore's fresh entropy, and the reseed of the kernel's random number generator with it |
| [`nvx::guest::root`](#nvxguestroot) | The operations of a layered root: its bootstrap, assembly, shares, export, mappings, and teardown |
Expand Down Expand Up @@ -1712,7 +1713,9 @@ A violation of the ABI stops the VM in enforcing mode, with the time ABI's
status, as `nvx-time` does; in strict mode, `TimeAbi::strict`, which a caller
that owns the VM's exit status and shutdown selects, the call fails with the
`Violation`, whose `status` is the one with which enforcing mode stops the VM,
and the time daemon and a capture's preparation enforce the ABI all the same.
and the time daemon and the legacy activation capture's preparation enforce
the ABI all the same. The single-call snapshot capture below uses strict
preparation instead.

A call takes the permission for OpenVMM's ports on the calling thread for each
port transaction alone, and fails with `TimeError::PortRelease` if the thread
Expand All @@ -1722,6 +1725,36 @@ did not start.

`nvx-time` runs its steps through them.

### `nvx::guest::snapshot`

x86_64: `capture(&Devices, Disks::Fresh, deadline)` composes the platform
template's mount checks, strict time checks, stall-detector suppression,
generation ID and capture request, without inspecting or freezing the
caller's executions. `Devices` names the distro and scratch devices by their
MMIO base values and the host filesystem by its virtio-fs tag. `Mounted`,
`Origin` and `MOUNT_TABLE_LIMIT` describe a mount refusal.

Every step and wait before the request, including runtime-lock acquisition
and rollback, shares the caller's absolute deadline. The request itself has
no deadline while the host saves stopped processors. If no snapshot was
taken, `Captured::Continued` reports that the detectors were restored.

`CaptureFailure` reports its `stage`, `cause` and `disposition`: `Resumed`
only when no suppression owned by this call remains, or `MustEnd { status }`.
A strict time violation preserves its actual status. A `PortRelease` remains
the cause even if detector rollback also fails. Request and release failures
are final.

The time core performs its existing clock correction after a restore, but
this interface does not complete the clone's repair and returns `MustEnd`.
Fresh platform restore packets must carry no network settings; even an
otherwise empty network policy is explicitly refused.

The caller owns its transport. After answering the capture request, send
nothing until `capture` returns. After `Continued` or a `Resumed` failure,
report on the same session. The activation module retains the legacy
Device-shaped capture interface for existing callers.

### `nvx::guest::activation`

x86_64: the preparation of a template's guest for a capture before its root's
Expand Down
45 changes: 45 additions & 0 deletions nvx-rs/public-api/linux.txt
Original file line number Diff line number Diff line change
Expand Up @@ -1734,6 +1734,50 @@ variant nvx::guest::root::UpperKind::Scratch (hidden)
variant nvx::guest::root::UpperKind::Tmpfs (hidden)
fn nvx::guest::root::assemble (hidden)
fn nvx::guest::root::bootstrap (hidden)
mod nvx::guest::snapshot
enum nvx::guest::snapshot::CaptureCause
variant nvx::guest::snapshot::CaptureCause::Io
field nvx::guest::snapshot::CaptureCause::Io::0
variant nvx::guest::snapshot::CaptureCause::Mounted
field nvx::guest::snapshot::CaptureCause::Mounted::0
variant nvx::guest::snapshot::CaptureCause::Pending
variant nvx::guest::snapshot::CaptureCause::Time
field nvx::guest::snapshot::CaptureCause::Time::0
struct nvx::guest::snapshot::CaptureFailure
field nvx::guest::snapshot::CaptureFailure::cause
field nvx::guest::snapshot::CaptureFailure::disposition
field nvx::guest::snapshot::CaptureFailure::stage
enum nvx::guest::snapshot::CaptureStage
variant nvx::guest::snapshot::CaptureStage::BootChecks
variant nvx::guest::snapshot::CaptureStage::Capture
variant nvx::guest::snapshot::CaptureStage::CaptureChecks
variant nvx::guest::snapshot::CaptureStage::Fence
variant nvx::guest::snapshot::CaptureStage::GenerationId
variant nvx::guest::snapshot::CaptureStage::Mounts
variant nvx::guest::snapshot::CaptureStage::Release
variant nvx::guest::snapshot::CaptureStage::Suppression
enum nvx::guest::snapshot::Captured
variant nvx::guest::snapshot::Captured::Continued
struct nvx::guest::snapshot::Devices
field nvx::guest::snapshot::Devices::distro
field nvx::guest::snapshot::Devices::filesystem_tag
field nvx::guest::snapshot::Devices::scratch
enum nvx::guest::snapshot::Disks
variant nvx::guest::snapshot::Disks::Fresh
enum nvx::guest::snapshot::Disposition
variant nvx::guest::snapshot::Disposition::MustEnd
field nvx::guest::snapshot::Disposition::MustEnd::status
variant nvx::guest::snapshot::Disposition::Resumed
const nvx::guest::snapshot::MOUNT_TABLE_LIMIT
struct nvx::guest::snapshot::Mounted
method nvx::guest::snapshot::Mounted::mount_point
method nvx::guest::snapshot::Mounted::origin
enum nvx::guest::snapshot::Origin
variant nvx::guest::snapshot::Origin::Distro
variant nvx::guest::snapshot::Origin::Filesystem
variant nvx::guest::snapshot::Origin::Overlay
variant nvx::guest::snapshot::Origin::Scratch
fn nvx::guest::snapshot::capture
mod nvx::guest::spawn
struct nvx::guest::spawn::Barrier
method nvx::guest::spawn::Barrier::release
Expand Down Expand Up @@ -1848,6 +1892,7 @@ method nvx::guest::time::TimeAbi::restore_finish
method nvx::guest::time::TimeAbi::start_daemon
method nvx::guest::time::TimeAbi::strict
enum nvx::guest::time::TimeError
variant nvx::guest::time::TimeError::Deadline
variant nvx::guest::time::TimeError::EntropyFile
field nvx::guest::time::TimeError::EntropyFile::path
field nvx::guest::time::TimeError::EntropyFile::source
Expand Down
102 changes: 55 additions & 47 deletions nvx-rs/src/guest/activation.rs
Original file line number Diff line number Diff line change
Expand Up @@ -273,7 +273,6 @@
//! writes the machine ID's and the resolver's files in the runtime directory, whose binds over
//! the root's `/etc/machine-id` and `/etc/resolv.conf` need Linux 6.8 or later.

mod mounts;
mod repair;
mod state;
#[cfg(test)]
Expand All @@ -284,16 +283,16 @@ use std::fs::{self, File};
use std::io::{self, Read};
use std::mem;
use std::path::{Path, PathBuf};
use std::time::Instant;

use self::mounts::DeviceNumber;
use super::GenerationId;
use super::root::{self, Device, Paths};
use super::snapshot::{self as mounts, DeviceNumber};
use super::sys;
use super::time::{
CaptureOutcome, CaptureRequest, PreCaptureError, Restore, ScratchPolicy, TimeAbi, TimeError,
};

pub use self::mounts::{Mounted, Origin};
pub use self::repair::{
RangeFault, Repair, RepairCause, RepairError, RepairProgress, RepairStage, Repaired, repair,
};
Expand All @@ -305,10 +304,7 @@ pub use self::state::{
pub use self::state::{
Applied, AppliedTeardownError, ApplyError, NetworkError, RootTemplate, apply,
};

/// The most bytes of the mount table that a [`CapturePreparation`] reads: 1 MiB, the lines of
/// thousands of mounts. It refuses a longer table, rather than check a part of it.
pub const MOUNT_TABLE_LIMIT: usize = 1 << 20;
pub use super::snapshot::{MOUNT_TABLE_LIMIT, Mounted, Origin};

/// The mount table of the calling process's mount namespace, below procfs.
const MOUNT_TABLE: &str = "self/mountinfo";
Expand Down Expand Up @@ -397,11 +393,7 @@ impl<F: AdmissionFence> CapturePreparation<F> {
/// the permission for the time ABI's ports fails with [`TimeError::PortRelease`] instead,
/// after which the thread must start neither a thread nor a process, which would inherit it.
pub fn prepare(self) -> Result<CaptureBarrier<F>, CaptureError> {
let system = LiveSystem {
paths: Paths::default(),
abi: live_abi(),
};
self.prepare_with(Box::new(system))
self.prepare_with(capture_system(None))
}

/// Implements [`prepare`](Self::prepare) on `system`.
Expand All @@ -421,22 +413,9 @@ impl<F: AdmissionFence> CapturePreparation<F> {
if let Err(error) = barrier.system.pre_capture() {
// The barrier restores a suppression that the time ABI holds after a failure, as
// after a preparation that succeeded.
barrier.suppressed = error.holds_suppression();
let error = match error {
PreCaptureError::BootChecks(error) => {
CaptureError::new(CaptureStage::BootChecks, CaptureCause::Time(error))
}
PreCaptureError::CaptureChecks(error)
| PreCaptureError::PortRelease { error, .. } => {
CaptureError::new(CaptureStage::CaptureChecks, CaptureCause::Time(error))
}
PreCaptureError::Pending => {
CaptureError::new(CaptureStage::Suppression, CaptureCause::Pending)
}
PreCaptureError::Suppression(error) | PreCaptureError::Unrestored(error) => {
CaptureError::new(CaptureStage::Suppression, CaptureCause::Io(error.into()))
}
};
let (stage, cause, suppressed) = mounts::preparation_failure(error);
barrier.suppressed = suppressed;
let error = CaptureError::new(stage, cause);
return Err(barrier.fail(error));
}
barrier.suppressed = true;
Expand All @@ -456,17 +435,12 @@ impl<F> fmt::Debug for CapturePreparation<F> {
/// Checks, in the mount table that `system` reads, that nothing is mounted from `devices` or an
/// overlay, and fails with the first mount that is, or with why the table could not be checked.
fn check_mounts(system: &mut dyn System, devices: &Devices) -> Result<(), CaptureCause> {
let table = system.mount_table().map_err(CaptureCause::Io)?;
if table.len() > MOUNT_TABLE_LIMIT {
return Err(CaptureCause::Io(io::Error::new(
io::ErrorKind::FileTooLarge,
format!("the mount table is longer than {MOUNT_TABLE_LIMIT} bytes"),
)));
}
match mounts::first_blocking(system, devices, &table).map_err(CaptureCause::Io)? {
Some(mounted) => Err(CaptureCause::Mounted(mounted)),
None => Ok(()),
}
let devices = mounts::Devices {
distro: devices.distro.mmio,
scratch: devices.scratch.mmio,
filesystem_tag: devices.filesystem_tag.clone(),
};
mounts::check_mounts(system, &devices, None)
}

/// The barrier of a prepared capture, which holds the caller's fence closed and the stall
Expand Down Expand Up @@ -517,7 +491,7 @@ impl<F: AdmissionFence> CaptureBarrier<F> {
let request = CaptureRequest::in_memory(ScratchPolicy::Fresh, generation_id);
let outcome = match self.system.capture(request) {
Ok(outcome) => outcome,
Err(error) => {
Err((error, _)) => {
// The fence is dropped closed.
self.fence = None;
let error = CaptureError::new(CaptureStage::Capture, CaptureCause::Time(error));
Expand Down Expand Up @@ -779,7 +753,7 @@ impl std::error::Error for CaptureError {
}

/// What a capture reads of the kernel and asks of the time ABI, which the tests' fake replaces.
trait System: Send {
pub(super) trait System: Send {
/// Reads the mount table of the calling process's mount namespace, at most one byte more than
/// [`MOUNT_TABLE_LIMIT`] of it.
fn mount_table(&mut self) -> io::Result<Vec<u8>>;
Expand All @@ -795,14 +769,17 @@ trait System: Send {
/// detectors, as [`TimeAbi::pre_capture`] does.
fn pre_capture(&mut self) -> Result<(), PreCaptureError>;

/// Prepares strictly by the caller deadline, retaining suppression ownership on failure.
fn pre_capture_until(&mut self, deadline: Instant) -> Result<(), (PreCaptureError, bool)>;

/// Restores the stall detectors' settings, as [`TimeAbi::cancel_capture`] does.
fn cancel_capture(&mut self) -> Result<(), Vec<u8>>;

/// Reads the VM generation ID, as [`TimeAbi::generation_id`] does.
fn generation_id(&mut self) -> Result<GenerationId, TimeError>;

/// Requests a capture, as [`TimeAbi::capture`] does.
fn capture(&mut self, request: CaptureRequest) -> Result<CaptureOutcome, TimeError>;
fn capture(&mut self, request: CaptureRequest) -> Result<CaptureOutcome, (TimeError, bool)>;
}

/// The guest's kernel, at the paths of its interfaces, and its time ABI.
Expand All @@ -811,6 +788,17 @@ struct LiveSystem {
paths: Paths,
/// The time ABI of [`live_abi`], whose preparation of a capture enforces the ABI all the same.
abi: TimeAbi,
/// The caller's pre-request deadline, cleared before the request.
deadline: Option<Instant>,
}

/// Returns the live capture seam with an explicit optional pre-request deadline.
pub(super) fn capture_system(deadline: Option<Instant>) -> Box<dyn System> {
Box::new(LiveSystem {
paths: Paths::default(),
abi: live_abi(),
deadline,
})
}

/// Returns the time ABI of the capture and of the repair: in strict mode, unless the kernel
Expand All @@ -821,6 +809,7 @@ fn live_abi() -> TimeAbi {

impl System for LiveSystem {
fn mount_table(&mut self) -> io::Result<Vec<u8>> {
sys::FileLock::check_deadline(self.deadline)?;
let path = self.paths.proc.join(MOUNT_TABLE);
let mut table = Vec::new();
File::open(&path)
Expand All @@ -829,14 +818,17 @@ impl System for LiveSystem {
.read_to_end(&mut table)
})
.map_err(|error| at(&path, &error))?;
sys::FileLock::check_deadline(self.deadline)?;
Ok(table)
}

fn windows(&mut self, mmio: u64) -> io::Result<Vec<PathBuf>> {
sys::FileLock::check_deadline(self.deadline)?;
let windows = root::window_directories(&self.paths, mmio)
.map_err(|error| at(&self.paths.proc.join("iomem"), &error))?;
let mut directories = Vec::with_capacity(windows.len());
for window in windows {
sys::FileLock::check_deadline(self.deadline)?;
match fs::canonicalize(&window) {
Ok(directory) => directories.push(directory),
// The kernel lists the window before it adds the window's device, from which
Expand All @@ -845,28 +837,44 @@ impl System for LiveSystem {
Err(error) => return Err(at(&window, &error)),
}
}
sys::FileLock::check_deadline(self.deadline)?;
Ok(directories)
}

fn block_device(&mut self, device: DeviceNumber) -> io::Result<PathBuf> {
sys::FileLock::check_deadline(self.deadline)?;
let path = self.paths.sys.join(BLOCK_DEVICES).join(device.to_string());
fs::canonicalize(&path).map_err(|error| at(&path, &error))
let directory = fs::canonicalize(&path).map_err(|error| at(&path, &error))?;
sys::FileLock::check_deadline(self.deadline)?;
Ok(directory)
}

fn pre_capture(&mut self) -> Result<(), PreCaptureError> {
self.abi.pre_capture()
}

fn pre_capture_until(&mut self, deadline: Instant) -> Result<(), (PreCaptureError, bool)> {
self.abi.pre_capture_until(deadline)
}

fn cancel_capture(&mut self) -> Result<(), Vec<u8>> {
self.abi.cancel_capture()
self.abi.cancel_capture_until(self.deadline)
}

fn generation_id(&mut self) -> Result<GenerationId, TimeError> {
self.abi.generation_id()
self.abi.generation_id_until(self.deadline)
}

fn capture(&mut self, request: CaptureRequest) -> Result<CaptureOutcome, TimeError> {
self.abi.capture(request)
fn capture(&mut self, request: CaptureRequest) -> Result<CaptureOutcome, (TimeError, bool)> {
let result = self.abi.capture_until(request, self.deadline);
if result
.as_ref()
.err()
.is_none_or(|(_, unbounded)| *unbounded)
{
self.deadline = None;
}
result
}
}

Expand Down
Loading
Loading