Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions doc/run.md
Original file line number Diff line number Diff line change
Expand Up @@ -1170,6 +1170,18 @@ as through a hard link or a bind mount. NVX no longer reads earlier
configuration formats, including those that recorded the shares as requested;
deprovision such a sandbox and provision it again.

A share must not reach the sandbox's own files, which the workload could
otherwise read or change: `provision` refuses a share whose resolved host path
equals, contains, or lies inside the sandbox's state directory, its image, its
scratch image, or the OpenVMM, kernel, or initramfs of the artifact root, before
it records the sandbox, and `start` refuses one again before it launches the VM.
`start` resolves the recorded host paths again, so a share that came to reach
one of these files through a link since `provision` is refused too. A
`provision` that finds no artifact root leaves the artifacts to `start`. Only
resolved paths are compared, so don't share a directory that holds a link
through which the host reaches one of these files, nor a hard link or bind
mount of one.

#### Access policy

Each share can narrow what the workload can see and modify inside it. OpenVMM
Expand Down
2 changes: 1 addition & 1 deletion doc/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -321,7 +321,7 @@ do, and their names cannot be abbreviated either.
| `--environment-file PATH` | omitted | Read the exact managed `exec` environment from a strict UTF-8 JSON array of `KEY=VALUE` strings, limited to 1 MiB of input. Invalid JSON, non-finite numbers, and unpaired surrogate escapes are refused. An empty array requests an empty environment. This option is mutually exclusive with `--environment`; omitting both preserves guest defaults. |
| `--inherit-default-environment` | off | Layer the managed `exec` environment from `--environment` or `--environment-file` over the guest default environment instead of replacing it; an entry replaces the default variable of the same name. Without either option the workload already receives the defaults, so the flag has no effect. |
| `--hypervisor {auto,whp,kvm,mshv}` | `auto` | Select the host hypervisor. |
| `--mount GUEST_TARGET,HOST_PATH[,ro\|rw]` | none | Live-share a host directory, or a single regular host file without its directory, at the absolute target inside the container rootfs for `run` or `provision`; defaults to `ro`. The target cannot contain whitespace, control characters, double quotes, backslashes, `=`, or any UTF-8 byte that Linux treats as whitespace, such as the `0xa0` in `à`, so that it stays one kernel command-line token in the shares of a one-shot `run`. The mode follows the last comma, so `HOST_PATH` may contain commas when the mode is given. Repeat to attach more shares, each with its own target and mode, for example a read-write workspace and a read-only tool cache, as the children of an aggregate that the guest binds at their targets; a shared file always uses the aggregate, and so does every share of a managed sandbox. Targets and host paths must not overlap. A one-shot `run` describes its shares in kernel command-line tokens that count toward the sandbox's 1024-byte budget; a managed sandbox announces only their number and sends their mapping table once its guest agent is ready, for at most 256 shares. An `rw` share accepts guest-created symbolic links, which the host never follows. `/`, `/etc`, `/run`, and `/run/nvx`, and the `/etc/machine-id`, `/etc/resolv.conf`, `/proc`, `/sys`, `/dev`, `/.nvx-agent`, and `/run/nvx/time` trees, are reserved. |
| `--mount GUEST_TARGET,HOST_PATH[,ro\|rw]` | none | Live-share a host directory, or a single regular host file without its directory, at the absolute target inside the container rootfs for `run` or `provision`; defaults to `ro`. The target cannot contain whitespace, control characters, double quotes, backslashes, `=`, or any UTF-8 byte that Linux treats as whitespace, such as the `0xa0` in `à`, so that it stays one kernel command-line token in the shares of a one-shot `run`. The mode follows the last comma, so `HOST_PATH` may contain commas when the mode is given. Repeat to attach more shares, each with its own target and mode, for example a read-write workspace and a read-only tool cache, as the children of an aggregate that the guest binds at their targets; a shared file always uses the aggregate, and so does every share of a managed sandbox. Targets and host paths must not overlap, and `provision` and `start` refuse a host path that equals, contains, or lies inside the sandbox's state directory, image, scratch image, OpenVMM, kernel, or initramfs; see [Live host shares](run.md#live-host-shares). A one-shot `run` describes its shares in kernel command-line tokens that count toward the sandbox's 1024-byte budget; a managed sandbox announces only their number and sends their mapping table once its guest agent is ready, for at most 256 shares. An `rw` share accepts guest-created symbolic links, which the host never follows. `/`, `/etc`, `/run`, and `/run/nvx`, and the `/etc/machine-id`, `/etc/resolv.conf`, `/proc`, `/sys`, `/dev`, `/.nvx-agent`, and `/run/nvx/time` trees, are reserved. |
| `--mount-deny HOST_PATH` | none | Hide one existing file or directory inside a `--mount` host directory; relative paths are resolved inside it. It may name the directory itself when `--mount-allow` paths inside it remain. With several shares, it applies to the `--mount` before it, which must share a directory. Repeat to deny multiple paths. |
| `--mount-allow HOST_PATH` | none | Expose one existing file or directory inside a `--mount-deny` path of the same share again, for example a readable `mcp-payloads` directory inside hidden logs. The hidden directories on the way list only the entries that lead to it, and the workload cannot modify them. Relative paths are resolved inside the share; with several shares, it applies to the `--mount` before it. Repeat to allow multiple paths. See [Access policy](run.md#access-policy). |
| `--mount-write HOST_PATH` | none | Make one existing file or directory one of the only writable parts of an `rw` share; the workload's writes anywhere else in the share fail with `EROFS`. Relative paths are resolved inside the share; with several shares, it applies to the `--mount` before it, which must be `rw`. Repeat to declare multiple writable paths. `provision` persists the denied, allowed, and writable paths. |
Expand Down
5 changes: 5 additions & 0 deletions nvx-rs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3042,6 +3042,11 @@ guest agents of NVX advertise bits 0, 4, and 6 in sandbox mode.
reported the exit.
- `exec` refuses more than 1 MiB of workload output, which the guest agent
never forwards.
- `provision`, and `start` again, refuse a share whose resolved host path
equals, contains, or lies inside the sandbox's state directory, image, or
scratch image, or the OpenVMM, kernel, or initramfs of the artifact root, as
[Live host shares](../doc/run.md#live-host-shares) describes, where `nvx.py`
shared them.
- The state directory holds the store's files: `sandbox.lock` replaces the
`lifecycle.lock` of `nvx.py`, `runtime.json` has the store's format, and each
start replaces `openvmm.log`, which `nvx.py` appended to. `exec` holds the
Expand Down
4 changes: 3 additions & 1 deletion nvx-rs/src/host/cli/sandbox/lifecycle/start.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,12 @@ use super::{
CONFIG_NAME, Found, REQUIRED_FEATURES, connect, find_vm, mismatched, require_config,
running_vm, runtime_for,
};
use crate::host::cli::artifacts::locate;
use crate::host::cli::deadline::{Deadline, RETRY_INTERVAL};
use crate::host::cli::exec_lock::ExecLock;
use crate::host::cli::sandbox::lock::LifecycleLock;
use crate::host::cli::sandbox::state::{Config, read_config};
use crate::host::cli::sandbox::{Artifacts, error_with_cause};
use crate::host::cli::sandbox::{Artifacts, error_with_cause, refuse_own_files};
use crate::host::cli::session::{ExecRequest, ExecResult, Session};
use crate::host::cli::{Exit, Result, output};
use crate::host::process::KILL_TIMEOUT;
Expand Down Expand Up @@ -63,6 +64,7 @@ fn start_locked(
}
let config = Config::new(config)?;
let launch = config.launch()?;
refuse_own_files(state_dir, &launch, Some(&locate(artifacts.root)?.dir))?;
let binding = config.distro_binding();
let scratch_binding = config.scratch_binding()?;
let distro = binding
Expand Down
87 changes: 85 additions & 2 deletions nvx-rs/src/host/cli/sandbox/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,16 @@ mod one_shot;
mod options;
mod state;

use std::path::{Path, PathBuf};
use std::time::Instant;

use super::artifacts::{Located, OPENVMM_VARIABLE, locate};
use super::download::{self, Hold};
use super::shell::format_command;
use super::{Exit, Result, output};
use super::{Exit, Result, io_error, output};
use crate::ErrorCode;
use crate::host::artifacts::{Guest, KERNEL_NAME, require_artifact};
use crate::host::artifacts::{ArtifactDir, Guest, KERNEL_NAME, require_artifact};
use crate::host::paths;
use crate::host::sandbox::distro::{DISTRO_POLICY_REVISION, DistroBinding, DistroFormat};
use crate::host::sandbox::mount::{plan_mounts, validate_mounts};
use crate::host::sandbox::options::{
Expand Down Expand Up @@ -250,6 +252,14 @@ pub(super) fn run(root: Option<&str>, options: Options) -> Result {
// binding, with the manifest beside that path.
let distro = RecordedDistro::new(&launch, binding)?;
let state_dir = state_dir("provision")?;
// A provision runs no artifact, so it needs no artifact root; without one, `start`
// checks the shares against the artifacts that it runs.
let located = locate(root).ok();
refuse_own_files(
Path::new(state_dir),
&launch,
located.as_ref().map(|located| &located.dir),
)?;
lifecycle::provision(
state_dir,
&launch,
Expand Down Expand Up @@ -386,6 +396,79 @@ pub(super) fn run(root: Option<&str>, options: Options) -> Result {
}
}

/// Refuses a share of the sandbox's own files: one whose host path equals, contains, or lies
/// inside the state directory `state_dir`, the image or scratch image of `launch`, or the OpenVMM,
/// kernel, or Alpine initramfs of `artifacts`, if known. Through it, the guest could read or
/// change the sandbox's records, its storage, or what the host runs.
///
/// The paths are compared once [`paths::resolve`] has resolved them, without a Windows verbatim
/// prefix. Resolution follows symbolic links and junctions, but not hard links or bind mounts.
/// `provision` checks before it records the sandbox, and `start` again before it launches it,
/// since the shares' paths may have come to resolve elsewhere.
fn refuse_own_files(
state_dir: &Path,
launch: &SandboxLaunch,
artifacts: Option<&ArtifactDir>,
) -> Result {
let Some(export) = launch.export() else {
return Ok(());
};
let mut owned = vec![
("state directory", state_dir.to_owned()),
("image", launch.distro().path().to_owned()),
];
owned.extend(
launch
.scratch()
.map(|path| ("scratch image", path.to_owned())),
);
if let Some(dir) = artifacts {
owned.extend([
("OpenVMM", dir.openvmm()),
("kernel", dir.kernel()),
("initramfs", dir.initramfs(Guest::Alpine)),
]);
}
let resolve = |path: &Path| {
paths::resolve(path)
.map(without_verbatim_prefix)
.map_err(|error| io_error(ErrorCode::Io, &error, path))
};
let owned = owned
.into_iter()
.map(|(name, path)| Ok((name, resolve(&path)?)))
.collect::<Result<Vec<_>>>()?;
for child in &export.children {
let shared = resolve(&child.root)?;
let overlap = owned
.iter()
.find(|(_, path)| shared.starts_with(path) || path.starts_with(&shared));
if let Some((name, path)) = overlap {
return Err(Exit::error(
ErrorCode::Unsupported,
format!(
"sandbox mount host path {} overlaps the sandbox's {name} {}; a share cannot \
expose the sandbox's own files",
shared.display(),
path.display()
),
));
}
}
Ok(())
}

/// Returns `path` without the verbatim prefix, `\\?\`, with which [`paths::resolve`] keeps a
/// Windows path that only that prefix names, such as one whose name ends in a period: its
/// components would otherwise differ from those of a path that it lies inside. A network path
/// keeps its prefix, since no share is one.
fn without_verbatim_prefix(path: PathBuf) -> PathBuf {
match path.to_str().and_then(|text| text.strip_prefix(r"\\?\")) {
Some(rest) if !rest.starts_with(r"UNC\") => PathBuf::from(rest),
_ => path,
}
}

/// Checks the launch of a one-shot run again, and returns its plan with the format and identity of
/// the distro layer that `binding`, which the run's verification of the layer made, records.
///
Expand Down
10 changes: 6 additions & 4 deletions nvx-rs/tests/cli/download.rs
Original file line number Diff line number Diff line change
Expand Up @@ -201,9 +201,10 @@ fn restores_download_nothing_for_snapshots_that_they_cannot_read() {

#[test]
fn provision_and_deprovision_need_no_artifact_root() {
// A provision runs no artifact, and a deprovision of a sandbox that records no launch reads
// none, so both succeed where nvx-rs finds no artifact root: from an executable outside any
// release or checkout, without --root.
// A provision runs no artifact, and checks its shares against the artifacts only where it
// finds them, and a deprovision of a sandbox that records no launch reads none, so both
// succeed where nvx-rs finds no artifact root: from an executable outside any release or
// checkout, without --root.
let root = Root::new();
let elsewhere = tempfile::tempdir().unwrap();
let executable = elsewhere
Expand Down Expand Up @@ -239,10 +240,11 @@ fn provision_and_deprovision_need_no_artifact_root() {
String::from_utf8_lossy(&output.stderr)
);
};
let provision: Vec<String> = crate::provision(&root, "state", "")
let mut provision: Vec<String> = crate::provision(&root, "state", "")
.iter()
.map(|argument| root.expand(argument))
.collect();
provision.extend(["--mount".to_owned(), root.expand("/data,{root/share}")]);
nvx_rs(&provision);
nvx_rs(&crate::lifecycle("deprovision", &root, "state"));
assert!(!root.below("state").exists());
Expand Down
Loading
Loading