Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/acceptance-case-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -357,7 +357,7 @@ suite in its shards, and the public sandbox scenarios in its guest and sandbox j
| `untiered-network-policy` | rewritten | On named snapshots with a NIC, which `snapshot capture --net` captures, as [Snapshots](#snapshots) says, keeping its checks of a restore's DNS, egress, forwards and listener publication. Its raw-snapshot parts retire: the raw capture and restores, the guest's repair marker, the console release, and OpenVMM's refusals of raw restores that the CLI cannot issue, in whose place the CLI must refuse a restore without `--network-profile`, and one with `--net` |
| `virtio-features` | kept | |
| `virtio-net` | kept | |
| `vm-lifecycle` | kept | A [cold boot](#cold-boots) through the public API: `nvx-rs/tests/vm_lifecycle.rs::a_cold_vm_is_reattached_from_saved_values_and_stopped` |
| `vm-lifecycle` | kept | A [cold boot](#cold-boots) through the public API: `nvx-rs/tests/vm_lifecycle.rs::a_cold_vm_is_reattached_from_saved_values_and_stopped`. Since SL4.4d, it pings and stops nvx-init with NVXC version 2's frames, where it sent version 1's |
| `workload-identity` | kept | Since SL6.3 (#1122), it also checks that a guest given OpenVMM's workload identity without `nvx_identity=`, or with another one, fails closed |

### Cold boots
Expand Down
3 changes: 1 addition & 2 deletions doc/design/4.3-isolation-and-resource-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,7 @@ the boot configuration requests it, nvx-init and the guest agents of the
blockless managed root instead create an account for a host-selected identity
that the image lacks, and replace an image account that uses the UID with
another primary group or without a usable home. The older layered sandbox
root creates no accounts, so it does not advertise
[account creation](4.4-host-guest-control-interface.md#442-feature-negotiation).
root creates no accounts.

Before the workload runs, its child clears supplementary groups and every
capability set and enables `no_new_privs`, so the workload keeps none of the
Expand Down
73 changes: 23 additions & 50 deletions doc/design/4.4-host-guest-control-interface.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,10 @@ Table 4.2 lists the operations of the control protocol.
| Operation | Purpose |
| --- | --- |
| `READY` | Reports readiness, and is the success reply to requests that return no other data |
| `FEATURES` | Reports which control behaviors the guest image provides |
| `MAPS` | Delivers a contiguous range of host mapping-table entries |
| `EXEC` | Runs one workload with a timeout, arguments, and an optional working directory and environment |
| `CANCEL` | Kills the running execution that the request names |
| Shutdown | Shuts the VM down gracefully between executions |
| Shutdown | Shuts the VM down gracefully, ending a running execution first |

A mapping request carries a contiguous range of the host mapping table's
entries, each with its access, a source relative to the guest's mount of the
Expand All @@ -40,14 +39,13 @@ and optionally an absolute working directory of at most 4,096 bytes and up to
256 `KEY=VALUE` environment entries of at most 4,096 bytes each, with
distinct, non-empty keys. The environment either replaces the default
environment or is layered over it, each entry replacing the default variable
of the same name. In version 2 of the protocol, which NVX's host speaks, the
host streams an execution's standard input, which the guest agents
acknowledge as they write it, with at most 64 KiB unacknowledged, until the
host ends it, and output returns as separate standard output and standard
error streams within the credit window, without a total limit. Version 1
carries no standard input, and forwards at most 1 MiB of output combined for
each execution. As Table 4.3 shows, an execution that runs ends with an exit
report, which carries an outcome category and a status.
of the same name. In version 2 of the protocol, which NVX's host and guest
agents speak alone, the host streams an execution's standard input, which the
guest agents acknowledge as they write it, with at most 64 KiB unacknowledged,
until the host ends it, and output returns as separate standard output and
standard error streams within the credit window, without a total limit. As
Table 4.3 shows, an execution that runs ends with an exit report, which carries
an outcome category and a status.

**Table 4.3.** Exit outcomes.

Expand All @@ -57,7 +55,6 @@ report, which carries an outcome category and a status.
| `signal` | A signal ended the workload; the status is 128 plus the signal number |
| `timeout` | The workload exceeded its timeout and was killed; the status is 124 |
| `cancelled` | A cancellation killed the workload; the status is 137 |
| `output-limit` | In version 1, the workload's combined output exceeded 1 MiB, and the guest agents killed it; the status is 125 |
| `failed` | The guest agents could not supervise the workload; the status is 125 |

Otherwise, the guest agents reply with an error, which carries one of the
Expand All @@ -81,50 +78,23 @@ account for.

## 4.4.2 Feature Negotiation

Version 2 of the protocol negotiates nothing: every frame names its version,
the guest's `READY` names the version that its agents speak, and NVX's host
requires version 2, whose guest agents provide every behavior that the host
depends on. Version 1 negotiates features instead.

A successful readiness probe does not prove that an image enforces the
behaviors that a host depends on. As Table 4.5 shows, a `FEATURES` request of
version 1 therefore returns a four-byte little-endian bit mask of control
features.

**Table 4.5.** Control feature bits.

| Bit | Feature |
| ---: | --- |
| 0 | Cancellation |
| 1 | Reserved; never set |
| 2 | Workload account creation |
| 3 | Per-execution cgroup containment |
| 4 | Per-execution environment |
| 5 | Working-directory refusal |
| 6 | Host mapping table |

The blockless managed root provides cancellation, workload account creation,
per-execution environments, working-directory refusal, and the host mapping
table. It runs plain processes and does not announce or require cgroup
containment (bit 3). The layered sandbox root provides cancellation,
per-execution environments, and the host mapping table. Its workloads run as
an account that the image already has, so it does not create accounts. A
feature bit is added together with the behavior it names.
Hosts ignore unknown bits and trailing bytes, read an `unsupported-operation`
refusal as no features, and terminate a guest that lacks a feature they need
instead of running workloads without the policy they requested.
The protocol negotiates nothing: every frame names its version, the guest's
`READY` names the version that its agents speak, and NVX's host requires
version 2, whose guest agents provide every behavior that the host depends on.
A peer of another version is refused: the host refuses a `READY` that names
another version, and the guest agents refuse a frame of another version as a
malformed one, which ends the session. Version 1, whose `FEATURES` request
reported the control behaviors that a guest image provided, is gone.

## 4.4.3 Session Semantics

Requests are typed, length-bounded, ordered, and associated with one VM
instance and epoch. Credit limits bound queued data in both directions.
Malformed, unauthenticated, stale, duplicated, or out-of-order records fail
closed. Only one execution runs at a time; any request other than a
cancellation, or in version 2 the execution's standard input or a shutdown,
that arrives during an execution is refused as `busy`. In version 2, a
shutdown ends the running execution first. In version 1, shutdown is therefore
graceful only between executions: a host cancels a running execution before
it requests shutdown, or else terminates the VM, which fails the execution.
cancellation, the execution's standard input, or a shutdown, that arrives
during an execution is refused as `busy`. A shutdown ends the running
execution first.

The control session is process-local even when guest parser state is
snapshotted. Restore establishes a fresh host capability, peer identity,
Expand All @@ -142,8 +112,11 @@ non-secret invariants needed before the control channel exists.
Conformance vectors give the bytes that every host and guest agent must
produce and accept, whatever code they share.
[`nvxc-v1.txt`](../../nvx-rs/tests/vectors/nvxc-v1.txt) holds those of
version 1 of the control protocol: its frames, their execution, mapping, and
feature payloads, and records of the session that carries them.
version 1 of the control protocol, which no peer speaks any longer: its frames,
which version 2 refuses, their execution and mapping payloads, which version 2
carries unchanged, its feature payloads, and records of the session that
carries them. Version 2's frames have no vector file: their tests pin their
bytes.
[`openvmm/`](../../nvx-rs/tests/vectors/openvmm) holds the VMM's own vectors
and cases of the session's records, unchanged. A vector changes only with a
protocol version, which gets vector files of its own, and the bytes of an
Expand Down
20 changes: 12 additions & 8 deletions doc/guest-agent-guidelines.md
Original file line number Diff line number Diff line change
Expand Up @@ -250,8 +250,9 @@ plain-process core:
root's user and group databases once, before its switch, and creates when
the boot, with `nvx_workload_account=create`, or the activate request asks
for it.
- **In the child**, which leads a process group of its own, with the null
device as its standard input and pipes as its standard output and error,
- **In the child**, which leads a process group of its own, with a pipe of
the input that the host streams as its standard input, which nvx-init
closes after the input's end, and pipes as its standard output and error,
the execution resets its priority, sets its credentials, changes to its
working directory and execs. It resets its out-of-memory score adjustment
and its nice value to 0, then drops its capability bounding set and takes
Expand All @@ -263,11 +264,14 @@ plain-process core:
in another directory.
- **End, cancellation, timeout and reset** each kill the execution's process
group: nvx-init kills it when the execution's leader ends, normally or not,
when the host cancels the execution, when its timeout passes, and when a
session reset abandons it, whose output it then discards rather than report
it on the new session. PID 1 collects the status of every child, orphans
included. A process group is a cleanup scope, not containment: a process
that leaves its group escapes it, as the capture's sweep allows for.
when the host cancels the execution or shuts the VM down while it runs,
when its timeout passes, and when a session reset abandons it. None of an
abandoned execution's output reaches the new session: nvx-init reads no
more of it, and what it had already queued goes out before the reset's
acknowledgement, in the earlier epoch, whose records the broker drops.
PID 1 collects the status of every child, orphans included. A process
group is a cleanup scope, not containment: a process that leaves its group
escapes it, as the capture's sweep allows for.

This is priority, not limits. PID 1 and its helpers run with an out-of-memory
score adjustment of -1000 and a nice value of -10, ahead of the executions'
Expand All @@ -294,7 +298,7 @@ enforces none of it.

nvx-init has no workspace: it refuses a capture while an execution runs, or
kills the process groups and sweeps when the host asks, as its capture
request says. Without permission to kill, it refuses the request with `EBUSY`
request says. Without permission to kill, it refuses the request as `busy`
while any process but PID 1 and its registered helpers runs; with it, it ends
a running execution's process group, then sweeps, killing every process but
PID 1 and its helpers again and again until none remains. The checks, the
Expand Down
1 change: 0 additions & 1 deletion doc/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,6 @@ each code for callers of the `nvx` library.
| `unsupported` | 1 | NVX refuses the request: options that it cannot combine, a value that it does not accept, or a policy that it cannot represent or enforce. The message names the option or field. |
| `unavailable` | 1 | A dependency is missing or unusable, other than as a finer code says: install metadata, the hypervisor, a release, a distro layer or scratch image that the request names, or a host that NVX does not support. |
| `artifact-missing` | 1 | An artifact that the command runs, such as OpenVMM, the guest kernel, or an initramfs, does not exist, or no artifact root holds them. |
| `guest-features-missing` | 1 | The guest image lacks control features that the sandbox needs, which the message names. |
| `sandbox-not-provisioned` | 1 | No sandbox is provisioned at `--state-dir`, which does not exist, is not a plain directory, or holds no configuration. |
| `sandbox-provisioned` | 1 | `provision` found the sandbox provisioned already, or it was provisioned again while `deprovision` removed it. |
| `sandbox-not-running` | 1 | `exec` or `stop` found the VM of a provisioned sandbox not running. |
Expand Down
Loading
Loading