doc: separate OpenVMM, nvx and its callers in the host design (SL6.10b)

doc/design/1.3-system-components-and-responsibilities.md

@@ -4,20 +4,22 @@
 [1. Architecture Overview](1-architecture-overview.md)
 
 NVX spans three layers (Figure 1.2), and their responsibilities follow the
-boundary between policy and mechanism. Host software owns policy,
-orchestration, placement, identity, and attachment authorization. The OpenVMM
-virtual machine monitor (VMM) owns the virtual platform and transition safety.
-Guest software owns workload construction, supervision, and guest-visible
-repair, while the workload owns only its execution inside the resulting
-sandbox.
+boundary between policy and mechanism. In the host, the `nvx` library holds
+the mechanisms that every user of a microVM needs, and its callers, such as an
+orchestrator or NVX's own command-line interface, hold the policy and the
+state: placement, identity, storage, attachment authorization, and recovery.
+The OpenVMM virtual machine monitor (VMM) owns the virtual platform and
+transition safety. Guest software owns the guest's space, its executions, and
+guest-visible repair, while the workload owns only its execution inside the
+resulting sandbox.
 
 ```mermaid
 flowchart TB
    subgraph HostLayer["Host"]
       direction LR
-      Launcher["NVX launcher"]
-      ControlLibrary["Host control library"]
-      Storage["Snapshot storage"]
+      Callers["Callers<br/>an orchestrator or NVX's CLI"]
+      Library["nvx library"]
+      Storage["Caller-owned VM and<br/>snapshot directories"]
       Attachments["External attachments"]
    end
 
@@ -35,14 +37,14 @@ flowchart TB
    subgraph GuestLayer["Guest"]
       direction LR
       Kernel["Linux kernel"]
-      Agents["Guest agents"]
+      Agents["Guest init and helpers,<br/>with the nvx guest library"]
       Workload["Workload"]
    end
 
-   Launcher --> Frontends
-   ControlLibrary --> Frontends
-   Launcher -. control session .-> Virtio
-   ControlLibrary -. control session .-> Virtio
+   Callers --> Library
+   Callers --> Storage
+   Library --> Frontends
+   Library -. control session .-> Virtio
    Frontends --> Profile
    Frontends --> Worker
    Profile --> Worker
@@ -62,16 +64,37 @@ flowchart TB
 
 ## 1.3.1 Host Components
 
-- **NVX launcher** runs one-shot microVMs and the managed sandbox lifecycle,
-  keeps per-sandbox state, prechecks host shares and places them in the guest,
-  and compiles structured egress-policy files into VMM egress rules.
-- **Host control library** implements the host side of the control protocol,
-  artifact discovery, blockless managed launch, VMM process supervision,
-  host-path export with its access policy, delivery of the host mapping table,
-  and egress-rule translation.
-- **Snapshot storage** retains immutable machine state, memory, and optional
-  paired scratch together with the identities needed to reconstruct external
-  resources.
+- **Callers**, such as an orchestrator or NVX's command-line interface (CLI),
+  `nvx-rs`, own the policy and the state that the library leaves to them: the
+  runtime files and their labels, each VM's configuration, the private
+  directories of their VMs and snapshots, with their names and retention, the
+  values with which they find a running VM again, their per-snapshot locks,
+  and the protocol that they speak with their guest over the control session,
+  with its timing, retries, and recovery. NVX's CLI is one such caller, which
+  the `nvx` crate builds with its `cli` feature, outside its documented API:
+  it keeps its own store of sandboxes and named snapshots, its locks, the path
+  rules of its exports, with the identities of the host objects that it pins
+  in them, the staging of its images, and the compilation of its egress-policy
+  files.
+- **The `nvx` library** holds the mechanisms and value types that every caller
+  needs. It checks a VM's configuration without reading the host, and lowers
+  it into OpenVMM's arguments and the guest's boot tokens. It creates a cold
+  boot, a capture, or a restore in a private VM directory that its caller
+  owns, starts OpenVMM detached, holds the VM's process by a verified
+  reference, and verifies, reattaches to, and stops it. It verifies the
+  control endpoint's server and plays the host's role of the control session,
+  which carries the caller's protocol. It finishes a capture with the key and
+  description that make it a strict snapshot directory, opens and checks such
+  a directory, checks a VM's compatibility with it, and orders a restore's
+  activation to Ready without I/O. It keeps no durable state, takes no lock
+  but the one that serializes its own spawns on Windows, and starts no process
+  but OpenVMM.
+- **Caller-owned directories** hold what the library and OpenVMM write for a
+  caller: each VM's private directory, with its log, outcome report,
+  endpoints, and snapshot files, and the strict snapshot directories in which
+  the caller keeps its captures, each with its immutable machine state and
+  memory, its key, and its description. The caller names, places, protects,
+  and removes them.
 - **External attachments** supply process-local resources such as console
   endpoints, network endpoints, host filesystem exports, and block files.
 
@@ -109,19 +132,30 @@ flowchart TB
 ## 1.3.3 Guest Components
 
 - **Linux kernel** boots from the fixed Linux-direct handoff, drives the
-  allowlisted virtual devices, and provides the namespaces, filesystems,
-  cgroups, and process primitives used by the runtime.
-- **Guest agents** mount runtime filesystems, apply boot configuration, mount
-  live host shares, and assemble the workload filesystem. They construct the
-  workload's root and credentials, and in the layered sandbox root its
-  namespaces and device view. In the blockless managed root, they create
-  workload accounts when requested. They supervise one-shot or managed
-  workloads, prepare snapshots, and repair
-  processors, memory, entropy, and identity after restore. They also announce
-  the guest's listeners when the host publishes them, check time ABI
-  conformance, watch for clock violations, discipline the wall clock, and
-  perform the snapshot time steps.
+  allowlisted virtual devices, and provides the filesystems, mounts, and
+  process primitives that the guest's software uses.
+- **Guest agents**, the guest's init, PID 1, and its helpers, build the VM's
+  one space: they mount runtime filesystems, apply boot configuration, mount
+  live host shares, and, in a layered root, assemble the root from its image
+  and upper and switch into it once. They run the workload's executions as
+  plain processes with the VM's workload credentials, without a namespace or
+  cgroup of their own, and create workload accounts when requested. They
+  supervise one-shot or managed workloads, prepare snapshots, repair
+  processors, memory, entropy, and identity after restore, and activate a
+  restored layered root before Ready. They also announce the guest's listeners
+  when the host publishes them, check time ABI conformance, watch for clock
+  violations, discipline the wall clock, and perform the snapshot time steps.
+  The space and the executions are the init's own code: nvx-init, NVX's init
+  for layered roots, implements NVX's reference space, which is private to
+  `nvx`. The `nvx` guest library gives any init the contracts that it reads,
+  the guest's side of a capture, which also repairs a restore, the activation
+  of a restore, and the account of the host's mapping table, `MappingTable`;
+  it starts no helper or workload but the time daemon, which
+  `TimeAbi::start_daemon` forks, and the
+  [guest-agent guidelines](../guest-agent-guidelines.md#ownership) give the
+  one other call that may start a process, and the init's obligations, which
+  `nvx` does not enforce.
 - **Workload** is the microVM's single tenant: it owns the microVM's lifetime,
   resource envelope, and network identity, and it may run any number of
-  processes, all inside the isolation boundary that the guest agents
-  construct. It has no direct authority over host resources or VMM policy.
+  processes, all inside the VM, its isolation boundary. It has no direct
+  authority over host resources or VMM policy.

doc/design/1.4-trust-boundaries.md

@@ -3,25 +3,31 @@
 [Architecture documentation](README.md) >
 [1. Architecture Overview](1-architecture-overview.md)
 
-As Figure 1.3 shows, the host and its orchestrator are trusted with guest
-memory, image contents, and attachment policy, while the workload is
-untrusted.
+As Figure 1.3 shows, the host's control plane, a caller and the `nvx` library
+that it uses, is trusted with guest memory, image contents, and attachment
+policy, while the workload is untrusted. Inside the control plane, the library
+checks what its caller gives it, but trusts the caller with what the caller
+owns, as [The Library and Its Callers](#146-the-library-and-its-callers)
+describes.
 
 ```mermaid
 flowchart LR
-   Control["Trusted host control plane"]
+   Caller["Trusted caller<br/>policy, paths, and state"]
+   Library["nvx library<br/>checks and mechanisms"]
    Vmm["VMM trust boundary"]
    Agent["Guest agent boundary"]
    Workload["Untrusted workload"]
    Resources["Host resources"]
 
-   Control -->|"policy and capabilities"| Vmm
+   Caller -->|"configuration, paths, and labels"| Library
+   Library -->|"checked launch and capability"| Vmm
    Resources -->|"validated attachments"| Vmm
    Vmm <-->|"bounded device protocols"| Agent
-   Agent -->|"isolation, identity,<br/>resources, and supervision"| Workload
+   Agent -->|"space, credentials,<br/>and supervision"| Workload
 ```
 
-**Figure 1.3.** Trust boundaries of the host, VMM, guest agents, and workload.
+**Figure 1.3.** Trust boundaries of the caller, the library, the VMM, the guest
+agents, and the workload.
 
 Because each microVM has a single tenant, the virtual machine boundary is also
 the boundary between workloads. A workload may use ordinary Linux interfaces
@@ -38,6 +44,10 @@ validation boundaries. Authority to provide an input does not make its
 structure valid. OpenVMM therefore validates the machine configuration before
 opening host resources, validates it again at worker construction, and
 validates all persisted state before any restored virtual processor runs.
+`nvx` checks a caller's configuration without reading the host, checks a
+launch completely before it starts OpenVMM, and checks a snapshot's directory,
+key, and manifest before it restores from them, in addition to OpenVMM's
+checks, never in their place.
 
 ## 1.4.1 Defense in Depth
 
@@ -62,12 +72,19 @@ of resuming with uncertain state.
 
 A host control client reaches the guest only through the control-console
 broker. The broker admits one local peer whose operating-system identity is
-the VMM's user, requires a launcher-provided capability before any host data
-reaches the guest, and bounds every record, queue, and credit window in both
-directions. Malformed, stale, or out-of-order records fail closed.
-Capabilities are process-local authority: they never appear in arguments,
-environment variables, logs, endpoint names, snapshots, or attachment
-identities.
+the VMM's user, requires the capability that the VM's launch provided before
+any host data reaches the guest, and bounds every record, queue, and credit
+window in both directions. Malformed, stale, or out-of-order records fail
+closed. Capabilities are process-local authority: they never appear in
+arguments, environment variables, logs, endpoint names, snapshots, or
+attachment identities.
+
+`nvx` generates a fresh capability for each launch, passes it to OpenVMM only
+on its standard input, never writes it to a file, and hides it from its
+debugging output: a caller that must reconnect after a restart keeps it
+itself. Before a caller presents the capability, `nvx` checks that the control
+endpoint's server is the VM's held process: through its socket's original peer
+on Linux, and through the named pipe's server process on Windows.
 
 ## 1.4.3 Device-Enforced Policy
 
@@ -81,40 +98,110 @@ The guest's announcements choose which of its ports are published, never the
 host addresses or the clients. The host filesystem server applies each
 export's access mode and path policy on every request, whichever guest mount
 or link reaches it. Guest mount flags, routes, and links are therefore never
-security boundaries.
+security boundaries. `nvx` lowers its caller's network policy and export into
+these device policies, and enforces none of its own.
 
 ## 1.4.4 Images and Snapshot Artifacts
 
 Image layers are trusted inputs: the overlay filesystem interprets metadata
 stored in lower layers, and bytes read from any layer can appear in captured
-memory. A snapshot identifies its blocks by a storage generation, which
-transfers trust for layer and scratch contents to the storage owner; the VMM
-trusts the control plane in the host to authenticate that generation.
+memory. A snapshot that binds its blocks identifies each, by default, by the
+SHA-256 digest of its file, which OpenVMM computes, or by a storage generation,
+which transfers trust for layer and scratch contents to the storage owner; the
+VMM trusts the control plane in the host to authenticate that generation. A
+platform snapshot, the only kind that `nvx` captures, binds no block: each
+restore attaches its caller's own image and upper, which the caller trusts as
+it trusts any image.
 
 The VMM trusts the host control plane not to give the same writable file, such
 as a caller-supplied RAM backing or scratch file, to more than one VMM. It
-takes no file locks to detect such sharing, and the RAM backing that it
-creates itself is a new file for every VM. Snapshot artifacts are shared by
-design: every VM restored from a snapshot maps the same immutable memory
-artifact with copy-on-write, so each VM's changes stay private and never reach
-the artifact. A capture always publishes new files, and a restored VM is not
-captured again.
+takes no file locks to detect such sharing, nor does `nvx`, and the RAM
+backing that the VMM creates itself is a new file for every VM. Snapshot
+artifacts are shared by design: every VM restored from a snapshot maps the same
+immutable memory artifact with copy-on-write, so each VM's changes stay private
+and never reach the artifact. A capture always publishes new files, and a
+restored VM is not captured again.
 
 Copy-on-write isolates writes, not physical pages. Clean pages of a shared
 memory artifact may remain shared through the host page cache among the VMs
 that map it, and guest kernel, guest agent, and cached workload pages can all
 originate in captured memory. Snapshot copy-on-write is therefore not, by
 itself, a tenant authorization or physical-page isolation mechanism. Artifact
 access control, sharing scope, backing-file placement, retention, and host
-memory deduplication remain deployment responsibilities.
+memory deduplication remain the responsibilities of the caller and its
+deployment.
 
 ## 1.4.5 Host-Facing Outputs
 
-Host-facing outputs are bounded. The optional outcome report is written after
-teardown to a previously absent local path. It records only a schema version,
-an opaque instance ID, the backend category, the operation and outcome
-category with a numeric status, whether the network policy was requested and
-applied with its mode, rule counts, and host-loopback action, and per-step
-teardown flags. It never contains commands, environment values, paths,
-network destinations, proxy details, workload output, credentials, or
-free-form error text, and the VMM does not upload it.
+A launch's host-facing outputs are OpenVMM's log and its outcome report. `nvx`
+gives both fixed names in the caller's private VM directory, beside the VM's
+endpoints: it creates the log anew, private to the caller, before OpenVMM
+starts, and refuses a launch whose outcome report or Linux socket endpoints
+already exist rather than replace them. OpenVMM writes the report after its
+VM's teardown, so an OpenVMM process that ends before then leaves none. A
+creation that fails removes only the entries that it created, and once a
+launch's creation succeeds, `nvx` removes none of its outputs: the caller
+reads and removes them.
+
+The outcome report is bounded. It is written after teardown to a previously
+absent local path. It records only a schema version, an opaque instance ID,
+the backend category, the operation and outcome category with a numeric
+status, whether the network policy was requested and applied with its mode,
+rule counts, and host-loopback action, and per-step teardown flags. It never
+contains commands, environment values, paths, network destinations, proxy
+details, workload output, credentials, or free-form error text, and the VMM
+does not upload it.
+
+## 1.4.6 The Library and Its Callers
+
+`nvx` checks what its caller gives it, but holds nothing for the caller
+between calls: it keeps no record, retains no handle of a snapshot, and takes
+no lock of a VM or a snapshot. It checks:
+
+- That each VM directory is private to the current user: on Linux, owned by
+  the user with mode 0700, which OpenVMM requires of the directory of its
+  control socket; on Windows, owned by the user, with a protected access
+  control list that grants access to the user alone. It never changes an
+  existing directory, since restricting it would not revoke access through
+  files that others already hold open.
+- That a snapshot directory has the strict form that
+  [Validation and Integrity](5.4-snapshot-artifact-and-attachment-model.md#543-validation-and-integrity)
+  describes, once, when it opens the snapshot.
+- That a VM's process is the one that the caller recorded, by its host boot,
+  process ID, and start time, and that the peers of its control and
+  restore-ready endpoints are that process.
+- That a saved VM's runtime identity is the current one, before it drives the
+  VM; see [Runtime Identity](1.5-compatibility-and-versioning-model.md#154-runtime-identity).
+
+The caller keeps what `nvx` cannot check once and for all:
+
+- **Paths.** It keeps the runtime files and the block images stable, and its
+  VM and snapshot directories stable and private, from a launch's creation to
+  the end of their use, and each snapshot directory immutable while it uses
+  the snapshot. A restore hard-links the snapshot's files into the VM's
+  directory, so it keeps both on one filesystem. The `ExportPlan` of an export
+  names host paths only, never the objects at them, so the caller also keeps
+  each path naming the object that it means, as the
+  [export's module](../../nvx-rs/src/host/filesystem/mod.rs) describes: across
+  the race between its own checks and OpenVMM's opens, through a recheck of a
+  workload's writable parts before each later launch, and for as long as each
+  denial is to hold. NVX's CLI pins the identities of the mapped and denied
+  objects inside those writable parts when it plans an export, and checks
+  them before each start.
+- **Outputs.** It owns each launch's log, outcome report, and endpoints, which
+  `nvx` never removes, though on Linux OpenVMM removes the control and
+  boot-console sockets that it bound, at its VM's teardown. It also owns each
+  capture's files, which it moves into a snapshot directory of its own. It
+  saves what finds a running VM again: its directory, control endpoint,
+  capability, process reference, and runtime identity.
+- **Per-snapshot locks.** On Linux, adding or removing a restore's links to a
+  snapshot's files changes their metadata, which OpenVMM checks until it
+  connects the restore-ready endpoint. The caller therefore holds a lock of
+  its own for each snapshot from the creation of a restore, which adds the
+  links, through its peer-verified restore-ready event or its VM's verified
+  exit, and around the removal of a restore's view of the snapshot. Windows
+  needs none.
+- **Spawns.** On Windows, `nvx` serializes its own spawns of OpenVMM, but a
+  process that the caller starts while a spawn passes handles to OpenVMM may
+  inherit them, so the caller keeps its other process starts out of a VM's
+  start.