diff --git a/doc/acceptance-case-matrix.md b/doc/acceptance-case-matrix.md index d5fa82c3b..612d74bc6 100644 --- a/doc/acceptance-case-matrix.md +++ b/doc/acceptance-case-matrix.md @@ -32,7 +32,7 @@ its own in turn. | [The adapter's other tests](#the-adapters-other-tests) | 169 | 0 | 9 | 160 | | [MicroVM scenarios](#microvm-scenarios) | 54 | 51 | 2 | 1 | | [CI](#ci) | 69 | 43 | 3 | 23 | -| [Docs](#docs) | 36 | 34 | 1 | 1 | +| [Docs](#docs) | 36 | 32 | 1 | 3 | ## The adapter's scenarios @@ -498,15 +498,15 @@ this matrix. The README's own code is no doctest. | `nvx-rs/src/guest/terminal.rs - guest::terminal (line 10)` | kept | | | `nvx-rs/src/guest/time/mod.rs - guest::time (line 15)` | kept | | | `nvx-rs/src/host/sandbox/prepared.rs - host::sandbox::prepared::PreparedLaunch (line 92)` | kept | Its launch type leaves with the old launch paths (#1087), whose PR retires it | -| `nvx-rs/src/host/store/mod.rs - host::store (line 75)` | kept | The sandbox store leaves (#1121), whose PR retires it | -| `nvx-rs/src/host/supervisor/mod.rs - host::supervisor (line 33)` | kept | The old supervisor leaves (#1087), whose PR retires it | +| `nvx-rs/src/host/store/mod.rs - host::store (line 72)` | kept | The sandbox store leaves (#1121), whose PR retires it | +| `nvx-rs/src/host/supervisor/mod.rs - host::supervisor (line 32)` | kept | The old supervisor leaves (#1087), whose PR retires it | | `nvx-rs/src/host/template/clone.rs - host::template::clone::ClonePlan (line 247)` | retired | `ClonePlan` and `CloneRequest` leave with the old clone path (SL6.2b, #1121) | -| `nvx-rs/src/host/template/plan.rs - host::template::plan::TemplatePlan (line 137)` | kept | The template store leaves (SL6.2a, #1121), whose PR retires it | -| `nvx-rs/src/host/template/store.rs - host::template::store::Store (line 473)` | kept | The template store leaves (SL6.2a, #1121), whose PR retires it | +| `nvx-rs/src/host/template/plan.rs - host::template::plan::TemplatePlan (line 137)` | retired | `TemplatePlan` leaves with the template store (SL6.2a, #1121) | +| `nvx-rs/src/host/template/store.rs - host::template::store::Store (line 473)` | retired | The template store leaves (SL6.2a, #1121) | | `nvx-rs/src/host/transport.rs - host::transport (line 22)` | kept | | | `nvx-rs/src/lib.rs - (line 58)` | kept | | | `nvx-rs/src/protocol/nvxs/mod.rs - protocol::nvxs (line 68)` | kept | | | `nvx-rs/src/host/snapshot/capture.rs - host::snapshot::capture::finish_capture (line 50)` | kept | | -| `nvx-rs/src/host/store/mod.rs - host::store (line 107)` | kept | The sandbox store leaves (#1121), whose PR retires it | +| `nvx-rs/src/host/store/mod.rs - host::store (line 104)` | kept | The sandbox store leaves (#1121), whose PR retires it | | `nvx-rs/README.md`, its Example: an instance through `LaunchSpec` and the sandbox store | rewritten | Nothing compiles it. `nvx-rs/examples/host-workflows.rs` compiles the documented host workflows, and the library's guide (SL6.10a) replaces the example | | `nvx-rs/examples/external-launch.rs` | kept | Compiled with the crate's targets, and run by hand on a host | diff --git a/doc/design/1.5-compatibility-and-versioning-model.md b/doc/design/1.5-compatibility-and-versioning-model.md index 01097f645..0a8926e9f 100644 --- a/doc/design/1.5-compatibility-and-versioning-model.md +++ b/doc/design/1.5-compatibility-and-versioning-model.md @@ -78,9 +78,8 @@ guest's address and the identity of the complete deny-all policy under which the capture attached it. Every restore of the template replaces that policy with its clone's own, so a template serves the clones of every network policy at its guest's address, and its key records none of them. The key's encoding, -version 1, has not been released, so a template store that an earlier build -created is not supported: delete it and create a new one. There is no -migration. +version 1, has not been released, so no key that an earlier build encoded is +supported. There is no migration. ## 1.5.3 Persisted State diff --git a/doc/design/5.3-snapshot-protocol.md b/doc/design/5.3-snapshot-protocol.md index 81e62151f..53f2cc16d 100644 --- a/doc/design/5.3-snapshot-protocol.md +++ b/doc/design/5.3-snapshot-protocol.md @@ -234,8 +234,7 @@ an endpoint that stops reading, stalls the capture, with nothing published and the source never resumed, until the process that launched the VMM ends it; the VMM logs each state unit that has not answered after five seconds, so that the log of such a capture names it. Every launcher therefore bounds a capture with -a deadline that ends the VMM, as the nvx crate's template store does with the -deadline that creating or refreshing a template requires. +a deadline that ends the VMM. ## 5.3.4 State-Unit Ordering diff --git a/nvx-rs/README.md b/nvx-rs/README.md index b66f3979c..f6deded0e 100644 --- a/nvx-rs/README.md +++ b/nvx-rs/README.md @@ -48,7 +48,7 @@ nvx-rs/ │ ├── host/ feature `host` (default): the runtime that every launch shares, VM │ │ directories and configurations, artifacts, launch, network and egress policy │ │ files, filesystem, NVXS connection, session, process, platform, VM instances, -│ │ sandbox stores, templates' manifests, compatibility keys, store, and leases, +│ │ sandbox stores, templates' manifests and compatibility keys, │ │ outcome reports, file seals, and the sandbox launch profile of `nvx-rs sandbox`, │ │ its distro and scratch bindings, and its prepared launches; with feature │ │ `cli`, the `nvx-rs` CLI @@ -64,7 +64,7 @@ nvx-rs/ │ │ the typed calls through which it runs the time ABI's steps and the reseed │ └── bin/ nvx-rs, nvx-fake-openvmm, and the guest binaries ├── tests/ fake-OpenVMM, CLI, NVXS session, process-supervision, checked-handoff, store, -│ host-module layering, template-capture, and guest-binary tests, the CLI's +│ host-module layering, and guest-binary tests, the CLI's │ scenario tests and their harness, in cli/, the store's record fixtures, the │ hand-written sandbox configurations of every format, the cases of the export │ and root boot tokens, and the template manifests that OpenVMM wrote, in @@ -139,7 +139,6 @@ changes only when the module's purpose does. | [`nvx::host::transport`](#nvxhosttransport) | A verified VM endpoint connection and the blocking host NVXS driver, with the existing timed transport | | [`nvx::host::process`](#nvxhostprocess) | Verified references to OpenVMM processes, through which callers wait for them and end them | | [`nvx::host::supervisor`](#nvxhostsupervisor) | The supervisor of a sandbox's VMM, beneath the CLI, `Instance`, and external orchestrators | -| [`nvx::host::template`](#nvxhosttemplate) | Legacy template capture plans and store, with aliases of the snapshot metadata | | [`nvx::host::seal`](#nvxhostseal) | `FileSeal`: what the host reports about an open file from its metadata alone, without reading it | | [`nvx::host::session`, `platform`, `report`](#nvxhostsession-platform-report) | The authenticated control session, Linux and Windows primitives, and OpenVMM's outcome report | | [`nvx::host::sandbox`](#nvxhostsandbox) | The launch profile of `nvx-rs sandbox`: its plans, live shares, option checks, bindings, and prepared launches | @@ -655,8 +654,39 @@ responsive shell input, and a verified clean exit; it downloads no artifact. `SnapshotKey` and `SnapshotManifest` own the compatibility key and strict OpenVMM manifest reader. `SnapshotManifest::time` and `SnapshotKey::time` expose the capture's `TimeContract`, whose `eligibility` checks its restore window at a `HostTime`; -`HostTime::now` samples the host's clock and identities. The existing -`host::template` names remain aliases of these same types. +`HostTime::now` samples the host's clock and identities. + +`SnapshotManifest::read` reads the `manifest.bin` that OpenVMM wrote for a +capture, at most `MAX_MANIFEST_LEN` bytes of a regular file that is not a link +or other reparse point, and refuses another snapshot format version, an unknown, +mistyped, or repeated field, truncated bytes, and a capture that is not a +template's: a platform-tier capture for clones, with a read-only `distro` whose +identity is unbound, a fresh scratch, and the dormant filesystem slot. + +`CaptureClass` holds the invariant inputs of a capture, known before it: the +runtime, the guest's build, the backend, the requested CPU profile, the +processors, those online at boot, and RAM, the geometries of the disks, the +network, of which it takes what the template captures alone, its NIC at the +guest's address under the complete deny-all policy, and the adapter's other +`Invariants`. + +`SnapshotKey::new` checks that the capture made the machine that its class +requested, and derives the template's compatibility key, from metadata alone: +the runtime and contract versions through which its clones are restored, the +backend, CPU profile and effective CPUID, topology, rates, resources, kernel +command line, disks, and devices of its machine, with the NIC's addresses and +the identity of the complete deny-all policy under which the capture attached +it, which each clone's restore replaces with the clone's own, so that a template +with a NIC serves the clones of every network policy at its guest's address. + +A key and a class each have one canonical, versioned encoding, whose +domain-separated SHA-256 digest is its identity, a `KeyIdentity`, which converts +to and from the `nvx::activation::Identity` by which a clone's lease and durable records +name its template's key, or a `ClassIdentity`; the +[conformance vectors](tests/vectors/README.md) of version 1, +`tests/vectors/snapshot-key-v1.txt`, pin both encodings and identities. The +key's encoding, version 1, leaves the network policy out, since templates +capture their NIC under the complete deny-all policy. The key's canonical encoding and identity use the snapshot names and omit `guest_build`. Version 1 changes in place: no snapshot outlives a deployment, @@ -669,6 +699,13 @@ anchors, host and boot identities, lineage and cut-to-anchor time: captures at different times can have different key bytes and the same `KeyIdentity`. The declared rates and time ABI remain compatibility inputs. +`TimeContract::eligibility` decides, apart from compatibility, whether a clone +may restore a capture at a host's time, a `HostTime`, which `HostTime::now` +samples as OpenVMM does: within the boot of the capture, by the host's monotonic +clock, and otherwise by UTC, the downtime must not be negative, and the interval +by which a restore advances the clone's VM time at most `MAX_DOWNTIME_NS`, 30 +days. + `Snapshot::open` checks exactly `key.bin`, `snapshot.json` and `openvmm/{manifest.bin,memory.bin,state.bin}`. Directories are private, files are regular and private to their owner, artifact lengths equal the strict @@ -832,7 +869,7 @@ with and what a restore may grow it to; and the caller's `Invariants`, at most 64 settings that NVX keeps without interpreting them, each named by 1 to 64 lowercase ASCII letters, digits, `.`, `-`, and `_` from a letter, with a value of at most 256 printable ASCII characters. `Invariants` and `BlockGeometry` -belong here, and `nvx::host::template` re-exports them where they were. +belong here. `CaptureConfig::check` checks a capture without reading the host. It fails with an `nvx::Error` of kind `Unsupported` for processors at boot or a capacity other @@ -1165,12 +1202,11 @@ orchestrators, which keep the configuration, the authorization, and the application protocol. `launch` runs the store's launch steps under the sandbox's lock, as a launch -whose `LaunchId` the caller reserved beforehand, a fresh one for each launch, or -that a clone's lease reserves: it writes the journal's marker, which names the -launch, spawns OpenVMM with the claimed log, records its `ProcessRef` right -after, and returns a `Starting` launch that the caller commits once the VMM -serves its control endpoint, or abandons. A launch that fails never leaves an -unrecorded VMM running. +whose `LaunchId` the caller reserved beforehand, a fresh one for each launch: it +writes the journal's marker, which names the launch, spawns OpenVMM with the +claimed log, records its `ProcessRef` right after, and returns a `Starting` +launch that the caller commits once the VMM serves its control endpoint, or +abandons. A launch that fails never leaves an unrecorded VMM running. `spawn_vm` starts a caller-owned command without a store, with the same strict Windows log-job requirement as `vm::spawn`: an untrusted job name or a log that @@ -1226,12 +1262,6 @@ identity. after the orchestrator restarts within the same host boot: `Live`, `Mismatched` with another runtime identity, `Exited`, or `Indeterminate`. -A clone's launch takes its locks in one order, the template store's, then the -sandbox's, which guards its clone binding and launch journal, then the scratch -image's claim, and holds none for the VM's lifetime; its lease lasts through -crashes and reattaches, until a release ends it once a stop, a termination, or a -reattach has verified the VM's exit. - #### Windows working-directory limit Windows' process-creation API limits `VmmCommand::working_dir` to 258 UTF-16 @@ -1241,157 +1271,6 @@ refuses an overlong working directory with `InvalidInput`; high-level instance and capture preflight return `Unsupported` with the limit in the top-level message. Neither silently changes the requested directory. -### `nvx::host::template` - -The legacy plans and store of templates, captures of a microVM from which -independent clones are restored. Their metadata names remain aliases of -[`host::snapshot`](#nvxhostsnapshot)'s types. - -`TemplatePlan::new` lowers an adapter's `SandboxPlan` into the plan of a -template's capture, without I/O: the placeholder `distro`, which its -`DistroBinding` binds, and whose length is the capacity of every clone's image, -over a dormant scratch device, the sparse file `SCRATCH_NAME` in the capture's -transaction directory, with the dormant filesystem slot, and no NIC or one at -the guest's address of the adapter's network, under the complete deny-all -policy, whatever the network's own, which each clone's restore replaces with the -clone's; `TemplateResources`, with the processors, 1, 2, 4, or 8, and those -online at boot, as many or fewer, and RAM and a `--memory-capacity` in blocks of -`MEMORY_BLOCK_SIZE`; the kernel command line `nvx_sandbox=1`, with `maxcpus=` -and the processors online at boot when they are fewer, so that the capture's -whole command line holds tokens of OpenVMM's platform allowlist alone, and no -tenant data, which the tests check with a copy of the allowlist's source; and -the capture's options, `--snapshot-tier platform`, a `--snapshot-destination`, -`SNAPSHOT_NAME` in the transaction directory, and -`--snapshot-block-identity generation` with the capture's bound-block -`StorageGeneration`, so that the capture hashes no block device. - -`TemplateManifest::read` reads the `manifest.bin` that OpenVMM wrote for a -capture, at most `MAX_MANIFEST_LEN` bytes of a regular file that is not a link -or other reparse point, and refuses another snapshot format version, an unknown, -mistyped, or repeated field, truncated bytes, and a capture that is not a -template's: a platform-tier capture for clones, with a read-only `distro` whose -identity is unbound, a fresh scratch, and the dormant filesystem slot. - -`CaptureClass` holds the invariant inputs of a capture, known before it: the -runtime, the guest's build, the backend, the requested CPU profile, the -processors, those online at boot, and RAM, the geometries of the disks, the -network, of which it takes what the template captures alone, its NIC at the -guest's address under the complete deny-all policy, and the adapter's other -`Invariants`. - -`TemplateKey::new` checks that the capture made the machine that its class -requested, and derives the template's compatibility key, from metadata alone: -the runtime and contract versions through which its clones are restored, the -backend, CPU profile and effective CPUID, topology, rates, resources, kernel -command line, disks, and devices of its machine, with the NIC's addresses and -the identity of the complete deny-all policy under which the capture attached -it, which each clone's restore replaces with the clone's own, so that a template -with a NIC serves the clones of every network policy at its guest's address. A -capture's own metadata, such as its time anchor and host, stays out of the key. - -A key and a class each have one canonical, versioned encoding, whose -domain-separated SHA-256 digest is its identity, a `KeyIdentity`, which converts -to and from the `nvx::activation::Identity` by which a clone's lease and durable records -name its template's key, or a `ClassIdentity`; the -[conformance vectors](tests/vectors/README.md) of version 1, -`tests/vectors/snapshot-key-v1.txt`, pin both encodings and identities. The -key's encoding, version 1, leaves the network policy out, since templates -capture their NIC under the complete deny-all policy, and has not been released, -so a template store that an earlier build created is not supported: delete it -and create a new one. There is no migration. - -`Store` keeps the generations of templates that a host published, apart from -every sandbox's store, at a location that the adapter chooses: a private -directory with an identity of its own, a `StoreId`, the immutable payload of -each generation, its key's canonical encoding beside the snapshot that OpenVMM -published, and the versioned `Index` of them, which lists each generation with -the identities of its key and capture class and its own metadata, such as its -time contract. - -`Store::create` creates a store, or opens the one that its directory holds, -crash-safely and once between concurrent creations, and refuses a directory with -foreign entries; `Store::open` opens one. - -Its records are JSON in one format, replaced atomically, and records of another -format fail rather than being migrated. Its directories must be private to the -current user, and it refuses links and, on Windows, other reparse points among -its directories and files. - -Four namespaces stay apart: the `nvx::activation::StoreGeneration`, the host's -publication ID, which the index assigns in the order of publication and never -twice, and by which a clone's lease and durable records name its template; the -bound-block `StorageGeneration` of a capture, which belongs to the storage -owner; the time lineage counter of the time contract; and the VM generation ID, -which the store never records. - -`TimeContract::eligibility` decides, apart from compatibility, whether a clone -may restore a capture at a host's time, a `HostTime`, which `HostTime::now` -samples as OpenVMM does: within the boot of the capture, by the host's monotonic -clock, and otherwise by UTC, the downtime must not be negative, and the interval -by which a restore advances the clone's VM time at most `MAX_DOWNTIME_NS`, 30 -days. - -`Store::select` selects the latest eligible generation of a key that no -tombstone quarantines. - -`Store::acquire` leases a generation to a clone in one transaction, under the -store's lock and then the sandbox's: it checks the generation's eligibility, -writes the lease's record in the store and the clone binding, `clone.json`, -beside the sandbox's launch journal, and returns the lease with the sandbox's -lock, the only one under which the caller launches the clone, once, as the -launch that the lease reserves, `Lease::launch`. - -A lease never expires: `Store::release` ends it once the sandbox records no VM -and no launch. - -On Linux, where OpenVMM requires a restore's control socket in the snapshot's -parent, the clone's VMM listens in the generation's payload, at `.sock`, -named after the launch that the lease reserves; on Windows, it listens at a -named pipe, as every launch does. - -A payload holds the sockets of its clones only while they lease it: the VMM -removes its socket as it exits, a stop or a recovery removes the one that an -abrupt exit leaves, and the release makes the socket's absence durable before -the lease ends, and removes the record and `restore.sock`. - -`Store::tombstone` quarantines a generation, which takes no new lease while its -leases keep working, and `Store::retire` deletes it once its leases have -drained, crash-safely, removing the records of leases that no clone binding -names any more. - -`Store::create_template` serves a `Capture`, an adapter's plan for a capture -class with the placeholder's binding and the runtime's artifacts, with a -generation of its class that the store holds and that is eligible at the host's -time, which it reads once it holds the class's slot, `Created::Existing`, or -captures one, `Created::Captured`, while `Store::refresh_template` always -captures, and leaves older generations and their leases untouched. - -A capture runs in its class's slot under `captures/`, a sandbox store whose lock -serializes the class's captures, which comes before the store's lock, and which -keeps the log and outcome report of the VMM of the class's latest capture: in a -private transaction on the store's volume, which is also the sandbox store of -its VM, since OpenVMM requires the control socket of a VM that captures a -snapshot, on Linux, beside the snapshot's destination, it creates the dormant -scratch, checks the placeholder through its binding, and both block devices at -their paths, regular files that are not links or other reparse points, that -belong to the user or an administrator, and that no one else may change, and -launches the template's VM through `supervisor::launch` from the transaction, -holding no handle that could write either file, since OpenVMM opens them itself -by path, as it does a restore's. Once the VMM has exited, it checks the outcome -report, the snapshot's manifest and artifacts, and the key, and publishes the -transaction in two commits, a rename into `generations/` that replaces nothing, -then the index. - -Capture preflight applies the -[Windows working-directory limit](#windows-working-directory-limit) to -`\captures\<16-hex-slot>\transaction` before creating the slot -or scratch. Without a trailing separator this leaves 220 units for the absolute -template-store path, or 210 for a state root whose store is `\templates`. - -A capture that fails publishes nothing, and `Store::recover_captures` ends the -captures that a crash interrupted, publishing those that were checked, and -retiring the others. - ### `nvx::host::seal` `FileSeal`: what the host reports about an open file from its metadata alone, @@ -3271,9 +3150,7 @@ tests, only the test hooks that [Test layout](../doc/contribute.md#test-layout) describes. The tests need no hypervisor. The fake-OpenVMM test drives `Instance` end to end -against `nvx-fake-openvmm`, and the template-capture test runs templates' -captures through the supervisor against it, up to a crash of the adapter whose -VMM a recovery ends. The CLI test runs the +against `nvx-fake-openvmm`. The CLI test runs the [scenario tests](#scenario-tests) of `nvx-rs`, and checks the guest-feature requirement and, on Linux, Ctrl-C. The [conformance vectors](tests/vectors/README.md) of the wire formats cover both diff --git a/nvx-rs/public-api/linux.txt b/nvx-rs/public-api/linux.txt index aeae95369..0bdbcb645 100644 --- a/nvx-rs/public-api/linux.txt +++ b/nvx-rs/public-api/linux.txt @@ -3314,228 +3314,6 @@ fn nvx::host::supervisor::reattach fn nvx::host::supervisor::spawn fn nvx::host::supervisor::spawn_command fn nvx::host::supervisor::spawn_vm -mod nvx::host::template -struct nvx::host::template::Acquired -field nvx::host::template::Acquired::lease -field nvx::host::template::Acquired::locked -struct nvx::host::template::BlockGeometry -field nvx::host::template::BlockGeometry::length -field nvx::host::template::BlockGeometry::logical_block_size -field nvx::host::template::BlockGeometry::physical_block_size -const nvx::host::template::CAPTURES_NAME -const nvx::host::template::CAPTURE_RECORD_NAME -struct nvx::host::template::Capture -field nvx::host::template::Capture::adapter -field nvx::host::template::Capture::artifacts -field nvx::host::template::Capture::binding -field nvx::host::template::Capture::class -field nvx::host::template::Capture::policy_revision -struct nvx::host::template::CaptureClass -const nvx::host::template::CaptureClass::VERSION -field nvx::host::template::CaptureClass::boot_processors -method nvx::host::template::CaptureClass::check -field nvx::host::template::CaptureClass::cpu_profile -field nvx::host::template::CaptureClass::distro -field nvx::host::template::CaptureClass::guest_build -field nvx::host::template::CaptureClass::hypervisor -method nvx::host::template::CaptureClass::identity -field nvx::host::template::CaptureClass::invariants -field nvx::host::template::CaptureClass::memory -field nvx::host::template::CaptureClass::memory_capacity -field nvx::host::template::CaptureClass::network -field nvx::host::template::CaptureClass::processors -field nvx::host::template::CaptureClass::runtime -field nvx::host::template::CaptureClass::scratch -method nvx::host::template::CaptureClass::to_bytes -struct nvx::host::template::ClassIdentity -method nvx::host::template::ClassIdentity::as_bytes -method nvx::host::template::ClassIdentity::from_bytes -enum nvx::host::template::Created -variant nvx::host::template::Created::Captured -field nvx::host::template::Created::Captured::0 -variant nvx::host::template::Created::Existing -field nvx::host::template::Created::Existing::0 -method nvx::host::template::Created::generation -method nvx::host::template::Created::into_generation -struct nvx::host::template::Downtime -field nvx::host::template::Downtime::nanos -field nvx::host::template::Downtime::source -field nvx::host::template::Downtime::vm_time_nanos -enum nvx::host::template::DowntimeSource -variant nvx::host::template::DowntimeSource::HostMonotonic -variant nvx::host::template::DowntimeSource::Utc -enum nvx::host::template::Eligibility -variant nvx::host::template::Eligibility::Early -field nvx::host::template::Eligibility::Early::before_ns -field nvx::host::template::Eligibility::Early::source -variant nvx::host::template::Eligibility::Eligible -field nvx::host::template::Eligibility::Eligible::0 -variant nvx::host::template::Eligibility::Expired -field nvx::host::template::Eligibility::Expired::0 -method nvx::host::template::Eligibility::is_eligible -const nvx::host::template::GENERATIONS_NAME -struct nvx::host::template::Generation -method nvx::host::template::Generation::class -method nvx::host::template::Generation::created_at -method nvx::host::template::Generation::eligibility -method nvx::host::template::Generation::generation -method nvx::host::template::Generation::key -method nvx::host::template::Generation::memory_len -method nvx::host::template::Generation::state_len -method nvx::host::template::Generation::storage_generation -method nvx::host::template::Generation::time -enum nvx::host::template::HostClock -variant nvx::host::template::HostClock::LinuxBoottime -variant nvx::host::template::HostClock::WindowsInterruptTime -method nvx::host::template::HostClock::as_str -struct nvx::host::template::HostTime -field nvx::host::template::HostTime::boot_id -field nvx::host::template::HostTime::clock -field nvx::host::template::HostTime::host_id -field nvx::host::template::HostTime::monotonic_ns -method nvx::host::template::HostTime::now -field nvx::host::template::HostTime::utc_ns -const nvx::host::template::INDEX_LOCK_NAME -const nvx::host::template::INDEX_NAME -struct nvx::host::template::Index -method nvx::host::template::Index::generations -method nvx::host::template::Index::get -method nvx::host::template::Index::next_generation -method nvx::host::template::Index::of_key -struct nvx::host::template::Invariants -const nvx::host::template::Invariants::MAX_NAME_LEN -const nvx::host::template::Invariants::MAX_SETTINGS -const nvx::host::template::Invariants::MAX_VALUE_LEN -method nvx::host::template::Invariants::get -method nvx::host::template::Invariants::is_empty -method nvx::host::template::Invariants::iter -method nvx::host::template::Invariants::len -method nvx::host::template::Invariants::new -method nvx::host::template::Invariants::with -const nvx::host::template::KEY_NAME -struct nvx::host::template::KeyIdentity -method nvx::host::template::KeyIdentity::as_bytes -method nvx::host::template::KeyIdentity::from_bytes -const nvx::host::template::LEASES_NAME -struct nvx::host::template::Lease -method nvx::host::template::Lease::generation -method nvx::host::template::Lease::id -method nvx::host::template::Lease::key -method nvx::host::template::Lease::launch -method nvx::host::template::Lease::of -method nvx::host::template::Lease::sandbox -method nvx::host::template::Lease::store -method nvx::host::template::Lease::template -struct nvx::host::template::LeaseId -method nvx::host::template::LeaseId::as_bytes -method nvx::host::template::LeaseId::from_bytes -const nvx::host::template::MANIFEST_NAME -const nvx::host::template::MAX_DOWNTIME_NS -const nvx::host::template::MAX_GENERATIONS -const nvx::host::template::MAX_MANIFEST_LEN -const nvx::host::template::MEMORY_BLOCK_SIZE -enum nvx::host::template::Retirement -variant nvx::host::template::Retirement::Draining -field nvx::host::template::Retirement::Draining::0 -variant nvx::host::template::Retirement::Retired -const nvx::host::template::SCRATCH_NAME -const nvx::host::template::SNAPSHOT_NAME -const nvx::host::template::STORE_FORMAT -const nvx::host::template::STORE_RECORD_NAME -struct nvx::host::template::StorageGeneration -method nvx::host::template::StorageGeneration::as_bytes -method nvx::host::template::StorageGeneration::from_bytes -method nvx::host::template::StorageGeneration::generate -struct nvx::host::template::Store -method nvx::host::template::Store::acquire -method nvx::host::template::Store::create -method nvx::host::template::Store::create_template -method nvx::host::template::Store::dir -method nvx::host::template::Store::id -method nvx::host::template::Store::index -method nvx::host::template::Store::leases -method nvx::host::template::Store::open -method nvx::host::template::Store::payload_dir -method nvx::host::template::Store::read_key -method nvx::host::template::Store::recover_captures -method nvx::host::template::Store::refresh_template -method nvx::host::template::Store::release -method nvx::host::template::Store::retire -method nvx::host::template::Store::select -method nvx::host::template::Store::snapshot_dir -method nvx::host::template::Store::tombstone -method nvx::host::template::Store::tombstones -struct nvx::host::template::StoreId -method nvx::host::template::StoreId::as_bytes -method nvx::host::template::StoreId::from_bytes -const nvx::host::template::TOMBSTONES_NAME -const nvx::host::template::TRANSACTION_NAME -struct nvx::host::template::TemplateKey -const nvx::host::template::TemplateKey::MAX_LEN -const nvx::host::template::TemplateKey::VERSION -method nvx::host::template::TemplateKey::boot_processors -method nvx::host::template::TemplateKey::command_line -method nvx::host::template::TemplateKey::cpu_profile -method nvx::host::template::TemplateKey::distro -method nvx::host::template::TemplateKey::filesystem_tag -method nvx::host::template::TemplateKey::from_bytes -method nvx::host::template::TemplateKey::guest_network -method nvx::host::template::TemplateKey::has_network -method nvx::host::template::TemplateKey::hypervisor -method nvx::host::template::TemplateKey::identity -method nvx::host::template::TemplateKey::invariants -method nvx::host::template::TemplateKey::memory -method nvx::host::template::TemplateKey::memory_block_size -method nvx::host::template::TemplateKey::memory_capacity -method nvx::host::template::TemplateKey::new -method nvx::host::template::TemplateKey::processors -method nvx::host::template::TemplateKey::runtime -method nvx::host::template::TemplateKey::scratch -method nvx::host::template::TemplateKey::speaks_current_contracts -method nvx::host::template::TemplateKey::time -method nvx::host::template::TemplateKey::to_bytes -struct nvx::host::template::TemplateManifest -method nvx::host::template::TemplateManifest::created_at -method nvx::host::template::TemplateManifest::hypervisor -method nvx::host::template::TemplateManifest::memory_len -method nvx::host::template::TemplateManifest::openvmm_version -method nvx::host::template::TemplateManifest::parse -method nvx::host::template::TemplateManifest::read -method nvx::host::template::TemplateManifest::state_len -method nvx::host::template::TemplateManifest::time -struct nvx::host::template::TemplatePlan -method nvx::host::template::TemplatePlan::binding -method nvx::host::template::TemplatePlan::capture_arguments -method nvx::host::template::TemplatePlan::destination -method nvx::host::template::TemplatePlan::generation -method nvx::host::template::TemplatePlan::image_capacity -method nvx::host::template::TemplatePlan::kernel_command_line -method nvx::host::template::TemplatePlan::new -method nvx::host::template::TemplatePlan::nic_arguments -method nvx::host::template::TemplatePlan::openvmm_arguments -method nvx::host::template::TemplatePlan::plan -method nvx::host::template::TemplatePlan::resources -method nvx::host::template::TemplatePlan::scratch_length -method nvx::host::template::TemplatePlan::scratch_path -method nvx::host::template::TemplatePlan::transaction -struct nvx::host::template::TemplateResources -field nvx::host::template::TemplateResources::boot_processors -field nvx::host::template::TemplateResources::memory -field nvx::host::template::TemplateResources::memory_capacity -field nvx::host::template::TemplateResources::processors -struct nvx::host::template::TimeContract -field nvx::host::template::TimeContract::apic_frequency_hz -field nvx::host::template::TimeContract::boot_id -field nvx::host::template::TimeContract::capture_monotonic_ns -field nvx::host::template::TimeContract::capture_tsc -field nvx::host::template::TimeContract::capture_utc_ns -field nvx::host::template::TimeContract::cut_to_anchor_ns -method nvx::host::template::TimeContract::eligibility -field nvx::host::template::TimeContract::host_clock -field nvx::host::template::TimeContract::host_id -field nvx::host::template::TimeContract::lineage -field nvx::host::template::TimeContract::tsc_frequency_hz -field nvx::host::template::TimeContract::tsc_tolerance_ppm mod nvx::host::transport struct nvx::host::transport::HostConnection method nvx::host::transport::HostConnection::attach diff --git a/nvx-rs/public-api/windows.txt b/nvx-rs/public-api/windows.txt index 1f5e64aff..20edbeac3 100644 --- a/nvx-rs/public-api/windows.txt +++ b/nvx-rs/public-api/windows.txt @@ -1916,228 +1916,6 @@ fn nvx::host::supervisor::reattach fn nvx::host::supervisor::spawn fn nvx::host::supervisor::spawn_command fn nvx::host::supervisor::spawn_vm -mod nvx::host::template -struct nvx::host::template::Acquired -field nvx::host::template::Acquired::lease -field nvx::host::template::Acquired::locked -struct nvx::host::template::BlockGeometry -field nvx::host::template::BlockGeometry::length -field nvx::host::template::BlockGeometry::logical_block_size -field nvx::host::template::BlockGeometry::physical_block_size -const nvx::host::template::CAPTURES_NAME -const nvx::host::template::CAPTURE_RECORD_NAME -struct nvx::host::template::Capture -field nvx::host::template::Capture::adapter -field nvx::host::template::Capture::artifacts -field nvx::host::template::Capture::binding -field nvx::host::template::Capture::class -field nvx::host::template::Capture::policy_revision -struct nvx::host::template::CaptureClass -const nvx::host::template::CaptureClass::VERSION -field nvx::host::template::CaptureClass::boot_processors -method nvx::host::template::CaptureClass::check -field nvx::host::template::CaptureClass::cpu_profile -field nvx::host::template::CaptureClass::distro -field nvx::host::template::CaptureClass::guest_build -field nvx::host::template::CaptureClass::hypervisor -method nvx::host::template::CaptureClass::identity -field nvx::host::template::CaptureClass::invariants -field nvx::host::template::CaptureClass::memory -field nvx::host::template::CaptureClass::memory_capacity -field nvx::host::template::CaptureClass::network -field nvx::host::template::CaptureClass::processors -field nvx::host::template::CaptureClass::runtime -field nvx::host::template::CaptureClass::scratch -method nvx::host::template::CaptureClass::to_bytes -struct nvx::host::template::ClassIdentity -method nvx::host::template::ClassIdentity::as_bytes -method nvx::host::template::ClassIdentity::from_bytes -enum nvx::host::template::Created -variant nvx::host::template::Created::Captured -field nvx::host::template::Created::Captured::0 -variant nvx::host::template::Created::Existing -field nvx::host::template::Created::Existing::0 -method nvx::host::template::Created::generation -method nvx::host::template::Created::into_generation -struct nvx::host::template::Downtime -field nvx::host::template::Downtime::nanos -field nvx::host::template::Downtime::source -field nvx::host::template::Downtime::vm_time_nanos -enum nvx::host::template::DowntimeSource -variant nvx::host::template::DowntimeSource::HostMonotonic -variant nvx::host::template::DowntimeSource::Utc -enum nvx::host::template::Eligibility -variant nvx::host::template::Eligibility::Early -field nvx::host::template::Eligibility::Early::before_ns -field nvx::host::template::Eligibility::Early::source -variant nvx::host::template::Eligibility::Eligible -field nvx::host::template::Eligibility::Eligible::0 -variant nvx::host::template::Eligibility::Expired -field nvx::host::template::Eligibility::Expired::0 -method nvx::host::template::Eligibility::is_eligible -const nvx::host::template::GENERATIONS_NAME -struct nvx::host::template::Generation -method nvx::host::template::Generation::class -method nvx::host::template::Generation::created_at -method nvx::host::template::Generation::eligibility -method nvx::host::template::Generation::generation -method nvx::host::template::Generation::key -method nvx::host::template::Generation::memory_len -method nvx::host::template::Generation::state_len -method nvx::host::template::Generation::storage_generation -method nvx::host::template::Generation::time -enum nvx::host::template::HostClock -variant nvx::host::template::HostClock::LinuxBoottime -variant nvx::host::template::HostClock::WindowsInterruptTime -method nvx::host::template::HostClock::as_str -struct nvx::host::template::HostTime -field nvx::host::template::HostTime::boot_id -field nvx::host::template::HostTime::clock -field nvx::host::template::HostTime::host_id -field nvx::host::template::HostTime::monotonic_ns -method nvx::host::template::HostTime::now -field nvx::host::template::HostTime::utc_ns -const nvx::host::template::INDEX_LOCK_NAME -const nvx::host::template::INDEX_NAME -struct nvx::host::template::Index -method nvx::host::template::Index::generations -method nvx::host::template::Index::get -method nvx::host::template::Index::next_generation -method nvx::host::template::Index::of_key -struct nvx::host::template::Invariants -const nvx::host::template::Invariants::MAX_NAME_LEN -const nvx::host::template::Invariants::MAX_SETTINGS -const nvx::host::template::Invariants::MAX_VALUE_LEN -method nvx::host::template::Invariants::get -method nvx::host::template::Invariants::is_empty -method nvx::host::template::Invariants::iter -method nvx::host::template::Invariants::len -method nvx::host::template::Invariants::new -method nvx::host::template::Invariants::with -const nvx::host::template::KEY_NAME -struct nvx::host::template::KeyIdentity -method nvx::host::template::KeyIdentity::as_bytes -method nvx::host::template::KeyIdentity::from_bytes -const nvx::host::template::LEASES_NAME -struct nvx::host::template::Lease -method nvx::host::template::Lease::generation -method nvx::host::template::Lease::id -method nvx::host::template::Lease::key -method nvx::host::template::Lease::launch -method nvx::host::template::Lease::of -method nvx::host::template::Lease::sandbox -method nvx::host::template::Lease::store -method nvx::host::template::Lease::template -struct nvx::host::template::LeaseId -method nvx::host::template::LeaseId::as_bytes -method nvx::host::template::LeaseId::from_bytes -const nvx::host::template::MANIFEST_NAME -const nvx::host::template::MAX_DOWNTIME_NS -const nvx::host::template::MAX_GENERATIONS -const nvx::host::template::MAX_MANIFEST_LEN -const nvx::host::template::MEMORY_BLOCK_SIZE -enum nvx::host::template::Retirement -variant nvx::host::template::Retirement::Draining -field nvx::host::template::Retirement::Draining::0 -variant nvx::host::template::Retirement::Retired -const nvx::host::template::SCRATCH_NAME -const nvx::host::template::SNAPSHOT_NAME -const nvx::host::template::STORE_FORMAT -const nvx::host::template::STORE_RECORD_NAME -struct nvx::host::template::StorageGeneration -method nvx::host::template::StorageGeneration::as_bytes -method nvx::host::template::StorageGeneration::from_bytes -method nvx::host::template::StorageGeneration::generate -struct nvx::host::template::Store -method nvx::host::template::Store::acquire -method nvx::host::template::Store::create -method nvx::host::template::Store::create_template -method nvx::host::template::Store::dir -method nvx::host::template::Store::id -method nvx::host::template::Store::index -method nvx::host::template::Store::leases -method nvx::host::template::Store::open -method nvx::host::template::Store::payload_dir -method nvx::host::template::Store::read_key -method nvx::host::template::Store::recover_captures -method nvx::host::template::Store::refresh_template -method nvx::host::template::Store::release -method nvx::host::template::Store::retire -method nvx::host::template::Store::select -method nvx::host::template::Store::snapshot_dir -method nvx::host::template::Store::tombstone -method nvx::host::template::Store::tombstones -struct nvx::host::template::StoreId -method nvx::host::template::StoreId::as_bytes -method nvx::host::template::StoreId::from_bytes -const nvx::host::template::TOMBSTONES_NAME -const nvx::host::template::TRANSACTION_NAME -struct nvx::host::template::TemplateKey -const nvx::host::template::TemplateKey::MAX_LEN -const nvx::host::template::TemplateKey::VERSION -method nvx::host::template::TemplateKey::boot_processors -method nvx::host::template::TemplateKey::command_line -method nvx::host::template::TemplateKey::cpu_profile -method nvx::host::template::TemplateKey::distro -method nvx::host::template::TemplateKey::filesystem_tag -method nvx::host::template::TemplateKey::from_bytes -method nvx::host::template::TemplateKey::guest_network -method nvx::host::template::TemplateKey::has_network -method nvx::host::template::TemplateKey::hypervisor -method nvx::host::template::TemplateKey::identity -method nvx::host::template::TemplateKey::invariants -method nvx::host::template::TemplateKey::memory -method nvx::host::template::TemplateKey::memory_block_size -method nvx::host::template::TemplateKey::memory_capacity -method nvx::host::template::TemplateKey::new -method nvx::host::template::TemplateKey::processors -method nvx::host::template::TemplateKey::runtime -method nvx::host::template::TemplateKey::scratch -method nvx::host::template::TemplateKey::speaks_current_contracts -method nvx::host::template::TemplateKey::time -method nvx::host::template::TemplateKey::to_bytes -struct nvx::host::template::TemplateManifest -method nvx::host::template::TemplateManifest::created_at -method nvx::host::template::TemplateManifest::hypervisor -method nvx::host::template::TemplateManifest::memory_len -method nvx::host::template::TemplateManifest::openvmm_version -method nvx::host::template::TemplateManifest::parse -method nvx::host::template::TemplateManifest::read -method nvx::host::template::TemplateManifest::state_len -method nvx::host::template::TemplateManifest::time -struct nvx::host::template::TemplatePlan -method nvx::host::template::TemplatePlan::binding -method nvx::host::template::TemplatePlan::capture_arguments -method nvx::host::template::TemplatePlan::destination -method nvx::host::template::TemplatePlan::generation -method nvx::host::template::TemplatePlan::image_capacity -method nvx::host::template::TemplatePlan::kernel_command_line -method nvx::host::template::TemplatePlan::new -method nvx::host::template::TemplatePlan::nic_arguments -method nvx::host::template::TemplatePlan::openvmm_arguments -method nvx::host::template::TemplatePlan::plan -method nvx::host::template::TemplatePlan::resources -method nvx::host::template::TemplatePlan::scratch_length -method nvx::host::template::TemplatePlan::scratch_path -method nvx::host::template::TemplatePlan::transaction -struct nvx::host::template::TemplateResources -field nvx::host::template::TemplateResources::boot_processors -field nvx::host::template::TemplateResources::memory -field nvx::host::template::TemplateResources::memory_capacity -field nvx::host::template::TemplateResources::processors -struct nvx::host::template::TimeContract -field nvx::host::template::TimeContract::apic_frequency_hz -field nvx::host::template::TimeContract::boot_id -field nvx::host::template::TimeContract::capture_monotonic_ns -field nvx::host::template::TimeContract::capture_tsc -field nvx::host::template::TimeContract::capture_utc_ns -field nvx::host::template::TimeContract::cut_to_anchor_ns -method nvx::host::template::TimeContract::eligibility -field nvx::host::template::TimeContract::host_clock -field nvx::host::template::TimeContract::host_id -field nvx::host::template::TimeContract::lineage -field nvx::host::template::TimeContract::tsc_frequency_hz -field nvx::host::template::TimeContract::tsc_tolerance_ppm mod nvx::host::transport struct nvx::host::transport::HostConnection method nvx::host::transport::HostConnection::attach diff --git a/nvx-rs/src/contract.rs b/nvx-rs/src/contract.rs index 666a817ca..33304cfb9 100644 --- a/nvx-rs/src/contract.rs +++ b/nvx-rs/src/contract.rs @@ -27,7 +27,7 @@ //! A release declares the microVM ABI, time ABI, restore-packet, control-session, and control //! contract versions of its artifacts in its `SOURCE-MANIFEST.json`. //! -//! The compatibility key of a template, `nvx::host::template::TemplateKey`, holds the boot and +//! The compatibility key of a template, `nvx::host::snapshot::SnapshotKey`, holds the boot and //! control versions, and the versions of the contracts through which a template's clones are //! restored, which no runtime identity records: //! diff --git a/nvx-rs/src/host/mod.rs b/nvx-rs/src/host/mod.rs index cecb93c0f..8ec3e1251 100644 --- a/nvx-rs/src/host/mod.rs +++ b/nvx-rs/src/host/mod.rs @@ -32,9 +32,6 @@ //! - [`snapshot`] holds the compatibility key, [`SnapshotKey`](snapshot::SnapshotKey), derived //! from a [`CaptureClass`](snapshot::CaptureClass) and [`SnapshotManifest`](snapshot::SnapshotManifest), //! the strict manifest reader, and the capture's public time and eligibility information. -//! - [`template`] retains the legacy capture and clone plans, the [`Store`](template::Store) of -//! published generations and their leases, and aliases of the snapshot metadata. Its -//! [`TemplatePlan`](template::TemplatePlan) lowers an adapter's plan for a capture. //! - [`session`] speaks NVX's control protocol, `NVXC`, over [`transport`], which verifies a VM's //! endpoint and runs the host's role of the NVXS session for it and for any other application //! protocol, over a platform transport. [`nvxs`] retains compatibility paths. [`platform`] holds @@ -105,7 +102,6 @@ pub mod supervisor; #[cfg(unix)] #[allow(unsafe_code)] mod sys; -pub mod template; #[cfg(test)] pub(crate) mod tests; pub mod transport; diff --git a/nvx-rs/src/host/platform/linux/identity.rs b/nvx-rs/src/host/platform/linux/identity.rs index e32d76604..c9c39b159 100644 --- a/nvx-rs/src/host/platform/linux/identity.rs +++ b/nvx-rs/src/host/platform/linux/identity.rs @@ -7,9 +7,7 @@ use std::os::unix::fs::{MetadataExt, OpenOptionsExt}; use std::path::Path; use nix::sys::stat::Mode; -use nix::unistd::Uid; -use super::private::{owner_and_mode, store_dir_problem}; use crate::host::sys::Errno; /// Returns the device and inode of a file or directory. @@ -168,13 +166,4 @@ impl Directory { pub(crate) fn sync(&self) -> io::Result<()> { self.0.sync_all() } - - /// Returns why the directory cannot hold a sandbox store, as [`check_store_dir`] returns it for - /// a path, if it cannot: unless it belongs to the current user with mode 0700. - /// - /// [`check_store_dir`]: crate::host::platform::linux::check_store_dir - pub(crate) fn store_dir_problem(&self) -> io::Result> { - let (uid, mode) = owner_and_mode(&self.0.metadata()?); - Ok(store_dir_problem(uid, mode, Uid::effective())) - } } diff --git a/nvx-rs/src/host/platform/linux/mod.rs b/nvx-rs/src/host/platform/linux/mod.rs index ae1721964..69fd736a1 100644 --- a/nvx-rs/src/host/platform/linux/mod.rs +++ b/nvx-rs/src/host/platform/linux/mod.rs @@ -27,9 +27,10 @@ pub(crate) use self::identity::{ }; pub(crate) use self::log_claim::{claim_log, log_claim_held, serves_launch}; pub use self::private::create_private_dir; +#[cfg(feature = "cli")] +pub(crate) use self::private::rename_no_replace; pub(crate) use self::private::{ - check_owner_only_dir, check_snapshot_dir, check_store_dir, create_owner_only_dir, - create_store_dir, foreign_writer_problem, make_sparse, private_file_problem, rename_no_replace, + check_snapshot_dir, check_store_dir, create_store_dir, make_sparse, private_file_problem, restore_private_file_mode, }; pub use self::process::process_start_time; diff --git a/nvx-rs/src/host/platform/linux/private.rs b/nvx-rs/src/host/platform/linux/private.rs index b14ee37f7..6aee426bc 100644 --- a/nvx-rs/src/host/platform/linux/private.rs +++ b/nvx-rs/src/host/platform/linux/private.rs @@ -12,7 +12,9 @@ use nix::sys::stat::Mode; use nix::unistd::Uid; use super::{GROUP_AND_OTHER_BITS, PRIVATE_DIR_MODE, STORE_DIR_MODE}; -use crate::host::sys::{self, Errno}; +#[cfg(feature = "cli")] +use crate::host::sys; +use crate::host::sys::Errno; /// Creates `path` if needed and restricts it to the current user. pub fn create_private_dir(path: &Path) -> io::Result<()> { @@ -44,18 +46,6 @@ pub(crate) fn create_store_dir(path: &Path) -> io::Result> { check_store_dir(path) } -/// Creates the directory `path` with mode 0700, or checks that an existing one is private to the -/// current user, as [`check_owner_only_dir`] does, and returns why it is not, if it is not. -/// -/// A new directory takes the setgid bit of its parent, and loses the bits that the umask clears, -/// even the current user's permission to read it, so its mode is set again. An existing directory -/// is never changed: others could have placed files in it, or still hold them open, which a -/// change of its mode would not revoke. -pub(crate) fn create_owner_only_dir(path: &Path) -> io::Result> { - create_dir_of_mode(path, PRIVATE_DIR_MODE)?; - check_owner_only_dir(path) -} - /// Creates the directory `path`, whose parent must exist, with exactly the mode `mode`, unless it /// exists already, which leaves it as it is. fn create_dir_of_mode(path: &Path, mode: Mode) -> io::Result<()> { @@ -101,8 +91,7 @@ pub(crate) fn check_store_dir(path: &Path) -> io::Result> { } /// Returns why the directory `path` is not private to the current user, if it is not: unless it -/// is a directory, not a link to one, that belongs to the user and grants no one else any access, -/// which [`create_owner_only_dir`] makes it. +/// is a directory, not a link to one, that belongs to the user and grants no one else any access. pub(crate) fn check_owner_only_dir(path: &Path) -> io::Result> { let Some(directory) = open_plain_dir(path)? else { return Ok(Some("is not a plain directory".to_owned())); @@ -155,28 +144,6 @@ pub(crate) fn check_snapshot_dir(path: &Path) -> io::Result> { check_owner_only_dir(path) } -/// Returns why another user than the current one, or root, may change the file that `file` holds -/// open, if one may: unless it belongs to the current user or to root, and grants neither its -/// group nor others the permission to write it. -pub(crate) fn foreign_writer_problem(file: &File) -> io::Result> { - let (uid, mode) = owner_and_mode(&file.metadata()?); - Ok(foreign_writer(uid, mode, Uid::effective())) -} - -/// Returns why another user than `euid`, or root, may change a file that the user `uid` owns with -/// the permission bits `mode`, if one may. -pub(crate) fn foreign_writer(uid: Uid, mode: Mode, euid: Uid) -> Option { - if uid != euid && !uid.is_root() { - Some(format!( - "belongs to user {uid}, neither the current user {euid} nor root" - )) - } else if mode.intersects(Mode::S_IWGRP | Mode::S_IWOTH) { - Some(format!("is writable by other users (mode {mode:03o})")) - } else { - None - } -} - /// Lets the regular file that `file` holds open for writing take no space for the blocks that /// nobody writes: nothing to do on Linux, whose filesystems allocate no block that a file's /// extension leaves unwritten. @@ -187,6 +154,7 @@ pub(crate) fn make_sparse(_file: &File) -> io::Result<()> { /// Renames `from` to `to` unless `to` exists, which it leaves as it is, failing with /// [`io::ErrorKind::AlreadyExists`]: `renameat2` with `RENAME_NOREPLACE`, which a filesystem that /// cannot rename without replacing refuses. +#[cfg(feature = "cli")] pub(crate) fn rename_no_replace(from: &Path, to: &Path) -> io::Result<()> { sys::rename_no_replace(from, to) } diff --git a/nvx-rs/src/host/platform/linux/tests.rs b/nvx-rs/src/host/platform/linux/tests.rs index 7a3029400..ed5a84da3 100644 --- a/nvx-rs/src/host/platform/linux/tests.rs +++ b/nvx-rs/src/host/platform/linux/tests.rs @@ -7,7 +7,7 @@ use std::thread; use std::time::{Duration, Instant}; use super::endpoint::{UnixTransport, endpoint_server_pid_until}; -pub(crate) use super::private::{owner_only_problem, store_dir_problem}; +pub(crate) use super::private::store_dir_problem; use super::process::wait_step; pub(crate) use super::process::{check_identity, start_time_from_stat}; use super::*; diff --git a/nvx-rs/src/host/platform/other.rs b/nvx-rs/src/host/platform/other.rs index c08450731..928518f46 100644 --- a/nvx-rs/src/host/platform/other.rs +++ b/nvx-rs/src/host/platform/other.rs @@ -183,11 +183,6 @@ pub(crate) fn check_store_dir(_path: &Path) -> io::Result> { Err(unsupported()) } -/// Fails: this host cannot check who may access a directory. -pub(crate) fn create_owner_only_dir(_path: &Path) -> io::Result> { - Err(unsupported()) -} - /// Fails: this host cannot check who may access a directory. pub(crate) fn check_owner_only_dir(_path: &Path) -> io::Result> { Err(unsupported()) @@ -203,17 +198,13 @@ pub(crate) fn check_snapshot_dir(path: &Path) -> io::Result> { check_owner_only_dir(path) } -/// Fails: this host cannot check who may change a file. -pub(crate) fn foreign_writer_problem(_file: &File) -> io::Result> { - Err(unsupported()) -} - /// Fails: this host makes no template's files. pub(crate) fn make_sparse(_file: &File) -> io::Result<()> { Err(unsupported()) } /// Fails: this host makes no template's files. +#[cfg(feature = "cli")] pub(crate) fn rename_no_replace(_from: &Path, _to: &Path) -> io::Result<()> { Err(unsupported()) } diff --git a/nvx-rs/src/host/platform/tests.rs b/nvx-rs/src/host/platform/tests.rs index 8cff9329f..0d8aa0abc 100644 --- a/nvx-rs/src/host/platform/tests.rs +++ b/nvx-rs/src/host/platform/tests.rs @@ -1,7 +1,5 @@ #[cfg(target_os = "linux")] -pub(crate) use super::linux::tests::{ - check_identity, owner_only_problem, start_time_from_stat, store_dir_problem, -}; +pub(crate) use super::linux::tests::{check_identity, start_time_from_stat, store_dir_problem}; #[cfg(all(windows, feature = "cli"))] pub(crate) use super::windows::tests::disconnect_control; #[cfg(windows)] diff --git a/nvx-rs/src/host/platform/windows/mod.rs b/nvx-rs/src/host/platform/windows/mod.rs index 52c5ccaed..bf18bc295 100644 --- a/nvx-rs/src/host/platform/windows/mod.rs +++ b/nvx-rs/src/host/platform/windows/mod.rs @@ -37,10 +37,11 @@ pub(crate) use self::identity::{ pub(crate) use self::interrupt::{interrupted, watch_interrupts}; pub(crate) use self::log_claim::{claim_log, create_log_job, log_claim_held, serves_launch}; pub use self::private::create_private_dir; +#[cfg(feature = "cli")] +pub(crate) use self::private::rename_no_replace; pub(crate) use self::private::{ - check_owner_only_dir, check_snapshot_dir, check_store_dir, check_vm_dir, create_owner_only_dir, - create_store_dir, create_vm_dir, foreign_writer_problem, make_sparse, private_file_problem, - rename_no_replace, + check_snapshot_dir, check_store_dir, check_vm_dir, create_store_dir, create_vm_dir, + make_sparse, private_file_problem, }; pub use self::process::process_start_time; pub(crate) use self::process::{ diff --git a/nvx-rs/src/host/platform/windows/private.rs b/nvx-rs/src/host/platform/windows/private.rs index 04d93414a..8fea4be75 100644 --- a/nvx-rs/src/host/platform/windows/private.rs +++ b/nvx-rs/src/host/platform/windows/private.rs @@ -9,7 +9,7 @@ use std::os::windows::io::AsRawHandle; use std::path::Path; use std::{mem, ptr}; -use windows_sys::Win32::Foundation::{ERROR_ALREADY_EXISTS, GENERIC_ALL, GENERIC_WRITE, HANDLE}; +use windows_sys::Win32::Foundation::{ERROR_ALREADY_EXISTS, GENERIC_ALL, HANDLE}; use windows_sys::Win32::Security::{ ACCESS_ALLOWED_ACE, ACE_HEADER, ACL, ACL_REVISION, ACL_SIZE_INFORMATION, AclSizeInformation, AddAccessAllowedAceEx, CONTAINER_INHERIT_ACE, DACL_SECURITY_INFORMATION, EqualSid, GetAce, @@ -21,11 +21,11 @@ use windows_sys::Win32::Security::{ SetSecurityDescriptorDacl, SetSecurityDescriptorOwner, TOKEN_QUERY, TOKEN_USER, TokenUser, WinBuiltinAdministratorsSid, WinCreatorOwnerRightsSid, WinCreatorOwnerSid, WinLocalSystemSid, }; +#[cfg(feature = "cli")] +use windows_sys::Win32::Storage::FileSystem::MoveFileExW; use windows_sys::Win32::Storage::FileSystem::{ - CreateDirectoryW, DELETE, FILE_ALL_ACCESS, FILE_APPEND_DATA, FILE_ATTRIBUTE_DIRECTORY, - FILE_ATTRIBUTE_REPARSE_POINT, FILE_DELETE_CHILD, FILE_FLAG_BACKUP_SEMANTICS, - FILE_FLAG_OPEN_REPARSE_POINT, FILE_WRITE_ATTRIBUTES, FILE_WRITE_DATA, FILE_WRITE_EA, - MoveFileExW, READ_CONTROL, WRITE_DAC, WRITE_OWNER, + CreateDirectoryW, FILE_ALL_ACCESS, FILE_ATTRIBUTE_DIRECTORY, FILE_ATTRIBUTE_REPARSE_POINT, + FILE_FLAG_BACKUP_SEMANTICS, FILE_FLAG_OPEN_REPARSE_POINT, READ_CONTROL, }; use windows_sys::Win32::System::IO::DeviceIoControl; use windows_sys::Win32::System::Threading::{GetCurrentProcess, OpenProcessToken}; @@ -198,21 +198,6 @@ pub(crate) fn check_store_dir(path: &Path) -> io::Result> { store_dir_problem(path, &CurrentUser::get()?) } -/// Creates the directory `path` private to the current user, or checks that an existing one is, -/// as [`create_store_dir`] does for the directory of a sandbox store. -pub(crate) fn create_owner_only_dir(path: &Path) -> io::Result> { - create_store_dir(path) -} - -/// Returns why the directory `path` is not private to the current user, if it is not, as -/// [`check_store_dir`] does for the directory of a sandbox store: unless it is a directory, not a -/// link or any other reparse point, that belongs to the user, or to SYSTEM or the Administrators -/// group, whose access control list allows no one else, and passes entries on to the files -/// created in it. -pub(crate) fn check_owner_only_dir(path: &Path) -> io::Result> { - check_store_dir(path) -} - /// Returns why the directory `path` cannot hold a sandbox store of `user`, if it cannot. fn store_dir_problem(path: &Path, user: &CurrentUser) -> io::Result> { plain_dir_problem(path, user, Required::PrivateDirectory) @@ -241,9 +226,9 @@ fn plain_dir_problem( /// current user, if it is not: unless it is a directory, not a link or any other reparse point, /// that belongs to the user, or to SYSTEM or the Administrators group, whose access control list /// is present, holds only entries that allow or deny access, and allows no one but those and the -/// owner's aliases, as [`check_owner_only_dir`] requires. +/// owner's aliases, as [`check_store_dir`] requires. /// -/// Unlike [`check_owner_only_dir`], it does not require the directory to pass entries on to the +/// Unlike [`check_store_dir`], it does not require the directory to pass entries on to the /// files created in it: OpenVMM gives the directory, and each file that it publishes in it, a /// protected list of their own, which allows SYSTEM and the owner, and the directory's own list /// already stops anyone else from creating a file in it. @@ -374,28 +359,6 @@ pub(crate) fn private_file_problem(file: &fs::File) -> io::Result privacy_problem(file, &CurrentUser::get()?, Required::Private) } -/// Returns why another user than the current one, SYSTEM, or the Administrators group may change -/// the file that `file` holds open, if one may: unless it belongs to the user, or to SYSTEM or the -/// Administrators group, and its access control list allows no one else to write or extend it, to -/// change its attributes, its list, or its owner, or to delete it. Others may read it. -pub(crate) fn foreign_writer_problem(file: &fs::File) -> io::Result> { - privacy_problem(file, &CurrentUser::get()?, Required::Unwritable) -} - -/// The rights of an access control entry that change a file or directory: write or extend it, or -/// its extended attributes, delete it or its children, change its attributes, its list, or its -/// owner, and the generic rights that include those. -const CHANGE_RIGHTS: u32 = FILE_WRITE_DATA - | FILE_APPEND_DATA - | FILE_WRITE_EA - | FILE_DELETE_CHILD - | FILE_WRITE_ATTRIBUTES - | DELETE - | WRITE_DAC - | WRITE_OWNER - | GENERIC_WRITE - | GENERIC_ALL; - /// What [`privacy_problem`] requires of the access control list of a file or directory, whose /// owner the user must trust in any case. #[derive(Clone, Copy, PartialEq, Eq)] @@ -405,8 +368,6 @@ pub(super) enum Required { /// That it allow no one else any access, and pass entries on to the files created in it, as /// a directory of a store must. PrivateDirectory, - /// That it allow no one else any of the [`CHANGE_RIGHTS`]. - Unwritable, } /// Returns why the file, directory, or other object that `object` holds open is not private to @@ -478,11 +439,6 @@ pub(super) fn privacy_problem( ACCESS_DENIED_ACE_TYPE => {} ACCESS_ALLOWED_ACE_TYPE => { let entry = ace.cast::(); - // SAFETY: an entry that allows access starts as an ACCESS_ALLOWED_ACE. - let mask = unsafe { (*entry).Mask }; - if required == Required::Unwritable && mask & CHANGE_RIGHTS == 0 { - continue; - } // SAFETY: an entry that allows access holds its identifier from `SidStart` on. let sid: PSID = unsafe { (&raw mut (*entry).SidStart).cast() }; // The creator owner and the owner rights stand for an owner: the owner, or the @@ -493,13 +449,7 @@ pub(super) fn privacy_problem( || IsWellKnownSid(sid, WinCreatorOwnerRightsSid) != 0 }; if !owner_alias && !user.trusts(sid) { - return Ok(Some( - match required { - Required::Unwritable => "allows other users to change it", - Required::Private | Required::PrivateDirectory => "allows other users", - } - .to_owned(), - )); + return Ok(Some("allows other users".to_owned())); } } other => { @@ -547,6 +497,7 @@ pub(crate) fn make_sparse(file: &fs::File) -> io::Result<()> { /// Renames `from` to `to` unless `to` exists, which it leaves as it is, failing with /// [`io::ErrorKind::AlreadyExists`]: `MoveFileExW` without `MOVEFILE_REPLACE_EXISTING`. +#[cfg(feature = "cli")] pub(crate) fn rename_no_replace(from: &Path, to: &Path) -> io::Result<()> { let (from, to) = (wide_path(from)?, wide_path(to)?); // SAFETY: both names are NUL-terminated. diff --git a/nvx-rs/src/host/platform/windows/private/tests.rs b/nvx-rs/src/host/platform/windows/private/tests.rs index 25270b46b..ef833c33e 100644 --- a/nvx-rs/src/host/platform/windows/private/tests.rs +++ b/nvx-rs/src/host/platform/windows/private/tests.rs @@ -41,6 +41,7 @@ fn private_directories_support_long_paths() { } } +#[cfg(feature = "cli")] #[test] fn renames_support_long_paths_without_replacing_existing_files() { let (_root, parent) = long_directory(); @@ -60,6 +61,7 @@ fn renames_support_long_paths_without_replacing_existing_files() { assert_eq!(fs::read(&destination).unwrap(), b"original"); } +#[cfg(feature = "cli")] #[test] fn invalid_paths_do_not_create_or_rename_their_prefix() { let root = tempfile::tempdir().unwrap(); diff --git a/nvx-rs/src/host/restore/tests/live_init.rs b/nvx-rs/src/host/restore/tests/live_init.rs index 4485da931..0bf9c4361 100644 --- a/nvx-rs/src/host/restore/tests/live_init.rs +++ b/nvx-rs/src/host/restore/tests/live_init.rs @@ -15,8 +15,8 @@ use crate::activation::{ use crate::host::instance::generate_capability; use crate::host::nvxs::HostConnection; use crate::host::session; +use crate::host::snapshot::SnapshotManifest; use crate::host::supervisor::{Launched, VmmCommand, spawn}; -use crate::host::template::TemplateManifest; use crate::protocol::{ APP_ERROR, APP_EXEC, APP_EXIT, APP_MAPS, APP_READY, APP_STDERR, APP_STDOUT, APP_STOP, APP_STOPPED, HostMap, Workload, WorkloadEnvironment, decode_app, encode_app, @@ -196,7 +196,7 @@ fn run() -> Result<(), AnyError> { for path in [&vmm, &snapshot, &distro, &original_console, &root, &result] { require(path.is_absolute(), "all fixture paths must be absolute")?; } - let manifest = TemplateManifest::read(&snapshot)?; + let manifest = SnapshotManifest::read(&snapshot)?; require( manifest.hypervisor() == backend() && manifest.memory_len() == 128 << 20, "wrong backend/memory snapshot", diff --git a/nvx-rs/src/host/sandbox/geometry.rs b/nvx-rs/src/host/sandbox/geometry.rs index fc88d3b8f..e1ece2a21 100644 --- a/nvx-rs/src/host/sandbox/geometry.rs +++ b/nvx-rs/src/host/sandbox/geometry.rs @@ -1,6 +1,5 @@ //! The block geometry of a sandbox block device, which the checks of its files read as OpenVMM //! presents it, and which a template records for the devices of its capture, from //! [`config`](crate::host::config), where a capture's configuration names it. -//! [`template`](crate::host::template) re-exports it. pub use crate::host::config::BlockGeometry; diff --git a/nvx-rs/src/host/snapshot.rs b/nvx-rs/src/host/snapshot.rs index 445c674f2..5c8328efa 100644 --- a/nvx-rs/src/host/snapshot.rs +++ b/nvx-rs/src/host/snapshot.rs @@ -58,8 +58,8 @@ use crate::host::{filesystem, platform, sandbox, seal}; use crate::net::Ipv4Cidr; use crate::{Error, Hypervisor, Result}; +pub(super) use self::capture::check_snapshot; pub use self::capture::{CapturedSnapshot, finish_capture}; -pub(super) use self::capture::{check_exit, check_snapshot}; pub use self::eligibility::{Downtime, DowntimeSource, Eligibility, HostTime, MAX_DOWNTIME_NS}; pub use self::key::{CaptureClass, ClassIdentity, Invariants, KeyIdentity, SnapshotKey}; pub use self::manifest::{ diff --git a/nvx-rs/src/host/snapshot/key.rs b/nvx-rs/src/host/snapshot/key.rs index b52ead232..c8fb3b436 100644 --- a/nvx-rs/src/host/snapshot/key.rs +++ b/nvx-rs/src/host/snapshot/key.rs @@ -506,8 +506,6 @@ pub struct SnapshotKey { time: TimeContract, } -pub(in crate::host) use SnapshotKey as TemplateKey; - /// The checked inputs shared by a legacy capture class and a caller-owned capture context. struct CaptureInputs<'a> { /// The runtime's artifact and contract identities. @@ -540,8 +538,8 @@ impl SnapshotKey { /// Version 1 omits the guest build, appends the complete capture-time contract, and holds /// the snapshot's NIC, if it has one, with the identity of the complete /// deny-all policy under which the capture attached it, but no network policy of its clones. - /// It has not been released, so a template store that an earlier build created is not - /// supported: delete it and create a new one. There is no migration. + /// It has not been released, so no key that an earlier build encoded is supported. There is + /// no migration. pub const VERSION: u16 = 1; /// Most bytes of a key's canonical encoding: what the largest key that a manifest of at most diff --git a/nvx-rs/src/host/snapshot/key/tests.rs b/nvx-rs/src/host/snapshot/key/tests.rs index 74fb1d7dd..76b9cb199 100644 --- a/nvx-rs/src/host/snapshot/key/tests.rs +++ b/nvx-rs/src/host/snapshot/key/tests.rs @@ -358,10 +358,6 @@ fn the_guest_build_is_not_part_of_a_snapshot_key() { assert_eq!(changed, made); assert_eq!(changed.to_bytes(), made.to_bytes()); assert_eq!(changed.identity(), made.identity()); - assert_eq!( - crate::host::template::TemplateKey::from_bytes(&changed.to_bytes()).unwrap(), - made - ); } #[test] @@ -740,12 +736,6 @@ fn keys_with_the_removed_guest_build_field_are_refused() { SnapshotKey::from_bytes(&old).unwrap_err().code(), ErrorCode::Io ); - assert_eq!( - crate::host::template::TemplateKey::from_bytes(&old) - .unwrap_err() - .code(), - ErrorCode::Io - ); } /// Returns the message of the refusal of the key bytes `bytes`, which must have the code `code`. diff --git a/nvx-rs/src/host/snapshot/manifest.rs b/nvx-rs/src/host/snapshot/manifest.rs index 20af1c9a0..f8bcf392a 100644 --- a/nvx-rs/src/host/snapshot/manifest.rs +++ b/nvx-rs/src/host/snapshot/manifest.rs @@ -650,8 +650,6 @@ pub struct SnapshotManifest { pub(super) attachments: Vec, } -pub(in crate::host) use SnapshotManifest as TemplateManifest; - impl SnapshotManifest { /// Reads the manifest of the snapshot in the directory `snapshot`: its [`MANIFEST_NAME`]. /// diff --git a/nvx-rs/src/host/snapshot/tests.rs b/nvx-rs/src/host/snapshot/tests.rs index 8c67db400..a07bc18e8 100644 --- a/nvx-rs/src/host/snapshot/tests.rs +++ b/nvx-rs/src/host/snapshot/tests.rs @@ -1015,9 +1015,9 @@ impl SnapshotFixture { let root = tempfile::tempdir().unwrap(); let dir = root.path().join("snapshot"); - assert_eq!(platform::create_owner_only_dir(&dir).unwrap(), None); + assert_eq!(platform::create_store_dir(&dir).unwrap(), None); let openvmm = dir.join(OPENVMM_DIR); - assert_eq!(platform::create_owner_only_dir(&openvmm).unwrap(), None); + assert_eq!(platform::create_store_dir(&openvmm).unwrap(), None); let spec = ManifestSpec::template(); let manifest = SnapshotManifest::parse(&spec.encode()).unwrap(); let key = SnapshotKey::new(&class(), &manifest).unwrap(); diff --git a/nvx-rs/src/host/store/files.rs b/nvx-rs/src/host/store/files.rs index f69206875..9807e59ac 100644 --- a/nvx-rs/src/host/store/files.rs +++ b/nvx-rs/src/host/store/files.rs @@ -5,9 +5,6 @@ //! and its log only as [`StoreFile`]s, each of which holds a file's path with what messages call //! it, so that no read or lock of one of them skips the checks that refuse a file that others may //! open. -//! -//! The template store, [`template::Store`](crate::host::template::Store), keeps its own records -//! with the same lock files, atomic replacements, removals, and syncs. use std::fs::{self, File, OpenOptions}; use std::io::{self, Read, Write}; diff --git a/nvx-rs/src/host/store/lease.rs b/nvx-rs/src/host/store/lease.rs index 7059e74f7..ba77f24ee 100644 --- a/nvx-rs/src/host/store/lease.rs +++ b/nvx-rs/src/host/store/lease.rs @@ -2,9 +2,8 @@ //! [clone binding](CLONE_BINDING_NAME) records it, and the reading of the records of the template //! store, in whose format the binding is. //! -//! The sandbox's store reads the binding before it begins a launch. The template store, -//! [`template::Store`](crate::host::template::Store), grants and ends leases, and reads its own -//! records, with these, and [`template`](crate::host::template) re-exports what is public here. +//! The sandbox's store reads the binding before it begins a launch, and as it removes the control +//! socket of a clone's VMM. use std::ffi::OsStr; use std::fmt; @@ -19,23 +18,23 @@ use serde::{Deserialize, Serialize}; use super::regular::{open_regular, open_regular_in, read_at_most, was_removed}; use super::{CLONE_BINDING_NAME, LaunchId, SandboxStore}; -use crate::activation::{Identity, StoreGeneration, TemplateReference}; +use crate::activation::{Identity, StoreGeneration}; use crate::error::{Error, ErrorCode, ErrorKind, Result}; use crate::host::platform::{self, parse_hex_id, parse_hex32}; /// Version of the store's records: its identity record and its index. -pub const STORE_FORMAT: u32 = 1; +pub(crate) const STORE_FORMAT: u32 = 1; /// Name of the directory of the generations' payloads. -pub const GENERATIONS_NAME: &str = "generations"; +pub(crate) const GENERATIONS_NAME: &str = "generations"; /// Name of the store's identity record. -pub const STORE_RECORD_NAME: &str = "store.json"; +pub(crate) const STORE_RECORD_NAME: &str = "store.json"; /// Longest identity record, in bytes. pub(crate) const MAX_STORE_RECORD_LEN: u64 = 4 * 1024; -/// Bytes of a [`StoreId`] and of a [`StorageGeneration`](crate::host::template::StorageGeneration). +/// Bytes of a [`StoreId`] and of a [`LeaseId`]. pub(crate) const ID_LEN: usize = 16; /// Longest record of a lease, of a clone binding, or of a tombstone, in bytes: more than a record @@ -55,19 +54,7 @@ pub(crate) fn hex(bytes: &[u8]) -> String { /// Its text form, which `Display` writes and `FromStr` parses, is 32 lowercase hexadecimal /// digits; parsing also accepts uppercase ones. #[derive(Clone, Copy, PartialEq, Eq, Hash)] -pub struct StoreId([u8; ID_LEN]); - -impl StoreId { - /// Returns the identity whose bytes are `bytes`. - pub const fn from_bytes(bytes: [u8; ID_LEN]) -> Self { - Self(bytes) - } - - /// Returns the bytes of the identity. - pub const fn as_bytes(&self) -> &[u8; ID_LEN] { - &self.0 - } -} +pub(crate) struct StoreId([u8; ID_LEN]); impl fmt::Display for StoreId { fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { @@ -100,19 +87,7 @@ impl FromStr for StoreId { /// Its text form, which `Display` writes and `FromStr` parses, is 32 lowercase hexadecimal /// digits; parsing also accepts uppercase ones. #[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] -pub struct LeaseId([u8; ID_LEN]); - -impl LeaseId { - /// Returns the identity whose bytes are `bytes`. - pub const fn from_bytes(bytes: [u8; ID_LEN]) -> Self { - Self(bytes) - } - - /// Returns the bytes of the identity. - pub const fn as_bytes(&self) -> &[u8; ID_LEN] { - &self.0 - } -} +pub(crate) struct LeaseId([u8; ID_LEN]); impl fmt::Display for LeaseId { fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { @@ -145,16 +120,13 @@ impl FromStr for LeaseId { /// /// The store keeps the record of each lease that it granted, and the sandbox keeps the same /// record as its clone binding, [`CLONE_BINDING_NAME`], beside its launch journal. A lease names -/// its sandbox by the path of its directory with its links resolved, so that no removal or change -/// of a link, or of a directory that a `..` component passes through, hides the sandbox from a -/// [retirement](crate::host::template::Store::retire). A sandbox that holds a lease must not move, -/// and goes only through its store's [removal](super::LockedStore::remove), which makes the -/// binding's absence durable before the sandbox goes. The lease allows one launch of the clone, as -/// the identity that it reserves, [`launch`](Self::launch), under the lock that acquired it, and no -/// other launch of the sandbox until a release ends it, as +/// its sandbox by the path of its directory with its links resolved. A sandbox that holds a lease +/// must not move, and goes only through its store's [removal](super::LockedStore::remove), which +/// makes the binding's absence durable before the sandbox goes. The lease reserves one launch of +/// the clone, [`launch`](Self::launch), and allows no launch of the sandbox, as /// [`LockedStore::begin_launch`](super::LockedStore::begin_launch) requires. #[derive(Clone, PartialEq, Eq)] -pub struct Lease { +pub(crate) struct Lease { /// The lease's identity. id: LeaseId, /// The identity of the store that granted it. @@ -171,76 +143,35 @@ pub struct Lease { } impl Lease { - /// Returns the lease `id`, which the store `store` granted, of its generation `generation`, - /// whose key's identity is `key`, to the sandbox in `sandbox`, an absolute path with its links - /// resolved, which is valid Unicode, for the launch `launch` of its clone. - pub(crate) fn new( - id: LeaseId, - store: StoreId, - generation: StoreGeneration, - key: Identity, - sandbox: PathBuf, - launch: LaunchId, - ) -> Self { - Self { - id, - store, - generation, - key, - sandbox, - launch, - } - } - /// Reads the clone binding of the sandbox of `sandbox`: the lease that it holds, if it holds /// one. /// /// A read that holds no lock sees one whole binding, as the latest transition left it. Fails /// with [`ErrorKind::Io`] for a binding that cannot be read, or is malformed, and with /// [`ErrorCode::FormatUnsupported`] for one of another format. - pub fn of(sandbox: &SandboxStore) -> Result> { + pub(crate) fn of(sandbox: &SandboxStore) -> Result> { read_binding(sandbox.dir()) } /// Returns the lease's identity. - pub fn id(&self) -> LeaseId { + pub(crate) fn id(&self) -> LeaseId { self.id } /// Returns the identity of the store that granted the lease. - pub fn store(&self) -> StoreId { + pub(crate) fn store(&self) -> StoreId { self.store } /// Returns the store generation of the leased generation. - pub fn generation(&self) -> StoreGeneration { + pub(crate) fn generation(&self) -> StoreGeneration { self.generation } - /// Returns the directory of the sandbox that holds the lease, as an absolute path with its - /// links resolved, which may differ from the path through which the sandbox was leased, and - /// through which a [release](crate::host::template::Store::release) takes the sandbox's lock. - pub fn sandbox(&self) -> &Path { - &self.sandbox - } - - /// Returns the launch of the clone that the lease reserves: the identity under which the - /// caller launches the clone, with [`supervisor::launch`](crate::host::supervisor::launch), - /// once, under the lock that acquired the lease, and the only launch of the sandbox that - /// [`LockedStore::begin_launch`](super::LockedStore::begin_launch) begins while the sandbox - /// holds the lease. - pub fn launch(&self) -> &LaunchId { + /// Returns the launch of the clone that the lease reserves. + pub(crate) fn launch(&self) -> &LaunchId { &self.launch } - - /// Returns the reference that the clone's lease and durable records keep of its template: - /// the leased generation's store generation and key identity. - pub fn template(&self) -> TemplateReference { - TemplateReference { - generation: self.generation, - key: self.key, - } - } } impl fmt::Debug for Lease { @@ -264,10 +195,9 @@ pub(crate) fn clone_socket_name(launch: &LaunchId) -> String { format!("{launch}.sock") } -/// Returns whether `path` names the control socket of the VMM of the clone that holds `lease`, as -/// [`template::Store::clone_socket_path`](crate::host::template::Store::clone_socket_path) returns -/// it: `generations//.sock`, in a template store, which a lease names by its -/// identity alone. +/// Returns whether `path` names the control socket of the VMM of the clone that holds `lease`, in +/// the payload of the leased generation: `generations//.sock`, in a template +/// store, which a lease names by its identity alone. pub(crate) fn is_clone_socket(lease: &Lease, path: &Path) -> bool { let name = clone_socket_name(&lease.launch); let generation = lease.generation.to_string(); @@ -318,17 +248,10 @@ pub(crate) fn open_payload( .map_err(failed()) } -/// Reads the identity of the store in `dir` from its identity record, or returns `None` if the -/// directory holds none. -pub(crate) fn read_identity(dir: &Path) -> Result> { - let path = dir.join(STORE_RECORD_NAME); - read_identity_with(&path, || open_existing(&path)) -} - -/// Reads the identity of the store whose directory `directory` holds open, in `dir`, as -/// [`read_identity`] reads it at its path, or returns `None` if the directory holds no identity -/// record, or, on Windows, one whose deletion is pending: through the handle, so that it reads -/// the identity of the directory that the handle holds, whatever `dir` names meanwhile. +/// Reads the identity of the store whose directory `directory` holds open, in `dir`, from its +/// identity record, or returns `None` if the directory holds none, or, on Windows, one whose +/// deletion is pending: through the handle, so that it reads the identity of the directory that the +/// handle holds, whatever `dir` names meanwhile. pub(crate) fn read_identity_in( directory: &platform::Directory, dir: &Path, @@ -375,19 +298,6 @@ pub(crate) fn read_binding(dir: &Path) -> Result> { read_lease(&dir.join(CLONE_BINDING_NAME), BINDING_DESCRIPTION) } -/// Reads the clone binding of the sandbox whose directory `directory` holds open, in `dir`, as -/// [`read_binding`] reads it at its path, or returns `None` if the directory holds none, or, on -/// Windows, one whose deletion is pending, which is removed: through the handle, so that it reads -/// the binding of the directory that the handle holds, whatever `dir` names meanwhile. -pub(crate) fn read_binding_in( - directory: &platform::Directory, - dir: &Path, -) -> Result> { - read_lease_with(&dir.join(CLONE_BINDING_NAME), BINDING_DESCRIPTION, || { - open_regular_in(directory, CLONE_BINDING_NAME) - }) -} - /// Reads the record of a lease at `path`, its `description`, or returns `None` if it does not /// exist. pub(crate) fn read_lease(path: &Path, description: &str) -> Result> { @@ -435,19 +345,6 @@ pub(crate) struct LeaseRecord { } impl LeaseRecord { - /// Returns the record of `lease`. - pub(crate) fn new(lease: &Lease) -> Self { - Self { - format: STORE_FORMAT, - store: lease.store.to_string(), - lease: lease.id.to_string(), - generation: lease.generation.get().get(), - key: lease.key.to_string(), - sandbox: lease.sandbox.to_string_lossy().into_owned(), - launch: lease.launch.to_string(), - } - } - /// Checks the record, and returns its lease, or says why it is malformed. fn into_lease(self) -> std::result::Result { let field = @@ -477,7 +374,7 @@ impl LeaseRecord { } /// Parses the text form of the identity of a template's key, 64 hexadecimal digits, as -/// [`KeyIdentity`](crate::host::template::KeyIdentity) parses it, failing with +/// [`KeyIdentity`](crate::host::snapshot::KeyIdentity) parses it, failing with /// [`ErrorKind::Unsupported`] for anything else. fn parse_key(text: &str) -> Result { parse_hex32(text).map(Identity::from_bytes).ok_or_else(|| { @@ -500,21 +397,9 @@ pub(crate) fn store_error(context: String) -> impl FnOnce(io::Error) -> Error { } } -/// Reads the private regular file at `path`, the store's `description`, of at most `limit` bytes, -/// or returns `None` if it does not exist, or was removed as this opened it, as [`was_removed`] -/// tells, which a transaction in another process may do. -/// -/// The file is opened without following a link, or on Windows any other reparse point, at its -/// own name, and refused unless it is a regular file, before anything is read from it. A file -/// that is not private to the current user is refused, since others could have written it: on -/// Windows, a file's own access control list decides who may open it, whatever the store's -/// directory allows. -pub(crate) fn read_private(path: &Path, description: &str, limit: u64) -> Result>> { - read_private_with(path, description, limit, || open_existing(path)) -} - /// Opens the regular file at `path`, as [`open_regular`] does, or returns `None` if it does not -/// exist, or was removed as this opened it, as [`was_removed`] tells. +/// exist, or was removed as this opened it, as [`was_removed`] tells, which a transaction in +/// another process may do. fn open_existing(path: &Path) -> io::Result> { match open_regular(path) { Ok(file) => Ok(Some(file)), @@ -526,7 +411,13 @@ fn open_existing(path: &Path) -> io::Result> { } /// Reads the private regular file that `open` opens, the store's `description` at `path`, of at -/// most `limit` bytes, as [`read_private`] reads it, or returns `None` if `open` finds none. +/// most `limit` bytes, or returns `None` if `open` finds none. +/// +/// `open` opens the file without following a link, or on Windows any other reparse point, at its +/// own name, and refuses it unless it is a regular file, before anything is read from it. A file +/// that is not private to the current user is refused, since others could have written it: on +/// Windows, a file's own access control list decides who may open it, whatever the store's +/// directory allows. fn read_private_with( path: &Path, description: &str, @@ -548,18 +439,9 @@ fn read_private_with( .map_err(failed()) } -/// Reads the store's record at `path`, its `description`, of at most `limit` bytes, in format -/// [`STORE_FORMAT`], or returns `None` if it does not exist. -pub(crate) fn read_record( - path: &Path, - description: &str, - limit: u64, -) -> Result> { - read_record_with(path, description, limit, || open_existing(path)) -} - -/// Reads the store's record that `open` opens, its `description` at `path`, as [`read_record`] -/// reads it, or returns `None` if `open` finds none. +/// Reads the store's record that `open` opens, its `description` at `path`, of at most `limit` +/// bytes, in format [`STORE_FORMAT`], as [`read_private_with`] reads it, or returns `None` if +/// `open` finds none. fn read_record_with( path: &Path, description: &str, diff --git a/nvx-rs/src/host/store/lifecycle.rs b/nvx-rs/src/host/store/lifecycle.rs index 0dedfb0c8..9bd928928 100644 --- a/nvx-rs/src/host/store/lifecycle.rs +++ b/nvx-rs/src/host/store/lifecycle.rs @@ -268,13 +268,13 @@ impl LockedStore { /// [`supervisor::reattach`](crate::host::supervisor::reattach) does. It also refuses, with /// [`RemovalError::Io`], and removes nothing, once the directory that its path names holds /// another lock file than this lock's, as after a link in the path changed. It makes both - /// checks, and removes what a clone kept of its launch, as a release does, and the sandbox's + /// checks, and removes what a clone kept of its launch, and the sandbox's /// [clone binding](CLONE_BINDING_NAME), if it holds one, through one handle of the directory /// that it opens, so that no change of a link in the path meanwhile redirects their removal /// to another sandbox, whose VM may run. It makes their absence durable before anything else /// goes, so that no crash after the removal of the directory, or of one of its ancestors, - /// brings back a binding whose lease a retirement may have ended. It then removes the store's - /// other files but its lock, the caller's files, its configuration record last, so that an + /// brings back a binding. It then removes the store's other files but its lock, the caller's + /// files, its configuration record last, so that an /// interrupted removal leaves a provisioned sandbox, which can be removed again, and the /// temporary files that the store owns. It refuses the entries that are neither the store's /// nor the caller's, which it names in the order in which the directory lists them, and keeps @@ -358,18 +358,17 @@ impl LockedStore { } } - /// Removes what the clone of the sandbox kept of its launch, as a release does, and then the - /// sandbox's clone binding, if it holds one, and makes their absence durable with `sync`, + /// Removes what the clone of the sandbox kept of its launch, and then the sandbox's clone + /// binding, if it holds one, and makes their absence durable with `sync`, /// before anything else of the sandbox goes, through one handle of the directory that the path /// names. Through that handle, it checks first that the directory holds this lock's file, and /// refuses otherwise, then that the store records no VM and no launch, and runs `checked` once /// both checks pass, before it removes anything. No change of a link in the path meanwhile /// redirects these steps to another sandbox, whose VM may run, and whose files then stay. /// - /// The binding's absence, after this removal or after a release that a crash interrupted - /// before its sync, is then durable while the directory, which keeps the sandbox provisioned, - /// remains. A binding, or a file of the clone, whose deletion is pending already counts as - /// removed. + /// The binding's absence, after this removal, is then durable while the directory, which + /// keeps the sandbox provisioned, remains. A binding, or a file of the clone, whose deletion + /// is pending already counts as removed. fn unbind( &self, checked: &dyn Fn(), @@ -405,8 +404,8 @@ impl LockedStore { } } checked(); - // What a clone kept of its launch goes before its binding, as a release removes it, so - // that the binding's durable absence makes theirs durable too. + // What a clone kept of its launch goes before its binding, so that the binding's durable + // absence makes theirs durable too. for name in CLONE_FILE_NAMES { directory.remove_file(name).map_err(entry_failed(name))?; } diff --git a/nvx-rs/src/host/store/lifecycle/tests.rs b/nvx-rs/src/host/store/lifecycle/tests.rs index c381d9559..cb6a3572d 100644 --- a/nvx-rs/src/host/store/lifecycle/tests.rs +++ b/nvx-rs/src/host/store/lifecycle/tests.rs @@ -1190,33 +1190,6 @@ mod windows { assert!(!dir.exists()); } - #[test] - fn a_retried_removal_of_the_clone_files_takes_those_whose_deletion_is_pending_for_removed() { - let (_directory, dir) = sandbox(); - for name in CLONE_FILE_NAMES { - fs::write(dir.join(name), b"").unwrap(); - } - // A release that failed once it removed the clone's files, which readers hold, retries - // their removal, which finds each one's deletion pending. - let pending: Vec = CLONE_FILE_NAMES - .into_iter() - .map(|name| delete_pending(&dir.join(name))) - .collect(); - let locked = SandboxStore::lock_existing(&dir, &files(), None).unwrap(); - locked - .remove_clone_files(&locked.open_dir().unwrap()) - .unwrap(); - drop(locked); - drop(pending); - let names = entries_of(&dir); - assert!( - names - .iter() - .all(|name| !CLONE_FILE_NAMES.contains(&name.as_str())), - "{names:?}" - ); - } - #[test] fn locks_refuse_a_lock_file_that_refuses_access_for_good() { // A read-only lock file, and a directory at its path, refuse an open for writing as a diff --git a/nvx-rs/src/host/store/locked.rs b/nvx-rs/src/host/store/locked.rs index f1d9326e4..9b52de4fe 100644 --- a/nvx-rs/src/host/store/locked.rs +++ b/nvx-rs/src/host/store/locked.rs @@ -4,7 +4,7 @@ use std::fs::File; use std::io; use std::ops::Deref; -use std::path::{Path, PathBuf}; +use std::path::Path; use std::thread; use std::time::Instant; @@ -13,7 +13,6 @@ use super::files::{io_error, remove_if_present}; use super::identity::RuntimeIdentity; use super::launch::{Launch, Recovery, end_process}; use super::lease::{Lease, clone_socket_name, is_clone_socket, open_payload}; -use super::names::{CLONE_FILE_NAMES, LOCK_NAME}; use super::record::{LaunchId, LaunchJournal, LogClaim, RuntimeRecord}; use super::{CLAIM_RELEASE_WAIT, LOCK_RETRY_INTERVAL, SandboxStore, record, store_error, vmm}; use crate::error::{Error, ErrorCode, Result}; @@ -42,26 +41,6 @@ pub struct LockedStore { pub(super) files: CallerFiles, /// Whether this holder removed its lock file, after which the path may name another's. pub(super) lock_removed: bool, - /// The launch of the sandbox's clone that the lease acquired under this lock reserves, until - /// [`begin_launch`](Self::begin_launch) begins it, or a release ends the lease. - reserved: Option, -} - -/// The launch of a sandbox's clone that a lease reserves, as -/// [`template::Store::acquire`](crate::host::template::Store::acquire) acquires the lease, and -/// where the clone's VMM listens. -#[derive(Debug, Clone, PartialEq, Eq)] -pub(crate) struct Reservation { - /// The identity of the launch, [`Lease::launch`]. - pub(crate) launch: LaunchId, - /// The path of the Linux socket at which the clone's VMM listens, its control endpoint: beside - /// the snapshot that it restores, in the payload of the leased generation, since OpenVMM - /// requires a restore's control socket in the snapshot's parent, rather than the sandbox's - /// own control socket. Windows endpoints are named pipes, which [`begin_launch`] draws as it - /// does for every launch. - /// - /// [`begin_launch`]: LockedStore::begin_launch - pub(crate) socket: PathBuf, } impl Deref for LockedStore { @@ -81,82 +60,7 @@ impl LockedStore { lock, files, lock_removed: false, - reserved: None, - } - } - - /// Lets [`begin_launch`](Self::begin_launch) begin the launch that `reservation` names, once, - /// as the launch of the sandbox's clone that the lease acquired under this lock reserves, - /// whose VMM listens at the reservation's socket, or, given `None`, no launch of a clone, once - /// a release ends the lease. - pub(crate) fn reserve(&mut self, reservation: Option) { - self.reserved = reservation; - } - - /// Returns the launch of the sandbox's clone that the lease acquired under this lock reserves, - /// until [`begin_launch`](Self::begin_launch) begins it, or a release ends the lease. - pub(crate) fn reserved(&self) -> Option<&LaunchId> { - self.reserved - .as_ref() - .map(|reservation| &reservation.launch) - } - - /// Opens the sandbox's directory through its path, as one handle through which a release of - /// the lease of its clone names the directory's entries, and checks through that handle that - /// the directory holds this lock's file, as a [removal] checks it: no change of the path - /// meanwhile, of a link in it or of one of its directories, then makes the release reach the - /// entries of another sandbox, whose lock it does not hold. - /// - /// Fails with [`ErrorKind::Other`] if the directory that the path names does not hold this - /// lock's file. - /// - /// [removal]: crate::host::store#removal - /// [`ErrorKind::Other`]: crate::ErrorKind::Other - pub(crate) fn open_dir(&self) -> Result { - let dir = self.dir(); - let directory = platform::Directory::open(dir).map_err(io_error(format!( - "cannot open the directory {}", - dir.display() - )))?; - let lock = self.path(LOCK_NAME); - let examine = || io_error(format!("cannot examine {}", lock.display())); - let held = platform::opened_file_id(&self.lock).map_err(examine())?; - if directory.entry_id(LOCK_NAME).map_err(examine())? != Some(held) { - return Err(Error::other(format!( - "the directory that {} names does not hold the lock of its sandbox: the path names \ - another directory than when the lock was taken", - dir.display() - ))); - } - Ok(directory) - } - - /// Removes what the clone of the sandbox kept of its launch in the sandbox's directory, whose - /// VM has exited, through `directory`, the handle of the directory that - /// [`open_dir`](Self::open_dir) returned: the file of its scratch device, - /// [`CLONE_SCRATCH_NAME`], if an earlier version left one, its activation record, - /// [`CLONE_ACTIVATION_NAME`], the socket of its restore-ready event, [`RESTORE_READY_NAME`], - /// its progress log, [`CLONE_PROGRESS_NAME`], and the record of its failure, - /// [`CLONE_FAILURE_NAME`]. A release, as a [removal] does, removes them before the clone - /// binding, whose removal, once durable, makes theirs durable too, since all are entries of the - /// sandbox's directory. A file whose deletion is pending, as a Windows file system without - /// POSIX deletion leaves a removed file that another handle holds, is removed already, so that - /// a release that failed once it removed the file can be retried. - /// - /// [`CLONE_SCRATCH_NAME`]: crate::host::store::CLONE_SCRATCH_NAME - /// [`CLONE_ACTIVATION_NAME`]: crate::host::store::CLONE_ACTIVATION_NAME - /// [`RESTORE_READY_NAME`]: crate::host::store::RESTORE_READY_NAME - /// [`CLONE_PROGRESS_NAME`]: crate::host::store::CLONE_PROGRESS_NAME - /// [`CLONE_FAILURE_NAME`]: crate::host::store::CLONE_FAILURE_NAME - /// [removal]: crate::host::store#removal - pub(crate) fn remove_clone_files(&self, directory: &platform::Directory) -> Result<()> { - for name in CLONE_FILE_NAMES { - directory.remove_file(name).map_err(io_error(format!( - "cannot remove {}", - self.path(name).display() - )))?; } - Ok(()) } /// Returns the caller's files that the lock knows: those that a [lifecycle lock] was taken @@ -177,19 +81,13 @@ impl LockedStore { /// of a caller that keeps the console of every launch, as [`CallerFiles::keeping_console`] /// declares. Nothing is spawned yet: the caller spawns the VMM with [`Launch::take_log`]. /// - /// The control endpoint is a named pipe of a random name on Windows. On Linux, it is the - /// sandbox's control socket, [`socket_path`](SandboxStore::socket_path), but for the launch - /// that the lease of the sandbox's clone reserves: its VMM restores the leased generation's - /// snapshot, and listens beside it, at the socket that the lease reserves in the generation's - /// payload, as [`template::Store::acquire`](crate::host::template::Store::acquire) describes. + /// The control endpoint is a named pipe of a random name on Windows, and the sandbox's control + /// socket, [`socket_path`](SandboxStore::socket_path), on Linux. /// /// Fails if the sandbox's VM runs, or a launch was interrupted and is not recovered yet, and /// with [`ErrorKind::Unsupported`] for an all-zero capability or an invalid runtime identity. /// A sandbox whose clone holds a lease of a template's generation, which its [clone binding] - /// records, launches once, as the launch that the lease reserves, under the lock that acquired - /// the lease, as [`template::Store::acquire`](crate::host::template::Store::acquire) returns - /// it: any other launch, and any launch under another lock or after that one began, fails with - /// [`ErrorCode::SandboxLeased`] until a release ends the lease. + /// records, launches nothing: any launch fails with [`ErrorCode::SandboxLeased`]. /// /// [`ErrorKind::Unsupported`]: crate::ErrorKind::Unsupported /// [clone binding]: crate::host::store::CLONE_BINDING_NAME @@ -217,23 +115,17 @@ impl LockedStore { self.dir.display() ))); } - if let Some(lease) = Lease::of(self)? - && (lease.launch() != launch || self.reserved() != Some(launch)) - { - let reservation = if lease.launch() == launch { - format!("whose launch {launch} begins once, under the lock that acquired the lease") - } else { - format!("which reserves the launch {}, not {launch}", lease.launch()) - }; + if let Some(lease) = Lease::of(self)? { return Err(Error::new( ErrorCode::SandboxLeased, format!( "the sandbox in {} holds the lease {} of the generation {} of the template \ - store {}, {reservation}", + store {}, which reserves the launch {}", self.dir.display(), lease.id(), lease.generation(), - lease.store() + lease.store(), + lease.launch() ), )); } @@ -245,13 +137,7 @@ impl LockedStore { remove_if_present(&self.console_path())?; } self.remove_report_staging()?; - // The VMM of a clone listens at the socket that its lease reserves, beside the snapshot - // that it restores. - let listening = match &self.reserved { - Some(reservation) if reservation.launch == *launch => reservation.socket.as_path(), - _ => &socket, - }; - let endpoint = platform::control_endpoint(listening) + let endpoint = platform::control_endpoint(&socket) .map_err(|error| Error::io("cannot choose a control endpoint").with_source(error))?; let (log, claim) = self.create_log()?; let capability_file = self.capability_file(); @@ -261,8 +147,6 @@ impl LockedStore { let _ = capability_file.remove_if_present(); return Err(error); } - // A lease reserves one launch, which has begun. - self.reserved = None; Ok(Launch { locked: self, journal, @@ -452,7 +336,7 @@ impl LockedStore { /// socket's generation. The log, console log, and outcome report outlive the run. pub fn clear_runtime(&mut self) -> Result<()> { // Only a Linux endpoint is a file, which may be a clone's beside its snapshot. A record that - // cannot be read names none, and a release removes a clone's socket all the same. + // cannot be read names none. let record = if cfg!(unix) { self.runtime().ok().flatten() } else { @@ -476,24 +360,19 @@ impl LockedStore { /// Removes the control socket at `endpoint`, the control endpoint that a launch of the /// sandbox recorded, if it is the socket of the sandbox's clone beside the snapshot that its - /// VMM restores, which the lease that the sandbox's clone binding records reserved, as - /// [`template::Store::clone_socket_path`](crate::host::template::Store::clone_socket_path) - /// names it. Any other endpoint stays: the sandbox's own control socket, which the caller - /// removes, and a named pipe. Neither reads the binding. + /// VMM restores, which the lease that the sandbox's clone binding records reserved, in the + /// payload of the leased generation of its template store. Any other endpoint stays: the + /// sandbox's own control socket, which the caller removes, and a named pipe. Neither reads the + /// binding. /// /// The socket goes through a handle of the generation's payload, which a handle of the - /// template store's directory opens, as a release opens it: following no link at the store's - /// directory, at its directory of the generations, or at the payload, and once it checked - /// through that handle that the directory holds the lease's store, so that neither a link that - /// takes the place of one of them nor a directory that takes the place of the store ever leads - /// the removal to the files of another directory. Such a link fails the removal; a payload - /// that is gone holds no socket, and neither does a directory that holds another store, or - /// none, in the place of the lease's, whose files stay. - /// - /// The removal is not made durable here: - /// [`template::Store::release`](crate::host::template::Store::release) makes it durable before - /// the lease ends, and a retirement removes whatever a crash brings back with the generation's - /// payload. + /// template store's directory opens: following no link at the store's directory, at its + /// directory of the generations, or at the payload, and once it checked through that handle + /// that the directory holds the lease's store, so that neither a link that takes the place of + /// one of them nor a directory that takes the place of the store ever leads the removal to the + /// files of another directory. Such a link fails the removal; a payload that is gone holds no + /// socket, and neither does a directory that holds another store, or none, in the place of the + /// lease's, whose files stay. The removal is not made durable here. fn remove_clone_socket(&self, endpoint: &str) -> Result<()> { let socket = Path::new(endpoint); if !cfg!(unix) || socket == self.socket_path() { diff --git a/nvx-rs/src/host/store/mod.rs b/nvx-rs/src/host/store/mod.rs index 912005938..f05a72060 100644 --- a/nvx-rs/src/host/store/mod.rs +++ b/nvx-rs/src/host/store/mod.rs @@ -8,8 +8,7 @@ //! launch.json the journal of a launch in progress, or of one that was interrupted //! runtime.json the runtime record of its running VM //! control.capability the launch capability of its VM -//! control.sock the control endpoint of its VM, on Linux, but for a clone, whose VMM -//! listens beside the snapshot that it restores +//! control.sock the control endpoint of its VM, on Linux //! openvmm.log the VMM's output, claimed by the launch that created it //! console.log the guest console of the latest launch, or of every launch, if the //! caller captures it @@ -17,7 +16,7 @@ //! .tmpXXXXXX the outcome report while the VMM writes it, which it then publishes //! clone.json the binding of a clone to the generation of a template that it leases //! clone-scratch.raw the file of the scratch device that earlier versions created for a -//! clone whose upper is in RAM, which a release removes if one remains +//! clone whose upper is in RAM, which a removal deletes if one remains //! clone-activation.bin the activation record of a clone, which its launch writes //! clone-progress.log the progress log of a clone's launch and restore sequence //! clone-failure.json the record of a clone's failure @@ -58,9 +57,7 @@ //! # Launches //! //! A launch holds the sandbox's lock throughout, under an identity that the caller reserved -//! before it began, with [`LaunchId::generate`], a fresh one for each launch, or that a clone's -//! lease reserved, which records it durably before any VMM of the clone exists, for the one -//! launch of the clone that the lease allows: +//! before it began, with [`LaunchId::generate`], a fresh one for each launch: //! //! 1. [`LockedStore::begin_launch`] stores the capability, creates a fresh log and claims it, and //! writes the journal's marker, which names the launch, before anything is spawned. @@ -172,8 +169,7 @@ //! the VMM through the control endpoint that the journal names, holds it, confirms that the //! endpoint still names it once held, and terminates it. //! On Windows the endpoint is a pipe whose name each launch draws at random, which names the -//! VMM by itself. On Linux it is the same socket path for every launch of the sandbox, but the -//! one launch of a clone, whose VMM listens at a socket that its lease reserves, so the +//! VMM by itself. On Linux it is the same socket path for every launch of the sandbox, so the //! VMM must also hold the claimed log as its standard output. Until the VMM serves its //! endpoint, recovery is indeterminate and keeps the journal; once the VMM exits, the released //! claim proves it. @@ -230,7 +226,7 @@ //! //! [`LockedStore::remove`] removes a sandbox that no VMM can run in: it refuses while the store //! records a VM or a launch, which the caller reattaches to, or recovers, first. It removes what a -//! clone kept of its launch, as a release does, and the sandbox's +//! clone kept of its launch, and the sandbox's //! [clone binding](CLONE_BINDING_NAME) first, and makes their absence durable before anything //! else goes. It checks for those records, and removes those files, through one handle of the //! directory, once it has checked through that handle that the directory holds its lock, so that @@ -304,7 +300,6 @@ pub use self::identity::{ pub use self::launch::{Launch, Recovery}; pub use self::lifecycle::{Lingering, RemovalError}; pub use self::locked::LockedStore; -pub(crate) use self::locked::Reservation; use self::names::CLONE_FILE_NAMES; pub use self::names::{ CAPABILITY_NAME, CLONE_ACTIVATION_NAME, CLONE_BINDING_NAME, CLONE_FAILURE_NAME, diff --git a/nvx-rs/src/host/store/names.rs b/nvx-rs/src/host/store/names.rs index 571ff7f57..67e4dc103 100644 --- a/nvx-rs/src/host/store/names.rs +++ b/nvx-rs/src/host/store/names.rs @@ -17,10 +17,7 @@ pub const JOURNAL_NAME: &str = "launch.json"; pub const RUNTIME_NAME: &str = "runtime.json"; /// Name of the file that holds the launch capability. pub const CAPABILITY_NAME: &str = "control.capability"; -/// Name of the Linux control socket, at which the VMM of every launch of the sandbox listens, but -/// that of a clone, which listens beside the snapshot that it restores, at the socket that its -/// lease reserves, as [`template::Store::acquire`](crate::host::template::Store::acquire) -/// describes. +/// Name of the Linux control socket, at which the VMM of every launch of the sandbox listens. pub const SOCKET_NAME: &str = "control.sock"; /// Name of the VMM's log. pub const LOG_NAME: &str = "openvmm.log"; @@ -28,50 +25,40 @@ pub const LOG_NAME: &str = "openvmm.log"; pub const CONSOLE_NAME: &str = "console.log"; /// Name of the VMM's outcome report. pub const OUTCOME_NAME: &str = "outcome.json"; -/// Name of the binding of the sandbox's clone to the generation of a template that it leases, -/// which [`template::Store::acquire`](crate::host::template::Store::acquire) writes, durably, -/// before the clone's launch, and [`template::Store::release`](crate::host::template::Store::release) -/// removes once its VM has exited. A [removal] of the sandbox removes it first, and makes its -/// absence durable before the sandbox goes. +/// Name of the binding of the sandbox's clone to the generation of a template that it leases. A +/// [removal] of the sandbox removes it first, and makes its absence durable before the sandbox +/// goes. /// /// [removal]: crate::host::store#removal pub const CLONE_BINDING_NAME: &str = "clone.json"; /// Name of the file of the `scratch` block device that earlier versions of this crate created for /// a clone whose upper is in RAM, a sparse file of the template's scratch length. Such a clone now /// restores with its scratch absent, and nothing creates the file, but a sandbox's directory may -/// still hold one that an earlier version left, which -/// [`template::Store::release`](crate::host::template::Store::release) removes once the clone's VM -/// has exited, as a [removal] of the sandbox does. +/// still hold one that an earlier version left, which a [removal] of the sandbox removes. /// /// [removal]: crate::host::store#removal pub const CLONE_SCRATCH_NAME: &str = "clone-scratch.raw"; -/// Name of the activation record of a clone, in its canonical encoding, which -/// [`template::Store::release`](crate::host::template::Store::release) removes once the clone's VM -/// has exited, as a [removal] of the sandbox does. +/// Name of the activation record of a clone, in its canonical encoding, which a [removal] of the +/// sandbox removes. /// /// [removal]: crate::host::store#removal pub const CLONE_ACTIVATION_NAME: &str = "clone-activation.bin"; -/// Name of the Linux socket of the restore-ready event of a clone's VMM, which -/// [`template::Store::release`](crate::host::template::Store::release) removes once the clone's VM -/// has exited, as a [removal] of the sandbox does. +/// Name of the Linux socket of the restore-ready event of a clone's VMM, which a [removal] of the +/// sandbox removes. /// /// [removal]: crate::host::store#removal pub const RESTORE_READY_NAME: &str = "restore.sock"; -/// Name of the progress log of a clone's launch, which -/// [`template::Store::release`](crate::host::template::Store::release) removes, as a [removal] of -/// the sandbox does. +/// Name of the progress log of a clone's launch, which a [removal] of the sandbox removes. /// /// [removal]: crate::host::store#removal pub const CLONE_PROGRESS_NAME: &str = "clone-progress.log"; -/// Name of the record of a clone's failure, which -/// [`template::Store::release`](crate::host::template::Store::release) removes, as a [removal] of -/// the sandbox does. +/// Name of the record of a clone's failure, which a [removal] of the sandbox removes. /// /// [removal]: crate::host::store#removal pub const CLONE_FAILURE_NAME: &str = "clone-failure.json"; /// The files that the clone of a sandbox keeps of its launch, and the file of the scratch device -/// that an earlier version created for a clone in RAM, which a release removes once the clone's VM -/// has exited, as a [removal] of the sandbox does, before the clone binding. +/// that an earlier version created for a clone in RAM, which a [removal] of the sandbox removes +/// before the clone binding. /// /// [removal]: crate::host::store#removal pub(super) const CLONE_FILE_NAMES: [&str; 5] = [ diff --git a/nvx-rs/src/host/store/record.rs b/nvx-rs/src/host/store/record.rs index fe86eac4f..a33e9d806 100644 --- a/nvx-rs/src/host/store/record.rs +++ b/nvx-rs/src/host/store/record.rs @@ -45,9 +45,7 @@ pub struct LaunchId(String); impl LaunchId { /// Draws a fresh launch identity, which a caller reserves for one launch before it begins it /// with [`LockedStore::begin_launch`](super::LockedStore::begin_launch), or - /// [`supervisor::launch`](crate::host::supervisor::launch), and never uses for another. A - /// clone's lease reserves one, as - /// [`template::Store::acquire`](crate::host::template::Store::acquire) draws it. + /// [`supervisor::launch`](crate::host::supervisor::launch), and never uses for another. pub fn generate() -> Result { let mut bytes = [0u8; LAUNCH_ID_BYTES]; getrandom::fill(&mut bytes) diff --git a/nvx-rs/src/host/store/regular.rs b/nvx-rs/src/host/store/regular.rs index 94060b5a7..10b9d7cbb 100644 --- a/nvx-rs/src/host/store/regular.rs +++ b/nvx-rs/src/host/store/regular.rs @@ -1,6 +1,5 @@ -//! The files that the template modules read, and that a sandbox's store reads as its clone -//! binding: regular files that are not links, which they read within a bound, and the identity of -//! a file that a template module holds open. +//! The files that a sandbox's store reads as its clone binding: regular files that are not links, +//! which it reads within a bound. //! //! The reads themselves are [`platform`]'s: this module re-exports its `open_regular` and //! `read_at_most` for its callers until #1121 deletes them. @@ -11,7 +10,6 @@ use std::path::Path; use crate::host::platform; pub(crate) use crate::host::platform::{open_regular, read_at_most}; -use crate::host::seal::FileSeal; /// Returns whether `error`, with which an open or a removal of the file at `path` failed, means /// that the file was removed, by a transaction of another process, say: on Windows, a file system @@ -40,11 +38,3 @@ pub(crate) fn open_regular_in( Err(error) => Err(platform::refused_open(error)), } } - -/// Returns the identity of the file that `file` holds open, as its [`FileSeal`] records it: its -/// device or volume, and its file ID, the 128-bit one of `FILE_ID_INFO` on Windows, which a file -/// system such as ReFS keeps unique where the 64-bit file index is not. -pub(crate) fn identity_of(file: &File) -> io::Result<(u64, u128)> { - let seal = FileSeal::of(file)?; - Ok((seal.device(), seal.file_id())) -} diff --git a/nvx-rs/src/host/store/tests.rs b/nvx-rs/src/host/store/tests.rs index 8a3991248..546b92765 100644 --- a/nvx-rs/src/host/store/tests.rs +++ b/nvx-rs/src/host/store/tests.rs @@ -315,39 +315,6 @@ fn a_lock_file_that_is_a_link_fails_at_once() { assert_eq!(fs::read(&target).unwrap(), CAPABILITY); } -// Other hosts open no handle of a sandbox's directory, through which a clone's files go. -#[cfg(any(target_os = "linux", windows))] -#[test] -fn the_files_that_a_clone_kept_of_its_launch_go_and_nothing_else() { - let (_directory, store) = store(); - let locked = store.lock(None).unwrap(); - for path in [ - // The file of the scratch device that an earlier version left for a clone in RAM. - store.dir().join(CLONE_SCRATCH_NAME), - store.clone_activation_path(), - store.clone_progress_path(), - store.clone_failure_path(), - store.restore_ready_path(), - store.log_path(), - store.binding_path(), - ] { - fs::write(path, b"left").unwrap(); - } - let directory = locked.open_dir().unwrap(); - locked.remove_clone_files(&directory).unwrap(); - let mut left: Vec = fs::read_dir(store.dir()) - .unwrap() - .map(|entry| entry.unwrap().file_name().into_string().unwrap()) - .collect(); - left.sort(); - // The binding goes later, once the release or the removal has removed the clone's files. - let mut expected = [CLONE_BINDING_NAME, LOCK_NAME, LOG_NAME]; - expected.sort_unstable(); - assert_eq!(left, expected); - // Nothing to remove is no failure. - locked.remove_clone_files(&directory).unwrap(); -} - #[cfg(unix)] #[test] fn records_and_capabilities_that_are_not_regular_files_fail_at_once() { diff --git a/nvx-rs/src/host/supervisor/launch.rs b/nvx-rs/src/host/supervisor/launch.rs index b2829722b..9377e28f5 100644 --- a/nvx-rs/src/host/supervisor/launch.rs +++ b/nvx-rs/src/host/supervisor/launch.rs @@ -11,8 +11,7 @@ use crate::host::store::{ use crate::protocol::CAPABILITY_LEN; /// Launches a VMM in the sandbox of `locked`, through the store's launch journal, as the launch -/// `reserved`, which the caller reserved with [`LaunchId::generate`], or which the sandbox's lease -/// reserves, [`Lease::launch`](crate::host::template::Lease::launch), under the runtime identity +/// `reserved`, which the caller reserved with [`LaunchId::generate`], under the runtime identity /// `runtime`, with `capability` on its standard input. /// /// It begins the launch with [`LockedStore::begin_launch`], asks `command` for the VMM's command, @@ -26,8 +25,7 @@ use crate::protocol::CAPABILITY_LEN; /// cannot be ended then, because its VMM did not exit, keeps its journal for /// [`LockedStore::recover`], and the error says so. Fails as [`LockedStore::begin_launch`] does if /// the sandbox's VM runs, an interrupted launch has not been recovered, or the sandbox holds a -/// lease that does not let this lock begin `reserved`: one that reserves another launch, or that -/// another lock acquired, or whose launch has begun. +/// lease. pub fn launch<'s, 'h>( locked: &'s mut LockedStore, reserved: &LaunchId, diff --git a/nvx-rs/src/host/supervisor/mod.rs b/nvx-rs/src/host/supervisor/mod.rs index bbf7463f8..9e1278c34 100644 --- a/nvx-rs/src/host/supervisor/mod.rs +++ b/nvx-rs/src/host/supervisor/mod.rs @@ -11,8 +11,7 @@ //! # Launches //! //! [`launch`] runs the store's launch steps under the sandbox's lock, as a launch whose identity -//! the caller reserved beforehand, with [`LaunchId::generate`], or through a clone's lease, as -//! [Clones](#clones) describes: +//! the caller reserved beforehand, with [`LaunchId::generate`]: //! //! 1. [`LockedStore::begin_launch`] writes the journal's marker, which names the launch, and the //! caller's `command` builds the VMM's [`VmmCommand`] with the control endpoint that the @@ -125,79 +124,6 @@ //! recovery of a launch that did not record its VMM, the process that recovery verified to be that //! VMM. It never signals a process that reuses a recorded ID. //! -//! # Clones -//! -//! A clone restores a generation of a template that it leases from a -//! [template store](crate::host::template::Store), and its lease reserves its launch: -//! -//! 1. [`Store::acquire`](crate::host::template::Store::acquire) leases the generation, and writes -//! the lease's record in the store and the clone binding in the sandbox, durably, with the -//! identity of the launch that the lease reserves, -//! [`Lease::launch`](crate::host::template::Lease::launch), before any VMM of the clone -//! exists. It returns the sandbox's lock. -//! 2. Under that lock, the caller launches the clone with [`launch`], as the reserved launch, -//! which [`LockedStore::begin_launch`] requires of a sandbox that holds a lease, and which -//! begins once, under that lock alone, whatever its outcome. The supervisor prepares nothing of -//! the restore: the caller binds what the restore needs before the launch, such as its -//! restore-ready endpoint, and its `command` gives OpenVMM the restore's options, such as -//! `--restore-ready-path`. The clone's VMM restores the generation's snapshot in place, and on -//! Linux listens beside it, at the control socket that the lease reserves in the generation's -//! payload, named after the launch, which [`Starting::endpoint`] and the runtime record name, -//! as every launch's endpoint, rather than at the sandbox's own socket. -//! 3. Still under that lock, the caller checks, through its own transport, that the clone's -//! restore completed: that the VMM sent its restore-ready event, and that the clone was -//! activated, reported Ready and started, as the caller's protocol requires. Only then does it -//! commit the launch with [`Starting::commit`], which writes the runtime record. A failure ends -//! the launch with [`Starting::abandon`], through the VMM's verified identity: a launch that -//! cannot be ended keeps its journal, and the lease. -//! 4. Once the clone's VM has exited, as a stop, a termination, or a reattach verifies, -//! [`Store::release`](crate::host::template::Store::release) ends the lease, under the -//! sandbox's lock, taken through the lease's path, -//! [`Lease::sandbox`](crate::host::template::Lease::sandbox), as the acquisition took it. The -//! VMM removes its control socket as it exits, the stop or the reattach removes the one that an -//! abrupt exit leaves, and the release makes the socket's absence durable before the lease -//! ends. -//! -//! ## The lock order -//! -//! These steps, and every other transition of a template store or a sandbox, take their locks in -//! one order: the lifecycle lock of a capture class's slot, which serializes the class's captures, -//! as [`Store::refresh_template`](crate::host::template::Store::refresh_template) describes, then -//! the template store's lock, then the [lifecycle lock](super::store#lifecycle-locks) of a sandbox, -//! which guards its clone binding and its launch journal, or of a capture's transaction, the -//! sandbox store of the capture's VM, then the scratch image's claim. None waits for a lock while -//! it holds a later one: under its slot's lock, a capture, or its recovery, takes its -//! transaction's lock and then, once it has released it, the store's lock, to publish, and no -//! other; an acquisition releases the store's lock before it returns; a release, a launch, a stop, -//! a reattach, and a removal of the sandbox take no lock of the store; and a retirement takes no -//! lock of a sandbox. No lock but those of a capture lasts for a VM's lifetime: a capture holds -//! its slot's lock and its transaction's until its VM has ended, while the caller of a clone's -//! launch releases the sandbox's lock once the launch commits or ends, and the scratch image's -//! claim belongs to the VMM, which holds it until it exits. -//! -//! ## Crashes -//! -//! A crash at any step leaves records from which a later transition, under the same locks, decides -//! what becomes of the lease, which a release ends only once the sandbox records no VM and no -//! launch: -//! -//! - After the acquisition, the binding reserves a launch that never began, and that no later lock -//! begins: a release ends the lease at once, and another acquisition, which checks the -//! generation's eligibility again, reserves another launch. -//! - After the launch's journal, before or after the spawn, before its commit, [`reattach`] ends -//! the launch, as [`LockedStore::recover`] does, and the lease lasts until it has: a launch -//! that cannot be ended yet is [`Reattached::Indeterminate`], and keeps it. -//! - After the commit, the runtime record names the VM, which [`reattach`] holds again: the lease -//! lasts as long as the VM runs, until a stop or a termination verifies its exit. -//! - During a release, the binding goes first, durably, and leaves the record of the lease, which -//! a retirement removes. -//! -//! The evidence of the VM's exit decides each outcome. A record of an earlier boot, or of a process -//! ID that another process now has, names a VM that has exited: reattach clears it, and the lease -//! may be released. A VM that cannot be examined is indeterminate, and keeps its records, and so -//! the lease. A forced teardown that fails, or that does not end the VM, keeps its runtime record, -//! and the lease. -//! //! [`launch`]: fn@crate::host::supervisor::launch //! [`LaunchId::generate`]: crate::host::store::LaunchId::generate //! [`LockedStore::begin_launch`]: crate::host::store::LockedStore::begin_launch diff --git a/nvx-rs/src/host/supervisor/tests.rs b/nvx-rs/src/host/supervisor/tests.rs index 91f5f8637..7080032b8 100644 --- a/nvx-rs/src/host/supervisor/tests.rs +++ b/nvx-rs/src/host/supervisor/tests.rs @@ -12,24 +12,21 @@ use super::launch::launch_reporting; use super::reattach::reattach_with; use super::vmm::{Stoppable, stop_process, terminate_process}; use super::*; +use crate::ErrorKind; use crate::error::{Error, Result}; use crate::host::launch::MAX_TIMEOUT; use crate::host::platform; use crate::host::process::{BootId, Process, ProcessRef}; use crate::host::store::{ - ArtifactIdentity, CallerFiles, FORMAT, IdentityPart, JOURNAL_NAME, LaunchId, LockedStore, - RUNTIME_NAME, Recovery, RuntimeIdentity, SandboxStore, + ArtifactIdentity, FORMAT, IdentityPart, JOURNAL_NAME, LaunchId, RUNTIME_NAME, Recovery, + RuntimeIdentity, SandboxStore, }; -use crate::host::template::{self, Acquired, HostTime, Lease}; use crate::protocol::CAPABILITY_LEN; -use crate::{ErrorCode, ErrorKind}; /// A launch capability. const CAPABILITY: [u8; CAPABILITY_LEN] = [7; CAPABILITY_LEN]; /// The launch of the runtime records that [`record`] writes. const RECORDED_LAUNCH: &str = "5d41402abc4b2a76b9719d911017c592"; -/// The configuration record of these tests' clones. -const CONFIG: &str = "config.json"; /// Returns the identity `sha256:` and 64 copies of `digit`. fn artifact(digit: char) -> ArtifactIdentity { @@ -87,52 +84,6 @@ fn record_launch( .unwrap(); } -/// A clone, in a fresh temporary directory, that leases a generation of a template from the -/// template store that it returns, and whose launch, the one that its lease reserved, committed a -/// VM whose VMM is `process`; and the lease, with the lock of the clone's sandbox that the -/// acquisition returned. -fn committed_clone( - process: &ProcessRef, -) -> (tempfile::TempDir, template::Store, Lease, LockedStore) { - let directory = tempfile::tempdir().unwrap(); - let templates = template::Store::create(directory.path().join("templates")).unwrap(); - let generation = templates.publish_for_tests(); - let sandbox = directory.path().join("sandbox"); - let files = CallerFiles::new(CONFIG).unwrap(); - let provisioned = SandboxStore::lock_for_provision(&sandbox, &files, None).unwrap(); - fs::write(sandbox.join(CONFIG), b"{}").unwrap(); - drop(provisioned); - // A second after the capture, on the host that captured the generation. - let time = generation.time(); - let now = HostTime { - host_id: time.host_id, - boot_id: time.boot_id, - clock: time.host_clock, - monotonic_ns: time.capture_monotonic_ns + 1_000_000_000, - utc_ns: time.capture_utc_ns + 1_000_000_000, - }; - let Acquired { lease, mut locked } = templates - .acquire(generation.generation(), &sandbox, &files, &now, None) - .unwrap(); - // The reserved launch begins, and its commit replaces its journal with its runtime record. - drop( - locked - .begin_launch(lease.launch(), &runtime(), &CAPABILITY) - .unwrap(), - ); - record_launch(&locked, lease.launch().as_str(), process, &runtime()); - fs::remove_file(sandbox.join(JOURNAL_NAME)).unwrap(); - (directory, templates, lease, locked) -} - -/// Checks that `templates` refuses to release the lease of the sandbox of `locked`, whose VM may -/// still run, and that the sandbox keeps it. -fn refuses_release(templates: &template::Store, locked: &mut LockedStore, lease: &Lease) { - let error = templates.release(locked).unwrap_err(); - assert_eq!(error.code(), ErrorCode::SandboxRunning, "{error}"); - assert_eq!(Lease::of(locked).unwrap().as_ref(), Some(lease)); -} - /// A VMM whose state drives a stop's decisions without a process: it exits once the graceful /// step runs, if it heeds that step, or once it is killed, if killing ends it. #[derive(Default)] @@ -618,7 +569,9 @@ fn a_launch_whose_caller_cannot_record_its_failure_keeps_its_journal_for_recover #[test] fn a_reattach_whose_caller_cannot_record_an_exit_keeps_the_record_until_one_can() { - let (_directory, templates, lease, mut locked) = committed_clone(&this_process()); + let (_directory, store) = store(); + record(&store, &this_process(), &runtime()); + let mut locked = store.lock(None).unwrap(); // The recorded VMM exited, but the caller cannot record what ended it. let exited = |_: &ProcessRef| -> io::Result> { Ok(None) }; let error = reattach_with(&mut locked, &runtime(), exited, |_| { @@ -626,18 +579,16 @@ fn a_reattach_whose_caller_cannot_record_an_exit_keeps_the_record_until_one_can( }) .unwrap_err(); assert_eq!(error.message(), "the test cannot record the exit"); - // The record, and with it the lease, remain for a later reattach. + // The record remains for a later reattach. assert!(locked.runtime().unwrap().is_some()); - refuses_release(&templates, &mut locked, &lease); let Reattached::Exited { record, recovery } = reattach_with(&mut locked, &runtime(), exited, |_| Ok(())).unwrap() else { panic!("a VMM that exited was not reported as exited"); }; - assert_eq!(record.unwrap().launch(), lease.launch()); + assert_eq!(record.unwrap().launch().as_str(), RECORDED_LAUNCH); assert_eq!(recovery, Recovery::Clean); assert!(locked.runtime().unwrap().is_none()); - assert_eq!(templates.release(&mut locked).unwrap(), Some(lease)); } #[test] @@ -671,62 +622,16 @@ fn a_launch_whose_vmm_cannot_start_ends() { } #[test] -fn a_clone_whose_record_names_a_vmm_that_exited_may_release_its_lease() { - let live = this_process(); - let mut earlier_boot = *live.boot.as_bytes(); - earlier_boot[0] ^= 1; - for exited in [ - // A record of an earlier boot names a VMM that has exited. - ProcessRef { - boot: BootId::from_bytes(earlier_boot), - ..live.clone() - }, - // The recorded ID now names another process: this one. - ProcessRef { - start_time: live.start_time + 1, - ..live.clone() - }, - ] { - let (_directory, templates, lease, mut locked) = committed_clone(&exited); - // While the record names the clone's VM, the lease stays. - refuses_release(&templates, &mut locked, &lease); - let Reattached::Exited { record, recovery } = reattach(&mut locked, &runtime()).unwrap() - else { - panic!("{exited:?} names a live VMM"); - }; - assert_eq!(record.unwrap().launch(), lease.launch()); - assert_eq!(recovery, Recovery::Clean); - // Reattach cleared the record of the VM that exited, so the release ends the lease. - assert_eq!(templates.release(&mut locked).unwrap(), Some(lease)); - assert_eq!(Lease::of(&locked).unwrap(), None); - } -} - -#[test] -fn a_clone_whose_vmm_cannot_be_examined_keeps_its_lease() { - let (_directory, templates, lease, mut locked) = committed_clone(&this_process()); - let hidden = |_: &ProcessRef| -> io::Result> { - Err(io::Error::new(io::ErrorKind::PermissionDenied, "hidden")) - }; - let Reattached::Indeterminate(reason) = - reattach_with(&mut locked, &runtime(), hidden, |_| Ok(())).unwrap() - else { - panic!("a VMM that cannot be examined was classified"); - }; - assert!(reason.ends_with("cannot be examined: hidden"), "{reason}"); - assert!(locked.runtime().unwrap().is_some()); - refuses_release(&templates, &mut locked, &lease); -} - -#[test] -fn a_clone_whose_forced_teardown_fails_keeps_its_lease_until_one_ends_its_vmm() { - let (_directory, templates, lease, mut locked) = committed_clone(&this_process()); +fn a_forced_teardown_that_fails_keeps_the_record_until_one_ends_the_vmm() { + let (_directory, store) = store(); + record(&store, &this_process(), &runtime()); + let mut locked = store.lock(None).unwrap(); let Reattached::Live(vmm) = reattach(&mut locked, &runtime()).unwrap() else { - panic!("the clone's VMM was not reattached"); + panic!("the recorded VMM was not reattached"); }; // This test process stands for the VMM, which these teardowns reach only through stand-ins. let bound = Duration::from_secs(10); - // A teardown that cannot terminate the VMM fails, and keeps its record, and so the lease. + // A teardown that cannot terminate the VMM fails, and keeps its record. let unkillable = Scripted { unkillable: true, ..Scripted::default() @@ -738,15 +643,15 @@ fn a_clone_whose_forced_teardown_fails_keeps_its_lease_until_one_ends_its_vmm() error.message(), format!("cannot terminate VMM process {}", std::process::id()) ); - refuses_release(&templates, &mut locked, &lease); - // One that does not end the VMM within its bound keeps them too. + assert_eq!(locked.runtime().unwrap().as_ref(), Some(vmm.record())); + // One that does not end the VMM within its bound keeps it too. let survivor = Scripted::default(); assert_eq!( vmm.terminate_with(&mut locked, bound, &survivor).unwrap(), StopOutcome::TimedOut ); - refuses_release(&templates, &mut locked, &lease); - // One that ends it clears the record, and the release ends the lease. + assert_eq!(locked.runtime().unwrap().as_ref(), Some(vmm.record())); + // One that ends it clears the record. let dying = Scripted { dies_when_killed: true, ..Scripted::default() @@ -756,14 +661,15 @@ fn a_clone_whose_forced_teardown_fails_keeps_its_lease_until_one_ends_its_vmm() StopOutcome::Forced ); assert!(locked.runtime().unwrap().is_none()); - assert_eq!(templates.release(&mut locked).unwrap(), Some(lease)); } #[test] fn only_a_stop_unless_exited_keeps_the_record_of_a_vmm_that_had_exited() { - let (_directory, templates, lease, mut locked) = committed_clone(&this_process()); + let (_directory, store) = store(); + record(&store, &this_process(), &runtime()); + let mut locked = store.lock(None).unwrap(); let Reattached::Live(vmm) = reattach(&mut locked, &runtime()).unwrap() else { - panic!("the clone's VMM was not reattached"); + panic!("the recorded VMM was not reattached"); }; // This test process stands for the VMM, which these stops reach only through a stand-in, // which exited once its holder found it running, before the stops began. @@ -775,16 +681,14 @@ fn only_a_stop_unless_exited_keeps_the_record_of_a_vmm_that_had_exited() { let exited = Scripted::default(); exited.exited.set(true); // The stop takes no step, and keeps the record, which tells a reattach that the VM exited - // unheld, and with it the lease. + // unheld. let stopped = vmm .stop_unless_exited_with(&mut locked, deadline, forced, never, &exited) .unwrap(); assert!(stopped.is_none(), "{stopped:?}"); assert_eq!(exited.operations(), ["check"]); assert_eq!(locked.runtime().unwrap().as_ref(), Some(vmm.record())); - refuses_release(&templates, &mut locked, &lease); - // A stop that ends such a VM as one that it stopped clears the record, so the release ends - // the lease. + // A stop that ends such a VM as one that it stopped clears the record. let stopped = vmm .stop_with(&mut locked, deadline, forced, never, &exited) .unwrap(); @@ -792,18 +696,19 @@ fn only_a_stop_unless_exited_keeps_the_record_of_a_vmm_that_had_exited() { assert!(stopped.graceful_error.is_none()); assert_eq!(exited.operations(), ["check", "check"]); assert!(locked.runtime().unwrap().is_none()); - assert_eq!(templates.release(&mut locked).unwrap(), Some(lease)); } #[test] fn a_stop_unless_exited_stops_a_running_vmm_as_a_stop_does() { - let (_directory, templates, lease, mut locked) = committed_clone(&this_process()); + let (_directory, store) = store(); + record(&store, &this_process(), &runtime()); + let mut locked = store.lock(None).unwrap(); let Reattached::Live(vmm) = reattach(&mut locked, &runtime()).unwrap() else { - panic!("the clone's VMM was not reattached"); + panic!("the recorded VMM was not reattached"); }; // This test process stands for the VMM, which these stops reach only through stand-ins. let deadline = Instant::now() + Duration::from_secs(60); - // One that outlives the stop keeps its record, and the lease. + // One that outlives the stop keeps its record. let survivor = Scripted::default(); let stopped = vmm .stop_unless_exited_with( @@ -817,7 +722,7 @@ fn a_stop_unless_exited_stops_a_running_vmm_as_a_stop_does() { .expect("the stop of a VMM that runs"); assert_eq!(stopped.outcome, StopOutcome::TimedOut); assert_eq!(survivor.operations(), ["check", "wait"]); - refuses_release(&templates, &mut locked, &lease); + assert_eq!(locked.runtime().unwrap().as_ref(), Some(vmm.record())); // One that heeds the graceful step exits through the stop, which clears its record. let heeding = Scripted::default(); let shut_down = |_: &Vmm, _: Instant| { @@ -831,7 +736,6 @@ fn a_stop_unless_exited_stops_a_running_vmm_as_a_stop_does() { assert_eq!(stopped.outcome, StopOutcome::Graceful); assert_eq!(heeding.operations(), ["check", "wait"]); assert!(locked.runtime().unwrap().is_none()); - assert_eq!(templates.release(&mut locked).unwrap(), Some(lease)); } #[test] diff --git a/nvx-rs/src/host/sys/mod.rs b/nvx-rs/src/host/sys/mod.rs index 574cbd89a..e9e87bba9 100644 --- a/nvx-rs/src/host/sys/mod.rs +++ b/nvx-rs/src/host/sys/mod.rs @@ -10,7 +10,7 @@ //! through libc's `prctl` too, since `nix` does not wrap seccomp. The Windows platform module //! still calls `windows-sys` directly. -#[cfg(target_os = "linux")] +#[cfg(all(feature = "cli", target_os = "linux"))] mod fs; #[cfg(feature = "cli")] mod interrupt; @@ -23,7 +23,7 @@ mod socket; pub(crate) use nix::errno::Errno; use nix::unistd::{getegid, geteuid}; -#[cfg(target_os = "linux")] +#[cfg(all(feature = "cli", target_os = "linux"))] pub(crate) use fs::rename_no_replace; #[cfg(feature = "cli")] pub(crate) use interrupt::{interrupted, watch_interrupts, write_interruptibly}; diff --git a/nvx-rs/src/host/template/mod.rs b/nvx-rs/src/host/template/mod.rs deleted file mode 100644 index 9f3997a5e..000000000 --- a/nvx-rs/src/host/template/mod.rs +++ /dev/null @@ -1,65 +0,0 @@ -//! Templates: captures of a microVM from which independent clones are restored, the metadata -//! that decides which clones a template can serve, and on which hosts, and the store that keeps -//! them. -//! -//! A template is a snapshot that OpenVMM captured at its platform tier, after it consumed the -//! invariant configuration of a microVM alone, for clones: a VM booted with a placeholder -//! `distro`, a fresh scratch, and a dormant filesystem slot, which each clone binds to its own -//! image, scratch, and export when it is restored. Its compatibility key, [`TemplateKey`], is what -//! a clone and the host that restores it must match: the runtime and the contracts through which -//! the template is restored, and the machine that it restores. -//! -//! - [`CaptureClass`] holds the invariant inputs of a capture, known before it: the runtime, the -//! guest's build, the backend, the CPU profile, the resources, the geometries of the disks, the -//! network, and the other invariant settings. -//! - [`TemplateManifest::read`] reads the manifest that OpenVMM wrote for a capture, with its time -//! contract, and refuses one that is not a template's. -//! - [`TemplateKey::new`] derives the key of a template from its class and its manifest, after it -//! checks that the capture made the machine that its class requested. -//! - [`Store`] holds the generations of templates that a host published, apart from every -//! sandbox's store: their payloads, which never change, and the versioned [`Index`] of them, -//! which names each by its [`StoreGeneration`](crate::activation::StoreGeneration), at a -//! location that the adapter chooses. -//! - [`Store::create_template`] and [`Store::refresh_template`] capture a template of a class, in -//! the VM that its [`TemplatePlan`] describes, which the supervisor launches, and publish it as -//! a generation of the store; [`Store::recover_captures`] ends the captures that a crash -//! interrupted. -//! - [`TimeContract::eligibility`] decides whether a clone may restore a capture at a host's -//! time, [`HostTime`], by the restore window of its time contract, apart from its -//! compatibility. -//! - [`Store::acquire`] leases an eligible generation to a clone, with a binding in the clone's -//! sandbox, until [`Store::release`] ends the lease once the clone's VM has exited. A tombstone -//! quarantines a generation, which takes no new lease, and [`Store::retire`] deletes it once its -//! leases have drained. -//! -//! # Snapshot metadata -//! -//! The metadata types remain aliases of [`snapshot`](super::snapshot)'s types for template -//! users. That module describes their canonical encoding and conformance vectors. - -mod plan; -mod store; - -pub use super::snapshot::{ - BlockGeometry, CaptureClass, ClassIdentity, Downtime, DowntimeSource, Eligibility, HostClock, - HostTime, Invariants, KeyIdentity, MANIFEST_NAME, MAX_DOWNTIME_NS, MAX_MANIFEST_LEN, - SnapshotKey as TemplateKey, SnapshotManifest as TemplateManifest, TimeContract, -}; -pub(super) use super::snapshot::{eligibility, key, manifest}; - -pub use self::plan::{MEMORY_BLOCK_SIZE, SCRATCH_NAME, TemplatePlan, TemplateResources}; -pub use self::store::{ - Acquired, CAPTURE_RECORD_NAME, CAPTURES_NAME, Capture, Created, GENERATIONS_NAME, Generation, - INDEX_LOCK_NAME, INDEX_NAME, Index, KEY_NAME, LEASES_NAME, Lease, LeaseId, MAX_GENERATIONS, - Retirement, SNAPSHOT_NAME, STORE_FORMAT, STORE_RECORD_NAME, StorageGeneration, Store, StoreId, - TOMBSTONES_NAME, TRANSACTION_NAME, -}; - -impl Generation { - /// Returns whether a clone may restore the generation at the host's time `now`, by the - /// restore window of its capture's time contract, as [`TimeContract::eligibility`] decides - /// it. - pub fn eligibility(&self, now: &HostTime) -> Eligibility { - self.time().eligibility(now) - } -} diff --git a/nvx-rs/src/host/template/plan.rs b/nvx-rs/src/host/template/plan.rs deleted file mode 100644 index c3b6ebda0..000000000 --- a/nvx-rs/src/host/template/plan.rs +++ /dev/null @@ -1,497 +0,0 @@ -//! The plan of a template's capture, [`TemplatePlan`]: the VM that a template's capture boots, -//! lowered from an adapter's plan, and the options with which OpenVMM captures it. - -use std::path::{Path, PathBuf}; - -use super::manifest::PROCESSOR_COUNTS; -use super::store::{SNAPSHOT_NAME, StorageGeneration}; -use crate::error::{Error, Result}; -use crate::host::paths; -#[cfg(doc)] -use crate::host::sandbox::SandboxNetwork; -use crate::host::sandbox::distro::DistroBinding; -use crate::host::sandbox::plan::bound_image; -use crate::host::sandbox::{SandboxPlan, SandboxUpper, reject_container_path, reject_disk_path}; - -/// Bytes in a mebibyte, the unit in which the plan writes OpenVMM's `--memory` and -/// `--memory-capacity`. -const MIB: u64 = 1024 * 1024; - -/// The size of the blocks of a microVM's RAM, in bytes: 128 MiB, as OpenVMM's microVM profile -/// fixes it. A template's RAM and its RAM capacity are whole blocks, and a restore expands the RAM -/// by whole blocks. -pub const MEMORY_BLOCK_SIZE: u64 = 128 * MIB; - -/// The name of the dormant scratch device's file in a capture's transaction directory: a sparse -/// file of the template's scratch length, which the template does not keep. -pub const SCRATCH_NAME: &str = "scratch.raw"; - -/// The kernel command line of a template's VM, with which the guest's init hands the VM to its -/// sandbox control. -const COMMAND_LINE: &str = "nvx_sandbox=1"; - -/// The kernel command-line parameter that limits the processors that the guest brings online at -/// boot, and with which OpenVMM records them as the template's processors online at boot. -const BOOT_PROCESSORS: &str = "maxcpus"; - -/// The snapshot tier of a template's capture: OpenVMM's clone point of the invariant -/// configuration alone. -const TIER: &str = "platform"; - -/// The block identity of a template's capture: the bound-block storage generation, which OpenVMM -/// takes in place of a digest of each block's content. -const BLOCK_IDENTITY: &str = "generation"; - -/// The logical block size that OpenVMM presents for a sandbox block device, of which the device's -/// length is a multiple. -const LOGICAL_BLOCK_SIZE: u64 = 512; - -/// The resources of a template's VM: its processors, of which a prefix is online at boot, and its -/// RAM, with the capacity to which a restore may expand it. -/// -/// A template's VM has the processors of the largest of its clones, its processor capacity, of -/// which the first, those online at boot, are online when it is captured. A restore brings a -/// prefix of the others online, so that one template serves clones of each processor count from -/// those online at boot to its capacity. A template whose processors are all online at boot -/// serves clones of its capacity alone. [`TemplatePlan::new`] checks the resources. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] -pub struct TemplateResources { - /// The processor capacity: 1, 2, 4, or 8. - pub processors: u32, - /// The processors online at boot: 1, 2, 4, or 8, at most [`processors`](Self::processors). - /// Fewer than the capacity limit those that the guest brings online at boot, as the - /// [kernel command line](TemplatePlan::kernel_command_line) says. - pub boot_processors: u32, - /// The RAM at boot, in bytes: a nonzero multiple of [`MEMORY_BLOCK_SIZE`]. - pub memory: u64, - /// The RAM capacity, in bytes, to which a restore may expand the RAM: a multiple of - /// [`MEMORY_BLOCK_SIZE`] of at least [`memory`](Self::memory). - pub memory_capacity: u64, -} - -/// The plan of a template's capture: the VM that the capture boots, lowered from an adapter's -/// plan, and the options with which OpenVMM captures it, which -/// [`openvmm_arguments`](Self::openvmm_arguments) and -/// [`kernel_command_line`](Self::kernel_command_line) render. -/// -/// A template's VM boots its invariant configuration alone, for clones, each of which binds its -/// own image, upper, and export when it is restored. [`new`](Self::new) lowers an adapter's plan, -/// which [`SandboxPlan::adapter`] makes, into that VM: -/// -/// - The plan's `distro` is the placeholder: a trusted platform artifact, which its -/// [`DistroBinding`] binds, and whose length is the capacity of every clone's image. -/// - A dormant scratch device takes the place of the plan's upper, which is a clone's: a sparse -/// file of the template's scratch length, [`SCRATCH_NAME`] in the capture's transaction -/// directory, which sets the device's length and block sizes, and which the template does not -/// keep. -/// - The filesystem slot, which every cold boot exposes, stays dormant: the plan exports nothing. -/// - The plan's network gives the template its fixed NIC, at the network's guest address, under -/// the complete deny-all policy, whatever the network's own, as -/// [`SandboxNetwork::template_arguments`] lowers it: egress denied without rules, ingress and -/// host-loopback access denied, without a proxy or forwards. It is the one policy that a -/// restore may replace, so each clone's restore installs the clone's own policy on the NIC. A -/// plan without a network, or with one that lets nothing through, which attaches no NIC, gives -/// the template none. -/// - Its [`TemplateResources`]: its processor capacity, of which those online at boot are online -/// when it is captured, and its RAM, with the capacity to which a restore may expand it, -/// OpenVMM's `--memory-capacity`. -/// -/// # The kernel command line -/// -/// OpenVMM keeps the effective kernel command line of a template, which its -/// [key](super::TemplateKey) holds and every clone restores, and refuses a capture whose command -/// line holds a token outside the allowlist of its platform tier. The template's own command line -/// is `nvx_sandbox=1`, with which the guest's init hands the VM to its sandbox control, and for a -/// template that boots with fewer processors online than its capacity, `maxcpus=` with the -/// processors online at boot, which limits those that the guest brings online at boot, and which -/// OpenVMM records as the template's, so that each restore may bring a prefix of the others -/// online. OpenVMM writes the other tokens itself, each on the allowlist: the early console, -/// console, reboot, and panic settings, `nr_cpus=` with the processor capacity, -/// `nvx_snapshot_tier=platform`, the virtio devices and the control tty that the guest discovers, -/// and the NIC's addresses. The arguments name no workload identity, lifecycle, share, or -/// published listener, for which OpenVMM would write other tokens, and under the NIC's deny-all -/// policy, OpenVMM names no DNS server either. The command line therefore holds no image -/// identity, hostname, share, proxy, DNS server, entrypoint, or other tenant data, and every -/// clone of the template has the same. -/// -/// # The capture -/// -/// The capture's options are `--snapshot-tier platform`; `--snapshot-destination`, the -/// destination [`SNAPSHOT_NAME`] in the capture's transaction directory, by which a generation of -/// a [`Store`](super::Store) names its snapshot too; and `--snapshot-block-identity generation`, -/// with the capture's [`StorageGeneration`] as `--snapshot-generation-id`, so that the capture -/// hashes no block device. The template keeps no storage generation: OpenVMM unbinds its -/// `distro`, and its scratch is fresh. -/// -/// The transaction directory is the capture's own, and the destination must not exist when the -/// capture launches. OpenVMM backs the VM's RAM with a file that it creates beside the -/// destination, and publishes the snapshot at the destination, so every file of the capture is in -/// the transaction directory. -/// -/// The caller creates the transaction directory and the sparse scratch file, checks the -/// placeholder against its binding, as [`DistroBinding::check`] does, and adds its own arguments, -/// such as the hypervisor, the kernel and the initramfs, the CPU profile, and the consoles. It -/// passes [`kernel_command_line`](Self::kernel_command_line) as OpenVMM's `--cmdline`, alone, and -/// names no workload identity, lifecycle, share, or RAM backing file of its own: -/// -/// ```no_run -/// use std::path::Path; -/// -/// use nvx::host::sandbox::distro::DistroBinding; -/// use nvx::host::sandbox::SandboxPlan; -/// use nvx::host::template::{MEMORY_BLOCK_SIZE, StorageGeneration, TemplatePlan, TemplateResources}; -/// -/// # fn capture( -/// # adapter: &SandboxPlan, -/// # binding: &DistroBinding, -/// # transaction: &Path, -/// # generation: StorageGeneration, -/// # ) -> nvx::Result<()> { -/// let resources = TemplateResources { -/// processors: 4, -/// boot_processors: 1, -/// memory: 4 * MEMORY_BLOCK_SIZE, -/// memory_capacity: 16 * MEMORY_BLOCK_SIZE, -/// }; -/// let template = -/// TemplatePlan::new(adapter, binding, 64 << 20, resources, transaction, generation)?; -/// let mut arguments = template.openvmm_arguments()?; -/// arguments.extend(["--cmdline".to_owned(), template.kernel_command_line().to_owned()]); -/// // ... the caller's own arguments: the hypervisor, the kernel and the initramfs, the CPU -/// // profile, and the consoles ... -/// # Ok(()) -/// # } -/// ``` -#[derive(Clone, Debug, PartialEq, Eq)] -pub struct TemplatePlan { - /// The capture's devices but its NIC: an adapter's plan of the placeholder over the dormant - /// scratch device, without a network. - plan: SandboxPlan, - /// OpenVMM's arguments of the template's NIC, under the complete deny-all policy, or none for - /// a template without a NIC. - nic: Vec, - /// The binding of the placeholder. - binding: DistroBinding, - /// The length of the dormant scratch device, in bytes. - scratch_length: u64, - /// The processors and RAM. - resources: TemplateResources, - /// The kernel command line of the capture's VM. - command_line: String, - /// The capture's transaction directory, spelled with its components alone. - transaction: PathBuf, - /// The snapshot's destination in the transaction directory. - destination: PathBuf, - /// The bound-block storage generation of the capture. - generation: StorageGeneration, -} - -impl TemplatePlan { - /// Lowers `adapter`, an adapter's plan, which [`SandboxPlan::adapter`] makes, into the plan of - /// a template's capture: of its placeholder, which `binding` binds, over a dormant scratch - /// device of `scratch_length` bytes, with the NIC of its network, the processors and RAM of - /// `resources`, the capture's transaction directory `transaction`, spelled with its components - /// alone, as [`Path::components`] reads them, and its bound-block storage generation - /// `generation`, which is apart from the other [namespaces](super::Store#four-namespaces) of a - /// template's generations. - /// - /// The plan's upper is no part of the template, which attaches its dormant scratch device in - /// the upper's place, whatever the upper. - /// - /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported), checking in this - /// order: a plan of the sandbox profile, with workload settings; a plan whose image is not the - /// format and identity that `binding` names, as for a binding whose identity is not a UUID in - /// its canonical text; a placeholder whose path is relative, which OpenVMM would resolve - /// against the directory in which it runs, holds a comma or a semicolon, which OpenVMM's block - /// device options would split, or whose name ends in `.vhd`, `.vhdx`, `.vmgs`, or `.iso`, - /// which OpenVMM would not attach as a raw image; a placeholder whose binding records - /// a length, or a scratch length, that is not a nonzero multiple of the 512 bytes of the - /// logical blocks that OpenVMM presents; - /// processors other than 1, 2, 4, or 8, or processors online at boot other than 1, 2, 4, or - /// 8, or more than the processors; RAM or a RAM capacity that is not a nonzero multiple - /// of [`MEMORY_BLOCK_SIZE`], or a capacity below the RAM; and a transaction directory that is - /// not valid Unicode, that is relative, which OpenVMM would resolve against the directory in - /// which it runs, or whose path holds a comma or a semicolon, which OpenVMM's block device - /// options would split. - /// - /// It reads, opens, and creates nothing. - pub fn new( - adapter: &SandboxPlan, - binding: &DistroBinding, - scratch_length: u64, - resources: TemplateResources, - transaction: &Path, - generation: StorageGeneration, - ) -> Result { - if adapter.settings().is_some() { - return Err(Error::unsupported( - "a template lowers an adapter's plan, not a plan of the sandbox profile, whose \ - workload settings no template captures", - )); - } - let distro = adapter.distro(); - if bound_image(binding)? != distro.image() { - return Err(Error::unsupported(format!( - "the placeholder {} is not the {} layer with identity {} that its distro binding \ - names", - distro.path().display(), - binding.format(), - binding.identity() - ))); - } - // OpenVMM attaches the placeholder at the path that its arguments name, which they must - // hold whole, as a raw image, and which OpenVMM would resolve against the directory in - // which it runs if it were relative. - let placeholder = distro.path(); - if !placeholder.is_absolute() { - return Err(Error::unsupported(format!( - "the placeholder of a template's capture must be absolute, since OpenVMM would \ - resolve it against the directory in which it runs: {}", - placeholder.display() - ))); - } - let shown = placeholder.display().to_string(); - reject_disk_path(placeholder, &shown)?; - reject_container_path(placeholder, &shown)?; - let capacity = binding.image().length(); - if !is_block_length(capacity) { - return Err(Error::unsupported(format!( - "the placeholder {} of {capacity} bytes cannot be attached: OpenVMM attaches block \ - devices of nonzero multiples of {LOGICAL_BLOCK_SIZE} bytes", - distro.path().display() - ))); - } - if !is_block_length(scratch_length) { - return Err(Error::unsupported(format!( - "a template's scratch of {scratch_length} bytes cannot be attached: OpenVMM \ - attaches block devices of nonzero multiples of {LOGICAL_BLOCK_SIZE} bytes" - ))); - } - check_resources(&resources)?; - let nic = match adapter.network() { - Some(network) => network.template_arguments()?, - None => Vec::new(), - }; - let transaction = transaction_directory(transaction)?; - let scratch = transaction.join(SCRATCH_NAME); - reject_disk_path(&scratch, &scratch.display().to_string())?; - let destination = transaction.join(SNAPSHOT_NAME); - let plan = SandboxPlan::adapter(distro.clone(), SandboxUpper::scratch_at(scratch)); - // A template whose processors are all online at boot needs no limit, and records none, - // as OpenVMM records none without the parameter. - let command_line = if resources.boot_processors < resources.processors { - format!( - "{COMMAND_LINE} {BOOT_PROCESSORS}={}", - resources.boot_processors - ) - } else { - COMMAND_LINE.to_owned() - }; - Ok(Self { - plan, - nic, - binding: binding.clone(), - scratch_length, - resources, - command_line, - transaction, - destination, - generation, - }) - } - - /// Returns the capture's devices but its NIC: an adapter's plan of the placeholder over the - /// dormant scratch device, without a network, whose NIC [`nic_arguments`](Self::nic_arguments) - /// attaches. - pub fn plan(&self) -> &SandboxPlan { - &self.plan - } - - /// Returns OpenVMM's arguments of the template's NIC, at the guest address of the adapter's - /// network, under the complete deny-all policy, as [`SandboxNetwork::template_arguments`] - /// lowers them, or none for a template without a NIC. - pub fn nic_arguments(&self) -> &[String] { - &self.nic - } - - /// Returns the binding of the placeholder. - pub fn binding(&self) -> &DistroBinding { - &self.binding - } - - /// Returns the length of the placeholder, in bytes, as its binding records it: the capacity - /// of every clone's image. - pub fn image_capacity(&self) -> u64 { - self.binding.image().length() - } - - /// Returns the dormant scratch device's file, [`SCRATCH_NAME`] in the transaction directory. - pub fn scratch_path(&self) -> &Path { - self.plan - .upper() - .scratch_path() - .expect("a template's upper is its dormant scratch device") - } - - /// Returns the length of the dormant scratch device, in bytes, which its sparse file sets. - pub fn scratch_length(&self) -> u64 { - self.scratch_length - } - - /// Returns the processors and RAM. - pub fn resources(&self) -> TemplateResources { - self.resources - } - - /// Returns the capture's transaction directory. - pub fn transaction(&self) -> &Path { - self.transaction.as_path() - } - - /// Returns the snapshot's destination, [`SNAPSHOT_NAME`] in the transaction directory. - pub fn destination(&self) -> &Path { - self.destination.as_path() - } - - /// Returns the bound-block storage generation of the capture. - pub fn generation(&self) -> StorageGeneration { - self.generation - } - - /// Returns the kernel command line of the capture's VM, which the caller passes as OpenVMM's - /// `--cmdline`, alone: `nvx_sandbox=1`, then, for a template that boots with fewer processors - /// online than its capacity, `maxcpus=` with the processors online at boot, as - /// [The kernel command line](Self#the-kernel-command-line) describes. - /// - /// It takes the place of the command line of [`plan`](Self::plan), which - /// [`SandboxPlan::kernel_command_line`] and a [`PreparedLaunch`] render: that one names the - /// boot tokens of the plan's root, the identity of its image among them, which a clone binds - /// when it is restored, and which OpenVMM's platform allowlist refuses. - /// - /// [`PreparedLaunch`]: crate::host::sandbox::PreparedLaunch - pub fn kernel_command_line(&self) -> &str { - &self.command_line - } - - /// Returns OpenVMM's arguments of the capture: those of its devices, as - /// [`SandboxPlan::openvmm_arguments`] renders them for [`plan`](Self::plan), the microVM - /// machine, the placeholder as the read-only `distro` block device, and the dormant scratch - /// device as the `scratch` one; then those of the NIC, if any, - /// [`nic_arguments`](Self::nic_arguments); then those of - /// [`capture_arguments`](Self::capture_arguments). - /// - /// The arguments name the block devices by their paths, which OpenVMM opens itself. A capture - /// opens each without following a link at its path, and on Windows without sharing write - /// access, so each path must name the file itself, which no other process holds open for - /// writing. The arguments hold neither the kernel command line, which - /// [`kernel_command_line`](Self::kernel_command_line) gives, nor the caller's own arguments. - /// - /// Fails as [`SandboxPlan::openvmm_arguments`] does. - pub fn openvmm_arguments(&self) -> Result> { - let mut arguments = self.plan.openvmm_arguments()?; - arguments.extend_from_slice(&self.nic); - arguments.extend(self.capture_arguments()); - Ok(arguments) - } - - /// Returns OpenVMM's arguments of the capture besides those of its devices: `--processors` - /// with the processor capacity, `--memory` and `--memory-capacity` in MiB, and the capture's - /// options, `--snapshot-destination`, `--snapshot-tier platform`, - /// `--snapshot-block-identity generation`, and `--snapshot-generation-id`. The processors - /// online at boot are the kernel command line's. - /// - /// A caller that renders the arguments of the devices itself adds these to them. - pub fn capture_arguments(&self) -> Vec { - let TemplateResources { - processors, - memory, - memory_capacity, - .. - } = self.resources; - [ - ("--processors", processors.to_string()), - ("--memory", mebibytes(memory)), - ("--memory-capacity", mebibytes(memory_capacity)), - ( - "--snapshot-destination", - self.destination.display().to_string(), - ), - ("--snapshot-tier", TIER.to_owned()), - ("--snapshot-block-identity", BLOCK_IDENTITY.to_owned()), - ("--snapshot-generation-id", self.generation.to_string()), - ] - .into_iter() - .flat_map(|(option, value)| [option.to_owned(), value]) - .collect() - } -} - -/// Returns whether OpenVMM attaches a sandbox block device of `length` bytes: a nonzero multiple -/// of [`LOGICAL_BLOCK_SIZE`]. -fn is_block_length(length: u64) -> bool { - length != 0 && length.is_multiple_of(LOGICAL_BLOCK_SIZE) -} - -/// Refuses `resources` that no template's VM has: processors other than 1, 2, 4, or 8, processors -/// online at boot other than 1, 2, 4, or 8, or more than the processors, and RAM or a RAM capacity -/// that is not a nonzero multiple of [`MEMORY_BLOCK_SIZE`], or a capacity below the RAM. -fn check_resources(resources: &TemplateResources) -> Result<()> { - let TemplateResources { - processors, - boot_processors, - memory, - memory_capacity, - } = *resources; - if !PROCESSOR_COUNTS.contains(&processors) { - return Err(Error::unsupported(format!( - "a template's VM has 1, 2, 4, or 8 processors, not {processors}" - ))); - } - if !PROCESSOR_COUNTS.contains(&boot_processors) || boot_processors > processors { - return Err(Error::unsupported(format!( - "a template's VM boots with 1, 2, 4, or 8 of its {processors} processors online, not \ - {boot_processors}" - ))); - } - if memory == 0 - || !memory.is_multiple_of(MEMORY_BLOCK_SIZE) - || !memory_capacity.is_multiple_of(MEMORY_BLOCK_SIZE) - || memory_capacity < memory - { - return Err(Error::unsupported(format!( - "a template's VM needs nonzero RAM, and a RAM capacity of at least it, each a multiple \ - of {} MiB, not {memory} bytes of RAM and a capacity of {memory_capacity} bytes", - MEMORY_BLOCK_SIZE / MIB - ))); - } - Ok(()) -} - -/// Returns the transaction directory `path`, spelled with its components alone, as -/// [`Path::components`] reads them, which must be valid Unicode and absolute. -fn transaction_directory(path: &Path) -> Result { - if path.to_str().is_none() { - return Err(Error::unsupported(format!( - "the transaction directory of a template's capture is not valid Unicode: {}", - path.display() - ))); - } - let directory = paths::normalized(path); - if !directory.is_absolute() { - return Err(Error::unsupported(format!( - "the transaction directory of a template's capture must be absolute, since OpenVMM \ - would resolve it against the directory in which it runs: {}", - directory.display() - ))); - } - Ok(directory) -} - -/// Returns `bytes`, a multiple of [`MIB`], as OpenVMM's memory sizes write it: the number of MiB -/// and `M`. -fn mebibytes(bytes: u64) -> String { - format!("{}M", bytes / MIB) -} - -#[cfg(test)] -mod tests; diff --git a/nvx-rs/src/host/template/plan/tests.rs b/nvx-rs/src/host/template/plan/tests.rs deleted file mode 100644 index 3144eb883..000000000 --- a/nvx-rs/src/host/template/plan/tests.rs +++ /dev/null @@ -1,1031 +0,0 @@ -//! Tests of the plans of templates' captures: the lowering of adapters' plans, the rendered -//! arguments, the kernel command line against OpenVMM's platform allowlist, and the refusals. - -use std::path::Path; - -use serde_json::json; - -use super::*; -use crate::ErrorKind; -use crate::host::network::{ - Access, EgressRule, ForwardProtocol, HostLoopback, LoopbackForward, NetworkPolicy, - NetworkProxy, Protocol, -}; -use crate::host::sandbox::distro::{BINDING_VERSION, DISTRO_POLICY_REVISION}; -use crate::host::sandbox::{SandboxDistro, SandboxNetwork, SandboxSettings}; -use crate::host::seal::SEAL_VERSION; -use crate::protocol::cmdline; -use crate::protocol::root::RootImage; -use crate::uuid::Uuid; - -/// OpenVMM's platform allowlist of kernel command-line tokens, compiled from the copies of its -/// source in `tests/vectors/openvmm/`, which an ignored test of `tests/openvmm_vectors.rs` -/// compares with the OpenVMM submodule. -#[allow(unreachable_pub, clippy::missing_docs_in_private_items)] -mod openvmm { - /// OpenVMM's `openvmm_defs` crate, of which the allowlist reads one constant. - mod openvmm_defs { - /// OpenVMM's `openvmm_defs::microvm`. - pub(super) mod microvm { - include!("../../../../tests/vectors/openvmm/microvm_control_tty_command_line.rs"); - } - } - - include!("../../../../tests/vectors/openvmm/platform_command_line_token_is_invariant.rs"); - - /// Returns whether OpenVMM's platform allowlist admits `token`. - pub(super) fn admits(token: &str) -> bool { - platform_command_line_token_is_invariant(token) - } -} - -/// Bytes in a GiB. -const GIB: u64 = 1024 * MIB; - -/// The identity of the placeholder. -const IDENTITY: &str = "11111111-1111-1111-1111-111111111111"; - -/// The placeholder. -const PLACEHOLDER: &str = if cfg!(windows) { - r"C:\platform\placeholder.img" -} else { - "/platform/placeholder.img" -}; - -/// A clone's scratch, which an adapter's plan names as its upper. -const CLONE_SCRATCH: &str = if cfg!(windows) { - r"C:\sandboxes\tenant-a\scratch.ext4" -} else { - "/sandboxes/tenant-a/scratch.ext4" -}; - -/// The capture's transaction directory. -const TRANSACTION: &str = if cfg!(windows) { - r"C:\templates\tx-1" -} else { - "/templates/tx-1" -}; - -/// The dormant scratch device's file in [`TRANSACTION`]. -const SCRATCH: &str = if cfg!(windows) { - r"C:\templates\tx-1\scratch.raw" -} else { - "/templates/tx-1/scratch.raw" -}; - -/// The snapshot's destination in [`TRANSACTION`]. -const DESTINATION: &str = if cfg!(windows) { - r"C:\templates\tx-1\snapshot" -} else { - "/templates/tx-1/snapshot" -}; - -/// The text of [`generation`]. -const GENERATION: &str = "000102030405060708090a0b0c0d0e0f"; - -/// The options of templates' arguments, each of which takes one value. -const OPTIONS: [&str; 14] = [ - "--machine", - "--microvm-sandbox-block", - "--net", - "--network-profile", - "--network-egress", - "--network-ingress", - "--host-loopback", - "--processors", - "--memory", - "--memory-capacity", - "--snapshot-destination", - "--snapshot-tier", - "--snapshot-block-identity", - "--snapshot-generation-id", -]; - -/// Returns the EROFS image of the placeholder. -fn erofs() -> RootImage { - RootImage::erofs(Some(Uuid::parse(IDENTITY).unwrap())) -} - -/// Returns the GPT image of the placeholder. -fn gpt() -> RootImage { - RootImage::gpt(Some(Uuid::parse(IDENTITY).unwrap())) -} - -/// Returns the binding of a placeholder of `format` with `identity`, of `length` bytes. -fn binding_of(format: &str, identity: &str, length: u64) -> DistroBinding { - serde_json::from_value(json!({ - "version": BINDING_VERSION, - "format": format, - "identity": identity, - "image": { - "version": SEAL_VERSION, - "scheme": "linux", - "device": 2049, - "fileId": "0000000000000000000000000004a2f1", - "kind": "regular", - "length": length, - "modified": {"seconds": 1_792_000_000, "nanoseconds": 5}, - "changed": {"seconds": 1_792_000_000, "nanoseconds": 5}, - }, - "manifest": null, - "policy": {"revision": DISTRO_POLICY_REVISION, "result": "accepted"}, - })) - .unwrap() -} - -/// Returns the binding of the EROFS placeholder of 1 GiB. -fn binding() -> DistroBinding { - binding_of("erofs", IDENTITY, GIB) -} - -/// Returns an adapter's plan of the placeholder as `image` over `upper`, with `network`. -fn adapter_with( - image: RootImage, - upper: SandboxUpper, - network: Option, -) -> SandboxPlan { - adapter_at(PLACEHOLDER, image, upper, network) -} - -/// Returns an adapter's plan of the placeholder at `path` as `image` over `upper`, with -/// `network`. -fn adapter_at( - path: &str, - image: RootImage, - upper: SandboxUpper, - network: Option, -) -> SandboxPlan { - SandboxPlan::adapter(SandboxDistro::new(path, image), upper).with_network(network) -} - -/// Returns an adapter's plan of the EROFS placeholder over a clone's scratch, with `network`. -fn adapter(network: Option) -> SandboxPlan { - adapter_with(erofs(), SandboxUpper::scratch(CLONE_SCRATCH), network) -} - -/// Returns the network of a guest at `10.0.0.2/24` under `policy`. -fn network(policy: NetworkPolicy) -> SandboxNetwork { - SandboxNetwork::new("10.0.0.2/24", policy).unwrap() -} - -/// Returns a policy that allows egress, and host-loopback access from the guest alone. -fn outbound() -> NetworkPolicy { - let mut policy = NetworkPolicy::new(Access::Allow); - policy.host_loopback = HostLoopback::Outbound; - policy -} - -/// Returns a policy that allows ingress. -fn listening() -> NetworkPolicy { - let mut policy = NetworkPolicy::new(Access::Deny); - policy.ingress = Access::Allow; - policy -} - -/// Returns a policy that allows egress, and host-loopback access both ways, with a forward of a -/// TCP port of host loopback to the guest. -fn forwarded() -> NetworkPolicy { - let mut policy = NetworkPolicy::new(Access::Allow); - policy.host_loopback = HostLoopback::Allow; - policy.host_loopback_forwards.push(LoopbackForward { - protocol: ForwardProtocol::Tcp, - host_port: 18080, - guest_port: 8080, - }); - policy -} - -/// Returns the policies of networks that attach a NIC: they allow egress, egress to some -/// destinations, egress and host-loopback access from the guest, a proxy alone, ingress, and -/// host-loopback access both ways, with a forward. -fn attaching() -> Vec { - vec![ - NetworkPolicy::new(Access::Allow), - NetworkPolicy::new(Access::Deny) - .with_allow(EgressRule::to("192.0.2.0/24").on_port(Protocol::Tcp, 443)) - .with_deny(EgressRule::to("192.0.2.128/25")), - outbound(), - NetworkPolicy::proxy_only(NetworkProxy::new(3128)), - listening(), - forwarded(), - ] -} - -/// Returns the networks of templates: none; one under a policy that lets nothing through, which -/// attaches no NIC; and those under each policy of [`attaching`]. -fn networks() -> Vec> { - let mut networks = vec![None, Some(network(NetworkPolicy::new(Access::Deny)))]; - networks.extend(attaching().into_iter().map(|policy| Some(network(policy)))); - networks -} - -/// Returns the resources of `processors`, all online at boot, with 512 MiB of RAM and a capacity -/// of 2 GiB. -fn resources(processors: u32) -> TemplateResources { - TemplateResources { - processors, - boot_processors: processors, - memory: 512 * MIB, - memory_capacity: 2 * GIB, - } -} - -/// Returns the storage generation whose bytes are 0 to 15. -fn generation() -> StorageGeneration { - StorageGeneration::from_bytes([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]).unwrap() -} - -/// The inputs of a lowering, which tests change from their defaults: an adapter's plan of the -/// EROFS placeholder over a clone's scratch, without a network, the placeholder's binding, 64 MiB -/// of scratch, the resources of 2 processors, and [`TRANSACTION`]. -#[derive(Clone)] -struct Inputs { - /// The adapter's plan. - adapter: SandboxPlan, - /// The binding of the placeholder. - binding: DistroBinding, - /// The length of the dormant scratch device. - scratch_length: u64, - /// The processors and RAM. - resources: TemplateResources, - /// The transaction directory. - transaction: String, -} - -impl Default for Inputs { - fn default() -> Self { - Self { - adapter: adapter(None), - binding: binding(), - scratch_length: 64 * MIB, - resources: resources(2), - transaction: TRANSACTION.to_owned(), - } - } -} - -impl Inputs { - /// Returns the default inputs with `adapter`. - fn of(adapter: SandboxPlan) -> Self { - Self { - adapter, - ..Self::default() - } - } - - /// Lowers the adapter's plan with the other inputs and [`generation`]. - fn lower(&self) -> Result { - TemplatePlan::new( - &self.adapter, - &self.binding, - self.scratch_length, - self.resources, - Path::new(&self.transaction), - generation(), - ) - } -} - -/// Returns the template of `adapter`, with the other inputs' defaults. -fn template(adapter: SandboxPlan) -> TemplatePlan { - Inputs::of(adapter).lower().unwrap() -} - -/// Returns the templates of every format of placeholder, of adapters' plans over a clone's scratch -/// and over RAM, of every network of [`networks`], and of every processor count, with every count -/// up to it online at boot. -fn every_template() -> Vec { - let mut templates = Vec::new(); - for (format, image) in [("erofs", erofs()), ("gpt", gpt())] { - for upper in [SandboxUpper::scratch(CLONE_SCRATCH), SandboxUpper::RAM] { - for network in networks() { - for processors in PROCESSOR_COUNTS { - for boot_processors in PROCESSOR_COUNTS - .into_iter() - .filter(|boot| *boot <= processors) - { - let inputs = Inputs { - adapter: adapter_with(image, upper.clone(), network.clone()), - binding: binding_of(format, IDENTITY, GIB), - resources: TemplateResources { - boot_processors, - ..resources(processors) - }, - ..Inputs::default() - }; - templates.push(inputs.lower().unwrap()); - } - } - } - } - } - templates -} - -/// Returns `texts` as owned strings. -fn strings(texts: &[&str]) -> Vec { - texts.iter().map(|text| (*text).to_owned()).collect() -} - -/// Returns the value of the one `option` of `arguments`, whose options each take one value. -fn value<'a>(arguments: &'a [String], option: &str) -> &'a str { - let values: Vec<&str> = arguments - .chunks(2) - .filter(|pair| pair[0] == option) - .map(|pair| pair[1].as_str()) - .collect(); - assert_eq!(values.len(), 1, "{option} in {arguments:?}"); - values[0] -} - -/// Returns the message of the error of `result`, which must be -/// [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported). -fn refusal(result: Result) -> String { - let error = result.unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); - error.to_string() -} - -#[test] -fn templates_render_their_devices_resources_and_capture_options() { - let devices = strings(&[ - "--machine", - "microvm", - "--microvm-sandbox-block", - &format!("distro:file:{PLACEHOLDER},ro"), - "--microvm-sandbox-block", - &format!("scratch:file:{SCRATCH}"), - ]); - let capture = strings(&[ - "--processors", - "2", - "--memory", - "512M", - "--memory-capacity", - "2048M", - "--snapshot-destination", - DESTINATION, - "--snapshot-tier", - "platform", - "--snapshot-block-identity", - "generation", - "--snapshot-generation-id", - GENERATION, - ]); - // No NIC, without a network or with one that lets nothing through. - for network in [None, Some(network(NetworkPolicy::new(Access::Deny)))] { - let template = template(adapter(network)); - assert!(template.nic_arguments().is_empty()); - assert_eq!(template.capture_arguments(), capture); - assert_eq!( - template.openvmm_arguments().unwrap(), - [devices.clone(), capture.clone()].concat() - ); - } - // One fixed NIC, at the network's guest address, under the complete deny-all policy, whatever - // the network's own, which each clone's restore replaces. - let nic = strings(&[ - "--net", - "10.0.0.2/24", - "--network-profile", - "portable", - "--network-egress", - "deny", - "--network-ingress", - "deny", - "--host-loopback", - "deny", - ]); - for policy in attaching() { - let template = template(adapter(Some(network(policy)))); - assert_eq!(template.nic_arguments(), nic); - assert_eq!(template.capture_arguments(), capture); - assert_eq!( - template.openvmm_arguments().unwrap(), - [devices.clone(), nic.clone(), capture.clone()].concat() - ); - } - // RAM and its capacity in MiB. - let inputs = Inputs { - resources: TemplateResources { - processors: 8, - boot_processors: 8, - memory: MEMORY_BLOCK_SIZE, - memory_capacity: MEMORY_BLOCK_SIZE, - }, - ..Inputs::default() - }; - assert_eq!( - inputs.lower().unwrap().capture_arguments()[..6], - strings(&[ - "--processors", - "8", - "--memory", - "128M", - "--memory-capacity", - "128M", - ]) - ); -} - -#[test] -fn templates_keep_the_placeholder_and_the_files_of_their_capture_in_its_transaction() { - let network = Some(network(NetworkPolicy::new(Access::Allow))); - let template = template(adapter(network.clone())); - let plan = template.plan(); - assert_eq!(plan.distro().path(), Path::new(PLACEHOLDER)); - assert_eq!(plan.distro().image(), erofs()); - assert_eq!(plan.upper().scratch_path(), Some(Path::new(SCRATCH))); - assert_eq!(plan.settings(), None); - assert_eq!(plan.export(), None); - assert_eq!(plan.network(), None); - assert_eq!(template.binding(), &binding()); - assert_eq!(template.image_capacity(), GIB); - assert_eq!(template.scratch_path(), Path::new(SCRATCH)); - assert_eq!(template.scratch_length(), 64 * MIB); - assert_eq!(template.resources(), resources(2)); - assert_eq!(template.transaction(), Path::new(TRANSACTION)); - assert_eq!(template.destination(), Path::new(DESTINATION)); - assert_eq!(template.generation(), generation()); - assert_eq!(template.kernel_command_line(), "nvx_sandbox=1"); - assert_eq!( - (SCRATCH_NAME, SNAPSHOT_NAME, MEMORY_BLOCK_SIZE), - ("scratch.raw", "snapshot", 128 * MIB) - ); - // The capacity of every clone's image is the placeholder's length, as its binding records it. - let inputs = Inputs { - binding: binding_of("erofs", IDENTITY, 8 * GIB + 512), - scratch_length: 512, - ..Inputs::default() - }; - let template = inputs.lower().unwrap(); - assert_eq!( - (template.image_capacity(), template.scratch_length()), - (8 * GIB + 512, 512) - ); -} - -#[test] -fn templates_limit_their_processors_online_at_boot_on_their_command_line() { - // A template whose processors are all online at boot sets no limit, and OpenVMM records none. - for processors in PROCESSOR_COUNTS { - let template = Inputs { - resources: resources(processors), - ..Inputs::default() - } - .lower() - .unwrap(); - assert_eq!(template.kernel_command_line(), "nvx_sandbox=1"); - } - // One that boots with fewer online limits them, and keeps its capacity. - let inputs = Inputs { - resources: TemplateResources { - boot_processors: 2, - ..resources(8) - }, - ..Inputs::default() - }; - let template = inputs.lower().unwrap(); - assert_eq!(template.kernel_command_line(), "nvx_sandbox=1 maxcpus=2"); - assert_eq!(value(&template.capture_arguments(), "--processors"), "8"); - assert_eq!(template.resources(), inputs.resources); -} - -#[test] -fn the_upper_of_an_adapters_plan_is_a_clones_and_no_part_of_its_template() { - let over_scratch = template(adapter(None)); - let over_ram = template(adapter_with(erofs(), SandboxUpper::RAM, None)); - assert_eq!(over_ram, over_scratch); - assert_eq!( - over_ram.openvmm_arguments().unwrap(), - over_scratch.openvmm_arguments().unwrap() - ); - assert_eq!(over_ram.scratch_path(), Path::new(SCRATCH)); -} - -#[test] -fn transaction_directories_are_spelled_with_their_components() { - let given = if cfg!(windows) { - r"C:/templates//tx-1/." - } else { - "/templates//tx-1/./" - }; - let inputs = Inputs { - transaction: given.to_owned(), - ..Inputs::default() - }; - let template = inputs.lower().unwrap(); - assert_eq!(template.transaction().to_str(), Some(TRANSACTION)); - assert_eq!(template.scratch_path().to_str(), Some(SCRATCH)); - assert_eq!(template.destination().to_str(), Some(DESTINATION)); - assert_eq!(template, Inputs::default().lower().unwrap()); -} - -#[test] -fn openvmms_platform_allowlist_admits_its_tokens_and_refuses_tenant_data() { - for token in [ - "earlycon=xe9", - "console=hvc0", - "console=hvc1", - "reboot=t", - "panic=-1", - "nvx_sandbox=1", - "nvx_config=0xd0010000,65536", - "nvx_snapshot_tier=platform", - "nr_cpus=8", - "maxcpus=1", - "nvx_control_tty=hvc2", - "virtio_mmio.device=0x1000@0xd0001000:17", - "virtnet_ip=10.0.0.2", - "virtnet_mask=255.255.255.0", - "virtnet_gw=10.0.0.1", - "virtnet_dns=10.0.0.1", - "virtnet_ip6=fd00::2/64", - "virtnet_gw6=fd00::1", - ] { - assert!(openvmm::admits(token), "{token}"); - } - // The tokens of a root, a workload, a share, a listener, and anything else that a tenant's - // sandbox or another tier sets. - let [layer, upper]: [String; 2] = adapter(None).layout().tokens().try_into().unwrap(); - let [_, ram]: [String; 2] = adapter_with(gpt(), SandboxUpper::RAM, None) - .layout() - .tokens() - .try_into() - .unwrap(); - for token in [ - layer.as_str(), - upper.as_str(), - ram.as_str(), - "nvx_overlay_mode=untrusted-lowers", - "nvx_scratch_journal=forbidden", - "nvx_first_partition_bound=4096", - "nvx_ram_upper_percent=50", - "nvx_hostname=tenant-a", - "hostname=tenant-a", - "nvx_entrypoint=/bin/sh", - "nvx_arg=-c", - "nvx_memory_max=1073741824", - "nvx_maps=1", - "nvx_workload_account=create", - "nvx_workload_uid=1000", - "nvx_workload_gid=1000", - "nvx_lifecycle=managed", - "virtfs_dir=0", - "virtnet_ingress=1", - "nvx_proxy=http://10.0.0.1:3128", - "nvx_config=tenant-data", - "nvx_sandbox=0", - "nvx_snapshot_tier=workload-start", - "console=ttyS0", - "quiet", - "", - ] { - assert!(!openvmm::admits(token), "{token}"); - } -} - -#[test] -fn the_command_line_holds_only_tokens_of_openvmms_platform_allowlist() { - for template in every_template() { - let arguments = template.openvmm_arguments().unwrap(); - let command_line = template.kernel_command_line(); - let mut tokens: Vec = command_line.split(' ').map(str::to_owned).collect(); - // The tokens that OpenVMM writes for the template's arguments. - tokens.push(format!("nr_cpus={}", value(&arguments, "--processors"))); - tokens.push(format!( - "nvx_snapshot_tier={}", - value(&arguments, "--snapshot-tier") - )); - // Under the NIC's complete deny-all policy, OpenVMM writes no DNS server and no - // published listeners. - if arguments.iter().any(|argument| argument == "--net") { - let guest_network = value(&arguments, "--net"); - let (address, _) = guest_network.split_once('/').unwrap(); - tokens.push(format!("virtnet_ip={address}")); - tokens.push(format!( - "virtnet_gw={}", - crate::net::guest_gateway(guest_network).unwrap() - )); - } - for token in &tokens { - assert!(openvmm::admits(token), "{token} of {template:?}"); - // OpenVMM splits the command line at ASCII whitespace, and the kernel at its own, - // which also holds the vertical tab and 0xa0: no token holds either, so both split - // the command line into the same tokens. - assert!( - token.is_ascii() && !token.bytes().any(cmdline::is_space), - "{token:?} of {template:?}" - ); - } - // The command line of the template's devices names the root of its plan, which a clone - // binds, and which the allowlist refuses: the template's replaces it. - let root = template.plan().kernel_command_line("").unwrap(); - for token in root.split(' ') { - assert!(!openvmm::admits(token), "{token} of {root}"); - assert!(!command_line.contains(token), "{token} in {command_line}"); - } - } -} - -#[test] -fn templates_hold_no_tenant_data() { - for template in every_template() { - let arguments = template.openvmm_arguments().unwrap(); - let command_line = template.kernel_command_line(); - // Neither the identity of the image, nor the clone's upper, nor the proxy. - for text in [IDENTITY, "tenant", "3128", "/sandboxes", r"\sandboxes"] { - assert!(!command_line.contains(text), "{text} in {command_line}"); - } - for text in [IDENTITY, "tenant", "3128", "/sandboxes", r"\sandboxes"] { - assert!( - arguments.iter().all(|argument| !argument.contains(text)), - "{text} in {arguments:?}" - ); - } - // No kernel command line, workload identity, lifecycle, share, published listener, or - // other option besides those of the devices, the resources, and the capture, each with one - // value. - for pair in arguments.chunks(2) { - let [option, value] = pair else { - panic!("{arguments:?} ends in an option without a value"); - }; - let option = option.as_str(); - assert!(OPTIONS.contains(&option), "{option} in {arguments:?}"); - assert!(!value.starts_with("--"), "{option} {value}"); - if ["--network-egress", "--network-ingress", "--host-loopback"].contains(&option) { - assert_eq!(value, "deny", "{option} in {arguments:?}"); - } - } - let devices = template.plan().openvmm_arguments().unwrap(); - assert_eq!( - [ - devices, - template.nic_arguments().to_vec(), - template.capture_arguments() - ] - .concat(), - arguments - ); - } -} - -#[test] -fn templates_refuse_what_no_capture_takes() { - // A plan of the sandbox profile. - let profile = SandboxPlan::new( - SandboxDistro::new(PLACEHOLDER, erofs()), - SandboxUpper::scratch(CLONE_SCRATCH), - SandboxSettings::default(), - ) - .unwrap(); - assert_eq!( - refusal(Inputs::of(profile).lower()), - "a template lowers an adapter's plan, not a plan of the sandbox profile, whose workload \ - settings no template captures" - ); - - // A placeholder that is not the image that its binding names. - let other = "22222222-2222-2222-2222-222222222222"; - for (image, binding) in [ - (gpt(), binding()), - (RootImage::erofs(None), binding()), - (erofs(), binding_of("erofs", other, GIB)), - (erofs(), binding_of("gpt", IDENTITY, GIB)), - ] { - let expected = format!( - "the placeholder {PLACEHOLDER} is not the {} layer with identity {} that its distro \ - binding names", - binding.format(), - binding.identity() - ); - let inputs = Inputs { - adapter: adapter_with(image, SandboxUpper::RAM, None), - binding, - ..Inputs::default() - }; - assert_eq!(refusal(inputs.lower()), expected, "{image:?}"); - } - let inputs = Inputs { - binding: binding_of("erofs", "11111111-1111-1111-1111-11111111111A", GIB), - ..Inputs::default() - }; - assert_eq!( - refusal(inputs.lower()), - "the distro binding names the identity \"11111111-1111-1111-1111-11111111111A\", which \ - is not a UUID in its canonical text" - ); - - // Placeholders that OpenVMM would resolve against its own directory. - let mut relative = vec!["platform/placeholder.img"]; - if cfg!(windows) { - // Rooted on the current drive, and relative to a drive's current directory. - relative.extend([r"\platform\placeholder.img", r"C:platform\placeholder.img"]); - } - for path in relative { - let inputs = Inputs::of(adapter_at(path, erofs(), SandboxUpper::RAM, None)); - assert_eq!( - refusal(inputs.lower()), - format!( - "the placeholder of a template's capture must be absolute, since OpenVMM would \ - resolve it against the directory in which it runs: {}", - paths::normalized(Path::new(path)).display() - ) - ); - } - // Placeholders that OpenVMM's block device options would split, and that it would not attach - // as raw images. - for (name, problem) in [ - ( - "placeholder,1.img", - "containing commas, semicolons, or unbalanced brackets are unsupported", - ), - ( - "placeholder;1.img", - "containing commas, semicolons, or unbalanced brackets are unsupported", - ), - ( - "placeholder[1.img", - "containing commas, semicolons, or unbalanced brackets are unsupported", - ), - ( - "placeholder]1.img", - "containing commas, semicolons, or unbalanced brackets are unsupported", - ), - ( - "placeholder][1.img", - "containing commas, semicolons, or unbalanced brackets are unsupported", - ), - ( - "placeholder.vhd", - "ending in .vhd are unsupported, since OpenVMM would not attach them as raw images", - ), - ( - "placeholder.vhdx", - "ending in .vhdx are unsupported, since OpenVMM would not attach them as raw images", - ), - ( - "placeholder.vmgs", - "ending in .vmgs are unsupported, since OpenVMM would not attach them as raw images", - ), - ( - "placeholder.iso", - "ending in .iso are unsupported, since OpenVMM would not attach them as raw images", - ), - ] { - let path = Path::new(PLACEHOLDER).with_file_name(name); - let path = path.to_str().unwrap(); - let inputs = Inputs::of(adapter_at(path, erofs(), SandboxUpper::RAM, None)); - assert_eq!( - refusal(inputs.lower()), - format!("disk paths {problem}: {path}") - ); - } - - // Block devices that OpenVMM does not attach. - for length in [0, 511, 1000, GIB + 1] { - let placeholder = Inputs { - binding: binding_of("erofs", IDENTITY, length), - ..Inputs::default() - }; - assert_eq!( - refusal(placeholder.lower()), - format!( - "the placeholder {PLACEHOLDER} of {length} bytes cannot be attached: OpenVMM \ - attaches block devices of nonzero multiples of 512 bytes" - ) - ); - let scratch = Inputs { - scratch_length: length, - ..Inputs::default() - }; - assert_eq!( - refusal(scratch.lower()), - format!( - "a template's scratch of {length} bytes cannot be attached: OpenVMM attaches \ - block devices of nonzero multiples of 512 bytes" - ) - ); - } - - // Processor counts other than 1, 2, 4, and 8. - for processors in [0, 3, 6, 16, u32::MAX] { - let inputs = Inputs { - resources: resources(processors), - ..Inputs::default() - }; - assert_eq!( - refusal(inputs.lower()), - format!("a template's VM has 1, 2, 4, or 8 processors, not {processors}") - ); - } - // Processors online at boot other than 1, 2, 4, and 8, or more than the processors. - for (processors, boot_processors) in [(2, 0), (8, 3), (2, 4), (8, 16), (8, u32::MAX)] { - let inputs = Inputs { - resources: TemplateResources { - boot_processors, - ..resources(processors) - }, - ..Inputs::default() - }; - assert_eq!( - refusal(inputs.lower()), - format!( - "a template's VM boots with 1, 2, 4, or 8 of its {processors} processors online, \ - not {boot_processors}" - ) - ); - } - // RAM and capacities that are not whole blocks, and capacities below the RAM. - for (memory, memory_capacity) in [ - (0, 0), - (0, 2 * GIB), - (100 * MIB, 2 * GIB), - (512 * MIB, 2 * GIB + MIB), - (512 * MIB, 384 * MIB), - (512 * MIB, 0), - ] { - let inputs = Inputs { - resources: TemplateResources { - processors: 2, - boot_processors: 2, - memory, - memory_capacity, - }, - ..Inputs::default() - }; - assert_eq!( - refusal(inputs.lower()), - format!( - "a template's VM needs nonzero RAM, and a RAM capacity of at least it, each a \ - multiple of 128 MiB, not {memory} bytes of RAM and a capacity of \ - {memory_capacity} bytes" - ) - ); - } - - // Transaction directories that OpenVMM would resolve against its own directory. - let mut relative = vec!["templates/tx-1", "."]; - if cfg!(windows) { - // Rooted on the current drive, and relative to a drive's current directory. - relative.extend([r"\templates\tx-1", r"C:templates\tx-1"]); - } - for transaction in relative { - let inputs = Inputs { - transaction: transaction.to_owned(), - ..Inputs::default() - }; - assert_eq!( - refusal(inputs.lower()), - format!( - "the transaction directory of a template's capture must be absolute, since \ - OpenVMM would resolve it against the directory in which it runs: {}", - paths::normalized(Path::new(transaction)).display() - ) - ); - } - // Transaction directories whose scratch device OpenVMM's options would split. - for name in ["a,b", "a;b", "a[b", "a]b", "a][b"] { - let inputs = Inputs { - transaction: format!("{TRANSACTION}-{name}"), - ..Inputs::default() - }; - assert_eq!( - refusal(inputs.lower()), - format!( - "disk paths containing commas, semicolons, or unbalanced brackets are unsupported: \ - {}", - Path::new(&inputs.transaction).join(SCRATCH_NAME).display() - ) - ); - } -} - -/// Lowers the default inputs with the transaction directory `transaction`. -fn lower_in(transaction: &Path) -> Result { - let inputs = Inputs::default(); - TemplatePlan::new( - &inputs.adapter, - &inputs.binding, - inputs.scratch_length, - inputs.resources, - transaction, - generation(), - ) -} - -#[cfg(unix)] -#[test] -fn transaction_directories_must_be_valid_unicode() { - use std::ffi::OsStr; - use std::os::unix::ffi::OsStrExt; - - let transaction = Path::new(OsStr::from_bytes(b"/templates/tx-\xff")); - assert_eq!( - refusal(lower_in(transaction)), - "the transaction directory of a template's capture is not valid Unicode: \ - /templates/tx-\u{fffd}" - ); -} - -#[cfg(windows)] -#[test] -fn transaction_directories_must_be_valid_unicode() { - use std::ffi::OsString; - use std::os::windows::ffi::OsStringExt; - use std::path::PathBuf; - - let mut units: Vec = r"C:\templates\tx-".encode_utf16().collect(); - units.push(0xd800); - let transaction = PathBuf::from(OsString::from_wide(&units)); - assert_eq!( - refusal(lower_in(&transaction)), - "the transaction directory of a template's capture is not valid Unicode: \ - C:\\templates\\tx-\u{fffd}" - ); -} - -/// A change to the inputs of a lowering. -type Fix = fn(&mut Inputs); - -#[test] -fn templates_check_their_inputs_in_order() { - // With every input refused, each fix in turn reveals the next refusal. - let mut inputs = Inputs { - adapter: SandboxPlan::new( - SandboxDistro::new(PLACEHOLDER, gpt()), - SandboxUpper::RAM, - SandboxSettings::default(), - ) - .unwrap() - .with_network(Some(network(listening()))), - binding: binding_of("erofs", IDENTITY, 1000), - scratch_length: 0, - resources: TemplateResources { - processors: 3, - boot_processors: 3, - memory: 0, - memory_capacity: 0, - }, - transaction: "templates".to_owned(), - }; - let steps: [(&str, Fix); 11] = [ - ("a template lowers an adapter's plan", |inputs| { - inputs.adapter = adapter_at( - "placeholder.vhd", - gpt(), - SandboxUpper::RAM, - Some(network(listening())), - ); - }), - ("is not the erofs layer", |inputs| { - inputs.adapter = adapter_at( - "placeholder.vhd", - erofs(), - SandboxUpper::RAM, - Some(network(listening())), - ); - }), - ( - "the placeholder of a template's capture must be absolute", - |inputs| { - inputs.adapter = adapter_at( - &PLACEHOLDER.replace(".img", ".vhd"), - erofs(), - SandboxUpper::RAM, - Some(network(listening())), - ); - }, - ), - ("disk paths ending in .vhd", |inputs| { - inputs.adapter = adapter(Some(network(listening()))); - }), - ("of 1000 bytes cannot be attached", |inputs| { - inputs.binding = binding(); - }), - ("a template's scratch of 0 bytes", |inputs| { - inputs.scratch_length = 512; - }), - ("processors, not 3", |inputs| { - inputs.resources.processors = 2 - }), - ("of its 2 processors online, not 3", |inputs| { - inputs.resources.boot_processors = 1 - }), - ("needs nonzero RAM", |inputs| { - inputs.resources = resources(2) - }), - ("must be absolute", |inputs| { - inputs.transaction = format!("{TRANSACTION},1"); - }), - ("commas, semicolons, or unbalanced brackets", |inputs| { - inputs.transaction = TRANSACTION.to_owned() - }), - ]; - for (refused, fix) in steps { - let message = refusal(inputs.lower()); - assert!(message.contains(refused), "{refused}: {message}"); - fix(&mut inputs); - } - assert_eq!(inputs.lower().unwrap().scratch_length(), 512); -} diff --git a/nvx-rs/src/host/template/store.rs b/nvx-rs/src/host/template/store.rs deleted file mode 100644 index 46e7a1f98..000000000 --- a/nvx-rs/src/host/template/store.rs +++ /dev/null @@ -1,1116 +0,0 @@ -//! The store of templates, [`Store`]: its directory, its records, the namespaces of its -//! generations, the captures that publish them, and their leases, tombstones, and retirement. - -mod capture; -mod leases; -#[cfg(test)] -pub(super) mod tests; - -use std::fmt; -use std::fs::{self, File, TryLockError}; -use std::io; -use std::num::NonZeroU64; -use std::path::{Path, PathBuf}; -use std::str::FromStr; -use std::thread; -use std::time::{Duration, Instant, SystemTime}; - -use serde::{Deserialize, Serialize}; - -use super::key::{ClassIdentity, KeyIdentity, TemplateKey}; -use super::manifest::{HostClock, TimeContract}; -use crate::activation::StoreGeneration; -use crate::error::{Error, ErrorCode, ErrorKind, Result}; -use crate::host::platform::{self, parse_hex_id}; -use crate::host::process::BootId; -use crate::host::store::files::{open_lock, write_private}; -use crate::host::store::lease::{ - ID_LEN, StoreRecord, hex, read_identity, read_private, read_record, store_error, -}; - -pub use self::capture::{CAPTURE_RECORD_NAME, CAPTURES_NAME, Capture, Created, TRANSACTION_NAME}; -pub use self::leases::{Acquired, LEASES_NAME, Retirement, TOMBSTONES_NAME}; -pub use crate::host::store::lease::{ - GENERATIONS_NAME, Lease, LeaseId, STORE_FORMAT, STORE_RECORD_NAME, StoreId, -}; - -/// Name of the store's index. -pub const INDEX_NAME: &str = "index.json"; -/// Name of the lock of the store's index. -pub const INDEX_LOCK_NAME: &str = "index.lock"; -/// Name of the canonical encoding of a generation's key in its payload. -pub const KEY_NAME: &str = "key.bin"; -/// Name of a generation's snapshot in its payload. -pub const SNAPSHOT_NAME: &str = "snapshot"; -/// Most generations that a store holds. -pub const MAX_GENERATIONS: usize = 1024; - -/// The store generation that a new store assigns first. -const FIRST_GENERATION: StoreGeneration = StoreGeneration::new(NonZeroU64::MIN); - -/// Longest index, in bytes: more than an index of [`MAX_GENERATIONS`] generations takes. -const MAX_INDEX_LEN: u64 = 4 * 1024 * 1024; - -/// Interval at which a lock that another holder has is tried again, by a caller that waits for it -/// until a deadline. -const LOCK_RETRY_INTERVAL: Duration = Duration::from_millis(25); - -/// Draws 16 random bytes, for `what`. -fn random_id(what: &str) -> Result<[u8; ID_LEN]> { - let mut bytes = [0; ID_LEN]; - getrandom::fill(&mut bytes) - .map_err(|error| Error::other(format!("cannot draw {what}")).with_source(error))?; - Ok(bytes) -} - -/// The bound-block storage generation of a capture: 16 bytes, not all zero, that a capture gives -/// OpenVMM for every block that it binds, which the storage owner keeps immutable for those -/// blocks, so that neither the capture nor a restore reads a block to identify it. -/// -/// It belongs to the storage owner, and is neither a [store generation](StoreGeneration), nor a -/// time lineage counter, nor a VM generation ID; see -/// [the four namespaces](Store#four-namespaces). Generations may share one. Its text form, which -/// `Display` writes and `FromStr` parses, is 32 lowercase hexadecimal digits, as OpenVMM's -/// `--snapshot-generation-id` takes it; parsing also accepts uppercase ones. -#[derive(Clone, Copy, PartialEq, Eq, Hash)] -pub struct StorageGeneration([u8; ID_LEN]); - -impl StorageGeneration { - /// Draws a fresh storage generation at random. - pub fn generate() -> Result { - loop { - if let Some(generation) = Self::from_bytes(random_id("a storage generation")?) { - return Ok(generation); - } - } - } - - /// Returns the storage generation whose bytes are `bytes`, unless they are all zero, which no - /// storage generation is. - pub fn from_bytes(bytes: [u8; ID_LEN]) -> Option { - (bytes != [0; ID_LEN]).then_some(Self(bytes)) - } - - /// Returns the bytes of the storage generation. - pub const fn as_bytes(&self) -> &[u8; ID_LEN] { - &self.0 - } -} - -impl fmt::Display for StorageGeneration { - fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { - formatter.write_str(&hex(&self.0)) - } -} - -impl fmt::Debug for StorageGeneration { - fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { - write!(formatter, "StorageGeneration({self})") - } -} - -impl FromStr for StorageGeneration { - type Err = Error; - - /// Parses the text form of a storage generation, failing with [`ErrorKind::Unsupported`] for - /// anything else, all zeros included. - fn from_str(text: &str) -> Result { - parse_hex_id(text) - .and_then(Self::from_bytes) - .ok_or_else(|| { - Error::unsupported(format!( - "{text:?} is not a storage generation, which is 32 hexadecimal digits, not \ - all zero" - )) - }) - } -} - -/// One generation of a template that a store holds, as its index lists it: its store generation, -/// the identities of its key and of its capture class, its storage generation, and what its -/// manifest records of the capture itself, in which the generations of one key differ. -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct Generation { - /// Its store generation. - generation: StoreGeneration, - /// The identity of its key. - key: KeyIdentity, - /// The identity of its capture class. - class: ClassIdentity, - /// The storage generation of its capture. - storage: StorageGeneration, - /// When OpenVMM created its snapshot. - created_at: SystemTime, - /// The time contract of its capture. - time: TimeContract, - /// The length of its saved state, in bytes. - state_len: u64, - /// The length of its RAM, in bytes. - memory_len: u64, -} - -impl Generation { - /// Returns the store generation, which names the generation in its store. - pub fn generation(&self) -> StoreGeneration { - self.generation - } - - /// Returns the identity of the generation's compatibility key, which - /// [`Store::read_key`] reads in full. - pub fn key(&self) -> KeyIdentity { - self.key - } - - /// Returns the identity of the class of the capture that made the generation. - pub fn class(&self) -> ClassIdentity { - self.class - } - - /// Returns the storage generation of the capture that made the generation. - pub fn storage_generation(&self) -> StorageGeneration { - self.storage - } - - /// Returns when OpenVMM created the generation's snapshot. - pub fn created_at(&self) -> SystemTime { - self.created_at - } - - /// Returns the time contract of the capture, with its anchor, host, boot, and time lineage - /// counter. - pub fn time(&self) -> &TimeContract { - &self.time - } - - /// Returns the length of the snapshot's saved state, in bytes. - pub fn state_len(&self) -> u64 { - self.state_len - } - - /// Returns the length of the snapshot's RAM, in bytes. - pub fn memory_len(&self) -> u64 { - self.memory_len - } -} - -/// The index of a store, as one read found it: its generations, in the order of their -/// publication, and the store generation that its next publication takes. -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct Index { - /// The store generation that the next publication takes. - next: StoreGeneration, - /// The generations, in the order of their store generations. - generations: Vec, -} - -impl Index { - /// Returns the generations, in the order of their publication. - pub fn generations(&self) -> &[Generation] { - &self.generations - } - - /// Returns the generation of store generation `generation`, if the index lists it. - pub fn get(&self, generation: StoreGeneration) -> Option<&Generation> { - self.generations - .binary_search_by_key(&generation, Generation::generation) - .ok() - .map(|index| &self.generations[index]) - } - - /// Returns the generations whose key has the identity `key`, in the order of their - /// publication. - pub fn of_key(&self, key: KeyIdentity) -> impl Iterator { - self.generations - .iter() - .filter(move |generation| generation.key == key) - } - - /// Returns the store generation that the store's next publication takes: one past every store - /// generation that it ever assigned. - pub fn next_generation(&self) -> StoreGeneration { - self.next - } -} - -/// A store of templates: a directory that the adapter chooses, apart from every sandbox's -/// store, with the generations of templates that its host published, and their index. -/// -/// ```text -/// / the store, private to the current user -/// store.json the store's identity record, which its creation writes last -/// index.json the index of the generations that the store holds -/// index.lock the lock that serializes the transactions of the index -/// generations/ the payloads of the generations, which never change once indexed -/// / the payload of the store generation G -/// capture.json the record of the capture that made it -/// key.bin the canonical encoding of its compatibility key -/// snapshot/ its snapshot, as OpenVMM published it -/// .sock on Linux, the control socket of the VMM of a clone that leases it, -/// named after the clone's launch, until the lease's release -/// leases/ the records of the leases that the store granted -/// .json the record of the lease L -/// tombstones/ the tombstones of the generations that take no new leases -/// .json the tombstone of the store generation G -/// captures/ the slots of the capture classes -/// / the slot of the classes whose identities start with the digits C, -/// a sandbox store whose lock serializes the classes' captures -/// openvmm.log the log of the VMM of the slot's latest capture -/// outcome.json the outcome report of the VMM of the slot's latest capture -/// transaction/ the capture in progress: the sandbox store of its VM until the VM has -/// ended, and then the payload of a generation -/// capture.json the record of the capture -/// scratch.raw the dormant scratch device, which the capture removes once checked -/// snapshot/ the snapshot, once OpenVMM has published it -/// key.bin the template's key, once the capture has checked the snapshot -/// ``` -/// -/// # Identity -/// -/// A store has an identity of its own, a [`StoreId`] that its creation draws, which its identity -/// record and its index both hold, so that the index of another store is refused. -/// [`Store::create`] creates a store, or opens the one that its directory already holds, and -/// [`Store::open`] opens an existing one. -/// -/// # The index -/// -/// The index lists the store's generations, in the order of their publication, with the metadata -/// of each that restores and their eligibility check without reading its payload: the identities -/// of its key and of its capture class, its storage generation, and what its manifest records of -/// the capture itself, its creation time, its time contract, and the lengths of its artifacts. -/// The payload of a generation never changes once the index lists it: the record of its capture, -/// the canonical encoding of its key, which [`Store::read_key`] reads, and its snapshot, which its -/// clones restore in place. Only the control sockets of the VMMs of the clones that lease it come -/// and go beside them, on Linux, as [`Store::acquire`] describes. The index -/// changes as generations are published and retired, and the records of leases and tombstones as -/// clones lease generations, and as generations are quarantined and retired. -/// -/// The store's records, its identity record, its index, and the records of its leases and -/// tombstones, are JSON in format [`STORE_FORMAT`]. -/// A record in any other format fails with a message that names its format, and is neither read -/// nor migrated: a store of another format is replaced by a new one, whose templates are captured -/// again. Records are replaced atomically: written to a temporary file beside them, flushed to -/// disk, and renamed over them. -/// -/// # Four namespaces -/// -/// Four numbers name generations of different things, and the store keeps them apart: -/// -/// - The store generation, [`StoreGeneration`], is the host's publication ID: the store assigns -/// one to each generation as it indexes it, from 1 on, in the order of publication, and never -/// assigns one twice, even once its generation is gone. A clone's lease and durable records name -/// the generation that it restores by its store generation and the identity of its key, in their -/// [`TemplateReference`](crate::activation::TemplateReference). -/// - The storage generation, [`StorageGeneration`], is the bound-block storage generation that a -/// capture gives OpenVMM, which the storage owner keeps immutable for the blocks that it binds. -/// It belongs to the storage owner, not to the store, and generations may share one. -/// - The time lineage counter, [`TimeContract::lineage`], is the generation counter of the -/// captured VM process, which each restore of a snapshot increments, so that the clones of one -/// generation share it. -/// - The VM generation ID is 16 random bytes that each VM process draws, which tell the clones of -/// one generation apart. Their guests read it, and the store records none. -/// -/// # Captures and publication -/// -/// [`Store::create_template`] and [`Store::refresh_template`] capture templates of a -/// [`CaptureClass`](crate::host::template::CaptureClass), and publish them as the store's -/// generations: a creation serves the class with a generation that the store holds, if one is -/// eligible, and captures one otherwise, while a refresh always captures. Each class has a slot in -/// `captures/`, a sandbox store named by the first 16 hexadecimal digits of the class's identity, -/// whose lock serializes the captures of the class, and which keeps the log and the outcome report -/// of the VMM of the class's latest capture. A capture holds its slot's lock until its VM has -/// ended, and takes the store's lock under it to publish, as the supervisor's -/// [lock order](crate::host::supervisor#the-lock-order) allows. -/// -/// A capture runs in a transaction, a directory private to the current user, in its slot, on the -/// store's volume. It writes the record of the capture there, `capture.json`, which names the -/// store, the class, and the [`StorageGeneration`] that it drew, and creates the template's -/// dormant scratch device, a sparse file, then lowers the adapter's plan to a -/// [`TemplatePlan`](crate::host::template::TemplatePlan), whose snapshot destination is in the -/// transaction. The transaction is also the sandbox store of the capture's VM, whose launch -/// journal records its VMM: OpenVMM requires the control socket of a VM that captures a snapshot, -/// on Linux, beside the snapshot's destination, and a sandbox store's VM listens in the store's -/// directory. The capture holds the transaction's lock, under its slot's, until the VM has ended, -/// and releases it before it publishes. It cold-boots the template's VM through -/// [`supervisor::launch`](crate::host::supervisor::launch), from the transaction, as a launch of -/// its own, and waits for the VMM to exit: the guest captures itself, and OpenVMM publishes the -/// snapshot at its destination, and exits. The capture then moves the VMM's log and outcome -/// report to its slot. -/// -/// ## Block devices by path -/// -/// OpenVMM opens the block devices of a VM whose snapshot it captures, or restores, itself, at the -/// paths that its command line names: without following a link, nor on Windows any other reparse -/// point, and sharing only reads and deletion, so that its open fails while another handle may -/// write the file. The host therefore hands OpenVMM the paths of these files, never handles, and -/// checks the file at each path just before the VM starts: -/// -/// - The directory that holds the VM's files, the capture's transaction, must be private to the -/// current user. -/// - The placeholder must pass its binding's check, -/// [`DistroBinding::check`](crate::host::sandbox::distro::DistroBinding::check), of its -/// identity and length, and then, at its path, be a regular file, not a link nor any other -/// reparse point, that is the file that the check found, that belongs to the current user or to -/// an administrator, and that no one else may change. -/// - The scratch must be, at its path, the regular file of the template's length that the capture -/// created, which no one else may change. -/// -/// The host tells these files apart by the identity that their seals record, -/// [`FileSeal`](crate::host::seal::FileSeal): their device or volume and their file ID, on Windows -/// the 128-bit ID of `FILE_ID_INFO`, which a file system such as ReFS keeps unique where the -/// 64-bit file index is not. -/// -/// As the VM starts, the host holds no handle through which it could write either file: it closes -/// the scratch once it has checked it, and holds only the placeholder that its binding's check -/// opened for reading, which on Windows lets no one write it, until the VM has ended. The -/// directories above the files must be ones that only the current user, or an administrator, may -/// change, since OpenVMM resolves each path again. A restore of a clone, whose block devices -/// OpenVMM also opens by their paths, takes the same rule; a restore of a clone in RAM declares its -/// scratch absent, which OpenVMM attaches with no file. -/// -/// ## What the VM leaves -/// -/// Once the VMM has exited, and before anything is published, the capture checks that the VM's -/// outcome report records the capture's success, and that the snapshot is a directory private to -/// the current user, as OpenVMM creates it, whose manifest -/// [`TemplateManifest::read`](crate::host::template::TemplateManifest::read) reads, whose artifacts -/// are exactly its manifest, RAM, and saved state, each a regular file private to the current -/// user, of the length that the manifest records, and whose key, [`TemplateKey::new`], is its -/// class's. It then closes the lock of the transaction's sandbox store, removes the lock's file, -/// the scratch, and what OpenVMM left beside the snapshot, and writes the key, `key.bin`, which -/// marks the transaction as checked, durably. -/// -/// ## Two commits -/// -/// A capture publishes its checked transaction under the store's lock, in two commits: it renames -/// the transaction to the payload of the index's next store generation, which no rename -/// replaces, and makes the rename durable; then it writes the index, which lists the generation, -/// and makes it durable, with the store's own entry in its parent, which the store's creation -/// does not sync. A payload at the index's next store generation is one whose publication a crash -/// interrupted, which the next publication, or recovery, indexes first, once it has made the -/// rename durable as the publication would have. Each capture completes these publications, and -/// first drops from the index the generations whose payloads a crash lost, which then take none -/// of the store's capacity, before it looks for a generation or counts the store's. A creation -/// that serves a generation that the store holds makes the index, and the store's own entry, -/// durable before it returns it, since the commit that listed it may have been interrupted. -/// Publications leave the older generations, and their leases, untouched. -/// -/// A capture that fails before it checks its transaction publishes nothing, and retires the -/// transaction once its VMM has ended. One that a crash of the host interrupts is ended by the -/// next capture of its class, or by [`Store::recover_captures`], which an adapter calls as it -/// starts: each ends the launch that the transaction's sandbox store records, as -/// [`LockedStore::recover`](crate::host::store::LockedStore::recover) ends a sandbox's, and then -/// publishes the transaction if the capture had checked it, once it has synced the transaction -/// again, and retires it otherwise, once it has moved the VMM's log and outcome report to the -/// slot. -/// -/// # Leases, tombstones, and retirement -/// -/// A clone that restores a generation leases it first, so that the generation stays in the store -/// until the clone's VM has exited. [`Store::select`] selects the generation of a key from which -/// a clone restores: the latest that is neither quarantined nor ineligible at the host's time, -/// as [`Generation::eligibility`] decides it. [`Store::acquire`] leases a generation in one -/// transaction, under the store's lock and then the sandbox's: it checks the generation's -/// eligibility, and writes the record of the lease in the store and the clone binding, -/// [`CLONE_BINDING_NAME`](crate::host::store::CLONE_BINDING_NAME), beside the sandbox's launch -/// journal, before any VMM of the clone is spawned. It returns the lease with the sandbox's lock, -/// under which the caller launches the clone, and holds no lock of the store for the VM's -/// lifetime. The lease reserves one launch of the clone, [`Lease::launch`], which only that lock -/// begins, with -/// [`LockedStore::begin_launch`](crate::host::store::LockedStore::begin_launch), and the sandbox -/// launches nothing else while the lease lasts; [the supervisor](crate::host::supervisor#clones) -/// describes the launch of a clone, its crashes, and the order of the locks that it takes. A lease -/// never expires: it lasts until [`Store::release`] ends it, which it does only -/// once the sandbox records no VM and no launch, and so once its VM has exited, as its store -/// verifies through the VM's recorded process, also after a reattach. A caller that reads a -/// generation's payload, as a restore does, holds a lease of it. -/// -/// On Linux, a payload holds the control socket of the VMM of each clone that leases it, -/// `.sock`, named after the launch that the lease reserves, beside the snapshot that the -/// clone restores, and of no clone whose lease has ended: the release makes the socket's absence -/// durable before it ends the lease. So a payload holds entries other than its capture's while -/// its generation has leases, which nothing that reads an indexed payload may refuse, and a -/// retirement deletes the payload with any socket that a crash, or a removal of a sandbox that -/// never released its lease, left in it. -/// -/// A tombstone, which [`Store::tombstone`] writes, quarantines a generation: it takes no new -/// lease, and selection skips it, while the leases that it granted keep working. -/// [`Store::retire`] tombstones a generation, and deletes it once its leases have drained: its -/// payload, then its entry in the index, and then its tombstone, so that an interrupted -/// retirement leaves a quarantined generation, which retiring it again deletes. A release, or a -/// removal of a sandbox, that is interrupted leaves the record of a lease that no clone binding -/// names, which a retirement removes. -/// -/// These transactions make each step durable before the next, so that their order holds through -/// a crash of the host too, even when a sandbox and the store are on different filesystems: an -/// acquisition makes the record of a lease durable before it writes the clone binding, and the -/// binding before it returns, and a release, or a retirement that finds a binding gone, makes the -/// binding's absence durable before it removes the record, as a -/// [removal](crate::host::store#removal) of a sandbox does before the sandbox goes, so that no -/// binding outlasts the record that keeps its generation; a retirement makes the tombstone durable -/// before it deletes anything, and each deletion before the next. On Windows, which has no -/// portable way to flush a directory, a rename or a removal is as durable as the filesystem makes -/// it, as OpenVMM's publication of a snapshot assumes. -/// -/// # Privacy -/// -/// The store's directories are private to the current user, the orchestrator's account, so that -/// no one else may write its records and payloads: each belongs to the user, and no one else may -/// access it. On Unix, a directory grants the group and others no permission, and those that the -/// store creates have mode 0700. On Windows, a directory that the store creates allows the user -/// alone, through a protected access control list that its files inherit; one that it did not -/// create may also allow SYSTEM and the Administrators group, as one under the user's profile -/// does, and must pass entries on to the files created in it. A snapshot's directory, which -/// OpenVMM creates, need not: OpenVMM gives it, and each file that it publishes in it, a protected -/// list of their own, which allows SYSTEM and the owner, and the directory's list already stops -/// anyone else from creating a file in it. The store refuses a record whose -/// own list allows anyone else. A directory or a file of the store that is a link, or on Windows -/// any other reparse point, is refused, and so is a record that is not a regular file. The store -/// names its files by their paths, so the directories above it must be ones that only the user, -/// or an administrator, may change. -/// -/// # Example -/// -/// ```no_run -/// use nvx::host::template::Store; -/// -/// # fn example() -> nvx::Result<()> { -/// let store = Store::create("/var/lib/example/templates")?; -/// for generation in store.index()?.generations() { -/// let key = store.read_key(generation)?; -/// println!( -/// "generation {} of key {} on {}", -/// generation.generation(), -/// generation.key(), -/// key.hypervisor() -/// ); -/// } -/// # Ok(()) -/// # } -/// ``` -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct Store { - /// The store's directory, as an absolute path. - dir: PathBuf, - /// The store's identity. - id: StoreId, -} - -impl Store { - /// Creates the directory `dir` if needed, whose parent must exist, private to the current - /// user, and a store in it, or opens the store that it holds. - /// - /// A directory that holds no store must hold nothing but what an interrupted creation leaves: - /// one with any other entry fails with [`ErrorCode::ForeignFiles`]. The creation writes the - /// store's identity record last, so a store whose creation is interrupted is none, and a - /// later creation replaces it. Creations of one store take its lock, so that they make one - /// store between them. An existing directory must already be private, as [`open`](Self::open) - /// requires: one that others could access is refused rather than restricted, since they could - /// hold its files open. - /// - /// A relative `dir` is resolved against the current directory. Fails with - /// [`ErrorKind::NotFound`] if the parent does not exist. - pub fn create(dir: impl Into) -> Result { - Self::create_racing(absolute(dir.into())?, || {}) - } - - /// Creates the store as [`create`](Self::create) does, in the absolute directory `dir`, and - /// runs `raced` once it has found no store there, before it checks the directory's entries, - /// where a concurrent creation may make one. - fn create_racing(dir: PathBuf, raced: impl FnOnce()) -> Result { - let problem = platform::create_owner_only_dir(&dir).map_err(store_error(format!( - "cannot create the template store {}", - dir.display() - )))?; - refuse_problem(&dir, "the template store", problem)?; - // A directory of anything else is refused before its lock file is created, unless a - // concurrent creation made a store in it meanwhile, and again under the lock. - if read_identity(&dir)?.is_none() { - raced(); - if let Err(error) = refuse_foreign_entries(&dir) - && read_identity(&dir)?.is_none() - { - return Err(error); - } - } - let lock = take_lock(&dir, None)?; - if let Some(id) = read_identity(&dir)? { - let store = Self { dir, id }; - store.check_generations_dir()?; - return Ok(store); - } - refuse_foreign_entries(&dir)?; - let generations = dir.join(GENERATIONS_NAME); - let problem = platform::create_owner_only_dir(&generations).map_err(store_error( - format!("cannot create the template store {}", dir.display()), - ))?; - refuse_problem( - &generations, - "the generations of the template store", - problem, - )?; - let store = Self { - dir, - id: StoreId::from_bytes(random_id("a template store's identity")?), - }; - store.write_index(&Index { - next: FIRST_GENERATION, - generations: Vec::new(), - })?; - write_record( - &store.dir.join(STORE_RECORD_NAME), - &StoreRecord { - format: STORE_FORMAT, - store: store.id.to_string(), - }, - "the identity record of the template store", - )?; - drop(lock); - Ok(store) - } - - /// Opens the store in the existing directory `dir`, which a relative path names from the - /// current directory, and which must be private to the current user, as - /// [`create`](Self::create) makes it. - /// - /// Fails with [`ErrorKind::NotFound`] if `dir` does not exist, or holds no store, and with - /// [`ErrorCode::FormatUnsupported`] if it holds a store of another format. - pub fn open(dir: impl Into) -> Result { - let dir = absolute(dir.into())?; - let problem = platform::check_owner_only_dir(&dir).map_err(store_error(format!( - "cannot open the template store {}", - dir.display() - )))?; - refuse_problem(&dir, "the template store", problem)?; - let Some(id) = read_identity(&dir)? else { - return Err(Error::new( - ErrorKind::NotFound, - format!("the directory {} holds no template store", dir.display()), - )); - }; - let store = Self { dir, id }; - store.check_generations_dir()?; - Ok(store) - } - - /// Returns the store's directory, as an absolute path. - pub fn dir(&self) -> &Path { - &self.dir - } - - /// Returns the store's identity. - pub fn id(&self) -> StoreId { - self.id - } - - /// Reads the store's index. - /// - /// A read that holds no lock sees one whole index, as the latest transaction left it. - /// Fails with [`ErrorCode::FormatUnsupported`] for an index of another format, and with - /// [`ErrorKind::Io`] for one that cannot be read, is malformed, or belongs to another store. - pub fn index(&self) -> Result { - let path = self.dir.join(INDEX_NAME); - let description = "index of the template store"; - let Some(record) = read_record::(&path, description, MAX_INDEX_LEN)? else { - return Err(Error::io(format!( - "the template store {} has no index", - self.dir.display() - ))); - }; - let malformed = |problem: String| { - Error::io(format!( - "the {description} {} is malformed: {problem}", - path.display() - )) - }; - if record.store != self.id.to_string() { - return Err(Error::io(format!( - "the {description} {} belongs to the template store {}, not to this one, {}", - path.display(), - record.store, - self.id - ))); - } - record.into_index().map_err(malformed) - } - - /// Returns the path of the payload of the store generation `generation`. - pub fn payload_dir(&self, generation: StoreGeneration) -> PathBuf { - self.dir.join(GENERATIONS_NAME).join(generation.to_string()) - } - - /// Checks the payload of `generation`, and returns the path of its snapshot, which OpenVMM - /// restores. - /// - /// The directories of the generations, of the payload, and of the snapshot must each be - /// private to the current user, and none of them a link, or on Windows any other reparse - /// point. The snapshot's directory is the one that OpenVMM published, which need not pass - /// access control entries on to the files created in it; see [Privacy](Store#privacy). Fails - /// with [`ErrorKind::NotFound`] if the payload or its snapshot is missing. - pub fn snapshot_dir(&self, generation: &Generation) -> Result { - let payload = self.payload_dir(generation.generation); - let snapshot = payload.join(SNAPSHOT_NAME); - self.check_generations_dir()?; - for (path, what, check) in [ - ( - &payload, - "the payload of the generation", - platform::check_owner_only_dir as fn(&Path) -> io::Result>, - ), - ( - &snapshot, - "the snapshot of the generation", - platform::check_snapshot_dir, - ), - ] { - let problem = check(path).map_err(store_error(format!( - "cannot open {what} {}", - path.display() - )))?; - refuse_problem(path, what, problem)?; - } - Ok(snapshot) - } - - /// Reads the compatibility key of `generation` from its payload. - /// - /// The key is read as it was written when the generation was indexed, without computing its - /// identity again. Fails with [`ErrorKind::NotFound`] if the payload or its key is missing, - /// and as [`TemplateKey::from_bytes`] does for a key that is not the canonical encoding of - /// one. - pub fn read_key(&self, generation: &Generation) -> Result { - let payload = self.payload_dir(generation.generation); - self.check_generations_dir()?; - let what = "the payload of the generation"; - let problem = platform::check_owner_only_dir(&payload).map_err(store_error(format!( - "cannot open {what} {}", - payload.display() - )))?; - refuse_problem(&payload, what, problem)?; - let path = payload.join(KEY_NAME); - let description = "template key"; - let bytes = - read_private(&path, description, TemplateKey::MAX_LEN as u64)?.ok_or_else(|| { - Error::new( - ErrorKind::NotFound, - format!("the {description} {} is missing", path.display()), - ) - })?; - TemplateKey::from_bytes(&bytes).map_err(|error| { - Error::new( - error.code(), - format!("cannot read {}: {}", path.display(), error.message()), - ) - }) - } - - /// Checks that the directory of the generations is private to the current user, and not a - /// link, as every read of a generation does, so that a clone's plan tells a store whose - /// directory fails it, the host's, from a generation that is malformed. - pub(super) fn check_generations_dir(&self) -> Result<()> { - let path = self.dir.join(GENERATIONS_NAME); - let what = "the generations of the template store"; - let problem = platform::check_owner_only_dir(&path).map_err(store_error(format!( - "cannot open {what} {}", - path.display() - )))?; - refuse_problem(&path, what, problem) - } - - /// Atomically replaces the store's index with `index`. - fn write_index(&self, index: &Index) -> Result<()> { - write_record( - &self.dir.join(INDEX_NAME), - &IndexRecord::new(self.id, index), - "the index of the template store", - ) - } -} - -/// Returns `dir` as an absolute path, which a relative one names from the current directory, -/// without trailing separators: Linux follows a link at a name that a separator ends, even -/// without following links at the final name, so the checks of the store's directory refuse a -/// link at its own name only without them. -fn absolute(dir: PathBuf) -> Result { - std::path::absolute(&dir) - .map(|absolute| absolute.components().collect()) - .map_err(|error| { - Error::io(format!( - "cannot resolve the template store {}", - dir.display() - )) - .with_source(error) - }) -} - -/// Fails unless `problem`, why the directory `path`, which `what` names, is not private to the -/// current user, is `None`. -fn refuse_problem(path: &Path, what: &str, problem: Option) -> Result<()> { - match problem { - None => Ok(()), - Some(problem) => Err(Error::io(format!("{what} {} {problem}", path.display()))), - } -} - -/// Returns the names of the entries of the directory `dir`. -fn entries(dir: &Path) -> Result> { - let failed = || store_error(format!("cannot list {}", dir.display())); - fs::read_dir(dir) - .map_err(failed())? - .map(|entry| { - Ok(entry - .map_err(failed())? - .file_name() - .to_string_lossy() - .into_owned()) - }) - .collect() -} - -/// Returns whether `name` is an entry that a creation of a store makes before its identity record: -/// the index, its lock, the directory of the generations, and the temporary files that an -/// interrupted replacement of a record leaves, `.NAME.NUMBER.tmp`. -fn is_creation_leftover(name: &str) -> bool { - [INDEX_NAME, INDEX_LOCK_NAME, GENERATIONS_NAME].contains(&name) - || name - .strip_prefix('.') - .and_then(|name| name.strip_suffix(".tmp")) - .and_then(|name| name.rsplit_once('.')) - .is_some_and(|(file, number)| { - [STORE_RECORD_NAME, INDEX_NAME].contains(&file) - && !number.is_empty() - && number.bytes().all(|byte| byte.is_ascii_digit()) - }) -} - -/// Fails with [`ErrorCode::ForeignFiles`] if the directory `dir`, which holds no store, holds -/// anything but what an interrupted creation of one leaves: any other entry, or any in the -/// directory of the generations, which must be private to the current user, if it exists. -fn refuse_foreign_entries(dir: &Path) -> Result<()> { - let mut foreign = entries(dir)? - .into_iter() - .filter(|name| !is_creation_leftover(name)) - .collect::>(); - let generations = dir.join(GENERATIONS_NAME); - let what = "the generations of the template store"; - match platform::check_owner_only_dir(&generations) { - Err(error) if error.kind() == io::ErrorKind::NotFound => {} - checked => { - let problem = checked.map_err(store_error(format!( - "cannot open {what} {}", - generations.display() - )))?; - refuse_problem(&generations, what, problem)?; - foreign.extend( - entries(&generations)? - .into_iter() - .map(|name| format!("{GENERATIONS_NAME}/{name}")), - ); - } - } - if foreign.is_empty() { - return Ok(()); - } - foreign.sort(); - Err(Error::new( - ErrorCode::ForeignFiles, - format!( - "the directory {} holds no template store, but entries that are not a store's: {}", - dir.display(), - foreign.join(", ") - ), - )) -} - -/// Opens the lock file of the store in `dir`, which it creates if needed, takes its lock, waiting -/// for it until `deadline`, or for as long as it takes without one, and checks that the path -/// still names the locked file. -/// -/// The store never removes or replaces its lock file, so one that its path no longer names once -/// it is locked was removed or replaced by someone else, and its lock serializes nothing: the -/// lock is then taken again, after a pause, until the deadline. -fn take_lock(dir: &Path, deadline: Option) -> Result { - let path = dir.join(INDEX_LOCK_NAME); - let failed = |error| store_error(format!("cannot lock {}", path.display()))(error); - let busy = || { - Error::new( - ErrorKind::TimedOut, - format!( - "another transaction of the template store {} is still in progress", - dir.display() - ), - ) - }; - loop { - let file = open_lock(&path).map_err(failed)?; - if let Some(problem) = platform::private_file_problem(&file).map_err(failed)? { - return Err(Error::io(format!( - "the lock of the template store {} {problem}", - path.display() - ))); - } - match deadline { - None => file.lock().map_err(failed)?, - Some(deadline) => loop { - match file.try_lock() { - Ok(()) => break, - Err(TryLockError::WouldBlock) => { - let left = deadline.saturating_duration_since(Instant::now()); - if left.is_zero() { - return Err(busy()); - } - thread::sleep(left.min(LOCK_RETRY_INTERVAL)); - } - Err(TryLockError::Error(error)) => return Err(failed(error)), - } - }, - } - let locked = platform::opened_file_id(&file).map_err(failed)?; - match platform::file_id(&path) { - Ok(current) if current == locked => return Ok(file), - Ok(_) => {} - Err(error) if error.kind() == io::ErrorKind::NotFound => {} - Err(error) if cfg!(windows) && error.kind() == io::ErrorKind::PermissionDenied => {} - Err(error) => return Err(failed(error)), - } - drop(file); - let left = deadline.map(|deadline| deadline.saturating_duration_since(Instant::now())); - if left.is_some_and(|left| left.is_zero()) { - return Err(busy()); - } - thread::sleep(left.map_or(LOCK_RETRY_INTERVAL, |left| left.min(LOCK_RETRY_INTERVAL))); - } -} - -/// Atomically replaces the store's record at `path`, its `description`, with `record`, as pretty -/// JSON and a newline. -fn write_record(path: &Path, record: &T, description: &str) -> Result<()> { - let mut text = serde_json::to_vec_pretty(record) - .map_err(|error| Error::other(format!("cannot encode {description}")).with_source(error))?; - text.push(b'\n'); - write_private(path, &text) -} - -/// The index of a store, as JSON. -#[derive(Debug, Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -struct IndexRecord { - /// [`STORE_FORMAT`]. - format: u32, - /// The identity of the store. - store: String, - /// The store generation that the next publication takes. - next_generation: u64, - /// The generations, in the order of their store generations. - generations: Vec, -} - -impl IndexRecord { - /// Returns the record of `index`, of the store `store`. - fn new(store: StoreId, index: &Index) -> Self { - Self { - format: STORE_FORMAT, - store: store.to_string(), - next_generation: index.next.get().get(), - generations: index - .generations - .iter() - .map(GenerationRecord::new) - .collect(), - } - } - - /// Checks the record, and returns its index, or says why it is malformed. - fn into_index(self) -> std::result::Result { - let next = NonZeroU64::new(self.next_generation) - .map(StoreGeneration::new) - .ok_or("its next store generation is 0")?; - if self.generations.len() > MAX_GENERATIONS { - return Err(format!( - "it lists {} generations, more than the {MAX_GENERATIONS} that a store holds", - self.generations.len() - )); - } - let mut generations = Vec::with_capacity(self.generations.len()); - for record in self.generations { - let generation = record.into_generation()?; - if generations - .last() - .is_some_and(|last: &Generation| last.generation >= generation.generation) - { - return Err(format!( - "it lists store generation {} out of order, or twice", - generation.generation - )); - } - if generation.generation >= next { - return Err(format!( - "it lists store generation {}, which its next store generation, {next}, does \ - not follow", - generation.generation - )); - } - generations.push(generation); - } - Ok(Index { next, generations }) - } -} - -/// A generation in the index of a store, as JSON. -#[derive(Debug, Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -struct GenerationRecord { - /// The store generation. - generation: u64, - /// The identity of the key. - key: String, - /// The identity of the capture class. - class: String, - /// The storage generation of the capture. - storage_generation: String, - /// When OpenVMM created the snapshot. - created_at: TimestampRecord, - /// The time contract of the capture. - time: TimeRecord, - /// The length of the saved state, in bytes. - state_len: u64, - /// The length of the RAM, in bytes. - memory_len: u64, -} - -impl GenerationRecord { - /// Returns the record of `generation`. - fn new(generation: &Generation) -> Self { - let created_at = generation - .created_at - .duration_since(SystemTime::UNIX_EPOCH) - .unwrap_or_default(); - Self { - generation: generation.generation.get().get(), - key: generation.key.to_string(), - class: generation.class.to_string(), - storage_generation: generation.storage.to_string(), - created_at: TimestampRecord { - seconds: created_at.as_secs(), - nanos: created_at.subsec_nanos(), - }, - time: TimeRecord::new(&generation.time), - state_len: generation.state_len, - memory_len: generation.memory_len, - } - } - - /// Checks the record, and returns its generation, or says why it is malformed. - fn into_generation(self) -> std::result::Result { - let generation = NonZeroU64::new(self.generation) - .map(StoreGeneration::new) - .ok_or("it lists store generation 0, which no generation has")?; - let field = |what: &str, error: Error| { - format!( - "the {what} of store generation {generation} is invalid: {}", - error.message() - ) - }; - let created_at = (self.created_at.nanos < 1_000_000_000) - .then(|| { - SystemTime::UNIX_EPOCH.checked_add(Duration::new( - self.created_at.seconds, - self.created_at.nanos, - )) - }) - .flatten() - .ok_or_else(|| { - format!("the creation time of store generation {generation} is invalid") - })?; - Ok(Generation { - generation, - key: self.key.parse().map_err(|error| field("key", error))?, - class: self.class.parse().map_err(|error| field("class", error))?, - storage: self - .storage_generation - .parse() - .map_err(|error| field("storage generation", error))?, - created_at, - time: self.time.into_contract().map_err(|problem| { - format!("the time contract of store generation {generation} {problem}") - })?, - state_len: self.state_len, - memory_len: self.memory_len, - }) - } -} - -/// A time, as JSON: seconds and nanoseconds since the Unix epoch. -#[derive(Debug, Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -struct TimestampRecord { - /// Whole seconds. - seconds: u64, - /// Nanoseconds within the second. - nanos: u32, -} - -/// The time contract of a capture, as JSON. -#[derive(Debug, Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -struct TimeRecord { - /// [`TimeContract::tsc_frequency_hz`]. - tsc_frequency_hz: u64, - /// [`TimeContract::tsc_tolerance_ppm`]. - tsc_tolerance_ppm: u32, - /// [`TimeContract::apic_frequency_hz`]. - apic_frequency_hz: u64, - /// [`TimeContract::capture_tsc`]. - capture_tsc: u64, - /// [`TimeContract::capture_utc_ns`]. - capture_utc_ns: u64, - /// [`TimeContract::capture_monotonic_ns`]. - capture_monotonic_ns: u64, - /// [`TimeContract::host_clock`], as a manifest spells it. - host_clock: String, - /// [`TimeContract::host_id`], as 32 hexadecimal digits. - host_id: String, - /// [`TimeContract::boot_id`]. - boot_id: BootId, - /// [`TimeContract::lineage`]. - lineage: u32, - /// [`TimeContract::cut_to_anchor_ns`]. - cut_to_anchor_ns: Option, -} - -impl TimeRecord { - /// Returns the record of `time`. - fn new(time: &TimeContract) -> Self { - Self { - tsc_frequency_hz: time.tsc_frequency_hz, - tsc_tolerance_ppm: time.tsc_tolerance_ppm, - apic_frequency_hz: time.apic_frequency_hz, - capture_tsc: time.capture_tsc, - capture_utc_ns: time.capture_utc_ns, - capture_monotonic_ns: time.capture_monotonic_ns, - host_clock: time.host_clock.as_str().to_owned(), - host_id: hex(&time.host_id), - boot_id: time.boot_id, - lineage: time.lineage, - cut_to_anchor_ns: time.cut_to_anchor_ns, - } - } - - /// Checks the record, and returns its time contract, or says why it is invalid. - fn into_contract(self) -> std::result::Result { - Ok(TimeContract { - tsc_frequency_hz: self.tsc_frequency_hz, - tsc_tolerance_ppm: self.tsc_tolerance_ppm, - apic_frequency_hz: self.apic_frequency_hz, - capture_tsc: self.capture_tsc, - capture_utc_ns: self.capture_utc_ns, - capture_monotonic_ns: self.capture_monotonic_ns, - host_clock: HostClock::parse(&self.host_clock) - .ok_or_else(|| format!("has the unknown host clock {:?}", self.host_clock))?, - host_id: parse_hex_id(&self.host_id) - .ok_or_else(|| format!("has the invalid host identity {:?}", self.host_id))?, - boot_id: self.boot_id, - lineage: self.lineage, - cut_to_anchor_ns: self.cut_to_anchor_ns, - }) - } -} diff --git a/nvx-rs/src/host/template/store/capture.rs b/nvx-rs/src/host/template/store/capture.rs deleted file mode 100644 index 660312d3e..000000000 --- a/nvx-rs/src/host/template/store/capture.rs +++ /dev/null @@ -1,1333 +0,0 @@ -//! The captures of templates, and their publication in a store: [`Store::create_template`], -//! [`Store::refresh_template`], and [`Store::recover_captures`]; see -//! [the store's documentation](Store#captures-and-publication). - -#[cfg(test)] -mod tests; - -use std::ffi::{OsStr, OsString}; -use std::fs; -use std::io; -use std::path::{Path, PathBuf}; -use std::thread; -use std::time::{Duration, Instant, SystemTime}; - -use serde::{Deserialize, Serialize}; - -use super::{ - GENERATIONS_NAME, Generation, Index, KEY_NAME, MAX_GENERATIONS, SNAPSHOT_NAME, STORE_FORMAT, - StorageGeneration, Store, entries, read_private, read_record, refuse_problem, store_error, - take_lock, write_record, -}; -use crate::activation::StoreGeneration; -use crate::error::{Error, ErrorCode, ErrorKind, Result}; -use crate::host::artifacts::{self, Artifacts}; -use crate::host::instance::generate_capability; -use crate::host::platform; -use crate::host::sandbox::distro::{CheckedDistro, DistroBinding, DistroFormat}; -use crate::host::sandbox::{LayerRole, SandboxLayer, SandboxPlan}; -use crate::host::snapshot::{check_exit, check_snapshot}; -use crate::host::store::files::{create_new, remove_if_present, sync_dir, write_private}; -use crate::host::store::regular::{identity_of, open_regular}; -use crate::host::store::{ - JOURNAL_NAME, LOCK_NAME, LOG_NAME, LaunchId, LockedStore, OUTCOME_NAME, Recovery, SOCKET_NAME, - SandboxStore, -}; -use crate::host::supervisor::{self, VmmCommand}; -use crate::host::template::eligibility::HostTime; -use crate::host::template::key::{CaptureClass, ClassIdentity, KeyIdentity, TemplateKey}; -use crate::host::template::manifest::{TemplateManifest, TimeContract}; -use crate::host::template::plan::{SCRATCH_NAME, TemplatePlan, TemplateResources}; - -/// Name of the directory of a store's captures, in which each capture class has a slot. -pub const CAPTURES_NAME: &str = "captures"; -/// Name of the transaction directory of a capture, in the slot of its class, which is also the -/// sandbox store of the capture's VM until the VM has ended. -pub const TRANSACTION_NAME: &str = "transaction"; -/// Name of the record of a capture, in its transaction and then in its generation's payload. -pub const CAPTURE_RECORD_NAME: &str = "capture.json"; - -/// Prefix of the name of the file that backs the RAM of a capture's VM, which OpenVMM creates -/// beside the snapshot's destination, and removes as the VM ends. -const MEMORY_FILE_PREFIX: &str = ".openvmm-microvm-memory-"; -/// Prefix of the names of the directories in which OpenVMM stages the snapshot, beside its -/// destination, and which it renames to the destination once the snapshot is complete: -/// `.snapshot.staging-`. -const STAGING_PREFIX: &str = ".snapshot.staging-"; - -/// Hexadecimal digits of the identity of a capture class that name its slot: few enough that the -/// path of the control socket of a capture's VMM fits a Unix socket's. -const SLOT_NAME_LEN: usize = 16; - -/// Most bytes of a Unix socket's path on Linux, `sun_path`, with its terminating NUL. -pub(in crate::host::template) const SOCKET_PATH_LIMIT: usize = 108; - -/// Longest record of a capture, in bytes. -const MAX_CAPTURE_RECORD_LEN: u64 = 4 * 1024; - -/// Interval at which a rename that Windows refuses, while another process holds a file below the -/// renamed directory open, is tried again, until the capture's deadline. -const RENAME_RETRY_INTERVAL: Duration = Duration::from_millis(25); - -/// The record of a capture, in messages. -const RECORD_DESCRIPTION: &str = "record of the capture"; -/// A capture's transaction, in messages. -const TRANSACTION_WHAT: &str = "the transaction of the capture"; -/// The directory of a store's captures, in messages. -const CAPTURES_WHAT: &str = "the captures of the template store"; - -/// What a template's capture takes: its class, the adapter's plan of its placeholder and its -/// network, the binding of the placeholder, and the artifacts that run its VM, which -/// [`Store::create_template`] and [`Store::refresh_template`] capture. -/// -/// The adapter's plan is lowered as [`TemplatePlan::new`] lowers it, with the processors, those -/// online at boot among them, and RAM of the class, and a scratch of the class's scratch length; -/// its upper is no part of the template. The class and the plan must describe one VM: a -/// placeholder whose binding records the class's `distro` length, and a network of the plan whose -/// NIC, as [`SandboxNetwork::template_arguments`] lowers it for a template, is the class's: none, -/// or one at the same guest's address, under the complete deny-all policy, whatever the policies -/// of the two networks. -/// -/// [`SandboxNetwork::template_arguments`]: crate::host::sandbox::SandboxNetwork::template_arguments -#[derive(Debug, Clone)] -pub struct Capture { - /// The class of the capture, which names its slot, and from which the capture takes its - /// backend, its CPU profile, its processors and those online at boot, its RAM, its scratch - /// length, and the runtime identity of its launch. - pub class: CaptureClass, - /// The adapter's plan, which [`SandboxPlan::adapter`] makes, of the placeholder `distro` and - /// the template's network. - pub adapter: SandboxPlan, - /// The binding of the placeholder, which the capture checks before its VM starts. - pub binding: DistroBinding, - /// The revision of the distro policy that the caller accepts, such as - /// [`DISTRO_POLICY_REVISION`](crate::host::sandbox::distro::DISTRO_POLICY_REVISION), which - /// [`DistroBinding::check`] requires of the binding's verdict. - pub policy_revision: u32, - /// The OpenVMM executable, the guest kernel, and the initramfs, whose identities the class's - /// runtime holds, which relative paths name from the current directory. - pub artifacts: Artifacts, -} - -/// How [`Store::create_template`] served a capture class: with a generation that the store held -/// already, or with one that it captured and published. -#[derive(Debug, Clone, PartialEq, Eq)] -pub enum Created { - /// A generation of the class that the store held already. - Existing(Generation), - /// A generation that a capture made, and that the store published. - Captured(Generation), -} - -impl Created { - /// Returns the generation. - pub fn generation(&self) -> &Generation { - match self { - Self::Existing(generation) | Self::Captured(generation) => generation, - } - } - - /// Returns the generation, consuming this. - pub fn into_generation(self) -> Generation { - match self { - Self::Existing(generation) | Self::Captured(generation) => generation, - } - } -} - -/// Syncs a directory, as [`sync_dir`] does. Captures take it as a parameter, so that tests -/// observe the order of their syncs, and fail them. -type SyncDir<'a> = &'a dyn Fn(&Path) -> Result<()>; - -/// Runs the VM of a capture in the slot whose lock the capture holds, as [`run_supervised`] does, -/// and returns once the VM has ended, and nothing of its launch runs. Tests run it in their own -/// process instead. -type RunVm<'a> = &'a dyn Fn(&mut LockedStore, &Vm<'_>) -> Result<()>; - -/// A point of a capture at which a test fails it, before the step that follows the point. -#[derive(Clone, Copy, Debug, PartialEq, Eq)] -enum Point { - /// The transaction holds the record of the capture and its scratch, and the VM has not - /// started. - Prepared, - /// The VM has ended, and nothing that it left is checked yet. - Exited, - /// The transaction is checked: it holds the template's key, durably. - Checked, - /// The envelope is renamed into the store, and the index does not list it yet. - Renamed, - /// The index lists the new generation. - Indexed, -} - -/// The seams of a capture, through which tests observe and interrupt it. -struct Hooks<'a> { - /// Syncs a directory. - sync: SyncDir<'a>, - /// Runs the capture's VM. - run: RunVm<'a>, - /// Fails the capture at a point, as a test asks. - interrupt: &'a dyn Fn(Point) -> Result<()>, - /// Whether a failure leaves the capture's files as they are, as a crash of the host would, - /// rather than retiring its transaction: in tests that crash captures. - crash: bool, -} - -impl Hooks<'static> { - /// Returns the seams of every capture outside tests. - fn supervised() -> Self { - Self { - sync: &sync_dir, - run: &run_supervised, - interrupt: &uninterrupted, - crash: false, - } - } -} - -impl Hooks<'_> { - /// Fails the capture at `point` if a test asks. - fn at(&self, point: Point) -> Result<()> { - (self.interrupt)(point) - } -} - -/// Lets a capture pass every point, as captures outside tests do. -fn uninterrupted(_: Point) -> Result<()> { - Ok(()) -} - -/// What a capture takes of its inputs once it has checked them, before it opens anything. -struct Inputs { - /// The identity of the class. - class: ClassIdentity, - /// The template's processors, those online at boot among them, and RAM, from the class. - resources: TemplateResources, - /// The artifacts, as absolute paths. - artifacts: Artifacts, -} - -impl Inputs { - /// Checks the inputs of `capture`, without reading or opening anything, and returns what the - /// capture takes of them. - /// - /// Fails with [`ErrorKind::Unsupported`] as [`CaptureClass::identity`] fails, and for a - /// placeholder whose binding records another length than the class's `distro`, and a plan - /// whose network attaches another NIC than the class's, as a template attaches it; and with - /// [`ErrorKind::Unavailable`] for an artifact whose path cannot be resolved. - fn check(capture: &Capture) -> Result { - let class = &capture.class; - let identity = class.identity()?; - let length = capture.binding.image().length(); - if length != class.distro.length { - return Err(Error::unsupported(format!( - "the placeholder {} is {length} bytes long, as its binding records it, but the \ - class's distro is {} bytes long", - capture.adapter.distro().path().display(), - class.distro.length - ))); - } - let planned = match capture.adapter.network() { - None => Vec::new(), - Some(network) => network.template_arguments()?, - }; - let requested = match &class.network { - None => Vec::new(), - Some(network) => network.template_arguments()?, - }; - if planned != requested { - return Err(Error::unsupported(format!( - "the adapter's plan attaches the NIC {planned:?}, but the class requests \ - {requested:?}" - ))); - } - let given = &capture.artifacts; - Ok(Self { - class: identity, - resources: TemplateResources { - processors: class.processors, - boot_processors: class.boot_processors, - memory: class.memory, - memory_capacity: class.memory_capacity, - }, - artifacts: Artifacts::new( - artifacts::absolute(&given.openvmm)?, - artifacts::absolute(&given.kernel)?, - artifacts::absolute(&given.initrd)?, - ), - }) - } -} - -/// The VM of a capture, as the capture's source runs it. -struct Vm<'a> { - /// The inputs of the capture. - capture: &'a Capture, - /// The plan of the template. - template: &'a TemplatePlan, - /// The OpenVMM executable, the guest kernel, and the initramfs, as absolute paths. - artifacts: &'a Artifacts, - /// When the capture must have ended. - deadline: Instant, -} - -impl Vm<'_> { - /// Returns the command of the capture's VMM, run in the capture's transaction `dir`, whose - /// control console listens on `endpoint`, and whose outcome report goes to `report`. - /// - /// Its arguments are the template's, as [`TemplatePlan::openvmm_arguments`] renders them, - /// then the backend, the CPU profile, the kernel and the initramfs, the template's kernel - /// command line, a boot console that discards its output, the control console, and the - /// outcome report. - fn command(&self, endpoint: &str, report: &Path, dir: &Path) -> Result> { - let class = &self.capture.class; - let mut arguments: Vec = self - .template - .openvmm_arguments()? - .into_iter() - .map(OsString::from) - .collect(); - let listen = format!("listen={endpoint}"); - for argument in [ - OsStr::new("--single-process"), - OsStr::new("--hypervisor"), - OsStr::new(class.hypervisor.as_str()), - OsStr::new("--cpu-profile"), - OsStr::new(&class.cpu_profile), - OsStr::new("--kernel"), - self.artifacts.kernel.as_os_str(), - OsStr::new("--initrd"), - self.artifacts.initrd.as_os_str(), - OsStr::new("--cmdline"), - OsStr::new(self.template.kernel_command_line()), - OsStr::new("--virtio-console"), - OsStr::new("none"), - OsStr::new("--microvm-control-console"), - OsStr::new(&listen), - OsStr::new("--microvm-control-auth-stdin"), - OsStr::new("--microvm-report"), - report.as_os_str(), - ] { - arguments.push(argument.to_owned()); - } - Ok(VmmCommand::new(&self.artifacts.openvmm, arguments, dir)) - } -} - -/// Runs the VM of a capture through the supervisor, from the sandbox store of `locked`, the -/// capture's transaction, as a launch of its own, until its VMM exits, or until the capture's -/// deadline, at which it ends the VMM, and then ends the launch. -/// -/// The capture opens no control session, and waits for no Ready: its guest captures itself, and -/// OpenVMM exits once it has published the snapshot, or once the capture failed. Fails with -/// [`ErrorKind::TimedOut`] if the deadline passes first, and with -/// [`ErrorCode::VmmIndeterminate`] if the launch cannot be ended, whose journal then remains for -/// the recovery of the slot; and as [`supervisor::launch`] fails. -fn run_supervised(locked: &mut LockedStore, vm: &Vm<'_>) -> Result<()> { - let report = locked.outcome_path(); - let dir = locked.dir().to_owned(); - let capability = generate_capability()?; - // A slot holds no lease, so each capture reserves its launch itself. - let launch = LaunchId::generate()?; - let starting = supervisor::launch( - locked, - &launch, - &vm.capture.class.runtime, - &capability, - |endpoint| vm.command(endpoint, &report, &dir), - )?; - let exited = starting.process().wait_bool(vm.deadline).map_err(|error| { - Error::io(format!( - "cannot wait for the VMM of the capture in {}", - dir.display() - )) - .with_source(error) - }); - if let Recovery::Indeterminate(reason) = starting.abandon()? { - return Err(indeterminate(&dir, &reason)); - } - if !exited? { - return Err(Error::timed_out(format!( - "the VM of the capture in {} did not end before the capture's deadline, and was torn \ - down", - dir.display() - ))); - } - Ok(()) -} - -/// Returns the error of a capture whose VMM, in the slot `dir`, may still run, for `reason`, -/// [`ErrorCode::VmmIndeterminate`]. -fn indeterminate(dir: &Path, reason: &str) -> Error { - Error::new( - ErrorCode::VmmIndeterminate, - format!( - "the VMM of the capture in {} may still run, and its launch remains for recovery: \ - {reason}", - dir.display() - ), - ) -} - -impl Store { - /// Returns a generation of the class of `capture` that the store holds, or captures one and - /// publishes it, waiting for the capture's slot and the store's lock until `deadline`, which - /// bounds the capture too. - /// - /// Captures of one class are serialized by the lock of the class's slot, under which a - /// creation first ends a capture of the class that a crash interrupted, and completes the - /// store's publications that a crash interrupted, as - /// [`recover_captures`](Self::recover_captures) does. It then calls `now` for the host's time, - /// once, and looks for a generation: the latest that the store indexes of the class, of the - /// key of the class's latest generation, that no tombstone quarantines, and that is eligible - /// at that time, as [`Generation::eligibility`] decides it, as [`select`](Self::select) - /// chooses among the generations of a key. Without one, it captures one, as - /// [`refresh_template`](Self::refresh_template) does. Since `now` is called once the creation - /// holds the slot, a generation that a racing creation captured while this one waited is - /// eligible at that time, so creations that race for one class make one generation between - /// them. [`HostTime::now`] samples this host's time; a caller that keeps a margin, as before - /// the window of a generation closes, returns a later time. - /// - /// Fails as `now` fails, and otherwise as [`refresh_template`](Self::refresh_template) does. - pub fn create_template( - &self, - capture: &Capture, - now: impl Fn() -> Result, - deadline: Instant, - ) -> Result { - self.capture(capture, Some(&now), deadline, &Hooks::supervised()) - } - - /// Captures a template of the class of `capture`, and publishes it as the store's next - /// generation, which later selections of its key take before the older ones, waiting for the - /// capture's slot and the store's lock until `deadline`, which bounds the capture too. - /// - /// The capture cold-boots the template's VM through [`supervisor::launch`], from its - /// transaction, in the slot of its class, and waits for it to exit: its guest captures itself, - /// and OpenVMM publishes the snapshot into the capture's transaction and exits. The capture - /// checks the files before the VM starts, and what the VM left once it has exited, publishes - /// the transaction in two commits, and leaves the older generations, and their leases, - /// untouched; see [the store's documentation](Store#captures-and-publication). - /// - /// Fails, and publishes nothing, but a transaction that it checked, which the next recovery - /// of the slot publishes: - /// - /// - with [`ErrorKind::Unsupported`] for inputs that [`Capture`] does not admit, as - /// [`TemplatePlan::new`] refuses them, for a store whose path is too long for the control - /// socket of a capture's VMM, on Linux, for one whose capture transaction path exceeds - /// 258 UTF-16 code units on Windows, including any verbatim prefix and trailing separator, - /// and for a store that holds - /// [`MAX_GENERATIONS`] generations; - /// - with [`ErrorKind::Unavailable`] for a placeholder that its binding's check refuses, as - /// [`DistroBinding::check`] fails, that is a link, or on Windows any other reparse point, - /// or that is not a regular file, and for one that belongs to another user than the current - /// one or an administrator, or that anyone else may change; - /// - with [`ErrorKind::Exited`] if the VM ended without a capture, as when its guest failed, - /// or OpenVMM did, even after it published the snapshot; - /// - with [`ErrorKind::TimedOut`] if the deadline passes before the VM ends, which the - /// capture then tears down, or before the capture takes its slot's lock or the store's; - /// - with [`ErrorCode::VmmIndeterminate`] if the capture cannot prove that the VMM of a - /// capture of the class, its own or an interrupted one, no longer runs: the slot then keeps - /// the launch and its transaction for a later recovery; - /// - with [`ErrorKind::Io`] for files of the slot, of the transaction, or of the store that - /// are links, that are not private to the current user, or that cannot be created, read, - /// synced, or renamed, and for a capture that did not leave what its class requests: a - /// snapshot whose manifest [`TemplateManifest::read`] refuses, that is not the class's, as - /// [`TemplateKey::new`] checks it, or whose artifacts are not exactly its manifest, RAM, and - /// saved state, each a regular file of the length that the manifest records. - pub fn refresh_template(&self, capture: &Capture, deadline: Instant) -> Result { - self.capture(capture, None, deadline, &Hooks::supervised()) - .map(Created::into_generation) - } - - /// Ends the captures that a crash interrupted, waiting for the lock of each slot and for the - /// store's until `deadline`, and returns the generations that it published, with those whose - /// publications it completed, in the order of their store generations. - /// - /// In each slot, it ends the launch of the capture's VMM, as [`LockedStore::recover`] ends - /// it, then publishes the capture's transaction if the capture had checked it, and retires it - /// otherwise. It then completes a publication that a crash interrupted once it had renamed - /// its envelope into the store, and drops from the index the generations whose payloads a - /// crash lost. A capture recovers the slot of its class, and the store's publications, the - /// same way, so an adapter recovers the other slots once, as it starts. - /// - /// Fails with [`ErrorCode::ForeignFiles`] if the directory of the captures holds entries that - /// are not slots, and otherwise as [`refresh_template`](Self::refresh_template) fails. - pub fn recover_captures(&self, deadline: Instant) -> Result> { - self.recover_with(deadline, &Hooks::supervised()) - } - - /// Serves `capture` as [`create_template`](Self::create_template) does with the host's time - /// that `now` returns, or, without it, as [`refresh_template`](Self::refresh_template) does, - /// through `hooks`. - fn capture( - &self, - capture: &Capture, - now: Option<&dyn Fn() -> Result>, - deadline: Instant, - hooks: &Hooks<'_>, - ) -> Result { - let inputs = Inputs::check(capture)?; - let slot = self.open_slot(inputs.class)?; - let locked = lock_slot(&slot, deadline)?; - self.recover_slot(&locked, deadline, hooks)?; - // The index then lists every generation that a publication renamed into the store, and - // none whose payload a crash lost, for the lookup and the store's capacity alike. - self.recover_publications(deadline, hooks)?; - if let Some(now) = now - && let Some(existing) = self.existing(inputs.class, &now()?)? - { - // The commit that listed it may have been interrupted before it made the index, or - // the store's own entry in its parent, durable. - self.sync_store(hooks)?; - return Ok(Created::Existing(existing)); - } - if self.index()?.generations.len() >= MAX_GENERATIONS { - return Err(self.full()); - } - let transaction = locked.dir().join(TRANSACTION_NAME); - let template = TemplatePlan::new( - &capture.adapter, - &capture.binding, - capture.class.scratch.length, - inputs.resources, - &transaction, - StorageGeneration::generate()?, - )?; - let envelope = - match self.check_capture(capture, &inputs, &template, &locked, deadline, hooks) { - Ok(envelope) => envelope, - Err(error) => { - // A transaction stays while its sandbox store records a launch, whose VMM may - // still write it, for the slot's recovery to retire once the VMM has ended; - // and so does one that a retirement interrupted, as a crash would leave it. - if !hooks.crash && !records_launch(&transaction) { - let _ = keep_diagnostics(&transaction, locked.dir()); - let _ = retire(&transaction); - } - return Err(error); - } - }; - hooks.at(Point::Checked)?; - self.publish(&transaction, &envelope, deadline, hooks) - .map(|(published, _)| Created::Captured(published)) - } - - /// Runs the capture of `template`, with the inputs `capture`, checked as `inputs`, in the slot - /// whose lock `slot` holds, through `hooks`, until the capture checks its transaction, or until - /// `deadline`, and returns its envelope. - /// - /// The transaction is the sandbox store of the capture's VM: OpenVMM requires the control - /// socket of a VM that captures a snapshot, on Linux, beside the snapshot's destination, and a - /// sandbox store's VM listens in the store's directory. - fn check_capture( - &self, - capture: &Capture, - inputs: &Inputs, - template: &TemplatePlan, - slot: &LockedStore, - deadline: Instant, - hooks: &Hooks<'_>, - ) -> Result { - let transaction = template.transaction(); - // The slot keeps the log and outcome report of its class's latest capture, this one, so - // that the capture never reads an earlier VM's report for its own. - for name in [LOG_NAME, OUTCOME_NAME] { - remove_if_present(&slot.dir().join(name))?; - } - let mut launch = SandboxStore::create(transaction)?.lock(Some(deadline))?; - write_record( - &transaction.join(CAPTURE_RECORD_NAME), - &CaptureRecord::new(self, inputs.class, template.generation()), - &format!("the {RECORD_DESCRIPTION}"), - )?; - let scratch = create_scratch(template.scratch_path(), template.scratch_length())?; - hooks.at(Point::Prepared)?; - let placeholder = check_files(capture, template, scratch)?; - let vm = Vm { - capture, - template, - artifacts: &inputs.artifacts, - deadline, - }; - (hooks.run)(&mut launch, &vm)?; - drop(placeholder); - keep_diagnostics(transaction, slot.dir())?; - hooks.at(Point::Exited)?; - check_exit(&slot.outcome_path())?; - let snapshot = template.destination(); - let manifest = check_snapshot(snapshot)?; - let key = TemplateKey::new(&capture.class, &manifest)?; - // The host closes its handles before it publishes: the transaction's lock goes, and its - // file with what else the capture leaves. - drop(launch); - clean_transaction(transaction)?; - write_private(&transaction.join(KEY_NAME), &key.to_bytes())?; - // The checked transaction must survive a crash, for the slot's recovery to publish it, - // and so must the store, whose own entry in its parent its creation does not sync. - for dir in [transaction, slot.dir(), &self.captures_dir()] { - (hooks.sync)(dir)?; - } - self.sync_store(hooks)?; - Ok(Envelope::new( - &key, - inputs.class, - template.generation(), - &manifest, - )) - } - - /// Publishes the checked transaction `transaction`, whose envelope is `envelope`, as the - /// store's next generation, under the store's lock, which it waits for until `deadline`, - /// through `hooks`, and returns the generation, with those of the publications that it - /// completed first. - /// - /// It first recovers the index, as [`recover_index`](Self::recover_index) does. Then it - /// renames the transaction to the payload of the next store generation, which it never - /// replaces, makes the rename durable, and commits the index, which lists the generation with - /// its full key, as [`commit_index`](Self::commit_index) does. - fn publish( - &self, - transaction: &Path, - envelope: &Envelope, - deadline: Instant, - hooks: &Hooks<'_>, - ) -> Result<(Generation, Vec)> { - let _lock = take_lock(&self.dir, Some(deadline))?; - let mut index = self.index()?; - let (completed, changed) = self.recover_index(&mut index, hooks)?; - if index.generations.len() >= MAX_GENERATIONS { - if changed { - self.commit_index(&index, hooks)?; - } - return Err(self.full()); - } - let generation = index.next; - rename_until(transaction, &self.payload_dir(generation), deadline)?; - (hooks.sync)(&self.dir.join(GENERATIONS_NAME))?; - if let Some(slot) = transaction.parent() { - (hooks.sync)(slot)?; - } - hooks.at(Point::Renamed)?; - let published = envelope.generation(generation); - index.generations.push(published.clone()); - index.next = following(generation)?; - self.commit_index(&index, hooks)?; - hooks.at(Point::Indexed)?; - Ok((published, completed)) - } - - /// Writes `index`, which the store's lock holds, and makes it durable through `hooks`, as - /// [`sync_store`](Self::sync_store) does, so that a crash of the host loses neither a - /// generation that it lists nor the store. - fn commit_index(&self, index: &Index, hooks: &Hooks<'_>) -> Result<()> { - self.write_index(index)?; - self.sync_store(hooks) - } - - /// Syncs the store's directory, and then its parent, through `hooks`: [`Store::create`] - /// syncs neither, so the store's own entry in its parent may not be durable yet. - fn sync_store(&self, hooks: &Hooks<'_>) -> Result<()> { - (hooks.sync)(&self.dir)?; - if let Some(parent) = self.dir.parent() { - (hooks.sync)(parent)?; - } - Ok(()) - } - - /// Completes the store's publications that a crash interrupted, and drops from the index the - /// generations whose payloads a crash lost, as [`recover_index`](Self::recover_index) does, - /// under the store's lock, which it waits for until `deadline`, through `hooks`. Commits the - /// index if that changed it, and returns the generations that it completed. - fn recover_publications( - &self, - deadline: Instant, - hooks: &Hooks<'_>, - ) -> Result> { - let _lock = take_lock(&self.dir, Some(deadline))?; - let mut index = self.index()?; - let (completed, changed) = self.recover_index(&mut index, hooks)?; - if changed { - self.commit_index(&index, hooks)?; - } - Ok(completed) - } - - /// Recovers `index`, which the store's lock holds, from the publications and retirements that - /// a crash interrupted, through `hooks`, and returns the generations that it indexed, and - /// whether it changed the index, which the caller writes. - /// - /// A generation whose payload is gone was lost in a crash, as one that a publication renamed - /// on a host that cannot flush a directory, or whose retirement removed its payload before it - /// wrote the index: the index first drops it, so that it takes none of the store's capacity. - /// Its tombstone, if any, stays for its retirement to remove. A publication renames its - /// envelope to the payload of the index's next store generation before it writes the index, - /// so a payload there is one whose publication was interrupted: the index then takes it, as - /// the publication would have, once the recovery has made the rename durable, as the - /// publication would have too: it syncs the payload, the directory of the payloads, and the - /// slot that the envelope left. - fn recover_index( - &self, - index: &mut Index, - hooks: &Hooks<'_>, - ) -> Result<(Vec, bool)> { - self.check_generations_dir()?; - let listed = index.generations.len(); - let mut kept = Vec::with_capacity(listed); - for generation in index.generations.drain(..) { - match fs::symlink_metadata(self.payload_dir(generation.generation)) { - Err(error) if error.kind() == io::ErrorKind::NotFound => {} - _ => kept.push(generation), - } - } - index.generations = kept; - let dropped = index.generations.len() != listed; - let mut completed = Vec::new(); - loop { - let payload = self.payload_dir(index.next); - match fs::symlink_metadata(&payload) { - Err(error) if error.kind() == io::ErrorKind::NotFound => break, - Err(error) => { - return Err(store_error(format!( - "cannot open the payload of the generation {}", - payload.display() - ))(error)); - } - Ok(_) => {} - } - if index.generations.len() >= MAX_GENERATIONS { - return Err(self.full()); - } - let generation = self.read_envelope(&payload)?.generation(index.next); - (hooks.sync)(&payload)?; - index.generations.push(generation.clone()); - index.next = following(index.next)?; - completed.push(generation); - } - if !completed.is_empty() { - (hooks.sync)(&self.dir.join(GENERATIONS_NAME))?; - let mut slots: Vec = completed - .iter() - .map(|generation| slot_name(generation.class)) - .collect(); - slots.sort(); - slots.dedup(); - for name in slots { - let slot = self.captures_dir().join(name); - match fs::symlink_metadata(&slot) { - Err(error) if error.kind() == io::ErrorKind::NotFound => {} - Err(error) => { - return Err(store_error(format!( - "cannot open the slot {}", - slot.display() - ))(error)); - } - Ok(metadata) if metadata.is_dir() => (hooks.sync)(&slot)?, - Ok(_) => {} - } - } - } - let changed = dropped || !completed.is_empty(); - Ok((completed, changed)) - } - - /// Reads the envelope of a checked capture in the directory `dir`, its transaction or its - /// payload, and returns what the index lists of it: the record of the capture, the template's - /// key, and the manifest of its snapshot, whose artifacts it checks again by their kinds and - /// lengths. - fn read_envelope(&self, dir: &Path) -> Result { - let what = "the envelope of the capture"; - let problem = platform::check_owner_only_dir(dir) - .map_err(store_error(format!("cannot open {what} {}", dir.display())))?; - refuse_problem(dir, what, problem)?; - let mut names = entries(dir)?; - names.sort(); - if names != [CAPTURE_RECORD_NAME, KEY_NAME, SNAPSHOT_NAME] { - return Err(Error::io(format!( - "{what} {} holds {}, not exactly the {RECORD_DESCRIPTION}, its key, and its \ - snapshot", - dir.display(), - names.join(", ") - ))); - } - let path = dir.join(CAPTURE_RECORD_NAME); - let record = - read_record::(&path, RECORD_DESCRIPTION, MAX_CAPTURE_RECORD_LEN)? - .ok_or_else(|| { - Error::io(format!( - "the {RECORD_DESCRIPTION} {} is missing", - path.display() - )) - })?; - let malformed = |problem: &str| { - Error::io(format!( - "the {RECORD_DESCRIPTION} {} is malformed: {problem}", - path.display() - )) - }; - if record.store != self.id.to_string() { - return Err(malformed(&format!( - "it is a capture of the template store {}", - record.store - ))); - } - let class: ClassIdentity = record - .class - .parse() - .map_err(|error: Error| malformed(error.message()))?; - let storage: StorageGeneration = record - .storage_generation - .parse() - .map_err(|error: Error| malformed(error.message()))?; - let path = dir.join(KEY_NAME); - let description = "template key"; - let bytes = read_private(&path, description, TemplateKey::MAX_LEN as u64)? - .ok_or_else(|| Error::io(format!("the {description} {} is missing", path.display())))?; - let key = TemplateKey::from_bytes(&bytes).map_err(|error| { - Error::new( - error.code(), - format!("cannot read {}: {}", path.display(), error.message()), - ) - })?; - let manifest = check_snapshot(&dir.join(SNAPSHOT_NAME))?; - Ok(Envelope::new(&key, class, storage, &manifest)) - } - - /// Ends the capture that a crash interrupted in the slot whose lock `slot` holds, if any, - /// through `hooks`: ends its VMM, which the transaction's sandbox store records, as - /// [`LockedStore::recover`] does, and then publishes its transaction if the capture checked - /// it, waiting for the store's lock until `deadline`, and retires it otherwise, once it has - /// moved the VMM's log and outcome report to the slot. Returns the generations that it - /// published: those of the publications that it completed first, as - /// [`publish`](Self::publish) does, and then the transaction's. - fn recover_slot( - &self, - slot: &LockedStore, - deadline: Instant, - hooks: &Hooks<'_>, - ) -> Result> { - let transaction = slot.dir().join(TRANSACTION_NAME); - match platform::check_owner_only_dir(&transaction) { - Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(Vec::new()), - checked => { - let problem = checked.map_err(store_error(format!( - "cannot open {TRANSACTION_WHAT} {}", - transaction.display() - )))?; - refuse_problem(&transaction, TRANSACTION_WHAT, problem)?; - } - } - end_launch(&transaction, deadline)?; - let key = transaction.join(KEY_NAME); - match fs::symlink_metadata(&key) { - Err(error) if error.kind() == io::ErrorKind::NotFound => { - keep_diagnostics(&transaction, slot.dir())?; - retire(&transaction)?; - Ok(Vec::new()) - } - Err(error) => Err(store_error(format!("cannot open {}", key.display()))(error)), - Ok(_) => { - let envelope = self.read_envelope(&transaction)?; - // The crash may have come before the capture synced the transaction that it - // checked, whose entries must be durable before the rename publishes them. - (hooks.sync)(&transaction)?; - let (published, mut completed) = - self.publish(&transaction, &envelope, deadline, hooks)?; - completed.push(published); - Ok(completed) - } - } - } - - /// Recovers the store's captures as [`recover_captures`](Self::recover_captures) does, - /// through `hooks`. - fn recover_with(&self, deadline: Instant, hooks: &Hooks<'_>) -> Result> { - let captures = self.captures_dir(); - let mut published = Vec::new(); - let exists = match platform::check_owner_only_dir(&captures) { - Err(error) if error.kind() == io::ErrorKind::NotFound => false, - checked => { - let problem = checked.map_err(store_error(format!( - "cannot open {CAPTURES_WHAT} {}", - captures.display() - )))?; - refuse_problem(&captures, CAPTURES_WHAT, problem)?; - true - } - }; - if exists { - let mut slots = entries(&captures)?; - slots.sort(); - let foreign: Vec<&str> = slots - .iter() - .map(String::as_str) - .filter(|name| !is_slot_name(name)) - .collect(); - if !foreign.is_empty() { - return Err(Error::new( - ErrorCode::ForeignFiles, - format!( - "{CAPTURES_WHAT} {} holds entries that are not the store's: {}", - captures.display(), - foreign.join(", ") - ), - )); - } - for name in slots { - let slot = SandboxStore::open(captures.join(name))?; - let locked = lock_slot(&slot, deadline)?; - published.extend(self.recover_slot(&locked, deadline, hooks)?); - } - } - published.extend(self.recover_publications(deadline, hooks)?); - published.sort_by_key(Generation::generation); - Ok(published) - } - - /// Returns the generation with which a creation serves the class `class` at the host's time - /// `now`, if the store holds one: the latest of the class, of the key of the class's latest - /// generation, that no tombstone quarantines, and that is eligible at `now`. - fn existing(&self, class: ClassIdentity, now: &HostTime) -> Result> { - let index = self.index()?; - let Some(latest) = index - .generations - .iter() - .rev() - .find(|generation| generation.class == class) - else { - return Ok(None); - }; - let key = latest.key; - let tombstoned = self.tombstones()?; - Ok(index - .generations - .iter() - .rev() - .find(|generation| { - generation.class == class - && generation.key == key - && !tombstoned.contains(&generation.generation) - && generation.eligibility(now).is_eligible() - }) - .cloned()) - } - - /// Creates the directory of the store's captures if needed, and the slot of the class - /// `class` in it, and returns the slot's store. - fn open_slot(&self, class: ClassIdentity) -> Result { - let captures = self.captures_dir(); - let slot = captures.join(slot_name(class)); - platform::check_working_dir(&slot.join(TRANSACTION_NAME))?; - create_private_dir(&captures, CAPTURES_WHAT)?; - let slot = SandboxStore::create(slot)?; - // A capture's VM listens in its transaction, the sandbox store from which it launches. - let socket = slot.dir().join(TRANSACTION_NAME).join(SOCKET_NAME); - if cfg!(target_os = "linux") && socket.as_os_str().len() >= SOCKET_PATH_LIMIT { - return Err(Error::unsupported(format!( - "the template store {} is too deep for the control socket of a capture's VMM, \ - {}, whose path must be shorter than {SOCKET_PATH_LIMIT} bytes", - self.dir.display(), - socket.display() - ))); - } - Ok(slot) - } - - /// Returns the path of the directory of the store's captures. - fn captures_dir(&self) -> PathBuf { - self.dir.join(CAPTURES_NAME) - } - - /// Returns the error of a store that holds [`MAX_GENERATIONS`] generations, which takes no - /// other. - fn full(&self) -> Error { - Error::unsupported(format!( - "the template store {} holds {MAX_GENERATIONS} generations, the most that a store \ - holds: retire some before a capture publishes another", - self.dir.display() - )) - } -} - -/// Takes the lock of the slot `slot`, waiting for it until `deadline`, as another capture of its -/// class may hold it. -fn lock_slot(slot: &SandboxStore, deadline: Instant) -> Result { - slot.lock(Some(deadline)).map_err(|error| { - if error.code() == ErrorCode::SandboxBusy { - Error::new( - ErrorKind::TimedOut, - format!( - "another capture of the class of the slot {} is still in progress", - slot.dir().display() - ), - ) - } else { - error - } - }) -} - -/// Returns the name of the slot of the class `class`: the first [`SLOT_NAME_LEN`] digits of its -/// identity. -fn slot_name(class: ClassIdentity) -> String { - class.to_string()[..SLOT_NAME_LEN].to_owned() -} - -/// Returns whether `name` is the name of a slot. -fn is_slot_name(name: &str) -> bool { - name.len() == SLOT_NAME_LEN - && name - .bytes() - .all(|byte| byte.is_ascii_digit() || (b'a'..=b'f').contains(&byte)) -} - -/// Creates the directory `path`, which `what` names, private to the current user, unless it -/// exists, which must be private already. -fn create_private_dir(path: &Path, what: &str) -> Result<()> { - let problem = platform::create_owner_only_dir(path).map_err(store_error(format!( - "cannot create {what} {}", - path.display() - )))?; - refuse_problem(path, what, problem) -} - -/// Creates the dormant scratch device's file at `path`, private to the current user: a sparse -/// file of `length` bytes, and returns its identity, as [`identity_of`] returns it. -fn create_scratch(path: &Path, length: u64) -> Result<(u64, u128)> { - let failed = |error| { - Error::io(format!( - "cannot create the scratch of the capture {}", - path.display() - )) - .with_source(error) - }; - let file = create_new(path).map_err(failed)?; - platform::make_sparse(&file).map_err(failed)?; - file.set_len(length).map_err(failed)?; - identity_of(&file).map_err(failed) -} - -/// Checks the files of the capture of `template`, with the inputs `capture`, before its VM -/// starts: its transaction directory, its placeholder, and its scratch, which the capture -/// created with the identity `scratch`. Returns the placeholder that the check of its binding -/// opened, which the capture holds, for reading alone, until its VM has ended. -/// -/// OpenVMM opens the block devices of a capture itself, at their paths, so the capture holds no -/// other handle of them as its VM starts, through which anything could write them. -fn check_files( - capture: &Capture, - template: &TemplatePlan, - scratch: (u64, u128), -) -> Result { - let transaction = template.transaction(); - let problem = platform::check_owner_only_dir(transaction).map_err(store_error(format!( - "cannot open {TRANSACTION_WHAT} {}", - transaction.display() - )))?; - refuse_problem(transaction, TRANSACTION_WHAT, problem)?; - let placeholder = check_placeholder(capture, template)?; - check_scratch(template, scratch)?; - Ok(placeholder) -} - -/// Checks the placeholder of `template` against the binding of `capture`, as -/// [`DistroBinding::check`] does, and then at its path: a regular file, not a link, nor on -/// Windows any other reparse point, that is the file that the check found, that belongs to the -/// current user or to an administrator, and that no one else may change. Returns the file that -/// the binding's check opened. -fn check_placeholder(capture: &Capture, template: &TemplatePlan) -> Result { - let path = template.plan().distro().path(); - let refused = |problem: String| { - Error::unavailable(format!("the placeholder {} {problem}", path.display())) - }; - let text = path - .to_str() - .ok_or_else(|| refused("has a path that is not valid Unicode".to_owned()))?; - let binding = &capture.binding; - let layer = match binding.format() { - DistroFormat::Erofs => SandboxLayer::new(LayerRole::Distro, text, binding.identity()), - DistroFormat::Gpt => SandboxLayer::gpt(LayerRole::Distro, text, binding.identity()), - }; - let checked = binding.check(&layer, capture.policy_revision)?; - let file = open_regular(path).map_err(|error| refused(format!("cannot be used: {error}")))?; - let identity = - |file| identity_of(file).map_err(|error| refused(format!("cannot be identified: {error}"))); - if identity(&file)? != identity(checked.file())? { - return Err(refused( - "changed while its binding was checked: its path names another file".to_owned(), - )); - } - if let Some(problem) = platform::foreign_writer_problem(&file) - .map_err(|error| refused(format!("cannot be checked: {error}")))? - { - return Err(refused(problem)); - } - Ok(checked) -} - -/// Checks the scratch of `template` at its path, which the capture created with the identity -/// `created`: a regular file of the template's scratch length, not a link, nor on Windows any -/// other reparse point, that is the file that the capture created, and that no one else may -/// change. The check's file is closed as it returns. -fn check_scratch(template: &TemplatePlan, created: (u64, u128)) -> Result<()> { - let path = template.scratch_path(); - let refused = |problem: String| { - Error::io(format!( - "the scratch of the capture {} {problem}", - path.display() - )) - }; - let file = open_regular(path).map_err(|error| refused(format!("cannot be used: {error}")))?; - let identity = - identity_of(&file).map_err(|error| refused(format!("cannot be identified: {error}")))?; - if identity != created { - return Err(refused( - "is not the file that the capture created".to_owned(), - )); - } - let length = file - .metadata() - .map_err(|error| refused(format!("cannot be examined: {error}")))? - .len(); - if length != template.scratch_length() { - return Err(refused(format!( - "is {length} bytes long, not the {} bytes of the template's scratch", - template.scratch_length() - ))); - } - if let Some(problem) = platform::foreign_writer_problem(&file) - .map_err(|error| refused(format!("cannot be checked: {error}")))? - { - return Err(refused(problem)); - } - Ok(()) -} - -/// Removes from the capture's transaction `transaction` what the template does not keep: the lock -/// file of its sandbox store, which the capture closed once its VM had ended, the dormant scratch -/// device's file, the file that backed the VM's RAM, which OpenVMM removes as the VM ends unless -/// it ends abruptly, and directories in which OpenVMM staged a snapshot that it did not publish. -/// Fails unless the transaction then holds the record of the capture and its snapshot alone. -fn clean_transaction(transaction: &Path) -> Result<()> { - for name in entries(transaction)? { - let path = transaction.join(&name); - if name == LOCK_NAME || name == SCRATCH_NAME || name.starts_with(MEMORY_FILE_PREFIX) { - remove_if_present(&path)?; - } else if name.starts_with(STAGING_PREFIX) { - match fs::remove_dir_all(&path) { - Err(error) if error.kind() != io::ErrorKind::NotFound => { - return Err(store_error(format!("cannot remove {}", path.display()))( - error, - )); - } - _ => {} - } - } - } - let mut names = entries(transaction)?; - names.sort(); - if names != [CAPTURE_RECORD_NAME, SNAPSHOT_NAME] { - return Err(Error::io(format!( - "{TRANSACTION_WHAT} {} holds {}, not exactly the {RECORD_DESCRIPTION} and its \ - snapshot", - transaction.display(), - names.join(", ") - ))); - } - Ok(()) -} - -/// Returns whether the sandbox store of the capture's transaction `transaction` records a launch, -/// whose VMM may still run, or whether that cannot be told: whether the journal of a launch is -/// there, unless nothing is. -fn records_launch(transaction: &Path) -> bool { - !matches!( - fs::symlink_metadata(transaction.join(JOURNAL_NAME)), - Err(error) if error.kind() == io::ErrorKind::NotFound - ) -} - -/// Ends the launch that the sandbox store of the capture's transaction `transaction` records, if -/// any, as [`LockedStore::recover`] ends it, waiting for the store's lock until `deadline`. -/// -/// A transaction without the store's lock file records no launch: a capture creates the lock file -/// before it launches, and removes it once its VM has ended, before it checks the transaction. -/// Fails with [`ErrorCode::VmmIndeterminate`] if the recovery cannot prove that the VMM no longer -/// runs, and the store then keeps the launch. -fn end_launch(transaction: &Path, deadline: Instant) -> Result<()> { - let lock = transaction.join(LOCK_NAME); - match fs::symlink_metadata(&lock) { - Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(()), - Err(error) => { - return Err(store_error(format!("cannot open {}", lock.display()))( - error, - )); - } - Ok(_) => {} - } - let mut launch = SandboxStore::open(transaction)?.lock(Some(deadline))?; - match launch.recover()? { - Recovery::Indeterminate(reason) => Err(indeterminate(transaction, &reason)), - _ => Ok(()), - } -} - -/// Moves the VMM's log and outcome report of the capture whose transaction is `transaction` to -/// its slot, `slot`, which keeps those of its class's latest capture, once the capture's launch -/// has ended. A file that the transaction lacks, such as one that this moved already, stays as it -/// is. -fn keep_diagnostics(transaction: &Path, slot: &Path) -> Result<()> { - for name in [LOG_NAME, OUTCOME_NAME] { - let from = transaction.join(name); - match fs::rename(&from, slot.join(name)) { - Err(error) if error.kind() != io::ErrorKind::NotFound => { - return Err(store_error(format!( - "cannot move {} to the slot {}", - from.display(), - slot.display() - ))(error)); - } - _ => {} - } - } - Ok(()) -} - -/// Removes the transaction `transaction` of a capture that did not check it, with everything in -/// it, once nothing of the capture runs, if it exists. One that is a link, or that is not private -/// to the current user, is refused, and kept. -fn retire(transaction: &Path) -> Result<()> { - match platform::check_owner_only_dir(transaction) { - Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(()), - checked => { - let problem = checked.map_err(store_error(format!( - "cannot open {TRANSACTION_WHAT} {}", - transaction.display() - )))?; - refuse_problem(transaction, TRANSACTION_WHAT, problem)?; - } - } - match fs::remove_dir_all(transaction) { - Err(error) if error.kind() != io::ErrorKind::NotFound => Err(store_error(format!( - "cannot retire {TRANSACTION_WHAT} {}", - transaction.display() - ))(error)), - _ => Ok(()), - } -} - -/// Renames the directory `from` to `to` unless `to` exists, which it leaves as it is, as -/// [`platform::rename_no_replace`] does. On Windows, which renames no directory while a file below -/// it is open, as a scanner may hold one, a refused rename is tried again until `deadline`. -fn rename_until(from: &Path, to: &Path, deadline: Instant) -> Result<()> { - loop { - match platform::rename_no_replace(from, to) { - Ok(()) => return Ok(()), - Err(error) - if cfg!(windows) - && error.kind() == io::ErrorKind::PermissionDenied - && Instant::now() < deadline => - { - thread::sleep(RENAME_RETRY_INTERVAL); - } - Err(error) => { - return Err(store_error(format!( - "cannot publish {} as {}", - from.display(), - to.display() - ))(error)); - } - } - } -} - -/// Returns the store generation that follows `generation`. -fn following(generation: StoreGeneration) -> Result { - generation - .get() - .checked_add(1) - .map(StoreGeneration::new) - .ok_or_else(|| Error::unsupported("the template store has assigned every store generation")) -} - -/// What a checked capture records of itself, which the index lists of its generation once a -/// publication gives it a store generation. -struct Envelope { - /// The identity of the template's key. - key: KeyIdentity, - /// The identity of the capture's class. - class: ClassIdentity, - /// The capture's storage generation. - storage: StorageGeneration, - /// When OpenVMM created the snapshot. - created_at: SystemTime, - /// The time contract of the capture. - time: TimeContract, - /// The length of the saved state, in bytes. - state_len: u64, - /// The length of the RAM, in bytes. - memory_len: u64, -} - -impl Envelope { - /// Returns the envelope of a capture of the class `class`, with the storage generation - /// `storage`, whose key is `key`, and whose snapshot's manifest is `manifest`. - fn new( - key: &TemplateKey, - class: ClassIdentity, - storage: StorageGeneration, - manifest: &TemplateManifest, - ) -> Self { - Self { - key: key.identity(), - class, - storage, - created_at: manifest.created_at(), - time: manifest.time().clone(), - state_len: manifest.state_len(), - memory_len: manifest.memory_len(), - } - } - - /// Returns the generation of the envelope, as the store generation `generation`. - fn generation(&self, generation: StoreGeneration) -> Generation { - Generation { - generation, - key: self.key, - class: self.class, - storage: self.storage, - created_at: self.created_at, - time: self.time.clone(), - state_len: self.state_len, - memory_len: self.memory_len, - } - } -} - -/// The record of a capture, as JSON, which the capture writes in its transaction before its VM -/// starts: the store, the class, and the storage generation of the capture. -#[derive(Debug, Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -struct CaptureRecord { - /// [`STORE_FORMAT`]. - format: u32, - /// The identity of the store. - store: String, - /// The identity of the capture's class. - class: String, - /// The capture's storage generation. - storage_generation: String, -} - -impl CaptureRecord { - /// Returns the record of a capture of the class `class` in `store`, with the storage - /// generation `storage`. - fn new(store: &Store, class: ClassIdentity, storage: StorageGeneration) -> Self { - Self { - format: STORE_FORMAT, - store: store.id.to_string(), - class: class.to_string(), - storage_generation: storage.to_string(), - } - } -} diff --git a/nvx-rs/src/host/template/store/capture/tests.rs b/nvx-rs/src/host/template/store/capture/tests.rs deleted file mode 100644 index 42b20b1d9..000000000 --- a/nvx-rs/src/host/template/store/capture/tests.rs +++ /dev/null @@ -1,1439 +0,0 @@ -//! Tests of the captures of templates and of their publication, whose VMs a source in this -//! process emulates as OpenVMM runs them: captures of a class and the generations that they -//! publish, the checks of their files before their VMs start and of what their VMs leave, their -//! failures and crashes around each commit, and their recovery. The tests of -//! `tests/template_capture.rs` run captures through the supervisor, with the fake OpenVMM. - -use std::cell::{Cell, RefCell}; -use std::fs::OpenOptions; -use std::sync::atomic::{AtomicUsize, Ordering}; -use std::sync::{Arc, Barrier}; - -use serde_json::{Value, json}; - -use super::*; -use crate::host::sandbox::distro::{DISTRO_POLICY_REVISION, PolicyVerdict}; -use crate::host::sandbox::{SandboxDistro, SandboxUpper}; -use crate::host::snapshot::tests::{MIB, ManifestSpec, class, network_class}; -use crate::host::snapshot::{MANIFEST_NAME, SNAPSHOT_ARTIFACTS}; -use crate::host::store::{CallerFiles, JOURNAL_NAME, LaunchId}; -use crate::host::template::eligibility::MAX_DOWNTIME_NS; -use crate::host::template::store::Lease; -use crate::host::template::store::tests::{ - cause, link_dir, link_file, private_dir, publish_with, storage, store, -}; -use crate::protocol::root::RootImage; -use crate::uuid::Uuid; - -/// Bytes in a GiB. -const GIB: u64 = 1024 * MIB; - -/// Nanoseconds in a second. -const SECOND: u64 = 1_000_000_000; - -/// Name of the saved state in the emulated snapshot. -const STATE_NAME: &str = "state.bin"; -/// Name of the RAM in the emulated snapshot. -const MEMORY_NAME: &str = "memory.bin"; - -/// The identity of the placeholder. -const IDENTITY: &str = "11111111-1111-1111-1111-111111111111"; - -/// The access control list that OpenVMM gives a snapshot's directory and each of its files on -/// Windows: SYSTEM and the owner alone, and nothing that files inherit. -#[cfg(windows)] -const OPENVMM_ACCESS: &str = "D:P(A;;FA;;;SY)(A;;FA;;;OW)"; - -/// Returns a deadline that no capture of these tests reaches. -fn deadline() -> Instant { - Instant::now() + Duration::from_secs(60) -} - -/// Returns the host's time `seconds` after the captures of [`ManifestSpec::template`], on the -/// host and in the boot that captured them. -fn after(seconds: u64) -> HostTime { - let manifest = TemplateManifest::parse(&ManifestSpec::template().encode()).unwrap(); - let time = manifest.time(); - HostTime { - host_id: time.host_id, - boot_id: time.boot_id, - clock: time.host_clock, - monotonic_ns: time.capture_monotonic_ns + seconds * SECOND, - utc_ns: time.capture_utc_ns + seconds * SECOND, - } -} - -/// Returns the host's time `seconds` before the captures of [`ManifestSpec::template`], at -/// which none of them is eligible yet. -fn before(seconds: u64) -> HostTime { - let anchor = after(0); - HostTime { - monotonic_ns: anchor.monotonic_ns - seconds * SECOND, - utc_ns: anchor.utc_ns - seconds * SECOND, - ..anchor - } -} - -/// Creates the sparse file `path` of `length` bytes, which no one else may write, whatever the -/// umask. -fn sparse(path: &Path, length: u64) { - let mut options = OpenOptions::new(); - options.write(true).create_new(true); - #[cfg(unix)] - std::os::unix::fs::OpenOptionsExt::mode(&mut options, 0o600); - let file = options.open(path).unwrap(); - platform::make_sparse(&file).unwrap(); - file.set_len(length).unwrap(); -} - -/// Lets anyone change the file or directory `path`. -fn let_anyone_write(path: &Path) { - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - - let mode = if path.is_dir() { 0o777 } else { 0o666 }; - fs::set_permissions(path, fs::Permissions::from_mode(mode)).unwrap(); - } - #[cfg(windows)] - platform::tests::set_access(path, "D:P(A;OICI;FA;;;OW)(A;OICI;FA;;;SY)(A;OICI;FA;;;WD)") - .unwrap(); -} - -/// Returns the names of the entries of `dir`, sorted. -fn sorted(dir: &Path) -> Vec { - let mut names = entries(dir).unwrap(); - names.sort(); - names -} - -/// Returns the file at `path` as JSON. -fn json_of(path: &Path) -> Value { - serde_json::from_slice(&fs::read(path).unwrap()).unwrap() -} - -/// What the emulated VM of a capture does, as OpenVMM would: it stages a snapshot of the -/// manifest that it publishes beside the destination, with the RAM as a link to the file that -/// backed it, renames it to the destination, and writes the outcome report. -#[derive(Clone)] -struct Emulation { - /// The manifest of the snapshot that it publishes, if it publishes one. - manifest: Option, - /// The operation, category, and status code that its outcome report records, if it writes - /// one. - outcome: Option<(&'static str, &'static str, i64)>, - /// An entry that it publishes in the snapshot besides its artifacts. - extra: Option<&'static str>, - /// An artifact that it leaves out of the snapshot. - missing: Option<&'static str>, - /// The length of the saved state that it publishes, if not the manifest's. - state_len: Option, - /// Whether it leaves the file that backed the RAM, and a staging directory, as OpenVMM leaves - /// them when it ends abruptly. - leftovers: bool, -} - -impl Emulation { - /// Returns the emulation of a capture of [`ManifestSpec::template`] that succeeds. - fn capture() -> Self { - Self { - manifest: Some(ManifestSpec::template()), - outcome: Some(("capture", "success", 0)), - extra: None, - missing: None, - state_len: None, - leftovers: false, - } - } - - /// Returns the emulation of a VM whose guest failed with 3 before it captured. - fn guest_failure() -> Self { - Self { - manifest: None, - outcome: Some(("capture", "guest-exit", 3)), - ..Self::capture() - } - } - - /// Runs the VM of `vm` from the sandbox store of `locked`, the capture's transaction, after - /// removing the outcome report of the store's previous VM, as a launch does. - fn run(&self, locked: &mut LockedStore, vm: &Vm<'_>) -> Result<()> { - // OpenVMM requires the control socket of a VM that captures a snapshot, on Linux, beside - // the snapshot's destination: the VM launches from its transaction. - assert_eq!(locked.dir(), vm.template.transaction()); - assert_eq!( - locked.socket_path().parent(), - vm.template.destination().parent() - ); - remove_if_present(&locked.outcome_path())?; - let transaction = vm.template.transaction(); - let ram = transaction.join(format!("{MEMORY_FILE_PREFIX}emulated")); - sparse(&ram, vm.capture.class.memory); - if let Some(spec) = &self.manifest { - let staging = transaction.join(format!("{STAGING_PREFIX}1-0-0")); - private_dir(&staging); - fs::write(staging.join(MANIFEST_NAME), spec.encode()).unwrap(); - sparse( - &staging.join(STATE_NAME), - self.state_len.unwrap_or(spec.state_size), - ); - fs::hard_link(&ram, staging.join(MEMORY_NAME)).unwrap(); - if let Some(extra) = self.extra { - fs::write(staging.join(extra), b"").unwrap(); - } - if let Some(missing) = self.missing { - fs::remove_file(staging.join(missing)).unwrap(); - } - #[cfg(windows)] - for name in entries(&staging).unwrap() { - platform::tests::set_access(&staging.join(name), OPENVMM_ACCESS).unwrap(); - } - #[cfg(windows)] - platform::tests::set_access(&staging, OPENVMM_ACCESS).unwrap(); - fs::rename(&staging, vm.template.destination()).unwrap(); - } - if self.leftovers { - private_dir(&transaction.join(format!("{STAGING_PREFIX}1-0-1"))); - } else { - fs::remove_file(&ram).unwrap(); - } - if let Some((operation, category, status_code)) = self.outcome { - let report = json!({ - "schema_version": 1, - "backend": vm.capture.class.hypervisor.as_str(), - "outcome": { - "operation": operation, - "category": category, - "status_code": status_code, - }, - "teardown": { "vm_stopped": true }, - }); - fs::write(locked.outcome_path(), report.to_string()).unwrap(); - } - Ok(()) - } -} - -/// A store, and the inputs of captures into it, of [`class`], whose VMs [`Emulation`]s run. -struct Bench { - /// The temporary directory that holds the store and the platform's files. - _directory: tempfile::TempDir, - /// The store. - store: Store, - /// The inputs of a capture. - capture: Capture, - /// The number of VMs that captures ran. - runs: Cell, -} - -impl Bench { - /// Returns a bench whose placeholder is a regular file, in a directory private to the current - /// user. - fn new() -> Self { - Self::with(Path::to_path_buf) - } - - /// Returns a bench whose placeholder is at the path that `prepare` returns, given a sparse - /// file of 1 GiB in a directory private to the current user, which it may change before the - /// placeholder's binding is made. - fn with(prepare: impl FnOnce(&Path) -> PathBuf) -> Self { - let (directory, store) = store(); - let platform = directory.path().join("platform"); - private_dir(&platform); - let image = platform.join("placeholder.erofs"); - sparse(&image, GIB); - let placeholder = prepare(&image); - let path = placeholder.to_str().unwrap(); - let layer = SandboxLayer::new(LayerRole::Distro, path, IDENTITY); - let binding = DistroBinding::import(&layer, PolicyVerdict::ACCEPTED).unwrap(); - let image = RootImage::erofs(Some(Uuid::parse(IDENTITY).unwrap())); - let upper = SandboxUpper::scratch(platform.join("clone.ext4").to_str().unwrap()); - let capture = Capture { - class: class(), - adapter: SandboxPlan::adapter(SandboxDistro::new(path, image), upper), - binding, - policy_revision: DISTRO_POLICY_REVISION, - artifacts: Artifacts::new( - platform.join("openvmm"), - platform.join("vmlinux"), - platform.join("initramfs.cpio.gz"), - ), - }; - Self { - _directory: directory, - store, - capture, - runs: Cell::new(0), - } - } - - /// Returns the slot of the class of the captures. - fn slot(&self) -> PathBuf { - self.store - .captures_dir() - .join(slot_name(self.capture.class.identity().unwrap())) - } - - /// Returns the transaction of the captures. - fn transaction(&self) -> PathBuf { - self.slot().join(TRANSACTION_NAME) - } - - /// Serves the capture as [`Store::create_template`] does with the host's time that `now` - /// returns, or, without it, as [`Store::refresh_template`] does, with `emulation` as its VM, - /// failing it at the points at which `interrupt` fails, and crashing it if `crash`. - fn serve( - &self, - now: Option<&dyn Fn() -> Result>, - emulation: &Emulation, - interrupt: &dyn Fn(Point) -> Result<()>, - crash: bool, - ) -> Result { - self.serve_with( - now, - &|locked, vm| emulation.run(locked, vm), - interrupt, - crash, - ) - } - - /// Serves the capture as [`serve`](Self::serve) does, with `run` as its source. - fn serve_with( - &self, - now: Option<&dyn Fn() -> Result>, - run: &dyn Fn(&mut LockedStore, &Vm<'_>) -> Result<()>, - interrupt: &dyn Fn(Point) -> Result<()>, - crash: bool, - ) -> Result { - let counted = |locked: &mut LockedStore, vm: &Vm<'_>| { - self.runs.set(self.runs.get() + 1); - run(locked, vm) - }; - let hooks = Hooks { - sync: &sync_dir, - run: &counted, - interrupt, - crash, - }; - self.store.capture(&self.capture, now, deadline(), &hooks) - } - - /// Serves the capture as [`Store::create_template`] does, a second after the captures. - fn create(&self, emulation: &Emulation) -> Result { - self.serve(Some(&|| Ok(after(1))), emulation, &uninterrupted, false) - } - - /// Serves the capture as [`Store::refresh_template`] does. - fn refresh(&self, emulation: &Emulation) -> Result { - self.serve(None, emulation, &uninterrupted, false) - .map(Created::into_generation) - } - - /// Refreshes the capture, failing it at `point`, or crashing it there if `crash`, and returns - /// the error. - fn interrupted(&self, point: Point, crash: bool) -> Error { - let interrupt = |at: Point| { - if at == point { - Err(Error::other(format!("failed at {at:?}"))) - } else { - Ok(()) - } - }; - self.serve(None, &Emulation::capture(), &interrupt, crash) - .unwrap_err() - } - - /// Recovers the store's captures. - fn recover(&self) -> Result> { - self.recover_syncing(&sync_dir) - } - - /// Recovers the store's captures, syncing directories through `sync`. - fn recover_syncing(&self, sync: SyncDir<'_>) -> Result> { - let run = |_: &mut LockedStore, _: &Vm<'_>| -> Result<()> { - unreachable!("a recovery runs no VM") - }; - let hooks = Hooks { - sync, - run: &run, - interrupt: &uninterrupted, - crash: false, - }; - self.store.recover_with(deadline(), &hooks) - } - - /// Returns the store generations that the index lists. - fn indexed(&self) -> Vec { - numbers(self.store.index().unwrap().generations()) - } - - /// Drops the index's last generation and keeps its payload, as a crash leaves a publication - /// that renamed its envelope into the store before it committed the index. Captures complete - /// such publications before they run, so tests that need one beside a checked transaction - /// make it this way. - fn unindex_last(&self) { - let mut index = self.store.index().unwrap(); - let last = index.generations.pop().unwrap(); - index.next = last.generation(); - self.store.write_index(&index).unwrap(); - } -} - -/// Returns the store generation `generation`, which must not be 0. -fn store_generation(generation: u64) -> StoreGeneration { - StoreGeneration::new(std::num::NonZeroU64::new(generation).unwrap()) -} - -/// Returns the store generations of `generations`, as numbers. -fn numbers(generations: &[Generation]) -> Vec { - generations - .iter() - .map(|generation| generation.generation().get().get()) - .collect() -} - -/// Returns the files that the sandboxes of these tests keep: a configuration record. -fn files() -> CallerFiles { - CallerFiles::new("config.json").unwrap() -} - -/// Provisions the sandbox `name` in `parent`, and returns its directory. -fn provision(parent: &Path, name: &str) -> PathBuf { - let dir = parent.join(name); - let locked = SandboxStore::lock_for_provision(&dir, &files(), None).unwrap(); - fs::write(dir.join("config.json"), b"{}").unwrap(); - drop(locked); - dir -} - -/// Asserts that the slot of `bench` holds no transaction, and records no launch. -fn assert_clean(bench: &Bench) { - assert!(!bench.transaction().exists()); - assert!(!bench.slot().join(JOURNAL_NAME).exists()); -} - -#[test] -fn captures_publish_generations_that_creations_then_serve() { - let bench = Bench::new(); - let Created::Captured(first) = bench.create(&Emulation::capture()).unwrap() else { - panic!("the first creation of a class captures it"); - }; - assert_eq!(bench.runs.get(), 1); - assert_eq!(first.generation(), store_generation(1)); - let class = class(); - let manifest = TemplateManifest::parse(&ManifestSpec::template().encode()).unwrap(); - let key = TemplateKey::new(&class, &manifest).unwrap(); - assert_eq!(first.key(), key.identity()); - assert_eq!(first.class(), class.identity().unwrap()); - assert_eq!(first.created_at(), manifest.created_at()); - assert_eq!(first.time(), manifest.time()); - assert_eq!( - (first.state_len(), first.memory_len()), - (manifest.state_len(), manifest.memory_len()) - ); - assert_eq!( - bench.store.index().unwrap().generations(), - std::slice::from_ref(&first) - ); - // The payload is the envelope: OpenVMM's snapshot, with the capture's record and the - // template's key beside it. - let payload = bench.store.payload_dir(first.generation()); - assert_eq!( - sorted(&payload), - [CAPTURE_RECORD_NAME, KEY_NAME, SNAPSHOT_NAME] - ); - assert_eq!(sorted(&payload.join(SNAPSHOT_NAME)), SNAPSHOT_ARTIFACTS); - assert_eq!(bench.store.read_key(&first).unwrap(), key); - assert_eq!( - bench.store.snapshot_dir(&first).unwrap(), - payload.join(SNAPSHOT_NAME) - ); - assert_eq!( - json_of(&payload.join(CAPTURE_RECORD_NAME)), - json!({ - "format": STORE_FORMAT, - "store": bench.store.id().to_string(), - "class": class.identity().unwrap().to_string(), - "storageGeneration": first.storage_generation().to_string(), - }) - ); - assert_clean(&bench); - // A creation serves the class with the generation that it holds, and runs no VM. - assert_eq!( - bench.create(&Emulation::capture()).unwrap(), - Created::Existing(first.clone()) - ); - assert_eq!(bench.runs.get(), 1); - // A refresh always captures, with a fresh storage generation, and creations then serve the - // new generation. - let second = bench.refresh(&Emulation::capture()).unwrap(); - assert_eq!(bench.runs.get(), 2); - assert_eq!(second.generation(), store_generation(2)); - assert_ne!(second.storage_generation(), first.storage_generation()); - assert_eq!(second.key(), first.key()); - assert_eq!( - bench.create(&Emulation::capture()).unwrap(), - Created::Existing(second.clone()) - ); - assert_eq!(bench.indexed(), [1, 2]); - assert_eq!(bench.store.read_key(&first).unwrap(), key); -} - -#[test] -fn captures_limit_the_processors_that_their_class_keeps_offline_at_boot() { - let mut bench = Bench::new(); - bench.capture.class.processors = 4; - bench.capture.class.boot_processors = 1; - // OpenVMM records the processors online at boot that the command line limits. - let mut spec = ManifestSpec::template(); - spec.vp_count = 4; - let contract = spec.contract(); - contract.topology = (1, 1, 4, 1, vec![0, 1, 2, 3]); - contract.boot_online_vp_count = 1; - let emulation = Emulation { - manifest: Some(spec), - ..Emulation::capture() - }; - let arguments = RefCell::new(Vec::new()); - let run = |locked: &mut LockedStore, vm: &Vm<'_>| { - let command = vm.command("control", Path::new("outcome.json"), locked.dir())?; - arguments.replace(command.arguments); - emulation.run(locked, vm) - }; - let created = bench.serve_with(Some(&|| Ok(after(1))), &run, &uninterrupted, false); - let Created::Captured(generation) = created.unwrap() else { - panic!("the first creation of a class captures it"); - }; - // The VM has the class's processors, and boots with those that it keeps online. - let arguments: Vec = arguments - .take() - .into_iter() - .map(|argument| argument.into_string().unwrap()) - .collect(); - let value = |option: &str| { - let at = arguments.iter().position(|given| given == option).unwrap(); - arguments[at + 1].as_str() - }; - assert_eq!(value("--processors"), "4"); - assert_eq!(value("--cmdline"), "nvx_sandbox=1 maxcpus=1"); - let key = bench.store.read_key(&generation).unwrap(); - assert_eq!((key.processors(), key.boot_processors()), (4, 1)); -} - -#[test] -fn refreshes_leave_older_generations_and_their_leases_untouched() { - let bench = Bench::new(); - let first = bench.refresh(&Emulation::capture()).unwrap(); - let sandboxes = tempfile::tempdir().unwrap(); - let sandbox = provision(sandboxes.path(), "clone"); - let lease = bench - .store - .acquire(first.generation(), &sandbox, &files(), &after(1), None) - .unwrap() - .lease; - let payload = bench.store.payload_dir(first.generation()); - let before = (sorted(&payload), bench.store.read_key(&first).unwrap()); - let second = bench.refresh(&Emulation::capture()).unwrap(); - assert_eq!(bench.indexed(), [1, 2]); - assert_eq!( - bench.store.index().unwrap().get(first.generation()), - Some(&first) - ); - assert_eq!( - (sorted(&payload), bench.store.read_key(&first).unwrap()), - before - ); - assert_eq!( - bench.store.leases(first.generation()).unwrap(), - std::slice::from_ref(&lease) - ); - assert!(bench.store.leases(second.generation()).unwrap().is_empty()); - assert_eq!( - Lease::of(&SandboxStore::open(&sandbox).unwrap()).unwrap(), - Some(lease) - ); -} - -#[test] -fn creations_capture_unless_the_class_has_a_generation_that_restores_take() { - let bench = Bench::new(); - // A generation of another class serves none of this one's. - let other = publish_with( - &bench.store, - &network_class(), - &ManifestSpec::with_network(), - storage(9), - ); - let Created::Captured(first) = bench.create(&Emulation::capture()).unwrap() else { - panic!("a class without generations is captured"); - }; - assert_eq!(first.generation(), store_generation(2)); - // A quarantined generation serves no creation. - bench.store.tombstone(first.generation(), None).unwrap(); - let Created::Captured(second) = bench.create(&Emulation::capture()).unwrap() else { - panic!("a class whose generations are quarantined is captured"); - }; - assert_eq!(second.generation(), store_generation(3)); - // Neither does one that no restore takes at the host's time. - let late = HostTime { - monotonic_ns: after(0).monotonic_ns + MAX_DOWNTIME_NS + SECOND, - ..after(0) - }; - assert!(!second.eligibility(&late).is_eligible()); - let Created::Captured(third) = bench - .serve( - Some(&|| Ok(late)), - &Emulation::capture(), - &uninterrupted, - false, - ) - .unwrap() - else { - panic!("a class whose generations are ineligible is captured"); - }; - assert_eq!(third.generation(), store_generation(4)); - assert_eq!( - bench.create(&Emulation::capture()).unwrap(), - Created::Existing(third) - ); - assert_eq!(bench.runs.get(), 3); - assert_eq!( - bench.store.index().unwrap().get(other.generation()), - Some(&other) - ); -} - -#[test] -fn failures_before_the_check_retire_the_transaction() { - let bench = Bench::new(); - for (point, runs) in [(Point::Prepared, 0), (Point::Exited, 1)] { - let error = bench.interrupted(point, false); - assert_eq!(error.message(), format!("failed at {point:?}")); - assert_eq!(bench.runs.replace(0), runs, "{point:?}"); - assert_clean(&bench); - } - let failures = [ - ( - Emulation::guest_failure(), - ErrorKind::Exited, - "records the outcome guest-exit with status 3 of the operation capture", - ), - // OpenVMM may fail once it has published the snapshot, as when it cannot make the - // publication durable. - ( - Emulation { - outcome: Some(("capture", "vmm-failure", 1)), - ..Emulation::capture() - }, - ErrorKind::Exited, - "records the outcome vmm-failure with status 1", - ), - ( - Emulation { - outcome: None, - ..Emulation::capture() - }, - ErrorKind::Exited, - "ended without the outcome report", - ), - ( - Emulation { - outcome: Some(("run", "success", 0)), - ..Emulation::capture() - }, - ErrorKind::Exited, - "of the operation run", - ), - ( - Emulation { - manifest: None, - ..Emulation::capture() - }, - ErrorKind::Io, - "cannot open the snapshot of the capture", - ), - ( - Emulation { - missing: Some(MANIFEST_NAME), - ..Emulation::capture() - }, - ErrorKind::Io, - "cannot read the snapshot manifest", - ), - ( - Emulation { - missing: Some(MEMORY_NAME), - ..Emulation::capture() - }, - ErrorKind::Io, - "holds manifest.bin, state.bin, not exactly the manifest", - ), - ( - Emulation { - manifest: Some(ManifestSpec::with_network()), - ..Emulation::capture() - }, - ErrorKind::Io, - "the capture does not match its class", - ), - ( - Emulation { - extra: Some("scratch.img"), - ..Emulation::capture() - }, - ErrorKind::Io, - "manifest.bin, memory.bin, scratch.img, state.bin, not exactly the manifest", - ), - ( - Emulation { - state_len: Some(ManifestSpec::template().state_size - 512), - ..Emulation::capture() - }, - ErrorKind::Io, - "bytes that its manifest records", - ), - ]; - for (emulation, kind, message) in failures { - let error = bench.refresh(&emulation).unwrap_err(); - assert_eq!(error.kind(), kind, "{error}"); - assert!(error.message().contains(message), "{error}"); - assert_eq!(bench.runs.replace(0), 1); - assert_clean(&bench); - // The slot keeps the outcome report of the latest capture, and none of an earlier one. - assert_eq!( - bench.slot().join(OUTCOME_NAME).exists(), - emulation.outcome.is_some() - ); - } - assert!(bench.indexed().is_empty()); -} - -#[test] -fn what_openvmm_leaves_beside_the_snapshot_is_removed() { - let bench = Bench::new(); - let emulation = Emulation { - leftovers: true, - ..Emulation::capture() - }; - let generation = bench.refresh(&emulation).unwrap(); - assert_eq!( - sorted(&bench.store.payload_dir(generation.generation())), - [CAPTURE_RECORD_NAME, KEY_NAME, SNAPSHOT_NAME] - ); - assert_clean(&bench); -} - -#[test] -fn checked_transactions_outlive_failures_until_recovery_publishes_them() { - let bench = Bench::new(); - bench.interrupted(Point::Checked, false); - assert_eq!( - sorted(&bench.transaction()), - [CAPTURE_RECORD_NAME, KEY_NAME, SNAPSHOT_NAME] - ); - assert!(bench.indexed().is_empty()); - let published = bench.recover().unwrap(); - assert_eq!(published.len(), 1); - assert_eq!(bench.store.index().unwrap().generations(), published); - assert_clean(&bench); - // The next capture of the class publishes it too, and then serves it. - let bench = Bench::new(); - bench.interrupted(Point::Checked, false); - let Created::Existing(generation) = bench.create(&Emulation::capture()).unwrap() else { - panic!("a creation serves the generation that it recovered"); - }; - assert_eq!(generation.generation(), store_generation(1)); - assert_eq!(bench.runs.get(), 1); - assert_clean(&bench); -} - -#[test] -fn publications_interrupted_once_they_renamed_their_envelope_are_completed() { - let bench = Bench::new(); - bench.interrupted(Point::Renamed, false); - assert!(bench.indexed().is_empty()); - assert_eq!( - sorted(&bench.store.payload_dir(store_generation(1))), - [CAPTURE_RECORD_NAME, KEY_NAME, SNAPSHOT_NAME] - ); - // The next capture completes it first. - let generation = bench.refresh(&Emulation::capture()).unwrap(); - assert_eq!(generation.generation(), store_generation(2)); - assert_eq!(bench.indexed(), [1, 2]); - // So does a recovery. - bench.interrupted(Point::Renamed, false); - let published = bench.recover().unwrap(); - assert_eq!(published.len(), 1); - assert_eq!(published[0].generation(), store_generation(3)); - assert_eq!(bench.indexed(), [1, 2, 3]); - // A recovery that completes one before it publishes a checked transaction returns both. - bench.interrupted(Point::Checked, false); - bench.unindex_last(); - assert_eq!(bench.indexed(), [1, 2]); - assert_eq!(numbers(&bench.recover().unwrap()), [3, 4]); - assert_eq!(bench.indexed(), [1, 2, 3, 4]); - assert_clean(&bench); -} - -#[test] -fn recoveries_make_what_they_publish_durable_before_they_index_it() { - let bench = Bench::new(); - bench.refresh(&Emulation::capture()).unwrap(); - bench.interrupted(Point::Checked, false); - bench.unindex_last(); - // Each directory that the recovery syncs, with what the index lists as it does. - let synced = RefCell::new(Vec::new()); - let sync = |dir: &Path| -> Result<()> { - synced - .borrow_mut() - .push((dir.to_path_buf(), bench.indexed())); - sync_dir(dir) - }; - assert_eq!(numbers(&bench.recover_syncing(&sync).unwrap()), [1, 2]); - let generations = bench.store.dir.join(GENERATIONS_NAME); - let parent = bench.store.dir.parent().unwrap().to_path_buf(); - assert_eq!( - synced.into_inner(), - [ - // The transaction that the crash left checked. - (bench.transaction(), vec![]), - // The publication that the crash interrupted once it had renamed its envelope. - (bench.store.payload_dir(store_generation(1)), vec![]), - (generations.clone(), vec![]), - (bench.slot(), vec![]), - // The transaction's publication. - (generations, vec![]), - (bench.slot(), vec![]), - (bench.store.dir.clone(), vec![1, 2]), - (parent, vec![1, 2]), - ] - ); - assert_clean(&bench); -} - -#[test] -fn captures_make_the_store_durable_before_they_report_a_generation() { - let bench = Bench::new(); - // Each directory that the capture syncs, with what the index lists as it does. - let synced = RefCell::new(Vec::new()); - let sync = |dir: &Path| -> Result<()> { - synced - .borrow_mut() - .push((dir.to_path_buf(), bench.indexed())); - sync_dir(dir) - }; - let run = |locked: &mut LockedStore, vm: &Vm<'_>| Emulation::capture().run(locked, vm); - let hooks = Hooks { - sync: &sync, - run: &run, - interrupt: &uninterrupted, - crash: false, - }; - let created = bench - .store - .capture(&bench.capture, None, deadline(), &hooks) - .unwrap(); - assert_eq!(created.generation().generation(), store_generation(1)); - let parent = bench.store.dir.parent().unwrap().to_path_buf(); - assert_eq!( - synced.into_inner(), - [ - // The checked transaction, and every directory above it, up to the store's parent, - // which the store's creation did not sync. - (bench.transaction(), vec![]), - (bench.slot(), vec![]), - (bench.store.captures_dir(), vec![]), - (bench.store.dir.clone(), vec![]), - (parent.clone(), vec![]), - // The rename into the store. - (bench.store.dir.join(GENERATIONS_NAME), vec![]), - (bench.slot(), vec![]), - // The index. - (bench.store.dir.clone(), vec![1]), - (parent, vec![1]), - ] - ); -} - -#[test] -fn creations_recover_the_index_before_they_look_for_a_generation() { - // A creation completes a publication that a crash interrupted, and serves its generation, - // the class's latest. - let bench = Bench::new(); - bench.refresh(&Emulation::capture()).unwrap(); - bench.interrupted(Point::Renamed, false); - assert_eq!(bench.indexed(), [1]); - let Created::Existing(generation) = bench.create(&Emulation::capture()).unwrap() else { - panic!("a creation serves the generation whose publication it completed"); - }; - assert_eq!(generation.generation(), store_generation(2)); - assert_eq!(bench.indexed(), [1, 2]); - assert_eq!(bench.runs.get(), 2); - // It serves no generation whose payload a crash lost, and captures one in its place. - let bench = Bench::new(); - bench.refresh(&Emulation::capture()).unwrap(); - fs::remove_dir_all(bench.store.payload_dir(store_generation(1))).unwrap(); - let Created::Captured(generation) = bench.create(&Emulation::capture()).unwrap() else { - panic!("a creation captures in place of a generation whose payload was lost"); - }; - assert_eq!(generation.generation(), store_generation(2)); - assert_eq!(bench.indexed(), [2]); - // A store whose index is full of generations whose payloads a crash lost still captures. - let bench = Bench::new(); - let first = bench.refresh(&Emulation::capture()).unwrap(); - let last = MAX_GENERATIONS as u64; - let mut index = bench.store.index().unwrap(); - index - .generations - .extend((2..=last).map(|generation| Generation { - generation: store_generation(generation), - ..first.clone() - })); - index.next = store_generation(last + 1); - bench.store.write_index(&index).unwrap(); - assert_eq!(bench.indexed().len(), MAX_GENERATIONS); - let generation = bench.refresh(&Emulation::capture()).unwrap(); - assert_eq!(generation.generation(), store_generation(last + 1)); - assert_eq!(bench.indexed(), [1, last + 1]); - assert_clean(&bench); -} - -#[test] -fn recoveries_drop_lost_generations_before_they_index_another() { - // A store whose index is full of generations that a crash lost, but one, beside a publication - // that a crash interrupted once it had renamed its envelope. - let bench = Bench::new(); - let first = bench.refresh(&Emulation::capture()).unwrap(); - bench.interrupted(Point::Renamed, false); - let last = MAX_GENERATIONS as u64; - fs::rename( - bench.store.payload_dir(store_generation(2)), - bench.store.payload_dir(store_generation(last + 1)), - ) - .unwrap(); - let mut index = bench.store.index().unwrap(); - index - .generations - .extend((2..=last).map(|generation| Generation { - generation: store_generation(generation), - ..first.clone() - })); - index.next = store_generation(last + 1); - bench.store.write_index(&index).unwrap(); - assert_eq!(bench.indexed().len(), MAX_GENERATIONS); - // The lost generations take none of the store's capacity from the publication. - assert_eq!(numbers(&bench.recover().unwrap()), [last + 1]); - assert_eq!(bench.indexed(), [1, last + 1]); - assert_clean(&bench); -} - -#[test] -fn creations_make_the_store_durable_before_they_serve_a_generation() { - let bench = Bench::new(); - let parent = bench.store.dir.parent().unwrap().to_path_buf(); - let run = |locked: &mut LockedStore, vm: &Vm<'_>| Emulation::capture().run(locked, vm); - // A capture whose host crashes as it syncs the store's parent, once the index lists the - // capture's generation. - let crashing = |dir: &Path| -> Result<()> { - if dir == parent && !bench.indexed().is_empty() { - return Err(Error::other("the host crashed")); - } - sync_dir(dir) - }; - let hooks = Hooks { - sync: &crashing, - run: &run, - interrupt: &uninterrupted, - crash: true, - }; - bench - .store - .capture(&bench.capture, None, deadline(), &hooks) - .unwrap_err(); - assert_eq!(bench.indexed(), [1]); - // A creation serves the generation once it has made the index, and the store's own entry in - // its parent, durable, although no recovery changed the index. - let synced = RefCell::new(Vec::new()); - let sync = |dir: &Path| -> Result<()> { - synced.borrow_mut().push(dir.to_path_buf()); - sync_dir(dir) - }; - let hooks = Hooks { - sync: &sync, - run: &run, - interrupt: &uninterrupted, - crash: false, - }; - let now = || -> Result { Ok(after(1)) }; - let Created::Existing(generation) = bench - .store - .capture(&bench.capture, Some(&now), deadline(), &hooks) - .unwrap() - else { - panic!("a creation serves the generation that the index lists"); - }; - assert_eq!(generation.generation(), store_generation(1)); - assert_eq!(synced.into_inner(), [bench.store.dir.clone(), parent]); - assert_clean(&bench); -} - -#[test] -fn recovery_ends_the_captures_that_crashes_interrupt() { - for (point, left, indexed) in [ - ( - Point::Prepared, - vec![CAPTURE_RECORD_NAME, LOCK_NAME, SCRATCH_NAME], - vec![], - ), - ( - Point::Exited, - vec![CAPTURE_RECORD_NAME, LOCK_NAME, SCRATCH_NAME, SNAPSHOT_NAME], - vec![], - ), - ( - Point::Checked, - vec![CAPTURE_RECORD_NAME, KEY_NAME, SNAPSHOT_NAME], - vec![1], - ), - (Point::Renamed, vec![], vec![1]), - (Point::Indexed, vec![], vec![1]), - ] { - let bench = Bench::new(); - bench.interrupted(point, true); - if left.is_empty() { - assert!(!bench.transaction().exists(), "{point:?}"); - } else { - assert_eq!(sorted(&bench.transaction()), left, "{point:?}"); - } - let published = bench.recover().unwrap(); - let recovered: Vec = published - .iter() - .map(|generation| generation.generation().get().get()) - .collect(); - let expected = if matches!(point, Point::Checked | Point::Renamed) { - vec![1] - } else { - vec![] - }; - assert_eq!(recovered, expected, "{point:?}"); - assert_eq!(bench.indexed(), indexed, "{point:?}"); - assert_clean(&bench); - // The slot captures again. - bench.refresh(&Emulation::capture()).unwrap(); - } -} - -#[test] -fn generations_whose_payloads_are_lost_are_dropped_from_the_index() { - let bench = Bench::new(); - let first = bench.refresh(&Emulation::capture()).unwrap(); - bench.refresh(&Emulation::capture()).unwrap(); - fs::remove_dir_all(bench.store.payload_dir(first.generation())).unwrap(); - bench.refresh(&Emulation::capture()).unwrap(); - assert_eq!(bench.indexed(), [2, 3]); - fs::remove_dir_all(bench.store.payload_dir(store_generation(3))).unwrap(); - assert!(bench.recover().unwrap().is_empty()); - assert_eq!(bench.indexed(), [2]); - // The index never assigns a store generation twice. - assert_eq!( - bench.store.index().unwrap().next_generation(), - store_generation(4) - ); -} - -#[test] -fn a_transaction_stays_while_it_records_a_launch() { - let bench = Bench::new(); - // A source that fails while its launch remains, as one whose VMM may still run: dropping a - // launch leaves its journal, as a crash would. - let failed = |locked: &mut LockedStore, vm: &Vm<'_>| -> Result<()> { - let capability = [7; crate::protocol::CAPABILITY_LEN]; - let launch = LaunchId::generate()?; - drop(locked.begin_launch(&launch, &vm.capture.class.runtime, &capability)?); - Err(Error::other("the VMM may still run")) - }; - let error = bench - .serve_with(None, &failed, &uninterrupted, false) - .unwrap_err(); - assert_eq!(error.message(), "the VMM may still run"); - // The transaction's sandbox store records the launch, and the slot none. - assert!(bench.transaction().join(JOURNAL_NAME).exists()); - assert!(!bench.slot().join(JOURNAL_NAME).exists()); - let left = sorted(&bench.transaction()); - for name in [CAPTURE_RECORD_NAME, SCRATCH_NAME, LOCK_NAME, LOG_NAME] { - assert!(left.iter().any(|left| left == name), "{name}: {left:?}"); - } - // The next capture ends the launch, which nothing holds, retires the transaction, and - // captures. - bench.refresh(&Emulation::capture()).unwrap(); - assert_clean(&bench); - assert_eq!(bench.indexed(), [1]); -} - -/// Returns whether this host lets the tests create a symbolic link to a file: Windows lets only -/// Developer Mode or a privilege create one. -fn can_link_files() -> bool { - let directory = tempfile::tempdir().unwrap(); - let target = directory.path().join("target"); - fs::write(&target, b"").unwrap(); - link_file(&target, &directory.path().join("link")) -} - -/// Asserts that `result` failed with `kind`, with a message that holds `message`, and that its -/// capture ran no VM, and left no transaction. -fn assert_refused(bench: &Bench, result: Result, kind: ErrorKind, message: &str) { - let error = result.unwrap_err(); - assert_eq!(error.kind(), kind, "{error}"); - assert!( - error.message().contains(message) || cause(&error).contains(message), - "{error}: {}", - cause(&error) - ); - assert_eq!(bench.runs.replace(0), 0); - assert!(bench.indexed().is_empty()); -} - -#[test] -fn links_and_unsafe_ownership_are_refused_before_the_vm_starts() { - // A placeholder that is a link, which OpenVMM would not follow, though the check of its - // binding follows it. - if can_link_files() { - let bench = Bench::with(|image| { - let link = image.with_file_name("link.erofs"); - assert!(link_file(image, &link)); - link - }); - let result = bench.create(&Emulation::capture()); - assert_refused( - &bench, - result, - ErrorKind::Unavailable, - "it is a link, or not a regular file", - ); - assert_clean(&bench); - } - // A placeholder that others may change. - let bench = Bench::with(|image| { - let_anyone_write(image); - image.to_path_buf() - }); - let result = bench.create(&Emulation::capture()); - let message = if cfg!(windows) { - "allows other users to change it" - } else { - "is writable by other users" - }; - assert_refused(&bench, result, ErrorKind::Unavailable, message); - assert_clean(&bench); - // A scratch that a link replaced, or that others may change. - let bench = Bench::new(); - let scratch = bench.transaction().join(SCRATCH_NAME); - let replaced = |point: Point| -> Result<()> { - if point == Point::Prepared { - let target = bench.slot().join("elsewhere.raw"); - sparse(&target, class().scratch.length); - fs::remove_file(&scratch).unwrap(); - assert!(link_file(&target, &scratch)); - } - Ok(()) - }; - if can_link_files() { - let result = bench.serve(None, &Emulation::capture(), &replaced, false); - assert_refused( - &bench, - result, - ErrorKind::Io, - "it is a link, or not a regular file", - ); - assert_clean(&bench); - } - let writable = |point: Point| -> Result<()> { - if point == Point::Prepared { - let_anyone_write(&scratch); - } - Ok(()) - }; - let result = bench.serve(None, &Emulation::capture(), &writable, false); - assert_refused(&bench, result, ErrorKind::Io, message); - assert_clean(&bench); - // A transaction that others may access, which the capture keeps rather than retire. - let exposed_message = if cfg!(windows) { - "allows other users" - } else { - "is accessible to other users" - }; - let transaction = bench.transaction(); - let exposed = |point: Point| -> Result<()> { - if point == Point::Prepared { - let_anyone_write(&transaction); - } - Ok(()) - }; - let result = bench.serve(None, &Emulation::capture(), &exposed, false); - assert_refused(&bench, result, ErrorKind::Io, exposed_message); - assert!(transaction.exists()); - let result = bench.create(&Emulation::capture()); - assert_refused(&bench, result, ErrorKind::Io, exposed_message); - fs::remove_dir_all(&transaction).unwrap(); - // A transaction that is a link. - let elsewhere = bench.slot().join("elsewhere"); - private_dir(&elsewhere); - link_dir(&elsewhere, &transaction); - let result = bench.create(&Emulation::capture()); - assert_refused(&bench, result, ErrorKind::Io, "is not a plain directory"); - assert!(fs::symlink_metadata(&transaction).is_ok()); -} - -#[test] -fn scratches_are_the_files_that_their_captures_created() { - // Another file of the scratch's length, renamed over the scratch. It is created while the - // scratch exists, so that the host cannot give it the scratch's identity again. - let bench = Bench::new(); - let scratch = bench.transaction().join(SCRATCH_NAME); - let replaced = |point: Point| -> Result<()> { - if point == Point::Prepared { - let other = bench.slot().join("other.raw"); - sparse(&other, class().scratch.length); - fs::rename(&other, &scratch).unwrap(); - } - Ok(()) - }; - let result = bench.serve(None, &Emulation::capture(), &replaced, false); - assert_refused( - &bench, - result, - ErrorKind::Io, - "is not the file that the capture created", - ); - assert_clean(&bench); - // A file whose ID differs from the scratch's only above its low 64 bits, as the 128-bit IDs - // of two files of a file system such as ReFS may: the check compares the whole ID. - let inputs = Inputs::check(&bench.capture).unwrap(); - let template = TemplatePlan::new( - &bench.capture.adapter, - &bench.capture.binding, - bench.capture.class.scratch.length, - inputs.resources, - &bench.transaction(), - StorageGeneration::generate().unwrap(), - ) - .unwrap(); - private_dir(&bench.transaction()); - let created = create_scratch(template.scratch_path(), template.scratch_length()).unwrap(); - check_scratch(&template, created).unwrap(); - let (device, file_id) = created; - let error = check_scratch(&template, (device, file_id ^ (1 << 64))).unwrap_err(); - assert!( - error - .message() - .contains("is not the file that the capture created"), - "{error}" - ); -} - -/// Returns whether this process holds the file at `path` open for writing. -#[cfg(target_os = "linux")] -fn held_for_writing(path: &Path) -> bool { - let target = fs::canonicalize(path).unwrap(); - fs::read_dir("/proc/self/fd") - .unwrap() - .filter_map(std::result::Result::ok) - .any(|entry| { - let flags = || { - let info = - fs::read_to_string(Path::new("/proc/self/fdinfo").join(entry.file_name())) - .ok()?; - let flags = info.lines().find_map(|line| line.strip_prefix("flags:"))?; - u32::from_str_radix(flags.trim(), 8).ok() - }; - fs::read_link(entry.path()).is_ok_and(|linked| linked == target) - && flags().is_some_and(|flags| flags & 0o3 != 0) - }) -} - -/// Opens the file at `path` as OpenVMM opens a block device of a capture on Windows: without -/// following a reparse point, for reading, and for writing if `write`, sharing reads and -/// deletion alone, which fails while another handle may write it. -#[cfg(windows)] -fn open_as_openvmm(path: &Path, write: bool) -> io::Result { - use std::os::windows::fs::OpenOptionsExt; - - /// `FILE_SHARE_READ | FILE_SHARE_DELETE`. - const SHARE_READ_DELETE: u32 = 0x1 | 0x4; - /// `FILE_FLAG_OPEN_REPARSE_POINT`. - const OPEN_REPARSE_POINT: u32 = 0x0020_0000; - OpenOptions::new() - .read(true) - .write(write) - .share_mode(SHARE_READ_DELETE) - .custom_flags(OPEN_REPARSE_POINT) - .open(path) -} - -#[test] -fn no_handle_through_which_the_host_could_write_the_block_devices_is_open_as_the_vm_starts() { - let bench = Bench::new(); - let checked = |locked: &mut LockedStore, vm: &Vm<'_>| -> Result<()> { - #[cfg(any(target_os = "linux", windows))] - let (placeholder, scratch) = ( - vm.template.plan().distro().path(), - vm.template.scratch_path(), - ); - #[cfg(target_os = "linux")] - { - assert!(!held_for_writing(placeholder)); - assert!(!held_for_writing(scratch)); - } - #[cfg(windows)] - { - open_as_openvmm(placeholder, false).unwrap(); - open_as_openvmm(scratch, true).unwrap(); - // The checked placeholder stays held, so that nothing writes it meanwhile. - let error = OpenOptions::new() - .write(true) - .open(placeholder) - .unwrap_err(); - assert_eq!(error.raw_os_error(), Some(32), "{error}"); - } - Emulation::capture().run(locked, vm) - }; - bench - .serve_with(None, &checked, &uninterrupted, false) - .unwrap(); - assert_eq!(bench.runs.get(), 1); - // The placeholder is released once the VM has ended. - OpenOptions::new() - .write(true) - .open(bench.capture.adapter.distro().path()) - .unwrap(); -} - -#[test] -fn captures_of_one_class_are_serialized() { - let active = AtomicUsize::new(0); - let most = AtomicUsize::new(0); - let runs = AtomicUsize::new(0); - let run = |locked: &mut LockedStore, vm: &Vm<'_>| -> Result<()> { - let now = active.fetch_add(1, Ordering::SeqCst) + 1; - most.fetch_max(now, Ordering::SeqCst); - runs.fetch_add(1, Ordering::SeqCst); - thread::sleep(Duration::from_millis(100)); - let ran = Emulation::capture().run(locked, vm); - active.fetch_sub(1, Ordering::SeqCst); - ran - }; - // Serves the capture of `bench` twice at once, as creations at the host's time that `now` - // returns, or as refreshes. - let race = - |bench: &Bench, now: Option<&(dyn Fn() -> Result + Sync)>| -> Vec { - let (store, capture) = (&bench.store, &bench.capture); - let barrier = Arc::new(Barrier::new(2)); - let serve = || { - let hooks = Hooks { - sync: &sync_dir, - run: &run, - interrupt: &uninterrupted, - crash: false, - }; - let now: Option<&dyn Fn() -> Result> = match now { - Some(now) => Some(now), - None => None, - }; - barrier.wait(); - store.capture(capture, now, deadline(), &hooks).unwrap() - }; - thread::scope(|scope| { - let threads: Vec<_> = (0..2).map(|_| scope.spawn(serve)).collect(); - threads - .into_iter() - .map(|thread| thread.join().unwrap()) - .collect() - }) - }; - // Refreshes capture one after the other. - let bench = Bench::new(); - let refreshed = race(&bench, None); - assert!( - refreshed - .iter() - .all(|created| matches!(created, Created::Captured(_))) - ); - assert_eq!(runs.swap(0, Ordering::SeqCst), 2); - assert_eq!(bench.indexed(), [1, 2]); - // Creations that race make one generation between them, although both start before the - // capture's anchor: the host's time is before it until a capture is published. - let bench = Bench::new(); - let store = &bench.store; - let now = || -> Result { - Ok(if store.index()?.generations().is_empty() { - before(1) - } else { - after(1) - }) - }; - assert_eq!(now().unwrap(), before(1)); - let created = race(&bench, Some(&now)); - assert_eq!(runs.load(Ordering::SeqCst), 1); - assert_eq!(created[0].generation(), created[1].generation()); - assert_eq!( - created - .iter() - .filter(|created| matches!(created, Created::Captured(_))) - .count(), - 1 - ); - assert_eq!(most.load(Ordering::SeqCst), 1); -} - -#[test] -fn recoveries_refuse_entries_of_the_captures_that_are_not_slots() { - let bench = Bench::new(); - assert!(bench.recover().unwrap().is_empty()); - bench.refresh(&Emulation::capture()).unwrap(); - fs::write(bench.store.captures_dir().join("notes.txt"), b"").unwrap(); - let error = bench.recover().unwrap_err(); - assert_eq!(error.code(), ErrorCode::ForeignFiles, "{error}"); - assert!(error.message().ends_with(": notes.txt"), "{error}"); -} - -#[cfg(target_os = "linux")] -#[test] -fn stores_too_deep_for_the_control_socket_of_a_capture_are_refused() { - let bench = Bench::new(); - let parent = bench._directory.path().join("d".repeat(80)); - fs::create_dir(&parent).unwrap(); - let bench = Bench { - store: Store::create(parent.join("templates")).unwrap(), - ..bench - }; - let error = bench.refresh(&Emulation::capture()).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); - assert!( - error.message().contains("too deep for the control socket"), - "{error}" - ); - assert_eq!(bench.runs.get(), 0); -} - -#[cfg(windows)] -#[test] -fn working_directory_limits_are_checked_before_captures() { - use std::os::windows::ffi::OsStrExt; - - let bench = Bench::new(); - let mut parent = bench._directory.path().to_path_buf(); - while parent.as_os_str().encode_wide().count() < 300 { - parent.push("long-state-root-component"); - } - fs::create_dir_all(&parent).unwrap(); - let bench = Bench { - store: Store::create(parent.join("templates")).unwrap(), - ..bench - }; - let error = bench.refresh(&Emulation::capture()).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported); - assert!(error.message().contains("working directory")); - assert!(error.message().contains("258")); - assert_eq!(bench.runs.get(), 0); - assert!(!bench.store.captures_dir().exists()); -} diff --git a/nvx-rs/src/host/template/store/leases.rs b/nvx-rs/src/host/template/store/leases.rs deleted file mode 100644 index 45e249530..000000000 --- a/nvx-rs/src/host/template/store/leases.rs +++ /dev/null @@ -1,1075 +0,0 @@ -//! The leases of a store's generations, the bindings of clones to them, the tombstones that -//! quarantine generations, and the retirement of generations whose leases have drained; see -//! [the store's documentation](Store#leases-tombstones-and-retirement). - -#[cfg(test)] -mod tests; - -use std::fs; -use std::io; -use std::num::NonZeroU64; -use std::path::{Path, PathBuf}; -use std::sync::{Mutex, MutexGuard, PoisonError}; -use std::time::Instant; - -use serde::{Deserialize, Serialize}; - -use super::{ - GENERATIONS_NAME, Generation, STORE_FORMAT, Store, StoreId, entries, random_id, read_record, - refuse_problem, store_error, take_lock, write_record, -}; -use crate::activation::{Identity, StoreGeneration}; -use crate::error::{Error, ErrorCode, ErrorKind, Result}; -use crate::host::filesystem; -use crate::host::platform; -use crate::host::store::files::{remove_if_present, sync_dir}; -use crate::host::store::lease::{ - Lease, LeaseId, LeaseRecord, MAX_LEASE_RECORD_LEN, clone_socket_name, open_payload, - read_binding, read_binding_in, read_lease, -}; -use crate::host::store::regular::{open_regular_in, was_removed}; -use crate::host::store::{ - CLONE_BINDING_NAME, CallerFiles, JOURNAL_NAME, LaunchId, LockedStore, RUNTIME_NAME, - Reservation, SandboxStore, -}; -use crate::host::template::eligibility::HostTime; -use crate::host::template::key::KeyIdentity; - -/// Name of the directory of the records of the leases that a store granted. -pub const LEASES_NAME: &str = "leases"; -/// Name of the directory of the tombstones of a store's generations. -pub const TOMBSTONES_NAME: &str = "tombstones"; - -/// Syncs a directory, as [`sync_dir`] does. The transactions of leases take it as a parameter, so -/// that tests observe the order of their syncs, and fail them. -type SyncDir<'a> = &'a dyn Fn(&Path) -> Result<()>; - -/// The directories of leases and tombstones whose entries in their stores this process made -/// durable, by the directory and the identity of the store, and the name of the directory. A store -/// creates each once, and never removes it, so one sync of the store's directory, and of its -/// parent, once a process sees one, keeps its entry, and the store's, durable, even if the -/// process that created them did not finish its own syncs. -static DURABLE_RECORD_DIRS: Mutex> = Mutex::new(Vec::new()); - -/// An entry of [`DURABLE_RECORD_DIRS`]: the directory and the identity of a store, and the name -/// of one of its directories. -type DurableRecordDir = (PathBuf, StoreId, &'static str); - -/// Returns [`DURABLE_RECORD_DIRS`], locked. -fn durable_record_dirs() -> MutexGuard<'static, Vec> { - DURABLE_RECORD_DIRS - .lock() - .unwrap_or_else(PoisonError::into_inner) -} - -/// The directory of a store's leases, in messages. -const LEASES_WHAT: &str = "the directory of the leases of the template store"; -/// The directory of a store's tombstones, in messages. -const TOMBSTONES_WHAT: &str = "the directory of the tombstones of the template store"; -/// The record of a lease in its store, in messages. -const LEASE_DESCRIPTION: &str = "lease of the template store"; -/// A tombstone, in messages. -const TOMBSTONE_DESCRIPTION: &str = "tombstone of the template store"; - -// A lease is the sandbox's store's, which reads it from the clone binding, and which holds its key -// as the identity by which the clone's activation record names it: the key's identity is a type of -// this module. -impl Lease { - /// Returns the identity of the leased generation's key. - pub fn key(&self) -> KeyIdentity { - KeyIdentity::from(self.template().key) - } -} - -/// A lease that [`Store::acquire`] granted, with the lock of the sandbox that holds it. -#[derive(Debug)] -pub struct Acquired { - /// The lease. - pub lease: Lease, - /// The lock of the sandbox, which names it by the lease's path, the only one under which the - /// caller launches its clone, as the launch that the lease reserves. Dropping it releases the - /// lock, and keeps the lease, whose launch no later lock begins: a release ends the lease, and - /// another acquisition reserves another launch. - pub locked: LockedStore, -} - -/// What [`Store::retire`] found of a generation. -#[derive(Debug, Clone, PartialEq, Eq)] -pub enum Retirement { - /// The generation is gone, with its payload, its entry in the index, and its tombstone, or was - /// gone already. - Retired, - /// A tombstone quarantines the generation, which keeps its payload and its entry in the index - /// for its leases, which this lists in the order of their identities, until their clones - /// release them. - Draining(Vec), -} - -impl Store { - /// Selects the generation of the key `key` from which a clone restores at the host's time - /// `now`: the latest that the store indexes, that no tombstone quarantines, and that is - /// eligible at `now`, as [`Generation::eligibility`] decides it, if one is. - /// - /// The selection reads the index and the tombstones without the store's lock, as - /// [`index`](Self::index) does, so a selected generation may be tombstoned or retired before - /// the caller leases it, which [`acquire`](Self::acquire) refuses. - pub fn select(&self, key: KeyIdentity, now: &HostTime) -> Result> { - let index = self.index()?; - let tombstoned = self.tombstones()?; - Ok(index - .generations - .iter() - .rev() - .find(|generation| { - generation.key == key - && !tombstoned.contains(&generation.generation) - && generation.eligibility(now).is_eligible() - }) - .cloned()) - } - - /// Leases the generation `generation` to the clone of the sandbox in the directory `sandbox`, - /// whose caller keeps `files` there, at the host's time `now`, as [`HostTime::now`] samples - /// it, waiting for the store's lock and the sandbox's until `deadline`, or for as long as it - /// takes without one. - /// - /// In one transaction, under the store's lock and then the sandbox's - /// [lifecycle lock](crate::host::store#lifecycle-locks), it checks that the store indexes the - /// generation, that no tombstone quarantines it, that it is eligible at `now`, as - /// [`Generation::eligibility`] decides it, that its payload, on Linux, is one that a launch of - /// the clone and a release of its lease can use, and that the sandbox is provisioned, records - /// no VM and no launch, and holds no lease; and it writes the lease's record in the store, and - /// then the same record as the sandbox's clone binding, beside its launch journal, each - /// durable before the next step, so that no binding outlasts, through a crash of the host, the - /// record that keeps its generation in the store. It resolves the links of the sandbox's path - /// before it takes the sandbox's lock, and the lease, the binding, and the lock that it - /// returns all name the sandbox by that path, as [`Lease::sandbox`] returns it, which no later - /// change of a link redirects to another directory. It returns the lease, with the sandbox's - /// lock, the only one under which the caller launches the clone, once, as the launch that the - /// lease reserves, [`Lease::launch`], and releases the store's lock. - /// - /// The launch that the lease reserves restores the generation's snapshot in place, and its VMM - /// listens beside the snapshot, as OpenVMM requires of a restore on Linux: at the control - /// socket `.sock`, in the generation's payload, after the launch's identity, rather - /// than at the sandbox's own socket. So a payload holds the sockets of the clones of its - /// generation, each named after a launch, while they lease it, and none once its leases have - /// drained: the VMM removes its socket as it exits, a stop or a recovery of the sandbox - /// removes the one that an abrupt exit leaves, and a [release](Self::release) makes the - /// socket's absence durable before the lease ends. On Windows, the VMM listens at a named - /// pipe, as every launch does. - /// - /// Fails, and leases nothing: - /// - /// - with [`ErrorKind::NotFound`] if the store does not index the generation; - /// - with [`ErrorCode::TemplateTombstoned`] if a tombstone quarantines it; - /// - with [`ErrorCode::TemplateIneligible`] if it is not eligible at `now`; - /// - on Linux, with [`ErrorKind::Io`] if the generation's payload is not a directory of the - /// current user with mode 0700, which a launch of the clone requires, as OpenVMM does of the - /// directory of a control socket, or if a release could not open it as it opens it, - /// following no link at the store's directory, at the directory of the generations, or at - /// the payload: a release opens the payload to remove the clone's socket from it and make - /// its absence durable; - /// - with [`ErrorCode::SandboxNotProvisioned`] if the directory does not exist, or holds no - /// configuration record of the caller, as `files` declares it; - /// - with [`ErrorCode::SandboxRunning`] if the sandbox records a VM, or a launch, which may - /// still run; - /// - with [`ErrorCode::SandboxLeased`] if the sandbox holds a lease already; - /// - with [`ErrorKind::Unsupported`] if the sandbox's path, with its links resolved, is not - /// valid Unicode, which its record cannot hold, or, on Windows, is a network path, or one - /// that names the directory only in its verbatim form; - /// - with [`ErrorKind::TimedOut`] if the deadline passes before it takes the store's lock, and - /// as [`SandboxStore::lock_existing`] fails before it takes the sandbox's. - /// - /// A failure to write the binding, or to make it durable, leases nothing either, unless the - /// binding's removal cannot be made durable: the lease's record then stays, and the error says - /// that the sandbox may hold the lease still, which a [release](Self::release) ends. - pub fn acquire( - &self, - generation: StoreGeneration, - sandbox: impl Into, - files: &CallerFiles, - now: &HostTime, - deadline: Option, - ) -> Result { - self.acquire_with(generation, sandbox.into(), files, now, deadline, &sync_dir) - } - - /// Leases a generation as [`acquire`](Self::acquire) does, making its records durable with - /// `sync`. - fn acquire_with( - &self, - generation: StoreGeneration, - sandbox: PathBuf, - files: &CallerFiles, - now: &HostTime, - deadline: Option, - sync: SyncDir<'_>, - ) -> Result { - let lock = take_lock(&self.dir, deadline)?; - let index = self.index()?; - let indexed = index - .get(generation) - .ok_or_else(|| self.not_indexed(generation))?; - if self.is_tombstoned(generation)? { - return Err(Error::new( - ErrorCode::TemplateTombstoned, - format!( - "a tombstone quarantines the generation {generation} of the template store \ - {}, which takes no new clones", - self.dir.display() - ), - )); - } - let eligibility = indexed.eligibility(now); - if !eligibility.is_eligible() { - return Err(Error::new( - ErrorCode::TemplateIneligible, - format!( - "the generation {generation} of the template store {} is {eligibility}", - self.dir.display() - ), - )); - } - // A release removes the socket of the clone's VMM through a handle of the payload, which - // it opens for reading, following no link from the store's directory on, so that it can - // make the removal durable: a payload that it could not open, or remove an entry from, - // would keep the lease and its binding past it. - #[cfg(target_os = "linux")] - self.check_payload(generation)?; - let not_provisioned = || { - Error::new( - ErrorCode::SandboxNotProvisioned, - format!("no sandbox is provisioned in {}", sandbox.display()), - ) - }; - // The lease, the binding, and the lock name the sandbox by its path with its links - // resolved, which no later change of a link redirects to another directory. - let resolved = match filesystem::canonicalize(&sandbox) { - Ok(resolved) => resolved, - Err(error) if error.kind() == io::ErrorKind::NotFound => { - return Err(not_provisioned().with_source(error)); - } - Err(error) => { - let message = format!( - "cannot resolve the path of the sandbox in {}", - sandbox.display() - ); - return Err(if error.kind() == io::ErrorKind::Unsupported { - Error::unsupported(message).with_source(error) - } else { - store_error(message)(error) - }); - } - }; - let mut locked = match SandboxStore::lock_existing(&resolved, files, deadline) { - Ok(locked) => locked, - Err(error) if error.kind() == ErrorKind::NotFound => { - return Err(not_provisioned().with_source(error)); - } - Err(error) => return Err(error), - }; - if let Some(config) = files.config() { - let path = locked.dir().join(config); - match fs::symlink_metadata(&path) { - Ok(_) => {} - Err(error) if error.kind() == io::ErrorKind::NotFound => { - return Err(not_provisioned()); - } - Err(error) => { - return Err(store_error(format!("cannot examine {}", path.display()))( - error, - )); - } - } - } - if locked.journal()?.is_some() || locked.runtime()?.is_some() { - return Err(may_run(locked.dir())); - } - if let Some(held) = read_binding(locked.dir())? { - return Err(Error::new( - ErrorCode::SandboxLeased, - format!( - "the sandbox in {} holds the lease {} of the generation {} of the template \ - store {} already", - locked.dir().display(), - held.id(), - held.generation(), - held.store() - ), - )); - } - if locked.dir().to_str().is_none() { - return Err(Error::unsupported(format!( - "the path of the sandbox {} is not valid Unicode, which the record of a lease \ - cannot hold", - locked.dir().display() - ))); - } - let lease = Lease::new( - LeaseId::from_bytes(random_id("a lease's identity")?), - self.id, - generation, - Identity::from(indexed.key), - locked.dir().to_owned(), - LaunchId::generate()?, - ); - let record = LeaseRecord::new(&lease); - self.create_records_dir(LEASES_NAME, LEASES_WHAT, sync)?; - let path = self.lease_path(lease.id()); - write_record(&path, &record, "the lease of the template store")?; - // A retirement deletes a generation that no record of the store leases, so the record - // must be durable before a binding names it. - if let Err(error) = sync(&self.dir.join(LEASES_NAME)) { - let _ = remove_if_present(&path); - return Err(error); - } - if let Err(error) = bind(locked.dir(), &record, sync) { - // The record goes only once the binding's removal is durable. Until then, it keeps - // the generation for a binding that may outlast a crash, and a retirement removes it - // once no binding names it. - if unbind(locked.dir(), sync).is_err() { - return Err(held_still(error, locked.dir(), lease.id())); - } - let _ = remove_if_present(&path); - return Err(error); - } - drop(lock); - locked.reserve(Some(Reservation { - socket: self.clone_socket_path(&lease), - launch: lease.launch().clone(), - })); - Ok(Acquired { lease, locked }) - } - - /// Returns the path of the Linux socket at which the VMM of the clone that holds `lease` - /// listens, its control endpoint: `.sock`, after the launch that the lease reserves, - /// in the payload of the leased generation, beside the snapshot that the VMM restores in - /// place, since OpenVMM requires a restore's control socket in the snapshot's parent. - /// - /// A payload holds the sockets of the clones of its generation only while they lease it: the - /// VMM removes its socket as it exits, a stop or a recovery removes what an abrupt exit - /// leaves, and a [release](Self::release) makes the socket's absence durable before the lease - /// ends. A [retirement](Self::retire) removes the payload with whatever a crash left in it. - pub(crate) fn clone_socket_path(&self, lease: &Lease) -> PathBuf { - self.payload_dir(lease.generation()) - .join(clone_socket_name(lease.launch())) - } - - /// Releases the lease that the sandbox of `locked` holds, and returns it, or returns `None` if - /// the sandbox holds none. - /// - /// A lease lasts until its clone's VM has exited: the release refuses with - /// [`ErrorCode::SandboxRunning`] while the sandbox records a VM or a launch, which may still - /// run. The sandbox's store clears those records only once it has verified that the VM's - /// process exited, through its recorded [`ProcessRef`](crate::host::process::ProcessRef), as - /// a stop or a [`reattach`](crate::host::supervisor::reattach) does. The release first ends - /// the reservation of the launch that `locked` holds, if it acquired the lease, so that one - /// that fails part of the way, once it removed files that a launch of the clone's plan needs, - /// leaves no launch that the lock may begin. It then removes what the clone kept of its - /// launch: the file of the scratch device that an earlier version created for a clone in RAM, - /// [`CLONE_SCRATCH_NAME`](crate::host::store::CLONE_SCRATCH_NAME), if one remains, the - /// clone's activation record, - /// [`CLONE_ACTIVATION_NAME`](crate::host::store::CLONE_ACTIVATION_NAME), the socket of its - /// restore-ready event, [`RESTORE_READY_NAME`](crate::host::store::RESTORE_READY_NAME), and - /// its records, which last as long as the lease, its progress log, - /// [`CLONE_PROGRESS_NAME`](crate::host::store::CLONE_PROGRESS_NAME), and the record of its - /// failure, [`CLONE_FAILURE_NAME`](crate::host::store::CLONE_FAILURE_NAME); - /// then, on Linux, the control socket of the clone's VMM, `.sock` in the leased - /// generation's payload, after the launch that the lease reserved, whose absence it makes - /// durable; then the clone binding, and once their removal is durable, the lease's record in - /// the store, so that an interrupted release leaves a record that no binding names, which a - /// [retirement](Self::retire) removes, and never a binding without its record. It takes no - /// lock of the store. - /// - /// The caller takes the sandbox's lock through the lease's path, as [`Lease::sandbox`] - /// returns it, and as [`acquire`](Self::acquire) takes it, with its links resolved. The - /// release names the entries of the sandbox's directory through one handle of the directory - /// that the path names, once it checked through that handle that the directory holds the - /// lock's file: through it, it checks that the sandbox records no VM and no launch, reads the - /// binding, removes the clone's files and the binding, and makes their absence durable. No - /// change of the path as the release runs, of a link in it or of one of its directories, then - /// makes it check or remove the files of another sandbox, whose lock it does not hold. - /// - /// Likewise, on Linux, the release opens the leased generation's payload before it removes - /// anything, through a handle of the store's directory: following no link at the store's - /// directory, at its directory of the generations, or at the payload, and once it checked - /// through that handle that the directory holds this store. It removes the clone's socket and - /// makes its absence durable through the payload's handle, so that neither a link that takes - /// the place of one of them nor a directory that takes the place of the store ever leads it to - /// remove the files of another directory. A directory that holds another store, or none, holds - /// no payload of this store's, so no socket goes. - /// - /// Fails with [`ErrorKind::Other`] if the sandbox holds a lease of another store, if the lock - /// does not name the sandbox by its lease's path, or if the directory that the path names does - /// not hold the lock's file, and with [`ErrorKind::Io`] if the store's directory, its - /// directory of the generations, or the payload is a link, or not a directory, and then - /// removes nothing. - pub fn release(&self, locked: &mut LockedStore) -> Result> { - self.release_with( - locked, - &|| {}, - &platform::Directory::sync, - &platform::Directory::sync, - ) - } - - /// Releases a lease as [`release`](Self::release) does, running `checked` once its checks - /// pass, before it removes anything, and making the removal of the clone's socket durable with - /// `sync_payload`, through the handle of the leased generation's payload, and that of the - /// binding with `sync_sandbox`, through the handle of the sandbox's directory. - fn release_with( - &self, - locked: &mut LockedStore, - checked: &dyn Fn(), - sync_payload: &dyn Fn(&platform::Directory) -> io::Result<()>, - sync_sandbox: &dyn Fn(&platform::Directory) -> io::Result<()>, - ) -> Result> { - let directory = locked.open_dir()?; - // A record whose deletion is pending, on Windows, is removed, as its readers take it. - for name in [JOURNAL_NAME, RUNTIME_NAME] { - let present = open_regular_in(&directory, name).map_err(store_error(format!( - "cannot examine {}", - locked.dir().join(name).display() - )))?; - if present.is_some() { - return Err(may_run(locked.dir())); - } - } - let Some(lease) = read_binding_in(&directory, locked.dir())? else { - return Ok(None); - }; - if lease.store() != self.id { - return Err(Error::other(format!( - "the sandbox in {} holds a lease of the template store {}, not of this one, {}", - locked.dir().display(), - lease.store(), - self.id - ))); - } - if locked.dir() != lease.sandbox() { - return Err(Error::other(format!( - "the lock of the sandbox in {} does not name it by the path of its lease, {}, \ - through which a release takes it", - locked.dir().display(), - lease.sandbox().display() - ))); - } - let recorded = self.check_records_dir(LEASES_NAME, LEASES_WHAT)?; - // The payload opens before anything goes, so that a link in its place refuses the release. - let payload = self.open_clone_payload(&lease)?; - checked(); - // A plan of the clone launches under the reservation that the lease's acquisition left in - // the lock, which the release ends before it removes anything, so that a release that - // fails once it removed the clone's files, such as the record of its failure, which ends - // the clone's launches until the release, leaves no launch that the lock may begin. - locked.reserve(None); - // The clone's VM has exited, and the sandbox launches nothing else under the lease, so no - // VM listens at the clone's control socket. The binding's removal makes the removal of the - // clone's files durable too, since - // all are entries of the sandbox's directory, but not that of the socket, an entry of the - // leased generation's payload, which must be durable before the lease ends, so that a - // generation whose leases have drained holds no clone's socket. - locked.remove_clone_files(&directory)?; - if let Some(payload) = &payload { - self.remove_clone_socket(&lease, payload, sync_payload)?; - } - let binding = locked.dir().join(CLONE_BINDING_NAME); - directory - .remove_file(CLONE_BINDING_NAME) - .map_err(store_error(format!("cannot remove {}", binding.display())))?; - // The binding's absence is durable before its record goes: a binding that outlasted the - // record through a crash would let a retirement delete the generation that it names. - sync_sandbox(&directory).map_err(store_error(format!( - "cannot sync the directory {}", - locked.dir().display() - )))?; - if recorded { - remove_lease_record(&self.lease_path(lease.id()))?; - } - Ok(Some(lease)) - } - - /// Refuses the payload of the generation `generation` unless a release of a clone's lease can - /// open it, as [`open_payload`] opens it, following no link at the store's directory, at the - /// directory of the generations, or at the payload, through a directory that holds this - /// store, and it is a directory of the current user with mode 0700. A launch of a clone - /// requires that mode, OpenVMM requiring it of the directory of a control socket, and so does - /// the release: it opens the payload for reading, and removes the clone's socket from it. - #[cfg(target_os = "linux")] - fn check_payload(&self, generation: StoreGeneration) -> Result<()> { - let payload = self.payload_dir(generation); - let problem = match open_payload(self.dir(), self.id, generation)? { - Some(directory) => directory.store_dir_problem().map_err(store_error(format!( - "cannot examine the payload of the generation {}", - payload.display() - )))?, - None => { - Some("is gone, or the store's directory holds another store, or none".to_owned()) - } - }; - match problem { - Some(problem) => Err(Error::io(format!( - "the payload of the generation {} {problem}", - payload.display() - ))), - None => Ok(()), - } - } - - /// Opens the payload of the generation that `lease` leases, which on Linux holds the control - /// socket of the clone's VMM, through the store's directory, following no link, as - /// [`open_payload`] opens it, or returns `None` elsewhere, where the VMM listens at a named - /// pipe, or if the payload is gone, or the store's directory holds another store, or none, - /// since neither holds a socket of this store's clones. - fn open_clone_payload(&self, lease: &Lease) -> Result> { - if !cfg!(unix) { - return Ok(None); - } - open_payload(self.dir(), self.id, lease.generation()) - } - - /// Removes the control socket of the VMM of the clone that holds `lease`, beside the snapshot - /// that the VMM restored, from `payload`, the handle of the leased generation's payload, if it - /// is there, and makes its absence durable with `sync`, through that handle. The VMM removes - /// its socket as it exits, and a stop or a recovery removes the one that an abrupt exit - /// leaves, but neither makes the removal durable. - fn remove_clone_socket( - &self, - lease: &Lease, - payload: &platform::Directory, - sync: &dyn Fn(&platform::Directory) -> io::Result<()>, - ) -> Result<()> { - payload - .remove_file(&clone_socket_name(lease.launch())) - .map_err(store_error(format!( - "cannot remove {}", - self.clone_socket_path(lease).display() - )))?; - sync(payload).map_err(store_error(format!( - "cannot sync the directory {}", - self.payload_dir(lease.generation()).display() - ))) - } - - /// Returns the leases of the generation `generation` that the store's records list, in the - /// order of their identities, including those that an interrupted release or removal of - /// their sandboxes left, which a [retirement](Self::retire) removes. - /// - /// Fails with [`ErrorCode::ForeignFiles`] if the directory of the leases holds entries that - /// are not the store's. - pub fn leases(&self, generation: StoreGeneration) -> Result> { - let mut leases = Vec::new(); - for id in self.records(LEASES_NAME, LEASES_WHAT, lease_name)?.records { - // A release may remove the record after the listing. - if let Some(lease) = self.read_lease(id)? - && lease.generation() == generation - { - leases.push(lease); - } - } - leases.sort_by_key(|lease| lease.id()); - Ok(leases) - } - - /// Quarantines the generation `generation` with a tombstone, waiting for the store's lock - /// until `deadline`, or for as long as it takes without one: the generation takes no new - /// lease, and [`select`](Self::select) skips it, while its leases keep working. A tombstone - /// is durable once this returns, and lasts until the generation's - /// [retirement](Self::retire). - /// - /// Tombstoning a generation again does nothing. Fails with [`ErrorKind::NotFound`] if the - /// store does not index the generation. - pub fn tombstone(&self, generation: StoreGeneration, deadline: Option) -> Result<()> { - let _lock = take_lock(&self.dir, deadline)?; - if self.index()?.get(generation).is_none() { - return Err(self.not_indexed(generation)); - } - self.write_tombstone(generation, &sync_dir) - } - - /// Returns the store generations that tombstones quarantine, in order. - /// - /// Each tombstone is read and checked, as an acquisition checks it: a tombstone that cannot be - /// read, or is malformed, fails with [`ErrorKind::Io`], and one of another format with - /// [`ErrorCode::FormatUnsupported`], rather than quarantining its generation. Fails with - /// [`ErrorCode::ForeignFiles`] if the directory of the tombstones holds entries that are not - /// the store's. - pub fn tombstones(&self) -> Result> { - let mut tombstoned = Vec::new(); - for generation in self - .records(TOMBSTONES_NAME, TOMBSTONES_WHAT, tombstone_name)? - .records - { - // A retirement may remove the tombstone after the listing. - if self.read_tombstone(generation)? { - tombstoned.push(generation); - } - } - tombstoned.sort(); - Ok(tombstoned) - } - - /// Retires the generation `generation`, waiting for the store's lock until `deadline`, or for - /// as long as it takes without one: it tombstones the generation, and deletes it once its - /// leases have drained. - /// - /// Under the store's lock, it tombstones the generation, and removes the records of its - /// leases whose sandboxes no longer hold them: a sandbox that is gone, that holds no clone - /// binding, or one of another lease, as an interrupted release, or a removal of the sandbox, - /// leaves it. It makes the binding's absence durable first, by syncing the sandbox's - /// directory, if it is one, and the directory that holds the sandbox's entry, the nearest of - /// its ancestors that remains, through the links that the sandbox's path names, as a release - /// that a crash or a failed sync interrupted may not have. A [removal](LockedStore::remove) of - /// the sandbox made it durable before the sandbox went; for a sandbox that went otherwise, - /// these syncs make it durable only while the directories that held the sandbox remain. If - /// any lease remains, the generation keeps its payload and its entry in the index, and the - /// retirement returns [`Retirement::Draining`] with the leases, whose clones - /// [release](Self::release) them; a later retirement deletes it. Otherwise it removes the - /// generation's payload, then its entry in the index, and then its tombstone, each durable - /// before the next, as the tombstone is before the payload goes, so that a retirement that a - /// crash interrupts, even one of the host, leaves a tombstoned generation, which retiring it - /// again deletes, and returns [`Retirement::Retired`]. A generation that the store indexed - /// once, and no longer indexes, is retired already. Its store generation is never assigned - /// again. - /// - /// It also removes the temporary files that interrupted replacements of the records of leases - /// and tombstones left, since the store writes those only under its lock. - /// - /// Fails with [`ErrorKind::NotFound`] for a store generation that the store never assigned, - /// and with [`ErrorCode::ForeignFiles`] if the directory of the leases, or of the tombstones, - /// holds entries that are not the store's. A clone binding that cannot be read, or whose - /// absence cannot be made durable, fails the retirement, which then deletes no generation. - pub fn retire( - &self, - generation: StoreGeneration, - deadline: Option, - ) -> Result { - self.retire_with(generation, deadline, &sync_dir) - } - - /// Retires a generation as [`retire`](Self::retire) does, making each step durable with `sync` - /// before the next. - fn retire_with( - &self, - generation: StoreGeneration, - deadline: Option, - sync: SyncDir<'_>, - ) -> Result { - let _lock = take_lock(&self.dir, deadline)?; - let mut index = self.index()?; - if index.get(generation).is_none() { - if generation >= index.next { - return Err(self.not_indexed(generation)); - } - // A retirement that was interrupted once it had written the index may not have made - // it durable, which must be before the tombstone goes. - sync(&self.dir)?; - self.remove_tombstone(generation)?; - return Ok(Retirement::Retired); - } - let leases = self.records(LEASES_NAME, LEASES_WHAT, lease_name)?; - let tombstones = self.records(TOMBSTONES_NAME, TOMBSTONES_WHAT, tombstone_name)?; - self.write_tombstone(generation, sync)?; - for temporary in leases.temporaries.iter().chain(&tombstones.temporaries) { - remove_if_present(temporary)?; - } - let mut live = Vec::new(); - for id in leases.records { - let Some(lease) = self.read_lease(id)? else { - continue; - }; - if lease.generation() != generation { - continue; - } - if is_bound(&lease)? { - live.push(lease); - } else { - // A release may have removed the binding without making that durable yet, so the - // binding's absence must be durable before its record goes. - for dir in binding_dirs(lease.sandbox())? { - sync(dir)?; - } - remove_lease_record(&self.lease_path(id))?; - } - } - if !live.is_empty() { - live.sort_by_key(|lease| lease.id()); - return Ok(Retirement::Draining(live)); - } - self.remove_payload(generation)?; - sync(&self.dir.join(GENERATIONS_NAME))?; - index - .generations - .retain(|indexed| indexed.generation != generation); - self.write_index(&index)?; - sync(&self.dir)?; - self.remove_tombstone(generation)?; - Ok(Retirement::Retired) - } - - /// Returns the error of the generation `generation`, which the store does not index. - fn not_indexed(&self, generation: StoreGeneration) -> Error { - Error::new( - ErrorKind::NotFound, - format!( - "the template store {} holds no generation {generation}", - self.dir.display() - ), - ) - } - - /// Returns the path of the record of the lease `id`. - fn lease_path(&self, id: LeaseId) -> PathBuf { - self.dir.join(LEASES_NAME).join(format!("{id}.json")) - } - - /// Returns the path of the tombstone of the store generation `generation`. - fn tombstone_path(&self, generation: StoreGeneration) -> PathBuf { - self.dir - .join(TOMBSTONES_NAME) - .join(format!("{generation}.json")) - } - - /// Creates the store's directory `name`, which `what` names, private to the current user, - /// unless it exists, under the store's lock, and makes its entry in the store durable with - /// `sync`, and the store's own entry in its parent: it syncs the store's directory, and then - /// the store's parent, once it creates the directory, and once it finds one that this process - /// has not synced yet, whose creation, or the store's, may not have finished its sync. A - /// binding then never names a record of a store whose own entry a crash of the host could - /// lose. - fn create_records_dir(&self, name: &'static str, what: &str, sync: SyncDir<'_>) -> Result<()> { - let key = (self.dir.clone(), self.id, name); - if !self.check_records_dir(name, what)? { - let dir = self.dir.join(name); - let problem = platform::create_owner_only_dir(&dir).map_err(store_error(format!( - "cannot create {what} {}", - dir.display() - )))?; - refuse_problem(&dir, what, problem)?; - } else if durable_record_dirs().contains(&key) { - return Ok(()); - } - sync(&self.dir)?; - if let Some(parent) = self.dir.parent() { - sync(parent)?; - } - let mut durable = durable_record_dirs(); - if !durable.contains(&key) { - durable.push(key); - } - Ok(()) - } - - /// Checks that the store's directory `name`, which `what` names, is private to the current - /// user, and not a link, if it exists, and returns whether it exists. - fn check_records_dir(&self, name: &str, what: &str) -> Result { - let dir = self.dir.join(name); - match platform::check_owner_only_dir(&dir) { - Err(error) if error.kind() == io::ErrorKind::NotFound => Ok(false), - checked => { - let problem = checked - .map_err(store_error(format!("cannot open {what} {}", dir.display())))?; - refuse_problem(&dir, what, problem)?; - Ok(true) - } - } - } - - /// Lists the store's directory `name`, which `what` names, whose records `parse` names: the - /// records, and the temporary files that interrupted replacements of them left. A directory - /// that does not exist holds none. - /// - /// Fails with [`ErrorCode::ForeignFiles`] for any other entry. - fn records( - &self, - name: &str, - what: &str, - parse: impl Fn(&str) -> Option, - ) -> Result> { - let mut listed = Records { - records: Vec::new(), - temporaries: Vec::new(), - }; - if !self.check_records_dir(name, what)? { - return Ok(listed); - } - let dir = self.dir.join(name); - let mut foreign = Vec::new(); - for entry in entries(&dir)? { - if let Some(record) = parse(&entry) { - listed.records.push(record); - } else if is_temporary_of(&entry, |file| parse(file).is_some()) { - listed.temporaries.push(dir.join(entry)); - } else { - foreign.push(entry); - } - } - if foreign.is_empty() { - return Ok(listed); - } - foreign.sort(); - Err(Error::new( - ErrorCode::ForeignFiles, - format!( - "{what} {} holds entries that are not the store's: {}", - dir.display(), - foreign.join(", ") - ), - )) - } - - /// Reads the record of the lease `id`, or returns `None` if it does not exist. - fn read_lease(&self, id: LeaseId) -> Result> { - let path = self.lease_path(id); - let Some(lease) = read_lease(&path, LEASE_DESCRIPTION)? else { - return Ok(None); - }; - if lease.id() != id || lease.store() != self.id { - return Err(Error::io(format!( - "the {LEASE_DESCRIPTION} {} is malformed: it is the lease {} of the template \ - store {}", - path.display(), - lease.id(), - lease.store() - ))); - } - Ok(Some(lease)) - } - - /// Returns whether a tombstone quarantines the generation `generation`. - fn is_tombstoned(&self, generation: StoreGeneration) -> Result { - if !self.check_records_dir(TOMBSTONES_NAME, TOMBSTONES_WHAT)? { - return Ok(false); - } - self.read_tombstone(generation) - } - - /// Reads the tombstone of the generation `generation`, in the directory of the tombstones, - /// which exists, and returns whether it exists: a tombstone that cannot be read, is malformed, - /// or is of another format fails. - fn read_tombstone(&self, generation: StoreGeneration) -> Result { - let path = self.tombstone_path(generation); - let Some(record) = - read_record::(&path, TOMBSTONE_DESCRIPTION, MAX_LEASE_RECORD_LEN)? - else { - return Ok(false); - }; - if record.store != self.id.to_string() || record.generation != generation.get().get() { - return Err(Error::io(format!( - "the {TOMBSTONE_DESCRIPTION} {} is malformed: it is the tombstone of the \ - generation {} of the template store {}", - path.display(), - record.generation, - record.store - ))); - } - Ok(true) - } - - /// Writes the tombstone of the generation `generation`, unless it has one, under the store's - /// lock, and makes it durable with `sync`, even one that an earlier transaction wrote, whose - /// sync may have failed. - fn write_tombstone(&self, generation: StoreGeneration, sync: SyncDir<'_>) -> Result<()> { - if !self.is_tombstoned(generation)? { - self.create_records_dir(TOMBSTONES_NAME, TOMBSTONES_WHAT, sync)?; - write_record( - &self.tombstone_path(generation), - &TombstoneRecord { - format: STORE_FORMAT, - store: self.id.to_string(), - generation: generation.get().get(), - }, - "the tombstone of the template store", - )?; - } - sync(&self.dir.join(TOMBSTONES_NAME)) - } - - /// Removes the tombstone of the generation `generation`, if it has one, under the store's - /// lock, once it has read and checked it: a tombstone that is not the store's tombstone of - /// that generation, or that is of another format, fails, and is kept. - fn remove_tombstone(&self, generation: StoreGeneration) -> Result<()> { - if self.is_tombstoned(generation)? { - remove_if_present(&self.tombstone_path(generation))?; - } - Ok(()) - } - - /// Removes the payload of the generation `generation`, if it exists, under the store's lock. - /// A payload that is a link, or not private to the current user, is refused, and kept. - fn remove_payload(&self, generation: StoreGeneration) -> Result<()> { - self.check_generations_dir()?; - let payload = self.payload_dir(generation); - let what = "the payload of the generation"; - match platform::check_owner_only_dir(&payload) { - Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(()), - checked => { - let problem = checked.map_err(store_error(format!( - "cannot open {what} {}", - payload.display() - )))?; - refuse_problem(&payload, what, problem)?; - } - } - match fs::remove_dir_all(&payload) { - Err(error) if error.kind() != io::ErrorKind::NotFound => Err(store_error(format!( - "cannot remove {what} {}", - payload.display() - ))(error)), - _ => Ok(()), - } - } -} - -/// Returns the error of a lease that the sandbox in `dir` cannot take or end while it records a -/// VM or a launch, [`ErrorCode::SandboxRunning`]. -fn may_run(dir: &Path) -> Error { - Error::new( - ErrorCode::SandboxRunning, - format!( - "a VM of the sandbox in {}, or a launch of it, may still run", - dir.display() - ), - ) -} - -/// Returns whether the clone binding of the sandbox of `lease` names it still: unless the sandbox -/// is gone, holds no clone binding, or holds one of another lease. -fn is_bound(lease: &Lease) -> Result { - Ok(read_binding(lease.sandbox())? - .is_some_and(|binding| binding.id() == lease.id() && binding.store() == lease.store())) -} - -/// Returns the directories whose entries changed as the clone binding of the sandbox in `sandbox` -/// went: the sandbox's own directory, if it is one, and the directory that holds the sandbox's -/// entry, the nearest of its ancestors that is a directory, following the links that its path -/// names, as the removal of the sandbox followed them. A sandbox that was removed, and provisioned -/// again at its path, is a new directory, whose entries never held the binding: the binding went -/// with the directory that the entry of the sandbox named before. -/// -/// Fails for an ancestor that cannot be examined, or that is a link to nothing, whose target, the -/// directory that held the sandbox, cannot be known. -fn binding_dirs(sandbox: &Path) -> Result> { - let mut dirs = Vec::with_capacity(2); - if fs::symlink_metadata(sandbox).is_ok_and(|metadata| metadata.is_dir()) { - dirs.push(sandbox); - } - for ancestor in sandbox.ancestors().skip(1) { - let unknown = |error: io::Error| { - Error::io(format!( - "cannot find the directory from which the sandbox in {} went, through {}", - sandbox.display(), - ancestor.display() - )) - .with_source(error) - }; - match fs::metadata(ancestor) { - Ok(metadata) if metadata.is_dir() => { - dirs.push(ancestor); - return Ok(dirs); - } - // A file replaced the directory, in the entries of its parent. - Ok(_) => {} - Err(error) if error.kind() == io::ErrorKind::NotFound => { - if fs::symlink_metadata(ancestor).is_ok() { - return Err(unknown(error)); - } - } - Err(error) => return Err(unknown(error)), - } - } - Err(Error::io(format!( - "no directory holds the sandbox in {}", - sandbox.display() - ))) -} - -/// Writes `record` as the clone binding of the sandbox in `dir`, and makes it durable with `sync`. -fn bind(dir: &Path, record: &LeaseRecord, sync: SyncDir<'_>) -> Result<()> { - write_record(&dir.join(CLONE_BINDING_NAME), record, "the clone binding")?; - sync(dir) -} - -/// Removes the clone binding of the sandbox in `dir`, if it has one, and makes its removal durable -/// with `sync`, before a record of its lease goes: a binding that outlasted the record through a -/// crash would let a retirement delete the generation that it names. -fn unbind(dir: &Path, sync: SyncDir<'_>) -> Result<()> { - remove_if_present(&dir.join(CLONE_BINDING_NAME))?; - sync(dir) -} - -/// Removes the record of a lease at `path`, if it exists, which a release and a retirement, in -/// other processes, may both remove at once: a record that another removal removed as this tried, -/// as [`was_removed`] tells, is gone already. -fn remove_lease_record(path: &Path) -> Result<()> { - match fs::remove_file(path) { - Err(error) if error.kind() != io::ErrorKind::NotFound && !was_removed(path, &error) => Err( - store_error(format!("cannot remove {}", path.display()))(error), - ), - _ => Ok(()), - } -} - -/// Returns `error`, which ended an acquisition of the lease `id` whose binding the sandbox in -/// `dir` may hold still, with the error's code and source, saying so. -fn held_still(error: Error, dir: &Path, id: LeaseId) -> Error { - let code = error.code(); - let (_, message, source) = error.into_parts(); - let error = Error::new( - code, - format!( - "{message}; the sandbox in {} may hold the lease {id} still, which a release ends", - dir.display() - ), - ); - match source { - Some(source) => error.with_source(source), - None => error, - } -} - -/// Parses the name of the record of a lease, `ID.json`, in the form in which the store writes -/// it. -fn lease_name(name: &str) -> Option { - let stem = name.strip_suffix(".json")?; - let id: LeaseId = stem.parse().ok()?; - (id.to_string() == stem).then_some(id) -} - -/// Parses the name of a tombstone, `G.json`, in the form in which the store writes it. -fn tombstone_name(name: &str) -> Option { - let stem = name.strip_suffix(".json")?; - let generation = NonZeroU64::new(stem.parse().ok()?).map(StoreGeneration::new)?; - (generation.to_string() == stem).then_some(generation) -} - -/// Returns whether `name` is a temporary file that an interrupted replacement of a record whose -/// name `record` accepts leaves beside it: `.NAME.NUMBER.tmp`. -fn is_temporary_of(name: &str, record: impl Fn(&str) -> bool) -> bool { - name.strip_prefix('.') - .and_then(|name| name.strip_suffix(".tmp")) - .and_then(|name| name.rsplit_once('.')) - .is_some_and(|(file, number)| { - record(file) && !number.is_empty() && number.bytes().all(|byte| byte.is_ascii_digit()) - }) -} - -/// The entries of the directory of a store's leases or tombstones. -struct Records { - /// What the names of the records name. - records: Vec, - /// The temporary files that interrupted replacements of records left. - temporaries: Vec, -} - -/// A tombstone, as JSON. -#[derive(Debug, Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -struct TombstoneRecord { - /// [`STORE_FORMAT`]. - format: u32, - /// The identity of the store. - store: String, - /// The store generation of the tombstoned generation. - generation: u64, -} diff --git a/nvx-rs/src/host/template/store/leases/tests.rs b/nvx-rs/src/host/template/store/leases/tests.rs deleted file mode 100644 index 2a8b6898d..000000000 --- a/nvx-rs/src/host/template/store/leases/tests.rs +++ /dev/null @@ -1,2956 +0,0 @@ -//! Tests of leases, clone bindings, tombstones, and retirement, by one process, and by -//! transactions in processes of their own, which [`transaction_role`] runs; and of the crashes of -//! a clone's orchestrator, which [`clone_role`] runs, with a [`vmm_role`] as the clone's VMM. - -use std::cell::{Cell, RefCell}; -use std::convert::Infallible; -use std::env; -use std::ffi::OsString; -use std::io::{BufRead, BufReader, Read, Write}; -use std::process::{Child, ChildStdin, ChildStdout, Command, Stdio}; -use std::thread; -use std::time::Duration; - -use serde_json::{Value, json}; - -use super::*; -use crate::activation::TemplateReference; -use crate::host::process::{Opened, Process, ProcessRef}; -use crate::host::snapshot::tests::{ManifestSpec, artifact, network_class}; -use crate::host::store::files::write_private; -use crate::host::store::{ - CLONE_ACTIVATION_NAME, CLONE_FAILURE_NAME, CLONE_PROGRESS_NAME, CLONE_SCRATCH_NAME, FORMAT, - LOCK_NAME, LaunchState, RESTORE_READY_NAME, RUNTIME_NAME, Recovery, RuntimeIdentity, -}; -use crate::host::supervisor::{self, Reattached, StopOutcome, VmmCommand}; -use crate::host::template::eligibility::MAX_DOWNTIME_NS; -use crate::host::template::store::SNAPSHOT_NAME; -use crate::host::template::store::tests::{ - cause, link_dir, link_file, publish, publish_with, remove, storage, store as new_store, - unlink_dir, -}; -use crate::host::tests::released; -use crate::protocol::CAPABILITY_LEN; - -/// A change to a record, as JSON. -type Change = fn(&mut Value); - -/// Nanoseconds in a second. -const SECOND: u64 = 1_000_000_000; -/// Nanoseconds in a day. -const DAY: u64 = 86_400 * SECOND; -/// The configuration record of these tests' sandboxes. -const CONFIG: &str = "config.json"; -/// The directory of the store in which [`transaction_role`] runs its transaction. -const ROLE_STORE: &str = "NVX_TEMPLATE_LEASE_TEST_STORE"; -/// The transaction that [`transaction_role`] runs: `acquire`, `release`, `tombstone`, or -/// `retire`. -const ROLE_ACTION: &str = "NVX_TEMPLATE_LEASE_TEST_ACTION"; -/// The store generation of the transaction of [`transaction_role`]. -const ROLE_GENERATION: &str = "NVX_TEMPLATE_LEASE_TEST_GENERATION"; -/// The directory of the sandbox of the transaction of [`transaction_role`], if it has one. -const ROLE_SANDBOX: &str = "NVX_TEMPLATE_LEASE_TEST_SANDBOX"; -/// The line that [`transaction_role`] prints once it is ready, before it waits for a line on its -/// standard input. -const READY: &str = "nvx-template-lease-test-ready"; -/// The start of the line in which [`transaction_role`] prints how its transaction ended. -const OUTCOME: &str = "nvx-template-lease-test-outcome "; -/// The step after which [`clone_role`] crashes: `after-record`, `after-binding`, or `acquired`, of -/// its acquisition; `before-spawn`, `after-spawn`, `after-identity`, or `committed`, of its -/// clone's launch; or `releasing`, of a release. -const ROLE_CRASH: &str = "NVX_TEMPLATE_LEASE_TEST_CRASH"; -/// The working directory of the VMM that [`clone_role`] launches. -const ROLE_VMM: &str = "NVX_TEMPLATE_LEASE_TEST_VMM"; -/// The start of the line in which [`clone_role`] prints the reference of the VMM that it spawned, -/// as JSON. -const REFERENCE: &str = "nvx-template-lease-test-vmm "; -/// The start of the line that [`clone_role`] prints as it crashes, before the step after which it -/// crashes. -const CRASHED: &str = "nvx-template-lease-test-crashed "; -/// File whose presence in its working directory marks a [`vmm_role`]. Without it, the role -/// returns at once, as it does when `cargo test -- --ignored` runs it. -const VMM_MARKER: &str = "vmm-role"; -/// File whose appearance in its working directory ends a [`vmm_role`]. -const EXIT_MARKER: &str = "exit"; -/// The capability of the VMMs of these tests' clones. -const CAPABILITY: [u8; CAPABILITY_LEN] = [7; CAPABILITY_LEN]; -/// Longest wait for a VMM that is expected to exit, for a sandbox's lock, or for a reattach to -/// become determinate. -const TIMEOUT: Duration = Duration::from_secs(60); -/// Longest life of a [`vmm_role`] that nothing ends, so that a failed test leaves no process. -const VMM_LIFETIME: Duration = Duration::from_secs(180); - -/// Returns the files that these tests' callers keep in their sandboxes. -fn files() -> CallerFiles { - CallerFiles::new(CONFIG).unwrap() -} - -/// Provisions the sandbox `name` in `parent`, and returns its directory, with its links resolved, -/// as a lease names it. -fn provision(parent: &Path, name: &str) -> PathBuf { - let dir = parent.join(name); - let locked = SandboxStore::lock_for_provision(&dir, &files(), None).unwrap(); - fs::write(dir.join(CONFIG), b"{}").unwrap(); - drop(locked); - filesystem::canonicalize(&dir).unwrap() -} - -/// Returns the time of the host that captured `generation`, during the boot of the capture, -/// `seconds` after it. -fn after(generation: &Generation, seconds: u64) -> HostTime { - let time = generation.time(); - HostTime { - host_id: time.host_id, - boot_id: time.boot_id, - clock: time.host_clock, - monotonic_ns: time.capture_monotonic_ns + seconds * SECOND, - utc_ns: time.capture_utc_ns + seconds * SECOND, - } -} - -/// Leases `generation` of `store` to the sandbox in `sandbox`, a second after its capture, and -/// releases the sandbox's lock. -fn leased(store: &Store, generation: &Generation, sandbox: &Path) -> Lease { - store - .acquire( - generation.generation(), - sandbox, - &files(), - &after(generation, 1), - None, - ) - .unwrap() - .lease -} - -/// Returns the error with which leasing `generation` of `store` to the sandbox in `sandbox` at -/// `now` fails. -fn refusal(store: &Store, generation: StoreGeneration, sandbox: &Path, now: &HostTime) -> Error { - store - .acquire(generation, sandbox, &files(), now, None) - .unwrap_err() -} - -/// Releases the lease of the sandbox in `sandbox` from `store`, under the sandbox's lock. -fn release(store: &Store, sandbox: &Path) -> Result> { - let mut locked = SandboxStore::lock_existing(sandbox, &files(), None).unwrap(); - store.release(&mut locked) -} - -/// Returns the lease that the clone binding of the sandbox in `sandbox` records, if any. -fn binding(sandbox: &Path) -> Option { - Lease::of(&SandboxStore::open(sandbox).unwrap()).unwrap() -} - -/// Returns the store generation `generation`, which must not be 0. -fn store_generation(generation: u64) -> StoreGeneration { - StoreGeneration::new(NonZeroU64::new(generation).unwrap()) -} - -/// Returns a runtime identity under this crate's contracts. -fn runtime() -> RuntimeIdentity { - RuntimeIdentity::new(artifact('a'), artifact('b'), artifact('c')) -} - -/// Records a VM of the sandbox in `sandbox`, as the commit of its launch does. -fn record_vm(sandbox: &Path) { - let record = json!({ - "format": FORMAT, - "launch": "0123456789abcdef0123456789abcdef", - "process": { "pid": 42, "startTime": 4242, "boot": "ab".repeat(16) }, - "endpoint": "/state/sandbox/control.sock", - "runtime": serde_json::to_value(runtime()).unwrap(), - }); - write_private(&sandbox.join(RUNTIME_NAME), record.to_string().as_bytes()).unwrap(); -} - -/// Ends, through `locked`, a launch that spawned nothing, and whose log is closed. -/// -/// A process that a concurrent test spawned may still hold the log, until it executes its -/// program, and a later recovery then ends the launch. -fn recover_unspawned(locked: &mut LockedStore) { - assert!(released(|| locked.recover().unwrap() == Recovery::Exited)); -} - -/// Returns the file at `path` as JSON. -fn json_of(path: &Path) -> Value { - serde_json::from_slice(&fs::read(path).unwrap()).unwrap() -} - -/// Returns the identities of `leases`, sorted, and separated by spaces. -fn ids(leases: &[Lease]) -> String { - let mut ids: Vec = leases.iter().map(|lease| lease.id().to_string()).collect(); - ids.sort(); - ids.join(" ") -} - -/// Returns the arguments that run only the role `name` of this test executable, on libtest's main -/// thread, so that libtest prints the role's name before the role's own output. -fn role(name: &str) -> Vec { - [ - "--exact", - &format!("host::template::store::leases::tests::{name}"), - "--ignored", - "--nocapture", - "--test-threads=1", - ] - .map(OsString::from) - .to_vec() -} - -/// Returns the error with which `locked` refuses to begin the launch `launch`. -fn launch_refusal(locked: &mut LockedStore, launch: &LaunchId) -> Error { - match locked.begin_launch(launch, &runtime(), &CAPABILITY) { - Ok(_) => panic!("the launch {launch} began"), - Err(error) => error, - } -} - -#[test] -fn leases_bind_clones_to_the_generations_that_they_restore() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let first = provision(directory.path(), "first"); - let acquired = store - .acquire( - published.generation(), - &first, - &files(), - &after(&published, 1), - None, - ) - .unwrap(); - let lease = acquired.lease.clone(); - assert_eq!(lease.store(), store.id()); - assert_eq!(lease.generation(), published.generation()); - assert_eq!(lease.key(), published.key()); - assert_eq!(lease.sandbox(), first); - assert_eq!( - lease.template(), - TemplateReference { - generation: published.generation(), - key: Identity::from(published.key()), - } - ); - // The caller keeps the sandbox's lock, under which it launches the clone, until it drops it. - assert_eq!(acquired.locked.dir(), first); - let busy = SandboxStore::lock_existing(&first, &files(), Some(Instant::now())).unwrap_err(); - assert_eq!(busy.code(), ErrorCode::SandboxBusy, "{busy}"); - drop(acquired); - // The clone binding in the sandbox and the record in the store are the same lease. - assert_eq!(binding(&first), Some(lease.clone())); - assert_eq!( - store.leases(published.generation()).unwrap(), - std::slice::from_ref(&lease) - ); - // A sandbox holds one lease at a time. - let error = refusal( - &store, - published.generation(), - &first, - &after(&published, 2), - ); - assert_eq!(error.code(), ErrorCode::SandboxLeased, "{error}"); - assert!(error.message().contains(&lease.id().to_string()), "{error}"); - // Other sandboxes lease the generation too, and other generations. - let second = provision(directory.path(), "second"); - let mut leases = vec![lease, leased(&store, &published, &second)]; - leases.sort_by_key(Lease::id); - let other = publish(&store, &ManifestSpec::template(), storage(2)); - let third = provision(directory.path(), "third"); - let other_lease = leased(&store, &other, &third); - assert_eq!(store.leases(published.generation()).unwrap(), leases); - assert_eq!(store.leases(other.generation()).unwrap(), [other_lease]); -} - -#[test] -fn acquisitions_refuse_what_they_cannot_lease_and_lease_nothing() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let now = after(&published, 1); - // A generation that the store does not index. - let error = refusal(&store, store_generation(9), &sandbox, &now); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); - // One that is not eligible at the host's time: no longer, or not yet. - let expired = after(&published, MAX_DOWNTIME_NS / SECOND + 1); - let mut early = after(&published, 0); - early.monotonic_ns -= 1; - for (now, condition) in [ - (expired, "is no longer eligible"), - (early, "is not eligible yet"), - ] { - let error = refusal(&store, generation, &sandbox, &now); - assert_eq!(error.code(), ErrorCode::TemplateIneligible, "{error}"); - assert!(error.message().contains(condition), "{error}"); - } - // On Linux, a payload that is not a directory of the current user with mode 0700, which a - // launch of the clone requires, and through which a release ends the lease: it opens the - // payload for reading, and removes the clone's socket from it. A user other than root cannot - // open a payload of mode 0300 for reading at all. - #[cfg(target_os = "linux")] - { - use std::os::unix::fs::PermissionsExt; - - let payload = store.payload_dir(generation); - let unopened = format!( - "cannot open the payload of the generation {}", - payload.display() - ); - for mode in [0o300, 0o500] { - fs::set_permissions(&payload, fs::Permissions::from_mode(mode)).unwrap(); - let error = refusal(&store, generation, &sandbox, &now); - fs::set_permissions(&payload, fs::Permissions::from_mode(0o700)).unwrap(); - assert_eq!(error.kind(), ErrorKind::Io, "{error}"); - let refused = format!( - "the payload of the generation {} has mode {mode:o}, not the mode 700 that the \ - directory of a control socket must have", - payload.display() - ); - assert!( - error.message() == refused || (mode == 0o300 && error.message() == unopened), - "{error}" - ); - } - // A link in the place of the store's directory, of the directory of the generations, or of - // the payload, even to a directory that holds the same files: a release, which follows - // none of them, could not open the payload. - let root = store.dir().to_owned(); - let generations = payload.parent().unwrap().to_owned(); - let elsewhere = directory.path().join("elsewhere"); - for linked in [&root, &generations, &payload] { - fs::rename(linked, &elsewhere).unwrap(); - std::os::unix::fs::symlink(&elsewhere, linked).unwrap(); - let error = refusal(&store, generation, &sandbox, &now); - fs::remove_file(linked).unwrap(); - fs::rename(&elsewhere, linked).unwrap(); - assert_eq!(error.kind(), ErrorKind::Io, "{error}"); - assert_eq!(error.message(), unopened, "{error}"); - let source = std::error::Error::source(&error).unwrap().to_string(); - assert!( - source.ends_with("is a link, or not a directory"), - "{source}" - ); - } - } - // A sandbox that is not provisioned: its directory is missing, or holds no configuration. - let unprovisioned = directory.path().join("unprovisioned"); - drop(SandboxStore::lock_for_provision(&unprovisioned, &files(), None).unwrap()); - for dir in [directory.path().join("missing"), unprovisioned] { - let error = refusal(&store, generation, &dir, &now); - assert_eq!(error.code(), ErrorCode::SandboxNotProvisioned, "{error}"); - } - // A sandbox that records a launch, or a VM, which may still run. - let mut locked = SandboxStore::lock_existing(&sandbox, &files(), None).unwrap(); - drop( - locked - .begin_launch( - &LaunchId::generate().unwrap(), - &runtime(), - &[7; CAPABILITY_LEN], - ) - .unwrap(), - ); - drop(locked); - let error = refusal(&store, generation, &sandbox, &now); - assert_eq!(error.code(), ErrorCode::SandboxRunning, "{error}"); - let mut locked = SandboxStore::lock_existing(&sandbox, &files(), None).unwrap(); - recover_unspawned(&mut locked); - record_vm(&sandbox); - drop(locked); - let error = refusal(&store, generation, &sandbox, &now); - assert_eq!(error.code(), ErrorCode::SandboxRunning, "{error}"); - let mut locked = SandboxStore::lock_existing(&sandbox, &files(), None).unwrap(); - locked.clear_runtime().unwrap(); - drop(locked); - // A generation that a tombstone quarantines. - store.tombstone(generation, None).unwrap(); - let error = refusal(&store, generation, &sandbox, &now); - assert_eq!(error.code(), ErrorCode::TemplateTombstoned, "{error}"); - // None of them leased anything. - assert!(store.leases(generation).unwrap().is_empty()); - assert_eq!(binding(&sandbox), None); - assert!(!store.dir().join(LEASES_NAME).exists()); -} - -#[test] -fn releases_end_leases_once_their_clones_have_exited() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let mut acquired = store - .acquire(generation, &sandbox, &files(), &after(&published, 1), None) - .unwrap(); - let lease = acquired.lease.clone(); - // While a launch of the clone, or its VM, may run, its lease stays. - drop( - acquired - .locked - .begin_launch(lease.launch(), &runtime(), &[7; CAPABILITY_LEN]) - .unwrap(), - ); - let error = store.release(&mut acquired.locked).unwrap_err(); - assert_eq!(error.code(), ErrorCode::SandboxRunning, "{error}"); - recover_unspawned(&mut acquired.locked); - record_vm(&sandbox); - let error = store.release(&mut acquired.locked).unwrap_err(); - assert_eq!(error.code(), ErrorCode::SandboxRunning, "{error}"); - assert_eq!(binding(&sandbox), Some(lease.clone())); - assert_eq!( - store.leases(generation).unwrap(), - std::slice::from_ref(&lease) - ); - // Once its VM has exited, the release removes the binding, and then the store's record. - acquired.locked.clear_runtime().unwrap(); - assert_eq!( - store.release(&mut acquired.locked).unwrap(), - Some(lease.clone()) - ); - drop(acquired); - assert_eq!(binding(&sandbox), None); - assert!(store.leases(generation).unwrap().is_empty()); - assert_eq!(release(&store, &sandbox).unwrap(), None); - // The sandbox may lease the generation again, under another identity. - let again = leased(&store, &published, &sandbox); - assert_ne!(again.id(), lease.id()); - // Another store's release keeps the lease. - let (_other_directory, other) = new_store(); - let error = release(&other, &sandbox).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Other, "{error}"); - assert_eq!(binding(&sandbox), Some(again.clone())); - assert_eq!(store.leases(generation).unwrap(), [again]); -} - -#[test] -fn a_lease_allows_one_launch_of_its_clone_under_the_lock_that_acquired_it() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let sandbox = provision(directory.path(), "sandbox"); - let mut acquired = store - .acquire( - published.generation(), - &sandbox, - &files(), - &after(&published, 1), - None, - ) - .unwrap(); - let reserved = acquired.lease.launch().clone(); - let once = - format!("whose launch {reserved} begins once, under the lock that acquired the lease"); - // No launch but the reserved one. - let other = LaunchId::generate().unwrap(); - let error = launch_refusal(&mut acquired.locked, &other); - assert_eq!(error.code(), ErrorCode::SandboxLeased, "{error}"); - assert!( - error.message().ends_with(&format!( - "which reserves the launch {reserved}, not {other}" - )), - "{error}" - ); - // The reserved launch, which its journal names, once. - drop( - acquired - .locked - .begin_launch(&reserved, &runtime(), &CAPABILITY) - .unwrap(), - ); - assert_eq!( - acquired.locked.journal().unwrap().unwrap().launch(), - &reserved - ); - recover_unspawned(&mut acquired.locked); - for launch in [&reserved, &other] { - let error = launch_refusal(&mut acquired.locked, launch); - assert_eq!(error.code(), ErrorCode::SandboxLeased, "{error}"); - } - let error = launch_refusal(&mut acquired.locked, &reserved); - assert!(error.message().ends_with(&once), "{error}"); - // Once a release ends the lease, the sandbox launches as any launch. - assert_eq!( - store.release(&mut acquired.locked).unwrap(), - Some(acquired.lease.clone()) - ); - drop( - acquired - .locked - .begin_launch(&other, &runtime(), &CAPABILITY) - .unwrap(), - ); - recover_unspawned(&mut acquired.locked); - drop(acquired); - // The launch of a lease whose acquisition's lock is gone, as a crash leaves it, begins under - // no other lock. - let lease = leased(&store, &published, &sandbox); - let mut locked = SandboxStore::lock_existing(&sandbox, &files(), None).unwrap(); - let error = launch_refusal(&mut locked, lease.launch()); - assert_eq!(error.code(), ErrorCode::SandboxLeased, "{error}"); - assert!( - error.message().ends_with(&format!( - "whose launch {} begins once, under the lock that acquired the lease", - lease.launch() - )), - "{error}" - ); -} - -#[test] -fn tombstones_quarantine_generations_and_keep_their_leases() { - let (directory, store) = new_store(); - let first = publish(&store, &ManifestSpec::template(), storage(1)); - let second = publish(&store, &ManifestSpec::template(), storage(2)); - let leasing = provision(directory.path(), "leasing"); - let lease = leased(&store, &second, &leasing); - let now = after(&second, 1); - assert_eq!( - store.select(second.key(), &now).unwrap(), - Some(second.clone()) - ); - assert!(store.tombstones().unwrap().is_empty()); - // A tombstone stops new leases, and selection skips the generation, whose leases stay. - store.tombstone(second.generation(), None).unwrap(); - store.tombstone(second.generation(), None).unwrap(); - assert_eq!(store.tombstones().unwrap(), [second.generation()]); - let sandbox = provision(directory.path(), "sandbox"); - let error = refusal(&store, second.generation(), &sandbox, &now); - assert_eq!(error.code(), ErrorCode::TemplateTombstoned, "{error}"); - assert_eq!(binding(&leasing), Some(lease.clone())); - assert_eq!(store.leases(second.generation()).unwrap(), [lease]); - assert_eq!( - store.select(second.key(), &now).unwrap(), - Some(first.clone()) - ); - leased(&store, &first, &sandbox); - // A tombstone needs a generation that the store indexes. - let error = store.tombstone(store_generation(9), None).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); - assert_eq!(store.tombstones().unwrap(), [second.generation()]); -} - -#[test] -fn selection_takes_the_latest_eligible_generation_of_a_key() { - let (_directory, store) = new_store(); - let first = publish(&store, &ManifestSpec::template(), storage(1)); - let mut spec = ManifestSpec::template(); - spec.time().capture_utc_ns += 10 * DAY; - spec.time().capture_monotonic_ns += 10 * DAY; - let second = publish(&store, &spec, storage(2)); - let other = publish_with( - &store, - &network_class(), - &ManifestSpec::with_network(), - storage(3), - ); - assert_eq!(second.key(), first.key()); - assert_ne!(other.key(), first.key()); - // The host's time `days` after the first capture, during its boot. - let at = |days: u64| HostTime { - monotonic_ns: first.time().capture_monotonic_ns + days * DAY, - utc_ns: first.time().capture_utc_ns + days * DAY, - ..after(&first, 0) - }; - let select = |now: &HostTime| store.select(first.key(), now).unwrap(); - // Before the second capture, only the first generation is eligible; then both are, and the - // latest is selected; after the first's window, only the second is, and after both, none. - assert_eq!(select(&at(1)), Some(first.clone())); - assert_eq!(select(&at(11)), Some(second.clone())); - assert_eq!(select(&at(35)), Some(second.clone())); - assert_eq!(select(&at(45)), None); - // A quarantined generation is skipped. - store.tombstone(second.generation(), None).unwrap(); - assert_eq!(select(&at(11)), Some(first.clone())); - assert_eq!(select(&at(35)), None); - // Each key selects among its own generations. - assert_eq!(store.select(other.key(), &at(1)).unwrap(), Some(other)); - assert_eq!( - store - .select(KeyIdentity::from_bytes([0; 32]), &at(1)) - .unwrap(), - None - ); -} - -#[test] -fn retirement_deletes_a_generation_once_its_leases_drain() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - // The generation stays, quarantined, while its lease lasts. - for _ in 0..2 { - assert_eq!( - store.retire(generation, None).unwrap(), - Retirement::Draining(vec![lease.clone()]) - ); - } - assert_eq!(store.tombstones().unwrap(), [generation]); - assert_eq!(store.index().unwrap().get(generation), Some(&published)); - assert!(store.snapshot_dir(&published).is_ok()); - let other = provision(directory.path(), "other"); - let error = refusal(&store, generation, &other, &after(&published, 1)); - assert_eq!(error.code(), ErrorCode::TemplateTombstoned, "{error}"); - // Once its clone releases it, the generation goes: its payload, its entry, and its tombstone. - assert_eq!(release(&store, &sandbox).unwrap(), Some(lease)); - assert_eq!(store.retire(generation, None).unwrap(), Retirement::Retired); - assert!(!store.payload_dir(generation).exists()); - assert_eq!(store.index().unwrap().get(generation), None); - assert!(store.tombstones().unwrap().is_empty()); - assert!(store.leases(generation).unwrap().is_empty()); - // Retiring it again finds it retired, and its store generation is never assigned again. - assert_eq!(store.retire(generation, None).unwrap(), Retirement::Retired); - let next = publish(&store, &ManifestSpec::template(), storage(1)); - assert!(next.generation() > generation); - let unassigned = store_generation(next.generation().get().get() + 1); - let error = store.retire(unassigned, None).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); - let error = refusal(&store, generation, &other, &after(&published, 1)); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); -} - -#[test] -fn retirement_removes_the_leases_that_no_clone_binds() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandboxes: Vec = ["released", "removed", "leased-again", "bound"] - .into_iter() - .map(|name| provision(directory.path(), name)) - .collect(); - let leases: Vec = sandboxes - .iter() - .map(|sandbox| leased(&store, &published, sandbox)) - .collect(); - // A release that was interrupted once it had removed the clone binding. - fs::remove_file(sandboxes[0].join(CLONE_BINDING_NAME)).unwrap(); - // A sandbox whose removal removed its binding with the rest. - let locked = SandboxStore::lock_existing(&sandboxes[1], &files(), None).unwrap(); - assert!(locked.remove().unwrap().is_none()); - assert!(!sandboxes[1].exists()); - // A sandbox that leased the generation again after such a release. - fs::remove_file(sandboxes[2].join(CLONE_BINDING_NAME)).unwrap(); - let again = leased(&store, &published, &sandboxes[2]); - let mut listed = leases.clone(); - listed.push(again.clone()); - listed.sort_by_key(Lease::id); - assert_eq!(store.leases(generation).unwrap(), listed); - // The retirement removes the records of the first three, and drains the bound ones. - let mut live = vec![again, leases[3].clone()]; - live.sort_by_key(Lease::id); - assert_eq!( - store.retire(generation, None).unwrap(), - Retirement::Draining(live.clone()) - ); - assert_eq!(store.leases(generation).unwrap(), live); - for sandbox in &sandboxes[2..] { - assert!(release(&store, sandbox).unwrap().is_some()); - } - assert_eq!(store.retire(generation, None).unwrap(), Retirement::Retired); -} - -#[test] -fn interrupted_retirements_finish() { - let (directory, store) = new_store(); - let first = publish(&store, &ManifestSpec::template(), storage(1)); - let second = publish(&store, &ManifestSpec::template(), storage(2)); - let third = publish(&store, &ManifestSpec::template(), storage(3)); - // One that was interrupted once it had removed the payload leaves the generation indexed and - // quarantined. - store.tombstone(first.generation(), None).unwrap(); - fs::remove_dir_all(store.payload_dir(first.generation())).unwrap(); - assert_eq!( - store.retire(first.generation(), None).unwrap(), - Retirement::Retired - ); - assert_eq!(store.index().unwrap().get(first.generation()), None); - // One that was interrupted once it had removed the entry from the index leaves the tombstone. - store.tombstone(second.generation(), None).unwrap(); - fs::remove_dir_all(store.payload_dir(second.generation())).unwrap(); - remove(&store, second.generation()); - assert_eq!(store.tombstones().unwrap(), [second.generation()]); - assert_eq!( - store.retire(second.generation(), None).unwrap(), - Retirement::Retired - ); - assert!(store.tombstones().unwrap().is_empty()); - // Interrupted replacements of records leave temporary files, which listings skip, and which - // a retirement removes under the store's lock. - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &third, &sandbox); - store.tombstone(third.generation(), None).unwrap(); - let temporaries = [ - store - .dir() - .join(LEASES_NAME) - .join(format!(".{}.json.42.tmp", lease.id())), - store.dir().join(TOMBSTONES_NAME).join(".7.json.1.tmp"), - ]; - for temporary in &temporaries { - fs::write(temporary, b"{").unwrap(); - } - assert_eq!( - store.leases(third.generation()).unwrap(), - std::slice::from_ref(&lease) - ); - assert_eq!(store.tombstones().unwrap(), [third.generation()]); - assert_eq!( - store.retire(third.generation(), None).unwrap(), - Retirement::Draining(vec![lease]) - ); - for temporary in &temporaries { - assert!(!temporary.exists(), "{}", temporary.display()); - } -} - -#[test] -fn acquisitions_make_their_records_durable_before_their_bindings_and_both_before_they_return() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let leases_dir = store.dir().join(LEASES_NAME); - let first = provision(directory.path(), "first"); - let second = provision(directory.path(), "second"); - // Each sync notes its directory, the records of leases in the store, and the bindings of the - // sandboxes. - let synced = RefCell::new(Vec::new()); - let sync = |dir: &Path| -> Result<()> { - let records = fs::read_dir(&leases_dir).map_or(0, Iterator::count); - let bindings = [&first, &second] - .into_iter() - .filter(|sandbox| sandbox.join(CLONE_BINDING_NAME).exists()) - .count(); - synced - .borrow_mut() - .push((dir.to_owned(), records, bindings)); - Ok(()) - }; - // An acquisition that creates the directory of the leases, but cannot make its entry in the - // store durable, leases nothing, and the next acquisition syncs the store's directory again. - let now = after(&published, 1); - let unsynced_store = |dir: &Path| { - if dir == store.dir() { - Err(Error::io("the disk failed")) - } else { - Ok(()) - } - }; - let error = store - .acquire_with( - published.generation(), - first.clone(), - &files(), - &now, - None, - &unsynced_store, - ) - .unwrap_err(); - assert_eq!(error.message(), "the disk failed"); - assert!(leases_dir.is_dir()); - assert_eq!(binding(&first), None); - for sandbox in [&first, &second] { - let acquired = store - .acquire_with( - published.generation(), - sandbox.clone(), - &files(), - &now, - None, - &sync, - ) - .unwrap(); - drop(acquired); - } - assert_eq!( - *synced.borrow(), - [ - // The first makes the entry of the directory of the leases durable, which no - // acquisition of this process did before, ahead of any record in it, and then the - // store's own entry. - (store.dir().to_owned(), 0, 0), - (store.dir().parent().unwrap().to_owned(), 0, 0), - // Each makes its record durable before its binding, and its binding before it - // returns. - (leases_dir.clone(), 1, 0), - (first.clone(), 1, 1), - (leases_dir.clone(), 2, 1), - (second.clone(), 2, 2), - ] - ); -} - -#[test] -fn an_acquisition_keeps_its_record_while_its_binding_may_outlast_a_crash() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let now = after(&published, 1); - let sandbox = provision(directory.path(), "sandbox"); - let leases_dir = store.dir().join(LEASES_NAME); - let failed = || Err(Error::io("the disk failed")); - let acquire = |sync: SyncDir<'_>| { - store - .acquire_with(generation, sandbox.clone(), &files(), &now, None, sync) - .unwrap_err() - }; - // A record that cannot be made durable goes before any binding names it. - let error = acquire(&|dir: &Path| if dir == leases_dir { failed() } else { Ok(()) }); - assert_eq!(error.message(), "the disk failed"); - assert_eq!(binding(&sandbox), None); - assert!(store.leases(generation).unwrap().is_empty()); - // A binding that cannot be made durable goes, and its record once the binding's removal is - // durable. - let first = Cell::new(true); - let error = acquire(&|dir: &Path| { - if dir == sandbox && first.replace(false) { - failed() - } else { - Ok(()) - } - }); - assert_eq!(error.message(), "the disk failed"); - assert_eq!(binding(&sandbox), None); - assert!(store.leases(generation).unwrap().is_empty()); - // A binding whose removal cannot be made durable either may outlast a crash, so its record - // stays, and keeps the generation, until a retirement finds that no binding names it. - let error = acquire(&|dir: &Path| if dir == sandbox { failed() } else { Ok(()) }); - let [kept]: [Lease; 1] = store.leases(generation).unwrap().try_into().unwrap(); - assert_eq!( - error.message(), - format!( - "the disk failed; the sandbox in {} may hold the lease {} still, which a release ends", - sandbox.display(), - kept.id() - ) - ); - assert_eq!(binding(&sandbox), None); - assert_eq!(store.retire(generation, None).unwrap(), Retirement::Retired); -} - -#[test] -fn a_release_makes_the_removal_of_its_binding_durable_before_it_removes_its_record() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - let record = store - .dir() - .join(LEASES_NAME) - .join(format!("{}.json", lease.id())); - let payload = store.payload_dir(generation); - let socket = store.clone_socket_path(&lease); - // Each sync notes its directory, and whether any file that the clone kept of its launch, the - // control socket of its VMM and the scratch file that an earlier version left for a clone in - // RAM included, the binding, and the record existed then. - let kept = [ - CLONE_SCRATCH_NAME, - CLONE_ACTIVATION_NAME, - CLONE_PROGRESS_NAME, - CLONE_FAILURE_NAME, - RESTORE_READY_NAME, - ]; - let synced = RefCell::new(Vec::new()); - let note = |dir: &Path| { - let clone = kept - .iter() - .any(|name| fs::symlink_metadata(sandbox.join(name)).is_ok()) - || fs::symlink_metadata(&socket).is_ok(); - let bound = sandbox.join(CLONE_BINDING_NAME).exists(); - synced - .borrow_mut() - .push((dir.to_owned(), clone, bound, record.exists())); - }; - // The release syncs the payload through its handle, which holds the generation's snapshot. - let snapshot = platform::file_id(&payload.join(SNAPSHOT_NAME)).unwrap(); - let sync_payload = |directory: &platform::Directory| -> io::Result<()> { - assert_eq!(directory.entry_id(SNAPSHOT_NAME).unwrap(), Some(snapshot)); - note(&payload); - Ok(()) - }; - // The release syncs the sandbox's directory through its handle, which holds the lock's file. - let lock = platform::file_id(&sandbox.join(LOCK_NAME)).unwrap(); - let sync_sandbox = |directory: &platform::Directory| -> io::Result<()> { - assert_eq!(directory.entry_id(LOCK_NAME).unwrap(), Some(lock)); - note(&sandbox); - Ok(()) - }; - for name in kept { - fs::write(sandbox.join(name), b"").unwrap(); - } - // A socket that a VMM on Linux left in the payload, which a file stands for. - if cfg!(unix) { - fs::write(&socket, b"").unwrap(); - } - let mut locked = SandboxStore::lock_existing(&sandbox, &files(), None).unwrap(); - assert_eq!( - store - .release_with(&mut locked, &|| {}, &sync_payload, &sync_sandbox) - .unwrap(), - Some(lease) - ); - drop(locked); - let mut expected = vec![(sandbox.clone(), false, false, true)]; - if cfg!(unix) { - expected.insert(0, (payload.clone(), false, true, true)); - } - assert_eq!(*synced.borrow(), expected); - assert!(!record.exists()); - assert!(fs::symlink_metadata(&socket).is_err()); - // A release under the lock that acquired the lease ends the lock's reservation before it - // removes the clone's files, such as the record of its failure, which ends the clone's - // launches until the release, so that one that fails after their removal leaves no launch to - // begin. - let Acquired { - lease: again, - mut locked, - } = store - .acquire( - generation, - sandbox.as_path(), - &files(), - &after(&published, 1), - None, - ) - .unwrap(); - assert_eq!(locked.reserved(), Some(again.launch())); - let failure = sandbox.join(CLONE_FAILURE_NAME); - fs::write(&failure, b"").unwrap(); - // A release that cannot make the socket's removal durable keeps the binding, and the lease. - if cfg!(unix) { - let error = store - .release_with( - &mut locked, - &|| {}, - &|_| Err(io::Error::other("the disk failed")), - &|_| Ok(()), - ) - .unwrap_err(); - assert_eq!( - error.message(), - format!("cannot sync the directory {}", payload.display()) - ); - assert_eq!(cause(&error), "the disk failed"); - assert_eq!(binding(&sandbox), Some(again.clone())); - assert_eq!(locked.reserved(), None); - assert!(fs::symlink_metadata(&failure).is_err()); - } - // A release that cannot make the binding's removal durable keeps the record, which a - // retirement removes once no binding names it. - let error = store - .release_with(&mut locked, &|| {}, &|_| Ok(()), &|_| { - Err(io::Error::other("the disk failed")) - }) - .unwrap_err(); - assert_eq!(locked.reserved(), None); - assert!(fs::symlink_metadata(&failure).is_err()); - drop(locked); - assert_eq!( - error.message(), - format!("cannot sync the directory {}", sandbox.display()) - ); - assert_eq!(cause(&error), "the disk failed"); - assert_eq!(binding(&sandbox), None); - assert_eq!(store.leases(generation).unwrap(), [again]); - assert_eq!(store.retire(generation, None).unwrap(), Retirement::Retired); -} - -#[test] -fn a_retirement_makes_each_step_durable_before_the_next() { - let (_directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let tombstones = store.dir().join(TOMBSTONES_NAME); - let tombstone = tombstones.join(format!("{generation}.json")); - let payload = store.payload_dir(generation); - // Each sync notes its directory, and whether the tombstone, the payload, and the - // generation's entry in the index existed then. - let synced = RefCell::new(Vec::new()); - let sync = |dir: &Path| -> Result<()> { - let indexed = store.index()?.get(generation).is_some(); - synced.borrow_mut().push(( - dir.to_owned(), - tombstone.exists(), - payload.exists(), - indexed, - )); - Ok(()) - }; - assert_eq!( - store.retire_with(generation, None, &sync).unwrap(), - Retirement::Retired - ); - assert_eq!( - *synced.borrow(), - [ - // The new directory of the tombstones, and the store's own entry, before the - // tombstone in it. - (store.dir().to_owned(), false, true, true), - (store.dir().parent().unwrap().to_owned(), false, true, true), - // The tombstone, before anything goes. - (tombstones.clone(), true, true, true), - // The payload's removal, before the index changes. - (store.dir().join(GENERATIONS_NAME), true, false, true), - // The index, before the tombstone goes. - (store.dir().to_owned(), true, false, false), - ] - ); - assert!(!tombstone.exists()); -} - -#[test] -fn a_retirement_makes_the_absence_of_a_binding_durable_before_it_removes_its_record() { - let (directory, store) = new_store(); - let root = filesystem::canonicalize(directory.path()).unwrap(); - let first = publish(&store, &ManifestSpec::template(), storage(1)); - let second = publish(&store, &ManifestSpec::template(), storage(2)); - let third = publish(&store, &ManifestSpec::template(), storage(3)); - let fourth = publish(&store, &ManifestSpec::template(), storage(4)); - let released = provision(&root, "released"); - let removed = provision(&root, "removed"); - let reprovisioned = provision(&root, "reprovisioned"); - let sandboxes = root.join("sandboxes"); - fs::create_dir(&sandboxes).unwrap(); - let removed_through_link = provision(&sandboxes, "removed"); - let released_lease = leased(&store, &first, &released); - let removed_lease = leased(&store, &second, &removed); - let linked_lease = leased(&store, &third, &removed_through_link); - let reprovisioned_lease = leased(&store, &fourth, &reprovisioned); - // The directory that holds a sandbox moves, and a link at its path names it, so the path that - // names the sandbox's lease runs through the link. - let moved = root.join("moved"); - fs::rename(&sandboxes, &moved).unwrap(); - link_dir(&moved, &sandboxes); - // A release that removed its binding, but did not make that durable, and removals of - // sandboxes, one of which is provisioned again at its path. - fs::remove_file(released.join(CLONE_BINDING_NAME)).unwrap(); - for sandbox in [&removed, &removed_through_link, &reprovisioned] { - let locked = SandboxStore::lock_existing(sandbox, &files(), None).unwrap(); - assert!(locked.remove().unwrap().is_none()); - } - assert_eq!(provision(&root, "reprovisioned"), reprovisioned); - let record = |lease: &Lease| { - store - .dir() - .join(LEASES_NAME) - .join(format!("{}.json", lease.id())) - }; - // A retirement that cannot make the binding's absence durable keeps its record, and the - // generation. - let unsynced_sandbox = |dir: &Path| { - if dir == released { - Err(Error::io("the disk failed")) - } else { - Ok(()) - } - }; - let error = store - .retire_with(first.generation(), None, &unsynced_sandbox) - .unwrap_err(); - assert_eq!(error.message(), "the disk failed"); - assert!(record(&released_lease).exists()); - assert!(store.snapshot_dir(&first).is_ok()); - // A record goes once the binding's absence is durable: in the sandbox's directory, if it is - // one, and in the directory that holds its entry, the nearest of its ancestors that remains, - // through the links that its path names. The directory of a sandbox that was provisioned - // again never held the binding, which went with the directory that its entry named before. - for (generation, lease, dirs) in [ - ( - &first, - &released_lease, - vec![released.as_path(), root.as_path()], - ), - (&second, &removed_lease, vec![root.as_path()]), - (&third, &linked_lease, vec![sandboxes.as_path()]), - ( - &fourth, - &reprovisioned_lease, - vec![reprovisioned.as_path(), root.as_path()], - ), - ] { - let path = record(lease); - let synced = RefCell::new(Vec::new()); - let sync = |synced_dir: &Path| -> Result<()> { - synced - .borrow_mut() - .push((synced_dir.to_owned(), path.exists())); - Ok(()) - }; - assert_eq!( - store - .retire_with(generation.generation(), None, &sync) - .unwrap(), - Retirement::Retired - ); - let mut expected = vec![(store.dir().join(TOMBSTONES_NAME), true)]; - expected.extend(dirs.into_iter().map(|dir| (dir.to_owned(), true))); - expected.extend([ - (store.dir().join(GENERATIONS_NAME), false), - (store.dir().to_owned(), false), - ]); - assert_eq!(*synced.borrow(), expected); - } - // A link to nothing hides the directory that held a sandbox, so the record stays. - let fifth = publish(&store, &ManifestSpec::template(), storage(5)); - let parked = root.join("parked"); - fs::create_dir(&parked).unwrap(); - let dangling = provision(&parked, "dangling"); - let dangling_lease = leased(&store, &fifth, &dangling); - let locked = SandboxStore::lock_existing(&dangling, &files(), None).unwrap(); - assert!(locked.remove().unwrap().is_none()); - let away = root.join("away"); - fs::rename(&parked, &away).unwrap(); - link_dir(&away, &parked); - fs::remove_dir(&away).unwrap(); - let error = store.retire(fifth.generation(), None).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Io, "{error}"); - assert!( - error.message().contains("cannot find the directory"), - "{error}" - ); - assert!(record(&dangling_lease).exists()); - assert!(store.snapshot_dir(&fifth).is_ok()); -} - -#[test] -fn a_lease_names_its_sandbox_by_its_path_with_its_links_resolved() { - let (directory, store) = new_store(); - let root = filesystem::canonicalize(directory.path()).unwrap(); - let first = publish(&store, &ManifestSpec::template(), storage(1)); - let second = publish(&store, &ManifestSpec::template(), storage(2)); - let sandboxes = root.join("sandboxes"); - fs::create_dir(&sandboxes).unwrap(); - let parent = provision(&sandboxes, "parent"); - let linked = provision(&sandboxes, "linked"); - // Paths through a directory that a `..` component passes through, and through a link, both of - // which go while the clones run. - let helper = root.join("helper"); - fs::create_dir(&helper).unwrap(); - let link = root.join("link"); - link_dir(&sandboxes, &link); - let mut leases = Vec::new(); - for (generation, sandbox, path) in [ - ( - &first, - &parent, - helper.join("..").join("sandboxes").join("parent"), - ), - (&second, &linked, link.join("linked")), - ] { - let acquired = store - .acquire( - generation.generation(), - &path, - &files(), - &after(generation, 1), - None, - ) - .unwrap(); - assert_eq!(acquired.lease.sandbox(), sandbox); - assert_eq!(acquired.locked.dir(), sandbox); - drop(acquired.locked); - assert_eq!(binding(sandbox), Some(acquired.lease.clone())); - leases.push(acquired.lease); - } - fs::remove_dir(&helper).unwrap(); - unlink_dir(&link); - // A retirement still finds both bindings, and keeps the generations. - for (generation, lease) in [&first, &second].into_iter().zip(leases) { - assert_eq!( - store.retire(generation.generation(), None).unwrap(), - Retirement::Draining(vec![lease]) - ); - } -} - -#[test] -fn an_acquisition_binds_the_sandbox_that_it_resolved_whatever_its_path_names_meanwhile() { - let (directory, store) = new_store(); - let root = filesystem::canonicalize(directory.path()).unwrap(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let (first, second) = (root.join("first"), root.join("second")); - fs::create_dir(&first).unwrap(); - fs::create_dir(&second).unwrap(); - let leased_sandbox = provision(&first, "sandbox"); - let other = provision(&second, "sandbox"); - let link = root.join("link"); - link_dir(&first, &link); - // The link names the other sandbox's directory once the acquisition has resolved its path, as - // it makes the lease's record durable. - let retargeted = Cell::new(false); - let retarget = |dir: &Path| -> Result<()> { - if dir == store.dir().join(LEASES_NAME) && !retargeted.replace(true) { - unlink_dir(&link); - link_dir(&second, &link); - } - Ok(()) - }; - let acquired = store - .acquire_with( - published.generation(), - link.join("sandbox"), - &files(), - &after(&published, 1), - None, - &retarget, - ) - .unwrap(); - assert!(retargeted.get()); - // The binding, and the lock, are those of the sandbox that the lease names. - assert_eq!(acquired.lease.sandbox(), leased_sandbox); - assert_eq!(acquired.locked.dir(), leased_sandbox); - drop(acquired.locked); - assert_eq!(binding(&leased_sandbox), Some(acquired.lease.clone())); - assert_eq!(binding(&other), None); - assert_eq!( - store.retire(published.generation(), None).unwrap(), - Retirement::Draining(vec![acquired.lease]) - ); -} - -#[test] -fn a_release_takes_the_lock_of_its_sandbox_through_the_path_of_its_lease() { - let (directory, store) = new_store(); - let root = filesystem::canonicalize(directory.path()).unwrap(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let (first, second) = (root.join("first"), root.join("second")); - fs::create_dir(&first).unwrap(); - fs::create_dir(&second).unwrap(); - let leased_sandbox = provision(&first, "sandbox"); - let other = provision(&second, "sandbox"); - let lease = leased(&store, &published, &leased_sandbox); - let other_lease = leased(&store, &published, &other); - let mut both = vec![lease.clone(), other_lease.clone()]; - both.sort_by_key(Lease::id); - let link = root.join("link"); - link_dir(&first, &link); - // A lock taken through a link refuses a release, which removes nothing, also once the link - // names another sandbox, whose lease a release would otherwise end without its lock: the - // directory that the link names then holds another lock file than the one locked. - let mut locked = SandboxStore::lock_existing(link.join("sandbox"), &files(), None).unwrap(); - for (retargeted, refusal) in [ - (false, "does not name it by the path of its lease"), - (true, "does not hold the lock of its sandbox"), - ] { - if retargeted { - unlink_dir(&link); - link_dir(&second, &link); - } - let error = store.release(&mut locked).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Other, "{error}"); - assert!(error.message().contains(refusal), "{error}"); - assert_eq!(binding(&leased_sandbox), Some(lease.clone())); - assert_eq!(binding(&other), Some(other_lease.clone())); - assert_eq!(store.leases(generation).unwrap(), both); - } - drop(locked); - // A lock taken through the lease's path ends it. - assert_eq!(release(&store, lease.sandbox()).unwrap(), Some(lease)); - assert_eq!(binding(&leased_sandbox), None); - assert_eq!(store.leases(generation).unwrap(), [other_lease]); -} - -/// A release whose path comes to name another sandbox, once a directory in the path is renamed, -/// and a link to the other sandbox's parent takes its place: only on Unix, since Windows refuses -/// to rename a directory that holds an open file, such as the lock's. -#[cfg(unix)] -#[test] -fn a_release_ends_the_lease_of_the_sandbox_whose_lock_it_holds_whatever_its_path_names_meanwhile() { - let (directory, store) = new_store(); - let root = filesystem::canonicalize(directory.path()).unwrap(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let (first, second) = (root.join("first"), root.join("second")); - fs::create_dir(&first).unwrap(); - fs::create_dir(&second).unwrap(); - let leased_sandbox = provision(&first, "sandbox"); - let other = provision(&second, "sandbox"); - let lease = leased(&store, &published, &leased_sandbox); - let other_lease = leased(&store, &published, &other); - let kept = [ - CLONE_SCRATCH_NAME, - CLONE_ACTIVATION_NAME, - CLONE_PROGRESS_NAME, - CLONE_FAILURE_NAME, - RESTORE_READY_NAME, - ]; - for sandbox in [&leased_sandbox, &other] { - for name in kept { - fs::write(sandbox.join(name), b"").unwrap(); - } - } - // Moves `parent` to `to`, and links `parent` to the other sandbox's parent. - let swap = |parent: &Path, to: &Path| { - fs::rename(parent, to).unwrap(); - link_dir(&second, parent); - }; - // The other sandbox's files, and its lease, stay. - let other_stays = || { - for name in kept { - assert!(other.join(name).exists(), "{name}"); - } - assert_eq!(binding(&other), Some(other_lease.clone())); - }; - // Once the release has checked the sandbox, its path comes to name the other sandbox, from - // the removal of its files on. - let moved = root.join("moved"); - let mut locked = SandboxStore::lock_existing(&leased_sandbox, &files(), None).unwrap(); - assert_eq!( - store - .release_with( - &mut locked, - &|| swap(&first, &moved), - &platform::Directory::sync, - &platform::Directory::sync - ) - .unwrap(), - Some(lease) - ); - drop(locked); - // The release removed the files and the binding of the sandbox whose lock it held. - let released = moved.join("sandbox"); - for name in kept { - assert!(fs::symlink_metadata(released.join(name)).is_err(), "{name}"); - } - assert_eq!(binding(&released), None); - other_stays(); - assert_eq!( - store.leases(generation).unwrap(), - std::slice::from_ref(&other_lease) - ); - // A release whose path names the other sandbox once the lock is taken, before the release - // begins, refuses, and removes nothing. - let again = leased(&store, &published, &released); - for name in kept { - fs::write(released.join(name), b"").unwrap(); - } - let mut locked = SandboxStore::lock_existing(&released, &files(), None).unwrap(); - let away = root.join("away"); - swap(&moved, &away); - let error = store.release(&mut locked).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Other, "{error}"); - assert!( - error - .message() - .contains("does not hold the lock of its sandbox"), - "{error}" - ); - drop(locked); - let kept_sandbox = away.join("sandbox"); - for name in kept { - assert!(kept_sandbox.join(name).exists(), "{name}"); - } - assert_eq!(binding(&kept_sandbox), Some(again.clone())); - other_stays(); - let mut both = vec![again, other_lease]; - both.sort_by_key(Lease::id); - assert_eq!(store.leases(generation).unwrap(), both); -} - -/// The payload of a generation, which holds the control sockets of the VMMs of its clones on -/// Linux, opens through the store's directory, following no link, or junction on Windows, at the -/// store's directory, at its directory of the generations, or at the payload, once it has found -/// that the store's directory holds the store of the caller's identity. On Unix, a store's -/// directory and its directory of the generations need only the right to be searched, which an -/// owner-only directory may grant without the right to read it. -#[test] -fn the_payload_of_a_generation_opens_through_no_link() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let id = store.id(); - let payload = store.payload_dir(generation); - let snapshot = platform::file_id(&payload.join(SNAPSHOT_NAME)).unwrap(); - let opened = open_payload(store.dir(), id, generation).unwrap().unwrap(); - assert_eq!(opened.entry_id(SNAPSHOT_NAME).unwrap(), Some(snapshot)); - drop(opened); - // A generation without a payload has none, nor has a store that is gone, or one of another - // identity. - assert!( - open_payload(store.dir(), id, store_generation(9)) - .unwrap() - .is_none() - ); - let missing = directory.path().join("missing"); - assert!(open_payload(&missing, id, generation).unwrap().is_none()); - let foreign = StoreId::from_bytes([0x5a; 16]); - assert!( - open_payload(store.dir(), foreign, generation) - .unwrap() - .is_none() - ); - // Another store in the store's place, whose generation of the same number has a payload, has - // none of the store's, nor has a directory that holds no store. - let other = Store::create(directory.path().join("other")).unwrap(); - fs::create_dir(other.payload_dir(generation)).unwrap(); - let moved = directory.path().join("moved"); - fs::rename(store.dir(), &moved).unwrap(); - fs::rename(other.dir(), store.dir()).unwrap(); - assert!(open_payload(store.dir(), id, generation).unwrap().is_none()); - assert!( - open_payload(store.dir(), other.id(), generation) - .unwrap() - .is_some() - ); - fs::rename(store.dir(), other.dir()).unwrap(); - fs::create_dir(store.dir()).unwrap(); - assert!(open_payload(store.dir(), id, generation).unwrap().is_none()); - fs::remove_dir(store.dir()).unwrap(); - // A link in the place of the store's directory, even to the store, of the directory of the - // generations, or of the payload, and a file in the place of the store's directory or of the - // payload, are refused. - let generations = store.dir().join(GENERATIONS_NAME); - let refused = |entry: &dyn std::fmt::Display| { - let error = open_payload(store.dir(), id, generation).err().unwrap(); - assert_eq!(error.kind(), ErrorKind::Io, "{error}"); - assert_eq!( - error.message(), - format!( - "cannot open the payload of the generation {}", - payload.display() - ) - ); - assert_eq!( - cause(&error), - format!("{entry} is a link, or not a directory") - ); - }; - link_dir(&moved, store.dir()); - refused(&store.dir().display()); - unlink_dir(store.dir()); - fs::write(store.dir(), b"").unwrap(); - refused(&store.dir().display()); - fs::remove_file(store.dir()).unwrap(); - fs::rename(&moved, store.dir()).unwrap(); - for (path, name, link) in [ - (&payload, generation.to_string(), true), - (&generations, GENERATIONS_NAME.to_owned(), true), - (&payload, generation.to_string(), false), - ] { - fs::rename(path, &moved).unwrap(); - if link { - link_dir(&moved, path); - } else { - fs::write(path, b"").unwrap(); - } - refused(&name); - if link { - unlink_dir(path); - } else { - fs::remove_file(path).unwrap(); - } - fs::rename(&moved, path).unwrap(); - } - assert!(open_payload(store.dir(), id, generation).unwrap().is_some()); - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - - let mode = |dir: &Path, mode| { - fs::set_permissions(dir, fs::Permissions::from_mode(mode)).unwrap(); - }; - for dir in [&generations, store.dir()] { - mode(dir, 0o300); - } - let opened = open_payload(store.dir(), id, generation); - for dir in [&generations, store.dir()] { - mode(dir, 0o700); - } - let opened = opened.unwrap().unwrap(); - assert_eq!(opened.entry_id(SNAPSHOT_NAME).unwrap(), Some(snapshot)); - opened.sync().unwrap(); - } -} - -/// A release whose generation's payload, or store's directory, a link takes the place of: before -/// the release begins, it refuses, and removes nothing; once it has opened the payload, it removes -/// the socket from the payload that it opened, and the link's target keeps its file. Only on Unix, -/// where the payload holds the socket of the clone's VMM. -#[cfg(unix)] -#[test] -fn a_release_removes_the_socket_from_the_payload_that_it_opened_whatever_its_path_names_meanwhile() -{ - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - let payload = store.payload_dir(generation); - let name = clone_socket_name(lease.launch()); - // A socket that the VMM left in the payload, which a file stands for, and a directory that - // holds a file of its name, to which a link takes the payload's place. - fs::write(payload.join(&name), b"").unwrap(); - let decoy = directory.path().join("decoy"); - fs::create_dir(&decoy).unwrap(); - fs::write(decoy.join(&name), b"").unwrap(); - // A directory of the store's layout, whose generation of the same number holds a file of the - // socket's name, to which a link takes the store's place. - let decoy_store = directory.path().join("decoy-store"); - let decoy_socket = decoy_store - .join(GENERATIONS_NAME) - .join(generation.to_string()) - .join(&name); - fs::create_dir_all(decoy_socket.parent().unwrap()).unwrap(); - fs::write(&decoy_socket, b"").unwrap(); - let moved = directory.path().join("moved"); - let swap = || { - fs::rename(&payload, &moved).unwrap(); - link_dir(&decoy, &payload); - }; - let restore = || { - unlink_dir(&payload); - fs::rename(&moved, &payload).unwrap(); - }; - let mut locked = SandboxStore::lock_existing(&sandbox, &files(), None).unwrap(); - let refused = |locked: &mut LockedStore, entry: &dyn std::fmt::Display| { - let error = store.release(locked).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Io, "{error}"); - assert_eq!( - error.message(), - format!( - "cannot open the payload of the generation {}", - payload.display() - ) - ); - assert_eq!( - cause(&error), - format!("{entry} is a link, or not a directory") - ); - assert!(decoy.join(&name).exists()); - assert!(decoy_socket.exists()); - assert_eq!(binding(&sandbox), Some(lease.clone())); - }; - swap(); - refused(&mut locked, &generation); - assert!(moved.join(&name).exists()); - restore(); - fs::rename(store.dir(), &moved).unwrap(); - link_dir(&decoy_store, store.dir()); - refused(&mut locked, &store.dir().display()); - unlink_dir(store.dir()); - fs::rename(&moved, store.dir()).unwrap(); - assert!(payload.join(&name).exists()); - assert_eq!( - store - .release_with( - &mut locked, - &swap, - &platform::Directory::sync, - &platform::Directory::sync - ) - .unwrap(), - Some(lease) - ); - drop(locked); - assert!(fs::symlink_metadata(moved.join(&name)).is_err()); - assert!(decoy.join(&name).exists()); - assert!(decoy_socket.exists()); - assert_eq!(binding(&sandbox), None); - restore(); - assert!(store.leases(generation).unwrap().is_empty()); -} - -/// A sandbox's store that clears the runtime record of its clone's VM removes the VMM's control -/// socket through the payload of its generation, as a recovery of its launch does: it refuses a -/// link in the place of the payload or of the template store's directory, keeping the record, and -/// leaves the files of another store in the template store's place. Only on Unix, where the socket -/// is a file. -#[cfg(unix)] -#[test] -fn a_sandbox_s_store_removes_the_socket_of_its_clone_through_no_link() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - let payload = store.payload_dir(generation); - let socket = store.clone_socket_path(&lease); - let name = clone_socket_name(lease.launch()); - fs::write(&socket, b"").unwrap(); - let decoy = directory.path().join("decoy"); - fs::create_dir(&decoy).unwrap(); - fs::write(decoy.join(&name), b"").unwrap(); - // Another store, whose generation of the same number holds a file of the socket's name. - let other = Store::create(directory.path().join("other")).unwrap(); - fs::create_dir(other.payload_dir(generation)).unwrap(); - let other_socket = other.payload_dir(generation).join(&name); - fs::write(&other_socket, b"").unwrap(); - // The runtime record of the clone's VM, whose control endpoint is the socket. - let record = json!({ - "format": FORMAT, - "launch": lease.launch().to_string(), - "process": { "pid": 42, "startTime": 4242, "boot": "ab".repeat(16) }, - "endpoint": socket.to_str().unwrap(), - "runtime": serde_json::to_value(runtime()).unwrap(), - }); - let write_record = || { - write_private(&sandbox.join(RUNTIME_NAME), record.to_string().as_bytes()).unwrap(); - }; - write_record(); - let mut locked = SandboxStore::lock_existing(&sandbox, &files(), None).unwrap(); - let refused = |locked: &mut LockedStore, entry: &dyn std::fmt::Display| { - let error = locked.clear_runtime().unwrap_err(); - assert_eq!( - error.message(), - format!( - "cannot open the payload of the generation {}", - payload.display() - ) - ); - assert_eq!( - cause(&error), - format!("{entry} is a link, or not a directory") - ); - assert!(decoy.join(&name).exists()); - assert!(other_socket.exists()); - assert!(sandbox.join(RUNTIME_NAME).exists()); - }; - let moved = directory.path().join("moved"); - fs::rename(&payload, &moved).unwrap(); - link_dir(&decoy, &payload); - refused(&mut locked, &generation); - assert!(moved.join(&name).exists()); - unlink_dir(&payload); - fs::rename(&moved, &payload).unwrap(); - fs::rename(store.dir(), &moved).unwrap(); - link_dir(other.dir(), store.dir()); - refused(&mut locked, &store.dir().display()); - unlink_dir(store.dir()); - // Another store in the store's place holds no socket of the clone's, so the record goes, and - // the other store's file stays, as does the socket in the store's own directory. - fs::rename(other.dir(), store.dir()).unwrap(); - locked.clear_runtime().unwrap(); - assert!(!sandbox.join(RUNTIME_NAME).exists()); - fs::rename(store.dir(), other.dir()).unwrap(); - fs::rename(&moved, store.dir()).unwrap(); - assert!(other_socket.exists()); - assert!(socket.exists()); - write_record(); - locked.clear_runtime().unwrap(); - assert!(fs::symlink_metadata(&socket).is_err()); - assert!(!sandbox.join(RUNTIME_NAME).exists()); - assert!(decoy.join(&name).exists()); - assert!(other_socket.exists()); - drop(locked); - assert_eq!(release(&store, &sandbox).unwrap(), Some(lease)); -} - -#[test] -fn a_retirement_that_finds_its_generation_unindexed_makes_the_index_durable_first() { - let (_directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - // A retirement that a crash interrupted once it had written the index, which it may not have - // made durable, leaves the tombstone. - store.tombstone(generation, None).unwrap(); - fs::remove_dir_all(store.payload_dir(generation)).unwrap(); - remove(&store, generation); - let tombstone = store - .dir() - .join(TOMBSTONES_NAME) - .join(format!("{generation}.json")); - let synced = RefCell::new(Vec::new()); - let sync = |dir: &Path| -> Result<()> { - synced - .borrow_mut() - .push((dir.to_owned(), tombstone.exists())); - Ok(()) - }; - assert_eq!( - store.retire_with(generation, None, &sync).unwrap(), - Retirement::Retired - ); - assert_eq!(*synced.borrow(), [(store.dir().to_owned(), true)]); - assert!(!tombstone.exists()); -} - -#[test] -fn a_retirement_keeps_a_tombstone_that_is_not_the_stores() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - // A retirement that a crash interrupted once it had written the index leaves the tombstone, - // which retiring the generation again removes only once it has checked it. - store.tombstone(generation, None).unwrap(); - fs::remove_dir_all(store.payload_dir(generation)).unwrap(); - remove(&store, generation); - let tombstone = store - .dir() - .join(TOMBSTONES_NAME) - .join(format!("{generation}.json")); - let valid = json_of(&tombstone); - let foreign: [(Change, ErrorCode); 2] = [ - ( - |value| value["store"] = json!("0".repeat(32)), - ErrorCode::Io, - ), - ( - |value| value["format"] = json!(2), - ErrorCode::FormatUnsupported, - ), - ]; - for (change, code) in foreign { - let mut value = valid.clone(); - change(&mut value); - fs::write(&tombstone, value.to_string()).unwrap(); - let error = store.retire(generation, None).unwrap_err(); - assert_eq!(error.code(), code, "{error}"); - assert_eq!(json_of(&tombstone), value); - } - // Nor does it remove a link in place of the tombstone. - fs::write(&tombstone, valid.to_string()).unwrap(); - let moved = directory.path().join("tombstone.json"); - fs::rename(&tombstone, &moved).unwrap(); - if link_file(&moved, &tombstone) { - let error = store.retire(generation, None).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Io, "{error}"); - assert!(fs::symlink_metadata(&tombstone).unwrap().is_symlink()); - fs::remove_file(&tombstone).unwrap(); - } - fs::rename(&moved, &tombstone).unwrap(); - assert_eq!(store.retire(generation, None).unwrap(), Retirement::Retired); - assert!(!tombstone.exists()); -} - -#[test] -fn directories_are_synced_where_the_platform_can() { - let directory = tempfile::tempdir().unwrap(); - sync_dir(directory.path()).unwrap(); - // On Unix, a sync opens the directory, which must exist. - #[cfg(unix)] - { - let error = sync_dir(&directory.path().join("missing")).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Io, "{error}"); - } -} - -#[test] -fn records_keep_their_json_form() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - let expected = json!({ - "format": 1, - "store": store.id().to_string(), - "lease": lease.id().to_string(), - "generation": generation.get().get(), - "key": published.key().to_string(), - "sandbox": sandbox.to_str().unwrap(), - "launch": lease.launch().as_str(), - }); - let record = store - .dir() - .join(LEASES_NAME) - .join(format!("{}.json", lease.id())); - assert_eq!(json_of(&record), expected); - assert_eq!(json_of(&sandbox.join(CLONE_BINDING_NAME)), expected); - store.tombstone(generation, None).unwrap(); - assert_eq!( - json_of( - &store - .dir() - .join(TOMBSTONES_NAME) - .join(format!("{generation}.json")) - ), - json!({ - "format": 1, - "store": store.id().to_string(), - "generation": generation.get().get(), - }) - ); - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - - for name in [LEASES_NAME, TOMBSTONES_NAME] { - let mode = fs::metadata(store.dir().join(name)) - .unwrap() - .permissions() - .mode(); - assert_eq!(mode & 0o777, 0o700, "{name}"); - } - } - // A lease's identity is its text form. - let text = lease.id().to_string(); - assert_eq!(text.len(), 32); - assert_eq!(text.parse::().unwrap(), lease.id()); - assert_eq!(text.to_uppercase().parse::().unwrap(), lease.id()); - assert_eq!(LeaseId::from_bytes(*lease.id().as_bytes()), lease.id()); - for invalid in ["", "0123", &"g".repeat(32)] { - let error = invalid.parse::().unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported, "{invalid}"); - } -} - -#[test] -fn malformed_records_and_foreign_entries_fail_clearly() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - let record = store - .dir() - .join(LEASES_NAME) - .join(format!("{}.json", lease.id())); - let bound = sandbox.join(CLONE_BINDING_NAME); - let valid = json_of(&record); - let rewrite = |path: &Path, change: Change| { - let mut value = valid.clone(); - change(&mut value); - fs::write(path, value.to_string()).unwrap(); - }; - // A record of another format is neither read nor migrated. - rewrite(&record, |value| value["format"] = json!(2)); - let error = store.leases(generation).unwrap_err(); - assert_eq!(error.code(), ErrorCode::FormatUnsupported, "{error}"); - // Records that are not leases. - let malformed: [(Change, &str); 7] = [ - ( - |value| value["generation"] = json!(0), - "it leases store generation 0", - ), - ( - |value| value["sandbox"] = json!("relative"), - "is not an absolute path", - ), - (|value| value["key"] = json!("00"), "its key is invalid"), - ( - |value| value["lease"] = json!("00"), - "its identity is invalid", - ), - ( - |value| value["launch"] = json!("00"), - "its launch is invalid", - ), - // A lease reserves its clone's launch. - ( - |value| { - value.as_object_mut().unwrap().remove("launch"); - }, - "is malformed", - ), - (|value| value["extra"] = json!(1), "is malformed"), - ]; - for (change, problem) in malformed { - rewrite(&bound, change); - let error = Lease::of(&SandboxStore::open(&sandbox).unwrap()).unwrap_err(); - assert_eq!(error.code(), ErrorCode::Io, "{error}"); - assert!(error.message().contains("is malformed"), "{error}"); - assert!(error.message().contains(problem), "{error}"); - } - // A key that is not a key's identity fails as its parse does. - rewrite(&bound, |value| value["key"] = json!("00")); - let error = Lease::of(&SandboxStore::open(&sandbox).unwrap()).unwrap_err(); - let parse = "00".parse::().unwrap_err(); - assert!( - error - .message() - .ends_with(&format!("its key is invalid: {}", parse.message())), - "{error}" - ); - fs::write(&bound, valid.to_string()).unwrap(); - // A record of another lease than its name names, or of another store. - let other_lease: Change = |value| value["lease"] = json!("0".repeat(32)); - let other_store: Change = |value| value["store"] = json!("0".repeat(32)); - for change in [other_lease, other_store] { - rewrite(&record, change); - let error = store.leases(generation).unwrap_err(); - assert_eq!(error.code(), ErrorCode::Io, "{error}"); - assert!(error.message().contains("is malformed"), "{error}"); - } - fs::write(&record, valid.to_string()).unwrap(); - assert_eq!(store.leases(generation).unwrap(), [lease]); - // Entries that are not the store's, which a retirement refuses before it changes anything. - let notes = store.dir().join(LEASES_NAME).join("notes.txt"); - fs::write(¬es, b"").unwrap(); - for error in [ - store.leases(generation).unwrap_err(), - store.retire(generation, None).unwrap_err(), - ] { - assert_eq!(error.code(), ErrorCode::ForeignFiles, "{error}"); - assert!( - error - .message() - .ends_with("holds entries that are not the store's: notes.txt"), - "{error}" - ); - } - assert!(store.tombstones().unwrap().is_empty()); - fs::remove_file(¬es).unwrap(); - // A tombstone of another store, or of another format, which neither an acquisition nor a - // listing or a selection takes for a quarantine, and entries that are not tombstones. - store.tombstone(generation, None).unwrap(); - let tombstones = store.dir().join(TOMBSTONES_NAME); - let tombstone = tombstones.join(format!("{generation}.json")); - let valid_tombstone = json_of(&tombstone); - let other = provision(directory.path(), "other"); - let foreign_tombstones: [(Change, ErrorCode); 2] = [ - ( - |value| value["store"] = json!("0".repeat(32)), - ErrorCode::Io, - ), - ( - |value| value["format"] = json!(2), - ErrorCode::FormatUnsupported, - ), - ]; - for (change, code) in foreign_tombstones { - let mut value = valid_tombstone.clone(); - change(&mut value); - fs::write(&tombstone, value.to_string()).unwrap(); - for error in [ - refusal(&store, generation, &other, &after(&published, 1)), - store.tombstones().unwrap_err(), - store - .select(published.key(), &after(&published, 1)) - .unwrap_err(), - ] { - assert_eq!(error.code(), code, "{error}"); - } - } - fs::write(tombstones.join("+1.json"), b"").unwrap(); - for error in [ - store.tombstones().unwrap_err(), - store - .select(published.key(), &after(&published, 1)) - .unwrap_err(), - ] { - assert_eq!(error.code(), ErrorCode::ForeignFiles, "{error}"); - assert!(error.message().ends_with(": +1.json"), "{error}"); - } -} - -#[test] -fn the_directories_of_leases_and_tombstones_refuse_links() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - store.tombstone(generation, None).unwrap(); - for name in [LEASES_NAME, TOMBSTONES_NAME] { - let path = store.dir().join(name); - let moved = path.with_extension("moved"); - fs::rename(&path, &moved).unwrap(); - link_dir(&moved, &path); - let mut errors = vec![store.retire(generation, None).unwrap_err()]; - if name == LEASES_NAME { - errors.push(store.leases(generation).unwrap_err()); - errors.push(release(&store, &sandbox).unwrap_err()); - } else { - errors.push(store.tombstones().unwrap_err()); - errors.push( - store - .select(published.key(), &after(&published, 1)) - .unwrap_err(), - ); - } - for error in errors { - assert!( - error.message().ends_with("is not a plain directory"), - "{name}: {error}" - ); - } - unlink_dir(&path); - fs::rename(&moved, &path).unwrap(); - } - // Nothing changed meanwhile. - assert_eq!(binding(&sandbox), Some(lease.clone())); - assert_eq!(store.leases(generation).unwrap(), [lease]); - assert_eq!(store.tombstones().unwrap(), [generation]); -} - -/// A transaction in a process of its own, which [`transaction_role`] runs. -struct Transaction { - /// The process. - child: Child, - /// Its standard input, a line on which starts the transaction. - input: ChildStdin, - /// Its standard output. - output: BufReader, -} - -impl Transaction { - /// Starts the transaction `action` of the store generation `generation` of the store in - /// `store`, for the sandbox in `sandbox`, if any, and returns once it is ready to run. - fn start( - store: &Path, - action: &str, - generation: StoreGeneration, - sandbox: Option<&Path>, - ) -> Self { - let mut command = Command::new(env::current_exe().unwrap()); - command - .args(role("transaction_role")) - .env(ROLE_STORE, store) - .env(ROLE_ACTION, action) - .env(ROLE_GENERATION, generation.to_string()) - .stdin(Stdio::piped()) - .stdout(Stdio::piped()); - if let Some(sandbox) = sandbox { - command.env(ROLE_SANDBOX, sandbox); - } - let mut child = supervisor::spawn_command(&mut command).unwrap(); - let input = child.stdin.take().unwrap(); - let mut output = BufReader::new(child.stdout.take().unwrap()); - let mut line = String::new(); - while line.trim_end() != READY { - line.clear(); - let read = output.read_line(&mut line).unwrap(); - assert_ne!(read, 0, "the transaction ended before it was ready"); - } - Self { - child, - input, - output, - } - } - - /// Lets the transaction run. - fn run(&mut self) { - self.input.write_all(b"run\n").unwrap(); - self.input.flush().unwrap(); - } - - /// Waits for the transaction to end, and returns how it ended, as [`transaction_role`] - /// prints it. - fn outcome(mut self) -> String { - let mut rest = String::new(); - self.output.read_to_string(&mut rest).unwrap(); - assert!(self.child.wait().unwrap().success(), "{rest}"); - rest.lines() - .find_map(|line| line.strip_prefix(OUTCOME)) - .expect("the transaction printed its outcome") - .to_owned() - } -} - -impl Drop for Transaction { - fn drop(&mut self) { - let _ = self.child.kill(); - let _ = self.child.wait(); - } -} - -/// Runs `transactions` at once, with `other` amid them, and returns how each of `transactions` -/// ended, in their order, and how `other` ended. -fn race(mut transactions: Vec, mut other: Transaction) -> (Vec, String) { - let middle = transactions.len() / 2; - for (index, transaction) in transactions.iter_mut().enumerate() { - if index == middle { - other.run(); - } - transaction.run(); - } - let outcomes = transactions.into_iter().map(Transaction::outcome).collect(); - (outcomes, other.outcome()) -} - -/// Returns the identities of the leases that `outcomes` report as acquired, sorted, and separated -/// by spaces. -fn acquired(outcomes: &[String]) -> String { - let mut ids: Vec<&str> = outcomes - .iter() - .filter_map(|outcome| outcome.strip_prefix("acquired ")) - .collect(); - ids.sort_unstable(); - ids.join(" ") -} - -/// A transaction in a process of its own, which [`Transaction::start`] runs: once a line arrives on -/// its standard input, it runs the transaction that [`ROLE_ACTION`] names, of the store generation -/// [`ROLE_GENERATION`] of the store in [`ROLE_STORE`], for the sandbox in [`ROLE_SANDBOX`], and -/// prints how it ended: `acquired ID`, `released ID` or `none`, `tombstoned`, `retired` or -/// `draining ID...`, or the code of its error. -#[test] -#[ignore = "a transaction that the lease tests run in a process of its own"] -fn transaction_role() { - let (Some(dir), Ok(action), Ok(generation)) = ( - env::var_os(ROLE_STORE), - env::var(ROLE_ACTION), - env::var(ROLE_GENERATION), - ) else { - return; - }; - let store = Store::open(dir).unwrap(); - let generation = StoreGeneration::new(generation.parse().unwrap()); - let sandbox = env::var_os(ROLE_SANDBOX).map(PathBuf::from); - // A second after the capture of the generation, on the host that captured it. - let now = store - .index() - .unwrap() - .get(generation) - .map(|indexed| after(indexed, 1)); - // The test harness has printed the test's name without ending the line. - println!("\n{READY}"); - let mut line = String::new(); - std::io::stdin().read_line(&mut line).unwrap(); - let deadline = Some(Instant::now() + Duration::from_secs(30)); - let code = |error: Error| error.code().to_string(); - let outcome = match action.as_str() { - "acquire" => store - .acquire( - generation, - sandbox.unwrap(), - &files(), - &now.unwrap(), - deadline, - ) - .map_or_else(code, |acquired| format!("acquired {}", acquired.lease.id())), - "release" => { - let mut locked = - SandboxStore::lock_existing(sandbox.unwrap(), &files(), deadline).unwrap(); - match store.release(&mut locked) { - Ok(Some(lease)) => format!("released {}", lease.id()), - Ok(None) => "none".to_owned(), - Err(error) => code(error), - } - } - "tombstone" => store - .tombstone(generation, deadline) - .map_or_else(code, |()| "tombstoned".to_owned()), - "retire" => match store.retire(generation, deadline) { - Ok(Retirement::Retired) => "retired".to_owned(), - Ok(Retirement::Draining(leases)) => format!("draining {}", ids(&leases)), - Err(error) => code(error), - }, - _ => panic!("unknown transaction {action}"), - }; - println!("{OUTCOME}{outcome}"); -} - -#[test] -fn processes_that_race_to_lease_one_sandbox_lease_it_once() { - /// The processes that race. - const PROCESSES: usize = 6; - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let mut acquisitions: Vec = (0..PROCESSES) - .map(|_| Transaction::start(store.dir(), "acquire", generation, Some(&sandbox))) - .collect(); - for acquisition in &mut acquisitions { - acquisition.run(); - } - let outcomes: Vec = acquisitions.into_iter().map(Transaction::outcome).collect(); - let leases = store.leases(generation).unwrap(); - assert_eq!(leases.len(), 1, "{outcomes:?}"); - assert_eq!(acquired(&outcomes), leases[0].id().to_string()); - assert_eq!( - outcomes - .iter() - .filter(|outcome| *outcome == "sandbox-leased") - .count(), - PROCESSES - 1, - "{outcomes:?}" - ); - assert_eq!(binding(&sandbox), Some(leases[0].clone())); -} - -#[test] -fn a_tombstone_that_races_acquisitions_in_other_processes_keeps_the_leases_before_it() { - /// The clones whose acquisitions race the tombstone. - const CLONES: usize = 6; - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandboxes: Vec = (0..CLONES) - .map(|clone| provision(directory.path(), &format!("clone-{clone}"))) - .collect(); - let acquisitions = sandboxes - .iter() - .map(|sandbox| Transaction::start(store.dir(), "acquire", generation, Some(sandbox))) - .collect(); - let tombstone = Transaction::start(store.dir(), "tombstone", generation, None); - let (outcomes, tombstoned) = race(acquisitions, tombstone); - assert_eq!(tombstoned, "tombstoned"); - // Each acquisition leased the generation before the tombstone, or found it quarantined after. - for (sandbox, outcome) in sandboxes.iter().zip(&outcomes) { - match outcome.strip_prefix("acquired ") { - Some(id) => assert_eq!(binding(sandbox).unwrap().id().to_string(), id), - None => { - assert_eq!(outcome, "template-tombstoned"); - assert_eq!(binding(sandbox), None); - let error = refusal(&store, generation, sandbox, &after(&published, 1)); - assert_eq!(error.code(), ErrorCode::TemplateTombstoned, "{error}"); - } - } - } - // The leases granted before it keep working. - assert_eq!(ids(&store.leases(generation).unwrap()), acquired(&outcomes)); - assert_eq!(store.tombstones().unwrap(), [generation]); -} - -#[test] -fn a_retirement_that_races_acquisitions_and_releases_in_other_processes_waits_for_its_leases() { - /// The clones whose acquisitions and releases race the retirements. - const CLONES: usize = 6; - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandboxes: Vec = (0..CLONES) - .map(|clone| provision(directory.path(), &format!("clone-{clone}"))) - .collect(); - let start = |action: &str, sandbox: Option<&Path>| { - Transaction::start(store.dir(), action, generation, sandbox) - }; - let acquisitions = sandboxes - .iter() - .map(|sandbox| start("acquire", Some(sandbox))) - .collect(); - let (outcomes, retired) = race(acquisitions, start("retire", None)); - // The retirement drains the leases of the acquisitions before it, and those after it find - // the generation quarantined, or gone if none leased it before. - let granted = acquired(&outcomes); - if granted.is_empty() { - assert_eq!(retired, "retired"); - assert!( - outcomes.iter().all(|outcome| outcome == "not-found"), - "{outcomes:?}" - ); - } else { - assert_eq!(retired, format!("draining {granted}")); - assert!( - outcomes.iter().all(|outcome| outcome.starts_with("acquired ") - || outcome == "template-tombstoned"), - "{outcomes:?}" - ); - assert!(store.snapshot_dir(&published).is_ok()); - } - assert_eq!(ids(&store.leases(generation).unwrap()), granted); - // The clones release their leases while another retirement runs, which deletes the - // generation only once none remains. - let releases = sandboxes - .iter() - .map(|sandbox| start("release", Some(sandbox))) - .collect(); - let (released, retired) = race(releases, start("retire", None)); - for (outcome, acquisition) in released.iter().zip(&outcomes) { - match acquisition.strip_prefix("acquired ") { - Some(id) => assert_eq!(outcome, &format!("released {id}")), - None => assert_eq!(outcome, "none"), - } - } - assert!( - retired == "retired" || retired.starts_with("draining "), - "{retired}" - ); - assert_eq!(store.retire(generation, None).unwrap(), Retirement::Retired); - assert!(!store.payload_dir(generation).exists()); - assert_eq!(store.index().unwrap().get(generation), None); - assert!(store.leases(generation).unwrap().is_empty()); - assert!(store.tombstones().unwrap().is_empty()); -} - -/// A clone's orchestrator in a process of its own, which [`run_clone_role`] runs: it leases the -/// store generation [`ROLE_GENERATION`] of the store in [`ROLE_STORE`] to the sandbox in -/// [`ROLE_SANDBOX`], and launches the clone with a [`vmm_role`] in [`ROLE_VMM`] as its VMM, or it -/// releases the sandbox's lease; and it exits without unwinding after the step that [`ROLE_CRASH`] -/// names, as a crash would, once it has printed the reference of the VMM that it spawned, if any. -#[test] -#[ignore = "a clone's orchestrator that the lease tests run in a process of its own"] -fn clone_role() { - let (Some(dir), Ok(generation), Some(sandbox), Ok(step), Some(vmm)) = ( - env::var_os(ROLE_STORE), - env::var(ROLE_GENERATION), - env::var_os(ROLE_SANDBOX).map(PathBuf::from), - env::var(ROLE_CRASH), - env::var_os(ROLE_VMM).map(PathBuf::from), - ) else { - return; - }; - let store = Store::open(dir).unwrap(); - let generation = StoreGeneration::new(generation.parse().unwrap()); - // The step's transaction crashes as it makes its write in `target` durable, or, for a - // release, the removal of the binding, in the sandbox's directory. - let target = match step.as_str() { - "after-record" => Some(store.dir().join(LEASES_NAME)), - "after-binding" => Some(sandbox.clone()), - _ => None, - }; - let sync = |dir: &Path| -> Result<()> { - if target.as_deref() == Some(dir) { - crash(&step); - } - sync_dir(dir) - }; - if step == "releasing" { - let mut locked = SandboxStore::lock_existing(&sandbox, &files(), None).unwrap(); - store - .release_with(&mut locked, &|| {}, &platform::Directory::sync, &|_| { - crash(&step) - }) - .unwrap(); - panic!("the release did not make the removal of its binding durable"); - } - let now = after(store.index().unwrap().get(generation).unwrap(), 1); - let mut acquired = store - .acquire_with(generation, sandbox.clone(), &files(), &now, None, &sync) - .unwrap(); - let reserved = acquired.lease.launch().clone(); - let command = |_: &str| -> Result> { - Ok(VmmCommand::new( - env::current_exe().unwrap(), - role("vmm_role"), - &vmm, - )) - }; - match step.as_str() { - "acquired" => crash(&step), - "before-spawn" => { - let _ = supervisor::launch( - &mut acquired.locked, - &reserved, - &runtime(), - &CAPABILITY, - |_| crash(&step), - ); - } - "after-spawn" => { - // The steps of `supervisor::launch`, up to the spawn. - let mut launch = acquired - .locked - .begin_launch(&reserved, &runtime(), &CAPABILITY) - .unwrap(); - let command = command(launch.endpoint()).unwrap(); - let log = launch.take_log().unwrap(); - let launched = supervisor::spawn(&command, &CAPABILITY, log).unwrap(); - print_reference(launched.process().unwrap().reference()); - crash(&step); - } - "after-identity" => { - let starting = supervisor::launch( - &mut acquired.locked, - &reserved, - &runtime(), - &CAPABILITY, - command, - ) - .unwrap(); - print_reference(starting.process().reference()); - crash(&step); - } - "committed" => { - let starting = supervisor::launch( - &mut acquired.locked, - &reserved, - &runtime(), - &CAPABILITY, - command, - ) - .unwrap(); - let running = starting.commit().unwrap(); - print_reference(running.record().process()); - crash(&step); - } - _ => {} - } - panic!("the clone's orchestrator did not crash after {step}"); -} - -/// The VMM of a clone of [`clone_role`]: reads its capability as OpenVMM does, then runs until a -/// file named [`EXIT_MARKER`] appears in its working directory, which [`VMM_MARKER`] marks, or for -/// [`VMM_LIFETIME`] at most. -#[test] -#[ignore = "a VMM that the lease tests run in a process of its own"] -fn vmm_role() { - let directory = env::current_dir().unwrap(); - if !directory.join(VMM_MARKER).exists() { - return; - } - let mut capability = [0; CAPABILITY_LEN]; - std::io::stdin().read_exact(&mut capability).unwrap(); - assert_eq!(capability, CAPABILITY); - let started = Instant::now(); - while !directory.join(EXIT_MARKER).exists() && started.elapsed() < VMM_LIFETIME { - thread::sleep(Duration::from_millis(10)); - } -} - -/// Prints `reference` as JSON, on a line of its own, after [`REFERENCE`]. -fn print_reference(reference: &ProcessRef) { - println!("\n{REFERENCE}{}", serde_json::to_string(reference).unwrap()); -} - -/// Ends this process at once, once it has printed that it crashes after `step`, without unwinding -/// or running destructors, as a crash would. -fn crash(step: &str) -> ! { - println!("\n{CRASHED}{step}"); - std::io::stdout().flush().unwrap(); - std::process::exit(0) -} - -/// Runs [`clone_role`] for the generation `generation` of `store`, the sandbox in `sandbox`, and a -/// VMM in `vmm`, until it crashes after `step`, and returns the reference of the VMM that it -/// spawned, if it spawned one. -fn run_clone_role( - step: &str, - store: &Store, - generation: StoreGeneration, - sandbox: &Path, - vmm: &Path, -) -> Option { - let mut command = Command::new(env::current_exe().unwrap()); - command - .args(role("clone_role")) - .env(ROLE_STORE, store.dir()) - .env(ROLE_GENERATION, generation.to_string()) - .env(ROLE_SANDBOX, sandbox) - .env(ROLE_CRASH, step) - .env(ROLE_VMM, vmm) - .stdin(Stdio::null()) - .stdout(Stdio::piped()) - .stderr(Stdio::piped()); - let output = supervisor::spawn_command(&mut command) - .unwrap() - .wait_with_output() - .unwrap(); - let stdout = String::from_utf8_lossy(&output.stdout); - let crashed = format!("{CRASHED}{step}"); - assert!( - output.status.success() && stdout.lines().any(|line| line == crashed), - "{}: {stdout}{}", - output.status, - String::from_utf8_lossy(&output.stderr) - ); - stdout - .lines() - .find_map(|line| line.strip_prefix(REFERENCE)) - .map(|reference| serde_json::from_str(reference).unwrap()) -} - -/// The working directory of a [`vmm_role`], which it marks. Dropping it ends the VMM, and waits -/// for it to exit, so that a failed test leaves no process behind. -struct VmmDir { - /// The reference of the VMM, once the test knows it. - reference: Option, - /// The directory, which is removed after the VMM exits. - dir: tempfile::TempDir, -} - -impl VmmDir { - /// Creates the directory, and marks it. - fn new() -> Self { - let dir = tempfile::tempdir().unwrap(); - fs::write(dir.path().join(VMM_MARKER), b"").unwrap(); - Self { - reference: None, - dir, - } - } - - /// Returns the path of the directory. - fn path(&self) -> &Path { - self.dir.path() - } - - /// Asks the VMM to exit. - fn end(&self) { - fs::write(self.path().join(EXIT_MARKER), b"").unwrap(); - } -} - -impl Drop for VmmDir { - fn drop(&mut self) { - let _ = fs::write(self.path().join(EXIT_MARKER), b""); - if let Some(reference) = &self.reference - && let Ok(Opened::Running(process)) = Process::open(reference) - && !process - .wait(Instant::now() + TIMEOUT) - .is_ok_and(|exited| exited.is_some()) - { - let _ = process.kill(); - let _ = process.wait(Instant::now() + TIMEOUT); - } - } -} - -/// A clone whose orchestrator, [`clone_role`], crashed after a step of the clone's launch, in a -/// fresh temporary directory: its VMM's working directory, its template store and sandbox, and -/// the lease that the sandbox holds. -struct Crashed { - /// The working directory of the clone's VMM, which ends the VMM as it drops, before the rest. - vmm: VmmDir, - /// The template store. - store: Store, - /// The sandbox's directory. - sandbox: PathBuf, - /// The lease that the sandbox holds. - lease: Lease, - /// The temporary directory of the store and the sandbox. - _directory: tempfile::TempDir, -} - -impl Crashed { - /// Publishes a generation in a new store, and leases it to a new sandbox, whose clone - /// [`clone_role`] launches until it crashes after `step`. - fn after(step: &str) -> Self { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let sandbox = provision(directory.path(), "sandbox"); - let mut vmm = VmmDir::new(); - vmm.reference = run_clone_role(step, &store, published.generation(), &sandbox, vmm.path()); - let lease = binding(&sandbox).expect("the crash kept the clone's lease"); - Self { - vmm, - store, - sandbox, - lease, - _directory: directory, - } - } - - /// Takes the lock of the clone's sandbox, as its restarted orchestrator does. - fn lock(&self) -> LockedStore { - SandboxStore::lock_existing(&self.sandbox, &files(), Some(Instant::now() + TIMEOUT)) - .unwrap() - } - - /// Checks that the store refuses to release the lease of the sandbox of `locked`, whose VM - /// may still run, and that the sandbox keeps it. - fn refuses_release(&self, locked: &mut LockedStore) { - let error = self.store.release(locked).unwrap_err(); - assert_eq!(error.code(), ErrorCode::SandboxRunning, "{error}"); - assert_eq!(binding(&self.sandbox), Some(self.lease.clone())); - } - - /// Checks that the store releases the lease of the sandbox of `locked`, and that the lease is - /// gone. - fn releases(&self, locked: &mut LockedStore) { - assert_eq!( - self.store.release(locked).unwrap(), - Some(self.lease.clone()) - ); - assert_eq!(binding(&self.sandbox), None); - assert!( - self.store - .leases(self.lease.generation()) - .unwrap() - .is_empty() - ); - } -} - -/// Reattaches to the VM of the sandbox of `locked` until the result is determinate, as a caller -/// retries an indeterminate reattach, and returns it. -/// -/// On Windows, a scanner that briefly opens the log of a launch can keep its claim held for a -/// moment. -fn reattach_until_determinate(locked: &mut LockedStore) -> Reattached { - let deadline = Instant::now() + TIMEOUT; - loop { - match supervisor::reattach(locked, &runtime()).unwrap() { - Reattached::Indeterminate(reason) if Instant::now() < deadline => { - eprintln!("reattach is indeterminate: {reason}"); - thread::sleep(Duration::from_millis(100)); - } - reattached => return reattached, - } - } -} - -#[test] -fn a_crash_during_an_acquisition_leaves_no_lease_or_one_that_launches_nothing() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - // No VMM runs: the orchestrator crashes before any launch. - let vmm = directory.path(); - // After the record, before the binding: the sandbox holds no lease, and leases the generation - // again, while the record, which no binding names, keeps the generation. - let unbound = provision(directory.path(), "unbound"); - assert_eq!( - run_clone_role("after-record", &store, generation, &unbound, vmm), - None - ); - assert_eq!(binding(&unbound), None); - let [orphan]: [Lease; 1] = store.leases(generation).unwrap().try_into().unwrap(); - assert_eq!(orphan.sandbox(), unbound); - let mut leases = vec![leased(&store, &published, &unbound)]; - // After the binding, before or after the acquisition returned: the sandbox holds the lease, - // whose launch no later lock begins, and which a release ends. Another acquisition then - // reserves another launch. - for step in ["after-binding", "acquired"] { - let sandbox = provision(directory.path(), step); - assert_eq!( - run_clone_role(step, &store, generation, &sandbox, vmm), - None - ); - let lease = binding(&sandbox).unwrap(); - assert!(store.leases(generation).unwrap().contains(&lease), "{step}"); - let mut locked = SandboxStore::lock_existing(&sandbox, &files(), None).unwrap(); - let error = launch_refusal(&mut locked, lease.launch()); - assert_eq!(error.code(), ErrorCode::SandboxLeased, "{step}: {error}"); - assert_eq!(store.release(&mut locked).unwrap(), Some(lease.clone())); - drop(locked); - let again = leased(&store, &published, &sandbox); - assert_ne!(again.launch(), lease.launch()); - leases.push(again); - } - // A retirement removes the record that no binding names, and waits for the leases. - leases.sort_by_key(Lease::id); - assert_eq!( - store.retire(generation, None).unwrap(), - Retirement::Draining(leases.clone()) - ); - assert_eq!(store.leases(generation).unwrap(), leases); -} - -#[test] -fn a_clone_that_crashes_before_its_spawn_keeps_its_lease_until_its_launch_ends() { - let crashed = Crashed::after("before-spawn"); - assert_eq!(crashed.vmm.reference, None); - let mut locked = crashed.lock(); - // The journal names the launch that the lease reserved, which spawned nothing. - let journal = locked.journal().unwrap().unwrap(); - assert_eq!(journal.launch(), crashed.lease.launch()); - assert_eq!(journal.state(), LaunchState::Prepared); - crashed.refuses_release(&mut locked); - match reattach_until_determinate(&mut locked) { - Reattached::Exited { - record: None, - recovery: Recovery::Exited, - } => {} - other => panic!("unexpected reattach: {other:?}"), - } - crashed.releases(&mut locked); -} - -#[test] -fn a_clone_that_crashes_after_its_spawn_keeps_its_lease_until_its_unrecorded_vmm_exits() { - let crashed = Crashed::after("after-spawn"); - let reference = crashed.vmm.reference.clone().unwrap(); - let Opened::Running(process) = Process::open(&reference).unwrap() else { - panic!("the unrecorded VMM runs"); - }; - let mut locked = crashed.lock(); - assert_eq!( - locked.journal().unwrap().unwrap().state(), - LaunchState::Prepared - ); - // Reattach can neither prove that the VMM, which no record names, exited, nor verify it: it - // signals nothing, and the lease stays. - let Reattached::Indeterminate(_) = supervisor::reattach(&mut locked, &runtime()).unwrap() - else { - panic!("reattach ended a launch whose VMM runs unverified"); - }; - assert!(!process.has_exited().unwrap()); - crashed.refuses_release(&mut locked); - // Once the VMM exits, the claim that it released proves it. - crashed.vmm.end(); - assert!(process.wait(Instant::now() + TIMEOUT).unwrap().is_some()); - match reattach_until_determinate(&mut locked) { - Reattached::Exited { - record: None, - recovery: Recovery::Exited, - } => {} - other => panic!("unexpected reattach: {other:?}"), - } - crashed.releases(&mut locked); -} - -#[test] -fn a_clone_that_crashes_after_recording_its_vmm_keeps_its_lease_until_reattach_ends_it() { - let crashed = Crashed::after("after-identity"); - let reference = crashed.vmm.reference.clone().unwrap(); - let Opened::Running(process) = Process::open(&reference).unwrap() else { - panic!("the recorded VMM runs"); - }; - let mut locked = crashed.lock(); - let journal = locked.journal().unwrap().unwrap(); - assert_eq!(journal.launch(), crashed.lease.launch()); - assert_eq!(journal.state(), LaunchState::Spawned); - assert_eq!(journal.process(), Some(&reference)); - crashed.refuses_release(&mut locked); - // The launch never committed, so reattach ends its VMM through its verified reference. - match supervisor::reattach(&mut locked, &runtime()).unwrap() { - Reattached::Exited { - record: None, - recovery: Recovery::Terminated(terminated), - } => assert_eq!(terminated, reference), - other => panic!("unexpected reattach: {other:?}"), - } - assert!(process.wait(Instant::now() + TIMEOUT).unwrap().is_some()); - crashed.releases(&mut locked); -} - -#[test] -fn a_committed_clone_keeps_its_lease_through_a_crash_until_its_vm_stops() { - let crashed = Crashed::after("committed"); - let reference = crashed.vmm.reference.clone().unwrap(); - let mut locked = crashed.lock(); - crashed.refuses_release(&mut locked); - // The restarted orchestrator holds the clone's VM again, and its lease lasts while it runs. - let Reattached::Live(running) = supervisor::reattach(&mut locked, &runtime()).unwrap() else { - panic!("the committed clone was not reattached"); - }; - assert_eq!(running.record().launch(), crashed.lease.launch()); - assert_eq!(running.record().process(), &reference); - crashed.refuses_release(&mut locked); - let stopped = running - .stop( - &mut locked, - Instant::now() + TIMEOUT, - None, - |_running, _deadline| { - crashed.vmm.end(); - Ok::<(), Infallible>(()) - }, - ) - .unwrap(); - assert_eq!(stopped.outcome, StopOutcome::Graceful); - crashed.releases(&mut locked); -} - -#[test] -fn a_crash_during_a_release_leaves_a_record_that_a_retirement_removes() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - assert_eq!( - run_clone_role("releasing", &store, generation, &sandbox, directory.path()), - None - ); - // The binding went first: the sandbox holds no lease, while the record, which no binding - // names, keeps the generation until a retirement removes it. - assert_eq!(binding(&sandbox), None); - assert_eq!(store.leases(generation).unwrap(), [lease]); - assert_eq!(release(&store, &sandbox).unwrap(), None); - assert_eq!(store.retire(generation, None).unwrap(), Retirement::Retired); - assert!(store.leases(generation).unwrap().is_empty()); -} - -/// Tests of the records that a Windows file system without POSIX deletion keeps under their names -/// while other handles hold them, after a transaction in another process removed them, and of the -/// records whose access control lists refuse access, which are not removed ones. -#[cfg(windows)] -#[allow(unsafe_code)] -mod windows { - use std::os::windows::fs::OpenOptionsExt; - use std::os::windows::io::AsRawHandle; - - use windows_sys::Win32::Foundation::HANDLE; - use windows_sys::Win32::Storage::FileSystem::{ - DELETE, FILE_DISPOSITION_INFO, FILE_SHARE_DELETE, FILE_SHARE_READ, FILE_SHARE_WRITE, - FileDispositionInfo, SetFileInformationByHandle, - }; - - use super::*; - use crate::host::store::lease::BINDING_DESCRIPTION; - - /// Deletes the file `path` as a file system without POSIX deletion deletes a file that other - /// handles hold: its name stays, and opening it fails, until the last handle closes, of which - /// the returned one is the last unless others hold the file. - fn delete_pending(path: &Path) -> fs::File { - let file = fs::OpenOptions::new() - .access_mode(DELETE) - .share_mode(FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE) - .open(path) - .unwrap(); - let disposition = FILE_DISPOSITION_INFO { DeleteFile: true }; - // SAFETY: the handle is open, and the information is a FILE_DISPOSITION_INFO of its size. - let set = unsafe { - SetFileInformationByHandle( - file.as_raw_handle() as HANDLE, - FileDispositionInfo, - (&raw const disposition).cast(), - size_of::() as u32, - ) - }; - assert_ne!(set, 0, "{}", io::Error::last_os_error()); - file - } - - /// The access control list of a file that allows no one anything, which its owner may still - /// change: every open of the file fails, also one without access rights, unless the process's - /// privileges bypass the list. - const NO_ACCESS: &str = "D:P"; - - /// The access control list of a file that allows its owner alone, as the store's files do. - const OWNER_ONLY: &str = "D:P(A;;FA;;;OW)"; - - #[test] - fn only_a_file_that_is_gone_or_whose_deletion_is_pending_was_removed() { - let directory = tempfile::tempdir().unwrap(); - let path = directory.path().join("record.json"); - fs::write(&path, b"{}").unwrap(); - let denied = io::Error::from(io::ErrorKind::PermissionDenied); - assert!(!was_removed(&path, &denied)); - // A file whose list refuses every open, also one without access rights, is not removed. - // Nor is it where the process's privileges bypass the list, as the backup privilege lets - // the open of `file_identity`, which has backup semantics, through. - platform::tests::set_access(&path, NO_ACCESS).unwrap(); - if let Err(error) = platform::file_identity(&path) { - assert_eq!(error.kind(), io::ErrorKind::PermissionDenied, "{error}"); - } - assert!(!was_removed(&path, &denied)); - platform::tests::set_access(&path, OWNER_ONLY).unwrap(); - // One whose deletion is pending is, as is one that is gone, but only a denied access tells - // so. - let held = delete_pending(&path); - assert!(was_removed(&path, &denied)); - assert!(!was_removed(&path, &io::Error::other("the open failed"))); - drop(held); - assert!(was_removed(&path, &denied)); - // So is a file whose directory is gone. - let gone = directory.path().join("gone").join("record.json"); - assert!(was_removed(&gone, &denied)); - } - - #[test] - fn records_that_refuse_access_are_not_taken_for_removed_ones() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - let record = store - .dir() - .join(LEASES_NAME) - .join(format!("{}.json", lease.id())); - // A retirement that cannot read the record of a lease, or its clone binding, cannot tell - // whether the lease is live, and fails, keeping the generation. - for (path, description) in [ - (record, LEASE_DESCRIPTION), - (sandbox.join(CLONE_BINDING_NAME), BINDING_DESCRIPTION), - ] { - platform::tests::set_access(&path, NO_ACCESS).unwrap(); - let error = store.retire(generation, None).unwrap_err(); - assert!( - error - .message() - .starts_with(&format!("cannot read the {description} ")), - "{error}" - ); - platform::tests::set_access(&path, OWNER_ONLY).unwrap(); - } - assert_eq!( - store.retire(generation, None).unwrap(), - Retirement::Draining(vec![lease.clone()]) - ); - assert_eq!(binding(&sandbox), Some(lease)); - // A tombstone that refuses access quarantines its generation still. - let other = publish(&store, &ManifestSpec::template(), storage(2)); - store.tombstone(other.generation(), None).unwrap(); - let tombstone = store - .dir() - .join(TOMBSTONES_NAME) - .join(format!("{}.json", other.generation())); - platform::tests::set_access(&tombstone, NO_ACCESS).unwrap(); - let now = after(&other, 1); - let leasing = provision(directory.path(), "leasing"); - for error in [ - store.tombstones().unwrap_err(), - store.select(other.key(), &now).unwrap_err(), - refusal(&store, other.generation(), &leasing, &now), - ] { - assert!( - error - .message() - .starts_with(&format!("cannot read the {TOMBSTONE_DESCRIPTION} ")), - "{error}" - ); - } - assert_eq!(binding(&leasing), None); - platform::tests::set_access(&tombstone, OWNER_ONLY).unwrap(); - assert_eq!( - store.tombstones().unwrap(), - [generation, other.generation()] - ); - } - - #[test] - fn records_that_another_transaction_removes_are_gone_for_their_readers() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - let record = store - .dir() - .join(LEASES_NAME) - .join(format!("{}.json", lease.id())); - // A release in another process removed the binding, and is removing the record, which a - // scanner, say, holds open. - let held_binding = delete_pending(&sandbox.join(CLONE_BINDING_NAME)); - let held_record = delete_pending(&record); - assert_eq!(binding(&sandbox), None); - assert!(store.leases(generation).unwrap().is_empty()); - // A tombstone that a retirement removes, likewise. - let other = publish(&store, &ManifestSpec::template(), storage(2)); - store.tombstone(other.generation(), None).unwrap(); - let tombstone = store - .dir() - .join(TOMBSTONES_NAME) - .join(format!("{}.json", other.generation())); - let held_tombstone = delete_pending(&tombstone); - assert!(store.tombstones().unwrap().is_empty()); - assert_eq!( - store.select(other.key(), &after(&other, 1)).unwrap(), - Some(other.clone()) - ); - // A retirement finds no lease of the generation. - assert_eq!(store.retire(generation, None).unwrap(), Retirement::Retired); - drop((held_binding, held_record, held_tombstone)); - assert!(!record.exists()); - assert!(!tombstone.exists()); - } - - #[test] - fn a_release_whose_record_another_transaction_removes_ends_its_lease() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - let record = store - .dir() - .join(LEASES_NAME) - .join(format!("{}.json", lease.id())); - // A retirement in another process is removing the record, as the release does. - let held_record = delete_pending(&record); - assert_eq!(release(&store, &sandbox).unwrap(), Some(lease)); - assert_eq!(binding(&sandbox), None); - drop(held_record); - assert!(store.leases(generation).unwrap().is_empty()); - } - - #[test] - fn a_release_takes_the_files_of_its_sandbox_whose_deletion_is_pending_for_removed() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - // A stop removed the runtime record of the clone's VM, and the journal of its launch, - // which a scanner, say, holds: the release, which reads them through its handle of the - // sandbox's directory, finds that the sandbox records no VM and no launch. - let held: Vec = [JOURNAL_NAME, RUNTIME_NAME] - .into_iter() - .map(|name| { - fs::write(sandbox.join(name), b"{}").unwrap(); - delete_pending(&sandbox.join(name)) - }) - .collect(); - assert_eq!(release(&store, &sandbox).unwrap(), Some(lease)); - assert_eq!(binding(&sandbox), None); - assert!(store.leases(generation).unwrap().is_empty()); - drop(held); - // A release that removed the binding, which a reader holds, and failed before its record - // went, leaves a binding whose deletion is pending, which a release retried finds removed. - let again = leased(&store, &published, &sandbox); - let held_binding = delete_pending(&sandbox.join(CLONE_BINDING_NAME)); - assert_eq!(release(&store, &sandbox).unwrap(), None); - drop(held_binding); - assert_eq!(binding(&sandbox), None); - assert_eq!(store.leases(generation).unwrap(), [again]); - assert_eq!(store.retire(generation, None).unwrap(), Retirement::Retired); - } -} diff --git a/nvx-rs/src/host/template/store/tests.rs b/nvx-rs/src/host/template/store/tests.rs deleted file mode 100644 index 45467d334..000000000 --- a/nvx-rs/src/host/template/store/tests.rs +++ /dev/null @@ -1,988 +0,0 @@ -//! Tests of the template store: its creation and opening, its records and their formats, the -//! namespaces of its generations, its lock, and its refusal of directories and files that are -//! not private to the current user, or that are links. - -use std::sync::{Arc, Barrier, mpsc}; - -use serde_json::{Value, json}; - -use super::*; -use crate::host::snapshot::tests::{ManifestSpec, class}; -use crate::host::template::MANIFEST_NAME; -use crate::host::template::key::CaptureClass; -use crate::host::template::manifest::TemplateManifest; - -impl Store { - /// Publishes a generation of a template, whose fixed time contract names a host and boot of - /// its own, for the tests of other modules, and returns it. - pub(crate) fn publish_for_tests(&self) -> Generation { - tests::publish( - self, - &crate::host::snapshot::tests::ManifestSpec::template(), - tests::storage(1), - ) - } -} - -/// A change to a record, as JSON. -type Change = fn(&mut Value); - -/// Creates a store in a fresh temporary directory, and returns the directory and the store. -pub(super) fn store() -> (tempfile::TempDir, Store) { - let directory = tempfile::tempdir().unwrap(); - let store = Store::create(directory.path().join("templates")).unwrap(); - (directory, store) -} - -/// Returns the storage generation whose bytes are all `byte`, which must not be 0. -pub(super) fn storage(byte: u8) -> StorageGeneration { - StorageGeneration::from_bytes([byte; ID_LEN]).unwrap() -} - -/// Creates the directory `path` private to the current user, as the store creates its own. -pub(super) fn private_dir(path: &Path) { - assert_eq!(platform::create_owner_only_dir(path).unwrap(), None); -} - -/// Publishes a capture of `class` with the storage generation `storage`, whose manifest `spec` -/// encodes, as a publication does, under the store's lock: its snapshot at the place of the next -/// store generation, its key beside it, and then the index, which lists it. -pub(super) fn publish_with( - store: &Store, - class: &CaptureClass, - spec: &ManifestSpec, - storage: StorageGeneration, -) -> Generation { - let _lock = take_lock(store.dir(), None).unwrap(); - let mut index = store.index().unwrap(); - let generation = index.next_generation(); - let payload = store.payload_dir(generation); - let snapshot = payload.join(SNAPSHOT_NAME); - private_dir(&payload); - private_dir(&snapshot); - fs::write(snapshot.join(MANIFEST_NAME), spec.encode()).unwrap(); - let manifest = TemplateManifest::read(&snapshot).unwrap(); - let key = TemplateKey::new(class, &manifest).unwrap(); - write_private(&payload.join(KEY_NAME), &key.to_bytes()).unwrap(); - let published = Generation { - generation, - key: key.identity(), - class: class.identity().unwrap(), - storage, - created_at: manifest.created_at(), - time: manifest.time().clone(), - state_len: manifest.state_len(), - memory_len: manifest.memory_len(), - }; - index.generations.push(published.clone()); - index.next = StoreGeneration::new(generation.get().checked_add(1).unwrap()); - store.write_index(&index).unwrap(); - published -} - -/// Publishes a capture of the class of [`ManifestSpec::template`], as [`publish_with`] does. -pub(super) fn publish( - store: &Store, - spec: &ManifestSpec, - storage: StorageGeneration, -) -> Generation { - publish_with(store, &class(), spec, storage) -} - -/// Removes the generation of store generation `generation` from the index of `store`, under its -/// lock, as a retirement does. -pub(super) fn remove(store: &Store, generation: StoreGeneration) { - let _lock = take_lock(store.dir(), None).unwrap(); - let mut index = store.index().unwrap(); - index - .generations - .retain(|indexed| indexed.generation != generation); - store.write_index(&index).unwrap(); -} - -/// Returns the store's record `name` as JSON. -fn record(store: &Store, name: &str) -> Value { - serde_json::from_slice(&fs::read(store.dir().join(name)).unwrap()).unwrap() -} - -/// Replaces the store's record `name` with `value`. -fn rewrite(store: &Store, name: &str, value: &Value) { - write_private(&store.dir().join(name), value.to_string().as_bytes()).unwrap(); -} - -/// Returns the sorted names of the entries of `dir`. -fn sorted_entries(dir: &Path) -> Vec { - let mut names = entries(dir).unwrap(); - names.sort(); - names -} - -/// Returns the message of the cause of `error`. -pub(super) fn cause(error: &Error) -> String { - std::error::Error::source(error) - .map(ToString::to_string) - .unwrap_or_default() -} - -/// Links `link` to the directory `target`: a symbolic link on Unix, and a junction on Windows, -/// which any user may create. -pub(super) fn link_dir(target: &Path, link: &Path) { - #[cfg(unix)] - std::os::unix::fs::symlink(target, link).unwrap(); - #[cfg(windows)] - crate::host::tests::junction(link, target); -} - -/// Links `link` to the file `target` with a symbolic link, and returns whether it could: Windows -/// lets only Developer Mode or a privilege create one. -pub(super) fn link_file(target: &Path, link: &Path) -> bool { - #[cfg(unix)] - let linked = std::os::unix::fs::symlink(target, link); - #[cfg(windows)] - let linked = std::os::windows::fs::symlink_file(target, link); - if linked.is_err() { - eprintln!("skipped: this host does not let the test create a symbolic link"); - } - linked.is_ok() -} - -/// Removes the link `link` to a directory, which leaves its target as it is. -pub(super) fn unlink_dir(link: &Path) { - #[cfg(unix)] - fs::remove_file(link).unwrap(); - #[cfg(windows)] - fs::remove_dir(link).unwrap(); -} - -#[test] -fn stores_are_created_opened_and_reopened() { - let directory = tempfile::tempdir().unwrap(); - let dir = directory.path().join("templates"); - let error = Store::open(&dir).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); - let store = Store::create(&dir).unwrap(); - assert_eq!(store.dir(), dir); - assert_eq!( - sorted_entries(&dir), - [ - GENERATIONS_NAME, - INDEX_NAME, - INDEX_LOCK_NAME, - STORE_RECORD_NAME - ] - ); - assert!(entries(&dir.join(GENERATIONS_NAME)).unwrap().is_empty()); - let index = store.index().unwrap(); - assert!(index.generations().is_empty()); - assert_eq!(index.next_generation(), FIRST_GENERATION); - // Opening it, creating it again, and reopening it find the same store, with its index. - let published = publish(&store, &ManifestSpec::template(), storage(1)); - assert_eq!(Store::open(&dir).unwrap(), store); - assert_eq!(Store::create(&dir).unwrap(), store); - let id = store.id(); - drop(store); - let reopened = Store::open(&dir).unwrap(); - assert_eq!(reopened.id(), id); - assert_eq!(reopened.index().unwrap().generations(), [published]); - // Another store has an identity of its own. - let other = Store::create(directory.path().join("other")).unwrap(); - assert_ne!(other.id(), id); - // A directory without a store holds none. - let empty = directory.path().join("empty"); - private_dir(&empty); - let error = Store::open(&empty).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); - assert!( - error.message().ends_with("holds no template store"), - "{error}" - ); - // A store needs its parent. - let error = Store::create(directory.path().join("missing").join("templates")).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); -} - -#[test] -fn a_creation_replaces_an_interrupted_one_and_refuses_foreign_entries() { - let directory = tempfile::tempdir().unwrap(); - // What a creation leaves before it writes the identity record is replaced. - let interrupted = directory.path().join("interrupted"); - private_dir(&interrupted); - private_dir(&interrupted.join(GENERATIONS_NAME)); - for name in [ - INDEX_NAME, - INDEX_LOCK_NAME, - ".index.json.42.tmp", - ".store.json.7.tmp", - ] { - fs::write(interrupted.join(name), b"{").unwrap(); - } - let store = Store::create(&interrupted).unwrap(); - assert!(store.index().unwrap().generations().is_empty()); - assert_eq!(Store::open(&interrupted).unwrap(), store); - // Anything else is refused, and kept as it is: other files, and the payloads of a store whose - // identity record is gone. - let foreign = directory.path().join("foreign"); - private_dir(&foreign); - for name in ["notes.txt", ".index.json.1a.tmp", ".config.json.1.tmp"] { - fs::write(foreign.join(name), b"").unwrap(); - } - private_dir(&foreign.join(GENERATIONS_NAME)); - private_dir(&foreign.join(GENERATIONS_NAME).join("7")); - let before = sorted_entries(&foreign); - let error = Store::create(&foreign).unwrap_err(); - assert_eq!(error.code(), ErrorCode::ForeignFiles, "{error}"); - assert!( - error.message().ends_with( - "holds no template store, but entries that are not a store's: .config.json.1.tmp, \ - .index.json.1a.tmp, generations/7, notes.txt" - ), - "{error}" - ); - assert_eq!(sorted_entries(&foreign), before); - let error = Store::open(&foreign).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); -} - -#[test] -fn concurrent_creations_make_one_store() { - /// The creations that race. - const CREATIONS: usize = 8; - let directory = tempfile::tempdir().unwrap(); - let dir = directory.path().join("templates"); - let barrier = Arc::new(Barrier::new(CREATIONS)); - let creations: Vec<_> = (0..CREATIONS) - .map(|_| { - let (dir, barrier) = (dir.clone(), Arc::clone(&barrier)); - thread::spawn(move || { - barrier.wait(); - Store::create(&dir).unwrap().id() - }) - }) - .collect(); - let ids: Vec = creations - .into_iter() - .map(|creation| creation.join().unwrap()) - .collect(); - assert!(ids.iter().all(|id| *id == ids[0]), "{ids:?}"); - assert_eq!(Store::open(&dir).unwrap().id(), ids[0]); -} - -#[test] -fn a_creation_that_another_overtakes_opens_its_store() { - let directory = tempfile::tempdir().unwrap(); - // A creation that finds no store, and then the entries of one that another creation made - // meanwhile, even with a generation, opens that store. - let dir = directory.path().join("templates"); - let mut overtaking = None; - let store = Store::create_racing(dir.clone(), || { - let store = Store::create(&dir).unwrap(); - publish(&store, &ManifestSpec::template(), storage(1)); - overtaking = Some(store); - }) - .unwrap(); - assert_eq!(Some(store), overtaking); - // Entries that no store explains are still refused. - let foreign = directory.path().join("foreign"); - let error = Store::create_racing(foreign.clone(), || { - fs::write(foreign.join("notes.txt"), b"").unwrap(); - }) - .unwrap_err(); - assert_eq!(error.code(), ErrorCode::ForeignFiles, "{error}"); -} - -#[test] -fn records_keep_their_json_form() { - let (_directory, store) = store(); - let mut spec = ManifestSpec::template(); - // Windows keeps times in units of 100 ns. - spec.created_at = (1_700_000_000, 123_456_700); - spec.time().cut_to_anchor_ns = None; - let published = publish(&store, &spec, storage(9)); - let manifest = TemplateManifest::parse(&spec.encode()).unwrap(); - let key = TemplateKey::new(&class(), &manifest).unwrap(); - let time = manifest.time(); - assert_eq!( - record(&store, STORE_RECORD_NAME), - json!({ "format": 1, "store": store.id().to_string() }) - ); - // The index holds no VM generation ID, which only the clones' guests know. - assert_eq!( - record(&store, INDEX_NAME), - json!({ - "format": 1, - "store": store.id().to_string(), - "nextGeneration": 2, - "generations": [{ - "generation": 1, - "key": key.identity().to_string(), - "class": class().identity().unwrap().to_string(), - "storageGeneration": "09090909090909090909090909090909", - "createdAt": { "seconds": 1_700_000_000, "nanos": 123_456_700 }, - "time": { - "tscFrequencyHz": time.tsc_frequency_hz, - "tscTolerancePpm": time.tsc_tolerance_ppm, - "apicFrequencyHz": time.apic_frequency_hz, - "captureTsc": time.capture_tsc, - "captureUtcNs": time.capture_utc_ns, - "captureMonotonicNs": time.capture_monotonic_ns, - "hostClock": "linux-boottime", - "hostId": hex(&time.host_id), - "bootId": time.boot_id.to_string(), - "lineage": time.lineage, - "cutToAnchorNs": null, - }, - "stateLen": manifest.state_len(), - "memoryLen": manifest.memory_len(), - }], - }) - ); - // Each generation keeps its own metadata, which its manifest recorded, and its key. - let index = Store::open(store.dir()).unwrap().index().unwrap(); - assert_eq!(index.generations(), std::slice::from_ref(&published)); - assert_eq!(index.get(FIRST_GENERATION), Some(&published)); - assert_eq!(published.generation(), FIRST_GENERATION); - assert_eq!(published.key(), key.identity()); - assert_eq!(published.class(), class().identity().unwrap()); - assert_eq!(published.storage_generation(), storage(9)); - assert_eq!(published.created_at(), manifest.created_at()); - assert_eq!(published.time(), manifest.time()); - assert_eq!(published.state_len(), manifest.state_len()); - assert_eq!(published.memory_len(), manifest.memory_len()); - assert_eq!(store.read_key(&published).unwrap(), key); - assert_eq!( - store.snapshot_dir(&published).unwrap(), - store - .dir() - .join(GENERATIONS_NAME) - .join("1") - .join(SNAPSHOT_NAME) - ); -} - -#[test] -fn records_of_other_formats_fail_clearly() { - let (_directory, store) = store(); - publish(&store, &ManifestSpec::template(), storage(1)); - let valid = record(&store, INDEX_NAME); - for format in [0, 2] { - let mut index = valid.clone(); - index["format"] = format.into(); - rewrite(&store, INDEX_NAME, &index); - let error = store.index().unwrap_err(); - assert_eq!(error.code(), ErrorCode::FormatUnsupported, "{error}"); - assert!( - error.message().contains(&format!( - "has the unsupported format {format}; this build reads format 1 only and \ - migrates no other" - )), - "{error}" - ); - } - let mut index = valid.clone(); - index.as_object_mut().unwrap().remove("format"); - rewrite(&store, INDEX_NAME, &index); - let error = store.index().unwrap_err(); - assert_eq!(error.code(), ErrorCode::Io, "{error}"); - assert!(error.message().ends_with("it has no format"), "{error}"); - rewrite(&store, INDEX_NAME, &valid); - assert_eq!(store.index().unwrap().generations().len(), 1); - // A store whose identity record is of another format is opened by no build of this one. - rewrite( - &store, - STORE_RECORD_NAME, - &json!({ "format": 2, "store": store.id().to_string() }), - ); - for error in [ - Store::open(store.dir()).unwrap_err(), - Store::create(store.dir()).unwrap_err(), - ] { - assert_eq!(error.code(), ErrorCode::FormatUnsupported, "{error}"); - } -} - -#[test] -fn malformed_indexes_and_those_of_other_stores_are_refused() { - let (_directory, store) = store(); - publish(&store, &ManifestSpec::template(), storage(1)); - publish(&store, &ManifestSpec::template(), storage(2)); - let valid = record(&store, INDEX_NAME); - let cases: [(&str, Change); 11] = [ - ("its next store generation is 0", |index| { - index["nextGeneration"] = 0.into(); - }), - ( - "it lists store generation 2, which its next store generation, 2, does not follow", - |index| index["nextGeneration"] = 2.into(), - ), - ( - "it lists store generation 1 out of order, or twice", - |index| { - index["generations"][1]["generation"] = 1.into(); - }, - ), - ( - "it lists store generation 0, which no generation has", - |index| index["generations"][0]["generation"] = 0.into(), - ), - ("the key of store generation 1 is invalid", |index| { - index["generations"][0]["key"] = "00".into(); - }), - ("the class of store generation 1 is invalid", |index| { - index["generations"][0]["class"] = "".into(); - }), - ( - "the storage generation of store generation 1 is invalid", - |index| index["generations"][0]["storageGeneration"] = "0".repeat(32).into(), - ), - ( - "the creation time of store generation 1 is invalid", - |index| { - index["generations"][0]["createdAt"]["nanos"] = 1_000_000_000.into(); - }, - ), - ( - "the time contract of store generation 1 has the unknown host clock \"tsc\"", - |index| index["generations"][0]["time"]["hostClock"] = "tsc".into(), - ), - ( - "the time contract of store generation 1 has the invalid host identity \"01\"", - |index| index["generations"][0]["time"]["hostId"] = "01".into(), - ), - ( - "it lists 1025 generations, more than the 1024 that a store holds", - |index| { - let entry = index["generations"][0].clone(); - index["generations"] = (1..=MAX_GENERATIONS + 1) - .map(|generation| { - let mut entry = entry.clone(); - entry["generation"] = generation.into(); - entry - }) - .collect(); - index["nextGeneration"] = (MAX_GENERATIONS + 2).into(); - }, - ), - ]; - for (problem, change) in cases { - let mut index = valid.clone(); - change(&mut index); - rewrite(&store, INDEX_NAME, &index); - let error = store.index().unwrap_err(); - assert_eq!(error.code(), ErrorCode::Io, "{error}"); - assert!( - error - .message() - .contains(&format!("is malformed: {problem}")), - "{problem}: {error}" - ); - } - // Fields that the format does not have are refused. - let mut index = valid.clone(); - index["generations"][0]["generationId"] = "00".repeat(16).into(); - rewrite(&store, INDEX_NAME, &index); - let error = store.index().unwrap_err(); - assert!(error.message().ends_with("is malformed"), "{error}"); - assert!( - cause(&error).starts_with("unknown field `generationId`"), - "{error}" - ); - // So is the index of another store. - let mut index = valid.clone(); - index["store"] = "07".repeat(16).into(); - rewrite(&store, INDEX_NAME, &index); - let error = store.index().unwrap_err(); - assert!( - error.message().ends_with(&format!( - "belongs to the template store {}, not to this one, {}", - "07".repeat(16), - store.id() - )), - "{error}" - ); - // And one longer than an index may be, before anything of it is parsed. - let path = store.dir().join(INDEX_NAME); - write_private(&path, &vec![b' '; MAX_INDEX_LEN as usize + 1]).unwrap(); - let error = store.index().unwrap_err(); - assert_eq!( - cause(&error), - format!( - "it holds more than the {MAX_INDEX_LEN} bytes that the index of the template store \ - may hold" - ) - ); - // A missing index is an error of the store, not an empty index. - fs::remove_file(&path).unwrap(); - let error = store.index().unwrap_err(); - assert_eq!(error.code(), ErrorCode::Io, "{error}"); - assert!(error.message().ends_with("has no index"), "{error}"); - rewrite(&store, INDEX_NAME, &valid); - assert_eq!(store.index().unwrap().generations().len(), 2); -} - -#[test] -fn the_four_namespaces_stay_apart() { - let (_directory, store) = store(); - // Store generations follow the order of publication alone: neither the time lineage counters, - // nor the storage generations, which captures may share, and which may look like a store - // generation. - let one = StorageGeneration::from_bytes(1u128.to_be_bytes()).unwrap(); - let mut published = Vec::new(); - for (lineage, storage) in [(7, one), (7, one), (1, storage(0xff))] { - let mut spec = ManifestSpec::template(); - spec.time().lineage = lineage; - let generation = publish(&store, &spec, storage); - published.push(( - generation.generation().get().get(), - generation.time().lineage, - generation.storage_generation(), - )); - } - assert_eq!(published, [(1, 7, one), (2, 7, one), (3, 1, storage(0xff))]); - // A store generation is never assigned again, even once its generation is gone. - let third = StoreGeneration::new(NonZeroU64::new(3).unwrap()); - remove(&store, third); - let index = store.index().unwrap(); - assert_eq!(index.get(third), None); - assert_eq!(index.next_generation().get().get(), 4); - let fourth = publish(&store, &ManifestSpec::template(), one); - assert_eq!(fourth.generation().get().get(), 4); - // The generations of one key, whatever their lineage counters and storage generations. - let of_key: Vec = store - .index() - .unwrap() - .of_key(fourth.key()) - .map(|generation| generation.generation().get().get()) - .collect(); - assert_eq!(of_key, [1, 2, 4]); - // The text form of each names nothing of another's. - assert_eq!(one.to_string(), "00000000000000000000000000000001"); - assert_eq!(fourth.generation().to_string(), "4"); - assert!("1".parse::().is_err()); - assert!("4".parse::().is_err()); - assert_eq!(StorageGeneration::from_bytes([0; ID_LEN]), None); - assert!("0".repeat(32).parse::().is_err()); - assert_eq!( - "00000000000000000000000000000001" - .parse::() - .unwrap(), - one - ); - assert_eq!( - "FF".repeat(16).parse::().unwrap(), - storage(0xff) - ); - assert_eq!(format!("{one:?}"), format!("StorageGeneration({one})")); - let drawn = StorageGeneration::generate().unwrap(); - assert_ne!(drawn, StorageGeneration::generate().unwrap()); - // A store's identity is none of them either. - let id = store.id(); - assert_eq!(id.to_string().parse::().unwrap(), id); - assert_eq!( - id.to_string().to_uppercase().parse::().unwrap(), - id - ); - assert_eq!(format!("{id:?}"), format!("StoreId({id})")); - assert!("1".parse::().is_err()); -} - -#[test] -fn payloads_are_checked_as_they_are_read() { - let (_directory, store) = store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let payload = store.payload_dir(published.generation()); - let key = payload.join(KEY_NAME); - let bytes = fs::read(&key).unwrap(); - // A key that is not a key's canonical encoding is refused as such. - write_private(&key, &[2, 0]).unwrap(); - let error = store.read_key(&published).unwrap_err(); - assert_eq!(error.code(), ErrorCode::FormatUnsupported, "{error}"); - assert!( - error - .message() - .starts_with(&format!("cannot read {}: ", key.display())), - "{error}" - ); - write_private(&key, &bytes[..bytes.len() - 1]).unwrap(); - let error = store.read_key(&published).unwrap_err(); - assert_eq!(error.code(), ErrorCode::Io, "{error}"); - write_private(&key, &vec![0; TemplateKey::MAX_LEN + 1]).unwrap(); - let error = store.read_key(&published).unwrap_err(); - assert_eq!( - cause(&error), - format!( - "it holds more than the {} bytes that the template key may hold", - TemplateKey::MAX_LEN - ) - ); - // A missing key, snapshot, or payload, which a crash may lose, is missing. - fs::remove_file(&key).unwrap(); - let error = store.read_key(&published).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); - fs::remove_dir_all(payload.join(SNAPSHOT_NAME)).unwrap(); - let error = store.snapshot_dir(&published).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); - fs::remove_dir_all(&payload).unwrap(); - for error in [ - store.snapshot_dir(&published).unwrap_err(), - store.read_key(&published).unwrap_err(), - ] { - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); - } -} - -#[test] -fn the_index_lock_excludes_other_holders() { - let (_directory, store) = store(); - let held = take_lock(store.dir(), None).unwrap(); - let started = Instant::now(); - let deadline = Some(Instant::now() + Duration::from_millis(100)); - let error = take_lock(store.dir(), deadline).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::TimedOut, "{error}"); - assert!(error.message().ends_with("is still in progress"), "{error}"); - assert!(started.elapsed() >= Duration::from_millis(100)); - // A creation takes the lock too, and waits for its holder to release it. - let (sender, receiver) = mpsc::channel(); - let dir = store.dir().to_owned(); - let creation = thread::spawn(move || { - let created = Store::create(&dir).map(|store| store.id()); - sender.send(()).unwrap(); - created - }); - assert!(receiver.recv_timeout(Duration::from_millis(200)).is_err()); - drop(held); - receiver.recv_timeout(Duration::from_secs(30)).unwrap(); - assert_eq!(creation.join().unwrap().unwrap(), store.id()); -} - -#[test] -fn links_are_refused() { - let (directory, store) = store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - // A path that a separator ends names the store's own directory. - assert_eq!(Store::open(store.dir().join("")).unwrap(), store); - assert_eq!(Store::create(store.dir().join("")).unwrap(), store); - // A link to a store is no store, however its path ends: Linux follows a link at a name that a - // separator ends, even when it follows none at the final name. - let link = directory.path().join("link"); - link_dir(store.dir(), &link); - for path in [link.clone(), link.join(""), link.join(".")] { - for error in [ - Store::create(&path).unwrap_err(), - Store::open(&path).unwrap_err(), - ] { - assert_eq!(error.code(), ErrorCode::Io, "{}: {error}", path.display()); - assert!( - error.message().ends_with("is not a plain directory"), - "{}: {error}", - path.display() - ); - } - } - // Nor is a link to the generations, to a payload, or to a snapshot followed. - let payload = store.payload_dir(published.generation()); - for path in [ - store.dir().join(GENERATIONS_NAME), - payload.clone(), - payload.join(SNAPSHOT_NAME), - ] { - let moved = path.with_extension("moved"); - fs::rename(&path, &moved).unwrap(); - link_dir(&moved, &path); - let mut errors = vec![store.snapshot_dir(&published).unwrap_err()]; - if path != payload.join(SNAPSHOT_NAME) { - errors.push(store.read_key(&published).unwrap_err()); - } - if path.ends_with(GENERATIONS_NAME) { - errors.push(Store::open(store.dir()).unwrap_err()); - } - for error in errors { - assert!( - error.message().ends_with("is not a plain directory"), - "{}: {error}", - path.display() - ); - } - unlink_dir(&path); - fs::rename(&moved, &path).unwrap(); - } - assert_eq!( - store.read_key(&published).unwrap().identity(), - published.key() - ); - // Nor is a link to a record, or to a key. - for path in [ - store.dir().join(STORE_RECORD_NAME), - store.dir().join(INDEX_NAME), - payload.join(KEY_NAME), - ] { - let moved = path.with_extension("moved"); - fs::rename(&path, &moved).unwrap(); - if !link_file(&moved, &path) { - fs::rename(&moved, &path).unwrap(); - return; - } - let error = if path.ends_with(STORE_RECORD_NAME) { - Store::open(store.dir()).unwrap_err() - } else if path.ends_with(INDEX_NAME) { - store.index().unwrap_err() - } else { - store.read_key(&published).unwrap_err() - }; - assert_eq!( - cause(&error), - "it is a link, or not a regular file", - "{}: {error}", - path.display() - ); - fs::remove_file(&path).unwrap(); - fs::rename(&moved, &path).unwrap(); - } - // Nor a link to the lock file. - let lock = store.dir().join(INDEX_LOCK_NAME); - let moved = lock.with_extension("moved"); - fs::rename(&lock, &moved).unwrap(); - assert!(link_file(&moved, &lock)); - let error = take_lock(store.dir(), None).unwrap_err(); - assert_eq!(cause(&error), "is not a regular file", "{error}"); -} - -#[cfg(unix)] -#[test] -fn records_that_are_not_regular_files_fail_at_once() { - use nix::sys::stat::Mode; - - let (_directory, store) = store(); - for name in [STORE_RECORD_NAME, INDEX_NAME, INDEX_LOCK_NAME] { - let path = store.dir().join(name); - let moved = path.with_extension("moved"); - fs::rename(&path, &moved).unwrap(); - nix::unistd::mkfifo(&path, Mode::S_IRUSR | Mode::S_IWUSR).unwrap(); - // A read that waited for a writer would never end, so it runs on a thread that the test - // can leave behind. - let (sender, receiver) = mpsc::channel(); - let opened = store.clone(); - thread::spawn(move || { - let result = match name { - STORE_RECORD_NAME => Store::open(opened.dir()).map(drop), - INDEX_NAME => opened.index().map(drop), - _ => take_lock(opened.dir(), None).map(drop), - }; - sender.send(result).unwrap(); - }); - let error = receiver - .recv_timeout(Duration::from_secs(10)) - .expect("the read does not wait for a writer") - .unwrap_err(); - assert!( - [ - "it is a link, or not a regular file", - "is not a regular file" - ] - .contains(&cause(&error).as_str()), - "{name}: {error}" - ); - fs::remove_file(&path).unwrap(); - fs::rename(&moved, &path).unwrap(); - } -} - -#[cfg(target_os = "linux")] -#[test] -fn the_store_is_private_to_the_current_user() { - use std::os::unix::fs::PermissionsExt; - - let (directory, store) = store(); - let mode = |path: &Path| fs::metadata(path).unwrap().permissions().mode() & 0o7777; - let set_mode = |path: &Path, mode: u32| { - fs::set_permissions(path, fs::Permissions::from_mode(mode)).unwrap(); - }; - // The store's directories and records are the current user's alone. - for path in [store.dir().to_owned(), store.dir().join(GENERATIONS_NAME)] { - assert_eq!(mode(&path), 0o700, "{}", path.display()); - } - for name in [STORE_RECORD_NAME, INDEX_NAME, INDEX_LOCK_NAME] { - assert_eq!(mode(&store.dir().join(name)), 0o600, "{name}"); - } - // A directory that others could access is refused, and left as it is, rather than - // restricted: another user could hold its files open. - let shared = directory.path().join("shared"); - fs::create_dir(&shared).unwrap(); - for shared_mode in [0o750, 0o705, 0o755, 0o777] { - set_mode(&shared, shared_mode); - for error in [ - Store::create(&shared).unwrap_err(), - Store::open(&shared).unwrap_err(), - ] { - let expected = format!("is accessible to other users (mode {shared_mode:03o})"); - assert!(error.message().ends_with(&expected), "{error}"); - } - assert_eq!(mode(&shared), shared_mode); - } - assert!(entries(&shared).unwrap().is_empty()); - // So are a store whose generations others could access, and its payloads. - let generations = store.dir().join(GENERATIONS_NAME); - set_mode(&generations, 0o750); - let error = Store::open(store.dir()).unwrap_err(); - assert!( - error - .message() - .ends_with("is accessible to other users (mode 750)"), - "{error}" - ); - set_mode(&generations, 0o700); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let snapshot = store - .payload_dir(published.generation()) - .join(SNAPSHOT_NAME); - set_mode(&snapshot, 0o755); - let error = store.snapshot_dir(&published).unwrap_err(); - assert!( - error - .message() - .ends_with("is accessible to other users (mode 755)"), - "{error}" - ); - // A store created under a setgid directory drops the bit for mode 0700, and an existing - // directory that grants no one else access is private, whatever else its mode holds. - let setgid = directory.path().join("setgid"); - fs::create_dir(&setgid).unwrap(); - set_mode(&setgid, 0o2700); - let nested = Store::create(setgid.join("templates")).unwrap(); - assert_eq!(mode(nested.dir()), 0o700); - let existing = directory.path().join("existing"); - fs::create_dir(&existing).unwrap(); - set_mode(&existing, 0o2700); - Store::create(&existing).unwrap(); - assert_eq!(mode(&existing), 0o2700); -} - -#[cfg(target_os = "linux")] -#[test] -fn private_directories_belong_to_the_current_user_and_grant_others_nothing() { - use nix::sys::stat::Mode; - use nix::unistd::Uid; - - // The check takes the owner and the current user as user IDs, and the permission bits of the - // directory's mode, without the bits of its type, as the store reads them from its metadata. - let owner_only_problem = |uid, mode, euid| { - platform::tests::owner_only_problem( - Uid::from_raw(uid), - Mode::from_bits_truncate(mode), - Uid::from_raw(euid), - ) - }; - // Another user's directory is refused whatever its mode, since its owner may change it. - assert_eq!( - owner_only_problem(1001, 0o40700, 1000), - Some("belongs to user 1001, not to the current user 1000".to_owned()) - ); - for mode in [0o40700, 0o42700, 0o41700, 0o40500, 0o40300] { - assert_eq!(owner_only_problem(1000, mode, 1000), None, "{mode:o}"); - } - for mode in [0o40750, 0o40705, 0o40770, 0o40701, 0o40710] { - let problem = owner_only_problem(1000, mode, 1000).unwrap(); - assert!( - problem.starts_with("is accessible to other users"), - "{problem}" - ); - } -} - -#[cfg(windows)] -#[test] -fn the_store_is_private_to_the_current_user() { - let directory = tempfile::tempdir().unwrap(); - platform::tests::set_access(directory.path(), "D:(A;OICI;FA;;;WD)").unwrap(); - // A directory that every user may change holds no store, nor does one created in it. - let error = Store::open(directory.path()).unwrap_err(); - assert!(error.message().ends_with("allows other users"), "{error}"); - let shared = directory.path().join("shared"); - fs::create_dir(&shared).unwrap(); - let error = Store::create(&shared).unwrap_err(); - assert!(error.message().ends_with("allows other users"), "{error}"); - assert!(entries(&shared).unwrap().is_empty()); - // Nor does one that allows its owner alone, but passes nothing on to the files created in it, - // which would take their creator's default list. - platform::tests::set_access(&shared, "D:P(A;;FA;;;OW)").unwrap(); - for error in [ - Store::create(&shared).unwrap_err(), - Store::open(&shared).unwrap_err(), - ] { - assert!( - error - .message() - .ends_with("passes no access control entries on to the files created in it"), - "{error}" - ); - } - // A store created there allows the current user alone, and so do its records. - let store = Store::create(directory.path().join("templates")).unwrap(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let payload = store.payload_dir(published.generation()); - let problem = |path: &Path| { - let file = File::open(path).unwrap(); - platform::private_file_problem(&file).unwrap() - }; - for path in [ - store.dir().join(STORE_RECORD_NAME), - store.dir().join(INDEX_NAME), - store.dir().join(INDEX_LOCK_NAME), - payload.join(KEY_NAME), - ] { - assert_eq!(problem(&path), None, "{}", path.display()); - } - // A payload that every user may change is refused. - let snapshot = payload.join(SNAPSHOT_NAME); - platform::tests::set_access(&snapshot, "D:(A;OICI;FA;;;WD)").unwrap(); - let error = store.snapshot_dir(&published).unwrap_err(); - assert!(error.message().ends_with("allows other users"), "{error}"); - // So is a record or a key whose own list allows others, who could have written it, whatever - // the store's directory allows, and a lock file that others could hold. - for name in [KEY_NAME, INDEX_NAME, INDEX_LOCK_NAME, STORE_RECORD_NAME] { - let path = if name == KEY_NAME { - payload.join(name) - } else { - store.dir().join(name) - }; - platform::tests::set_access(&path, "D:P(A;;FA;;;WD)").unwrap(); - let error = match name { - KEY_NAME => store.read_key(&published).unwrap_err(), - INDEX_NAME => store.index().unwrap_err(), - INDEX_LOCK_NAME => take_lock(store.dir(), None).unwrap_err(), - _ => Store::open(store.dir()).unwrap_err(), - }; - assert!( - error.message().ends_with("allows other users"), - "{name}: {error}" - ); - } -} - -#[cfg(windows)] -#[test] -fn snapshots_that_openvmm_published_are_private_without_entries_for_their_files() { - let (_directory, store) = store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let payload = store.payload_dir(published.generation()); - // OpenVMM gives a snapshot's directory, and each of its files, a protected list of their own, - // which allows SYSTEM and the owner, and passes nothing on to the files created in it. - let openvmm = "D:P(A;;FA;;;SY)(A;;FA;;;OW)"; - let snapshot = payload.join(SNAPSHOT_NAME); - platform::tests::set_access(&snapshot.join(MANIFEST_NAME), openvmm).unwrap(); - platform::tests::set_access(&snapshot, openvmm).unwrap(); - assert_eq!(store.snapshot_dir(&published).unwrap(), snapshot); - // The host's own directories still pass entries on to the files created in them. - platform::tests::set_access(&payload, openvmm).unwrap(); - let error = store.snapshot_dir(&published).unwrap_err(); - assert!( - error - .message() - .ends_with("passes no access control entries on to the files created in it"), - "{error}" - ); - // A snapshot whose list allows anyone else is refused. - platform::tests::set_access(&payload, "D:P(A;OICI;FA;;;OW)").unwrap(); - platform::tests::set_access(&snapshot, "D:P(A;;FA;;;SY)(A;;FA;;;OW)(A;;FR;;;WD)").unwrap(); - let error = store.snapshot_dir(&published).unwrap_err(); - assert!(error.message().ends_with("allows other users"), "{error}"); -} diff --git a/nvx-rs/src/host/vm_directory/tests.rs b/nvx-rs/src/host/vm_directory/tests.rs index 1a284542b..15d7e1da4 100644 --- a/nvx-rs/src/host/vm_directory/tests.rs +++ b/nvx-rs/src/host/vm_directory/tests.rs @@ -103,7 +103,6 @@ fn an_existing_directory_that_is_not_private_is_refused_and_left_unchanged() { ); assert_eq!(mode(&path), other); } - assert_eq!(platform::check_owner_only_dir(&path).unwrap(), None); assert_eq!( platform::tests::store_dir_problem(Uid::from_raw(1001), Mode::S_IRWXU, Uid::from_raw(1000)), Some("belongs to user 1001, not to the current user 1000".to_owned()) diff --git a/nvx-rs/src/lib.rs b/nvx-rs/src/lib.rs index 90c2b4fdf..5b635d012 100644 --- a/nvx-rs/src/lib.rs +++ b/nvx-rs/src/lib.rs @@ -35,7 +35,7 @@ //! network and filesystem policies, the host's NVXS connection, control sessions, process and //! platform primitives, the supervision of VM instances: starting, reattaching, executing //! workloads, and stopping, the store of a sandbox's runtime records and launch journal, and the -//! manifests, compatibility keys, store, and leases of templates. +//! manifests and compatibility keys of templates. //! - `guest` (feature `guest`, Linux): the programs that run in NVX guests, such as //! `nvx-console-pending`, the guest's NVXS connection, the libraries through which any guest //! init or agent opens the terminal on which it serves its session (`guest::terminal`), starts diff --git a/nvx-rs/tests/openvmm_vectors.rs b/nvx-rs/tests/openvmm_vectors.rs index 803910b40..9febc1b6f 100644 --- a/nvx-rs/tests/openvmm_vectors.rs +++ b/nvx-rs/tests/openvmm_vectors.rs @@ -2,10 +2,10 @@ //! protocol vectors and the excerpts of OpenVMM's sources in `tests/vectors/openvmm/`, and the //! versions of OpenVMM's contracts in `nvx::contract`. //! -//! The protocol's tests read the copies, and the unit tests of `nvx::host::template` compile or -//! read the excerpts, because the `nvx-rs` job cannot check out the submodule, whose deploy key -//! fork pull requests lack. This test is ignored by default: the NVX CLI jobs, which check out the -//! submodule, run it, and so does an OpenVMM pin promotion. +//! The protocol's tests read the copies, and the unit tests of `nvx::host::config` compile the +//! excerpts of OpenVMM's allowlist, because the `nvx-rs` job cannot check out the submodule, whose +//! deploy key fork pull requests lack. This test is ignored by default: the NVX CLI jobs, which +//! check out the submodule, run it, and so does an OpenVMM pin promotion. use std::fs; use std::path::Path; diff --git a/nvx-rs/tests/template_capture.rs b/nvx-rs/tests/template_capture.rs deleted file mode 100644 index d03074966..000000000 --- a/nvx-rs/tests/template_capture.rs +++ /dev/null @@ -1,697 +0,0 @@ -//! Captures of templates through the supervisor, with the fake OpenVMM as their VMM: the block -//! devices that the VMM opens, which no handle of the host may write as it starts, the -//! generations that captures publish and that creations then serve, VMs that end without a -//! capture or never end, captures of one class that race, and an adapter that crashes while its -//! capture's VM runs, whose VMM a recovery ends. -//! -//! The fake OpenVMM publishes the snapshot whose manifest OpenVMM wrote for the fixture -//! `tests/fixtures/template/platform-kvm`, or fails, as the scenario next to the kernel asks; see -//! `nvx::testing::fake_openvmm`. The crash is real: a child process of this test binary, the -//! adapter, captures in the store, and the test kills it while its capture's VM runs. -#![cfg(all( - feature = "host", - feature = "testing", - any(target_os = "linux", windows) -))] - -use std::env; -use std::fs; -use std::path::{Path, PathBuf}; -use std::process::{Command, Stdio}; -use std::sync::{Mutex, MutexGuard, PoisonError}; -use std::thread; -use std::time::{Duration, Instant}; - -use nvx::host::platform; -use nvx::host::process::{BootId, Opened, Process, ProcessRef}; -use nvx::host::sandbox::distro::{DISTRO_POLICY_REVISION, DistroBinding, PolicyVerdict}; -use nvx::host::sandbox::{LayerRole, SandboxDistro, SandboxLayer, SandboxPlan, SandboxUpper}; -#[cfg(target_os = "linux")] -use nvx::host::store::SOCKET_NAME; -use nvx::host::store::{ArtifactIdentity, OUTCOME_NAME, RuntimeIdentity, SandboxStore}; -use nvx::host::template::{ - BlockGeometry, CAPTURES_NAME, Capture, CaptureClass, Created, GENERATIONS_NAME, Generation, - HostTime, Invariants, SCRATCH_NAME, SNAPSHOT_NAME, Store, TRANSACTION_NAME, TemplateKey, - TemplateManifest, -}; -use nvx::protocol::root::RootImage; -use nvx::uuid::Uuid; -use nvx::{Artifacts, ErrorCode, ErrorKind, Hypervisor}; -use serde_json::{Value, json}; - -/// Bytes in a MiB. -const MIB: u64 = 1024 * 1024; -/// Nanoseconds in a second. -const SECOND_NS: u64 = 1_000_000_000; -/// The identity of the placeholder. -const IDENTITY: &str = "11111111-1111-1111-1111-111111111111"; -/// Name of the scenario of the fake OpenVMM's captures, next to the kernel. -const SCENARIO_NAME: &str = "fake-openvmm-capture.json"; -/// Name of the marker that the fake OpenVMM holds while a capture's VM runs, in the platform's -/// directory. -const ACTIVE_NAME: &str = "active"; -/// The longest wait of these tests for a capture, a recovery, or a process. -const TIMEOUT: Duration = Duration::from_secs(60); -/// Environment variable that marks the adapter that [`adapter_role`] runs, and names the -/// directory of its bench. -const ADAPTER_BENCH: &str = "NVX_TEMPLATE_CAPTURE_TEST_BENCH"; - -/// Serializes the tests that spawn processes: a process that one test spawns while another test's -/// log is inheritable for its own spawn could hold that log's claim for a while. -static SPAWNS: Mutex<()> = Mutex::new(()); - -/// Takes [`SPAWNS`]. -fn serialized() -> MutexGuard<'static, ()> { - SPAWNS.lock().unwrap_or_else(PoisonError::into_inner) -} - -/// Returns the directory of the fixture whose manifest OpenVMM wrote for a capture of [`class`]. -fn fixture() -> PathBuf { - Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures/template/platform-kvm") -} - -/// Returns the manifest of [`fixture`]. -fn manifest() -> TemplateManifest { - TemplateManifest::read(&fixture()).unwrap() -} - -/// Returns the identity `sha256:` and 64 copies of `digit`. -fn artifact(digit: char) -> ArtifactIdentity { - ArtifactIdentity::sha256(&digit.to_string().repeat(64)).unwrap() -} - -/// Returns the class of the capture of [`fixture`]. -fn class() -> CaptureClass { - CaptureClass { - runtime: RuntimeIdentity::new(artifact('a'), artifact('b'), artifact('c')), - guest_build: ArtifactIdentity::new("guest-build-1").unwrap(), - hypervisor: Hypervisor::Kvm, - cpu_profile: "auto".to_owned(), - processors: 2, - boot_processors: 2, - memory: 512 * MIB, - memory_capacity: 2048 * MIB, - distro: BlockGeometry { - length: 1024 * MIB, - logical_block_size: 512, - physical_block_size: 4096, - }, - scratch: BlockGeometry { - length: 64 * MIB, - logical_block_size: 512, - physical_block_size: 512, - }, - network: None, - invariants: Invariants::new(), - } -} - -/// Returns the host's time `seconds` after the capture of [`fixture`], on the host and in the -/// boot that captured it. -fn after(seconds: u64) -> HostTime { - let manifest = manifest(); - let time = manifest.time(); - HostTime { - host_id: time.host_id, - boot_id: time.boot_id, - clock: time.host_clock, - monotonic_ns: time.capture_monotonic_ns + seconds * SECOND_NS, - utc_ns: time.capture_utc_ns + seconds * SECOND_NS, - } -} - -/// Returns a deadline that no capture of these tests reaches. -fn deadline() -> Instant { - Instant::now() + TIMEOUT -} - -/// Creates the directory `path`, private to the current user: on Windows, with a protected list -/// that allows the owner and SYSTEM alone, which the files created in it inherit. -fn private_dir(path: &Path) { - fs::create_dir(path).unwrap(); - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - - fs::set_permissions(path, fs::Permissions::from_mode(0o700)).unwrap(); - } - #[cfg(windows)] - windows::set_access(path, "D:P(A;OICI;FA;;;OW)(A;OICI;FA;;;SY)"); -} - -/// Creates the sparse file `path` of `length` bytes, which no one else may write, whatever the -/// umask. -fn sparse(path: &Path, length: u64) { - let mut options = fs::OpenOptions::new(); - options.write(true).create_new(true); - #[cfg(unix)] - std::os::unix::fs::OpenOptionsExt::mode(&mut options, 0o600); - let file = options.open(path).unwrap(); - #[cfg(windows)] - windows::set_sparse(&file); - file.set_len(length).unwrap(); -} - -/// Returns the file at `path` as JSON. -fn json_of(path: &Path) -> Value { - serde_json::from_slice(&fs::read(path).unwrap()).unwrap() -} - -/// Returns the names of the entries of `dir`, sorted. -fn sorted(dir: &Path) -> Vec { - let mut names: Vec = fs::read_dir(dir) - .unwrap() - .map(|entry| entry.unwrap().file_name().into_string().unwrap()) - .collect(); - names.sort(); - names -} - -/// Returns the store generation of `generation`, as a number. -fn number(generation: &Generation) -> u64 { - generation.generation().get().get() -} - -/// A store, and the inputs of captures of [`class`] into it, in a temporary directory: the -/// platform's files in `platform`, a placeholder, a kernel, an initramfs, and the fake OpenVMM's -/// scenario, which a capture's VM follows, and the store in `store`. -struct Bench { - /// The temporary directory. - directory: tempfile::TempDir, - /// The store. - store: Store, - /// The inputs of a capture. - capture: Capture, -} - -impl Bench { - /// Returns a bench whose VMs publish the fixture's snapshot, as `scenario` modifies the - /// default scenario. - fn new(scenario: &Value) -> Self { - let directory = tempfile::tempdir().unwrap(); - let platform = directory.path().join("platform"); - private_dir(&platform); - sparse(&platform.join("placeholder.erofs"), class().distro.length); - fs::write(platform.join("vmlinux"), b"kernel").unwrap(); - fs::write(platform.join("initramfs.cpio.gz"), b"initrd").unwrap(); - let store = Store::create(directory.path().join("store")).unwrap(); - let bench = Self { - store, - capture: capture_in(&platform), - directory, - }; - bench.follow(scenario); - bench - } - - /// Returns the bench in `dir`, as another process made it. - fn open(dir: &Path) -> (Store, Capture) { - ( - Store::open(dir.join("store")).unwrap(), - capture_in(&dir.join("platform")), - ) - } - - /// Returns the platform's directory. - fn platform(&self) -> PathBuf { - self.directory.path().join("platform") - } - - /// Lets the next VMs follow the default scenario, as `fields` modify it: the fake OpenVMM - /// publishes the fixture's snapshot. - fn follow(&self, fields: &Value) { - let mut scenario = json!({ - "manifest": fixture().join("manifest.bin"), - "stateLen": manifest().state_len(), - }); - for (name, value) in fields.as_object().unwrap() { - scenario[name] = value.clone(); - } - fs::write(self.platform().join(SCENARIO_NAME), scenario.to_string()).unwrap(); - } - - /// Returns the path of the marker of an active VM. - fn active(&self) -> PathBuf { - self.platform().join(ACTIVE_NAME) - } - - /// Returns the slot of the class, the only one. - fn slot(&self) -> PathBuf { - let captures = self.store.dir().join(CAPTURES_NAME); - let slots = sorted(&captures); - assert_eq!(slots.len(), 1, "{slots:?}"); - captures.join(&slots[0]) - } - - /// Returns the path of the file `suffix` that the fake OpenVMM wrote of the slot's last VM, - /// which it names after the directory of the VM's outcome report: the capture's transaction, - /// from whose sandbox store the VM launched. - fn dump(&self, suffix: &str) -> PathBuf { - self.platform() - .join(format!("fake-openvmm-{TRANSACTION_NAME}{suffix}.json")) - } - - /// Creates a template at a second after the fixture's capture. - fn create(&self) -> nvx::Result { - self.store - .create_template(&self.capture, || Ok(after(1)), deadline()) - } - - /// Refreshes the template. - fn refresh(&self) -> nvx::Result { - self.store.refresh_template(&self.capture, deadline()) - } - - /// Returns the store generations that the index lists. - fn indexed(&self) -> Vec { - self.store - .index() - .unwrap() - .generations() - .iter() - .map(number) - .collect() - } - - /// Asserts that no capture of the class runs or remains: its slot holds no transaction and no - /// launch. - fn assert_idle(&self) { - let slot = self.slot(); - assert!(!slot.join(TRANSACTION_NAME).exists()); - assert!( - SandboxStore::open(&slot) - .unwrap() - .journal() - .unwrap() - .is_none() - ); - } -} - -/// Returns the inputs of a capture of [`class`] whose placeholder, kernel, initramfs, and -/// scenario are in `platform`, and whose VMM is the fake OpenVMM. -fn capture_in(platform: &Path) -> Capture { - let placeholder = platform.join("placeholder.erofs"); - let path = placeholder.to_str().unwrap(); - let layer = SandboxLayer::new(LayerRole::Distro, path, IDENTITY); - let binding = DistroBinding::import(&layer, PolicyVerdict::ACCEPTED).unwrap(); - let image = RootImage::erofs(Some(Uuid::parse(IDENTITY).unwrap())); - let upper = SandboxUpper::scratch(platform.join("clone.ext4").to_str().unwrap()); - Capture { - class: class(), - adapter: SandboxPlan::adapter(SandboxDistro::new(path, image), upper), - binding, - policy_revision: DISTRO_POLICY_REVISION, - artifacts: Artifacts::new( - env!("CARGO_BIN_EXE_nvx-fake-openvmm"), - platform.join("vmlinux"), - platform.join("initramfs.cpio.gz"), - ), - } -} - -/// Returns the value of `option` in the command line `arguments`. -fn value_of<'a>(arguments: &'a [String], option: &str) -> &'a str { - let at = arguments - .iter() - .position(|argument| argument == option) - .unwrap_or_else(|| panic!("{option} is missing from {arguments:?}")); - &arguments[at + 1] -} - -#[test] -fn captures_publish_generations_that_creations_then_serve() { - let _spawns = serialized(); - let bench = Bench::new(&json!({})); - let Created::Captured(generation) = bench.create().unwrap() else { - panic!("the first creation captures"); - }; - assert_eq!(bench.indexed(), [1]); - assert_eq!(number(&generation), 1); - // The VMM received the template's command line, with the generation's storage generation as - // its block devices' identity. - let arguments: Vec = serde_json::from_value(json_of(&bench.dump(""))).unwrap(); - assert_eq!(value_of(&arguments, "--cmdline"), "nvx_sandbox=1"); - assert_eq!( - value_of(&arguments, "--snapshot-generation-id"), - generation.storage_generation().to_string() - ); - let transaction = bench.slot().join(TRANSACTION_NAME); - assert_eq!( - Path::new(value_of(&arguments, "--snapshot-destination")), - transaction.join(SNAPSHOT_NAME) - ); - // The VM launched from the transaction's sandbox store, so that its control socket, on - // Linux, is beside the snapshot's destination, as OpenVMM requires, and its outcome report - // too, which the slot then keeps. - assert_eq!( - Path::new(value_of(&arguments, "--microvm-report")), - transaction.join(OUTCOME_NAME) - ); - #[cfg(target_os = "linux")] - assert_eq!( - value_of(&arguments, "--microvm-control-console"), - format!("listen={}", transaction.join(SOCKET_NAME).display()) - ); - assert!(bench.slot().join(OUTCOME_NAME).is_file()); - // The VMM opened the placeholder and the scratch at their paths, and found no handle that - // could write them: on Windows, its open would have failed, and on Linux, its read lease - // finds one, unless the file system takes no lease. - let blocks = json_of(&bench.dump("-blocks")); - let opened: Vec<(&str, PathBuf, u64)> = blocks - .as_array() - .unwrap() - .iter() - .map(|block| { - assert_ne!(block["writers"], true, "{block}"); - ( - block["role"].as_str().unwrap(), - PathBuf::from(block["path"].as_str().unwrap()), - block["length"].as_u64().unwrap(), - ) - }) - .collect(); - assert_eq!( - opened, - [ - ( - "distro", - bench.platform().join("placeholder.erofs"), - 1024 * MIB - ), - ("scratch", transaction.join(SCRATCH_NAME), 64 * MIB), - ] - ); - // The generation's payload holds the snapshot that the VMM published, with its key, and - // the slot keeps nothing of the capture. - let snapshot = bench.store.snapshot_dir(&generation).unwrap(); - assert_eq!( - sorted(&snapshot), - ["manifest.bin", "memory.bin", "state.bin"] - ); - let key = TemplateKey::new(&bench.capture.class, &manifest()).unwrap(); - assert_eq!(bench.store.read_key(&generation).unwrap(), key); - assert_eq!(generation.key(), key.identity()); - bench.assert_idle(); - // A second creation serves the generation without a VM. - fs::remove_file(bench.dump("")).unwrap(); - assert_eq!(bench.create().unwrap(), Created::Existing(generation)); - assert!(!bench.dump("").exists()); -} - -#[test] -fn refreshes_publish_new_generations_and_leave_the_older_ones() { - let _spawns = serialized(); - let bench = Bench::new(&json!({})); - let first = bench.create().unwrap().into_generation(); - // A VMM that ends abruptly once it has published leaves the file that backed the RAM, which - // the publication does not take. - bench.follow(&json!({ "keepMemory": true })); - let second = bench.refresh().unwrap(); - assert_eq!(number(&second), 2); - assert_ne!(second.storage_generation(), first.storage_generation()); - assert_eq!(bench.indexed(), [1, 2]); - for generation in [&first, &second] { - let snapshot = bench.store.snapshot_dir(generation).unwrap(); - assert_eq!( - sorted(&snapshot), - ["manifest.bin", "memory.bin", "state.bin"] - ); - } - assert_eq!( - sorted(&bench.store.dir().join(GENERATIONS_NAME)), - ["1", "2"] - ); - bench.assert_idle(); - assert_eq!(bench.create().unwrap(), Created::Existing(second)); -} - -#[test] -fn vms_that_end_without_a_capture_publish_nothing() { - let _spawns = serialized(); - let bench = Bench::new(&json!({})); - for scenario in [ - json!({ "outcome": "fail", "status": 3 }), - // The snapshot that the VMM published before it failed is not the capture's. - json!({ "outcome": "publish-then-fail" }), - json!({ "outcome": "crash" }), - ] { - bench.follow(&scenario); - let error = bench.refresh().unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Exited, "{scenario}: {error}"); - assert!(bench.indexed().is_empty(), "{scenario}"); - bench.assert_idle(); - } - // The slot serves the next capture. - bench.follow(&json!({})); - bench.refresh().unwrap(); - assert_eq!(bench.indexed(), [1]); -} - -/// Returns the process ID that the marker of an active VM at `path` holds. -fn active_pid(path: &Path) -> u32 { - fs::read_to_string(path).unwrap().parse().unwrap() -} - -#[test] -fn vms_that_never_end_are_torn_down_at_the_deadline() { - let _spawns = serialized(); - let bench = Bench::new(&json!({})); - for outcome in ["hang", "publish-then-hang"] { - bench.follow(&json!({ "outcome": outcome, "active": bench.active() })); - let deadline = Instant::now() + Duration::from_secs(2); - let error = bench - .store - .refresh_template(&bench.capture, deadline) - .unwrap_err(); - assert_eq!(error.kind(), ErrorKind::TimedOut, "{outcome}: {error}"); - // The VMM, which was killed, left its marker. - let pid = active_pid(&bench.active()); - assert_eq!( - platform::process_start_time(pid).unwrap(), - None, - "{outcome}" - ); - fs::remove_file(bench.active()).unwrap(); - assert!(bench.indexed().is_empty(), "{outcome}"); - bench.assert_idle(); - } -} - -#[test] -fn captures_of_one_class_are_serialized() { - let _spawns = serialized(); - // A VM that started while another ran would fail to mark itself active. - let bench = Bench::new(&json!({})); - bench.follow(&json!({ "active": bench.active(), "delayMs": 300 })); - let (store, capture) = (&bench.store, &bench.capture); - let generations: Vec = thread::scope(|scope| { - let refreshes: Vec<_> = (0..2) - .map(|_| scope.spawn(|| store.refresh_template(capture, deadline()))) - .collect(); - refreshes - .into_iter() - .map(|refresh| refresh.join().unwrap().unwrap()) - .collect() - }); - let mut published: Vec = generations.iter().map(number).collect(); - published.sort_unstable(); - assert_eq!(published, [1, 2]); - assert_eq!(bench.indexed(), [1, 2]); - assert!(!bench.active().exists()); - bench.assert_idle(); -} - -/// The adapter that [`a_vm_that_outlives_its_crashed_adapter_is_ended_by_recovery`] runs in a -/// process of its own: it refreshes the template of the bench that [`ADAPTER_BENCH`] names, until -/// the test kills it. -#[test] -#[ignore = "an adapter that the capture tests run in a process of its own"] -fn adapter_role() { - let Some(dir) = env::var_os(ADAPTER_BENCH) else { - return; - }; - let (store, capture) = Bench::open(Path::new(&dir)); - let result = store.refresh_template(&capture, deadline()); - panic!("the capture ended before the test killed its adapter: {result:?}"); -} - -/// A VMM that a crashed adapter left behind, which is killed when the test ends if it still runs. -struct Orphan(ProcessRef); - -impl Orphan { - /// Returns whether the VMM still runs. - fn runs(&self) -> bool { - platform::process_start_time(self.0.pid).unwrap() == Some(self.0.start_time) - } -} - -impl Drop for Orphan { - fn drop(&mut self) { - if let Ok(Opened::Running(process)) = Process::open(&self.0) { - let _ = process.kill(); - let _ = process.wait(Instant::now() + TIMEOUT); - } - } -} - -/// Runs [`adapter_role`] on `bench` until its capture's VM is active, the journal of the -/// transaction's sandbox store records the VMM, and, if `published`, the VM has published its -/// snapshot, then kills the adapter, as a crash would end it, and returns the VMM, which still -/// runs. -fn crash_adapter(bench: &Bench, published: bool) -> Orphan { - let mut adapter = Command::new(env::current_exe().unwrap()) - .args([ - "--exact", - "adapter_role", - "--ignored", - "--nocapture", - "--test-threads=1", - ]) - .env(ADAPTER_BENCH, bench.directory.path()) - .stdin(Stdio::null()) - .stdout(Stdio::null()) - .spawn() - .unwrap(); - let started = Instant::now(); - let recorded = loop { - assert!( - adapter.try_wait().unwrap().is_none(), - "the adapter ended before the test killed it" - ); - // The VM marks itself active before it publishes its snapshot. - let recorded = bench.active().exists().then(|| { - let transaction = bench.slot().join(TRANSACTION_NAME); - if published && !transaction.join(SNAPSHOT_NAME).is_dir() { - return None; - } - SandboxStore::open(transaction) - .ok()? - .journal() - .ok()?? - .process() - .cloned() - }); - if let Some(Some(process)) = recorded { - break process; - } - assert!( - started.elapsed() < TIMEOUT, - "the capture's VM did not start" - ); - thread::sleep(Duration::from_millis(20)); - }; - adapter.kill().unwrap(); - adapter.wait().unwrap(); - assert_eq!(recorded.pid, active_pid(&bench.active())); - assert_eq!(recorded.boot, BootId::current().unwrap()); - let orphan = Orphan(recorded); - assert!(orphan.runs()); - orphan -} - -#[test] -fn a_vm_that_outlives_its_crashed_adapter_is_ended_by_recovery() { - let _spawns = serialized(); - let bench = Bench::new(&json!({})); - // A VM that published its snapshot, but whose exit no capture verified, is not published - // either. - for outcome in ["hang", "publish-then-hang"] { - bench.follow(&json!({ "outcome": outcome, "active": bench.active() })); - let orphan = crash_adapter(&bench, outcome == "publish-then-hang"); - let deadline = deadline(); - // A scanner that briefly opens the launch's log can keep its claim held for a moment. - let recovered = loop { - match bench.store.recover_captures(deadline) { - Err(error) - if error.code() == ErrorCode::VmmIndeterminate && Instant::now() < deadline => - { - eprintln!("the recovery is indeterminate: {error}"); - thread::sleep(Duration::from_millis(100)); - } - recovered => break recovered.unwrap(), - } - }; - assert!(recovered.is_empty(), "{outcome}"); - assert!(!orphan.runs(), "{outcome}"); - assert!(bench.indexed().is_empty(), "{outcome}"); - bench.assert_idle(); - fs::remove_file(bench.active()).unwrap(); - } - // The slot serves the next capture. - bench.follow(&json!({})); - bench.refresh().unwrap(); - assert_eq!(bench.indexed(), [1]); -} - -/// What the tests need of Windows that the crate does not export. -#[cfg(windows)] -mod windows { - use std::fs::File; - use std::os::windows::ffi::OsStrExt; - use std::os::windows::io::AsRawHandle; - use std::path::Path; - use std::{io, ptr}; - - use windows_sys::Win32::Foundation::{HANDLE, LocalFree}; - use windows_sys::Win32::Security::Authorization::{ - ConvertStringSecurityDescriptorToSecurityDescriptorW, SDDL_REVISION_1, - }; - use windows_sys::Win32::Security::{ - DACL_SECURITY_INFORMATION, PROTECTED_DACL_SECURITY_INFORMATION, SetFileSecurityW, - }; - use windows_sys::Win32::System::IO::DeviceIoControl; - - /// Returns `text` as a NUL-terminated UTF-16 string. - fn wide(text: impl AsRef) -> Vec { - text.as_ref().encode_wide().chain([0]).collect() - } - - /// Replaces the access control list of the file or directory `path` with the protected list - /// of the descriptor `sddl`, in the security descriptor definition language. - pub(super) fn set_access(path: &Path, sddl: &str) { - let sddl = wide(sddl); - let name = wide(path); - let mut descriptor = ptr::null_mut(); - // SAFETY: the string is NUL-terminated, and the output pointer is valid. - let converted = unsafe { - ConvertStringSecurityDescriptorToSecurityDescriptorW( - sddl.as_ptr(), - SDDL_REVISION_1, - &mut descriptor, - ptr::null_mut(), - ) - }; - assert_ne!(converted, 0, "{}", io::Error::last_os_error()); - let information = DACL_SECURITY_INFORMATION | PROTECTED_DACL_SECURITY_INFORMATION; - // SAFETY: the name is NUL-terminated, and the descriptor is valid. - let set = unsafe { SetFileSecurityW(name.as_ptr(), information, descriptor) }; - let error = io::Error::last_os_error(); - // SAFETY: the conversion allocated the descriptor, which nothing uses any more. - unsafe { LocalFree(descriptor) }; - assert_ne!(set, 0, "{error}"); - } - - /// Marks the file that `file` holds open for writing sparse, which NTFS otherwise allocates - /// whole as the file is extended. - pub(super) fn set_sparse(file: &File) { - /// `FSCTL_SET_SPARSE`, in `winioctl.h`, which marks a file sparse without a buffer. - const FSCTL_SET_SPARSE: u32 = 0x0009_00c4; - let mut returned = 0; - // SAFETY: the handle is open for as long as `file` is borrowed, the control takes no - // input or output buffer, and the count's pointer is valid. - let set = unsafe { - DeviceIoControl( - file.as_raw_handle() as HANDLE, - FSCTL_SET_SPARSE, - ptr::null(), - 0, - ptr::null_mut(), - 0, - &mut returned, - ptr::null_mut(), - ) - }; - assert_ne!(set, 0, "{}", io::Error::last_os_error()); - } -} diff --git a/nvx-rs/tests/vectors/README.md b/nvx-rs/tests/vectors/README.md index b53ae6063..547866e3d 100644 --- a/nvx-rs/tests/vectors/README.md +++ b/nvx-rs/tests/vectors/README.md @@ -70,10 +70,10 @@ the key that the vector's bytes decode to. `CLASS|NETWORK` of `snapshot-key-v1.txt` came to encode its network in place as the template captures it, under the complete deny-all policy: the encodings of a key and - of a class have not been released, so a template store that an earlier build - created is not supported, and is deleted and created again, since nothing - migrates it. The snapshot key also drops the obsolete guest-build field and - uses the `nvx snapshot key` hash domain in place, and now appends the full + of a class have not been released, so no key or class that an earlier build + encoded is supported, since nothing migrates it. The snapshot key also drops + the obsolete guest-build field and uses the `nvx snapshot key` hash domain in + place, and now appends the full capture-time contract while its compatibility identity still digests only the preceding compatibility fields: no snapshot outlives a deployment, and the prior key encoding is not supported. @@ -98,15 +98,15 @@ of `openvmm/openvmm_helpers/src/snapshot/microvm.rs` that admits the kernel command-line tokens of a capture at OpenVMM's platform tier, and `microvm_control_tty_command_line.rs`, the constant of `openvmm/openvmm_defs/src/microvm.rs` that names the control tty's token, which -the function reads. The tests of `nvx::host::template` compile them, and check -each token of a template's kernel command line with OpenVMM's own function. The -third, `port_forward_socket_creation.rs`, is the variant of the error of -`vm/devices/net/net_consomme/src/resolver.rs` with which OpenVMM reports a port -forward whose socket on the host it cannot bind, as when a clone's VMM restores -its NIC with a forward whose port another socket holds. OpenVMM gives that -error no code, so the launch of a clone attributes it to the host by its text, -and the tests of `nvx::host::template` read the excerpt, and check that the -text is OpenVMM's. +the function reads. The tests of `nvx::host::config` compile them, and check +that its allowlist of a capture's kernel command-line tokens admits each token +as OpenVMM's own function does. The third, `port_forward_socket_creation.rs`, +is the variant of the error of `vm/devices/net/net_consomme/src/resolver.rs` +with which OpenVMM reports a port forward whose socket on the host it cannot +bind, as when a clone's VMM restores its NIC with a forward whose port another +socket holds. OpenVMM gives that error no code, so the launch of a clone, which +#1121 deleted, attributed it to the host by its text; only the comparison below +reads the excerpt now. The tests cannot call OpenVMM's broker or its allowlist: OpenVMM's workspace needs a later Rust release than `nvx`'s minimum, and the `nvx-rs` job cannot