From 2c0033083c35cf3799e25d06157edcab07f3deae Mon Sep 17 00:00:00 2001 From: Pedro Henrique Penna Date: Wed, 11 Nov 2026 17:56:58 -0800 Subject: [PATCH] host: delete the template store With the old clone path gone (SL6.2b), nothing uses the template store: Store, with its index, generations, payloads, leases, tombstones and retirement, and the capture transactions that published its generations, with TemplatePlan, which lowered an adapter's plan for a capture, and Generation::eligibility, which checked a generation's time contract for the store. host::template leaves whole: its plan and store modules, their 95 unit tests and the doc tests of TemplatePlan and Store, the template-capture integration test, and the aliases of the snapshot metadata that it kept for the store's users. Their last users take the snapshot names: a restore test's TemplateManifest, and two key tests that checked the TemplateKey alias. The public API snapshots lose its 222 items on each platform. TimeContract::eligibility and its tests stay. The crate-private code that only the template store used leaves with it, as the dead-code check of each feature set finds it on Linux and Windows: create_owner_only_dir and foreign_writer_problem on every host, with Linux's foreign_writer and Windows' Unwritable check and CHANGE_RIGHTS, Windows' check_owner_only_dir, Linux's re-export of it and Directory::store_dir_problem; the snapshot module's internal TemplateKey and TemplateManifest aliases and its re-export of check_exit; and, in the sandbox store, which SL6.1 deletes, LockedStore's reservation of a lease's launch, open_dir and remove_clone_files, regular::identity_of, and the lease module's readers and accessors that only the store called, whose remaining items become crate-private now that nothing re-exports them. Without a reservation, begin_launch refuses every launch of a sandbox whose clone binding records a lease, as it refused all but the reserved one. rename_no_replace, which only the CLI calls now, is gated by its cli feature, with the Windows tests that call it. The tests of what leaves go with it. The supervisor's tests no longer lease from a template store: four keep their checks of the runtime record on a sandbox of their own, one of them renamed, and two retire, since other tests make the same checks without a lease. The sandbox store's two tests of remove_clone_files retire with it, a VM directory test drops its check of the template store's owner-only policy, and the snapshot fixture creates its directories with create_store_dir, which create_owner_only_dir called on Windows. The docs stop naming the template store: the crate's and the host's module docs, the supervisor's Clones section, the sandbox store's launch, recovery, removal, file-name, lease and record docs, and the other rustdoc that linked to it or told to recreate a store; the nvx README's module table, its layout, its template section, whose paragraphs on the manifest reader, the capture class, the key and eligibility move to the snapshot section under the snapshot names, and its supervisor and test sections; the vectors README; design docs 1.5 and 5.3; and the case matrix, which retires the doc tests of TemplatePlan and Store and follows the supervisor's and the sandbox store's doc tests to their new lines. Part of #1121 (SL6.2a). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- doc/acceptance-case-matrix.md | 12 +- .../1.5-compatibility-and-versioning-model.md | 5 +- doc/design/5.3-snapshot-protocol.md | 3 +- nvx-rs/README.md | 221 +- nvx-rs/public-api/linux.txt | 222 -- nvx-rs/public-api/windows.txt | 222 -- nvx-rs/src/contract.rs | 2 +- nvx-rs/src/host/mod.rs | 4 - nvx-rs/src/host/platform/linux/identity.rs | 11 - nvx-rs/src/host/platform/linux/mod.rs | 5 +- nvx-rs/src/host/platform/linux/private.rs | 42 +- nvx-rs/src/host/platform/linux/tests.rs | 2 +- nvx-rs/src/host/platform/other.rs | 11 +- nvx-rs/src/host/platform/tests.rs | 4 +- nvx-rs/src/host/platform/windows/mod.rs | 7 +- nvx-rs/src/host/platform/windows/private.rs | 67 +- .../host/platform/windows/private/tests.rs | 2 + nvx-rs/src/host/restore/tests/live_init.rs | 4 +- nvx-rs/src/host/sandbox/geometry.rs | 1 - nvx-rs/src/host/snapshot.rs | 2 +- nvx-rs/src/host/snapshot/key.rs | 6 +- nvx-rs/src/host/snapshot/key/tests.rs | 10 - nvx-rs/src/host/snapshot/manifest.rs | 2 - nvx-rs/src/host/snapshot/tests.rs | 4 +- nvx-rs/src/host/store/files.rs | 3 - nvx-rs/src/host/store/lease.rs | 198 +- nvx-rs/src/host/store/lifecycle.rs | 21 +- nvx-rs/src/host/store/lifecycle/tests.rs | 27 - nvx-rs/src/host/store/locked.rs | 163 +- nvx-rs/src/host/store/mod.rs | 15 +- nvx-rs/src/host/store/names.rs | 39 +- nvx-rs/src/host/store/record.rs | 4 +- nvx-rs/src/host/store/regular.rs | 14 +- nvx-rs/src/host/store/tests.rs | 33 - nvx-rs/src/host/supervisor/launch.rs | 6 +- nvx-rs/src/host/supervisor/mod.rs | 76 +- nvx-rs/src/host/supervisor/tests.rs | 156 +- nvx-rs/src/host/sys/mod.rs | 4 +- nvx-rs/src/host/template/mod.rs | 65 - nvx-rs/src/host/template/plan.rs | 497 --- nvx-rs/src/host/template/plan/tests.rs | 1031 ------ nvx-rs/src/host/template/store.rs | 1116 ------- nvx-rs/src/host/template/store/capture.rs | 1333 -------- .../src/host/template/store/capture/tests.rs | 1439 -------- nvx-rs/src/host/template/store/leases.rs | 1075 ------ .../src/host/template/store/leases/tests.rs | 2956 ----------------- nvx-rs/src/host/template/store/tests.rs | 988 ------ nvx-rs/src/host/vm_directory/tests.rs | 1 - nvx-rs/src/lib.rs | 2 +- nvx-rs/tests/openvmm_vectors.rs | 8 +- nvx-rs/tests/template_capture.rs | 697 ---- nvx-rs/tests/vectors/README.md | 26 +- 52 files changed, 237 insertions(+), 12627 deletions(-) delete mode 100644 nvx-rs/src/host/template/mod.rs delete mode 100644 nvx-rs/src/host/template/plan.rs delete mode 100644 nvx-rs/src/host/template/plan/tests.rs delete mode 100644 nvx-rs/src/host/template/store.rs delete mode 100644 nvx-rs/src/host/template/store/capture.rs delete mode 100644 nvx-rs/src/host/template/store/capture/tests.rs delete mode 100644 nvx-rs/src/host/template/store/leases.rs delete mode 100644 nvx-rs/src/host/template/store/leases/tests.rs delete mode 100644 nvx-rs/src/host/template/store/tests.rs delete mode 100644 nvx-rs/tests/template_capture.rs diff --git a/doc/acceptance-case-matrix.md b/doc/acceptance-case-matrix.md index d5fa82c3b..612d74bc6 100644 --- a/doc/acceptance-case-matrix.md +++ b/doc/acceptance-case-matrix.md @@ -32,7 +32,7 @@ its own in turn. | [The adapter's other tests](#the-adapters-other-tests) | 169 | 0 | 9 | 160 | | [MicroVM scenarios](#microvm-scenarios) | 54 | 51 | 2 | 1 | | [CI](#ci) | 69 | 43 | 3 | 23 | -| [Docs](#docs) | 36 | 34 | 1 | 1 | +| [Docs](#docs) | 36 | 32 | 1 | 3 | ## The adapter's scenarios @@ -498,15 +498,15 @@ this matrix. The README's own code is no doctest. | `nvx-rs/src/guest/terminal.rs - guest::terminal (line 10)` | kept | | | `nvx-rs/src/guest/time/mod.rs - guest::time (line 15)` | kept | | | `nvx-rs/src/host/sandbox/prepared.rs - host::sandbox::prepared::PreparedLaunch (line 92)` | kept | Its launch type leaves with the old launch paths (#1087), whose PR retires it | -| `nvx-rs/src/host/store/mod.rs - host::store (line 75)` | kept | The sandbox store leaves (#1121), whose PR retires it | -| `nvx-rs/src/host/supervisor/mod.rs - host::supervisor (line 33)` | kept | The old supervisor leaves (#1087), whose PR retires it | +| `nvx-rs/src/host/store/mod.rs - host::store (line 72)` | kept | The sandbox store leaves (#1121), whose PR retires it | +| `nvx-rs/src/host/supervisor/mod.rs - host::supervisor (line 32)` | kept | The old supervisor leaves (#1087), whose PR retires it | | `nvx-rs/src/host/template/clone.rs - host::template::clone::ClonePlan (line 247)` | retired | `ClonePlan` and `CloneRequest` leave with the old clone path (SL6.2b, #1121) | -| `nvx-rs/src/host/template/plan.rs - host::template::plan::TemplatePlan (line 137)` | kept | The template store leaves (SL6.2a, #1121), whose PR retires it | -| `nvx-rs/src/host/template/store.rs - host::template::store::Store (line 473)` | kept | The template store leaves (SL6.2a, #1121), whose PR retires it | +| `nvx-rs/src/host/template/plan.rs - host::template::plan::TemplatePlan (line 137)` | retired | `TemplatePlan` leaves with the template store (SL6.2a, #1121) | +| `nvx-rs/src/host/template/store.rs - host::template::store::Store (line 473)` | retired | The template store leaves (SL6.2a, #1121) | | `nvx-rs/src/host/transport.rs - host::transport (line 22)` | kept | | | `nvx-rs/src/lib.rs - (line 58)` | kept | | | `nvx-rs/src/protocol/nvxs/mod.rs - protocol::nvxs (line 68)` | kept | | | `nvx-rs/src/host/snapshot/capture.rs - host::snapshot::capture::finish_capture (line 50)` | kept | | -| `nvx-rs/src/host/store/mod.rs - host::store (line 107)` | kept | The sandbox store leaves (#1121), whose PR retires it | +| `nvx-rs/src/host/store/mod.rs - host::store (line 104)` | kept | The sandbox store leaves (#1121), whose PR retires it | | `nvx-rs/README.md`, its Example: an instance through `LaunchSpec` and the sandbox store | rewritten | Nothing compiles it. `nvx-rs/examples/host-workflows.rs` compiles the documented host workflows, and the library's guide (SL6.10a) replaces the example | | `nvx-rs/examples/external-launch.rs` | kept | Compiled with the crate's targets, and run by hand on a host | diff --git a/doc/design/1.5-compatibility-and-versioning-model.md b/doc/design/1.5-compatibility-and-versioning-model.md index 01097f645..0a8926e9f 100644 --- a/doc/design/1.5-compatibility-and-versioning-model.md +++ b/doc/design/1.5-compatibility-and-versioning-model.md @@ -78,9 +78,8 @@ guest's address and the identity of the complete deny-all policy under which the capture attached it. Every restore of the template replaces that policy with its clone's own, so a template serves the clones of every network policy at its guest's address, and its key records none of them. The key's encoding, -version 1, has not been released, so a template store that an earlier build -created is not supported: delete it and create a new one. There is no -migration. +version 1, has not been released, so no key that an earlier build encoded is +supported. There is no migration. ## 1.5.3 Persisted State diff --git a/doc/design/5.3-snapshot-protocol.md b/doc/design/5.3-snapshot-protocol.md index 81e62151f..53f2cc16d 100644 --- a/doc/design/5.3-snapshot-protocol.md +++ b/doc/design/5.3-snapshot-protocol.md @@ -234,8 +234,7 @@ an endpoint that stops reading, stalls the capture, with nothing published and the source never resumed, until the process that launched the VMM ends it; the VMM logs each state unit that has not answered after five seconds, so that the log of such a capture names it. Every launcher therefore bounds a capture with -a deadline that ends the VMM, as the nvx crate's template store does with the -deadline that creating or refreshing a template requires. +a deadline that ends the VMM. ## 5.3.4 State-Unit Ordering diff --git a/nvx-rs/README.md b/nvx-rs/README.md index b66f3979c..f6deded0e 100644 --- a/nvx-rs/README.md +++ b/nvx-rs/README.md @@ -48,7 +48,7 @@ nvx-rs/ │ ├── host/ feature `host` (default): the runtime that every launch shares, VM │ │ directories and configurations, artifacts, launch, network and egress policy │ │ files, filesystem, NVXS connection, session, process, platform, VM instances, -│ │ sandbox stores, templates' manifests, compatibility keys, store, and leases, +│ │ sandbox stores, templates' manifests and compatibility keys, │ │ outcome reports, file seals, and the sandbox launch profile of `nvx-rs sandbox`, │ │ its distro and scratch bindings, and its prepared launches; with feature │ │ `cli`, the `nvx-rs` CLI @@ -64,7 +64,7 @@ nvx-rs/ │ │ the typed calls through which it runs the time ABI's steps and the reseed │ └── bin/ nvx-rs, nvx-fake-openvmm, and the guest binaries ├── tests/ fake-OpenVMM, CLI, NVXS session, process-supervision, checked-handoff, store, -│ host-module layering, template-capture, and guest-binary tests, the CLI's +│ host-module layering, and guest-binary tests, the CLI's │ scenario tests and their harness, in cli/, the store's record fixtures, the │ hand-written sandbox configurations of every format, the cases of the export │ and root boot tokens, and the template manifests that OpenVMM wrote, in @@ -139,7 +139,6 @@ changes only when the module's purpose does. | [`nvx::host::transport`](#nvxhosttransport) | A verified VM endpoint connection and the blocking host NVXS driver, with the existing timed transport | | [`nvx::host::process`](#nvxhostprocess) | Verified references to OpenVMM processes, through which callers wait for them and end them | | [`nvx::host::supervisor`](#nvxhostsupervisor) | The supervisor of a sandbox's VMM, beneath the CLI, `Instance`, and external orchestrators | -| [`nvx::host::template`](#nvxhosttemplate) | Legacy template capture plans and store, with aliases of the snapshot metadata | | [`nvx::host::seal`](#nvxhostseal) | `FileSeal`: what the host reports about an open file from its metadata alone, without reading it | | [`nvx::host::session`, `platform`, `report`](#nvxhostsession-platform-report) | The authenticated control session, Linux and Windows primitives, and OpenVMM's outcome report | | [`nvx::host::sandbox`](#nvxhostsandbox) | The launch profile of `nvx-rs sandbox`: its plans, live shares, option checks, bindings, and prepared launches | @@ -655,8 +654,39 @@ responsive shell input, and a verified clean exit; it downloads no artifact. `SnapshotKey` and `SnapshotManifest` own the compatibility key and strict OpenVMM manifest reader. `SnapshotManifest::time` and `SnapshotKey::time` expose the capture's `TimeContract`, whose `eligibility` checks its restore window at a `HostTime`; -`HostTime::now` samples the host's clock and identities. The existing -`host::template` names remain aliases of these same types. +`HostTime::now` samples the host's clock and identities. + +`SnapshotManifest::read` reads the `manifest.bin` that OpenVMM wrote for a +capture, at most `MAX_MANIFEST_LEN` bytes of a regular file that is not a link +or other reparse point, and refuses another snapshot format version, an unknown, +mistyped, or repeated field, truncated bytes, and a capture that is not a +template's: a platform-tier capture for clones, with a read-only `distro` whose +identity is unbound, a fresh scratch, and the dormant filesystem slot. + +`CaptureClass` holds the invariant inputs of a capture, known before it: the +runtime, the guest's build, the backend, the requested CPU profile, the +processors, those online at boot, and RAM, the geometries of the disks, the +network, of which it takes what the template captures alone, its NIC at the +guest's address under the complete deny-all policy, and the adapter's other +`Invariants`. + +`SnapshotKey::new` checks that the capture made the machine that its class +requested, and derives the template's compatibility key, from metadata alone: +the runtime and contract versions through which its clones are restored, the +backend, CPU profile and effective CPUID, topology, rates, resources, kernel +command line, disks, and devices of its machine, with the NIC's addresses and +the identity of the complete deny-all policy under which the capture attached +it, which each clone's restore replaces with the clone's own, so that a template +with a NIC serves the clones of every network policy at its guest's address. + +A key and a class each have one canonical, versioned encoding, whose +domain-separated SHA-256 digest is its identity, a `KeyIdentity`, which converts +to and from the `nvx::activation::Identity` by which a clone's lease and durable records +name its template's key, or a `ClassIdentity`; the +[conformance vectors](tests/vectors/README.md) of version 1, +`tests/vectors/snapshot-key-v1.txt`, pin both encodings and identities. The +key's encoding, version 1, leaves the network policy out, since templates +capture their NIC under the complete deny-all policy. The key's canonical encoding and identity use the snapshot names and omit `guest_build`. Version 1 changes in place: no snapshot outlives a deployment, @@ -669,6 +699,13 @@ anchors, host and boot identities, lineage and cut-to-anchor time: captures at different times can have different key bytes and the same `KeyIdentity`. The declared rates and time ABI remain compatibility inputs. +`TimeContract::eligibility` decides, apart from compatibility, whether a clone +may restore a capture at a host's time, a `HostTime`, which `HostTime::now` +samples as OpenVMM does: within the boot of the capture, by the host's monotonic +clock, and otherwise by UTC, the downtime must not be negative, and the interval +by which a restore advances the clone's VM time at most `MAX_DOWNTIME_NS`, 30 +days. + `Snapshot::open` checks exactly `key.bin`, `snapshot.json` and `openvmm/{manifest.bin,memory.bin,state.bin}`. Directories are private, files are regular and private to their owner, artifact lengths equal the strict @@ -832,7 +869,7 @@ with and what a restore may grow it to; and the caller's `Invariants`, at most 64 settings that NVX keeps without interpreting them, each named by 1 to 64 lowercase ASCII letters, digits, `.`, `-`, and `_` from a letter, with a value of at most 256 printable ASCII characters. `Invariants` and `BlockGeometry` -belong here, and `nvx::host::template` re-exports them where they were. +belong here. `CaptureConfig::check` checks a capture without reading the host. It fails with an `nvx::Error` of kind `Unsupported` for processors at boot or a capacity other @@ -1165,12 +1202,11 @@ orchestrators, which keep the configuration, the authorization, and the application protocol. `launch` runs the store's launch steps under the sandbox's lock, as a launch -whose `LaunchId` the caller reserved beforehand, a fresh one for each launch, or -that a clone's lease reserves: it writes the journal's marker, which names the -launch, spawns OpenVMM with the claimed log, records its `ProcessRef` right -after, and returns a `Starting` launch that the caller commits once the VMM -serves its control endpoint, or abandons. A launch that fails never leaves an -unrecorded VMM running. +whose `LaunchId` the caller reserved beforehand, a fresh one for each launch: it +writes the journal's marker, which names the launch, spawns OpenVMM with the +claimed log, records its `ProcessRef` right after, and returns a `Starting` +launch that the caller commits once the VMM serves its control endpoint, or +abandons. A launch that fails never leaves an unrecorded VMM running. `spawn_vm` starts a caller-owned command without a store, with the same strict Windows log-job requirement as `vm::spawn`: an untrusted job name or a log that @@ -1226,12 +1262,6 @@ identity. after the orchestrator restarts within the same host boot: `Live`, `Mismatched` with another runtime identity, `Exited`, or `Indeterminate`. -A clone's launch takes its locks in one order, the template store's, then the -sandbox's, which guards its clone binding and launch journal, then the scratch -image's claim, and holds none for the VM's lifetime; its lease lasts through -crashes and reattaches, until a release ends it once a stop, a termination, or a -reattach has verified the VM's exit. - #### Windows working-directory limit Windows' process-creation API limits `VmmCommand::working_dir` to 258 UTF-16 @@ -1241,157 +1271,6 @@ refuses an overlong working directory with `InvalidInput`; high-level instance and capture preflight return `Unsupported` with the limit in the top-level message. Neither silently changes the requested directory. -### `nvx::host::template` - -The legacy plans and store of templates, captures of a microVM from which -independent clones are restored. Their metadata names remain aliases of -[`host::snapshot`](#nvxhostsnapshot)'s types. - -`TemplatePlan::new` lowers an adapter's `SandboxPlan` into the plan of a -template's capture, without I/O: the placeholder `distro`, which its -`DistroBinding` binds, and whose length is the capacity of every clone's image, -over a dormant scratch device, the sparse file `SCRATCH_NAME` in the capture's -transaction directory, with the dormant filesystem slot, and no NIC or one at -the guest's address of the adapter's network, under the complete deny-all -policy, whatever the network's own, which each clone's restore replaces with the -clone's; `TemplateResources`, with the processors, 1, 2, 4, or 8, and those -online at boot, as many or fewer, and RAM and a `--memory-capacity` in blocks of -`MEMORY_BLOCK_SIZE`; the kernel command line `nvx_sandbox=1`, with `maxcpus=` -and the processors online at boot when they are fewer, so that the capture's -whole command line holds tokens of OpenVMM's platform allowlist alone, and no -tenant data, which the tests check with a copy of the allowlist's source; and -the capture's options, `--snapshot-tier platform`, a `--snapshot-destination`, -`SNAPSHOT_NAME` in the transaction directory, and -`--snapshot-block-identity generation` with the capture's bound-block -`StorageGeneration`, so that the capture hashes no block device. - -`TemplateManifest::read` reads the `manifest.bin` that OpenVMM wrote for a -capture, at most `MAX_MANIFEST_LEN` bytes of a regular file that is not a link -or other reparse point, and refuses another snapshot format version, an unknown, -mistyped, or repeated field, truncated bytes, and a capture that is not a -template's: a platform-tier capture for clones, with a read-only `distro` whose -identity is unbound, a fresh scratch, and the dormant filesystem slot. - -`CaptureClass` holds the invariant inputs of a capture, known before it: the -runtime, the guest's build, the backend, the requested CPU profile, the -processors, those online at boot, and RAM, the geometries of the disks, the -network, of which it takes what the template captures alone, its NIC at the -guest's address under the complete deny-all policy, and the adapter's other -`Invariants`. - -`TemplateKey::new` checks that the capture made the machine that its class -requested, and derives the template's compatibility key, from metadata alone: -the runtime and contract versions through which its clones are restored, the -backend, CPU profile and effective CPUID, topology, rates, resources, kernel -command line, disks, and devices of its machine, with the NIC's addresses and -the identity of the complete deny-all policy under which the capture attached -it, which each clone's restore replaces with the clone's own, so that a template -with a NIC serves the clones of every network policy at its guest's address. A -capture's own metadata, such as its time anchor and host, stays out of the key. - -A key and a class each have one canonical, versioned encoding, whose -domain-separated SHA-256 digest is its identity, a `KeyIdentity`, which converts -to and from the `nvx::activation::Identity` by which a clone's lease and durable records -name its template's key, or a `ClassIdentity`; the -[conformance vectors](tests/vectors/README.md) of version 1, -`tests/vectors/snapshot-key-v1.txt`, pin both encodings and identities. The -key's encoding, version 1, leaves the network policy out, since templates -capture their NIC under the complete deny-all policy, and has not been released, -so a template store that an earlier build created is not supported: delete it -and create a new one. There is no migration. - -`Store` keeps the generations of templates that a host published, apart from -every sandbox's store, at a location that the adapter chooses: a private -directory with an identity of its own, a `StoreId`, the immutable payload of -each generation, its key's canonical encoding beside the snapshot that OpenVMM -published, and the versioned `Index` of them, which lists each generation with -the identities of its key and capture class and its own metadata, such as its -time contract. - -`Store::create` creates a store, or opens the one that its directory holds, -crash-safely and once between concurrent creations, and refuses a directory with -foreign entries; `Store::open` opens one. - -Its records are JSON in one format, replaced atomically, and records of another -format fail rather than being migrated. Its directories must be private to the -current user, and it refuses links and, on Windows, other reparse points among -its directories and files. - -Four namespaces stay apart: the `nvx::activation::StoreGeneration`, the host's -publication ID, which the index assigns in the order of publication and never -twice, and by which a clone's lease and durable records name its template; the -bound-block `StorageGeneration` of a capture, which belongs to the storage -owner; the time lineage counter of the time contract; and the VM generation ID, -which the store never records. - -`TimeContract::eligibility` decides, apart from compatibility, whether a clone -may restore a capture at a host's time, a `HostTime`, which `HostTime::now` -samples as OpenVMM does: within the boot of the capture, by the host's monotonic -clock, and otherwise by UTC, the downtime must not be negative, and the interval -by which a restore advances the clone's VM time at most `MAX_DOWNTIME_NS`, 30 -days. - -`Store::select` selects the latest eligible generation of a key that no -tombstone quarantines. - -`Store::acquire` leases a generation to a clone in one transaction, under the -store's lock and then the sandbox's: it checks the generation's eligibility, -writes the lease's record in the store and the clone binding, `clone.json`, -beside the sandbox's launch journal, and returns the lease with the sandbox's -lock, the only one under which the caller launches the clone, once, as the -launch that the lease reserves, `Lease::launch`. - -A lease never expires: `Store::release` ends it once the sandbox records no VM -and no launch. - -On Linux, where OpenVMM requires a restore's control socket in the snapshot's -parent, the clone's VMM listens in the generation's payload, at `.sock`, -named after the launch that the lease reserves; on Windows, it listens at a -named pipe, as every launch does. - -A payload holds the sockets of its clones only while they lease it: the VMM -removes its socket as it exits, a stop or a recovery removes the one that an -abrupt exit leaves, and the release makes the socket's absence durable before -the lease ends, and removes the record and `restore.sock`. - -`Store::tombstone` quarantines a generation, which takes no new lease while its -leases keep working, and `Store::retire` deletes it once its leases have -drained, crash-safely, removing the records of leases that no clone binding -names any more. - -`Store::create_template` serves a `Capture`, an adapter's plan for a capture -class with the placeholder's binding and the runtime's artifacts, with a -generation of its class that the store holds and that is eligible at the host's -time, which it reads once it holds the class's slot, `Created::Existing`, or -captures one, `Created::Captured`, while `Store::refresh_template` always -captures, and leaves older generations and their leases untouched. - -A capture runs in its class's slot under `captures/`, a sandbox store whose lock -serializes the class's captures, which comes before the store's lock, and which -keeps the log and outcome report of the VMM of the class's latest capture: in a -private transaction on the store's volume, which is also the sandbox store of -its VM, since OpenVMM requires the control socket of a VM that captures a -snapshot, on Linux, beside the snapshot's destination, it creates the dormant -scratch, checks the placeholder through its binding, and both block devices at -their paths, regular files that are not links or other reparse points, that -belong to the user or an administrator, and that no one else may change, and -launches the template's VM through `supervisor::launch` from the transaction, -holding no handle that could write either file, since OpenVMM opens them itself -by path, as it does a restore's. Once the VMM has exited, it checks the outcome -report, the snapshot's manifest and artifacts, and the key, and publishes the -transaction in two commits, a rename into `generations/` that replaces nothing, -then the index. - -Capture preflight applies the -[Windows working-directory limit](#windows-working-directory-limit) to -`\captures\<16-hex-slot>\transaction` before creating the slot -or scratch. Without a trailing separator this leaves 220 units for the absolute -template-store path, or 210 for a state root whose store is `\templates`. - -A capture that fails publishes nothing, and `Store::recover_captures` ends the -captures that a crash interrupted, publishing those that were checked, and -retiring the others. - ### `nvx::host::seal` `FileSeal`: what the host reports about an open file from its metadata alone, @@ -3271,9 +3150,7 @@ tests, only the test hooks that [Test layout](../doc/contribute.md#test-layout) describes. The tests need no hypervisor. The fake-OpenVMM test drives `Instance` end to end -against `nvx-fake-openvmm`, and the template-capture test runs templates' -captures through the supervisor against it, up to a crash of the adapter whose -VMM a recovery ends. The CLI test runs the +against `nvx-fake-openvmm`. The CLI test runs the [scenario tests](#scenario-tests) of `nvx-rs`, and checks the guest-feature requirement and, on Linux, Ctrl-C. The [conformance vectors](tests/vectors/README.md) of the wire formats cover both diff --git a/nvx-rs/public-api/linux.txt b/nvx-rs/public-api/linux.txt index aeae95369..0bdbcb645 100644 --- a/nvx-rs/public-api/linux.txt +++ b/nvx-rs/public-api/linux.txt @@ -3314,228 +3314,6 @@ fn nvx::host::supervisor::reattach fn nvx::host::supervisor::spawn fn nvx::host::supervisor::spawn_command fn nvx::host::supervisor::spawn_vm -mod nvx::host::template -struct nvx::host::template::Acquired -field nvx::host::template::Acquired::lease -field nvx::host::template::Acquired::locked -struct nvx::host::template::BlockGeometry -field nvx::host::template::BlockGeometry::length -field nvx::host::template::BlockGeometry::logical_block_size -field nvx::host::template::BlockGeometry::physical_block_size -const nvx::host::template::CAPTURES_NAME -const nvx::host::template::CAPTURE_RECORD_NAME -struct nvx::host::template::Capture -field nvx::host::template::Capture::adapter -field nvx::host::template::Capture::artifacts -field nvx::host::template::Capture::binding -field nvx::host::template::Capture::class -field nvx::host::template::Capture::policy_revision -struct nvx::host::template::CaptureClass -const nvx::host::template::CaptureClass::VERSION -field nvx::host::template::CaptureClass::boot_processors -method nvx::host::template::CaptureClass::check -field nvx::host::template::CaptureClass::cpu_profile -field nvx::host::template::CaptureClass::distro -field nvx::host::template::CaptureClass::guest_build -field nvx::host::template::CaptureClass::hypervisor -method nvx::host::template::CaptureClass::identity -field nvx::host::template::CaptureClass::invariants -field nvx::host::template::CaptureClass::memory -field nvx::host::template::CaptureClass::memory_capacity -field nvx::host::template::CaptureClass::network -field nvx::host::template::CaptureClass::processors -field nvx::host::template::CaptureClass::runtime -field nvx::host::template::CaptureClass::scratch -method nvx::host::template::CaptureClass::to_bytes -struct nvx::host::template::ClassIdentity -method nvx::host::template::ClassIdentity::as_bytes -method nvx::host::template::ClassIdentity::from_bytes -enum nvx::host::template::Created -variant nvx::host::template::Created::Captured -field nvx::host::template::Created::Captured::0 -variant nvx::host::template::Created::Existing -field nvx::host::template::Created::Existing::0 -method nvx::host::template::Created::generation -method nvx::host::template::Created::into_generation -struct nvx::host::template::Downtime -field nvx::host::template::Downtime::nanos -field nvx::host::template::Downtime::source -field nvx::host::template::Downtime::vm_time_nanos -enum nvx::host::template::DowntimeSource -variant nvx::host::template::DowntimeSource::HostMonotonic -variant nvx::host::template::DowntimeSource::Utc -enum nvx::host::template::Eligibility -variant nvx::host::template::Eligibility::Early -field nvx::host::template::Eligibility::Early::before_ns -field nvx::host::template::Eligibility::Early::source -variant nvx::host::template::Eligibility::Eligible -field nvx::host::template::Eligibility::Eligible::0 -variant nvx::host::template::Eligibility::Expired -field nvx::host::template::Eligibility::Expired::0 -method nvx::host::template::Eligibility::is_eligible -const nvx::host::template::GENERATIONS_NAME -struct nvx::host::template::Generation -method nvx::host::template::Generation::class -method nvx::host::template::Generation::created_at -method nvx::host::template::Generation::eligibility -method nvx::host::template::Generation::generation -method nvx::host::template::Generation::key -method nvx::host::template::Generation::memory_len -method nvx::host::template::Generation::state_len -method nvx::host::template::Generation::storage_generation -method nvx::host::template::Generation::time -enum nvx::host::template::HostClock -variant nvx::host::template::HostClock::LinuxBoottime -variant nvx::host::template::HostClock::WindowsInterruptTime -method nvx::host::template::HostClock::as_str -struct nvx::host::template::HostTime -field nvx::host::template::HostTime::boot_id -field nvx::host::template::HostTime::clock -field nvx::host::template::HostTime::host_id -field nvx::host::template::HostTime::monotonic_ns -method nvx::host::template::HostTime::now -field nvx::host::template::HostTime::utc_ns -const nvx::host::template::INDEX_LOCK_NAME -const nvx::host::template::INDEX_NAME -struct nvx::host::template::Index -method nvx::host::template::Index::generations -method nvx::host::template::Index::get -method nvx::host::template::Index::next_generation -method nvx::host::template::Index::of_key -struct nvx::host::template::Invariants -const nvx::host::template::Invariants::MAX_NAME_LEN -const nvx::host::template::Invariants::MAX_SETTINGS -const nvx::host::template::Invariants::MAX_VALUE_LEN -method nvx::host::template::Invariants::get -method nvx::host::template::Invariants::is_empty -method nvx::host::template::Invariants::iter -method nvx::host::template::Invariants::len -method nvx::host::template::Invariants::new -method nvx::host::template::Invariants::with -const nvx::host::template::KEY_NAME -struct nvx::host::template::KeyIdentity -method nvx::host::template::KeyIdentity::as_bytes -method nvx::host::template::KeyIdentity::from_bytes -const nvx::host::template::LEASES_NAME -struct nvx::host::template::Lease -method nvx::host::template::Lease::generation -method nvx::host::template::Lease::id -method nvx::host::template::Lease::key -method nvx::host::template::Lease::launch -method nvx::host::template::Lease::of -method nvx::host::template::Lease::sandbox -method nvx::host::template::Lease::store -method nvx::host::template::Lease::template -struct nvx::host::template::LeaseId -method nvx::host::template::LeaseId::as_bytes -method nvx::host::template::LeaseId::from_bytes -const nvx::host::template::MANIFEST_NAME -const nvx::host::template::MAX_DOWNTIME_NS -const nvx::host::template::MAX_GENERATIONS -const nvx::host::template::MAX_MANIFEST_LEN -const nvx::host::template::MEMORY_BLOCK_SIZE -enum nvx::host::template::Retirement -variant nvx::host::template::Retirement::Draining -field nvx::host::template::Retirement::Draining::0 -variant nvx::host::template::Retirement::Retired -const nvx::host::template::SCRATCH_NAME -const nvx::host::template::SNAPSHOT_NAME -const nvx::host::template::STORE_FORMAT -const nvx::host::template::STORE_RECORD_NAME -struct nvx::host::template::StorageGeneration -method nvx::host::template::StorageGeneration::as_bytes -method nvx::host::template::StorageGeneration::from_bytes -method nvx::host::template::StorageGeneration::generate -struct nvx::host::template::Store -method nvx::host::template::Store::acquire -method nvx::host::template::Store::create -method nvx::host::template::Store::create_template -method nvx::host::template::Store::dir -method nvx::host::template::Store::id -method nvx::host::template::Store::index -method nvx::host::template::Store::leases -method nvx::host::template::Store::open -method nvx::host::template::Store::payload_dir -method nvx::host::template::Store::read_key -method nvx::host::template::Store::recover_captures -method nvx::host::template::Store::refresh_template -method nvx::host::template::Store::release -method nvx::host::template::Store::retire -method nvx::host::template::Store::select -method nvx::host::template::Store::snapshot_dir -method nvx::host::template::Store::tombstone -method nvx::host::template::Store::tombstones -struct nvx::host::template::StoreId -method nvx::host::template::StoreId::as_bytes -method nvx::host::template::StoreId::from_bytes -const nvx::host::template::TOMBSTONES_NAME -const nvx::host::template::TRANSACTION_NAME -struct nvx::host::template::TemplateKey -const nvx::host::template::TemplateKey::MAX_LEN -const nvx::host::template::TemplateKey::VERSION -method nvx::host::template::TemplateKey::boot_processors -method nvx::host::template::TemplateKey::command_line -method nvx::host::template::TemplateKey::cpu_profile -method nvx::host::template::TemplateKey::distro -method nvx::host::template::TemplateKey::filesystem_tag -method nvx::host::template::TemplateKey::from_bytes -method nvx::host::template::TemplateKey::guest_network -method nvx::host::template::TemplateKey::has_network -method nvx::host::template::TemplateKey::hypervisor -method nvx::host::template::TemplateKey::identity -method nvx::host::template::TemplateKey::invariants -method nvx::host::template::TemplateKey::memory -method nvx::host::template::TemplateKey::memory_block_size -method nvx::host::template::TemplateKey::memory_capacity -method nvx::host::template::TemplateKey::new -method nvx::host::template::TemplateKey::processors -method nvx::host::template::TemplateKey::runtime -method nvx::host::template::TemplateKey::scratch -method nvx::host::template::TemplateKey::speaks_current_contracts -method nvx::host::template::TemplateKey::time -method nvx::host::template::TemplateKey::to_bytes -struct nvx::host::template::TemplateManifest -method nvx::host::template::TemplateManifest::created_at -method nvx::host::template::TemplateManifest::hypervisor -method nvx::host::template::TemplateManifest::memory_len -method nvx::host::template::TemplateManifest::openvmm_version -method nvx::host::template::TemplateManifest::parse -method nvx::host::template::TemplateManifest::read -method nvx::host::template::TemplateManifest::state_len -method nvx::host::template::TemplateManifest::time -struct nvx::host::template::TemplatePlan -method nvx::host::template::TemplatePlan::binding -method nvx::host::template::TemplatePlan::capture_arguments -method nvx::host::template::TemplatePlan::destination -method nvx::host::template::TemplatePlan::generation -method nvx::host::template::TemplatePlan::image_capacity -method nvx::host::template::TemplatePlan::kernel_command_line -method nvx::host::template::TemplatePlan::new -method nvx::host::template::TemplatePlan::nic_arguments -method nvx::host::template::TemplatePlan::openvmm_arguments -method nvx::host::template::TemplatePlan::plan -method nvx::host::template::TemplatePlan::resources -method nvx::host::template::TemplatePlan::scratch_length -method nvx::host::template::TemplatePlan::scratch_path -method nvx::host::template::TemplatePlan::transaction -struct nvx::host::template::TemplateResources -field nvx::host::template::TemplateResources::boot_processors -field nvx::host::template::TemplateResources::memory -field nvx::host::template::TemplateResources::memory_capacity -field nvx::host::template::TemplateResources::processors -struct nvx::host::template::TimeContract -field nvx::host::template::TimeContract::apic_frequency_hz -field nvx::host::template::TimeContract::boot_id -field nvx::host::template::TimeContract::capture_monotonic_ns -field nvx::host::template::TimeContract::capture_tsc -field nvx::host::template::TimeContract::capture_utc_ns -field nvx::host::template::TimeContract::cut_to_anchor_ns -method nvx::host::template::TimeContract::eligibility -field nvx::host::template::TimeContract::host_clock -field nvx::host::template::TimeContract::host_id -field nvx::host::template::TimeContract::lineage -field nvx::host::template::TimeContract::tsc_frequency_hz -field nvx::host::template::TimeContract::tsc_tolerance_ppm mod nvx::host::transport struct nvx::host::transport::HostConnection method nvx::host::transport::HostConnection::attach diff --git a/nvx-rs/public-api/windows.txt b/nvx-rs/public-api/windows.txt index 1f5e64aff..20edbeac3 100644 --- a/nvx-rs/public-api/windows.txt +++ b/nvx-rs/public-api/windows.txt @@ -1916,228 +1916,6 @@ fn nvx::host::supervisor::reattach fn nvx::host::supervisor::spawn fn nvx::host::supervisor::spawn_command fn nvx::host::supervisor::spawn_vm -mod nvx::host::template -struct nvx::host::template::Acquired -field nvx::host::template::Acquired::lease -field nvx::host::template::Acquired::locked -struct nvx::host::template::BlockGeometry -field nvx::host::template::BlockGeometry::length -field nvx::host::template::BlockGeometry::logical_block_size -field nvx::host::template::BlockGeometry::physical_block_size -const nvx::host::template::CAPTURES_NAME -const nvx::host::template::CAPTURE_RECORD_NAME -struct nvx::host::template::Capture -field nvx::host::template::Capture::adapter -field nvx::host::template::Capture::artifacts -field nvx::host::template::Capture::binding -field nvx::host::template::Capture::class -field nvx::host::template::Capture::policy_revision -struct nvx::host::template::CaptureClass -const nvx::host::template::CaptureClass::VERSION -field nvx::host::template::CaptureClass::boot_processors -method nvx::host::template::CaptureClass::check -field nvx::host::template::CaptureClass::cpu_profile -field nvx::host::template::CaptureClass::distro -field nvx::host::template::CaptureClass::guest_build -field nvx::host::template::CaptureClass::hypervisor -method nvx::host::template::CaptureClass::identity -field nvx::host::template::CaptureClass::invariants -field nvx::host::template::CaptureClass::memory -field nvx::host::template::CaptureClass::memory_capacity -field nvx::host::template::CaptureClass::network -field nvx::host::template::CaptureClass::processors -field nvx::host::template::CaptureClass::runtime -field nvx::host::template::CaptureClass::scratch -method nvx::host::template::CaptureClass::to_bytes -struct nvx::host::template::ClassIdentity -method nvx::host::template::ClassIdentity::as_bytes -method nvx::host::template::ClassIdentity::from_bytes -enum nvx::host::template::Created -variant nvx::host::template::Created::Captured -field nvx::host::template::Created::Captured::0 -variant nvx::host::template::Created::Existing -field nvx::host::template::Created::Existing::0 -method nvx::host::template::Created::generation -method nvx::host::template::Created::into_generation -struct nvx::host::template::Downtime -field nvx::host::template::Downtime::nanos -field nvx::host::template::Downtime::source -field nvx::host::template::Downtime::vm_time_nanos -enum nvx::host::template::DowntimeSource -variant nvx::host::template::DowntimeSource::HostMonotonic -variant nvx::host::template::DowntimeSource::Utc -enum nvx::host::template::Eligibility -variant nvx::host::template::Eligibility::Early -field nvx::host::template::Eligibility::Early::before_ns -field nvx::host::template::Eligibility::Early::source -variant nvx::host::template::Eligibility::Eligible -field nvx::host::template::Eligibility::Eligible::0 -variant nvx::host::template::Eligibility::Expired -field nvx::host::template::Eligibility::Expired::0 -method nvx::host::template::Eligibility::is_eligible -const nvx::host::template::GENERATIONS_NAME -struct nvx::host::template::Generation -method nvx::host::template::Generation::class -method nvx::host::template::Generation::created_at -method nvx::host::template::Generation::eligibility -method nvx::host::template::Generation::generation -method nvx::host::template::Generation::key -method nvx::host::template::Generation::memory_len -method nvx::host::template::Generation::state_len -method nvx::host::template::Generation::storage_generation -method nvx::host::template::Generation::time -enum nvx::host::template::HostClock -variant nvx::host::template::HostClock::LinuxBoottime -variant nvx::host::template::HostClock::WindowsInterruptTime -method nvx::host::template::HostClock::as_str -struct nvx::host::template::HostTime -field nvx::host::template::HostTime::boot_id -field nvx::host::template::HostTime::clock -field nvx::host::template::HostTime::host_id -field nvx::host::template::HostTime::monotonic_ns -method nvx::host::template::HostTime::now -field nvx::host::template::HostTime::utc_ns -const nvx::host::template::INDEX_LOCK_NAME -const nvx::host::template::INDEX_NAME -struct nvx::host::template::Index -method nvx::host::template::Index::generations -method nvx::host::template::Index::get -method nvx::host::template::Index::next_generation -method nvx::host::template::Index::of_key -struct nvx::host::template::Invariants -const nvx::host::template::Invariants::MAX_NAME_LEN -const nvx::host::template::Invariants::MAX_SETTINGS -const nvx::host::template::Invariants::MAX_VALUE_LEN -method nvx::host::template::Invariants::get -method nvx::host::template::Invariants::is_empty -method nvx::host::template::Invariants::iter -method nvx::host::template::Invariants::len -method nvx::host::template::Invariants::new -method nvx::host::template::Invariants::with -const nvx::host::template::KEY_NAME -struct nvx::host::template::KeyIdentity -method nvx::host::template::KeyIdentity::as_bytes -method nvx::host::template::KeyIdentity::from_bytes -const nvx::host::template::LEASES_NAME -struct nvx::host::template::Lease -method nvx::host::template::Lease::generation -method nvx::host::template::Lease::id -method nvx::host::template::Lease::key -method nvx::host::template::Lease::launch -method nvx::host::template::Lease::of -method nvx::host::template::Lease::sandbox -method nvx::host::template::Lease::store -method nvx::host::template::Lease::template -struct nvx::host::template::LeaseId -method nvx::host::template::LeaseId::as_bytes -method nvx::host::template::LeaseId::from_bytes -const nvx::host::template::MANIFEST_NAME -const nvx::host::template::MAX_DOWNTIME_NS -const nvx::host::template::MAX_GENERATIONS -const nvx::host::template::MAX_MANIFEST_LEN -const nvx::host::template::MEMORY_BLOCK_SIZE -enum nvx::host::template::Retirement -variant nvx::host::template::Retirement::Draining -field nvx::host::template::Retirement::Draining::0 -variant nvx::host::template::Retirement::Retired -const nvx::host::template::SCRATCH_NAME -const nvx::host::template::SNAPSHOT_NAME -const nvx::host::template::STORE_FORMAT -const nvx::host::template::STORE_RECORD_NAME -struct nvx::host::template::StorageGeneration -method nvx::host::template::StorageGeneration::as_bytes -method nvx::host::template::StorageGeneration::from_bytes -method nvx::host::template::StorageGeneration::generate -struct nvx::host::template::Store -method nvx::host::template::Store::acquire -method nvx::host::template::Store::create -method nvx::host::template::Store::create_template -method nvx::host::template::Store::dir -method nvx::host::template::Store::id -method nvx::host::template::Store::index -method nvx::host::template::Store::leases -method nvx::host::template::Store::open -method nvx::host::template::Store::payload_dir -method nvx::host::template::Store::read_key -method nvx::host::template::Store::recover_captures -method nvx::host::template::Store::refresh_template -method nvx::host::template::Store::release -method nvx::host::template::Store::retire -method nvx::host::template::Store::select -method nvx::host::template::Store::snapshot_dir -method nvx::host::template::Store::tombstone -method nvx::host::template::Store::tombstones -struct nvx::host::template::StoreId -method nvx::host::template::StoreId::as_bytes -method nvx::host::template::StoreId::from_bytes -const nvx::host::template::TOMBSTONES_NAME -const nvx::host::template::TRANSACTION_NAME -struct nvx::host::template::TemplateKey -const nvx::host::template::TemplateKey::MAX_LEN -const nvx::host::template::TemplateKey::VERSION -method nvx::host::template::TemplateKey::boot_processors -method nvx::host::template::TemplateKey::command_line -method nvx::host::template::TemplateKey::cpu_profile -method nvx::host::template::TemplateKey::distro -method nvx::host::template::TemplateKey::filesystem_tag -method nvx::host::template::TemplateKey::from_bytes -method nvx::host::template::TemplateKey::guest_network -method nvx::host::template::TemplateKey::has_network -method nvx::host::template::TemplateKey::hypervisor -method nvx::host::template::TemplateKey::identity -method nvx::host::template::TemplateKey::invariants -method nvx::host::template::TemplateKey::memory -method nvx::host::template::TemplateKey::memory_block_size -method nvx::host::template::TemplateKey::memory_capacity -method nvx::host::template::TemplateKey::new -method nvx::host::template::TemplateKey::processors -method nvx::host::template::TemplateKey::runtime -method nvx::host::template::TemplateKey::scratch -method nvx::host::template::TemplateKey::speaks_current_contracts -method nvx::host::template::TemplateKey::time -method nvx::host::template::TemplateKey::to_bytes -struct nvx::host::template::TemplateManifest -method nvx::host::template::TemplateManifest::created_at -method nvx::host::template::TemplateManifest::hypervisor -method nvx::host::template::TemplateManifest::memory_len -method nvx::host::template::TemplateManifest::openvmm_version -method nvx::host::template::TemplateManifest::parse -method nvx::host::template::TemplateManifest::read -method nvx::host::template::TemplateManifest::state_len -method nvx::host::template::TemplateManifest::time -struct nvx::host::template::TemplatePlan -method nvx::host::template::TemplatePlan::binding -method nvx::host::template::TemplatePlan::capture_arguments -method nvx::host::template::TemplatePlan::destination -method nvx::host::template::TemplatePlan::generation -method nvx::host::template::TemplatePlan::image_capacity -method nvx::host::template::TemplatePlan::kernel_command_line -method nvx::host::template::TemplatePlan::new -method nvx::host::template::TemplatePlan::nic_arguments -method nvx::host::template::TemplatePlan::openvmm_arguments -method nvx::host::template::TemplatePlan::plan -method nvx::host::template::TemplatePlan::resources -method nvx::host::template::TemplatePlan::scratch_length -method nvx::host::template::TemplatePlan::scratch_path -method nvx::host::template::TemplatePlan::transaction -struct nvx::host::template::TemplateResources -field nvx::host::template::TemplateResources::boot_processors -field nvx::host::template::TemplateResources::memory -field nvx::host::template::TemplateResources::memory_capacity -field nvx::host::template::TemplateResources::processors -struct nvx::host::template::TimeContract -field nvx::host::template::TimeContract::apic_frequency_hz -field nvx::host::template::TimeContract::boot_id -field nvx::host::template::TimeContract::capture_monotonic_ns -field nvx::host::template::TimeContract::capture_tsc -field nvx::host::template::TimeContract::capture_utc_ns -field nvx::host::template::TimeContract::cut_to_anchor_ns -method nvx::host::template::TimeContract::eligibility -field nvx::host::template::TimeContract::host_clock -field nvx::host::template::TimeContract::host_id -field nvx::host::template::TimeContract::lineage -field nvx::host::template::TimeContract::tsc_frequency_hz -field nvx::host::template::TimeContract::tsc_tolerance_ppm mod nvx::host::transport struct nvx::host::transport::HostConnection method nvx::host::transport::HostConnection::attach diff --git a/nvx-rs/src/contract.rs b/nvx-rs/src/contract.rs index 666a817ca..33304cfb9 100644 --- a/nvx-rs/src/contract.rs +++ b/nvx-rs/src/contract.rs @@ -27,7 +27,7 @@ //! A release declares the microVM ABI, time ABI, restore-packet, control-session, and control //! contract versions of its artifacts in its `SOURCE-MANIFEST.json`. //! -//! The compatibility key of a template, `nvx::host::template::TemplateKey`, holds the boot and +//! The compatibility key of a template, `nvx::host::snapshot::SnapshotKey`, holds the boot and //! control versions, and the versions of the contracts through which a template's clones are //! restored, which no runtime identity records: //! diff --git a/nvx-rs/src/host/mod.rs b/nvx-rs/src/host/mod.rs index cecb93c0f..8ec3e1251 100644 --- a/nvx-rs/src/host/mod.rs +++ b/nvx-rs/src/host/mod.rs @@ -32,9 +32,6 @@ //! - [`snapshot`] holds the compatibility key, [`SnapshotKey`](snapshot::SnapshotKey), derived //! from a [`CaptureClass`](snapshot::CaptureClass) and [`SnapshotManifest`](snapshot::SnapshotManifest), //! the strict manifest reader, and the capture's public time and eligibility information. -//! - [`template`] retains the legacy capture and clone plans, the [`Store`](template::Store) of -//! published generations and their leases, and aliases of the snapshot metadata. Its -//! [`TemplatePlan`](template::TemplatePlan) lowers an adapter's plan for a capture. //! - [`session`] speaks NVX's control protocol, `NVXC`, over [`transport`], which verifies a VM's //! endpoint and runs the host's role of the NVXS session for it and for any other application //! protocol, over a platform transport. [`nvxs`] retains compatibility paths. [`platform`] holds @@ -105,7 +102,6 @@ pub mod supervisor; #[cfg(unix)] #[allow(unsafe_code)] mod sys; -pub mod template; #[cfg(test)] pub(crate) mod tests; pub mod transport; diff --git a/nvx-rs/src/host/platform/linux/identity.rs b/nvx-rs/src/host/platform/linux/identity.rs index e32d76604..c9c39b159 100644 --- a/nvx-rs/src/host/platform/linux/identity.rs +++ b/nvx-rs/src/host/platform/linux/identity.rs @@ -7,9 +7,7 @@ use std::os::unix::fs::{MetadataExt, OpenOptionsExt}; use std::path::Path; use nix::sys::stat::Mode; -use nix::unistd::Uid; -use super::private::{owner_and_mode, store_dir_problem}; use crate::host::sys::Errno; /// Returns the device and inode of a file or directory. @@ -168,13 +166,4 @@ impl Directory { pub(crate) fn sync(&self) -> io::Result<()> { self.0.sync_all() } - - /// Returns why the directory cannot hold a sandbox store, as [`check_store_dir`] returns it for - /// a path, if it cannot: unless it belongs to the current user with mode 0700. - /// - /// [`check_store_dir`]: crate::host::platform::linux::check_store_dir - pub(crate) fn store_dir_problem(&self) -> io::Result> { - let (uid, mode) = owner_and_mode(&self.0.metadata()?); - Ok(store_dir_problem(uid, mode, Uid::effective())) - } } diff --git a/nvx-rs/src/host/platform/linux/mod.rs b/nvx-rs/src/host/platform/linux/mod.rs index ae1721964..69fd736a1 100644 --- a/nvx-rs/src/host/platform/linux/mod.rs +++ b/nvx-rs/src/host/platform/linux/mod.rs @@ -27,9 +27,10 @@ pub(crate) use self::identity::{ }; pub(crate) use self::log_claim::{claim_log, log_claim_held, serves_launch}; pub use self::private::create_private_dir; +#[cfg(feature = "cli")] +pub(crate) use self::private::rename_no_replace; pub(crate) use self::private::{ - check_owner_only_dir, check_snapshot_dir, check_store_dir, create_owner_only_dir, - create_store_dir, foreign_writer_problem, make_sparse, private_file_problem, rename_no_replace, + check_snapshot_dir, check_store_dir, create_store_dir, make_sparse, private_file_problem, restore_private_file_mode, }; pub use self::process::process_start_time; diff --git a/nvx-rs/src/host/platform/linux/private.rs b/nvx-rs/src/host/platform/linux/private.rs index b14ee37f7..6aee426bc 100644 --- a/nvx-rs/src/host/platform/linux/private.rs +++ b/nvx-rs/src/host/platform/linux/private.rs @@ -12,7 +12,9 @@ use nix::sys::stat::Mode; use nix::unistd::Uid; use super::{GROUP_AND_OTHER_BITS, PRIVATE_DIR_MODE, STORE_DIR_MODE}; -use crate::host::sys::{self, Errno}; +#[cfg(feature = "cli")] +use crate::host::sys; +use crate::host::sys::Errno; /// Creates `path` if needed and restricts it to the current user. pub fn create_private_dir(path: &Path) -> io::Result<()> { @@ -44,18 +46,6 @@ pub(crate) fn create_store_dir(path: &Path) -> io::Result> { check_store_dir(path) } -/// Creates the directory `path` with mode 0700, or checks that an existing one is private to the -/// current user, as [`check_owner_only_dir`] does, and returns why it is not, if it is not. -/// -/// A new directory takes the setgid bit of its parent, and loses the bits that the umask clears, -/// even the current user's permission to read it, so its mode is set again. An existing directory -/// is never changed: others could have placed files in it, or still hold them open, which a -/// change of its mode would not revoke. -pub(crate) fn create_owner_only_dir(path: &Path) -> io::Result> { - create_dir_of_mode(path, PRIVATE_DIR_MODE)?; - check_owner_only_dir(path) -} - /// Creates the directory `path`, whose parent must exist, with exactly the mode `mode`, unless it /// exists already, which leaves it as it is. fn create_dir_of_mode(path: &Path, mode: Mode) -> io::Result<()> { @@ -101,8 +91,7 @@ pub(crate) fn check_store_dir(path: &Path) -> io::Result> { } /// Returns why the directory `path` is not private to the current user, if it is not: unless it -/// is a directory, not a link to one, that belongs to the user and grants no one else any access, -/// which [`create_owner_only_dir`] makes it. +/// is a directory, not a link to one, that belongs to the user and grants no one else any access. pub(crate) fn check_owner_only_dir(path: &Path) -> io::Result> { let Some(directory) = open_plain_dir(path)? else { return Ok(Some("is not a plain directory".to_owned())); @@ -155,28 +144,6 @@ pub(crate) fn check_snapshot_dir(path: &Path) -> io::Result> { check_owner_only_dir(path) } -/// Returns why another user than the current one, or root, may change the file that `file` holds -/// open, if one may: unless it belongs to the current user or to root, and grants neither its -/// group nor others the permission to write it. -pub(crate) fn foreign_writer_problem(file: &File) -> io::Result> { - let (uid, mode) = owner_and_mode(&file.metadata()?); - Ok(foreign_writer(uid, mode, Uid::effective())) -} - -/// Returns why another user than `euid`, or root, may change a file that the user `uid` owns with -/// the permission bits `mode`, if one may. -pub(crate) fn foreign_writer(uid: Uid, mode: Mode, euid: Uid) -> Option { - if uid != euid && !uid.is_root() { - Some(format!( - "belongs to user {uid}, neither the current user {euid} nor root" - )) - } else if mode.intersects(Mode::S_IWGRP | Mode::S_IWOTH) { - Some(format!("is writable by other users (mode {mode:03o})")) - } else { - None - } -} - /// Lets the regular file that `file` holds open for writing take no space for the blocks that /// nobody writes: nothing to do on Linux, whose filesystems allocate no block that a file's /// extension leaves unwritten. @@ -187,6 +154,7 @@ pub(crate) fn make_sparse(_file: &File) -> io::Result<()> { /// Renames `from` to `to` unless `to` exists, which it leaves as it is, failing with /// [`io::ErrorKind::AlreadyExists`]: `renameat2` with `RENAME_NOREPLACE`, which a filesystem that /// cannot rename without replacing refuses. +#[cfg(feature = "cli")] pub(crate) fn rename_no_replace(from: &Path, to: &Path) -> io::Result<()> { sys::rename_no_replace(from, to) } diff --git a/nvx-rs/src/host/platform/linux/tests.rs b/nvx-rs/src/host/platform/linux/tests.rs index 7a3029400..ed5a84da3 100644 --- a/nvx-rs/src/host/platform/linux/tests.rs +++ b/nvx-rs/src/host/platform/linux/tests.rs @@ -7,7 +7,7 @@ use std::thread; use std::time::{Duration, Instant}; use super::endpoint::{UnixTransport, endpoint_server_pid_until}; -pub(crate) use super::private::{owner_only_problem, store_dir_problem}; +pub(crate) use super::private::store_dir_problem; use super::process::wait_step; pub(crate) use super::process::{check_identity, start_time_from_stat}; use super::*; diff --git a/nvx-rs/src/host/platform/other.rs b/nvx-rs/src/host/platform/other.rs index c08450731..928518f46 100644 --- a/nvx-rs/src/host/platform/other.rs +++ b/nvx-rs/src/host/platform/other.rs @@ -183,11 +183,6 @@ pub(crate) fn check_store_dir(_path: &Path) -> io::Result> { Err(unsupported()) } -/// Fails: this host cannot check who may access a directory. -pub(crate) fn create_owner_only_dir(_path: &Path) -> io::Result> { - Err(unsupported()) -} - /// Fails: this host cannot check who may access a directory. pub(crate) fn check_owner_only_dir(_path: &Path) -> io::Result> { Err(unsupported()) @@ -203,17 +198,13 @@ pub(crate) fn check_snapshot_dir(path: &Path) -> io::Result> { check_owner_only_dir(path) } -/// Fails: this host cannot check who may change a file. -pub(crate) fn foreign_writer_problem(_file: &File) -> io::Result> { - Err(unsupported()) -} - /// Fails: this host makes no template's files. pub(crate) fn make_sparse(_file: &File) -> io::Result<()> { Err(unsupported()) } /// Fails: this host makes no template's files. +#[cfg(feature = "cli")] pub(crate) fn rename_no_replace(_from: &Path, _to: &Path) -> io::Result<()> { Err(unsupported()) } diff --git a/nvx-rs/src/host/platform/tests.rs b/nvx-rs/src/host/platform/tests.rs index 8cff9329f..0d8aa0abc 100644 --- a/nvx-rs/src/host/platform/tests.rs +++ b/nvx-rs/src/host/platform/tests.rs @@ -1,7 +1,5 @@ #[cfg(target_os = "linux")] -pub(crate) use super::linux::tests::{ - check_identity, owner_only_problem, start_time_from_stat, store_dir_problem, -}; +pub(crate) use super::linux::tests::{check_identity, start_time_from_stat, store_dir_problem}; #[cfg(all(windows, feature = "cli"))] pub(crate) use super::windows::tests::disconnect_control; #[cfg(windows)] diff --git a/nvx-rs/src/host/platform/windows/mod.rs b/nvx-rs/src/host/platform/windows/mod.rs index 52c5ccaed..bf18bc295 100644 --- a/nvx-rs/src/host/platform/windows/mod.rs +++ b/nvx-rs/src/host/platform/windows/mod.rs @@ -37,10 +37,11 @@ pub(crate) use self::identity::{ pub(crate) use self::interrupt::{interrupted, watch_interrupts}; pub(crate) use self::log_claim::{claim_log, create_log_job, log_claim_held, serves_launch}; pub use self::private::create_private_dir; +#[cfg(feature = "cli")] +pub(crate) use self::private::rename_no_replace; pub(crate) use self::private::{ - check_owner_only_dir, check_snapshot_dir, check_store_dir, check_vm_dir, create_owner_only_dir, - create_store_dir, create_vm_dir, foreign_writer_problem, make_sparse, private_file_problem, - rename_no_replace, + check_snapshot_dir, check_store_dir, check_vm_dir, create_store_dir, create_vm_dir, + make_sparse, private_file_problem, }; pub use self::process::process_start_time; pub(crate) use self::process::{ diff --git a/nvx-rs/src/host/platform/windows/private.rs b/nvx-rs/src/host/platform/windows/private.rs index 04d93414a..8fea4be75 100644 --- a/nvx-rs/src/host/platform/windows/private.rs +++ b/nvx-rs/src/host/platform/windows/private.rs @@ -9,7 +9,7 @@ use std::os::windows::io::AsRawHandle; use std::path::Path; use std::{mem, ptr}; -use windows_sys::Win32::Foundation::{ERROR_ALREADY_EXISTS, GENERIC_ALL, GENERIC_WRITE, HANDLE}; +use windows_sys::Win32::Foundation::{ERROR_ALREADY_EXISTS, GENERIC_ALL, HANDLE}; use windows_sys::Win32::Security::{ ACCESS_ALLOWED_ACE, ACE_HEADER, ACL, ACL_REVISION, ACL_SIZE_INFORMATION, AclSizeInformation, AddAccessAllowedAceEx, CONTAINER_INHERIT_ACE, DACL_SECURITY_INFORMATION, EqualSid, GetAce, @@ -21,11 +21,11 @@ use windows_sys::Win32::Security::{ SetSecurityDescriptorDacl, SetSecurityDescriptorOwner, TOKEN_QUERY, TOKEN_USER, TokenUser, WinBuiltinAdministratorsSid, WinCreatorOwnerRightsSid, WinCreatorOwnerSid, WinLocalSystemSid, }; +#[cfg(feature = "cli")] +use windows_sys::Win32::Storage::FileSystem::MoveFileExW; use windows_sys::Win32::Storage::FileSystem::{ - CreateDirectoryW, DELETE, FILE_ALL_ACCESS, FILE_APPEND_DATA, FILE_ATTRIBUTE_DIRECTORY, - FILE_ATTRIBUTE_REPARSE_POINT, FILE_DELETE_CHILD, FILE_FLAG_BACKUP_SEMANTICS, - FILE_FLAG_OPEN_REPARSE_POINT, FILE_WRITE_ATTRIBUTES, FILE_WRITE_DATA, FILE_WRITE_EA, - MoveFileExW, READ_CONTROL, WRITE_DAC, WRITE_OWNER, + CreateDirectoryW, FILE_ALL_ACCESS, FILE_ATTRIBUTE_DIRECTORY, FILE_ATTRIBUTE_REPARSE_POINT, + FILE_FLAG_BACKUP_SEMANTICS, FILE_FLAG_OPEN_REPARSE_POINT, READ_CONTROL, }; use windows_sys::Win32::System::IO::DeviceIoControl; use windows_sys::Win32::System::Threading::{GetCurrentProcess, OpenProcessToken}; @@ -198,21 +198,6 @@ pub(crate) fn check_store_dir(path: &Path) -> io::Result> { store_dir_problem(path, &CurrentUser::get()?) } -/// Creates the directory `path` private to the current user, or checks that an existing one is, -/// as [`create_store_dir`] does for the directory of a sandbox store. -pub(crate) fn create_owner_only_dir(path: &Path) -> io::Result> { - create_store_dir(path) -} - -/// Returns why the directory `path` is not private to the current user, if it is not, as -/// [`check_store_dir`] does for the directory of a sandbox store: unless it is a directory, not a -/// link or any other reparse point, that belongs to the user, or to SYSTEM or the Administrators -/// group, whose access control list allows no one else, and passes entries on to the files -/// created in it. -pub(crate) fn check_owner_only_dir(path: &Path) -> io::Result> { - check_store_dir(path) -} - /// Returns why the directory `path` cannot hold a sandbox store of `user`, if it cannot. fn store_dir_problem(path: &Path, user: &CurrentUser) -> io::Result> { plain_dir_problem(path, user, Required::PrivateDirectory) @@ -241,9 +226,9 @@ fn plain_dir_problem( /// current user, if it is not: unless it is a directory, not a link or any other reparse point, /// that belongs to the user, or to SYSTEM or the Administrators group, whose access control list /// is present, holds only entries that allow or deny access, and allows no one but those and the -/// owner's aliases, as [`check_owner_only_dir`] requires. +/// owner's aliases, as [`check_store_dir`] requires. /// -/// Unlike [`check_owner_only_dir`], it does not require the directory to pass entries on to the +/// Unlike [`check_store_dir`], it does not require the directory to pass entries on to the /// files created in it: OpenVMM gives the directory, and each file that it publishes in it, a /// protected list of their own, which allows SYSTEM and the owner, and the directory's own list /// already stops anyone else from creating a file in it. @@ -374,28 +359,6 @@ pub(crate) fn private_file_problem(file: &fs::File) -> io::Result privacy_problem(file, &CurrentUser::get()?, Required::Private) } -/// Returns why another user than the current one, SYSTEM, or the Administrators group may change -/// the file that `file` holds open, if one may: unless it belongs to the user, or to SYSTEM or the -/// Administrators group, and its access control list allows no one else to write or extend it, to -/// change its attributes, its list, or its owner, or to delete it. Others may read it. -pub(crate) fn foreign_writer_problem(file: &fs::File) -> io::Result> { - privacy_problem(file, &CurrentUser::get()?, Required::Unwritable) -} - -/// The rights of an access control entry that change a file or directory: write or extend it, or -/// its extended attributes, delete it or its children, change its attributes, its list, or its -/// owner, and the generic rights that include those. -const CHANGE_RIGHTS: u32 = FILE_WRITE_DATA - | FILE_APPEND_DATA - | FILE_WRITE_EA - | FILE_DELETE_CHILD - | FILE_WRITE_ATTRIBUTES - | DELETE - | WRITE_DAC - | WRITE_OWNER - | GENERIC_WRITE - | GENERIC_ALL; - /// What [`privacy_problem`] requires of the access control list of a file or directory, whose /// owner the user must trust in any case. #[derive(Clone, Copy, PartialEq, Eq)] @@ -405,8 +368,6 @@ pub(super) enum Required { /// That it allow no one else any access, and pass entries on to the files created in it, as /// a directory of a store must. PrivateDirectory, - /// That it allow no one else any of the [`CHANGE_RIGHTS`]. - Unwritable, } /// Returns why the file, directory, or other object that `object` holds open is not private to @@ -478,11 +439,6 @@ pub(super) fn privacy_problem( ACCESS_DENIED_ACE_TYPE => {} ACCESS_ALLOWED_ACE_TYPE => { let entry = ace.cast::(); - // SAFETY: an entry that allows access starts as an ACCESS_ALLOWED_ACE. - let mask = unsafe { (*entry).Mask }; - if required == Required::Unwritable && mask & CHANGE_RIGHTS == 0 { - continue; - } // SAFETY: an entry that allows access holds its identifier from `SidStart` on. let sid: PSID = unsafe { (&raw mut (*entry).SidStart).cast() }; // The creator owner and the owner rights stand for an owner: the owner, or the @@ -493,13 +449,7 @@ pub(super) fn privacy_problem( || IsWellKnownSid(sid, WinCreatorOwnerRightsSid) != 0 }; if !owner_alias && !user.trusts(sid) { - return Ok(Some( - match required { - Required::Unwritable => "allows other users to change it", - Required::Private | Required::PrivateDirectory => "allows other users", - } - .to_owned(), - )); + return Ok(Some("allows other users".to_owned())); } } other => { @@ -547,6 +497,7 @@ pub(crate) fn make_sparse(file: &fs::File) -> io::Result<()> { /// Renames `from` to `to` unless `to` exists, which it leaves as it is, failing with /// [`io::ErrorKind::AlreadyExists`]: `MoveFileExW` without `MOVEFILE_REPLACE_EXISTING`. +#[cfg(feature = "cli")] pub(crate) fn rename_no_replace(from: &Path, to: &Path) -> io::Result<()> { let (from, to) = (wide_path(from)?, wide_path(to)?); // SAFETY: both names are NUL-terminated. diff --git a/nvx-rs/src/host/platform/windows/private/tests.rs b/nvx-rs/src/host/platform/windows/private/tests.rs index 25270b46b..ef833c33e 100644 --- a/nvx-rs/src/host/platform/windows/private/tests.rs +++ b/nvx-rs/src/host/platform/windows/private/tests.rs @@ -41,6 +41,7 @@ fn private_directories_support_long_paths() { } } +#[cfg(feature = "cli")] #[test] fn renames_support_long_paths_without_replacing_existing_files() { let (_root, parent) = long_directory(); @@ -60,6 +61,7 @@ fn renames_support_long_paths_without_replacing_existing_files() { assert_eq!(fs::read(&destination).unwrap(), b"original"); } +#[cfg(feature = "cli")] #[test] fn invalid_paths_do_not_create_or_rename_their_prefix() { let root = tempfile::tempdir().unwrap(); diff --git a/nvx-rs/src/host/restore/tests/live_init.rs b/nvx-rs/src/host/restore/tests/live_init.rs index 4485da931..0bf9c4361 100644 --- a/nvx-rs/src/host/restore/tests/live_init.rs +++ b/nvx-rs/src/host/restore/tests/live_init.rs @@ -15,8 +15,8 @@ use crate::activation::{ use crate::host::instance::generate_capability; use crate::host::nvxs::HostConnection; use crate::host::session; +use crate::host::snapshot::SnapshotManifest; use crate::host::supervisor::{Launched, VmmCommand, spawn}; -use crate::host::template::TemplateManifest; use crate::protocol::{ APP_ERROR, APP_EXEC, APP_EXIT, APP_MAPS, APP_READY, APP_STDERR, APP_STDOUT, APP_STOP, APP_STOPPED, HostMap, Workload, WorkloadEnvironment, decode_app, encode_app, @@ -196,7 +196,7 @@ fn run() -> Result<(), AnyError> { for path in [&vmm, &snapshot, &distro, &original_console, &root, &result] { require(path.is_absolute(), "all fixture paths must be absolute")?; } - let manifest = TemplateManifest::read(&snapshot)?; + let manifest = SnapshotManifest::read(&snapshot)?; require( manifest.hypervisor() == backend() && manifest.memory_len() == 128 << 20, "wrong backend/memory snapshot", diff --git a/nvx-rs/src/host/sandbox/geometry.rs b/nvx-rs/src/host/sandbox/geometry.rs index fc88d3b8f..e1ece2a21 100644 --- a/nvx-rs/src/host/sandbox/geometry.rs +++ b/nvx-rs/src/host/sandbox/geometry.rs @@ -1,6 +1,5 @@ //! The block geometry of a sandbox block device, which the checks of its files read as OpenVMM //! presents it, and which a template records for the devices of its capture, from //! [`config`](crate::host::config), where a capture's configuration names it. -//! [`template`](crate::host::template) re-exports it. pub use crate::host::config::BlockGeometry; diff --git a/nvx-rs/src/host/snapshot.rs b/nvx-rs/src/host/snapshot.rs index 445c674f2..5c8328efa 100644 --- a/nvx-rs/src/host/snapshot.rs +++ b/nvx-rs/src/host/snapshot.rs @@ -58,8 +58,8 @@ use crate::host::{filesystem, platform, sandbox, seal}; use crate::net::Ipv4Cidr; use crate::{Error, Hypervisor, Result}; +pub(super) use self::capture::check_snapshot; pub use self::capture::{CapturedSnapshot, finish_capture}; -pub(super) use self::capture::{check_exit, check_snapshot}; pub use self::eligibility::{Downtime, DowntimeSource, Eligibility, HostTime, MAX_DOWNTIME_NS}; pub use self::key::{CaptureClass, ClassIdentity, Invariants, KeyIdentity, SnapshotKey}; pub use self::manifest::{ diff --git a/nvx-rs/src/host/snapshot/key.rs b/nvx-rs/src/host/snapshot/key.rs index b52ead232..c8fb3b436 100644 --- a/nvx-rs/src/host/snapshot/key.rs +++ b/nvx-rs/src/host/snapshot/key.rs @@ -506,8 +506,6 @@ pub struct SnapshotKey { time: TimeContract, } -pub(in crate::host) use SnapshotKey as TemplateKey; - /// The checked inputs shared by a legacy capture class and a caller-owned capture context. struct CaptureInputs<'a> { /// The runtime's artifact and contract identities. @@ -540,8 +538,8 @@ impl SnapshotKey { /// Version 1 omits the guest build, appends the complete capture-time contract, and holds /// the snapshot's NIC, if it has one, with the identity of the complete /// deny-all policy under which the capture attached it, but no network policy of its clones. - /// It has not been released, so a template store that an earlier build created is not - /// supported: delete it and create a new one. There is no migration. + /// It has not been released, so no key that an earlier build encoded is supported. There is + /// no migration. pub const VERSION: u16 = 1; /// Most bytes of a key's canonical encoding: what the largest key that a manifest of at most diff --git a/nvx-rs/src/host/snapshot/key/tests.rs b/nvx-rs/src/host/snapshot/key/tests.rs index 74fb1d7dd..76b9cb199 100644 --- a/nvx-rs/src/host/snapshot/key/tests.rs +++ b/nvx-rs/src/host/snapshot/key/tests.rs @@ -358,10 +358,6 @@ fn the_guest_build_is_not_part_of_a_snapshot_key() { assert_eq!(changed, made); assert_eq!(changed.to_bytes(), made.to_bytes()); assert_eq!(changed.identity(), made.identity()); - assert_eq!( - crate::host::template::TemplateKey::from_bytes(&changed.to_bytes()).unwrap(), - made - ); } #[test] @@ -740,12 +736,6 @@ fn keys_with_the_removed_guest_build_field_are_refused() { SnapshotKey::from_bytes(&old).unwrap_err().code(), ErrorCode::Io ); - assert_eq!( - crate::host::template::TemplateKey::from_bytes(&old) - .unwrap_err() - .code(), - ErrorCode::Io - ); } /// Returns the message of the refusal of the key bytes `bytes`, which must have the code `code`. diff --git a/nvx-rs/src/host/snapshot/manifest.rs b/nvx-rs/src/host/snapshot/manifest.rs index 20af1c9a0..f8bcf392a 100644 --- a/nvx-rs/src/host/snapshot/manifest.rs +++ b/nvx-rs/src/host/snapshot/manifest.rs @@ -650,8 +650,6 @@ pub struct SnapshotManifest { pub(super) attachments: Vec, } -pub(in crate::host) use SnapshotManifest as TemplateManifest; - impl SnapshotManifest { /// Reads the manifest of the snapshot in the directory `snapshot`: its [`MANIFEST_NAME`]. /// diff --git a/nvx-rs/src/host/snapshot/tests.rs b/nvx-rs/src/host/snapshot/tests.rs index 8c67db400..a07bc18e8 100644 --- a/nvx-rs/src/host/snapshot/tests.rs +++ b/nvx-rs/src/host/snapshot/tests.rs @@ -1015,9 +1015,9 @@ impl SnapshotFixture { let root = tempfile::tempdir().unwrap(); let dir = root.path().join("snapshot"); - assert_eq!(platform::create_owner_only_dir(&dir).unwrap(), None); + assert_eq!(platform::create_store_dir(&dir).unwrap(), None); let openvmm = dir.join(OPENVMM_DIR); - assert_eq!(platform::create_owner_only_dir(&openvmm).unwrap(), None); + assert_eq!(platform::create_store_dir(&openvmm).unwrap(), None); let spec = ManifestSpec::template(); let manifest = SnapshotManifest::parse(&spec.encode()).unwrap(); let key = SnapshotKey::new(&class(), &manifest).unwrap(); diff --git a/nvx-rs/src/host/store/files.rs b/nvx-rs/src/host/store/files.rs index f69206875..9807e59ac 100644 --- a/nvx-rs/src/host/store/files.rs +++ b/nvx-rs/src/host/store/files.rs @@ -5,9 +5,6 @@ //! and its log only as [`StoreFile`]s, each of which holds a file's path with what messages call //! it, so that no read or lock of one of them skips the checks that refuse a file that others may //! open. -//! -//! The template store, [`template::Store`](crate::host::template::Store), keeps its own records -//! with the same lock files, atomic replacements, removals, and syncs. use std::fs::{self, File, OpenOptions}; use std::io::{self, Read, Write}; diff --git a/nvx-rs/src/host/store/lease.rs b/nvx-rs/src/host/store/lease.rs index 7059e74f7..ba77f24ee 100644 --- a/nvx-rs/src/host/store/lease.rs +++ b/nvx-rs/src/host/store/lease.rs @@ -2,9 +2,8 @@ //! [clone binding](CLONE_BINDING_NAME) records it, and the reading of the records of the template //! store, in whose format the binding is. //! -//! The sandbox's store reads the binding before it begins a launch. The template store, -//! [`template::Store`](crate::host::template::Store), grants and ends leases, and reads its own -//! records, with these, and [`template`](crate::host::template) re-exports what is public here. +//! The sandbox's store reads the binding before it begins a launch, and as it removes the control +//! socket of a clone's VMM. use std::ffi::OsStr; use std::fmt; @@ -19,23 +18,23 @@ use serde::{Deserialize, Serialize}; use super::regular::{open_regular, open_regular_in, read_at_most, was_removed}; use super::{CLONE_BINDING_NAME, LaunchId, SandboxStore}; -use crate::activation::{Identity, StoreGeneration, TemplateReference}; +use crate::activation::{Identity, StoreGeneration}; use crate::error::{Error, ErrorCode, ErrorKind, Result}; use crate::host::platform::{self, parse_hex_id, parse_hex32}; /// Version of the store's records: its identity record and its index. -pub const STORE_FORMAT: u32 = 1; +pub(crate) const STORE_FORMAT: u32 = 1; /// Name of the directory of the generations' payloads. -pub const GENERATIONS_NAME: &str = "generations"; +pub(crate) const GENERATIONS_NAME: &str = "generations"; /// Name of the store's identity record. -pub const STORE_RECORD_NAME: &str = "store.json"; +pub(crate) const STORE_RECORD_NAME: &str = "store.json"; /// Longest identity record, in bytes. pub(crate) const MAX_STORE_RECORD_LEN: u64 = 4 * 1024; -/// Bytes of a [`StoreId`] and of a [`StorageGeneration`](crate::host::template::StorageGeneration). +/// Bytes of a [`StoreId`] and of a [`LeaseId`]. pub(crate) const ID_LEN: usize = 16; /// Longest record of a lease, of a clone binding, or of a tombstone, in bytes: more than a record @@ -55,19 +54,7 @@ pub(crate) fn hex(bytes: &[u8]) -> String { /// Its text form, which `Display` writes and `FromStr` parses, is 32 lowercase hexadecimal /// digits; parsing also accepts uppercase ones. #[derive(Clone, Copy, PartialEq, Eq, Hash)] -pub struct StoreId([u8; ID_LEN]); - -impl StoreId { - /// Returns the identity whose bytes are `bytes`. - pub const fn from_bytes(bytes: [u8; ID_LEN]) -> Self { - Self(bytes) - } - - /// Returns the bytes of the identity. - pub const fn as_bytes(&self) -> &[u8; ID_LEN] { - &self.0 - } -} +pub(crate) struct StoreId([u8; ID_LEN]); impl fmt::Display for StoreId { fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { @@ -100,19 +87,7 @@ impl FromStr for StoreId { /// Its text form, which `Display` writes and `FromStr` parses, is 32 lowercase hexadecimal /// digits; parsing also accepts uppercase ones. #[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] -pub struct LeaseId([u8; ID_LEN]); - -impl LeaseId { - /// Returns the identity whose bytes are `bytes`. - pub const fn from_bytes(bytes: [u8; ID_LEN]) -> Self { - Self(bytes) - } - - /// Returns the bytes of the identity. - pub const fn as_bytes(&self) -> &[u8; ID_LEN] { - &self.0 - } -} +pub(crate) struct LeaseId([u8; ID_LEN]); impl fmt::Display for LeaseId { fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { @@ -145,16 +120,13 @@ impl FromStr for LeaseId { /// /// The store keeps the record of each lease that it granted, and the sandbox keeps the same /// record as its clone binding, [`CLONE_BINDING_NAME`], beside its launch journal. A lease names -/// its sandbox by the path of its directory with its links resolved, so that no removal or change -/// of a link, or of a directory that a `..` component passes through, hides the sandbox from a -/// [retirement](crate::host::template::Store::retire). A sandbox that holds a lease must not move, -/// and goes only through its store's [removal](super::LockedStore::remove), which makes the -/// binding's absence durable before the sandbox goes. The lease allows one launch of the clone, as -/// the identity that it reserves, [`launch`](Self::launch), under the lock that acquired it, and no -/// other launch of the sandbox until a release ends it, as +/// its sandbox by the path of its directory with its links resolved. A sandbox that holds a lease +/// must not move, and goes only through its store's [removal](super::LockedStore::remove), which +/// makes the binding's absence durable before the sandbox goes. The lease reserves one launch of +/// the clone, [`launch`](Self::launch), and allows no launch of the sandbox, as /// [`LockedStore::begin_launch`](super::LockedStore::begin_launch) requires. #[derive(Clone, PartialEq, Eq)] -pub struct Lease { +pub(crate) struct Lease { /// The lease's identity. id: LeaseId, /// The identity of the store that granted it. @@ -171,76 +143,35 @@ pub struct Lease { } impl Lease { - /// Returns the lease `id`, which the store `store` granted, of its generation `generation`, - /// whose key's identity is `key`, to the sandbox in `sandbox`, an absolute path with its links - /// resolved, which is valid Unicode, for the launch `launch` of its clone. - pub(crate) fn new( - id: LeaseId, - store: StoreId, - generation: StoreGeneration, - key: Identity, - sandbox: PathBuf, - launch: LaunchId, - ) -> Self { - Self { - id, - store, - generation, - key, - sandbox, - launch, - } - } - /// Reads the clone binding of the sandbox of `sandbox`: the lease that it holds, if it holds /// one. /// /// A read that holds no lock sees one whole binding, as the latest transition left it. Fails /// with [`ErrorKind::Io`] for a binding that cannot be read, or is malformed, and with /// [`ErrorCode::FormatUnsupported`] for one of another format. - pub fn of(sandbox: &SandboxStore) -> Result> { + pub(crate) fn of(sandbox: &SandboxStore) -> Result> { read_binding(sandbox.dir()) } /// Returns the lease's identity. - pub fn id(&self) -> LeaseId { + pub(crate) fn id(&self) -> LeaseId { self.id } /// Returns the identity of the store that granted the lease. - pub fn store(&self) -> StoreId { + pub(crate) fn store(&self) -> StoreId { self.store } /// Returns the store generation of the leased generation. - pub fn generation(&self) -> StoreGeneration { + pub(crate) fn generation(&self) -> StoreGeneration { self.generation } - /// Returns the directory of the sandbox that holds the lease, as an absolute path with its - /// links resolved, which may differ from the path through which the sandbox was leased, and - /// through which a [release](crate::host::template::Store::release) takes the sandbox's lock. - pub fn sandbox(&self) -> &Path { - &self.sandbox - } - - /// Returns the launch of the clone that the lease reserves: the identity under which the - /// caller launches the clone, with [`supervisor::launch`](crate::host::supervisor::launch), - /// once, under the lock that acquired the lease, and the only launch of the sandbox that - /// [`LockedStore::begin_launch`](super::LockedStore::begin_launch) begins while the sandbox - /// holds the lease. - pub fn launch(&self) -> &LaunchId { + /// Returns the launch of the clone that the lease reserves. + pub(crate) fn launch(&self) -> &LaunchId { &self.launch } - - /// Returns the reference that the clone's lease and durable records keep of its template: - /// the leased generation's store generation and key identity. - pub fn template(&self) -> TemplateReference { - TemplateReference { - generation: self.generation, - key: self.key, - } - } } impl fmt::Debug for Lease { @@ -264,10 +195,9 @@ pub(crate) fn clone_socket_name(launch: &LaunchId) -> String { format!("{launch}.sock") } -/// Returns whether `path` names the control socket of the VMM of the clone that holds `lease`, as -/// [`template::Store::clone_socket_path`](crate::host::template::Store::clone_socket_path) returns -/// it: `generations//.sock`, in a template store, which a lease names by its -/// identity alone. +/// Returns whether `path` names the control socket of the VMM of the clone that holds `lease`, in +/// the payload of the leased generation: `generations//.sock`, in a template +/// store, which a lease names by its identity alone. pub(crate) fn is_clone_socket(lease: &Lease, path: &Path) -> bool { let name = clone_socket_name(&lease.launch); let generation = lease.generation.to_string(); @@ -318,17 +248,10 @@ pub(crate) fn open_payload( .map_err(failed()) } -/// Reads the identity of the store in `dir` from its identity record, or returns `None` if the -/// directory holds none. -pub(crate) fn read_identity(dir: &Path) -> Result> { - let path = dir.join(STORE_RECORD_NAME); - read_identity_with(&path, || open_existing(&path)) -} - -/// Reads the identity of the store whose directory `directory` holds open, in `dir`, as -/// [`read_identity`] reads it at its path, or returns `None` if the directory holds no identity -/// record, or, on Windows, one whose deletion is pending: through the handle, so that it reads -/// the identity of the directory that the handle holds, whatever `dir` names meanwhile. +/// Reads the identity of the store whose directory `directory` holds open, in `dir`, from its +/// identity record, or returns `None` if the directory holds none, or, on Windows, one whose +/// deletion is pending: through the handle, so that it reads the identity of the directory that the +/// handle holds, whatever `dir` names meanwhile. pub(crate) fn read_identity_in( directory: &platform::Directory, dir: &Path, @@ -375,19 +298,6 @@ pub(crate) fn read_binding(dir: &Path) -> Result> { read_lease(&dir.join(CLONE_BINDING_NAME), BINDING_DESCRIPTION) } -/// Reads the clone binding of the sandbox whose directory `directory` holds open, in `dir`, as -/// [`read_binding`] reads it at its path, or returns `None` if the directory holds none, or, on -/// Windows, one whose deletion is pending, which is removed: through the handle, so that it reads -/// the binding of the directory that the handle holds, whatever `dir` names meanwhile. -pub(crate) fn read_binding_in( - directory: &platform::Directory, - dir: &Path, -) -> Result> { - read_lease_with(&dir.join(CLONE_BINDING_NAME), BINDING_DESCRIPTION, || { - open_regular_in(directory, CLONE_BINDING_NAME) - }) -} - /// Reads the record of a lease at `path`, its `description`, or returns `None` if it does not /// exist. pub(crate) fn read_lease(path: &Path, description: &str) -> Result> { @@ -435,19 +345,6 @@ pub(crate) struct LeaseRecord { } impl LeaseRecord { - /// Returns the record of `lease`. - pub(crate) fn new(lease: &Lease) -> Self { - Self { - format: STORE_FORMAT, - store: lease.store.to_string(), - lease: lease.id.to_string(), - generation: lease.generation.get().get(), - key: lease.key.to_string(), - sandbox: lease.sandbox.to_string_lossy().into_owned(), - launch: lease.launch.to_string(), - } - } - /// Checks the record, and returns its lease, or says why it is malformed. fn into_lease(self) -> std::result::Result { let field = @@ -477,7 +374,7 @@ impl LeaseRecord { } /// Parses the text form of the identity of a template's key, 64 hexadecimal digits, as -/// [`KeyIdentity`](crate::host::template::KeyIdentity) parses it, failing with +/// [`KeyIdentity`](crate::host::snapshot::KeyIdentity) parses it, failing with /// [`ErrorKind::Unsupported`] for anything else. fn parse_key(text: &str) -> Result { parse_hex32(text).map(Identity::from_bytes).ok_or_else(|| { @@ -500,21 +397,9 @@ pub(crate) fn store_error(context: String) -> impl FnOnce(io::Error) -> Error { } } -/// Reads the private regular file at `path`, the store's `description`, of at most `limit` bytes, -/// or returns `None` if it does not exist, or was removed as this opened it, as [`was_removed`] -/// tells, which a transaction in another process may do. -/// -/// The file is opened without following a link, or on Windows any other reparse point, at its -/// own name, and refused unless it is a regular file, before anything is read from it. A file -/// that is not private to the current user is refused, since others could have written it: on -/// Windows, a file's own access control list decides who may open it, whatever the store's -/// directory allows. -pub(crate) fn read_private(path: &Path, description: &str, limit: u64) -> Result>> { - read_private_with(path, description, limit, || open_existing(path)) -} - /// Opens the regular file at `path`, as [`open_regular`] does, or returns `None` if it does not -/// exist, or was removed as this opened it, as [`was_removed`] tells. +/// exist, or was removed as this opened it, as [`was_removed`] tells, which a transaction in +/// another process may do. fn open_existing(path: &Path) -> io::Result> { match open_regular(path) { Ok(file) => Ok(Some(file)), @@ -526,7 +411,13 @@ fn open_existing(path: &Path) -> io::Result> { } /// Reads the private regular file that `open` opens, the store's `description` at `path`, of at -/// most `limit` bytes, as [`read_private`] reads it, or returns `None` if `open` finds none. +/// most `limit` bytes, or returns `None` if `open` finds none. +/// +/// `open` opens the file without following a link, or on Windows any other reparse point, at its +/// own name, and refuses it unless it is a regular file, before anything is read from it. A file +/// that is not private to the current user is refused, since others could have written it: on +/// Windows, a file's own access control list decides who may open it, whatever the store's +/// directory allows. fn read_private_with( path: &Path, description: &str, @@ -548,18 +439,9 @@ fn read_private_with( .map_err(failed()) } -/// Reads the store's record at `path`, its `description`, of at most `limit` bytes, in format -/// [`STORE_FORMAT`], or returns `None` if it does not exist. -pub(crate) fn read_record( - path: &Path, - description: &str, - limit: u64, -) -> Result> { - read_record_with(path, description, limit, || open_existing(path)) -} - -/// Reads the store's record that `open` opens, its `description` at `path`, as [`read_record`] -/// reads it, or returns `None` if `open` finds none. +/// Reads the store's record that `open` opens, its `description` at `path`, of at most `limit` +/// bytes, in format [`STORE_FORMAT`], as [`read_private_with`] reads it, or returns `None` if +/// `open` finds none. fn read_record_with( path: &Path, description: &str, diff --git a/nvx-rs/src/host/store/lifecycle.rs b/nvx-rs/src/host/store/lifecycle.rs index 0dedfb0c8..9bd928928 100644 --- a/nvx-rs/src/host/store/lifecycle.rs +++ b/nvx-rs/src/host/store/lifecycle.rs @@ -268,13 +268,13 @@ impl LockedStore { /// [`supervisor::reattach`](crate::host::supervisor::reattach) does. It also refuses, with /// [`RemovalError::Io`], and removes nothing, once the directory that its path names holds /// another lock file than this lock's, as after a link in the path changed. It makes both - /// checks, and removes what a clone kept of its launch, as a release does, and the sandbox's + /// checks, and removes what a clone kept of its launch, and the sandbox's /// [clone binding](CLONE_BINDING_NAME), if it holds one, through one handle of the directory /// that it opens, so that no change of a link in the path meanwhile redirects their removal /// to another sandbox, whose VM may run. It makes their absence durable before anything else /// goes, so that no crash after the removal of the directory, or of one of its ancestors, - /// brings back a binding whose lease a retirement may have ended. It then removes the store's - /// other files but its lock, the caller's files, its configuration record last, so that an + /// brings back a binding. It then removes the store's other files but its lock, the caller's + /// files, its configuration record last, so that an /// interrupted removal leaves a provisioned sandbox, which can be removed again, and the /// temporary files that the store owns. It refuses the entries that are neither the store's /// nor the caller's, which it names in the order in which the directory lists them, and keeps @@ -358,18 +358,17 @@ impl LockedStore { } } - /// Removes what the clone of the sandbox kept of its launch, as a release does, and then the - /// sandbox's clone binding, if it holds one, and makes their absence durable with `sync`, + /// Removes what the clone of the sandbox kept of its launch, and then the sandbox's clone + /// binding, if it holds one, and makes their absence durable with `sync`, /// before anything else of the sandbox goes, through one handle of the directory that the path /// names. Through that handle, it checks first that the directory holds this lock's file, and /// refuses otherwise, then that the store records no VM and no launch, and runs `checked` once /// both checks pass, before it removes anything. No change of a link in the path meanwhile /// redirects these steps to another sandbox, whose VM may run, and whose files then stay. /// - /// The binding's absence, after this removal or after a release that a crash interrupted - /// before its sync, is then durable while the directory, which keeps the sandbox provisioned, - /// remains. A binding, or a file of the clone, whose deletion is pending already counts as - /// removed. + /// The binding's absence, after this removal, is then durable while the directory, which + /// keeps the sandbox provisioned, remains. A binding, or a file of the clone, whose deletion + /// is pending already counts as removed. fn unbind( &self, checked: &dyn Fn(), @@ -405,8 +404,8 @@ impl LockedStore { } } checked(); - // What a clone kept of its launch goes before its binding, as a release removes it, so - // that the binding's durable absence makes theirs durable too. + // What a clone kept of its launch goes before its binding, so that the binding's durable + // absence makes theirs durable too. for name in CLONE_FILE_NAMES { directory.remove_file(name).map_err(entry_failed(name))?; } diff --git a/nvx-rs/src/host/store/lifecycle/tests.rs b/nvx-rs/src/host/store/lifecycle/tests.rs index c381d9559..cb6a3572d 100644 --- a/nvx-rs/src/host/store/lifecycle/tests.rs +++ b/nvx-rs/src/host/store/lifecycle/tests.rs @@ -1190,33 +1190,6 @@ mod windows { assert!(!dir.exists()); } - #[test] - fn a_retried_removal_of_the_clone_files_takes_those_whose_deletion_is_pending_for_removed() { - let (_directory, dir) = sandbox(); - for name in CLONE_FILE_NAMES { - fs::write(dir.join(name), b"").unwrap(); - } - // A release that failed once it removed the clone's files, which readers hold, retries - // their removal, which finds each one's deletion pending. - let pending: Vec = CLONE_FILE_NAMES - .into_iter() - .map(|name| delete_pending(&dir.join(name))) - .collect(); - let locked = SandboxStore::lock_existing(&dir, &files(), None).unwrap(); - locked - .remove_clone_files(&locked.open_dir().unwrap()) - .unwrap(); - drop(locked); - drop(pending); - let names = entries_of(&dir); - assert!( - names - .iter() - .all(|name| !CLONE_FILE_NAMES.contains(&name.as_str())), - "{names:?}" - ); - } - #[test] fn locks_refuse_a_lock_file_that_refuses_access_for_good() { // A read-only lock file, and a directory at its path, refuse an open for writing as a diff --git a/nvx-rs/src/host/store/locked.rs b/nvx-rs/src/host/store/locked.rs index f1d9326e4..9b52de4fe 100644 --- a/nvx-rs/src/host/store/locked.rs +++ b/nvx-rs/src/host/store/locked.rs @@ -4,7 +4,7 @@ use std::fs::File; use std::io; use std::ops::Deref; -use std::path::{Path, PathBuf}; +use std::path::Path; use std::thread; use std::time::Instant; @@ -13,7 +13,6 @@ use super::files::{io_error, remove_if_present}; use super::identity::RuntimeIdentity; use super::launch::{Launch, Recovery, end_process}; use super::lease::{Lease, clone_socket_name, is_clone_socket, open_payload}; -use super::names::{CLONE_FILE_NAMES, LOCK_NAME}; use super::record::{LaunchId, LaunchJournal, LogClaim, RuntimeRecord}; use super::{CLAIM_RELEASE_WAIT, LOCK_RETRY_INTERVAL, SandboxStore, record, store_error, vmm}; use crate::error::{Error, ErrorCode, Result}; @@ -42,26 +41,6 @@ pub struct LockedStore { pub(super) files: CallerFiles, /// Whether this holder removed its lock file, after which the path may name another's. pub(super) lock_removed: bool, - /// The launch of the sandbox's clone that the lease acquired under this lock reserves, until - /// [`begin_launch`](Self::begin_launch) begins it, or a release ends the lease. - reserved: Option, -} - -/// The launch of a sandbox's clone that a lease reserves, as -/// [`template::Store::acquire`](crate::host::template::Store::acquire) acquires the lease, and -/// where the clone's VMM listens. -#[derive(Debug, Clone, PartialEq, Eq)] -pub(crate) struct Reservation { - /// The identity of the launch, [`Lease::launch`]. - pub(crate) launch: LaunchId, - /// The path of the Linux socket at which the clone's VMM listens, its control endpoint: beside - /// the snapshot that it restores, in the payload of the leased generation, since OpenVMM - /// requires a restore's control socket in the snapshot's parent, rather than the sandbox's - /// own control socket. Windows endpoints are named pipes, which [`begin_launch`] draws as it - /// does for every launch. - /// - /// [`begin_launch`]: LockedStore::begin_launch - pub(crate) socket: PathBuf, } impl Deref for LockedStore { @@ -81,82 +60,7 @@ impl LockedStore { lock, files, lock_removed: false, - reserved: None, - } - } - - /// Lets [`begin_launch`](Self::begin_launch) begin the launch that `reservation` names, once, - /// as the launch of the sandbox's clone that the lease acquired under this lock reserves, - /// whose VMM listens at the reservation's socket, or, given `None`, no launch of a clone, once - /// a release ends the lease. - pub(crate) fn reserve(&mut self, reservation: Option) { - self.reserved = reservation; - } - - /// Returns the launch of the sandbox's clone that the lease acquired under this lock reserves, - /// until [`begin_launch`](Self::begin_launch) begins it, or a release ends the lease. - pub(crate) fn reserved(&self) -> Option<&LaunchId> { - self.reserved - .as_ref() - .map(|reservation| &reservation.launch) - } - - /// Opens the sandbox's directory through its path, as one handle through which a release of - /// the lease of its clone names the directory's entries, and checks through that handle that - /// the directory holds this lock's file, as a [removal] checks it: no change of the path - /// meanwhile, of a link in it or of one of its directories, then makes the release reach the - /// entries of another sandbox, whose lock it does not hold. - /// - /// Fails with [`ErrorKind::Other`] if the directory that the path names does not hold this - /// lock's file. - /// - /// [removal]: crate::host::store#removal - /// [`ErrorKind::Other`]: crate::ErrorKind::Other - pub(crate) fn open_dir(&self) -> Result { - let dir = self.dir(); - let directory = platform::Directory::open(dir).map_err(io_error(format!( - "cannot open the directory {}", - dir.display() - )))?; - let lock = self.path(LOCK_NAME); - let examine = || io_error(format!("cannot examine {}", lock.display())); - let held = platform::opened_file_id(&self.lock).map_err(examine())?; - if directory.entry_id(LOCK_NAME).map_err(examine())? != Some(held) { - return Err(Error::other(format!( - "the directory that {} names does not hold the lock of its sandbox: the path names \ - another directory than when the lock was taken", - dir.display() - ))); - } - Ok(directory) - } - - /// Removes what the clone of the sandbox kept of its launch in the sandbox's directory, whose - /// VM has exited, through `directory`, the handle of the directory that - /// [`open_dir`](Self::open_dir) returned: the file of its scratch device, - /// [`CLONE_SCRATCH_NAME`], if an earlier version left one, its activation record, - /// [`CLONE_ACTIVATION_NAME`], the socket of its restore-ready event, [`RESTORE_READY_NAME`], - /// its progress log, [`CLONE_PROGRESS_NAME`], and the record of its failure, - /// [`CLONE_FAILURE_NAME`]. A release, as a [removal] does, removes them before the clone - /// binding, whose removal, once durable, makes theirs durable too, since all are entries of the - /// sandbox's directory. A file whose deletion is pending, as a Windows file system without - /// POSIX deletion leaves a removed file that another handle holds, is removed already, so that - /// a release that failed once it removed the file can be retried. - /// - /// [`CLONE_SCRATCH_NAME`]: crate::host::store::CLONE_SCRATCH_NAME - /// [`CLONE_ACTIVATION_NAME`]: crate::host::store::CLONE_ACTIVATION_NAME - /// [`RESTORE_READY_NAME`]: crate::host::store::RESTORE_READY_NAME - /// [`CLONE_PROGRESS_NAME`]: crate::host::store::CLONE_PROGRESS_NAME - /// [`CLONE_FAILURE_NAME`]: crate::host::store::CLONE_FAILURE_NAME - /// [removal]: crate::host::store#removal - pub(crate) fn remove_clone_files(&self, directory: &platform::Directory) -> Result<()> { - for name in CLONE_FILE_NAMES { - directory.remove_file(name).map_err(io_error(format!( - "cannot remove {}", - self.path(name).display() - )))?; } - Ok(()) } /// Returns the caller's files that the lock knows: those that a [lifecycle lock] was taken @@ -177,19 +81,13 @@ impl LockedStore { /// of a caller that keeps the console of every launch, as [`CallerFiles::keeping_console`] /// declares. Nothing is spawned yet: the caller spawns the VMM with [`Launch::take_log`]. /// - /// The control endpoint is a named pipe of a random name on Windows. On Linux, it is the - /// sandbox's control socket, [`socket_path`](SandboxStore::socket_path), but for the launch - /// that the lease of the sandbox's clone reserves: its VMM restores the leased generation's - /// snapshot, and listens beside it, at the socket that the lease reserves in the generation's - /// payload, as [`template::Store::acquire`](crate::host::template::Store::acquire) describes. + /// The control endpoint is a named pipe of a random name on Windows, and the sandbox's control + /// socket, [`socket_path`](SandboxStore::socket_path), on Linux. /// /// Fails if the sandbox's VM runs, or a launch was interrupted and is not recovered yet, and /// with [`ErrorKind::Unsupported`] for an all-zero capability or an invalid runtime identity. /// A sandbox whose clone holds a lease of a template's generation, which its [clone binding] - /// records, launches once, as the launch that the lease reserves, under the lock that acquired - /// the lease, as [`template::Store::acquire`](crate::host::template::Store::acquire) returns - /// it: any other launch, and any launch under another lock or after that one began, fails with - /// [`ErrorCode::SandboxLeased`] until a release ends the lease. + /// records, launches nothing: any launch fails with [`ErrorCode::SandboxLeased`]. /// /// [`ErrorKind::Unsupported`]: crate::ErrorKind::Unsupported /// [clone binding]: crate::host::store::CLONE_BINDING_NAME @@ -217,23 +115,17 @@ impl LockedStore { self.dir.display() ))); } - if let Some(lease) = Lease::of(self)? - && (lease.launch() != launch || self.reserved() != Some(launch)) - { - let reservation = if lease.launch() == launch { - format!("whose launch {launch} begins once, under the lock that acquired the lease") - } else { - format!("which reserves the launch {}, not {launch}", lease.launch()) - }; + if let Some(lease) = Lease::of(self)? { return Err(Error::new( ErrorCode::SandboxLeased, format!( "the sandbox in {} holds the lease {} of the generation {} of the template \ - store {}, {reservation}", + store {}, which reserves the launch {}", self.dir.display(), lease.id(), lease.generation(), - lease.store() + lease.store(), + lease.launch() ), )); } @@ -245,13 +137,7 @@ impl LockedStore { remove_if_present(&self.console_path())?; } self.remove_report_staging()?; - // The VMM of a clone listens at the socket that its lease reserves, beside the snapshot - // that it restores. - let listening = match &self.reserved { - Some(reservation) if reservation.launch == *launch => reservation.socket.as_path(), - _ => &socket, - }; - let endpoint = platform::control_endpoint(listening) + let endpoint = platform::control_endpoint(&socket) .map_err(|error| Error::io("cannot choose a control endpoint").with_source(error))?; let (log, claim) = self.create_log()?; let capability_file = self.capability_file(); @@ -261,8 +147,6 @@ impl LockedStore { let _ = capability_file.remove_if_present(); return Err(error); } - // A lease reserves one launch, which has begun. - self.reserved = None; Ok(Launch { locked: self, journal, @@ -452,7 +336,7 @@ impl LockedStore { /// socket's generation. The log, console log, and outcome report outlive the run. pub fn clear_runtime(&mut self) -> Result<()> { // Only a Linux endpoint is a file, which may be a clone's beside its snapshot. A record that - // cannot be read names none, and a release removes a clone's socket all the same. + // cannot be read names none. let record = if cfg!(unix) { self.runtime().ok().flatten() } else { @@ -476,24 +360,19 @@ impl LockedStore { /// Removes the control socket at `endpoint`, the control endpoint that a launch of the /// sandbox recorded, if it is the socket of the sandbox's clone beside the snapshot that its - /// VMM restores, which the lease that the sandbox's clone binding records reserved, as - /// [`template::Store::clone_socket_path`](crate::host::template::Store::clone_socket_path) - /// names it. Any other endpoint stays: the sandbox's own control socket, which the caller - /// removes, and a named pipe. Neither reads the binding. + /// VMM restores, which the lease that the sandbox's clone binding records reserved, in the + /// payload of the leased generation of its template store. Any other endpoint stays: the + /// sandbox's own control socket, which the caller removes, and a named pipe. Neither reads the + /// binding. /// /// The socket goes through a handle of the generation's payload, which a handle of the - /// template store's directory opens, as a release opens it: following no link at the store's - /// directory, at its directory of the generations, or at the payload, and once it checked - /// through that handle that the directory holds the lease's store, so that neither a link that - /// takes the place of one of them nor a directory that takes the place of the store ever leads - /// the removal to the files of another directory. Such a link fails the removal; a payload - /// that is gone holds no socket, and neither does a directory that holds another store, or - /// none, in the place of the lease's, whose files stay. - /// - /// The removal is not made durable here: - /// [`template::Store::release`](crate::host::template::Store::release) makes it durable before - /// the lease ends, and a retirement removes whatever a crash brings back with the generation's - /// payload. + /// template store's directory opens: following no link at the store's directory, at its + /// directory of the generations, or at the payload, and once it checked through that handle + /// that the directory holds the lease's store, so that neither a link that takes the place of + /// one of them nor a directory that takes the place of the store ever leads the removal to the + /// files of another directory. Such a link fails the removal; a payload that is gone holds no + /// socket, and neither does a directory that holds another store, or none, in the place of the + /// lease's, whose files stay. The removal is not made durable here. fn remove_clone_socket(&self, endpoint: &str) -> Result<()> { let socket = Path::new(endpoint); if !cfg!(unix) || socket == self.socket_path() { diff --git a/nvx-rs/src/host/store/mod.rs b/nvx-rs/src/host/store/mod.rs index 912005938..f05a72060 100644 --- a/nvx-rs/src/host/store/mod.rs +++ b/nvx-rs/src/host/store/mod.rs @@ -8,8 +8,7 @@ //! launch.json the journal of a launch in progress, or of one that was interrupted //! runtime.json the runtime record of its running VM //! control.capability the launch capability of its VM -//! control.sock the control endpoint of its VM, on Linux, but for a clone, whose VMM -//! listens beside the snapshot that it restores +//! control.sock the control endpoint of its VM, on Linux //! openvmm.log the VMM's output, claimed by the launch that created it //! console.log the guest console of the latest launch, or of every launch, if the //! caller captures it @@ -17,7 +16,7 @@ //! .tmpXXXXXX the outcome report while the VMM writes it, which it then publishes //! clone.json the binding of a clone to the generation of a template that it leases //! clone-scratch.raw the file of the scratch device that earlier versions created for a -//! clone whose upper is in RAM, which a release removes if one remains +//! clone whose upper is in RAM, which a removal deletes if one remains //! clone-activation.bin the activation record of a clone, which its launch writes //! clone-progress.log the progress log of a clone's launch and restore sequence //! clone-failure.json the record of a clone's failure @@ -58,9 +57,7 @@ //! # Launches //! //! A launch holds the sandbox's lock throughout, under an identity that the caller reserved -//! before it began, with [`LaunchId::generate`], a fresh one for each launch, or that a clone's -//! lease reserved, which records it durably before any VMM of the clone exists, for the one -//! launch of the clone that the lease allows: +//! before it began, with [`LaunchId::generate`], a fresh one for each launch: //! //! 1. [`LockedStore::begin_launch`] stores the capability, creates a fresh log and claims it, and //! writes the journal's marker, which names the launch, before anything is spawned. @@ -172,8 +169,7 @@ //! the VMM through the control endpoint that the journal names, holds it, confirms that the //! endpoint still names it once held, and terminates it. //! On Windows the endpoint is a pipe whose name each launch draws at random, which names the -//! VMM by itself. On Linux it is the same socket path for every launch of the sandbox, but the -//! one launch of a clone, whose VMM listens at a socket that its lease reserves, so the +//! VMM by itself. On Linux it is the same socket path for every launch of the sandbox, so the //! VMM must also hold the claimed log as its standard output. Until the VMM serves its //! endpoint, recovery is indeterminate and keeps the journal; once the VMM exits, the released //! claim proves it. @@ -230,7 +226,7 @@ //! //! [`LockedStore::remove`] removes a sandbox that no VMM can run in: it refuses while the store //! records a VM or a launch, which the caller reattaches to, or recovers, first. It removes what a -//! clone kept of its launch, as a release does, and the sandbox's +//! clone kept of its launch, and the sandbox's //! [clone binding](CLONE_BINDING_NAME) first, and makes their absence durable before anything //! else goes. It checks for those records, and removes those files, through one handle of the //! directory, once it has checked through that handle that the directory holds its lock, so that @@ -304,7 +300,6 @@ pub use self::identity::{ pub use self::launch::{Launch, Recovery}; pub use self::lifecycle::{Lingering, RemovalError}; pub use self::locked::LockedStore; -pub(crate) use self::locked::Reservation; use self::names::CLONE_FILE_NAMES; pub use self::names::{ CAPABILITY_NAME, CLONE_ACTIVATION_NAME, CLONE_BINDING_NAME, CLONE_FAILURE_NAME, diff --git a/nvx-rs/src/host/store/names.rs b/nvx-rs/src/host/store/names.rs index 571ff7f57..67e4dc103 100644 --- a/nvx-rs/src/host/store/names.rs +++ b/nvx-rs/src/host/store/names.rs @@ -17,10 +17,7 @@ pub const JOURNAL_NAME: &str = "launch.json"; pub const RUNTIME_NAME: &str = "runtime.json"; /// Name of the file that holds the launch capability. pub const CAPABILITY_NAME: &str = "control.capability"; -/// Name of the Linux control socket, at which the VMM of every launch of the sandbox listens, but -/// that of a clone, which listens beside the snapshot that it restores, at the socket that its -/// lease reserves, as [`template::Store::acquire`](crate::host::template::Store::acquire) -/// describes. +/// Name of the Linux control socket, at which the VMM of every launch of the sandbox listens. pub const SOCKET_NAME: &str = "control.sock"; /// Name of the VMM's log. pub const LOG_NAME: &str = "openvmm.log"; @@ -28,50 +25,40 @@ pub const LOG_NAME: &str = "openvmm.log"; pub const CONSOLE_NAME: &str = "console.log"; /// Name of the VMM's outcome report. pub const OUTCOME_NAME: &str = "outcome.json"; -/// Name of the binding of the sandbox's clone to the generation of a template that it leases, -/// which [`template::Store::acquire`](crate::host::template::Store::acquire) writes, durably, -/// before the clone's launch, and [`template::Store::release`](crate::host::template::Store::release) -/// removes once its VM has exited. A [removal] of the sandbox removes it first, and makes its -/// absence durable before the sandbox goes. +/// Name of the binding of the sandbox's clone to the generation of a template that it leases. A +/// [removal] of the sandbox removes it first, and makes its absence durable before the sandbox +/// goes. /// /// [removal]: crate::host::store#removal pub const CLONE_BINDING_NAME: &str = "clone.json"; /// Name of the file of the `scratch` block device that earlier versions of this crate created for /// a clone whose upper is in RAM, a sparse file of the template's scratch length. Such a clone now /// restores with its scratch absent, and nothing creates the file, but a sandbox's directory may -/// still hold one that an earlier version left, which -/// [`template::Store::release`](crate::host::template::Store::release) removes once the clone's VM -/// has exited, as a [removal] of the sandbox does. +/// still hold one that an earlier version left, which a [removal] of the sandbox removes. /// /// [removal]: crate::host::store#removal pub const CLONE_SCRATCH_NAME: &str = "clone-scratch.raw"; -/// Name of the activation record of a clone, in its canonical encoding, which -/// [`template::Store::release`](crate::host::template::Store::release) removes once the clone's VM -/// has exited, as a [removal] of the sandbox does. +/// Name of the activation record of a clone, in its canonical encoding, which a [removal] of the +/// sandbox removes. /// /// [removal]: crate::host::store#removal pub const CLONE_ACTIVATION_NAME: &str = "clone-activation.bin"; -/// Name of the Linux socket of the restore-ready event of a clone's VMM, which -/// [`template::Store::release`](crate::host::template::Store::release) removes once the clone's VM -/// has exited, as a [removal] of the sandbox does. +/// Name of the Linux socket of the restore-ready event of a clone's VMM, which a [removal] of the +/// sandbox removes. /// /// [removal]: crate::host::store#removal pub const RESTORE_READY_NAME: &str = "restore.sock"; -/// Name of the progress log of a clone's launch, which -/// [`template::Store::release`](crate::host::template::Store::release) removes, as a [removal] of -/// the sandbox does. +/// Name of the progress log of a clone's launch, which a [removal] of the sandbox removes. /// /// [removal]: crate::host::store#removal pub const CLONE_PROGRESS_NAME: &str = "clone-progress.log"; -/// Name of the record of a clone's failure, which -/// [`template::Store::release`](crate::host::template::Store::release) removes, as a [removal] of -/// the sandbox does. +/// Name of the record of a clone's failure, which a [removal] of the sandbox removes. /// /// [removal]: crate::host::store#removal pub const CLONE_FAILURE_NAME: &str = "clone-failure.json"; /// The files that the clone of a sandbox keeps of its launch, and the file of the scratch device -/// that an earlier version created for a clone in RAM, which a release removes once the clone's VM -/// has exited, as a [removal] of the sandbox does, before the clone binding. +/// that an earlier version created for a clone in RAM, which a [removal] of the sandbox removes +/// before the clone binding. /// /// [removal]: crate::host::store#removal pub(super) const CLONE_FILE_NAMES: [&str; 5] = [ diff --git a/nvx-rs/src/host/store/record.rs b/nvx-rs/src/host/store/record.rs index fe86eac4f..a33e9d806 100644 --- a/nvx-rs/src/host/store/record.rs +++ b/nvx-rs/src/host/store/record.rs @@ -45,9 +45,7 @@ pub struct LaunchId(String); impl LaunchId { /// Draws a fresh launch identity, which a caller reserves for one launch before it begins it /// with [`LockedStore::begin_launch`](super::LockedStore::begin_launch), or - /// [`supervisor::launch`](crate::host::supervisor::launch), and never uses for another. A - /// clone's lease reserves one, as - /// [`template::Store::acquire`](crate::host::template::Store::acquire) draws it. + /// [`supervisor::launch`](crate::host::supervisor::launch), and never uses for another. pub fn generate() -> Result { let mut bytes = [0u8; LAUNCH_ID_BYTES]; getrandom::fill(&mut bytes) diff --git a/nvx-rs/src/host/store/regular.rs b/nvx-rs/src/host/store/regular.rs index 94060b5a7..10b9d7cbb 100644 --- a/nvx-rs/src/host/store/regular.rs +++ b/nvx-rs/src/host/store/regular.rs @@ -1,6 +1,5 @@ -//! The files that the template modules read, and that a sandbox's store reads as its clone -//! binding: regular files that are not links, which they read within a bound, and the identity of -//! a file that a template module holds open. +//! The files that a sandbox's store reads as its clone binding: regular files that are not links, +//! which it reads within a bound. //! //! The reads themselves are [`platform`]'s: this module re-exports its `open_regular` and //! `read_at_most` for its callers until #1121 deletes them. @@ -11,7 +10,6 @@ use std::path::Path; use crate::host::platform; pub(crate) use crate::host::platform::{open_regular, read_at_most}; -use crate::host::seal::FileSeal; /// Returns whether `error`, with which an open or a removal of the file at `path` failed, means /// that the file was removed, by a transaction of another process, say: on Windows, a file system @@ -40,11 +38,3 @@ pub(crate) fn open_regular_in( Err(error) => Err(platform::refused_open(error)), } } - -/// Returns the identity of the file that `file` holds open, as its [`FileSeal`] records it: its -/// device or volume, and its file ID, the 128-bit one of `FILE_ID_INFO` on Windows, which a file -/// system such as ReFS keeps unique where the 64-bit file index is not. -pub(crate) fn identity_of(file: &File) -> io::Result<(u64, u128)> { - let seal = FileSeal::of(file)?; - Ok((seal.device(), seal.file_id())) -} diff --git a/nvx-rs/src/host/store/tests.rs b/nvx-rs/src/host/store/tests.rs index 8a3991248..546b92765 100644 --- a/nvx-rs/src/host/store/tests.rs +++ b/nvx-rs/src/host/store/tests.rs @@ -315,39 +315,6 @@ fn a_lock_file_that_is_a_link_fails_at_once() { assert_eq!(fs::read(&target).unwrap(), CAPABILITY); } -// Other hosts open no handle of a sandbox's directory, through which a clone's files go. -#[cfg(any(target_os = "linux", windows))] -#[test] -fn the_files_that_a_clone_kept_of_its_launch_go_and_nothing_else() { - let (_directory, store) = store(); - let locked = store.lock(None).unwrap(); - for path in [ - // The file of the scratch device that an earlier version left for a clone in RAM. - store.dir().join(CLONE_SCRATCH_NAME), - store.clone_activation_path(), - store.clone_progress_path(), - store.clone_failure_path(), - store.restore_ready_path(), - store.log_path(), - store.binding_path(), - ] { - fs::write(path, b"left").unwrap(); - } - let directory = locked.open_dir().unwrap(); - locked.remove_clone_files(&directory).unwrap(); - let mut left: Vec = fs::read_dir(store.dir()) - .unwrap() - .map(|entry| entry.unwrap().file_name().into_string().unwrap()) - .collect(); - left.sort(); - // The binding goes later, once the release or the removal has removed the clone's files. - let mut expected = [CLONE_BINDING_NAME, LOCK_NAME, LOG_NAME]; - expected.sort_unstable(); - assert_eq!(left, expected); - // Nothing to remove is no failure. - locked.remove_clone_files(&directory).unwrap(); -} - #[cfg(unix)] #[test] fn records_and_capabilities_that_are_not_regular_files_fail_at_once() { diff --git a/nvx-rs/src/host/supervisor/launch.rs b/nvx-rs/src/host/supervisor/launch.rs index b2829722b..9377e28f5 100644 --- a/nvx-rs/src/host/supervisor/launch.rs +++ b/nvx-rs/src/host/supervisor/launch.rs @@ -11,8 +11,7 @@ use crate::host::store::{ use crate::protocol::CAPABILITY_LEN; /// Launches a VMM in the sandbox of `locked`, through the store's launch journal, as the launch -/// `reserved`, which the caller reserved with [`LaunchId::generate`], or which the sandbox's lease -/// reserves, [`Lease::launch`](crate::host::template::Lease::launch), under the runtime identity +/// `reserved`, which the caller reserved with [`LaunchId::generate`], under the runtime identity /// `runtime`, with `capability` on its standard input. /// /// It begins the launch with [`LockedStore::begin_launch`], asks `command` for the VMM's command, @@ -26,8 +25,7 @@ use crate::protocol::CAPABILITY_LEN; /// cannot be ended then, because its VMM did not exit, keeps its journal for /// [`LockedStore::recover`], and the error says so. Fails as [`LockedStore::begin_launch`] does if /// the sandbox's VM runs, an interrupted launch has not been recovered, or the sandbox holds a -/// lease that does not let this lock begin `reserved`: one that reserves another launch, or that -/// another lock acquired, or whose launch has begun. +/// lease. pub fn launch<'s, 'h>( locked: &'s mut LockedStore, reserved: &LaunchId, diff --git a/nvx-rs/src/host/supervisor/mod.rs b/nvx-rs/src/host/supervisor/mod.rs index bbf7463f8..9e1278c34 100644 --- a/nvx-rs/src/host/supervisor/mod.rs +++ b/nvx-rs/src/host/supervisor/mod.rs @@ -11,8 +11,7 @@ //! # Launches //! //! [`launch`] runs the store's launch steps under the sandbox's lock, as a launch whose identity -//! the caller reserved beforehand, with [`LaunchId::generate`], or through a clone's lease, as -//! [Clones](#clones) describes: +//! the caller reserved beforehand, with [`LaunchId::generate`]: //! //! 1. [`LockedStore::begin_launch`] writes the journal's marker, which names the launch, and the //! caller's `command` builds the VMM's [`VmmCommand`] with the control endpoint that the @@ -125,79 +124,6 @@ //! recovery of a launch that did not record its VMM, the process that recovery verified to be that //! VMM. It never signals a process that reuses a recorded ID. //! -//! # Clones -//! -//! A clone restores a generation of a template that it leases from a -//! [template store](crate::host::template::Store), and its lease reserves its launch: -//! -//! 1. [`Store::acquire`](crate::host::template::Store::acquire) leases the generation, and writes -//! the lease's record in the store and the clone binding in the sandbox, durably, with the -//! identity of the launch that the lease reserves, -//! [`Lease::launch`](crate::host::template::Lease::launch), before any VMM of the clone -//! exists. It returns the sandbox's lock. -//! 2. Under that lock, the caller launches the clone with [`launch`], as the reserved launch, -//! which [`LockedStore::begin_launch`] requires of a sandbox that holds a lease, and which -//! begins once, under that lock alone, whatever its outcome. The supervisor prepares nothing of -//! the restore: the caller binds what the restore needs before the launch, such as its -//! restore-ready endpoint, and its `command` gives OpenVMM the restore's options, such as -//! `--restore-ready-path`. The clone's VMM restores the generation's snapshot in place, and on -//! Linux listens beside it, at the control socket that the lease reserves in the generation's -//! payload, named after the launch, which [`Starting::endpoint`] and the runtime record name, -//! as every launch's endpoint, rather than at the sandbox's own socket. -//! 3. Still under that lock, the caller checks, through its own transport, that the clone's -//! restore completed: that the VMM sent its restore-ready event, and that the clone was -//! activated, reported Ready and started, as the caller's protocol requires. Only then does it -//! commit the launch with [`Starting::commit`], which writes the runtime record. A failure ends -//! the launch with [`Starting::abandon`], through the VMM's verified identity: a launch that -//! cannot be ended keeps its journal, and the lease. -//! 4. Once the clone's VM has exited, as a stop, a termination, or a reattach verifies, -//! [`Store::release`](crate::host::template::Store::release) ends the lease, under the -//! sandbox's lock, taken through the lease's path, -//! [`Lease::sandbox`](crate::host::template::Lease::sandbox), as the acquisition took it. The -//! VMM removes its control socket as it exits, the stop or the reattach removes the one that an -//! abrupt exit leaves, and the release makes the socket's absence durable before the lease -//! ends. -//! -//! ## The lock order -//! -//! These steps, and every other transition of a template store or a sandbox, take their locks in -//! one order: the lifecycle lock of a capture class's slot, which serializes the class's captures, -//! as [`Store::refresh_template`](crate::host::template::Store::refresh_template) describes, then -//! the template store's lock, then the [lifecycle lock](super::store#lifecycle-locks) of a sandbox, -//! which guards its clone binding and its launch journal, or of a capture's transaction, the -//! sandbox store of the capture's VM, then the scratch image's claim. None waits for a lock while -//! it holds a later one: under its slot's lock, a capture, or its recovery, takes its -//! transaction's lock and then, once it has released it, the store's lock, to publish, and no -//! other; an acquisition releases the store's lock before it returns; a release, a launch, a stop, -//! a reattach, and a removal of the sandbox take no lock of the store; and a retirement takes no -//! lock of a sandbox. No lock but those of a capture lasts for a VM's lifetime: a capture holds -//! its slot's lock and its transaction's until its VM has ended, while the caller of a clone's -//! launch releases the sandbox's lock once the launch commits or ends, and the scratch image's -//! claim belongs to the VMM, which holds it until it exits. -//! -//! ## Crashes -//! -//! A crash at any step leaves records from which a later transition, under the same locks, decides -//! what becomes of the lease, which a release ends only once the sandbox records no VM and no -//! launch: -//! -//! - After the acquisition, the binding reserves a launch that never began, and that no later lock -//! begins: a release ends the lease at once, and another acquisition, which checks the -//! generation's eligibility again, reserves another launch. -//! - After the launch's journal, before or after the spawn, before its commit, [`reattach`] ends -//! the launch, as [`LockedStore::recover`] does, and the lease lasts until it has: a launch -//! that cannot be ended yet is [`Reattached::Indeterminate`], and keeps it. -//! - After the commit, the runtime record names the VM, which [`reattach`] holds again: the lease -//! lasts as long as the VM runs, until a stop or a termination verifies its exit. -//! - During a release, the binding goes first, durably, and leaves the record of the lease, which -//! a retirement removes. -//! -//! The evidence of the VM's exit decides each outcome. A record of an earlier boot, or of a process -//! ID that another process now has, names a VM that has exited: reattach clears it, and the lease -//! may be released. A VM that cannot be examined is indeterminate, and keeps its records, and so -//! the lease. A forced teardown that fails, or that does not end the VM, keeps its runtime record, -//! and the lease. -//! //! [`launch`]: fn@crate::host::supervisor::launch //! [`LaunchId::generate`]: crate::host::store::LaunchId::generate //! [`LockedStore::begin_launch`]: crate::host::store::LockedStore::begin_launch diff --git a/nvx-rs/src/host/supervisor/tests.rs b/nvx-rs/src/host/supervisor/tests.rs index 91f5f8637..7080032b8 100644 --- a/nvx-rs/src/host/supervisor/tests.rs +++ b/nvx-rs/src/host/supervisor/tests.rs @@ -12,24 +12,21 @@ use super::launch::launch_reporting; use super::reattach::reattach_with; use super::vmm::{Stoppable, stop_process, terminate_process}; use super::*; +use crate::ErrorKind; use crate::error::{Error, Result}; use crate::host::launch::MAX_TIMEOUT; use crate::host::platform; use crate::host::process::{BootId, Process, ProcessRef}; use crate::host::store::{ - ArtifactIdentity, CallerFiles, FORMAT, IdentityPart, JOURNAL_NAME, LaunchId, LockedStore, - RUNTIME_NAME, Recovery, RuntimeIdentity, SandboxStore, + ArtifactIdentity, FORMAT, IdentityPart, JOURNAL_NAME, LaunchId, RUNTIME_NAME, Recovery, + RuntimeIdentity, SandboxStore, }; -use crate::host::template::{self, Acquired, HostTime, Lease}; use crate::protocol::CAPABILITY_LEN; -use crate::{ErrorCode, ErrorKind}; /// A launch capability. const CAPABILITY: [u8; CAPABILITY_LEN] = [7; CAPABILITY_LEN]; /// The launch of the runtime records that [`record`] writes. const RECORDED_LAUNCH: &str = "5d41402abc4b2a76b9719d911017c592"; -/// The configuration record of these tests' clones. -const CONFIG: &str = "config.json"; /// Returns the identity `sha256:` and 64 copies of `digit`. fn artifact(digit: char) -> ArtifactIdentity { @@ -87,52 +84,6 @@ fn record_launch( .unwrap(); } -/// A clone, in a fresh temporary directory, that leases a generation of a template from the -/// template store that it returns, and whose launch, the one that its lease reserved, committed a -/// VM whose VMM is `process`; and the lease, with the lock of the clone's sandbox that the -/// acquisition returned. -fn committed_clone( - process: &ProcessRef, -) -> (tempfile::TempDir, template::Store, Lease, LockedStore) { - let directory = tempfile::tempdir().unwrap(); - let templates = template::Store::create(directory.path().join("templates")).unwrap(); - let generation = templates.publish_for_tests(); - let sandbox = directory.path().join("sandbox"); - let files = CallerFiles::new(CONFIG).unwrap(); - let provisioned = SandboxStore::lock_for_provision(&sandbox, &files, None).unwrap(); - fs::write(sandbox.join(CONFIG), b"{}").unwrap(); - drop(provisioned); - // A second after the capture, on the host that captured the generation. - let time = generation.time(); - let now = HostTime { - host_id: time.host_id, - boot_id: time.boot_id, - clock: time.host_clock, - monotonic_ns: time.capture_monotonic_ns + 1_000_000_000, - utc_ns: time.capture_utc_ns + 1_000_000_000, - }; - let Acquired { lease, mut locked } = templates - .acquire(generation.generation(), &sandbox, &files, &now, None) - .unwrap(); - // The reserved launch begins, and its commit replaces its journal with its runtime record. - drop( - locked - .begin_launch(lease.launch(), &runtime(), &CAPABILITY) - .unwrap(), - ); - record_launch(&locked, lease.launch().as_str(), process, &runtime()); - fs::remove_file(sandbox.join(JOURNAL_NAME)).unwrap(); - (directory, templates, lease, locked) -} - -/// Checks that `templates` refuses to release the lease of the sandbox of `locked`, whose VM may -/// still run, and that the sandbox keeps it. -fn refuses_release(templates: &template::Store, locked: &mut LockedStore, lease: &Lease) { - let error = templates.release(locked).unwrap_err(); - assert_eq!(error.code(), ErrorCode::SandboxRunning, "{error}"); - assert_eq!(Lease::of(locked).unwrap().as_ref(), Some(lease)); -} - /// A VMM whose state drives a stop's decisions without a process: it exits once the graceful /// step runs, if it heeds that step, or once it is killed, if killing ends it. #[derive(Default)] @@ -618,7 +569,9 @@ fn a_launch_whose_caller_cannot_record_its_failure_keeps_its_journal_for_recover #[test] fn a_reattach_whose_caller_cannot_record_an_exit_keeps_the_record_until_one_can() { - let (_directory, templates, lease, mut locked) = committed_clone(&this_process()); + let (_directory, store) = store(); + record(&store, &this_process(), &runtime()); + let mut locked = store.lock(None).unwrap(); // The recorded VMM exited, but the caller cannot record what ended it. let exited = |_: &ProcessRef| -> io::Result> { Ok(None) }; let error = reattach_with(&mut locked, &runtime(), exited, |_| { @@ -626,18 +579,16 @@ fn a_reattach_whose_caller_cannot_record_an_exit_keeps_the_record_until_one_can( }) .unwrap_err(); assert_eq!(error.message(), "the test cannot record the exit"); - // The record, and with it the lease, remain for a later reattach. + // The record remains for a later reattach. assert!(locked.runtime().unwrap().is_some()); - refuses_release(&templates, &mut locked, &lease); let Reattached::Exited { record, recovery } = reattach_with(&mut locked, &runtime(), exited, |_| Ok(())).unwrap() else { panic!("a VMM that exited was not reported as exited"); }; - assert_eq!(record.unwrap().launch(), lease.launch()); + assert_eq!(record.unwrap().launch().as_str(), RECORDED_LAUNCH); assert_eq!(recovery, Recovery::Clean); assert!(locked.runtime().unwrap().is_none()); - assert_eq!(templates.release(&mut locked).unwrap(), Some(lease)); } #[test] @@ -671,62 +622,16 @@ fn a_launch_whose_vmm_cannot_start_ends() { } #[test] -fn a_clone_whose_record_names_a_vmm_that_exited_may_release_its_lease() { - let live = this_process(); - let mut earlier_boot = *live.boot.as_bytes(); - earlier_boot[0] ^= 1; - for exited in [ - // A record of an earlier boot names a VMM that has exited. - ProcessRef { - boot: BootId::from_bytes(earlier_boot), - ..live.clone() - }, - // The recorded ID now names another process: this one. - ProcessRef { - start_time: live.start_time + 1, - ..live.clone() - }, - ] { - let (_directory, templates, lease, mut locked) = committed_clone(&exited); - // While the record names the clone's VM, the lease stays. - refuses_release(&templates, &mut locked, &lease); - let Reattached::Exited { record, recovery } = reattach(&mut locked, &runtime()).unwrap() - else { - panic!("{exited:?} names a live VMM"); - }; - assert_eq!(record.unwrap().launch(), lease.launch()); - assert_eq!(recovery, Recovery::Clean); - // Reattach cleared the record of the VM that exited, so the release ends the lease. - assert_eq!(templates.release(&mut locked).unwrap(), Some(lease)); - assert_eq!(Lease::of(&locked).unwrap(), None); - } -} - -#[test] -fn a_clone_whose_vmm_cannot_be_examined_keeps_its_lease() { - let (_directory, templates, lease, mut locked) = committed_clone(&this_process()); - let hidden = |_: &ProcessRef| -> io::Result> { - Err(io::Error::new(io::ErrorKind::PermissionDenied, "hidden")) - }; - let Reattached::Indeterminate(reason) = - reattach_with(&mut locked, &runtime(), hidden, |_| Ok(())).unwrap() - else { - panic!("a VMM that cannot be examined was classified"); - }; - assert!(reason.ends_with("cannot be examined: hidden"), "{reason}"); - assert!(locked.runtime().unwrap().is_some()); - refuses_release(&templates, &mut locked, &lease); -} - -#[test] -fn a_clone_whose_forced_teardown_fails_keeps_its_lease_until_one_ends_its_vmm() { - let (_directory, templates, lease, mut locked) = committed_clone(&this_process()); +fn a_forced_teardown_that_fails_keeps_the_record_until_one_ends_the_vmm() { + let (_directory, store) = store(); + record(&store, &this_process(), &runtime()); + let mut locked = store.lock(None).unwrap(); let Reattached::Live(vmm) = reattach(&mut locked, &runtime()).unwrap() else { - panic!("the clone's VMM was not reattached"); + panic!("the recorded VMM was not reattached"); }; // This test process stands for the VMM, which these teardowns reach only through stand-ins. let bound = Duration::from_secs(10); - // A teardown that cannot terminate the VMM fails, and keeps its record, and so the lease. + // A teardown that cannot terminate the VMM fails, and keeps its record. let unkillable = Scripted { unkillable: true, ..Scripted::default() @@ -738,15 +643,15 @@ fn a_clone_whose_forced_teardown_fails_keeps_its_lease_until_one_ends_its_vmm() error.message(), format!("cannot terminate VMM process {}", std::process::id()) ); - refuses_release(&templates, &mut locked, &lease); - // One that does not end the VMM within its bound keeps them too. + assert_eq!(locked.runtime().unwrap().as_ref(), Some(vmm.record())); + // One that does not end the VMM within its bound keeps it too. let survivor = Scripted::default(); assert_eq!( vmm.terminate_with(&mut locked, bound, &survivor).unwrap(), StopOutcome::TimedOut ); - refuses_release(&templates, &mut locked, &lease); - // One that ends it clears the record, and the release ends the lease. + assert_eq!(locked.runtime().unwrap().as_ref(), Some(vmm.record())); + // One that ends it clears the record. let dying = Scripted { dies_when_killed: true, ..Scripted::default() @@ -756,14 +661,15 @@ fn a_clone_whose_forced_teardown_fails_keeps_its_lease_until_one_ends_its_vmm() StopOutcome::Forced ); assert!(locked.runtime().unwrap().is_none()); - assert_eq!(templates.release(&mut locked).unwrap(), Some(lease)); } #[test] fn only_a_stop_unless_exited_keeps_the_record_of_a_vmm_that_had_exited() { - let (_directory, templates, lease, mut locked) = committed_clone(&this_process()); + let (_directory, store) = store(); + record(&store, &this_process(), &runtime()); + let mut locked = store.lock(None).unwrap(); let Reattached::Live(vmm) = reattach(&mut locked, &runtime()).unwrap() else { - panic!("the clone's VMM was not reattached"); + panic!("the recorded VMM was not reattached"); }; // This test process stands for the VMM, which these stops reach only through a stand-in, // which exited once its holder found it running, before the stops began. @@ -775,16 +681,14 @@ fn only_a_stop_unless_exited_keeps_the_record_of_a_vmm_that_had_exited() { let exited = Scripted::default(); exited.exited.set(true); // The stop takes no step, and keeps the record, which tells a reattach that the VM exited - // unheld, and with it the lease. + // unheld. let stopped = vmm .stop_unless_exited_with(&mut locked, deadline, forced, never, &exited) .unwrap(); assert!(stopped.is_none(), "{stopped:?}"); assert_eq!(exited.operations(), ["check"]); assert_eq!(locked.runtime().unwrap().as_ref(), Some(vmm.record())); - refuses_release(&templates, &mut locked, &lease); - // A stop that ends such a VM as one that it stopped clears the record, so the release ends - // the lease. + // A stop that ends such a VM as one that it stopped clears the record. let stopped = vmm .stop_with(&mut locked, deadline, forced, never, &exited) .unwrap(); @@ -792,18 +696,19 @@ fn only_a_stop_unless_exited_keeps_the_record_of_a_vmm_that_had_exited() { assert!(stopped.graceful_error.is_none()); assert_eq!(exited.operations(), ["check", "check"]); assert!(locked.runtime().unwrap().is_none()); - assert_eq!(templates.release(&mut locked).unwrap(), Some(lease)); } #[test] fn a_stop_unless_exited_stops_a_running_vmm_as_a_stop_does() { - let (_directory, templates, lease, mut locked) = committed_clone(&this_process()); + let (_directory, store) = store(); + record(&store, &this_process(), &runtime()); + let mut locked = store.lock(None).unwrap(); let Reattached::Live(vmm) = reattach(&mut locked, &runtime()).unwrap() else { - panic!("the clone's VMM was not reattached"); + panic!("the recorded VMM was not reattached"); }; // This test process stands for the VMM, which these stops reach only through stand-ins. let deadline = Instant::now() + Duration::from_secs(60); - // One that outlives the stop keeps its record, and the lease. + // One that outlives the stop keeps its record. let survivor = Scripted::default(); let stopped = vmm .stop_unless_exited_with( @@ -817,7 +722,7 @@ fn a_stop_unless_exited_stops_a_running_vmm_as_a_stop_does() { .expect("the stop of a VMM that runs"); assert_eq!(stopped.outcome, StopOutcome::TimedOut); assert_eq!(survivor.operations(), ["check", "wait"]); - refuses_release(&templates, &mut locked, &lease); + assert_eq!(locked.runtime().unwrap().as_ref(), Some(vmm.record())); // One that heeds the graceful step exits through the stop, which clears its record. let heeding = Scripted::default(); let shut_down = |_: &Vmm, _: Instant| { @@ -831,7 +736,6 @@ fn a_stop_unless_exited_stops_a_running_vmm_as_a_stop_does() { assert_eq!(stopped.outcome, StopOutcome::Graceful); assert_eq!(heeding.operations(), ["check", "wait"]); assert!(locked.runtime().unwrap().is_none()); - assert_eq!(templates.release(&mut locked).unwrap(), Some(lease)); } #[test] diff --git a/nvx-rs/src/host/sys/mod.rs b/nvx-rs/src/host/sys/mod.rs index 574cbd89a..e9e87bba9 100644 --- a/nvx-rs/src/host/sys/mod.rs +++ b/nvx-rs/src/host/sys/mod.rs @@ -10,7 +10,7 @@ //! through libc's `prctl` too, since `nix` does not wrap seccomp. The Windows platform module //! still calls `windows-sys` directly. -#[cfg(target_os = "linux")] +#[cfg(all(feature = "cli", target_os = "linux"))] mod fs; #[cfg(feature = "cli")] mod interrupt; @@ -23,7 +23,7 @@ mod socket; pub(crate) use nix::errno::Errno; use nix::unistd::{getegid, geteuid}; -#[cfg(target_os = "linux")] +#[cfg(all(feature = "cli", target_os = "linux"))] pub(crate) use fs::rename_no_replace; #[cfg(feature = "cli")] pub(crate) use interrupt::{interrupted, watch_interrupts, write_interruptibly}; diff --git a/nvx-rs/src/host/template/mod.rs b/nvx-rs/src/host/template/mod.rs deleted file mode 100644 index 9f3997a5e..000000000 --- a/nvx-rs/src/host/template/mod.rs +++ /dev/null @@ -1,65 +0,0 @@ -//! Templates: captures of a microVM from which independent clones are restored, the metadata -//! that decides which clones a template can serve, and on which hosts, and the store that keeps -//! them. -//! -//! A template is a snapshot that OpenVMM captured at its platform tier, after it consumed the -//! invariant configuration of a microVM alone, for clones: a VM booted with a placeholder -//! `distro`, a fresh scratch, and a dormant filesystem slot, which each clone binds to its own -//! image, scratch, and export when it is restored. Its compatibility key, [`TemplateKey`], is what -//! a clone and the host that restores it must match: the runtime and the contracts through which -//! the template is restored, and the machine that it restores. -//! -//! - [`CaptureClass`] holds the invariant inputs of a capture, known before it: the runtime, the -//! guest's build, the backend, the CPU profile, the resources, the geometries of the disks, the -//! network, and the other invariant settings. -//! - [`TemplateManifest::read`] reads the manifest that OpenVMM wrote for a capture, with its time -//! contract, and refuses one that is not a template's. -//! - [`TemplateKey::new`] derives the key of a template from its class and its manifest, after it -//! checks that the capture made the machine that its class requested. -//! - [`Store`] holds the generations of templates that a host published, apart from every -//! sandbox's store: their payloads, which never change, and the versioned [`Index`] of them, -//! which names each by its [`StoreGeneration`](crate::activation::StoreGeneration), at a -//! location that the adapter chooses. -//! - [`Store::create_template`] and [`Store::refresh_template`] capture a template of a class, in -//! the VM that its [`TemplatePlan`] describes, which the supervisor launches, and publish it as -//! a generation of the store; [`Store::recover_captures`] ends the captures that a crash -//! interrupted. -//! - [`TimeContract::eligibility`] decides whether a clone may restore a capture at a host's -//! time, [`HostTime`], by the restore window of its time contract, apart from its -//! compatibility. -//! - [`Store::acquire`] leases an eligible generation to a clone, with a binding in the clone's -//! sandbox, until [`Store::release`] ends the lease once the clone's VM has exited. A tombstone -//! quarantines a generation, which takes no new lease, and [`Store::retire`] deletes it once its -//! leases have drained. -//! -//! # Snapshot metadata -//! -//! The metadata types remain aliases of [`snapshot`](super::snapshot)'s types for template -//! users. That module describes their canonical encoding and conformance vectors. - -mod plan; -mod store; - -pub use super::snapshot::{ - BlockGeometry, CaptureClass, ClassIdentity, Downtime, DowntimeSource, Eligibility, HostClock, - HostTime, Invariants, KeyIdentity, MANIFEST_NAME, MAX_DOWNTIME_NS, MAX_MANIFEST_LEN, - SnapshotKey as TemplateKey, SnapshotManifest as TemplateManifest, TimeContract, -}; -pub(super) use super::snapshot::{eligibility, key, manifest}; - -pub use self::plan::{MEMORY_BLOCK_SIZE, SCRATCH_NAME, TemplatePlan, TemplateResources}; -pub use self::store::{ - Acquired, CAPTURE_RECORD_NAME, CAPTURES_NAME, Capture, Created, GENERATIONS_NAME, Generation, - INDEX_LOCK_NAME, INDEX_NAME, Index, KEY_NAME, LEASES_NAME, Lease, LeaseId, MAX_GENERATIONS, - Retirement, SNAPSHOT_NAME, STORE_FORMAT, STORE_RECORD_NAME, StorageGeneration, Store, StoreId, - TOMBSTONES_NAME, TRANSACTION_NAME, -}; - -impl Generation { - /// Returns whether a clone may restore the generation at the host's time `now`, by the - /// restore window of its capture's time contract, as [`TimeContract::eligibility`] decides - /// it. - pub fn eligibility(&self, now: &HostTime) -> Eligibility { - self.time().eligibility(now) - } -} diff --git a/nvx-rs/src/host/template/plan.rs b/nvx-rs/src/host/template/plan.rs deleted file mode 100644 index c3b6ebda0..000000000 --- a/nvx-rs/src/host/template/plan.rs +++ /dev/null @@ -1,497 +0,0 @@ -//! The plan of a template's capture, [`TemplatePlan`]: the VM that a template's capture boots, -//! lowered from an adapter's plan, and the options with which OpenVMM captures it. - -use std::path::{Path, PathBuf}; - -use super::manifest::PROCESSOR_COUNTS; -use super::store::{SNAPSHOT_NAME, StorageGeneration}; -use crate::error::{Error, Result}; -use crate::host::paths; -#[cfg(doc)] -use crate::host::sandbox::SandboxNetwork; -use crate::host::sandbox::distro::DistroBinding; -use crate::host::sandbox::plan::bound_image; -use crate::host::sandbox::{SandboxPlan, SandboxUpper, reject_container_path, reject_disk_path}; - -/// Bytes in a mebibyte, the unit in which the plan writes OpenVMM's `--memory` and -/// `--memory-capacity`. -const MIB: u64 = 1024 * 1024; - -/// The size of the blocks of a microVM's RAM, in bytes: 128 MiB, as OpenVMM's microVM profile -/// fixes it. A template's RAM and its RAM capacity are whole blocks, and a restore expands the RAM -/// by whole blocks. -pub const MEMORY_BLOCK_SIZE: u64 = 128 * MIB; - -/// The name of the dormant scratch device's file in a capture's transaction directory: a sparse -/// file of the template's scratch length, which the template does not keep. -pub const SCRATCH_NAME: &str = "scratch.raw"; - -/// The kernel command line of a template's VM, with which the guest's init hands the VM to its -/// sandbox control. -const COMMAND_LINE: &str = "nvx_sandbox=1"; - -/// The kernel command-line parameter that limits the processors that the guest brings online at -/// boot, and with which OpenVMM records them as the template's processors online at boot. -const BOOT_PROCESSORS: &str = "maxcpus"; - -/// The snapshot tier of a template's capture: OpenVMM's clone point of the invariant -/// configuration alone. -const TIER: &str = "platform"; - -/// The block identity of a template's capture: the bound-block storage generation, which OpenVMM -/// takes in place of a digest of each block's content. -const BLOCK_IDENTITY: &str = "generation"; - -/// The logical block size that OpenVMM presents for a sandbox block device, of which the device's -/// length is a multiple. -const LOGICAL_BLOCK_SIZE: u64 = 512; - -/// The resources of a template's VM: its processors, of which a prefix is online at boot, and its -/// RAM, with the capacity to which a restore may expand it. -/// -/// A template's VM has the processors of the largest of its clones, its processor capacity, of -/// which the first, those online at boot, are online when it is captured. A restore brings a -/// prefix of the others online, so that one template serves clones of each processor count from -/// those online at boot to its capacity. A template whose processors are all online at boot -/// serves clones of its capacity alone. [`TemplatePlan::new`] checks the resources. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] -pub struct TemplateResources { - /// The processor capacity: 1, 2, 4, or 8. - pub processors: u32, - /// The processors online at boot: 1, 2, 4, or 8, at most [`processors`](Self::processors). - /// Fewer than the capacity limit those that the guest brings online at boot, as the - /// [kernel command line](TemplatePlan::kernel_command_line) says. - pub boot_processors: u32, - /// The RAM at boot, in bytes: a nonzero multiple of [`MEMORY_BLOCK_SIZE`]. - pub memory: u64, - /// The RAM capacity, in bytes, to which a restore may expand the RAM: a multiple of - /// [`MEMORY_BLOCK_SIZE`] of at least [`memory`](Self::memory). - pub memory_capacity: u64, -} - -/// The plan of a template's capture: the VM that the capture boots, lowered from an adapter's -/// plan, and the options with which OpenVMM captures it, which -/// [`openvmm_arguments`](Self::openvmm_arguments) and -/// [`kernel_command_line`](Self::kernel_command_line) render. -/// -/// A template's VM boots its invariant configuration alone, for clones, each of which binds its -/// own image, upper, and export when it is restored. [`new`](Self::new) lowers an adapter's plan, -/// which [`SandboxPlan::adapter`] makes, into that VM: -/// -/// - The plan's `distro` is the placeholder: a trusted platform artifact, which its -/// [`DistroBinding`] binds, and whose length is the capacity of every clone's image. -/// - A dormant scratch device takes the place of the plan's upper, which is a clone's: a sparse -/// file of the template's scratch length, [`SCRATCH_NAME`] in the capture's transaction -/// directory, which sets the device's length and block sizes, and which the template does not -/// keep. -/// - The filesystem slot, which every cold boot exposes, stays dormant: the plan exports nothing. -/// - The plan's network gives the template its fixed NIC, at the network's guest address, under -/// the complete deny-all policy, whatever the network's own, as -/// [`SandboxNetwork::template_arguments`] lowers it: egress denied without rules, ingress and -/// host-loopback access denied, without a proxy or forwards. It is the one policy that a -/// restore may replace, so each clone's restore installs the clone's own policy on the NIC. A -/// plan without a network, or with one that lets nothing through, which attaches no NIC, gives -/// the template none. -/// - Its [`TemplateResources`]: its processor capacity, of which those online at boot are online -/// when it is captured, and its RAM, with the capacity to which a restore may expand it, -/// OpenVMM's `--memory-capacity`. -/// -/// # The kernel command line -/// -/// OpenVMM keeps the effective kernel command line of a template, which its -/// [key](super::TemplateKey) holds and every clone restores, and refuses a capture whose command -/// line holds a token outside the allowlist of its platform tier. The template's own command line -/// is `nvx_sandbox=1`, with which the guest's init hands the VM to its sandbox control, and for a -/// template that boots with fewer processors online than its capacity, `maxcpus=` with the -/// processors online at boot, which limits those that the guest brings online at boot, and which -/// OpenVMM records as the template's, so that each restore may bring a prefix of the others -/// online. OpenVMM writes the other tokens itself, each on the allowlist: the early console, -/// console, reboot, and panic settings, `nr_cpus=` with the processor capacity, -/// `nvx_snapshot_tier=platform`, the virtio devices and the control tty that the guest discovers, -/// and the NIC's addresses. The arguments name no workload identity, lifecycle, share, or -/// published listener, for which OpenVMM would write other tokens, and under the NIC's deny-all -/// policy, OpenVMM names no DNS server either. The command line therefore holds no image -/// identity, hostname, share, proxy, DNS server, entrypoint, or other tenant data, and every -/// clone of the template has the same. -/// -/// # The capture -/// -/// The capture's options are `--snapshot-tier platform`; `--snapshot-destination`, the -/// destination [`SNAPSHOT_NAME`] in the capture's transaction directory, by which a generation of -/// a [`Store`](super::Store) names its snapshot too; and `--snapshot-block-identity generation`, -/// with the capture's [`StorageGeneration`] as `--snapshot-generation-id`, so that the capture -/// hashes no block device. The template keeps no storage generation: OpenVMM unbinds its -/// `distro`, and its scratch is fresh. -/// -/// The transaction directory is the capture's own, and the destination must not exist when the -/// capture launches. OpenVMM backs the VM's RAM with a file that it creates beside the -/// destination, and publishes the snapshot at the destination, so every file of the capture is in -/// the transaction directory. -/// -/// The caller creates the transaction directory and the sparse scratch file, checks the -/// placeholder against its binding, as [`DistroBinding::check`] does, and adds its own arguments, -/// such as the hypervisor, the kernel and the initramfs, the CPU profile, and the consoles. It -/// passes [`kernel_command_line`](Self::kernel_command_line) as OpenVMM's `--cmdline`, alone, and -/// names no workload identity, lifecycle, share, or RAM backing file of its own: -/// -/// ```no_run -/// use std::path::Path; -/// -/// use nvx::host::sandbox::distro::DistroBinding; -/// use nvx::host::sandbox::SandboxPlan; -/// use nvx::host::template::{MEMORY_BLOCK_SIZE, StorageGeneration, TemplatePlan, TemplateResources}; -/// -/// # fn capture( -/// # adapter: &SandboxPlan, -/// # binding: &DistroBinding, -/// # transaction: &Path, -/// # generation: StorageGeneration, -/// # ) -> nvx::Result<()> { -/// let resources = TemplateResources { -/// processors: 4, -/// boot_processors: 1, -/// memory: 4 * MEMORY_BLOCK_SIZE, -/// memory_capacity: 16 * MEMORY_BLOCK_SIZE, -/// }; -/// let template = -/// TemplatePlan::new(adapter, binding, 64 << 20, resources, transaction, generation)?; -/// let mut arguments = template.openvmm_arguments()?; -/// arguments.extend(["--cmdline".to_owned(), template.kernel_command_line().to_owned()]); -/// // ... the caller's own arguments: the hypervisor, the kernel and the initramfs, the CPU -/// // profile, and the consoles ... -/// # Ok(()) -/// # } -/// ``` -#[derive(Clone, Debug, PartialEq, Eq)] -pub struct TemplatePlan { - /// The capture's devices but its NIC: an adapter's plan of the placeholder over the dormant - /// scratch device, without a network. - plan: SandboxPlan, - /// OpenVMM's arguments of the template's NIC, under the complete deny-all policy, or none for - /// a template without a NIC. - nic: Vec, - /// The binding of the placeholder. - binding: DistroBinding, - /// The length of the dormant scratch device, in bytes. - scratch_length: u64, - /// The processors and RAM. - resources: TemplateResources, - /// The kernel command line of the capture's VM. - command_line: String, - /// The capture's transaction directory, spelled with its components alone. - transaction: PathBuf, - /// The snapshot's destination in the transaction directory. - destination: PathBuf, - /// The bound-block storage generation of the capture. - generation: StorageGeneration, -} - -impl TemplatePlan { - /// Lowers `adapter`, an adapter's plan, which [`SandboxPlan::adapter`] makes, into the plan of - /// a template's capture: of its placeholder, which `binding` binds, over a dormant scratch - /// device of `scratch_length` bytes, with the NIC of its network, the processors and RAM of - /// `resources`, the capture's transaction directory `transaction`, spelled with its components - /// alone, as [`Path::components`] reads them, and its bound-block storage generation - /// `generation`, which is apart from the other [namespaces](super::Store#four-namespaces) of a - /// template's generations. - /// - /// The plan's upper is no part of the template, which attaches its dormant scratch device in - /// the upper's place, whatever the upper. - /// - /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported), checking in this - /// order: a plan of the sandbox profile, with workload settings; a plan whose image is not the - /// format and identity that `binding` names, as for a binding whose identity is not a UUID in - /// its canonical text; a placeholder whose path is relative, which OpenVMM would resolve - /// against the directory in which it runs, holds a comma or a semicolon, which OpenVMM's block - /// device options would split, or whose name ends in `.vhd`, `.vhdx`, `.vmgs`, or `.iso`, - /// which OpenVMM would not attach as a raw image; a placeholder whose binding records - /// a length, or a scratch length, that is not a nonzero multiple of the 512 bytes of the - /// logical blocks that OpenVMM presents; - /// processors other than 1, 2, 4, or 8, or processors online at boot other than 1, 2, 4, or - /// 8, or more than the processors; RAM or a RAM capacity that is not a nonzero multiple - /// of [`MEMORY_BLOCK_SIZE`], or a capacity below the RAM; and a transaction directory that is - /// not valid Unicode, that is relative, which OpenVMM would resolve against the directory in - /// which it runs, or whose path holds a comma or a semicolon, which OpenVMM's block device - /// options would split. - /// - /// It reads, opens, and creates nothing. - pub fn new( - adapter: &SandboxPlan, - binding: &DistroBinding, - scratch_length: u64, - resources: TemplateResources, - transaction: &Path, - generation: StorageGeneration, - ) -> Result { - if adapter.settings().is_some() { - return Err(Error::unsupported( - "a template lowers an adapter's plan, not a plan of the sandbox profile, whose \ - workload settings no template captures", - )); - } - let distro = adapter.distro(); - if bound_image(binding)? != distro.image() { - return Err(Error::unsupported(format!( - "the placeholder {} is not the {} layer with identity {} that its distro binding \ - names", - distro.path().display(), - binding.format(), - binding.identity() - ))); - } - // OpenVMM attaches the placeholder at the path that its arguments name, which they must - // hold whole, as a raw image, and which OpenVMM would resolve against the directory in - // which it runs if it were relative. - let placeholder = distro.path(); - if !placeholder.is_absolute() { - return Err(Error::unsupported(format!( - "the placeholder of a template's capture must be absolute, since OpenVMM would \ - resolve it against the directory in which it runs: {}", - placeholder.display() - ))); - } - let shown = placeholder.display().to_string(); - reject_disk_path(placeholder, &shown)?; - reject_container_path(placeholder, &shown)?; - let capacity = binding.image().length(); - if !is_block_length(capacity) { - return Err(Error::unsupported(format!( - "the placeholder {} of {capacity} bytes cannot be attached: OpenVMM attaches block \ - devices of nonzero multiples of {LOGICAL_BLOCK_SIZE} bytes", - distro.path().display() - ))); - } - if !is_block_length(scratch_length) { - return Err(Error::unsupported(format!( - "a template's scratch of {scratch_length} bytes cannot be attached: OpenVMM \ - attaches block devices of nonzero multiples of {LOGICAL_BLOCK_SIZE} bytes" - ))); - } - check_resources(&resources)?; - let nic = match adapter.network() { - Some(network) => network.template_arguments()?, - None => Vec::new(), - }; - let transaction = transaction_directory(transaction)?; - let scratch = transaction.join(SCRATCH_NAME); - reject_disk_path(&scratch, &scratch.display().to_string())?; - let destination = transaction.join(SNAPSHOT_NAME); - let plan = SandboxPlan::adapter(distro.clone(), SandboxUpper::scratch_at(scratch)); - // A template whose processors are all online at boot needs no limit, and records none, - // as OpenVMM records none without the parameter. - let command_line = if resources.boot_processors < resources.processors { - format!( - "{COMMAND_LINE} {BOOT_PROCESSORS}={}", - resources.boot_processors - ) - } else { - COMMAND_LINE.to_owned() - }; - Ok(Self { - plan, - nic, - binding: binding.clone(), - scratch_length, - resources, - command_line, - transaction, - destination, - generation, - }) - } - - /// Returns the capture's devices but its NIC: an adapter's plan of the placeholder over the - /// dormant scratch device, without a network, whose NIC [`nic_arguments`](Self::nic_arguments) - /// attaches. - pub fn plan(&self) -> &SandboxPlan { - &self.plan - } - - /// Returns OpenVMM's arguments of the template's NIC, at the guest address of the adapter's - /// network, under the complete deny-all policy, as [`SandboxNetwork::template_arguments`] - /// lowers them, or none for a template without a NIC. - pub fn nic_arguments(&self) -> &[String] { - &self.nic - } - - /// Returns the binding of the placeholder. - pub fn binding(&self) -> &DistroBinding { - &self.binding - } - - /// Returns the length of the placeholder, in bytes, as its binding records it: the capacity - /// of every clone's image. - pub fn image_capacity(&self) -> u64 { - self.binding.image().length() - } - - /// Returns the dormant scratch device's file, [`SCRATCH_NAME`] in the transaction directory. - pub fn scratch_path(&self) -> &Path { - self.plan - .upper() - .scratch_path() - .expect("a template's upper is its dormant scratch device") - } - - /// Returns the length of the dormant scratch device, in bytes, which its sparse file sets. - pub fn scratch_length(&self) -> u64 { - self.scratch_length - } - - /// Returns the processors and RAM. - pub fn resources(&self) -> TemplateResources { - self.resources - } - - /// Returns the capture's transaction directory. - pub fn transaction(&self) -> &Path { - self.transaction.as_path() - } - - /// Returns the snapshot's destination, [`SNAPSHOT_NAME`] in the transaction directory. - pub fn destination(&self) -> &Path { - self.destination.as_path() - } - - /// Returns the bound-block storage generation of the capture. - pub fn generation(&self) -> StorageGeneration { - self.generation - } - - /// Returns the kernel command line of the capture's VM, which the caller passes as OpenVMM's - /// `--cmdline`, alone: `nvx_sandbox=1`, then, for a template that boots with fewer processors - /// online than its capacity, `maxcpus=` with the processors online at boot, as - /// [The kernel command line](Self#the-kernel-command-line) describes. - /// - /// It takes the place of the command line of [`plan`](Self::plan), which - /// [`SandboxPlan::kernel_command_line`] and a [`PreparedLaunch`] render: that one names the - /// boot tokens of the plan's root, the identity of its image among them, which a clone binds - /// when it is restored, and which OpenVMM's platform allowlist refuses. - /// - /// [`PreparedLaunch`]: crate::host::sandbox::PreparedLaunch - pub fn kernel_command_line(&self) -> &str { - &self.command_line - } - - /// Returns OpenVMM's arguments of the capture: those of its devices, as - /// [`SandboxPlan::openvmm_arguments`] renders them for [`plan`](Self::plan), the microVM - /// machine, the placeholder as the read-only `distro` block device, and the dormant scratch - /// device as the `scratch` one; then those of the NIC, if any, - /// [`nic_arguments`](Self::nic_arguments); then those of - /// [`capture_arguments`](Self::capture_arguments). - /// - /// The arguments name the block devices by their paths, which OpenVMM opens itself. A capture - /// opens each without following a link at its path, and on Windows without sharing write - /// access, so each path must name the file itself, which no other process holds open for - /// writing. The arguments hold neither the kernel command line, which - /// [`kernel_command_line`](Self::kernel_command_line) gives, nor the caller's own arguments. - /// - /// Fails as [`SandboxPlan::openvmm_arguments`] does. - pub fn openvmm_arguments(&self) -> Result> { - let mut arguments = self.plan.openvmm_arguments()?; - arguments.extend_from_slice(&self.nic); - arguments.extend(self.capture_arguments()); - Ok(arguments) - } - - /// Returns OpenVMM's arguments of the capture besides those of its devices: `--processors` - /// with the processor capacity, `--memory` and `--memory-capacity` in MiB, and the capture's - /// options, `--snapshot-destination`, `--snapshot-tier platform`, - /// `--snapshot-block-identity generation`, and `--snapshot-generation-id`. The processors - /// online at boot are the kernel command line's. - /// - /// A caller that renders the arguments of the devices itself adds these to them. - pub fn capture_arguments(&self) -> Vec { - let TemplateResources { - processors, - memory, - memory_capacity, - .. - } = self.resources; - [ - ("--processors", processors.to_string()), - ("--memory", mebibytes(memory)), - ("--memory-capacity", mebibytes(memory_capacity)), - ( - "--snapshot-destination", - self.destination.display().to_string(), - ), - ("--snapshot-tier", TIER.to_owned()), - ("--snapshot-block-identity", BLOCK_IDENTITY.to_owned()), - ("--snapshot-generation-id", self.generation.to_string()), - ] - .into_iter() - .flat_map(|(option, value)| [option.to_owned(), value]) - .collect() - } -} - -/// Returns whether OpenVMM attaches a sandbox block device of `length` bytes: a nonzero multiple -/// of [`LOGICAL_BLOCK_SIZE`]. -fn is_block_length(length: u64) -> bool { - length != 0 && length.is_multiple_of(LOGICAL_BLOCK_SIZE) -} - -/// Refuses `resources` that no template's VM has: processors other than 1, 2, 4, or 8, processors -/// online at boot other than 1, 2, 4, or 8, or more than the processors, and RAM or a RAM capacity -/// that is not a nonzero multiple of [`MEMORY_BLOCK_SIZE`], or a capacity below the RAM. -fn check_resources(resources: &TemplateResources) -> Result<()> { - let TemplateResources { - processors, - boot_processors, - memory, - memory_capacity, - } = *resources; - if !PROCESSOR_COUNTS.contains(&processors) { - return Err(Error::unsupported(format!( - "a template's VM has 1, 2, 4, or 8 processors, not {processors}" - ))); - } - if !PROCESSOR_COUNTS.contains(&boot_processors) || boot_processors > processors { - return Err(Error::unsupported(format!( - "a template's VM boots with 1, 2, 4, or 8 of its {processors} processors online, not \ - {boot_processors}" - ))); - } - if memory == 0 - || !memory.is_multiple_of(MEMORY_BLOCK_SIZE) - || !memory_capacity.is_multiple_of(MEMORY_BLOCK_SIZE) - || memory_capacity < memory - { - return Err(Error::unsupported(format!( - "a template's VM needs nonzero RAM, and a RAM capacity of at least it, each a multiple \ - of {} MiB, not {memory} bytes of RAM and a capacity of {memory_capacity} bytes", - MEMORY_BLOCK_SIZE / MIB - ))); - } - Ok(()) -} - -/// Returns the transaction directory `path`, spelled with its components alone, as -/// [`Path::components`] reads them, which must be valid Unicode and absolute. -fn transaction_directory(path: &Path) -> Result { - if path.to_str().is_none() { - return Err(Error::unsupported(format!( - "the transaction directory of a template's capture is not valid Unicode: {}", - path.display() - ))); - } - let directory = paths::normalized(path); - if !directory.is_absolute() { - return Err(Error::unsupported(format!( - "the transaction directory of a template's capture must be absolute, since OpenVMM \ - would resolve it against the directory in which it runs: {}", - directory.display() - ))); - } - Ok(directory) -} - -/// Returns `bytes`, a multiple of [`MIB`], as OpenVMM's memory sizes write it: the number of MiB -/// and `M`. -fn mebibytes(bytes: u64) -> String { - format!("{}M", bytes / MIB) -} - -#[cfg(test)] -mod tests; diff --git a/nvx-rs/src/host/template/plan/tests.rs b/nvx-rs/src/host/template/plan/tests.rs deleted file mode 100644 index 3144eb883..000000000 --- a/nvx-rs/src/host/template/plan/tests.rs +++ /dev/null @@ -1,1031 +0,0 @@ -//! Tests of the plans of templates' captures: the lowering of adapters' plans, the rendered -//! arguments, the kernel command line against OpenVMM's platform allowlist, and the refusals. - -use std::path::Path; - -use serde_json::json; - -use super::*; -use crate::ErrorKind; -use crate::host::network::{ - Access, EgressRule, ForwardProtocol, HostLoopback, LoopbackForward, NetworkPolicy, - NetworkProxy, Protocol, -}; -use crate::host::sandbox::distro::{BINDING_VERSION, DISTRO_POLICY_REVISION}; -use crate::host::sandbox::{SandboxDistro, SandboxNetwork, SandboxSettings}; -use crate::host::seal::SEAL_VERSION; -use crate::protocol::cmdline; -use crate::protocol::root::RootImage; -use crate::uuid::Uuid; - -/// OpenVMM's platform allowlist of kernel command-line tokens, compiled from the copies of its -/// source in `tests/vectors/openvmm/`, which an ignored test of `tests/openvmm_vectors.rs` -/// compares with the OpenVMM submodule. -#[allow(unreachable_pub, clippy::missing_docs_in_private_items)] -mod openvmm { - /// OpenVMM's `openvmm_defs` crate, of which the allowlist reads one constant. - mod openvmm_defs { - /// OpenVMM's `openvmm_defs::microvm`. - pub(super) mod microvm { - include!("../../../../tests/vectors/openvmm/microvm_control_tty_command_line.rs"); - } - } - - include!("../../../../tests/vectors/openvmm/platform_command_line_token_is_invariant.rs"); - - /// Returns whether OpenVMM's platform allowlist admits `token`. - pub(super) fn admits(token: &str) -> bool { - platform_command_line_token_is_invariant(token) - } -} - -/// Bytes in a GiB. -const GIB: u64 = 1024 * MIB; - -/// The identity of the placeholder. -const IDENTITY: &str = "11111111-1111-1111-1111-111111111111"; - -/// The placeholder. -const PLACEHOLDER: &str = if cfg!(windows) { - r"C:\platform\placeholder.img" -} else { - "/platform/placeholder.img" -}; - -/// A clone's scratch, which an adapter's plan names as its upper. -const CLONE_SCRATCH: &str = if cfg!(windows) { - r"C:\sandboxes\tenant-a\scratch.ext4" -} else { - "/sandboxes/tenant-a/scratch.ext4" -}; - -/// The capture's transaction directory. -const TRANSACTION: &str = if cfg!(windows) { - r"C:\templates\tx-1" -} else { - "/templates/tx-1" -}; - -/// The dormant scratch device's file in [`TRANSACTION`]. -const SCRATCH: &str = if cfg!(windows) { - r"C:\templates\tx-1\scratch.raw" -} else { - "/templates/tx-1/scratch.raw" -}; - -/// The snapshot's destination in [`TRANSACTION`]. -const DESTINATION: &str = if cfg!(windows) { - r"C:\templates\tx-1\snapshot" -} else { - "/templates/tx-1/snapshot" -}; - -/// The text of [`generation`]. -const GENERATION: &str = "000102030405060708090a0b0c0d0e0f"; - -/// The options of templates' arguments, each of which takes one value. -const OPTIONS: [&str; 14] = [ - "--machine", - "--microvm-sandbox-block", - "--net", - "--network-profile", - "--network-egress", - "--network-ingress", - "--host-loopback", - "--processors", - "--memory", - "--memory-capacity", - "--snapshot-destination", - "--snapshot-tier", - "--snapshot-block-identity", - "--snapshot-generation-id", -]; - -/// Returns the EROFS image of the placeholder. -fn erofs() -> RootImage { - RootImage::erofs(Some(Uuid::parse(IDENTITY).unwrap())) -} - -/// Returns the GPT image of the placeholder. -fn gpt() -> RootImage { - RootImage::gpt(Some(Uuid::parse(IDENTITY).unwrap())) -} - -/// Returns the binding of a placeholder of `format` with `identity`, of `length` bytes. -fn binding_of(format: &str, identity: &str, length: u64) -> DistroBinding { - serde_json::from_value(json!({ - "version": BINDING_VERSION, - "format": format, - "identity": identity, - "image": { - "version": SEAL_VERSION, - "scheme": "linux", - "device": 2049, - "fileId": "0000000000000000000000000004a2f1", - "kind": "regular", - "length": length, - "modified": {"seconds": 1_792_000_000, "nanoseconds": 5}, - "changed": {"seconds": 1_792_000_000, "nanoseconds": 5}, - }, - "manifest": null, - "policy": {"revision": DISTRO_POLICY_REVISION, "result": "accepted"}, - })) - .unwrap() -} - -/// Returns the binding of the EROFS placeholder of 1 GiB. -fn binding() -> DistroBinding { - binding_of("erofs", IDENTITY, GIB) -} - -/// Returns an adapter's plan of the placeholder as `image` over `upper`, with `network`. -fn adapter_with( - image: RootImage, - upper: SandboxUpper, - network: Option, -) -> SandboxPlan { - adapter_at(PLACEHOLDER, image, upper, network) -} - -/// Returns an adapter's plan of the placeholder at `path` as `image` over `upper`, with -/// `network`. -fn adapter_at( - path: &str, - image: RootImage, - upper: SandboxUpper, - network: Option, -) -> SandboxPlan { - SandboxPlan::adapter(SandboxDistro::new(path, image), upper).with_network(network) -} - -/// Returns an adapter's plan of the EROFS placeholder over a clone's scratch, with `network`. -fn adapter(network: Option) -> SandboxPlan { - adapter_with(erofs(), SandboxUpper::scratch(CLONE_SCRATCH), network) -} - -/// Returns the network of a guest at `10.0.0.2/24` under `policy`. -fn network(policy: NetworkPolicy) -> SandboxNetwork { - SandboxNetwork::new("10.0.0.2/24", policy).unwrap() -} - -/// Returns a policy that allows egress, and host-loopback access from the guest alone. -fn outbound() -> NetworkPolicy { - let mut policy = NetworkPolicy::new(Access::Allow); - policy.host_loopback = HostLoopback::Outbound; - policy -} - -/// Returns a policy that allows ingress. -fn listening() -> NetworkPolicy { - let mut policy = NetworkPolicy::new(Access::Deny); - policy.ingress = Access::Allow; - policy -} - -/// Returns a policy that allows egress, and host-loopback access both ways, with a forward of a -/// TCP port of host loopback to the guest. -fn forwarded() -> NetworkPolicy { - let mut policy = NetworkPolicy::new(Access::Allow); - policy.host_loopback = HostLoopback::Allow; - policy.host_loopback_forwards.push(LoopbackForward { - protocol: ForwardProtocol::Tcp, - host_port: 18080, - guest_port: 8080, - }); - policy -} - -/// Returns the policies of networks that attach a NIC: they allow egress, egress to some -/// destinations, egress and host-loopback access from the guest, a proxy alone, ingress, and -/// host-loopback access both ways, with a forward. -fn attaching() -> Vec { - vec![ - NetworkPolicy::new(Access::Allow), - NetworkPolicy::new(Access::Deny) - .with_allow(EgressRule::to("192.0.2.0/24").on_port(Protocol::Tcp, 443)) - .with_deny(EgressRule::to("192.0.2.128/25")), - outbound(), - NetworkPolicy::proxy_only(NetworkProxy::new(3128)), - listening(), - forwarded(), - ] -} - -/// Returns the networks of templates: none; one under a policy that lets nothing through, which -/// attaches no NIC; and those under each policy of [`attaching`]. -fn networks() -> Vec> { - let mut networks = vec![None, Some(network(NetworkPolicy::new(Access::Deny)))]; - networks.extend(attaching().into_iter().map(|policy| Some(network(policy)))); - networks -} - -/// Returns the resources of `processors`, all online at boot, with 512 MiB of RAM and a capacity -/// of 2 GiB. -fn resources(processors: u32) -> TemplateResources { - TemplateResources { - processors, - boot_processors: processors, - memory: 512 * MIB, - memory_capacity: 2 * GIB, - } -} - -/// Returns the storage generation whose bytes are 0 to 15. -fn generation() -> StorageGeneration { - StorageGeneration::from_bytes([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]).unwrap() -} - -/// The inputs of a lowering, which tests change from their defaults: an adapter's plan of the -/// EROFS placeholder over a clone's scratch, without a network, the placeholder's binding, 64 MiB -/// of scratch, the resources of 2 processors, and [`TRANSACTION`]. -#[derive(Clone)] -struct Inputs { - /// The adapter's plan. - adapter: SandboxPlan, - /// The binding of the placeholder. - binding: DistroBinding, - /// The length of the dormant scratch device. - scratch_length: u64, - /// The processors and RAM. - resources: TemplateResources, - /// The transaction directory. - transaction: String, -} - -impl Default for Inputs { - fn default() -> Self { - Self { - adapter: adapter(None), - binding: binding(), - scratch_length: 64 * MIB, - resources: resources(2), - transaction: TRANSACTION.to_owned(), - } - } -} - -impl Inputs { - /// Returns the default inputs with `adapter`. - fn of(adapter: SandboxPlan) -> Self { - Self { - adapter, - ..Self::default() - } - } - - /// Lowers the adapter's plan with the other inputs and [`generation`]. - fn lower(&self) -> Result { - TemplatePlan::new( - &self.adapter, - &self.binding, - self.scratch_length, - self.resources, - Path::new(&self.transaction), - generation(), - ) - } -} - -/// Returns the template of `adapter`, with the other inputs' defaults. -fn template(adapter: SandboxPlan) -> TemplatePlan { - Inputs::of(adapter).lower().unwrap() -} - -/// Returns the templates of every format of placeholder, of adapters' plans over a clone's scratch -/// and over RAM, of every network of [`networks`], and of every processor count, with every count -/// up to it online at boot. -fn every_template() -> Vec { - let mut templates = Vec::new(); - for (format, image) in [("erofs", erofs()), ("gpt", gpt())] { - for upper in [SandboxUpper::scratch(CLONE_SCRATCH), SandboxUpper::RAM] { - for network in networks() { - for processors in PROCESSOR_COUNTS { - for boot_processors in PROCESSOR_COUNTS - .into_iter() - .filter(|boot| *boot <= processors) - { - let inputs = Inputs { - adapter: adapter_with(image, upper.clone(), network.clone()), - binding: binding_of(format, IDENTITY, GIB), - resources: TemplateResources { - boot_processors, - ..resources(processors) - }, - ..Inputs::default() - }; - templates.push(inputs.lower().unwrap()); - } - } - } - } - } - templates -} - -/// Returns `texts` as owned strings. -fn strings(texts: &[&str]) -> Vec { - texts.iter().map(|text| (*text).to_owned()).collect() -} - -/// Returns the value of the one `option` of `arguments`, whose options each take one value. -fn value<'a>(arguments: &'a [String], option: &str) -> &'a str { - let values: Vec<&str> = arguments - .chunks(2) - .filter(|pair| pair[0] == option) - .map(|pair| pair[1].as_str()) - .collect(); - assert_eq!(values.len(), 1, "{option} in {arguments:?}"); - values[0] -} - -/// Returns the message of the error of `result`, which must be -/// [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported). -fn refusal(result: Result) -> String { - let error = result.unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); - error.to_string() -} - -#[test] -fn templates_render_their_devices_resources_and_capture_options() { - let devices = strings(&[ - "--machine", - "microvm", - "--microvm-sandbox-block", - &format!("distro:file:{PLACEHOLDER},ro"), - "--microvm-sandbox-block", - &format!("scratch:file:{SCRATCH}"), - ]); - let capture = strings(&[ - "--processors", - "2", - "--memory", - "512M", - "--memory-capacity", - "2048M", - "--snapshot-destination", - DESTINATION, - "--snapshot-tier", - "platform", - "--snapshot-block-identity", - "generation", - "--snapshot-generation-id", - GENERATION, - ]); - // No NIC, without a network or with one that lets nothing through. - for network in [None, Some(network(NetworkPolicy::new(Access::Deny)))] { - let template = template(adapter(network)); - assert!(template.nic_arguments().is_empty()); - assert_eq!(template.capture_arguments(), capture); - assert_eq!( - template.openvmm_arguments().unwrap(), - [devices.clone(), capture.clone()].concat() - ); - } - // One fixed NIC, at the network's guest address, under the complete deny-all policy, whatever - // the network's own, which each clone's restore replaces. - let nic = strings(&[ - "--net", - "10.0.0.2/24", - "--network-profile", - "portable", - "--network-egress", - "deny", - "--network-ingress", - "deny", - "--host-loopback", - "deny", - ]); - for policy in attaching() { - let template = template(adapter(Some(network(policy)))); - assert_eq!(template.nic_arguments(), nic); - assert_eq!(template.capture_arguments(), capture); - assert_eq!( - template.openvmm_arguments().unwrap(), - [devices.clone(), nic.clone(), capture.clone()].concat() - ); - } - // RAM and its capacity in MiB. - let inputs = Inputs { - resources: TemplateResources { - processors: 8, - boot_processors: 8, - memory: MEMORY_BLOCK_SIZE, - memory_capacity: MEMORY_BLOCK_SIZE, - }, - ..Inputs::default() - }; - assert_eq!( - inputs.lower().unwrap().capture_arguments()[..6], - strings(&[ - "--processors", - "8", - "--memory", - "128M", - "--memory-capacity", - "128M", - ]) - ); -} - -#[test] -fn templates_keep_the_placeholder_and_the_files_of_their_capture_in_its_transaction() { - let network = Some(network(NetworkPolicy::new(Access::Allow))); - let template = template(adapter(network.clone())); - let plan = template.plan(); - assert_eq!(plan.distro().path(), Path::new(PLACEHOLDER)); - assert_eq!(plan.distro().image(), erofs()); - assert_eq!(plan.upper().scratch_path(), Some(Path::new(SCRATCH))); - assert_eq!(plan.settings(), None); - assert_eq!(plan.export(), None); - assert_eq!(plan.network(), None); - assert_eq!(template.binding(), &binding()); - assert_eq!(template.image_capacity(), GIB); - assert_eq!(template.scratch_path(), Path::new(SCRATCH)); - assert_eq!(template.scratch_length(), 64 * MIB); - assert_eq!(template.resources(), resources(2)); - assert_eq!(template.transaction(), Path::new(TRANSACTION)); - assert_eq!(template.destination(), Path::new(DESTINATION)); - assert_eq!(template.generation(), generation()); - assert_eq!(template.kernel_command_line(), "nvx_sandbox=1"); - assert_eq!( - (SCRATCH_NAME, SNAPSHOT_NAME, MEMORY_BLOCK_SIZE), - ("scratch.raw", "snapshot", 128 * MIB) - ); - // The capacity of every clone's image is the placeholder's length, as its binding records it. - let inputs = Inputs { - binding: binding_of("erofs", IDENTITY, 8 * GIB + 512), - scratch_length: 512, - ..Inputs::default() - }; - let template = inputs.lower().unwrap(); - assert_eq!( - (template.image_capacity(), template.scratch_length()), - (8 * GIB + 512, 512) - ); -} - -#[test] -fn templates_limit_their_processors_online_at_boot_on_their_command_line() { - // A template whose processors are all online at boot sets no limit, and OpenVMM records none. - for processors in PROCESSOR_COUNTS { - let template = Inputs { - resources: resources(processors), - ..Inputs::default() - } - .lower() - .unwrap(); - assert_eq!(template.kernel_command_line(), "nvx_sandbox=1"); - } - // One that boots with fewer online limits them, and keeps its capacity. - let inputs = Inputs { - resources: TemplateResources { - boot_processors: 2, - ..resources(8) - }, - ..Inputs::default() - }; - let template = inputs.lower().unwrap(); - assert_eq!(template.kernel_command_line(), "nvx_sandbox=1 maxcpus=2"); - assert_eq!(value(&template.capture_arguments(), "--processors"), "8"); - assert_eq!(template.resources(), inputs.resources); -} - -#[test] -fn the_upper_of_an_adapters_plan_is_a_clones_and_no_part_of_its_template() { - let over_scratch = template(adapter(None)); - let over_ram = template(adapter_with(erofs(), SandboxUpper::RAM, None)); - assert_eq!(over_ram, over_scratch); - assert_eq!( - over_ram.openvmm_arguments().unwrap(), - over_scratch.openvmm_arguments().unwrap() - ); - assert_eq!(over_ram.scratch_path(), Path::new(SCRATCH)); -} - -#[test] -fn transaction_directories_are_spelled_with_their_components() { - let given = if cfg!(windows) { - r"C:/templates//tx-1/." - } else { - "/templates//tx-1/./" - }; - let inputs = Inputs { - transaction: given.to_owned(), - ..Inputs::default() - }; - let template = inputs.lower().unwrap(); - assert_eq!(template.transaction().to_str(), Some(TRANSACTION)); - assert_eq!(template.scratch_path().to_str(), Some(SCRATCH)); - assert_eq!(template.destination().to_str(), Some(DESTINATION)); - assert_eq!(template, Inputs::default().lower().unwrap()); -} - -#[test] -fn openvmms_platform_allowlist_admits_its_tokens_and_refuses_tenant_data() { - for token in [ - "earlycon=xe9", - "console=hvc0", - "console=hvc1", - "reboot=t", - "panic=-1", - "nvx_sandbox=1", - "nvx_config=0xd0010000,65536", - "nvx_snapshot_tier=platform", - "nr_cpus=8", - "maxcpus=1", - "nvx_control_tty=hvc2", - "virtio_mmio.device=0x1000@0xd0001000:17", - "virtnet_ip=10.0.0.2", - "virtnet_mask=255.255.255.0", - "virtnet_gw=10.0.0.1", - "virtnet_dns=10.0.0.1", - "virtnet_ip6=fd00::2/64", - "virtnet_gw6=fd00::1", - ] { - assert!(openvmm::admits(token), "{token}"); - } - // The tokens of a root, a workload, a share, a listener, and anything else that a tenant's - // sandbox or another tier sets. - let [layer, upper]: [String; 2] = adapter(None).layout().tokens().try_into().unwrap(); - let [_, ram]: [String; 2] = adapter_with(gpt(), SandboxUpper::RAM, None) - .layout() - .tokens() - .try_into() - .unwrap(); - for token in [ - layer.as_str(), - upper.as_str(), - ram.as_str(), - "nvx_overlay_mode=untrusted-lowers", - "nvx_scratch_journal=forbidden", - "nvx_first_partition_bound=4096", - "nvx_ram_upper_percent=50", - "nvx_hostname=tenant-a", - "hostname=tenant-a", - "nvx_entrypoint=/bin/sh", - "nvx_arg=-c", - "nvx_memory_max=1073741824", - "nvx_maps=1", - "nvx_workload_account=create", - "nvx_workload_uid=1000", - "nvx_workload_gid=1000", - "nvx_lifecycle=managed", - "virtfs_dir=0", - "virtnet_ingress=1", - "nvx_proxy=http://10.0.0.1:3128", - "nvx_config=tenant-data", - "nvx_sandbox=0", - "nvx_snapshot_tier=workload-start", - "console=ttyS0", - "quiet", - "", - ] { - assert!(!openvmm::admits(token), "{token}"); - } -} - -#[test] -fn the_command_line_holds_only_tokens_of_openvmms_platform_allowlist() { - for template in every_template() { - let arguments = template.openvmm_arguments().unwrap(); - let command_line = template.kernel_command_line(); - let mut tokens: Vec = command_line.split(' ').map(str::to_owned).collect(); - // The tokens that OpenVMM writes for the template's arguments. - tokens.push(format!("nr_cpus={}", value(&arguments, "--processors"))); - tokens.push(format!( - "nvx_snapshot_tier={}", - value(&arguments, "--snapshot-tier") - )); - // Under the NIC's complete deny-all policy, OpenVMM writes no DNS server and no - // published listeners. - if arguments.iter().any(|argument| argument == "--net") { - let guest_network = value(&arguments, "--net"); - let (address, _) = guest_network.split_once('/').unwrap(); - tokens.push(format!("virtnet_ip={address}")); - tokens.push(format!( - "virtnet_gw={}", - crate::net::guest_gateway(guest_network).unwrap() - )); - } - for token in &tokens { - assert!(openvmm::admits(token), "{token} of {template:?}"); - // OpenVMM splits the command line at ASCII whitespace, and the kernel at its own, - // which also holds the vertical tab and 0xa0: no token holds either, so both split - // the command line into the same tokens. - assert!( - token.is_ascii() && !token.bytes().any(cmdline::is_space), - "{token:?} of {template:?}" - ); - } - // The command line of the template's devices names the root of its plan, which a clone - // binds, and which the allowlist refuses: the template's replaces it. - let root = template.plan().kernel_command_line("").unwrap(); - for token in root.split(' ') { - assert!(!openvmm::admits(token), "{token} of {root}"); - assert!(!command_line.contains(token), "{token} in {command_line}"); - } - } -} - -#[test] -fn templates_hold_no_tenant_data() { - for template in every_template() { - let arguments = template.openvmm_arguments().unwrap(); - let command_line = template.kernel_command_line(); - // Neither the identity of the image, nor the clone's upper, nor the proxy. - for text in [IDENTITY, "tenant", "3128", "/sandboxes", r"\sandboxes"] { - assert!(!command_line.contains(text), "{text} in {command_line}"); - } - for text in [IDENTITY, "tenant", "3128", "/sandboxes", r"\sandboxes"] { - assert!( - arguments.iter().all(|argument| !argument.contains(text)), - "{text} in {arguments:?}" - ); - } - // No kernel command line, workload identity, lifecycle, share, published listener, or - // other option besides those of the devices, the resources, and the capture, each with one - // value. - for pair in arguments.chunks(2) { - let [option, value] = pair else { - panic!("{arguments:?} ends in an option without a value"); - }; - let option = option.as_str(); - assert!(OPTIONS.contains(&option), "{option} in {arguments:?}"); - assert!(!value.starts_with("--"), "{option} {value}"); - if ["--network-egress", "--network-ingress", "--host-loopback"].contains(&option) { - assert_eq!(value, "deny", "{option} in {arguments:?}"); - } - } - let devices = template.plan().openvmm_arguments().unwrap(); - assert_eq!( - [ - devices, - template.nic_arguments().to_vec(), - template.capture_arguments() - ] - .concat(), - arguments - ); - } -} - -#[test] -fn templates_refuse_what_no_capture_takes() { - // A plan of the sandbox profile. - let profile = SandboxPlan::new( - SandboxDistro::new(PLACEHOLDER, erofs()), - SandboxUpper::scratch(CLONE_SCRATCH), - SandboxSettings::default(), - ) - .unwrap(); - assert_eq!( - refusal(Inputs::of(profile).lower()), - "a template lowers an adapter's plan, not a plan of the sandbox profile, whose workload \ - settings no template captures" - ); - - // A placeholder that is not the image that its binding names. - let other = "22222222-2222-2222-2222-222222222222"; - for (image, binding) in [ - (gpt(), binding()), - (RootImage::erofs(None), binding()), - (erofs(), binding_of("erofs", other, GIB)), - (erofs(), binding_of("gpt", IDENTITY, GIB)), - ] { - let expected = format!( - "the placeholder {PLACEHOLDER} is not the {} layer with identity {} that its distro \ - binding names", - binding.format(), - binding.identity() - ); - let inputs = Inputs { - adapter: adapter_with(image, SandboxUpper::RAM, None), - binding, - ..Inputs::default() - }; - assert_eq!(refusal(inputs.lower()), expected, "{image:?}"); - } - let inputs = Inputs { - binding: binding_of("erofs", "11111111-1111-1111-1111-11111111111A", GIB), - ..Inputs::default() - }; - assert_eq!( - refusal(inputs.lower()), - "the distro binding names the identity \"11111111-1111-1111-1111-11111111111A\", which \ - is not a UUID in its canonical text" - ); - - // Placeholders that OpenVMM would resolve against its own directory. - let mut relative = vec!["platform/placeholder.img"]; - if cfg!(windows) { - // Rooted on the current drive, and relative to a drive's current directory. - relative.extend([r"\platform\placeholder.img", r"C:platform\placeholder.img"]); - } - for path in relative { - let inputs = Inputs::of(adapter_at(path, erofs(), SandboxUpper::RAM, None)); - assert_eq!( - refusal(inputs.lower()), - format!( - "the placeholder of a template's capture must be absolute, since OpenVMM would \ - resolve it against the directory in which it runs: {}", - paths::normalized(Path::new(path)).display() - ) - ); - } - // Placeholders that OpenVMM's block device options would split, and that it would not attach - // as raw images. - for (name, problem) in [ - ( - "placeholder,1.img", - "containing commas, semicolons, or unbalanced brackets are unsupported", - ), - ( - "placeholder;1.img", - "containing commas, semicolons, or unbalanced brackets are unsupported", - ), - ( - "placeholder[1.img", - "containing commas, semicolons, or unbalanced brackets are unsupported", - ), - ( - "placeholder]1.img", - "containing commas, semicolons, or unbalanced brackets are unsupported", - ), - ( - "placeholder][1.img", - "containing commas, semicolons, or unbalanced brackets are unsupported", - ), - ( - "placeholder.vhd", - "ending in .vhd are unsupported, since OpenVMM would not attach them as raw images", - ), - ( - "placeholder.vhdx", - "ending in .vhdx are unsupported, since OpenVMM would not attach them as raw images", - ), - ( - "placeholder.vmgs", - "ending in .vmgs are unsupported, since OpenVMM would not attach them as raw images", - ), - ( - "placeholder.iso", - "ending in .iso are unsupported, since OpenVMM would not attach them as raw images", - ), - ] { - let path = Path::new(PLACEHOLDER).with_file_name(name); - let path = path.to_str().unwrap(); - let inputs = Inputs::of(adapter_at(path, erofs(), SandboxUpper::RAM, None)); - assert_eq!( - refusal(inputs.lower()), - format!("disk paths {problem}: {path}") - ); - } - - // Block devices that OpenVMM does not attach. - for length in [0, 511, 1000, GIB + 1] { - let placeholder = Inputs { - binding: binding_of("erofs", IDENTITY, length), - ..Inputs::default() - }; - assert_eq!( - refusal(placeholder.lower()), - format!( - "the placeholder {PLACEHOLDER} of {length} bytes cannot be attached: OpenVMM \ - attaches block devices of nonzero multiples of 512 bytes" - ) - ); - let scratch = Inputs { - scratch_length: length, - ..Inputs::default() - }; - assert_eq!( - refusal(scratch.lower()), - format!( - "a template's scratch of {length} bytes cannot be attached: OpenVMM attaches \ - block devices of nonzero multiples of 512 bytes" - ) - ); - } - - // Processor counts other than 1, 2, 4, and 8. - for processors in [0, 3, 6, 16, u32::MAX] { - let inputs = Inputs { - resources: resources(processors), - ..Inputs::default() - }; - assert_eq!( - refusal(inputs.lower()), - format!("a template's VM has 1, 2, 4, or 8 processors, not {processors}") - ); - } - // Processors online at boot other than 1, 2, 4, and 8, or more than the processors. - for (processors, boot_processors) in [(2, 0), (8, 3), (2, 4), (8, 16), (8, u32::MAX)] { - let inputs = Inputs { - resources: TemplateResources { - boot_processors, - ..resources(processors) - }, - ..Inputs::default() - }; - assert_eq!( - refusal(inputs.lower()), - format!( - "a template's VM boots with 1, 2, 4, or 8 of its {processors} processors online, \ - not {boot_processors}" - ) - ); - } - // RAM and capacities that are not whole blocks, and capacities below the RAM. - for (memory, memory_capacity) in [ - (0, 0), - (0, 2 * GIB), - (100 * MIB, 2 * GIB), - (512 * MIB, 2 * GIB + MIB), - (512 * MIB, 384 * MIB), - (512 * MIB, 0), - ] { - let inputs = Inputs { - resources: TemplateResources { - processors: 2, - boot_processors: 2, - memory, - memory_capacity, - }, - ..Inputs::default() - }; - assert_eq!( - refusal(inputs.lower()), - format!( - "a template's VM needs nonzero RAM, and a RAM capacity of at least it, each a \ - multiple of 128 MiB, not {memory} bytes of RAM and a capacity of \ - {memory_capacity} bytes" - ) - ); - } - - // Transaction directories that OpenVMM would resolve against its own directory. - let mut relative = vec!["templates/tx-1", "."]; - if cfg!(windows) { - // Rooted on the current drive, and relative to a drive's current directory. - relative.extend([r"\templates\tx-1", r"C:templates\tx-1"]); - } - for transaction in relative { - let inputs = Inputs { - transaction: transaction.to_owned(), - ..Inputs::default() - }; - assert_eq!( - refusal(inputs.lower()), - format!( - "the transaction directory of a template's capture must be absolute, since \ - OpenVMM would resolve it against the directory in which it runs: {}", - paths::normalized(Path::new(transaction)).display() - ) - ); - } - // Transaction directories whose scratch device OpenVMM's options would split. - for name in ["a,b", "a;b", "a[b", "a]b", "a][b"] { - let inputs = Inputs { - transaction: format!("{TRANSACTION}-{name}"), - ..Inputs::default() - }; - assert_eq!( - refusal(inputs.lower()), - format!( - "disk paths containing commas, semicolons, or unbalanced brackets are unsupported: \ - {}", - Path::new(&inputs.transaction).join(SCRATCH_NAME).display() - ) - ); - } -} - -/// Lowers the default inputs with the transaction directory `transaction`. -fn lower_in(transaction: &Path) -> Result { - let inputs = Inputs::default(); - TemplatePlan::new( - &inputs.adapter, - &inputs.binding, - inputs.scratch_length, - inputs.resources, - transaction, - generation(), - ) -} - -#[cfg(unix)] -#[test] -fn transaction_directories_must_be_valid_unicode() { - use std::ffi::OsStr; - use std::os::unix::ffi::OsStrExt; - - let transaction = Path::new(OsStr::from_bytes(b"/templates/tx-\xff")); - assert_eq!( - refusal(lower_in(transaction)), - "the transaction directory of a template's capture is not valid Unicode: \ - /templates/tx-\u{fffd}" - ); -} - -#[cfg(windows)] -#[test] -fn transaction_directories_must_be_valid_unicode() { - use std::ffi::OsString; - use std::os::windows::ffi::OsStringExt; - use std::path::PathBuf; - - let mut units: Vec = r"C:\templates\tx-".encode_utf16().collect(); - units.push(0xd800); - let transaction = PathBuf::from(OsString::from_wide(&units)); - assert_eq!( - refusal(lower_in(&transaction)), - "the transaction directory of a template's capture is not valid Unicode: \ - C:\\templates\\tx-\u{fffd}" - ); -} - -/// A change to the inputs of a lowering. -type Fix = fn(&mut Inputs); - -#[test] -fn templates_check_their_inputs_in_order() { - // With every input refused, each fix in turn reveals the next refusal. - let mut inputs = Inputs { - adapter: SandboxPlan::new( - SandboxDistro::new(PLACEHOLDER, gpt()), - SandboxUpper::RAM, - SandboxSettings::default(), - ) - .unwrap() - .with_network(Some(network(listening()))), - binding: binding_of("erofs", IDENTITY, 1000), - scratch_length: 0, - resources: TemplateResources { - processors: 3, - boot_processors: 3, - memory: 0, - memory_capacity: 0, - }, - transaction: "templates".to_owned(), - }; - let steps: [(&str, Fix); 11] = [ - ("a template lowers an adapter's plan", |inputs| { - inputs.adapter = adapter_at( - "placeholder.vhd", - gpt(), - SandboxUpper::RAM, - Some(network(listening())), - ); - }), - ("is not the erofs layer", |inputs| { - inputs.adapter = adapter_at( - "placeholder.vhd", - erofs(), - SandboxUpper::RAM, - Some(network(listening())), - ); - }), - ( - "the placeholder of a template's capture must be absolute", - |inputs| { - inputs.adapter = adapter_at( - &PLACEHOLDER.replace(".img", ".vhd"), - erofs(), - SandboxUpper::RAM, - Some(network(listening())), - ); - }, - ), - ("disk paths ending in .vhd", |inputs| { - inputs.adapter = adapter(Some(network(listening()))); - }), - ("of 1000 bytes cannot be attached", |inputs| { - inputs.binding = binding(); - }), - ("a template's scratch of 0 bytes", |inputs| { - inputs.scratch_length = 512; - }), - ("processors, not 3", |inputs| { - inputs.resources.processors = 2 - }), - ("of its 2 processors online, not 3", |inputs| { - inputs.resources.boot_processors = 1 - }), - ("needs nonzero RAM", |inputs| { - inputs.resources = resources(2) - }), - ("must be absolute", |inputs| { - inputs.transaction = format!("{TRANSACTION},1"); - }), - ("commas, semicolons, or unbalanced brackets", |inputs| { - inputs.transaction = TRANSACTION.to_owned() - }), - ]; - for (refused, fix) in steps { - let message = refusal(inputs.lower()); - assert!(message.contains(refused), "{refused}: {message}"); - fix(&mut inputs); - } - assert_eq!(inputs.lower().unwrap().scratch_length(), 512); -} diff --git a/nvx-rs/src/host/template/store.rs b/nvx-rs/src/host/template/store.rs deleted file mode 100644 index 46e7a1f98..000000000 --- a/nvx-rs/src/host/template/store.rs +++ /dev/null @@ -1,1116 +0,0 @@ -//! The store of templates, [`Store`]: its directory, its records, the namespaces of its -//! generations, the captures that publish them, and their leases, tombstones, and retirement. - -mod capture; -mod leases; -#[cfg(test)] -pub(super) mod tests; - -use std::fmt; -use std::fs::{self, File, TryLockError}; -use std::io; -use std::num::NonZeroU64; -use std::path::{Path, PathBuf}; -use std::str::FromStr; -use std::thread; -use std::time::{Duration, Instant, SystemTime}; - -use serde::{Deserialize, Serialize}; - -use super::key::{ClassIdentity, KeyIdentity, TemplateKey}; -use super::manifest::{HostClock, TimeContract}; -use crate::activation::StoreGeneration; -use crate::error::{Error, ErrorCode, ErrorKind, Result}; -use crate::host::platform::{self, parse_hex_id}; -use crate::host::process::BootId; -use crate::host::store::files::{open_lock, write_private}; -use crate::host::store::lease::{ - ID_LEN, StoreRecord, hex, read_identity, read_private, read_record, store_error, -}; - -pub use self::capture::{CAPTURE_RECORD_NAME, CAPTURES_NAME, Capture, Created, TRANSACTION_NAME}; -pub use self::leases::{Acquired, LEASES_NAME, Retirement, TOMBSTONES_NAME}; -pub use crate::host::store::lease::{ - GENERATIONS_NAME, Lease, LeaseId, STORE_FORMAT, STORE_RECORD_NAME, StoreId, -}; - -/// Name of the store's index. -pub const INDEX_NAME: &str = "index.json"; -/// Name of the lock of the store's index. -pub const INDEX_LOCK_NAME: &str = "index.lock"; -/// Name of the canonical encoding of a generation's key in its payload. -pub const KEY_NAME: &str = "key.bin"; -/// Name of a generation's snapshot in its payload. -pub const SNAPSHOT_NAME: &str = "snapshot"; -/// Most generations that a store holds. -pub const MAX_GENERATIONS: usize = 1024; - -/// The store generation that a new store assigns first. -const FIRST_GENERATION: StoreGeneration = StoreGeneration::new(NonZeroU64::MIN); - -/// Longest index, in bytes: more than an index of [`MAX_GENERATIONS`] generations takes. -const MAX_INDEX_LEN: u64 = 4 * 1024 * 1024; - -/// Interval at which a lock that another holder has is tried again, by a caller that waits for it -/// until a deadline. -const LOCK_RETRY_INTERVAL: Duration = Duration::from_millis(25); - -/// Draws 16 random bytes, for `what`. -fn random_id(what: &str) -> Result<[u8; ID_LEN]> { - let mut bytes = [0; ID_LEN]; - getrandom::fill(&mut bytes) - .map_err(|error| Error::other(format!("cannot draw {what}")).with_source(error))?; - Ok(bytes) -} - -/// The bound-block storage generation of a capture: 16 bytes, not all zero, that a capture gives -/// OpenVMM for every block that it binds, which the storage owner keeps immutable for those -/// blocks, so that neither the capture nor a restore reads a block to identify it. -/// -/// It belongs to the storage owner, and is neither a [store generation](StoreGeneration), nor a -/// time lineage counter, nor a VM generation ID; see -/// [the four namespaces](Store#four-namespaces). Generations may share one. Its text form, which -/// `Display` writes and `FromStr` parses, is 32 lowercase hexadecimal digits, as OpenVMM's -/// `--snapshot-generation-id` takes it; parsing also accepts uppercase ones. -#[derive(Clone, Copy, PartialEq, Eq, Hash)] -pub struct StorageGeneration([u8; ID_LEN]); - -impl StorageGeneration { - /// Draws a fresh storage generation at random. - pub fn generate() -> Result { - loop { - if let Some(generation) = Self::from_bytes(random_id("a storage generation")?) { - return Ok(generation); - } - } - } - - /// Returns the storage generation whose bytes are `bytes`, unless they are all zero, which no - /// storage generation is. - pub fn from_bytes(bytes: [u8; ID_LEN]) -> Option { - (bytes != [0; ID_LEN]).then_some(Self(bytes)) - } - - /// Returns the bytes of the storage generation. - pub const fn as_bytes(&self) -> &[u8; ID_LEN] { - &self.0 - } -} - -impl fmt::Display for StorageGeneration { - fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { - formatter.write_str(&hex(&self.0)) - } -} - -impl fmt::Debug for StorageGeneration { - fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { - write!(formatter, "StorageGeneration({self})") - } -} - -impl FromStr for StorageGeneration { - type Err = Error; - - /// Parses the text form of a storage generation, failing with [`ErrorKind::Unsupported`] for - /// anything else, all zeros included. - fn from_str(text: &str) -> Result { - parse_hex_id(text) - .and_then(Self::from_bytes) - .ok_or_else(|| { - Error::unsupported(format!( - "{text:?} is not a storage generation, which is 32 hexadecimal digits, not \ - all zero" - )) - }) - } -} - -/// One generation of a template that a store holds, as its index lists it: its store generation, -/// the identities of its key and of its capture class, its storage generation, and what its -/// manifest records of the capture itself, in which the generations of one key differ. -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct Generation { - /// Its store generation. - generation: StoreGeneration, - /// The identity of its key. - key: KeyIdentity, - /// The identity of its capture class. - class: ClassIdentity, - /// The storage generation of its capture. - storage: StorageGeneration, - /// When OpenVMM created its snapshot. - created_at: SystemTime, - /// The time contract of its capture. - time: TimeContract, - /// The length of its saved state, in bytes. - state_len: u64, - /// The length of its RAM, in bytes. - memory_len: u64, -} - -impl Generation { - /// Returns the store generation, which names the generation in its store. - pub fn generation(&self) -> StoreGeneration { - self.generation - } - - /// Returns the identity of the generation's compatibility key, which - /// [`Store::read_key`] reads in full. - pub fn key(&self) -> KeyIdentity { - self.key - } - - /// Returns the identity of the class of the capture that made the generation. - pub fn class(&self) -> ClassIdentity { - self.class - } - - /// Returns the storage generation of the capture that made the generation. - pub fn storage_generation(&self) -> StorageGeneration { - self.storage - } - - /// Returns when OpenVMM created the generation's snapshot. - pub fn created_at(&self) -> SystemTime { - self.created_at - } - - /// Returns the time contract of the capture, with its anchor, host, boot, and time lineage - /// counter. - pub fn time(&self) -> &TimeContract { - &self.time - } - - /// Returns the length of the snapshot's saved state, in bytes. - pub fn state_len(&self) -> u64 { - self.state_len - } - - /// Returns the length of the snapshot's RAM, in bytes. - pub fn memory_len(&self) -> u64 { - self.memory_len - } -} - -/// The index of a store, as one read found it: its generations, in the order of their -/// publication, and the store generation that its next publication takes. -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct Index { - /// The store generation that the next publication takes. - next: StoreGeneration, - /// The generations, in the order of their store generations. - generations: Vec, -} - -impl Index { - /// Returns the generations, in the order of their publication. - pub fn generations(&self) -> &[Generation] { - &self.generations - } - - /// Returns the generation of store generation `generation`, if the index lists it. - pub fn get(&self, generation: StoreGeneration) -> Option<&Generation> { - self.generations - .binary_search_by_key(&generation, Generation::generation) - .ok() - .map(|index| &self.generations[index]) - } - - /// Returns the generations whose key has the identity `key`, in the order of their - /// publication. - pub fn of_key(&self, key: KeyIdentity) -> impl Iterator { - self.generations - .iter() - .filter(move |generation| generation.key == key) - } - - /// Returns the store generation that the store's next publication takes: one past every store - /// generation that it ever assigned. - pub fn next_generation(&self) -> StoreGeneration { - self.next - } -} - -/// A store of templates: a directory that the adapter chooses, apart from every sandbox's -/// store, with the generations of templates that its host published, and their index. -/// -/// ```text -/// / the store, private to the current user -/// store.json the store's identity record, which its creation writes last -/// index.json the index of the generations that the store holds -/// index.lock the lock that serializes the transactions of the index -/// generations/ the payloads of the generations, which never change once indexed -/// / the payload of the store generation G -/// capture.json the record of the capture that made it -/// key.bin the canonical encoding of its compatibility key -/// snapshot/ its snapshot, as OpenVMM published it -/// .sock on Linux, the control socket of the VMM of a clone that leases it, -/// named after the clone's launch, until the lease's release -/// leases/ the records of the leases that the store granted -/// .json the record of the lease L -/// tombstones/ the tombstones of the generations that take no new leases -/// .json the tombstone of the store generation G -/// captures/ the slots of the capture classes -/// / the slot of the classes whose identities start with the digits C, -/// a sandbox store whose lock serializes the classes' captures -/// openvmm.log the log of the VMM of the slot's latest capture -/// outcome.json the outcome report of the VMM of the slot's latest capture -/// transaction/ the capture in progress: the sandbox store of its VM until the VM has -/// ended, and then the payload of a generation -/// capture.json the record of the capture -/// scratch.raw the dormant scratch device, which the capture removes once checked -/// snapshot/ the snapshot, once OpenVMM has published it -/// key.bin the template's key, once the capture has checked the snapshot -/// ``` -/// -/// # Identity -/// -/// A store has an identity of its own, a [`StoreId`] that its creation draws, which its identity -/// record and its index both hold, so that the index of another store is refused. -/// [`Store::create`] creates a store, or opens the one that its directory already holds, and -/// [`Store::open`] opens an existing one. -/// -/// # The index -/// -/// The index lists the store's generations, in the order of their publication, with the metadata -/// of each that restores and their eligibility check without reading its payload: the identities -/// of its key and of its capture class, its storage generation, and what its manifest records of -/// the capture itself, its creation time, its time contract, and the lengths of its artifacts. -/// The payload of a generation never changes once the index lists it: the record of its capture, -/// the canonical encoding of its key, which [`Store::read_key`] reads, and its snapshot, which its -/// clones restore in place. Only the control sockets of the VMMs of the clones that lease it come -/// and go beside them, on Linux, as [`Store::acquire`] describes. The index -/// changes as generations are published and retired, and the records of leases and tombstones as -/// clones lease generations, and as generations are quarantined and retired. -/// -/// The store's records, its identity record, its index, and the records of its leases and -/// tombstones, are JSON in format [`STORE_FORMAT`]. -/// A record in any other format fails with a message that names its format, and is neither read -/// nor migrated: a store of another format is replaced by a new one, whose templates are captured -/// again. Records are replaced atomically: written to a temporary file beside them, flushed to -/// disk, and renamed over them. -/// -/// # Four namespaces -/// -/// Four numbers name generations of different things, and the store keeps them apart: -/// -/// - The store generation, [`StoreGeneration`], is the host's publication ID: the store assigns -/// one to each generation as it indexes it, from 1 on, in the order of publication, and never -/// assigns one twice, even once its generation is gone. A clone's lease and durable records name -/// the generation that it restores by its store generation and the identity of its key, in their -/// [`TemplateReference`](crate::activation::TemplateReference). -/// - The storage generation, [`StorageGeneration`], is the bound-block storage generation that a -/// capture gives OpenVMM, which the storage owner keeps immutable for the blocks that it binds. -/// It belongs to the storage owner, not to the store, and generations may share one. -/// - The time lineage counter, [`TimeContract::lineage`], is the generation counter of the -/// captured VM process, which each restore of a snapshot increments, so that the clones of one -/// generation share it. -/// - The VM generation ID is 16 random bytes that each VM process draws, which tell the clones of -/// one generation apart. Their guests read it, and the store records none. -/// -/// # Captures and publication -/// -/// [`Store::create_template`] and [`Store::refresh_template`] capture templates of a -/// [`CaptureClass`](crate::host::template::CaptureClass), and publish them as the store's -/// generations: a creation serves the class with a generation that the store holds, if one is -/// eligible, and captures one otherwise, while a refresh always captures. Each class has a slot in -/// `captures/`, a sandbox store named by the first 16 hexadecimal digits of the class's identity, -/// whose lock serializes the captures of the class, and which keeps the log and the outcome report -/// of the VMM of the class's latest capture. A capture holds its slot's lock until its VM has -/// ended, and takes the store's lock under it to publish, as the supervisor's -/// [lock order](crate::host::supervisor#the-lock-order) allows. -/// -/// A capture runs in a transaction, a directory private to the current user, in its slot, on the -/// store's volume. It writes the record of the capture there, `capture.json`, which names the -/// store, the class, and the [`StorageGeneration`] that it drew, and creates the template's -/// dormant scratch device, a sparse file, then lowers the adapter's plan to a -/// [`TemplatePlan`](crate::host::template::TemplatePlan), whose snapshot destination is in the -/// transaction. The transaction is also the sandbox store of the capture's VM, whose launch -/// journal records its VMM: OpenVMM requires the control socket of a VM that captures a snapshot, -/// on Linux, beside the snapshot's destination, and a sandbox store's VM listens in the store's -/// directory. The capture holds the transaction's lock, under its slot's, until the VM has ended, -/// and releases it before it publishes. It cold-boots the template's VM through -/// [`supervisor::launch`](crate::host::supervisor::launch), from the transaction, as a launch of -/// its own, and waits for the VMM to exit: the guest captures itself, and OpenVMM publishes the -/// snapshot at its destination, and exits. The capture then moves the VMM's log and outcome -/// report to its slot. -/// -/// ## Block devices by path -/// -/// OpenVMM opens the block devices of a VM whose snapshot it captures, or restores, itself, at the -/// paths that its command line names: without following a link, nor on Windows any other reparse -/// point, and sharing only reads and deletion, so that its open fails while another handle may -/// write the file. The host therefore hands OpenVMM the paths of these files, never handles, and -/// checks the file at each path just before the VM starts: -/// -/// - The directory that holds the VM's files, the capture's transaction, must be private to the -/// current user. -/// - The placeholder must pass its binding's check, -/// [`DistroBinding::check`](crate::host::sandbox::distro::DistroBinding::check), of its -/// identity and length, and then, at its path, be a regular file, not a link nor any other -/// reparse point, that is the file that the check found, that belongs to the current user or to -/// an administrator, and that no one else may change. -/// - The scratch must be, at its path, the regular file of the template's length that the capture -/// created, which no one else may change. -/// -/// The host tells these files apart by the identity that their seals record, -/// [`FileSeal`](crate::host::seal::FileSeal): their device or volume and their file ID, on Windows -/// the 128-bit ID of `FILE_ID_INFO`, which a file system such as ReFS keeps unique where the -/// 64-bit file index is not. -/// -/// As the VM starts, the host holds no handle through which it could write either file: it closes -/// the scratch once it has checked it, and holds only the placeholder that its binding's check -/// opened for reading, which on Windows lets no one write it, until the VM has ended. The -/// directories above the files must be ones that only the current user, or an administrator, may -/// change, since OpenVMM resolves each path again. A restore of a clone, whose block devices -/// OpenVMM also opens by their paths, takes the same rule; a restore of a clone in RAM declares its -/// scratch absent, which OpenVMM attaches with no file. -/// -/// ## What the VM leaves -/// -/// Once the VMM has exited, and before anything is published, the capture checks that the VM's -/// outcome report records the capture's success, and that the snapshot is a directory private to -/// the current user, as OpenVMM creates it, whose manifest -/// [`TemplateManifest::read`](crate::host::template::TemplateManifest::read) reads, whose artifacts -/// are exactly its manifest, RAM, and saved state, each a regular file private to the current -/// user, of the length that the manifest records, and whose key, [`TemplateKey::new`], is its -/// class's. It then closes the lock of the transaction's sandbox store, removes the lock's file, -/// the scratch, and what OpenVMM left beside the snapshot, and writes the key, `key.bin`, which -/// marks the transaction as checked, durably. -/// -/// ## Two commits -/// -/// A capture publishes its checked transaction under the store's lock, in two commits: it renames -/// the transaction to the payload of the index's next store generation, which no rename -/// replaces, and makes the rename durable; then it writes the index, which lists the generation, -/// and makes it durable, with the store's own entry in its parent, which the store's creation -/// does not sync. A payload at the index's next store generation is one whose publication a crash -/// interrupted, which the next publication, or recovery, indexes first, once it has made the -/// rename durable as the publication would have. Each capture completes these publications, and -/// first drops from the index the generations whose payloads a crash lost, which then take none -/// of the store's capacity, before it looks for a generation or counts the store's. A creation -/// that serves a generation that the store holds makes the index, and the store's own entry, -/// durable before it returns it, since the commit that listed it may have been interrupted. -/// Publications leave the older generations, and their leases, untouched. -/// -/// A capture that fails before it checks its transaction publishes nothing, and retires the -/// transaction once its VMM has ended. One that a crash of the host interrupts is ended by the -/// next capture of its class, or by [`Store::recover_captures`], which an adapter calls as it -/// starts: each ends the launch that the transaction's sandbox store records, as -/// [`LockedStore::recover`](crate::host::store::LockedStore::recover) ends a sandbox's, and then -/// publishes the transaction if the capture had checked it, once it has synced the transaction -/// again, and retires it otherwise, once it has moved the VMM's log and outcome report to the -/// slot. -/// -/// # Leases, tombstones, and retirement -/// -/// A clone that restores a generation leases it first, so that the generation stays in the store -/// until the clone's VM has exited. [`Store::select`] selects the generation of a key from which -/// a clone restores: the latest that is neither quarantined nor ineligible at the host's time, -/// as [`Generation::eligibility`] decides it. [`Store::acquire`] leases a generation in one -/// transaction, under the store's lock and then the sandbox's: it checks the generation's -/// eligibility, and writes the record of the lease in the store and the clone binding, -/// [`CLONE_BINDING_NAME`](crate::host::store::CLONE_BINDING_NAME), beside the sandbox's launch -/// journal, before any VMM of the clone is spawned. It returns the lease with the sandbox's lock, -/// under which the caller launches the clone, and holds no lock of the store for the VM's -/// lifetime. The lease reserves one launch of the clone, [`Lease::launch`], which only that lock -/// begins, with -/// [`LockedStore::begin_launch`](crate::host::store::LockedStore::begin_launch), and the sandbox -/// launches nothing else while the lease lasts; [the supervisor](crate::host::supervisor#clones) -/// describes the launch of a clone, its crashes, and the order of the locks that it takes. A lease -/// never expires: it lasts until [`Store::release`] ends it, which it does only -/// once the sandbox records no VM and no launch, and so once its VM has exited, as its store -/// verifies through the VM's recorded process, also after a reattach. A caller that reads a -/// generation's payload, as a restore does, holds a lease of it. -/// -/// On Linux, a payload holds the control socket of the VMM of each clone that leases it, -/// `.sock`, named after the launch that the lease reserves, beside the snapshot that the -/// clone restores, and of no clone whose lease has ended: the release makes the socket's absence -/// durable before it ends the lease. So a payload holds entries other than its capture's while -/// its generation has leases, which nothing that reads an indexed payload may refuse, and a -/// retirement deletes the payload with any socket that a crash, or a removal of a sandbox that -/// never released its lease, left in it. -/// -/// A tombstone, which [`Store::tombstone`] writes, quarantines a generation: it takes no new -/// lease, and selection skips it, while the leases that it granted keep working. -/// [`Store::retire`] tombstones a generation, and deletes it once its leases have drained: its -/// payload, then its entry in the index, and then its tombstone, so that an interrupted -/// retirement leaves a quarantined generation, which retiring it again deletes. A release, or a -/// removal of a sandbox, that is interrupted leaves the record of a lease that no clone binding -/// names, which a retirement removes. -/// -/// These transactions make each step durable before the next, so that their order holds through -/// a crash of the host too, even when a sandbox and the store are on different filesystems: an -/// acquisition makes the record of a lease durable before it writes the clone binding, and the -/// binding before it returns, and a release, or a retirement that finds a binding gone, makes the -/// binding's absence durable before it removes the record, as a -/// [removal](crate::host::store#removal) of a sandbox does before the sandbox goes, so that no -/// binding outlasts the record that keeps its generation; a retirement makes the tombstone durable -/// before it deletes anything, and each deletion before the next. On Windows, which has no -/// portable way to flush a directory, a rename or a removal is as durable as the filesystem makes -/// it, as OpenVMM's publication of a snapshot assumes. -/// -/// # Privacy -/// -/// The store's directories are private to the current user, the orchestrator's account, so that -/// no one else may write its records and payloads: each belongs to the user, and no one else may -/// access it. On Unix, a directory grants the group and others no permission, and those that the -/// store creates have mode 0700. On Windows, a directory that the store creates allows the user -/// alone, through a protected access control list that its files inherit; one that it did not -/// create may also allow SYSTEM and the Administrators group, as one under the user's profile -/// does, and must pass entries on to the files created in it. A snapshot's directory, which -/// OpenVMM creates, need not: OpenVMM gives it, and each file that it publishes in it, a protected -/// list of their own, which allows SYSTEM and the owner, and the directory's list already stops -/// anyone else from creating a file in it. The store refuses a record whose -/// own list allows anyone else. A directory or a file of the store that is a link, or on Windows -/// any other reparse point, is refused, and so is a record that is not a regular file. The store -/// names its files by their paths, so the directories above it must be ones that only the user, -/// or an administrator, may change. -/// -/// # Example -/// -/// ```no_run -/// use nvx::host::template::Store; -/// -/// # fn example() -> nvx::Result<()> { -/// let store = Store::create("/var/lib/example/templates")?; -/// for generation in store.index()?.generations() { -/// let key = store.read_key(generation)?; -/// println!( -/// "generation {} of key {} on {}", -/// generation.generation(), -/// generation.key(), -/// key.hypervisor() -/// ); -/// } -/// # Ok(()) -/// # } -/// ``` -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct Store { - /// The store's directory, as an absolute path. - dir: PathBuf, - /// The store's identity. - id: StoreId, -} - -impl Store { - /// Creates the directory `dir` if needed, whose parent must exist, private to the current - /// user, and a store in it, or opens the store that it holds. - /// - /// A directory that holds no store must hold nothing but what an interrupted creation leaves: - /// one with any other entry fails with [`ErrorCode::ForeignFiles`]. The creation writes the - /// store's identity record last, so a store whose creation is interrupted is none, and a - /// later creation replaces it. Creations of one store take its lock, so that they make one - /// store between them. An existing directory must already be private, as [`open`](Self::open) - /// requires: one that others could access is refused rather than restricted, since they could - /// hold its files open. - /// - /// A relative `dir` is resolved against the current directory. Fails with - /// [`ErrorKind::NotFound`] if the parent does not exist. - pub fn create(dir: impl Into) -> Result { - Self::create_racing(absolute(dir.into())?, || {}) - } - - /// Creates the store as [`create`](Self::create) does, in the absolute directory `dir`, and - /// runs `raced` once it has found no store there, before it checks the directory's entries, - /// where a concurrent creation may make one. - fn create_racing(dir: PathBuf, raced: impl FnOnce()) -> Result { - let problem = platform::create_owner_only_dir(&dir).map_err(store_error(format!( - "cannot create the template store {}", - dir.display() - )))?; - refuse_problem(&dir, "the template store", problem)?; - // A directory of anything else is refused before its lock file is created, unless a - // concurrent creation made a store in it meanwhile, and again under the lock. - if read_identity(&dir)?.is_none() { - raced(); - if let Err(error) = refuse_foreign_entries(&dir) - && read_identity(&dir)?.is_none() - { - return Err(error); - } - } - let lock = take_lock(&dir, None)?; - if let Some(id) = read_identity(&dir)? { - let store = Self { dir, id }; - store.check_generations_dir()?; - return Ok(store); - } - refuse_foreign_entries(&dir)?; - let generations = dir.join(GENERATIONS_NAME); - let problem = platform::create_owner_only_dir(&generations).map_err(store_error( - format!("cannot create the template store {}", dir.display()), - ))?; - refuse_problem( - &generations, - "the generations of the template store", - problem, - )?; - let store = Self { - dir, - id: StoreId::from_bytes(random_id("a template store's identity")?), - }; - store.write_index(&Index { - next: FIRST_GENERATION, - generations: Vec::new(), - })?; - write_record( - &store.dir.join(STORE_RECORD_NAME), - &StoreRecord { - format: STORE_FORMAT, - store: store.id.to_string(), - }, - "the identity record of the template store", - )?; - drop(lock); - Ok(store) - } - - /// Opens the store in the existing directory `dir`, which a relative path names from the - /// current directory, and which must be private to the current user, as - /// [`create`](Self::create) makes it. - /// - /// Fails with [`ErrorKind::NotFound`] if `dir` does not exist, or holds no store, and with - /// [`ErrorCode::FormatUnsupported`] if it holds a store of another format. - pub fn open(dir: impl Into) -> Result { - let dir = absolute(dir.into())?; - let problem = platform::check_owner_only_dir(&dir).map_err(store_error(format!( - "cannot open the template store {}", - dir.display() - )))?; - refuse_problem(&dir, "the template store", problem)?; - let Some(id) = read_identity(&dir)? else { - return Err(Error::new( - ErrorKind::NotFound, - format!("the directory {} holds no template store", dir.display()), - )); - }; - let store = Self { dir, id }; - store.check_generations_dir()?; - Ok(store) - } - - /// Returns the store's directory, as an absolute path. - pub fn dir(&self) -> &Path { - &self.dir - } - - /// Returns the store's identity. - pub fn id(&self) -> StoreId { - self.id - } - - /// Reads the store's index. - /// - /// A read that holds no lock sees one whole index, as the latest transaction left it. - /// Fails with [`ErrorCode::FormatUnsupported`] for an index of another format, and with - /// [`ErrorKind::Io`] for one that cannot be read, is malformed, or belongs to another store. - pub fn index(&self) -> Result { - let path = self.dir.join(INDEX_NAME); - let description = "index of the template store"; - let Some(record) = read_record::(&path, description, MAX_INDEX_LEN)? else { - return Err(Error::io(format!( - "the template store {} has no index", - self.dir.display() - ))); - }; - let malformed = |problem: String| { - Error::io(format!( - "the {description} {} is malformed: {problem}", - path.display() - )) - }; - if record.store != self.id.to_string() { - return Err(Error::io(format!( - "the {description} {} belongs to the template store {}, not to this one, {}", - path.display(), - record.store, - self.id - ))); - } - record.into_index().map_err(malformed) - } - - /// Returns the path of the payload of the store generation `generation`. - pub fn payload_dir(&self, generation: StoreGeneration) -> PathBuf { - self.dir.join(GENERATIONS_NAME).join(generation.to_string()) - } - - /// Checks the payload of `generation`, and returns the path of its snapshot, which OpenVMM - /// restores. - /// - /// The directories of the generations, of the payload, and of the snapshot must each be - /// private to the current user, and none of them a link, or on Windows any other reparse - /// point. The snapshot's directory is the one that OpenVMM published, which need not pass - /// access control entries on to the files created in it; see [Privacy](Store#privacy). Fails - /// with [`ErrorKind::NotFound`] if the payload or its snapshot is missing. - pub fn snapshot_dir(&self, generation: &Generation) -> Result { - let payload = self.payload_dir(generation.generation); - let snapshot = payload.join(SNAPSHOT_NAME); - self.check_generations_dir()?; - for (path, what, check) in [ - ( - &payload, - "the payload of the generation", - platform::check_owner_only_dir as fn(&Path) -> io::Result>, - ), - ( - &snapshot, - "the snapshot of the generation", - platform::check_snapshot_dir, - ), - ] { - let problem = check(path).map_err(store_error(format!( - "cannot open {what} {}", - path.display() - )))?; - refuse_problem(path, what, problem)?; - } - Ok(snapshot) - } - - /// Reads the compatibility key of `generation` from its payload. - /// - /// The key is read as it was written when the generation was indexed, without computing its - /// identity again. Fails with [`ErrorKind::NotFound`] if the payload or its key is missing, - /// and as [`TemplateKey::from_bytes`] does for a key that is not the canonical encoding of - /// one. - pub fn read_key(&self, generation: &Generation) -> Result { - let payload = self.payload_dir(generation.generation); - self.check_generations_dir()?; - let what = "the payload of the generation"; - let problem = platform::check_owner_only_dir(&payload).map_err(store_error(format!( - "cannot open {what} {}", - payload.display() - )))?; - refuse_problem(&payload, what, problem)?; - let path = payload.join(KEY_NAME); - let description = "template key"; - let bytes = - read_private(&path, description, TemplateKey::MAX_LEN as u64)?.ok_or_else(|| { - Error::new( - ErrorKind::NotFound, - format!("the {description} {} is missing", path.display()), - ) - })?; - TemplateKey::from_bytes(&bytes).map_err(|error| { - Error::new( - error.code(), - format!("cannot read {}: {}", path.display(), error.message()), - ) - }) - } - - /// Checks that the directory of the generations is private to the current user, and not a - /// link, as every read of a generation does, so that a clone's plan tells a store whose - /// directory fails it, the host's, from a generation that is malformed. - pub(super) fn check_generations_dir(&self) -> Result<()> { - let path = self.dir.join(GENERATIONS_NAME); - let what = "the generations of the template store"; - let problem = platform::check_owner_only_dir(&path).map_err(store_error(format!( - "cannot open {what} {}", - path.display() - )))?; - refuse_problem(&path, what, problem) - } - - /// Atomically replaces the store's index with `index`. - fn write_index(&self, index: &Index) -> Result<()> { - write_record( - &self.dir.join(INDEX_NAME), - &IndexRecord::new(self.id, index), - "the index of the template store", - ) - } -} - -/// Returns `dir` as an absolute path, which a relative one names from the current directory, -/// without trailing separators: Linux follows a link at a name that a separator ends, even -/// without following links at the final name, so the checks of the store's directory refuse a -/// link at its own name only without them. -fn absolute(dir: PathBuf) -> Result { - std::path::absolute(&dir) - .map(|absolute| absolute.components().collect()) - .map_err(|error| { - Error::io(format!( - "cannot resolve the template store {}", - dir.display() - )) - .with_source(error) - }) -} - -/// Fails unless `problem`, why the directory `path`, which `what` names, is not private to the -/// current user, is `None`. -fn refuse_problem(path: &Path, what: &str, problem: Option) -> Result<()> { - match problem { - None => Ok(()), - Some(problem) => Err(Error::io(format!("{what} {} {problem}", path.display()))), - } -} - -/// Returns the names of the entries of the directory `dir`. -fn entries(dir: &Path) -> Result> { - let failed = || store_error(format!("cannot list {}", dir.display())); - fs::read_dir(dir) - .map_err(failed())? - .map(|entry| { - Ok(entry - .map_err(failed())? - .file_name() - .to_string_lossy() - .into_owned()) - }) - .collect() -} - -/// Returns whether `name` is an entry that a creation of a store makes before its identity record: -/// the index, its lock, the directory of the generations, and the temporary files that an -/// interrupted replacement of a record leaves, `.NAME.NUMBER.tmp`. -fn is_creation_leftover(name: &str) -> bool { - [INDEX_NAME, INDEX_LOCK_NAME, GENERATIONS_NAME].contains(&name) - || name - .strip_prefix('.') - .and_then(|name| name.strip_suffix(".tmp")) - .and_then(|name| name.rsplit_once('.')) - .is_some_and(|(file, number)| { - [STORE_RECORD_NAME, INDEX_NAME].contains(&file) - && !number.is_empty() - && number.bytes().all(|byte| byte.is_ascii_digit()) - }) -} - -/// Fails with [`ErrorCode::ForeignFiles`] if the directory `dir`, which holds no store, holds -/// anything but what an interrupted creation of one leaves: any other entry, or any in the -/// directory of the generations, which must be private to the current user, if it exists. -fn refuse_foreign_entries(dir: &Path) -> Result<()> { - let mut foreign = entries(dir)? - .into_iter() - .filter(|name| !is_creation_leftover(name)) - .collect::>(); - let generations = dir.join(GENERATIONS_NAME); - let what = "the generations of the template store"; - match platform::check_owner_only_dir(&generations) { - Err(error) if error.kind() == io::ErrorKind::NotFound => {} - checked => { - let problem = checked.map_err(store_error(format!( - "cannot open {what} {}", - generations.display() - )))?; - refuse_problem(&generations, what, problem)?; - foreign.extend( - entries(&generations)? - .into_iter() - .map(|name| format!("{GENERATIONS_NAME}/{name}")), - ); - } - } - if foreign.is_empty() { - return Ok(()); - } - foreign.sort(); - Err(Error::new( - ErrorCode::ForeignFiles, - format!( - "the directory {} holds no template store, but entries that are not a store's: {}", - dir.display(), - foreign.join(", ") - ), - )) -} - -/// Opens the lock file of the store in `dir`, which it creates if needed, takes its lock, waiting -/// for it until `deadline`, or for as long as it takes without one, and checks that the path -/// still names the locked file. -/// -/// The store never removes or replaces its lock file, so one that its path no longer names once -/// it is locked was removed or replaced by someone else, and its lock serializes nothing: the -/// lock is then taken again, after a pause, until the deadline. -fn take_lock(dir: &Path, deadline: Option) -> Result { - let path = dir.join(INDEX_LOCK_NAME); - let failed = |error| store_error(format!("cannot lock {}", path.display()))(error); - let busy = || { - Error::new( - ErrorKind::TimedOut, - format!( - "another transaction of the template store {} is still in progress", - dir.display() - ), - ) - }; - loop { - let file = open_lock(&path).map_err(failed)?; - if let Some(problem) = platform::private_file_problem(&file).map_err(failed)? { - return Err(Error::io(format!( - "the lock of the template store {} {problem}", - path.display() - ))); - } - match deadline { - None => file.lock().map_err(failed)?, - Some(deadline) => loop { - match file.try_lock() { - Ok(()) => break, - Err(TryLockError::WouldBlock) => { - let left = deadline.saturating_duration_since(Instant::now()); - if left.is_zero() { - return Err(busy()); - } - thread::sleep(left.min(LOCK_RETRY_INTERVAL)); - } - Err(TryLockError::Error(error)) => return Err(failed(error)), - } - }, - } - let locked = platform::opened_file_id(&file).map_err(failed)?; - match platform::file_id(&path) { - Ok(current) if current == locked => return Ok(file), - Ok(_) => {} - Err(error) if error.kind() == io::ErrorKind::NotFound => {} - Err(error) if cfg!(windows) && error.kind() == io::ErrorKind::PermissionDenied => {} - Err(error) => return Err(failed(error)), - } - drop(file); - let left = deadline.map(|deadline| deadline.saturating_duration_since(Instant::now())); - if left.is_some_and(|left| left.is_zero()) { - return Err(busy()); - } - thread::sleep(left.map_or(LOCK_RETRY_INTERVAL, |left| left.min(LOCK_RETRY_INTERVAL))); - } -} - -/// Atomically replaces the store's record at `path`, its `description`, with `record`, as pretty -/// JSON and a newline. -fn write_record(path: &Path, record: &T, description: &str) -> Result<()> { - let mut text = serde_json::to_vec_pretty(record) - .map_err(|error| Error::other(format!("cannot encode {description}")).with_source(error))?; - text.push(b'\n'); - write_private(path, &text) -} - -/// The index of a store, as JSON. -#[derive(Debug, Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -struct IndexRecord { - /// [`STORE_FORMAT`]. - format: u32, - /// The identity of the store. - store: String, - /// The store generation that the next publication takes. - next_generation: u64, - /// The generations, in the order of their store generations. - generations: Vec, -} - -impl IndexRecord { - /// Returns the record of `index`, of the store `store`. - fn new(store: StoreId, index: &Index) -> Self { - Self { - format: STORE_FORMAT, - store: store.to_string(), - next_generation: index.next.get().get(), - generations: index - .generations - .iter() - .map(GenerationRecord::new) - .collect(), - } - } - - /// Checks the record, and returns its index, or says why it is malformed. - fn into_index(self) -> std::result::Result { - let next = NonZeroU64::new(self.next_generation) - .map(StoreGeneration::new) - .ok_or("its next store generation is 0")?; - if self.generations.len() > MAX_GENERATIONS { - return Err(format!( - "it lists {} generations, more than the {MAX_GENERATIONS} that a store holds", - self.generations.len() - )); - } - let mut generations = Vec::with_capacity(self.generations.len()); - for record in self.generations { - let generation = record.into_generation()?; - if generations - .last() - .is_some_and(|last: &Generation| last.generation >= generation.generation) - { - return Err(format!( - "it lists store generation {} out of order, or twice", - generation.generation - )); - } - if generation.generation >= next { - return Err(format!( - "it lists store generation {}, which its next store generation, {next}, does \ - not follow", - generation.generation - )); - } - generations.push(generation); - } - Ok(Index { next, generations }) - } -} - -/// A generation in the index of a store, as JSON. -#[derive(Debug, Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -struct GenerationRecord { - /// The store generation. - generation: u64, - /// The identity of the key. - key: String, - /// The identity of the capture class. - class: String, - /// The storage generation of the capture. - storage_generation: String, - /// When OpenVMM created the snapshot. - created_at: TimestampRecord, - /// The time contract of the capture. - time: TimeRecord, - /// The length of the saved state, in bytes. - state_len: u64, - /// The length of the RAM, in bytes. - memory_len: u64, -} - -impl GenerationRecord { - /// Returns the record of `generation`. - fn new(generation: &Generation) -> Self { - let created_at = generation - .created_at - .duration_since(SystemTime::UNIX_EPOCH) - .unwrap_or_default(); - Self { - generation: generation.generation.get().get(), - key: generation.key.to_string(), - class: generation.class.to_string(), - storage_generation: generation.storage.to_string(), - created_at: TimestampRecord { - seconds: created_at.as_secs(), - nanos: created_at.subsec_nanos(), - }, - time: TimeRecord::new(&generation.time), - state_len: generation.state_len, - memory_len: generation.memory_len, - } - } - - /// Checks the record, and returns its generation, or says why it is malformed. - fn into_generation(self) -> std::result::Result { - let generation = NonZeroU64::new(self.generation) - .map(StoreGeneration::new) - .ok_or("it lists store generation 0, which no generation has")?; - let field = |what: &str, error: Error| { - format!( - "the {what} of store generation {generation} is invalid: {}", - error.message() - ) - }; - let created_at = (self.created_at.nanos < 1_000_000_000) - .then(|| { - SystemTime::UNIX_EPOCH.checked_add(Duration::new( - self.created_at.seconds, - self.created_at.nanos, - )) - }) - .flatten() - .ok_or_else(|| { - format!("the creation time of store generation {generation} is invalid") - })?; - Ok(Generation { - generation, - key: self.key.parse().map_err(|error| field("key", error))?, - class: self.class.parse().map_err(|error| field("class", error))?, - storage: self - .storage_generation - .parse() - .map_err(|error| field("storage generation", error))?, - created_at, - time: self.time.into_contract().map_err(|problem| { - format!("the time contract of store generation {generation} {problem}") - })?, - state_len: self.state_len, - memory_len: self.memory_len, - }) - } -} - -/// A time, as JSON: seconds and nanoseconds since the Unix epoch. -#[derive(Debug, Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -struct TimestampRecord { - /// Whole seconds. - seconds: u64, - /// Nanoseconds within the second. - nanos: u32, -} - -/// The time contract of a capture, as JSON. -#[derive(Debug, Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -struct TimeRecord { - /// [`TimeContract::tsc_frequency_hz`]. - tsc_frequency_hz: u64, - /// [`TimeContract::tsc_tolerance_ppm`]. - tsc_tolerance_ppm: u32, - /// [`TimeContract::apic_frequency_hz`]. - apic_frequency_hz: u64, - /// [`TimeContract::capture_tsc`]. - capture_tsc: u64, - /// [`TimeContract::capture_utc_ns`]. - capture_utc_ns: u64, - /// [`TimeContract::capture_monotonic_ns`]. - capture_monotonic_ns: u64, - /// [`TimeContract::host_clock`], as a manifest spells it. - host_clock: String, - /// [`TimeContract::host_id`], as 32 hexadecimal digits. - host_id: String, - /// [`TimeContract::boot_id`]. - boot_id: BootId, - /// [`TimeContract::lineage`]. - lineage: u32, - /// [`TimeContract::cut_to_anchor_ns`]. - cut_to_anchor_ns: Option, -} - -impl TimeRecord { - /// Returns the record of `time`. - fn new(time: &TimeContract) -> Self { - Self { - tsc_frequency_hz: time.tsc_frequency_hz, - tsc_tolerance_ppm: time.tsc_tolerance_ppm, - apic_frequency_hz: time.apic_frequency_hz, - capture_tsc: time.capture_tsc, - capture_utc_ns: time.capture_utc_ns, - capture_monotonic_ns: time.capture_monotonic_ns, - host_clock: time.host_clock.as_str().to_owned(), - host_id: hex(&time.host_id), - boot_id: time.boot_id, - lineage: time.lineage, - cut_to_anchor_ns: time.cut_to_anchor_ns, - } - } - - /// Checks the record, and returns its time contract, or says why it is invalid. - fn into_contract(self) -> std::result::Result { - Ok(TimeContract { - tsc_frequency_hz: self.tsc_frequency_hz, - tsc_tolerance_ppm: self.tsc_tolerance_ppm, - apic_frequency_hz: self.apic_frequency_hz, - capture_tsc: self.capture_tsc, - capture_utc_ns: self.capture_utc_ns, - capture_monotonic_ns: self.capture_monotonic_ns, - host_clock: HostClock::parse(&self.host_clock) - .ok_or_else(|| format!("has the unknown host clock {:?}", self.host_clock))?, - host_id: parse_hex_id(&self.host_id) - .ok_or_else(|| format!("has the invalid host identity {:?}", self.host_id))?, - boot_id: self.boot_id, - lineage: self.lineage, - cut_to_anchor_ns: self.cut_to_anchor_ns, - }) - } -} diff --git a/nvx-rs/src/host/template/store/capture.rs b/nvx-rs/src/host/template/store/capture.rs deleted file mode 100644 index 660312d3e..000000000 --- a/nvx-rs/src/host/template/store/capture.rs +++ /dev/null @@ -1,1333 +0,0 @@ -//! The captures of templates, and their publication in a store: [`Store::create_template`], -//! [`Store::refresh_template`], and [`Store::recover_captures`]; see -//! [the store's documentation](Store#captures-and-publication). - -#[cfg(test)] -mod tests; - -use std::ffi::{OsStr, OsString}; -use std::fs; -use std::io; -use std::path::{Path, PathBuf}; -use std::thread; -use std::time::{Duration, Instant, SystemTime}; - -use serde::{Deserialize, Serialize}; - -use super::{ - GENERATIONS_NAME, Generation, Index, KEY_NAME, MAX_GENERATIONS, SNAPSHOT_NAME, STORE_FORMAT, - StorageGeneration, Store, entries, read_private, read_record, refuse_problem, store_error, - take_lock, write_record, -}; -use crate::activation::StoreGeneration; -use crate::error::{Error, ErrorCode, ErrorKind, Result}; -use crate::host::artifacts::{self, Artifacts}; -use crate::host::instance::generate_capability; -use crate::host::platform; -use crate::host::sandbox::distro::{CheckedDistro, DistroBinding, DistroFormat}; -use crate::host::sandbox::{LayerRole, SandboxLayer, SandboxPlan}; -use crate::host::snapshot::{check_exit, check_snapshot}; -use crate::host::store::files::{create_new, remove_if_present, sync_dir, write_private}; -use crate::host::store::regular::{identity_of, open_regular}; -use crate::host::store::{ - JOURNAL_NAME, LOCK_NAME, LOG_NAME, LaunchId, LockedStore, OUTCOME_NAME, Recovery, SOCKET_NAME, - SandboxStore, -}; -use crate::host::supervisor::{self, VmmCommand}; -use crate::host::template::eligibility::HostTime; -use crate::host::template::key::{CaptureClass, ClassIdentity, KeyIdentity, TemplateKey}; -use crate::host::template::manifest::{TemplateManifest, TimeContract}; -use crate::host::template::plan::{SCRATCH_NAME, TemplatePlan, TemplateResources}; - -/// Name of the directory of a store's captures, in which each capture class has a slot. -pub const CAPTURES_NAME: &str = "captures"; -/// Name of the transaction directory of a capture, in the slot of its class, which is also the -/// sandbox store of the capture's VM until the VM has ended. -pub const TRANSACTION_NAME: &str = "transaction"; -/// Name of the record of a capture, in its transaction and then in its generation's payload. -pub const CAPTURE_RECORD_NAME: &str = "capture.json"; - -/// Prefix of the name of the file that backs the RAM of a capture's VM, which OpenVMM creates -/// beside the snapshot's destination, and removes as the VM ends. -const MEMORY_FILE_PREFIX: &str = ".openvmm-microvm-memory-"; -/// Prefix of the names of the directories in which OpenVMM stages the snapshot, beside its -/// destination, and which it renames to the destination once the snapshot is complete: -/// `.snapshot.staging-`. -const STAGING_PREFIX: &str = ".snapshot.staging-"; - -/// Hexadecimal digits of the identity of a capture class that name its slot: few enough that the -/// path of the control socket of a capture's VMM fits a Unix socket's. -const SLOT_NAME_LEN: usize = 16; - -/// Most bytes of a Unix socket's path on Linux, `sun_path`, with its terminating NUL. -pub(in crate::host::template) const SOCKET_PATH_LIMIT: usize = 108; - -/// Longest record of a capture, in bytes. -const MAX_CAPTURE_RECORD_LEN: u64 = 4 * 1024; - -/// Interval at which a rename that Windows refuses, while another process holds a file below the -/// renamed directory open, is tried again, until the capture's deadline. -const RENAME_RETRY_INTERVAL: Duration = Duration::from_millis(25); - -/// The record of a capture, in messages. -const RECORD_DESCRIPTION: &str = "record of the capture"; -/// A capture's transaction, in messages. -const TRANSACTION_WHAT: &str = "the transaction of the capture"; -/// The directory of a store's captures, in messages. -const CAPTURES_WHAT: &str = "the captures of the template store"; - -/// What a template's capture takes: its class, the adapter's plan of its placeholder and its -/// network, the binding of the placeholder, and the artifacts that run its VM, which -/// [`Store::create_template`] and [`Store::refresh_template`] capture. -/// -/// The adapter's plan is lowered as [`TemplatePlan::new`] lowers it, with the processors, those -/// online at boot among them, and RAM of the class, and a scratch of the class's scratch length; -/// its upper is no part of the template. The class and the plan must describe one VM: a -/// placeholder whose binding records the class's `distro` length, and a network of the plan whose -/// NIC, as [`SandboxNetwork::template_arguments`] lowers it for a template, is the class's: none, -/// or one at the same guest's address, under the complete deny-all policy, whatever the policies -/// of the two networks. -/// -/// [`SandboxNetwork::template_arguments`]: crate::host::sandbox::SandboxNetwork::template_arguments -#[derive(Debug, Clone)] -pub struct Capture { - /// The class of the capture, which names its slot, and from which the capture takes its - /// backend, its CPU profile, its processors and those online at boot, its RAM, its scratch - /// length, and the runtime identity of its launch. - pub class: CaptureClass, - /// The adapter's plan, which [`SandboxPlan::adapter`] makes, of the placeholder `distro` and - /// the template's network. - pub adapter: SandboxPlan, - /// The binding of the placeholder, which the capture checks before its VM starts. - pub binding: DistroBinding, - /// The revision of the distro policy that the caller accepts, such as - /// [`DISTRO_POLICY_REVISION`](crate::host::sandbox::distro::DISTRO_POLICY_REVISION), which - /// [`DistroBinding::check`] requires of the binding's verdict. - pub policy_revision: u32, - /// The OpenVMM executable, the guest kernel, and the initramfs, whose identities the class's - /// runtime holds, which relative paths name from the current directory. - pub artifacts: Artifacts, -} - -/// How [`Store::create_template`] served a capture class: with a generation that the store held -/// already, or with one that it captured and published. -#[derive(Debug, Clone, PartialEq, Eq)] -pub enum Created { - /// A generation of the class that the store held already. - Existing(Generation), - /// A generation that a capture made, and that the store published. - Captured(Generation), -} - -impl Created { - /// Returns the generation. - pub fn generation(&self) -> &Generation { - match self { - Self::Existing(generation) | Self::Captured(generation) => generation, - } - } - - /// Returns the generation, consuming this. - pub fn into_generation(self) -> Generation { - match self { - Self::Existing(generation) | Self::Captured(generation) => generation, - } - } -} - -/// Syncs a directory, as [`sync_dir`] does. Captures take it as a parameter, so that tests -/// observe the order of their syncs, and fail them. -type SyncDir<'a> = &'a dyn Fn(&Path) -> Result<()>; - -/// Runs the VM of a capture in the slot whose lock the capture holds, as [`run_supervised`] does, -/// and returns once the VM has ended, and nothing of its launch runs. Tests run it in their own -/// process instead. -type RunVm<'a> = &'a dyn Fn(&mut LockedStore, &Vm<'_>) -> Result<()>; - -/// A point of a capture at which a test fails it, before the step that follows the point. -#[derive(Clone, Copy, Debug, PartialEq, Eq)] -enum Point { - /// The transaction holds the record of the capture and its scratch, and the VM has not - /// started. - Prepared, - /// The VM has ended, and nothing that it left is checked yet. - Exited, - /// The transaction is checked: it holds the template's key, durably. - Checked, - /// The envelope is renamed into the store, and the index does not list it yet. - Renamed, - /// The index lists the new generation. - Indexed, -} - -/// The seams of a capture, through which tests observe and interrupt it. -struct Hooks<'a> { - /// Syncs a directory. - sync: SyncDir<'a>, - /// Runs the capture's VM. - run: RunVm<'a>, - /// Fails the capture at a point, as a test asks. - interrupt: &'a dyn Fn(Point) -> Result<()>, - /// Whether a failure leaves the capture's files as they are, as a crash of the host would, - /// rather than retiring its transaction: in tests that crash captures. - crash: bool, -} - -impl Hooks<'static> { - /// Returns the seams of every capture outside tests. - fn supervised() -> Self { - Self { - sync: &sync_dir, - run: &run_supervised, - interrupt: &uninterrupted, - crash: false, - } - } -} - -impl Hooks<'_> { - /// Fails the capture at `point` if a test asks. - fn at(&self, point: Point) -> Result<()> { - (self.interrupt)(point) - } -} - -/// Lets a capture pass every point, as captures outside tests do. -fn uninterrupted(_: Point) -> Result<()> { - Ok(()) -} - -/// What a capture takes of its inputs once it has checked them, before it opens anything. -struct Inputs { - /// The identity of the class. - class: ClassIdentity, - /// The template's processors, those online at boot among them, and RAM, from the class. - resources: TemplateResources, - /// The artifacts, as absolute paths. - artifacts: Artifacts, -} - -impl Inputs { - /// Checks the inputs of `capture`, without reading or opening anything, and returns what the - /// capture takes of them. - /// - /// Fails with [`ErrorKind::Unsupported`] as [`CaptureClass::identity`] fails, and for a - /// placeholder whose binding records another length than the class's `distro`, and a plan - /// whose network attaches another NIC than the class's, as a template attaches it; and with - /// [`ErrorKind::Unavailable`] for an artifact whose path cannot be resolved. - fn check(capture: &Capture) -> Result { - let class = &capture.class; - let identity = class.identity()?; - let length = capture.binding.image().length(); - if length != class.distro.length { - return Err(Error::unsupported(format!( - "the placeholder {} is {length} bytes long, as its binding records it, but the \ - class's distro is {} bytes long", - capture.adapter.distro().path().display(), - class.distro.length - ))); - } - let planned = match capture.adapter.network() { - None => Vec::new(), - Some(network) => network.template_arguments()?, - }; - let requested = match &class.network { - None => Vec::new(), - Some(network) => network.template_arguments()?, - }; - if planned != requested { - return Err(Error::unsupported(format!( - "the adapter's plan attaches the NIC {planned:?}, but the class requests \ - {requested:?}" - ))); - } - let given = &capture.artifacts; - Ok(Self { - class: identity, - resources: TemplateResources { - processors: class.processors, - boot_processors: class.boot_processors, - memory: class.memory, - memory_capacity: class.memory_capacity, - }, - artifacts: Artifacts::new( - artifacts::absolute(&given.openvmm)?, - artifacts::absolute(&given.kernel)?, - artifacts::absolute(&given.initrd)?, - ), - }) - } -} - -/// The VM of a capture, as the capture's source runs it. -struct Vm<'a> { - /// The inputs of the capture. - capture: &'a Capture, - /// The plan of the template. - template: &'a TemplatePlan, - /// The OpenVMM executable, the guest kernel, and the initramfs, as absolute paths. - artifacts: &'a Artifacts, - /// When the capture must have ended. - deadline: Instant, -} - -impl Vm<'_> { - /// Returns the command of the capture's VMM, run in the capture's transaction `dir`, whose - /// control console listens on `endpoint`, and whose outcome report goes to `report`. - /// - /// Its arguments are the template's, as [`TemplatePlan::openvmm_arguments`] renders them, - /// then the backend, the CPU profile, the kernel and the initramfs, the template's kernel - /// command line, a boot console that discards its output, the control console, and the - /// outcome report. - fn command(&self, endpoint: &str, report: &Path, dir: &Path) -> Result> { - let class = &self.capture.class; - let mut arguments: Vec = self - .template - .openvmm_arguments()? - .into_iter() - .map(OsString::from) - .collect(); - let listen = format!("listen={endpoint}"); - for argument in [ - OsStr::new("--single-process"), - OsStr::new("--hypervisor"), - OsStr::new(class.hypervisor.as_str()), - OsStr::new("--cpu-profile"), - OsStr::new(&class.cpu_profile), - OsStr::new("--kernel"), - self.artifacts.kernel.as_os_str(), - OsStr::new("--initrd"), - self.artifacts.initrd.as_os_str(), - OsStr::new("--cmdline"), - OsStr::new(self.template.kernel_command_line()), - OsStr::new("--virtio-console"), - OsStr::new("none"), - OsStr::new("--microvm-control-console"), - OsStr::new(&listen), - OsStr::new("--microvm-control-auth-stdin"), - OsStr::new("--microvm-report"), - report.as_os_str(), - ] { - arguments.push(argument.to_owned()); - } - Ok(VmmCommand::new(&self.artifacts.openvmm, arguments, dir)) - } -} - -/// Runs the VM of a capture through the supervisor, from the sandbox store of `locked`, the -/// capture's transaction, as a launch of its own, until its VMM exits, or until the capture's -/// deadline, at which it ends the VMM, and then ends the launch. -/// -/// The capture opens no control session, and waits for no Ready: its guest captures itself, and -/// OpenVMM exits once it has published the snapshot, or once the capture failed. Fails with -/// [`ErrorKind::TimedOut`] if the deadline passes first, and with -/// [`ErrorCode::VmmIndeterminate`] if the launch cannot be ended, whose journal then remains for -/// the recovery of the slot; and as [`supervisor::launch`] fails. -fn run_supervised(locked: &mut LockedStore, vm: &Vm<'_>) -> Result<()> { - let report = locked.outcome_path(); - let dir = locked.dir().to_owned(); - let capability = generate_capability()?; - // A slot holds no lease, so each capture reserves its launch itself. - let launch = LaunchId::generate()?; - let starting = supervisor::launch( - locked, - &launch, - &vm.capture.class.runtime, - &capability, - |endpoint| vm.command(endpoint, &report, &dir), - )?; - let exited = starting.process().wait_bool(vm.deadline).map_err(|error| { - Error::io(format!( - "cannot wait for the VMM of the capture in {}", - dir.display() - )) - .with_source(error) - }); - if let Recovery::Indeterminate(reason) = starting.abandon()? { - return Err(indeterminate(&dir, &reason)); - } - if !exited? { - return Err(Error::timed_out(format!( - "the VM of the capture in {} did not end before the capture's deadline, and was torn \ - down", - dir.display() - ))); - } - Ok(()) -} - -/// Returns the error of a capture whose VMM, in the slot `dir`, may still run, for `reason`, -/// [`ErrorCode::VmmIndeterminate`]. -fn indeterminate(dir: &Path, reason: &str) -> Error { - Error::new( - ErrorCode::VmmIndeterminate, - format!( - "the VMM of the capture in {} may still run, and its launch remains for recovery: \ - {reason}", - dir.display() - ), - ) -} - -impl Store { - /// Returns a generation of the class of `capture` that the store holds, or captures one and - /// publishes it, waiting for the capture's slot and the store's lock until `deadline`, which - /// bounds the capture too. - /// - /// Captures of one class are serialized by the lock of the class's slot, under which a - /// creation first ends a capture of the class that a crash interrupted, and completes the - /// store's publications that a crash interrupted, as - /// [`recover_captures`](Self::recover_captures) does. It then calls `now` for the host's time, - /// once, and looks for a generation: the latest that the store indexes of the class, of the - /// key of the class's latest generation, that no tombstone quarantines, and that is eligible - /// at that time, as [`Generation::eligibility`] decides it, as [`select`](Self::select) - /// chooses among the generations of a key. Without one, it captures one, as - /// [`refresh_template`](Self::refresh_template) does. Since `now` is called once the creation - /// holds the slot, a generation that a racing creation captured while this one waited is - /// eligible at that time, so creations that race for one class make one generation between - /// them. [`HostTime::now`] samples this host's time; a caller that keeps a margin, as before - /// the window of a generation closes, returns a later time. - /// - /// Fails as `now` fails, and otherwise as [`refresh_template`](Self::refresh_template) does. - pub fn create_template( - &self, - capture: &Capture, - now: impl Fn() -> Result, - deadline: Instant, - ) -> Result { - self.capture(capture, Some(&now), deadline, &Hooks::supervised()) - } - - /// Captures a template of the class of `capture`, and publishes it as the store's next - /// generation, which later selections of its key take before the older ones, waiting for the - /// capture's slot and the store's lock until `deadline`, which bounds the capture too. - /// - /// The capture cold-boots the template's VM through [`supervisor::launch`], from its - /// transaction, in the slot of its class, and waits for it to exit: its guest captures itself, - /// and OpenVMM publishes the snapshot into the capture's transaction and exits. The capture - /// checks the files before the VM starts, and what the VM left once it has exited, publishes - /// the transaction in two commits, and leaves the older generations, and their leases, - /// untouched; see [the store's documentation](Store#captures-and-publication). - /// - /// Fails, and publishes nothing, but a transaction that it checked, which the next recovery - /// of the slot publishes: - /// - /// - with [`ErrorKind::Unsupported`] for inputs that [`Capture`] does not admit, as - /// [`TemplatePlan::new`] refuses them, for a store whose path is too long for the control - /// socket of a capture's VMM, on Linux, for one whose capture transaction path exceeds - /// 258 UTF-16 code units on Windows, including any verbatim prefix and trailing separator, - /// and for a store that holds - /// [`MAX_GENERATIONS`] generations; - /// - with [`ErrorKind::Unavailable`] for a placeholder that its binding's check refuses, as - /// [`DistroBinding::check`] fails, that is a link, or on Windows any other reparse point, - /// or that is not a regular file, and for one that belongs to another user than the current - /// one or an administrator, or that anyone else may change; - /// - with [`ErrorKind::Exited`] if the VM ended without a capture, as when its guest failed, - /// or OpenVMM did, even after it published the snapshot; - /// - with [`ErrorKind::TimedOut`] if the deadline passes before the VM ends, which the - /// capture then tears down, or before the capture takes its slot's lock or the store's; - /// - with [`ErrorCode::VmmIndeterminate`] if the capture cannot prove that the VMM of a - /// capture of the class, its own or an interrupted one, no longer runs: the slot then keeps - /// the launch and its transaction for a later recovery; - /// - with [`ErrorKind::Io`] for files of the slot, of the transaction, or of the store that - /// are links, that are not private to the current user, or that cannot be created, read, - /// synced, or renamed, and for a capture that did not leave what its class requests: a - /// snapshot whose manifest [`TemplateManifest::read`] refuses, that is not the class's, as - /// [`TemplateKey::new`] checks it, or whose artifacts are not exactly its manifest, RAM, and - /// saved state, each a regular file of the length that the manifest records. - pub fn refresh_template(&self, capture: &Capture, deadline: Instant) -> Result { - self.capture(capture, None, deadline, &Hooks::supervised()) - .map(Created::into_generation) - } - - /// Ends the captures that a crash interrupted, waiting for the lock of each slot and for the - /// store's until `deadline`, and returns the generations that it published, with those whose - /// publications it completed, in the order of their store generations. - /// - /// In each slot, it ends the launch of the capture's VMM, as [`LockedStore::recover`] ends - /// it, then publishes the capture's transaction if the capture had checked it, and retires it - /// otherwise. It then completes a publication that a crash interrupted once it had renamed - /// its envelope into the store, and drops from the index the generations whose payloads a - /// crash lost. A capture recovers the slot of its class, and the store's publications, the - /// same way, so an adapter recovers the other slots once, as it starts. - /// - /// Fails with [`ErrorCode::ForeignFiles`] if the directory of the captures holds entries that - /// are not slots, and otherwise as [`refresh_template`](Self::refresh_template) fails. - pub fn recover_captures(&self, deadline: Instant) -> Result> { - self.recover_with(deadline, &Hooks::supervised()) - } - - /// Serves `capture` as [`create_template`](Self::create_template) does with the host's time - /// that `now` returns, or, without it, as [`refresh_template`](Self::refresh_template) does, - /// through `hooks`. - fn capture( - &self, - capture: &Capture, - now: Option<&dyn Fn() -> Result>, - deadline: Instant, - hooks: &Hooks<'_>, - ) -> Result { - let inputs = Inputs::check(capture)?; - let slot = self.open_slot(inputs.class)?; - let locked = lock_slot(&slot, deadline)?; - self.recover_slot(&locked, deadline, hooks)?; - // The index then lists every generation that a publication renamed into the store, and - // none whose payload a crash lost, for the lookup and the store's capacity alike. - self.recover_publications(deadline, hooks)?; - if let Some(now) = now - && let Some(existing) = self.existing(inputs.class, &now()?)? - { - // The commit that listed it may have been interrupted before it made the index, or - // the store's own entry in its parent, durable. - self.sync_store(hooks)?; - return Ok(Created::Existing(existing)); - } - if self.index()?.generations.len() >= MAX_GENERATIONS { - return Err(self.full()); - } - let transaction = locked.dir().join(TRANSACTION_NAME); - let template = TemplatePlan::new( - &capture.adapter, - &capture.binding, - capture.class.scratch.length, - inputs.resources, - &transaction, - StorageGeneration::generate()?, - )?; - let envelope = - match self.check_capture(capture, &inputs, &template, &locked, deadline, hooks) { - Ok(envelope) => envelope, - Err(error) => { - // A transaction stays while its sandbox store records a launch, whose VMM may - // still write it, for the slot's recovery to retire once the VMM has ended; - // and so does one that a retirement interrupted, as a crash would leave it. - if !hooks.crash && !records_launch(&transaction) { - let _ = keep_diagnostics(&transaction, locked.dir()); - let _ = retire(&transaction); - } - return Err(error); - } - }; - hooks.at(Point::Checked)?; - self.publish(&transaction, &envelope, deadline, hooks) - .map(|(published, _)| Created::Captured(published)) - } - - /// Runs the capture of `template`, with the inputs `capture`, checked as `inputs`, in the slot - /// whose lock `slot` holds, through `hooks`, until the capture checks its transaction, or until - /// `deadline`, and returns its envelope. - /// - /// The transaction is the sandbox store of the capture's VM: OpenVMM requires the control - /// socket of a VM that captures a snapshot, on Linux, beside the snapshot's destination, and a - /// sandbox store's VM listens in the store's directory. - fn check_capture( - &self, - capture: &Capture, - inputs: &Inputs, - template: &TemplatePlan, - slot: &LockedStore, - deadline: Instant, - hooks: &Hooks<'_>, - ) -> Result { - let transaction = template.transaction(); - // The slot keeps the log and outcome report of its class's latest capture, this one, so - // that the capture never reads an earlier VM's report for its own. - for name in [LOG_NAME, OUTCOME_NAME] { - remove_if_present(&slot.dir().join(name))?; - } - let mut launch = SandboxStore::create(transaction)?.lock(Some(deadline))?; - write_record( - &transaction.join(CAPTURE_RECORD_NAME), - &CaptureRecord::new(self, inputs.class, template.generation()), - &format!("the {RECORD_DESCRIPTION}"), - )?; - let scratch = create_scratch(template.scratch_path(), template.scratch_length())?; - hooks.at(Point::Prepared)?; - let placeholder = check_files(capture, template, scratch)?; - let vm = Vm { - capture, - template, - artifacts: &inputs.artifacts, - deadline, - }; - (hooks.run)(&mut launch, &vm)?; - drop(placeholder); - keep_diagnostics(transaction, slot.dir())?; - hooks.at(Point::Exited)?; - check_exit(&slot.outcome_path())?; - let snapshot = template.destination(); - let manifest = check_snapshot(snapshot)?; - let key = TemplateKey::new(&capture.class, &manifest)?; - // The host closes its handles before it publishes: the transaction's lock goes, and its - // file with what else the capture leaves. - drop(launch); - clean_transaction(transaction)?; - write_private(&transaction.join(KEY_NAME), &key.to_bytes())?; - // The checked transaction must survive a crash, for the slot's recovery to publish it, - // and so must the store, whose own entry in its parent its creation does not sync. - for dir in [transaction, slot.dir(), &self.captures_dir()] { - (hooks.sync)(dir)?; - } - self.sync_store(hooks)?; - Ok(Envelope::new( - &key, - inputs.class, - template.generation(), - &manifest, - )) - } - - /// Publishes the checked transaction `transaction`, whose envelope is `envelope`, as the - /// store's next generation, under the store's lock, which it waits for until `deadline`, - /// through `hooks`, and returns the generation, with those of the publications that it - /// completed first. - /// - /// It first recovers the index, as [`recover_index`](Self::recover_index) does. Then it - /// renames the transaction to the payload of the next store generation, which it never - /// replaces, makes the rename durable, and commits the index, which lists the generation with - /// its full key, as [`commit_index`](Self::commit_index) does. - fn publish( - &self, - transaction: &Path, - envelope: &Envelope, - deadline: Instant, - hooks: &Hooks<'_>, - ) -> Result<(Generation, Vec)> { - let _lock = take_lock(&self.dir, Some(deadline))?; - let mut index = self.index()?; - let (completed, changed) = self.recover_index(&mut index, hooks)?; - if index.generations.len() >= MAX_GENERATIONS { - if changed { - self.commit_index(&index, hooks)?; - } - return Err(self.full()); - } - let generation = index.next; - rename_until(transaction, &self.payload_dir(generation), deadline)?; - (hooks.sync)(&self.dir.join(GENERATIONS_NAME))?; - if let Some(slot) = transaction.parent() { - (hooks.sync)(slot)?; - } - hooks.at(Point::Renamed)?; - let published = envelope.generation(generation); - index.generations.push(published.clone()); - index.next = following(generation)?; - self.commit_index(&index, hooks)?; - hooks.at(Point::Indexed)?; - Ok((published, completed)) - } - - /// Writes `index`, which the store's lock holds, and makes it durable through `hooks`, as - /// [`sync_store`](Self::sync_store) does, so that a crash of the host loses neither a - /// generation that it lists nor the store. - fn commit_index(&self, index: &Index, hooks: &Hooks<'_>) -> Result<()> { - self.write_index(index)?; - self.sync_store(hooks) - } - - /// Syncs the store's directory, and then its parent, through `hooks`: [`Store::create`] - /// syncs neither, so the store's own entry in its parent may not be durable yet. - fn sync_store(&self, hooks: &Hooks<'_>) -> Result<()> { - (hooks.sync)(&self.dir)?; - if let Some(parent) = self.dir.parent() { - (hooks.sync)(parent)?; - } - Ok(()) - } - - /// Completes the store's publications that a crash interrupted, and drops from the index the - /// generations whose payloads a crash lost, as [`recover_index`](Self::recover_index) does, - /// under the store's lock, which it waits for until `deadline`, through `hooks`. Commits the - /// index if that changed it, and returns the generations that it completed. - fn recover_publications( - &self, - deadline: Instant, - hooks: &Hooks<'_>, - ) -> Result> { - let _lock = take_lock(&self.dir, Some(deadline))?; - let mut index = self.index()?; - let (completed, changed) = self.recover_index(&mut index, hooks)?; - if changed { - self.commit_index(&index, hooks)?; - } - Ok(completed) - } - - /// Recovers `index`, which the store's lock holds, from the publications and retirements that - /// a crash interrupted, through `hooks`, and returns the generations that it indexed, and - /// whether it changed the index, which the caller writes. - /// - /// A generation whose payload is gone was lost in a crash, as one that a publication renamed - /// on a host that cannot flush a directory, or whose retirement removed its payload before it - /// wrote the index: the index first drops it, so that it takes none of the store's capacity. - /// Its tombstone, if any, stays for its retirement to remove. A publication renames its - /// envelope to the payload of the index's next store generation before it writes the index, - /// so a payload there is one whose publication was interrupted: the index then takes it, as - /// the publication would have, once the recovery has made the rename durable, as the - /// publication would have too: it syncs the payload, the directory of the payloads, and the - /// slot that the envelope left. - fn recover_index( - &self, - index: &mut Index, - hooks: &Hooks<'_>, - ) -> Result<(Vec, bool)> { - self.check_generations_dir()?; - let listed = index.generations.len(); - let mut kept = Vec::with_capacity(listed); - for generation in index.generations.drain(..) { - match fs::symlink_metadata(self.payload_dir(generation.generation)) { - Err(error) if error.kind() == io::ErrorKind::NotFound => {} - _ => kept.push(generation), - } - } - index.generations = kept; - let dropped = index.generations.len() != listed; - let mut completed = Vec::new(); - loop { - let payload = self.payload_dir(index.next); - match fs::symlink_metadata(&payload) { - Err(error) if error.kind() == io::ErrorKind::NotFound => break, - Err(error) => { - return Err(store_error(format!( - "cannot open the payload of the generation {}", - payload.display() - ))(error)); - } - Ok(_) => {} - } - if index.generations.len() >= MAX_GENERATIONS { - return Err(self.full()); - } - let generation = self.read_envelope(&payload)?.generation(index.next); - (hooks.sync)(&payload)?; - index.generations.push(generation.clone()); - index.next = following(index.next)?; - completed.push(generation); - } - if !completed.is_empty() { - (hooks.sync)(&self.dir.join(GENERATIONS_NAME))?; - let mut slots: Vec = completed - .iter() - .map(|generation| slot_name(generation.class)) - .collect(); - slots.sort(); - slots.dedup(); - for name in slots { - let slot = self.captures_dir().join(name); - match fs::symlink_metadata(&slot) { - Err(error) if error.kind() == io::ErrorKind::NotFound => {} - Err(error) => { - return Err(store_error(format!( - "cannot open the slot {}", - slot.display() - ))(error)); - } - Ok(metadata) if metadata.is_dir() => (hooks.sync)(&slot)?, - Ok(_) => {} - } - } - } - let changed = dropped || !completed.is_empty(); - Ok((completed, changed)) - } - - /// Reads the envelope of a checked capture in the directory `dir`, its transaction or its - /// payload, and returns what the index lists of it: the record of the capture, the template's - /// key, and the manifest of its snapshot, whose artifacts it checks again by their kinds and - /// lengths. - fn read_envelope(&self, dir: &Path) -> Result { - let what = "the envelope of the capture"; - let problem = platform::check_owner_only_dir(dir) - .map_err(store_error(format!("cannot open {what} {}", dir.display())))?; - refuse_problem(dir, what, problem)?; - let mut names = entries(dir)?; - names.sort(); - if names != [CAPTURE_RECORD_NAME, KEY_NAME, SNAPSHOT_NAME] { - return Err(Error::io(format!( - "{what} {} holds {}, not exactly the {RECORD_DESCRIPTION}, its key, and its \ - snapshot", - dir.display(), - names.join(", ") - ))); - } - let path = dir.join(CAPTURE_RECORD_NAME); - let record = - read_record::(&path, RECORD_DESCRIPTION, MAX_CAPTURE_RECORD_LEN)? - .ok_or_else(|| { - Error::io(format!( - "the {RECORD_DESCRIPTION} {} is missing", - path.display() - )) - })?; - let malformed = |problem: &str| { - Error::io(format!( - "the {RECORD_DESCRIPTION} {} is malformed: {problem}", - path.display() - )) - }; - if record.store != self.id.to_string() { - return Err(malformed(&format!( - "it is a capture of the template store {}", - record.store - ))); - } - let class: ClassIdentity = record - .class - .parse() - .map_err(|error: Error| malformed(error.message()))?; - let storage: StorageGeneration = record - .storage_generation - .parse() - .map_err(|error: Error| malformed(error.message()))?; - let path = dir.join(KEY_NAME); - let description = "template key"; - let bytes = read_private(&path, description, TemplateKey::MAX_LEN as u64)? - .ok_or_else(|| Error::io(format!("the {description} {} is missing", path.display())))?; - let key = TemplateKey::from_bytes(&bytes).map_err(|error| { - Error::new( - error.code(), - format!("cannot read {}: {}", path.display(), error.message()), - ) - })?; - let manifest = check_snapshot(&dir.join(SNAPSHOT_NAME))?; - Ok(Envelope::new(&key, class, storage, &manifest)) - } - - /// Ends the capture that a crash interrupted in the slot whose lock `slot` holds, if any, - /// through `hooks`: ends its VMM, which the transaction's sandbox store records, as - /// [`LockedStore::recover`] does, and then publishes its transaction if the capture checked - /// it, waiting for the store's lock until `deadline`, and retires it otherwise, once it has - /// moved the VMM's log and outcome report to the slot. Returns the generations that it - /// published: those of the publications that it completed first, as - /// [`publish`](Self::publish) does, and then the transaction's. - fn recover_slot( - &self, - slot: &LockedStore, - deadline: Instant, - hooks: &Hooks<'_>, - ) -> Result> { - let transaction = slot.dir().join(TRANSACTION_NAME); - match platform::check_owner_only_dir(&transaction) { - Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(Vec::new()), - checked => { - let problem = checked.map_err(store_error(format!( - "cannot open {TRANSACTION_WHAT} {}", - transaction.display() - )))?; - refuse_problem(&transaction, TRANSACTION_WHAT, problem)?; - } - } - end_launch(&transaction, deadline)?; - let key = transaction.join(KEY_NAME); - match fs::symlink_metadata(&key) { - Err(error) if error.kind() == io::ErrorKind::NotFound => { - keep_diagnostics(&transaction, slot.dir())?; - retire(&transaction)?; - Ok(Vec::new()) - } - Err(error) => Err(store_error(format!("cannot open {}", key.display()))(error)), - Ok(_) => { - let envelope = self.read_envelope(&transaction)?; - // The crash may have come before the capture synced the transaction that it - // checked, whose entries must be durable before the rename publishes them. - (hooks.sync)(&transaction)?; - let (published, mut completed) = - self.publish(&transaction, &envelope, deadline, hooks)?; - completed.push(published); - Ok(completed) - } - } - } - - /// Recovers the store's captures as [`recover_captures`](Self::recover_captures) does, - /// through `hooks`. - fn recover_with(&self, deadline: Instant, hooks: &Hooks<'_>) -> Result> { - let captures = self.captures_dir(); - let mut published = Vec::new(); - let exists = match platform::check_owner_only_dir(&captures) { - Err(error) if error.kind() == io::ErrorKind::NotFound => false, - checked => { - let problem = checked.map_err(store_error(format!( - "cannot open {CAPTURES_WHAT} {}", - captures.display() - )))?; - refuse_problem(&captures, CAPTURES_WHAT, problem)?; - true - } - }; - if exists { - let mut slots = entries(&captures)?; - slots.sort(); - let foreign: Vec<&str> = slots - .iter() - .map(String::as_str) - .filter(|name| !is_slot_name(name)) - .collect(); - if !foreign.is_empty() { - return Err(Error::new( - ErrorCode::ForeignFiles, - format!( - "{CAPTURES_WHAT} {} holds entries that are not the store's: {}", - captures.display(), - foreign.join(", ") - ), - )); - } - for name in slots { - let slot = SandboxStore::open(captures.join(name))?; - let locked = lock_slot(&slot, deadline)?; - published.extend(self.recover_slot(&locked, deadline, hooks)?); - } - } - published.extend(self.recover_publications(deadline, hooks)?); - published.sort_by_key(Generation::generation); - Ok(published) - } - - /// Returns the generation with which a creation serves the class `class` at the host's time - /// `now`, if the store holds one: the latest of the class, of the key of the class's latest - /// generation, that no tombstone quarantines, and that is eligible at `now`. - fn existing(&self, class: ClassIdentity, now: &HostTime) -> Result> { - let index = self.index()?; - let Some(latest) = index - .generations - .iter() - .rev() - .find(|generation| generation.class == class) - else { - return Ok(None); - }; - let key = latest.key; - let tombstoned = self.tombstones()?; - Ok(index - .generations - .iter() - .rev() - .find(|generation| { - generation.class == class - && generation.key == key - && !tombstoned.contains(&generation.generation) - && generation.eligibility(now).is_eligible() - }) - .cloned()) - } - - /// Creates the directory of the store's captures if needed, and the slot of the class - /// `class` in it, and returns the slot's store. - fn open_slot(&self, class: ClassIdentity) -> Result { - let captures = self.captures_dir(); - let slot = captures.join(slot_name(class)); - platform::check_working_dir(&slot.join(TRANSACTION_NAME))?; - create_private_dir(&captures, CAPTURES_WHAT)?; - let slot = SandboxStore::create(slot)?; - // A capture's VM listens in its transaction, the sandbox store from which it launches. - let socket = slot.dir().join(TRANSACTION_NAME).join(SOCKET_NAME); - if cfg!(target_os = "linux") && socket.as_os_str().len() >= SOCKET_PATH_LIMIT { - return Err(Error::unsupported(format!( - "the template store {} is too deep for the control socket of a capture's VMM, \ - {}, whose path must be shorter than {SOCKET_PATH_LIMIT} bytes", - self.dir.display(), - socket.display() - ))); - } - Ok(slot) - } - - /// Returns the path of the directory of the store's captures. - fn captures_dir(&self) -> PathBuf { - self.dir.join(CAPTURES_NAME) - } - - /// Returns the error of a store that holds [`MAX_GENERATIONS`] generations, which takes no - /// other. - fn full(&self) -> Error { - Error::unsupported(format!( - "the template store {} holds {MAX_GENERATIONS} generations, the most that a store \ - holds: retire some before a capture publishes another", - self.dir.display() - )) - } -} - -/// Takes the lock of the slot `slot`, waiting for it until `deadline`, as another capture of its -/// class may hold it. -fn lock_slot(slot: &SandboxStore, deadline: Instant) -> Result { - slot.lock(Some(deadline)).map_err(|error| { - if error.code() == ErrorCode::SandboxBusy { - Error::new( - ErrorKind::TimedOut, - format!( - "another capture of the class of the slot {} is still in progress", - slot.dir().display() - ), - ) - } else { - error - } - }) -} - -/// Returns the name of the slot of the class `class`: the first [`SLOT_NAME_LEN`] digits of its -/// identity. -fn slot_name(class: ClassIdentity) -> String { - class.to_string()[..SLOT_NAME_LEN].to_owned() -} - -/// Returns whether `name` is the name of a slot. -fn is_slot_name(name: &str) -> bool { - name.len() == SLOT_NAME_LEN - && name - .bytes() - .all(|byte| byte.is_ascii_digit() || (b'a'..=b'f').contains(&byte)) -} - -/// Creates the directory `path`, which `what` names, private to the current user, unless it -/// exists, which must be private already. -fn create_private_dir(path: &Path, what: &str) -> Result<()> { - let problem = platform::create_owner_only_dir(path).map_err(store_error(format!( - "cannot create {what} {}", - path.display() - )))?; - refuse_problem(path, what, problem) -} - -/// Creates the dormant scratch device's file at `path`, private to the current user: a sparse -/// file of `length` bytes, and returns its identity, as [`identity_of`] returns it. -fn create_scratch(path: &Path, length: u64) -> Result<(u64, u128)> { - let failed = |error| { - Error::io(format!( - "cannot create the scratch of the capture {}", - path.display() - )) - .with_source(error) - }; - let file = create_new(path).map_err(failed)?; - platform::make_sparse(&file).map_err(failed)?; - file.set_len(length).map_err(failed)?; - identity_of(&file).map_err(failed) -} - -/// Checks the files of the capture of `template`, with the inputs `capture`, before its VM -/// starts: its transaction directory, its placeholder, and its scratch, which the capture -/// created with the identity `scratch`. Returns the placeholder that the check of its binding -/// opened, which the capture holds, for reading alone, until its VM has ended. -/// -/// OpenVMM opens the block devices of a capture itself, at their paths, so the capture holds no -/// other handle of them as its VM starts, through which anything could write them. -fn check_files( - capture: &Capture, - template: &TemplatePlan, - scratch: (u64, u128), -) -> Result { - let transaction = template.transaction(); - let problem = platform::check_owner_only_dir(transaction).map_err(store_error(format!( - "cannot open {TRANSACTION_WHAT} {}", - transaction.display() - )))?; - refuse_problem(transaction, TRANSACTION_WHAT, problem)?; - let placeholder = check_placeholder(capture, template)?; - check_scratch(template, scratch)?; - Ok(placeholder) -} - -/// Checks the placeholder of `template` against the binding of `capture`, as -/// [`DistroBinding::check`] does, and then at its path: a regular file, not a link, nor on -/// Windows any other reparse point, that is the file that the check found, that belongs to the -/// current user or to an administrator, and that no one else may change. Returns the file that -/// the binding's check opened. -fn check_placeholder(capture: &Capture, template: &TemplatePlan) -> Result { - let path = template.plan().distro().path(); - let refused = |problem: String| { - Error::unavailable(format!("the placeholder {} {problem}", path.display())) - }; - let text = path - .to_str() - .ok_or_else(|| refused("has a path that is not valid Unicode".to_owned()))?; - let binding = &capture.binding; - let layer = match binding.format() { - DistroFormat::Erofs => SandboxLayer::new(LayerRole::Distro, text, binding.identity()), - DistroFormat::Gpt => SandboxLayer::gpt(LayerRole::Distro, text, binding.identity()), - }; - let checked = binding.check(&layer, capture.policy_revision)?; - let file = open_regular(path).map_err(|error| refused(format!("cannot be used: {error}")))?; - let identity = - |file| identity_of(file).map_err(|error| refused(format!("cannot be identified: {error}"))); - if identity(&file)? != identity(checked.file())? { - return Err(refused( - "changed while its binding was checked: its path names another file".to_owned(), - )); - } - if let Some(problem) = platform::foreign_writer_problem(&file) - .map_err(|error| refused(format!("cannot be checked: {error}")))? - { - return Err(refused(problem)); - } - Ok(checked) -} - -/// Checks the scratch of `template` at its path, which the capture created with the identity -/// `created`: a regular file of the template's scratch length, not a link, nor on Windows any -/// other reparse point, that is the file that the capture created, and that no one else may -/// change. The check's file is closed as it returns. -fn check_scratch(template: &TemplatePlan, created: (u64, u128)) -> Result<()> { - let path = template.scratch_path(); - let refused = |problem: String| { - Error::io(format!( - "the scratch of the capture {} {problem}", - path.display() - )) - }; - let file = open_regular(path).map_err(|error| refused(format!("cannot be used: {error}")))?; - let identity = - identity_of(&file).map_err(|error| refused(format!("cannot be identified: {error}")))?; - if identity != created { - return Err(refused( - "is not the file that the capture created".to_owned(), - )); - } - let length = file - .metadata() - .map_err(|error| refused(format!("cannot be examined: {error}")))? - .len(); - if length != template.scratch_length() { - return Err(refused(format!( - "is {length} bytes long, not the {} bytes of the template's scratch", - template.scratch_length() - ))); - } - if let Some(problem) = platform::foreign_writer_problem(&file) - .map_err(|error| refused(format!("cannot be checked: {error}")))? - { - return Err(refused(problem)); - } - Ok(()) -} - -/// Removes from the capture's transaction `transaction` what the template does not keep: the lock -/// file of its sandbox store, which the capture closed once its VM had ended, the dormant scratch -/// device's file, the file that backed the VM's RAM, which OpenVMM removes as the VM ends unless -/// it ends abruptly, and directories in which OpenVMM staged a snapshot that it did not publish. -/// Fails unless the transaction then holds the record of the capture and its snapshot alone. -fn clean_transaction(transaction: &Path) -> Result<()> { - for name in entries(transaction)? { - let path = transaction.join(&name); - if name == LOCK_NAME || name == SCRATCH_NAME || name.starts_with(MEMORY_FILE_PREFIX) { - remove_if_present(&path)?; - } else if name.starts_with(STAGING_PREFIX) { - match fs::remove_dir_all(&path) { - Err(error) if error.kind() != io::ErrorKind::NotFound => { - return Err(store_error(format!("cannot remove {}", path.display()))( - error, - )); - } - _ => {} - } - } - } - let mut names = entries(transaction)?; - names.sort(); - if names != [CAPTURE_RECORD_NAME, SNAPSHOT_NAME] { - return Err(Error::io(format!( - "{TRANSACTION_WHAT} {} holds {}, not exactly the {RECORD_DESCRIPTION} and its \ - snapshot", - transaction.display(), - names.join(", ") - ))); - } - Ok(()) -} - -/// Returns whether the sandbox store of the capture's transaction `transaction` records a launch, -/// whose VMM may still run, or whether that cannot be told: whether the journal of a launch is -/// there, unless nothing is. -fn records_launch(transaction: &Path) -> bool { - !matches!( - fs::symlink_metadata(transaction.join(JOURNAL_NAME)), - Err(error) if error.kind() == io::ErrorKind::NotFound - ) -} - -/// Ends the launch that the sandbox store of the capture's transaction `transaction` records, if -/// any, as [`LockedStore::recover`] ends it, waiting for the store's lock until `deadline`. -/// -/// A transaction without the store's lock file records no launch: a capture creates the lock file -/// before it launches, and removes it once its VM has ended, before it checks the transaction. -/// Fails with [`ErrorCode::VmmIndeterminate`] if the recovery cannot prove that the VMM no longer -/// runs, and the store then keeps the launch. -fn end_launch(transaction: &Path, deadline: Instant) -> Result<()> { - let lock = transaction.join(LOCK_NAME); - match fs::symlink_metadata(&lock) { - Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(()), - Err(error) => { - return Err(store_error(format!("cannot open {}", lock.display()))( - error, - )); - } - Ok(_) => {} - } - let mut launch = SandboxStore::open(transaction)?.lock(Some(deadline))?; - match launch.recover()? { - Recovery::Indeterminate(reason) => Err(indeterminate(transaction, &reason)), - _ => Ok(()), - } -} - -/// Moves the VMM's log and outcome report of the capture whose transaction is `transaction` to -/// its slot, `slot`, which keeps those of its class's latest capture, once the capture's launch -/// has ended. A file that the transaction lacks, such as one that this moved already, stays as it -/// is. -fn keep_diagnostics(transaction: &Path, slot: &Path) -> Result<()> { - for name in [LOG_NAME, OUTCOME_NAME] { - let from = transaction.join(name); - match fs::rename(&from, slot.join(name)) { - Err(error) if error.kind() != io::ErrorKind::NotFound => { - return Err(store_error(format!( - "cannot move {} to the slot {}", - from.display(), - slot.display() - ))(error)); - } - _ => {} - } - } - Ok(()) -} - -/// Removes the transaction `transaction` of a capture that did not check it, with everything in -/// it, once nothing of the capture runs, if it exists. One that is a link, or that is not private -/// to the current user, is refused, and kept. -fn retire(transaction: &Path) -> Result<()> { - match platform::check_owner_only_dir(transaction) { - Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(()), - checked => { - let problem = checked.map_err(store_error(format!( - "cannot open {TRANSACTION_WHAT} {}", - transaction.display() - )))?; - refuse_problem(transaction, TRANSACTION_WHAT, problem)?; - } - } - match fs::remove_dir_all(transaction) { - Err(error) if error.kind() != io::ErrorKind::NotFound => Err(store_error(format!( - "cannot retire {TRANSACTION_WHAT} {}", - transaction.display() - ))(error)), - _ => Ok(()), - } -} - -/// Renames the directory `from` to `to` unless `to` exists, which it leaves as it is, as -/// [`platform::rename_no_replace`] does. On Windows, which renames no directory while a file below -/// it is open, as a scanner may hold one, a refused rename is tried again until `deadline`. -fn rename_until(from: &Path, to: &Path, deadline: Instant) -> Result<()> { - loop { - match platform::rename_no_replace(from, to) { - Ok(()) => return Ok(()), - Err(error) - if cfg!(windows) - && error.kind() == io::ErrorKind::PermissionDenied - && Instant::now() < deadline => - { - thread::sleep(RENAME_RETRY_INTERVAL); - } - Err(error) => { - return Err(store_error(format!( - "cannot publish {} as {}", - from.display(), - to.display() - ))(error)); - } - } - } -} - -/// Returns the store generation that follows `generation`. -fn following(generation: StoreGeneration) -> Result { - generation - .get() - .checked_add(1) - .map(StoreGeneration::new) - .ok_or_else(|| Error::unsupported("the template store has assigned every store generation")) -} - -/// What a checked capture records of itself, which the index lists of its generation once a -/// publication gives it a store generation. -struct Envelope { - /// The identity of the template's key. - key: KeyIdentity, - /// The identity of the capture's class. - class: ClassIdentity, - /// The capture's storage generation. - storage: StorageGeneration, - /// When OpenVMM created the snapshot. - created_at: SystemTime, - /// The time contract of the capture. - time: TimeContract, - /// The length of the saved state, in bytes. - state_len: u64, - /// The length of the RAM, in bytes. - memory_len: u64, -} - -impl Envelope { - /// Returns the envelope of a capture of the class `class`, with the storage generation - /// `storage`, whose key is `key`, and whose snapshot's manifest is `manifest`. - fn new( - key: &TemplateKey, - class: ClassIdentity, - storage: StorageGeneration, - manifest: &TemplateManifest, - ) -> Self { - Self { - key: key.identity(), - class, - storage, - created_at: manifest.created_at(), - time: manifest.time().clone(), - state_len: manifest.state_len(), - memory_len: manifest.memory_len(), - } - } - - /// Returns the generation of the envelope, as the store generation `generation`. - fn generation(&self, generation: StoreGeneration) -> Generation { - Generation { - generation, - key: self.key, - class: self.class, - storage: self.storage, - created_at: self.created_at, - time: self.time.clone(), - state_len: self.state_len, - memory_len: self.memory_len, - } - } -} - -/// The record of a capture, as JSON, which the capture writes in its transaction before its VM -/// starts: the store, the class, and the storage generation of the capture. -#[derive(Debug, Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -struct CaptureRecord { - /// [`STORE_FORMAT`]. - format: u32, - /// The identity of the store. - store: String, - /// The identity of the capture's class. - class: String, - /// The capture's storage generation. - storage_generation: String, -} - -impl CaptureRecord { - /// Returns the record of a capture of the class `class` in `store`, with the storage - /// generation `storage`. - fn new(store: &Store, class: ClassIdentity, storage: StorageGeneration) -> Self { - Self { - format: STORE_FORMAT, - store: store.id.to_string(), - class: class.to_string(), - storage_generation: storage.to_string(), - } - } -} diff --git a/nvx-rs/src/host/template/store/capture/tests.rs b/nvx-rs/src/host/template/store/capture/tests.rs deleted file mode 100644 index 42b20b1d9..000000000 --- a/nvx-rs/src/host/template/store/capture/tests.rs +++ /dev/null @@ -1,1439 +0,0 @@ -//! Tests of the captures of templates and of their publication, whose VMs a source in this -//! process emulates as OpenVMM runs them: captures of a class and the generations that they -//! publish, the checks of their files before their VMs start and of what their VMs leave, their -//! failures and crashes around each commit, and their recovery. The tests of -//! `tests/template_capture.rs` run captures through the supervisor, with the fake OpenVMM. - -use std::cell::{Cell, RefCell}; -use std::fs::OpenOptions; -use std::sync::atomic::{AtomicUsize, Ordering}; -use std::sync::{Arc, Barrier}; - -use serde_json::{Value, json}; - -use super::*; -use crate::host::sandbox::distro::{DISTRO_POLICY_REVISION, PolicyVerdict}; -use crate::host::sandbox::{SandboxDistro, SandboxUpper}; -use crate::host::snapshot::tests::{MIB, ManifestSpec, class, network_class}; -use crate::host::snapshot::{MANIFEST_NAME, SNAPSHOT_ARTIFACTS}; -use crate::host::store::{CallerFiles, JOURNAL_NAME, LaunchId}; -use crate::host::template::eligibility::MAX_DOWNTIME_NS; -use crate::host::template::store::Lease; -use crate::host::template::store::tests::{ - cause, link_dir, link_file, private_dir, publish_with, storage, store, -}; -use crate::protocol::root::RootImage; -use crate::uuid::Uuid; - -/// Bytes in a GiB. -const GIB: u64 = 1024 * MIB; - -/// Nanoseconds in a second. -const SECOND: u64 = 1_000_000_000; - -/// Name of the saved state in the emulated snapshot. -const STATE_NAME: &str = "state.bin"; -/// Name of the RAM in the emulated snapshot. -const MEMORY_NAME: &str = "memory.bin"; - -/// The identity of the placeholder. -const IDENTITY: &str = "11111111-1111-1111-1111-111111111111"; - -/// The access control list that OpenVMM gives a snapshot's directory and each of its files on -/// Windows: SYSTEM and the owner alone, and nothing that files inherit. -#[cfg(windows)] -const OPENVMM_ACCESS: &str = "D:P(A;;FA;;;SY)(A;;FA;;;OW)"; - -/// Returns a deadline that no capture of these tests reaches. -fn deadline() -> Instant { - Instant::now() + Duration::from_secs(60) -} - -/// Returns the host's time `seconds` after the captures of [`ManifestSpec::template`], on the -/// host and in the boot that captured them. -fn after(seconds: u64) -> HostTime { - let manifest = TemplateManifest::parse(&ManifestSpec::template().encode()).unwrap(); - let time = manifest.time(); - HostTime { - host_id: time.host_id, - boot_id: time.boot_id, - clock: time.host_clock, - monotonic_ns: time.capture_monotonic_ns + seconds * SECOND, - utc_ns: time.capture_utc_ns + seconds * SECOND, - } -} - -/// Returns the host's time `seconds` before the captures of [`ManifestSpec::template`], at -/// which none of them is eligible yet. -fn before(seconds: u64) -> HostTime { - let anchor = after(0); - HostTime { - monotonic_ns: anchor.monotonic_ns - seconds * SECOND, - utc_ns: anchor.utc_ns - seconds * SECOND, - ..anchor - } -} - -/// Creates the sparse file `path` of `length` bytes, which no one else may write, whatever the -/// umask. -fn sparse(path: &Path, length: u64) { - let mut options = OpenOptions::new(); - options.write(true).create_new(true); - #[cfg(unix)] - std::os::unix::fs::OpenOptionsExt::mode(&mut options, 0o600); - let file = options.open(path).unwrap(); - platform::make_sparse(&file).unwrap(); - file.set_len(length).unwrap(); -} - -/// Lets anyone change the file or directory `path`. -fn let_anyone_write(path: &Path) { - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - - let mode = if path.is_dir() { 0o777 } else { 0o666 }; - fs::set_permissions(path, fs::Permissions::from_mode(mode)).unwrap(); - } - #[cfg(windows)] - platform::tests::set_access(path, "D:P(A;OICI;FA;;;OW)(A;OICI;FA;;;SY)(A;OICI;FA;;;WD)") - .unwrap(); -} - -/// Returns the names of the entries of `dir`, sorted. -fn sorted(dir: &Path) -> Vec { - let mut names = entries(dir).unwrap(); - names.sort(); - names -} - -/// Returns the file at `path` as JSON. -fn json_of(path: &Path) -> Value { - serde_json::from_slice(&fs::read(path).unwrap()).unwrap() -} - -/// What the emulated VM of a capture does, as OpenVMM would: it stages a snapshot of the -/// manifest that it publishes beside the destination, with the RAM as a link to the file that -/// backed it, renames it to the destination, and writes the outcome report. -#[derive(Clone)] -struct Emulation { - /// The manifest of the snapshot that it publishes, if it publishes one. - manifest: Option, - /// The operation, category, and status code that its outcome report records, if it writes - /// one. - outcome: Option<(&'static str, &'static str, i64)>, - /// An entry that it publishes in the snapshot besides its artifacts. - extra: Option<&'static str>, - /// An artifact that it leaves out of the snapshot. - missing: Option<&'static str>, - /// The length of the saved state that it publishes, if not the manifest's. - state_len: Option, - /// Whether it leaves the file that backed the RAM, and a staging directory, as OpenVMM leaves - /// them when it ends abruptly. - leftovers: bool, -} - -impl Emulation { - /// Returns the emulation of a capture of [`ManifestSpec::template`] that succeeds. - fn capture() -> Self { - Self { - manifest: Some(ManifestSpec::template()), - outcome: Some(("capture", "success", 0)), - extra: None, - missing: None, - state_len: None, - leftovers: false, - } - } - - /// Returns the emulation of a VM whose guest failed with 3 before it captured. - fn guest_failure() -> Self { - Self { - manifest: None, - outcome: Some(("capture", "guest-exit", 3)), - ..Self::capture() - } - } - - /// Runs the VM of `vm` from the sandbox store of `locked`, the capture's transaction, after - /// removing the outcome report of the store's previous VM, as a launch does. - fn run(&self, locked: &mut LockedStore, vm: &Vm<'_>) -> Result<()> { - // OpenVMM requires the control socket of a VM that captures a snapshot, on Linux, beside - // the snapshot's destination: the VM launches from its transaction. - assert_eq!(locked.dir(), vm.template.transaction()); - assert_eq!( - locked.socket_path().parent(), - vm.template.destination().parent() - ); - remove_if_present(&locked.outcome_path())?; - let transaction = vm.template.transaction(); - let ram = transaction.join(format!("{MEMORY_FILE_PREFIX}emulated")); - sparse(&ram, vm.capture.class.memory); - if let Some(spec) = &self.manifest { - let staging = transaction.join(format!("{STAGING_PREFIX}1-0-0")); - private_dir(&staging); - fs::write(staging.join(MANIFEST_NAME), spec.encode()).unwrap(); - sparse( - &staging.join(STATE_NAME), - self.state_len.unwrap_or(spec.state_size), - ); - fs::hard_link(&ram, staging.join(MEMORY_NAME)).unwrap(); - if let Some(extra) = self.extra { - fs::write(staging.join(extra), b"").unwrap(); - } - if let Some(missing) = self.missing { - fs::remove_file(staging.join(missing)).unwrap(); - } - #[cfg(windows)] - for name in entries(&staging).unwrap() { - platform::tests::set_access(&staging.join(name), OPENVMM_ACCESS).unwrap(); - } - #[cfg(windows)] - platform::tests::set_access(&staging, OPENVMM_ACCESS).unwrap(); - fs::rename(&staging, vm.template.destination()).unwrap(); - } - if self.leftovers { - private_dir(&transaction.join(format!("{STAGING_PREFIX}1-0-1"))); - } else { - fs::remove_file(&ram).unwrap(); - } - if let Some((operation, category, status_code)) = self.outcome { - let report = json!({ - "schema_version": 1, - "backend": vm.capture.class.hypervisor.as_str(), - "outcome": { - "operation": operation, - "category": category, - "status_code": status_code, - }, - "teardown": { "vm_stopped": true }, - }); - fs::write(locked.outcome_path(), report.to_string()).unwrap(); - } - Ok(()) - } -} - -/// A store, and the inputs of captures into it, of [`class`], whose VMs [`Emulation`]s run. -struct Bench { - /// The temporary directory that holds the store and the platform's files. - _directory: tempfile::TempDir, - /// The store. - store: Store, - /// The inputs of a capture. - capture: Capture, - /// The number of VMs that captures ran. - runs: Cell, -} - -impl Bench { - /// Returns a bench whose placeholder is a regular file, in a directory private to the current - /// user. - fn new() -> Self { - Self::with(Path::to_path_buf) - } - - /// Returns a bench whose placeholder is at the path that `prepare` returns, given a sparse - /// file of 1 GiB in a directory private to the current user, which it may change before the - /// placeholder's binding is made. - fn with(prepare: impl FnOnce(&Path) -> PathBuf) -> Self { - let (directory, store) = store(); - let platform = directory.path().join("platform"); - private_dir(&platform); - let image = platform.join("placeholder.erofs"); - sparse(&image, GIB); - let placeholder = prepare(&image); - let path = placeholder.to_str().unwrap(); - let layer = SandboxLayer::new(LayerRole::Distro, path, IDENTITY); - let binding = DistroBinding::import(&layer, PolicyVerdict::ACCEPTED).unwrap(); - let image = RootImage::erofs(Some(Uuid::parse(IDENTITY).unwrap())); - let upper = SandboxUpper::scratch(platform.join("clone.ext4").to_str().unwrap()); - let capture = Capture { - class: class(), - adapter: SandboxPlan::adapter(SandboxDistro::new(path, image), upper), - binding, - policy_revision: DISTRO_POLICY_REVISION, - artifacts: Artifacts::new( - platform.join("openvmm"), - platform.join("vmlinux"), - platform.join("initramfs.cpio.gz"), - ), - }; - Self { - _directory: directory, - store, - capture, - runs: Cell::new(0), - } - } - - /// Returns the slot of the class of the captures. - fn slot(&self) -> PathBuf { - self.store - .captures_dir() - .join(slot_name(self.capture.class.identity().unwrap())) - } - - /// Returns the transaction of the captures. - fn transaction(&self) -> PathBuf { - self.slot().join(TRANSACTION_NAME) - } - - /// Serves the capture as [`Store::create_template`] does with the host's time that `now` - /// returns, or, without it, as [`Store::refresh_template`] does, with `emulation` as its VM, - /// failing it at the points at which `interrupt` fails, and crashing it if `crash`. - fn serve( - &self, - now: Option<&dyn Fn() -> Result>, - emulation: &Emulation, - interrupt: &dyn Fn(Point) -> Result<()>, - crash: bool, - ) -> Result { - self.serve_with( - now, - &|locked, vm| emulation.run(locked, vm), - interrupt, - crash, - ) - } - - /// Serves the capture as [`serve`](Self::serve) does, with `run` as its source. - fn serve_with( - &self, - now: Option<&dyn Fn() -> Result>, - run: &dyn Fn(&mut LockedStore, &Vm<'_>) -> Result<()>, - interrupt: &dyn Fn(Point) -> Result<()>, - crash: bool, - ) -> Result { - let counted = |locked: &mut LockedStore, vm: &Vm<'_>| { - self.runs.set(self.runs.get() + 1); - run(locked, vm) - }; - let hooks = Hooks { - sync: &sync_dir, - run: &counted, - interrupt, - crash, - }; - self.store.capture(&self.capture, now, deadline(), &hooks) - } - - /// Serves the capture as [`Store::create_template`] does, a second after the captures. - fn create(&self, emulation: &Emulation) -> Result { - self.serve(Some(&|| Ok(after(1))), emulation, &uninterrupted, false) - } - - /// Serves the capture as [`Store::refresh_template`] does. - fn refresh(&self, emulation: &Emulation) -> Result { - self.serve(None, emulation, &uninterrupted, false) - .map(Created::into_generation) - } - - /// Refreshes the capture, failing it at `point`, or crashing it there if `crash`, and returns - /// the error. - fn interrupted(&self, point: Point, crash: bool) -> Error { - let interrupt = |at: Point| { - if at == point { - Err(Error::other(format!("failed at {at:?}"))) - } else { - Ok(()) - } - }; - self.serve(None, &Emulation::capture(), &interrupt, crash) - .unwrap_err() - } - - /// Recovers the store's captures. - fn recover(&self) -> Result> { - self.recover_syncing(&sync_dir) - } - - /// Recovers the store's captures, syncing directories through `sync`. - fn recover_syncing(&self, sync: SyncDir<'_>) -> Result> { - let run = |_: &mut LockedStore, _: &Vm<'_>| -> Result<()> { - unreachable!("a recovery runs no VM") - }; - let hooks = Hooks { - sync, - run: &run, - interrupt: &uninterrupted, - crash: false, - }; - self.store.recover_with(deadline(), &hooks) - } - - /// Returns the store generations that the index lists. - fn indexed(&self) -> Vec { - numbers(self.store.index().unwrap().generations()) - } - - /// Drops the index's last generation and keeps its payload, as a crash leaves a publication - /// that renamed its envelope into the store before it committed the index. Captures complete - /// such publications before they run, so tests that need one beside a checked transaction - /// make it this way. - fn unindex_last(&self) { - let mut index = self.store.index().unwrap(); - let last = index.generations.pop().unwrap(); - index.next = last.generation(); - self.store.write_index(&index).unwrap(); - } -} - -/// Returns the store generation `generation`, which must not be 0. -fn store_generation(generation: u64) -> StoreGeneration { - StoreGeneration::new(std::num::NonZeroU64::new(generation).unwrap()) -} - -/// Returns the store generations of `generations`, as numbers. -fn numbers(generations: &[Generation]) -> Vec { - generations - .iter() - .map(|generation| generation.generation().get().get()) - .collect() -} - -/// Returns the files that the sandboxes of these tests keep: a configuration record. -fn files() -> CallerFiles { - CallerFiles::new("config.json").unwrap() -} - -/// Provisions the sandbox `name` in `parent`, and returns its directory. -fn provision(parent: &Path, name: &str) -> PathBuf { - let dir = parent.join(name); - let locked = SandboxStore::lock_for_provision(&dir, &files(), None).unwrap(); - fs::write(dir.join("config.json"), b"{}").unwrap(); - drop(locked); - dir -} - -/// Asserts that the slot of `bench` holds no transaction, and records no launch. -fn assert_clean(bench: &Bench) { - assert!(!bench.transaction().exists()); - assert!(!bench.slot().join(JOURNAL_NAME).exists()); -} - -#[test] -fn captures_publish_generations_that_creations_then_serve() { - let bench = Bench::new(); - let Created::Captured(first) = bench.create(&Emulation::capture()).unwrap() else { - panic!("the first creation of a class captures it"); - }; - assert_eq!(bench.runs.get(), 1); - assert_eq!(first.generation(), store_generation(1)); - let class = class(); - let manifest = TemplateManifest::parse(&ManifestSpec::template().encode()).unwrap(); - let key = TemplateKey::new(&class, &manifest).unwrap(); - assert_eq!(first.key(), key.identity()); - assert_eq!(first.class(), class.identity().unwrap()); - assert_eq!(first.created_at(), manifest.created_at()); - assert_eq!(first.time(), manifest.time()); - assert_eq!( - (first.state_len(), first.memory_len()), - (manifest.state_len(), manifest.memory_len()) - ); - assert_eq!( - bench.store.index().unwrap().generations(), - std::slice::from_ref(&first) - ); - // The payload is the envelope: OpenVMM's snapshot, with the capture's record and the - // template's key beside it. - let payload = bench.store.payload_dir(first.generation()); - assert_eq!( - sorted(&payload), - [CAPTURE_RECORD_NAME, KEY_NAME, SNAPSHOT_NAME] - ); - assert_eq!(sorted(&payload.join(SNAPSHOT_NAME)), SNAPSHOT_ARTIFACTS); - assert_eq!(bench.store.read_key(&first).unwrap(), key); - assert_eq!( - bench.store.snapshot_dir(&first).unwrap(), - payload.join(SNAPSHOT_NAME) - ); - assert_eq!( - json_of(&payload.join(CAPTURE_RECORD_NAME)), - json!({ - "format": STORE_FORMAT, - "store": bench.store.id().to_string(), - "class": class.identity().unwrap().to_string(), - "storageGeneration": first.storage_generation().to_string(), - }) - ); - assert_clean(&bench); - // A creation serves the class with the generation that it holds, and runs no VM. - assert_eq!( - bench.create(&Emulation::capture()).unwrap(), - Created::Existing(first.clone()) - ); - assert_eq!(bench.runs.get(), 1); - // A refresh always captures, with a fresh storage generation, and creations then serve the - // new generation. - let second = bench.refresh(&Emulation::capture()).unwrap(); - assert_eq!(bench.runs.get(), 2); - assert_eq!(second.generation(), store_generation(2)); - assert_ne!(second.storage_generation(), first.storage_generation()); - assert_eq!(second.key(), first.key()); - assert_eq!( - bench.create(&Emulation::capture()).unwrap(), - Created::Existing(second.clone()) - ); - assert_eq!(bench.indexed(), [1, 2]); - assert_eq!(bench.store.read_key(&first).unwrap(), key); -} - -#[test] -fn captures_limit_the_processors_that_their_class_keeps_offline_at_boot() { - let mut bench = Bench::new(); - bench.capture.class.processors = 4; - bench.capture.class.boot_processors = 1; - // OpenVMM records the processors online at boot that the command line limits. - let mut spec = ManifestSpec::template(); - spec.vp_count = 4; - let contract = spec.contract(); - contract.topology = (1, 1, 4, 1, vec![0, 1, 2, 3]); - contract.boot_online_vp_count = 1; - let emulation = Emulation { - manifest: Some(spec), - ..Emulation::capture() - }; - let arguments = RefCell::new(Vec::new()); - let run = |locked: &mut LockedStore, vm: &Vm<'_>| { - let command = vm.command("control", Path::new("outcome.json"), locked.dir())?; - arguments.replace(command.arguments); - emulation.run(locked, vm) - }; - let created = bench.serve_with(Some(&|| Ok(after(1))), &run, &uninterrupted, false); - let Created::Captured(generation) = created.unwrap() else { - panic!("the first creation of a class captures it"); - }; - // The VM has the class's processors, and boots with those that it keeps online. - let arguments: Vec = arguments - .take() - .into_iter() - .map(|argument| argument.into_string().unwrap()) - .collect(); - let value = |option: &str| { - let at = arguments.iter().position(|given| given == option).unwrap(); - arguments[at + 1].as_str() - }; - assert_eq!(value("--processors"), "4"); - assert_eq!(value("--cmdline"), "nvx_sandbox=1 maxcpus=1"); - let key = bench.store.read_key(&generation).unwrap(); - assert_eq!((key.processors(), key.boot_processors()), (4, 1)); -} - -#[test] -fn refreshes_leave_older_generations_and_their_leases_untouched() { - let bench = Bench::new(); - let first = bench.refresh(&Emulation::capture()).unwrap(); - let sandboxes = tempfile::tempdir().unwrap(); - let sandbox = provision(sandboxes.path(), "clone"); - let lease = bench - .store - .acquire(first.generation(), &sandbox, &files(), &after(1), None) - .unwrap() - .lease; - let payload = bench.store.payload_dir(first.generation()); - let before = (sorted(&payload), bench.store.read_key(&first).unwrap()); - let second = bench.refresh(&Emulation::capture()).unwrap(); - assert_eq!(bench.indexed(), [1, 2]); - assert_eq!( - bench.store.index().unwrap().get(first.generation()), - Some(&first) - ); - assert_eq!( - (sorted(&payload), bench.store.read_key(&first).unwrap()), - before - ); - assert_eq!( - bench.store.leases(first.generation()).unwrap(), - std::slice::from_ref(&lease) - ); - assert!(bench.store.leases(second.generation()).unwrap().is_empty()); - assert_eq!( - Lease::of(&SandboxStore::open(&sandbox).unwrap()).unwrap(), - Some(lease) - ); -} - -#[test] -fn creations_capture_unless_the_class_has_a_generation_that_restores_take() { - let bench = Bench::new(); - // A generation of another class serves none of this one's. - let other = publish_with( - &bench.store, - &network_class(), - &ManifestSpec::with_network(), - storage(9), - ); - let Created::Captured(first) = bench.create(&Emulation::capture()).unwrap() else { - panic!("a class without generations is captured"); - }; - assert_eq!(first.generation(), store_generation(2)); - // A quarantined generation serves no creation. - bench.store.tombstone(first.generation(), None).unwrap(); - let Created::Captured(second) = bench.create(&Emulation::capture()).unwrap() else { - panic!("a class whose generations are quarantined is captured"); - }; - assert_eq!(second.generation(), store_generation(3)); - // Neither does one that no restore takes at the host's time. - let late = HostTime { - monotonic_ns: after(0).monotonic_ns + MAX_DOWNTIME_NS + SECOND, - ..after(0) - }; - assert!(!second.eligibility(&late).is_eligible()); - let Created::Captured(third) = bench - .serve( - Some(&|| Ok(late)), - &Emulation::capture(), - &uninterrupted, - false, - ) - .unwrap() - else { - panic!("a class whose generations are ineligible is captured"); - }; - assert_eq!(third.generation(), store_generation(4)); - assert_eq!( - bench.create(&Emulation::capture()).unwrap(), - Created::Existing(third) - ); - assert_eq!(bench.runs.get(), 3); - assert_eq!( - bench.store.index().unwrap().get(other.generation()), - Some(&other) - ); -} - -#[test] -fn failures_before_the_check_retire_the_transaction() { - let bench = Bench::new(); - for (point, runs) in [(Point::Prepared, 0), (Point::Exited, 1)] { - let error = bench.interrupted(point, false); - assert_eq!(error.message(), format!("failed at {point:?}")); - assert_eq!(bench.runs.replace(0), runs, "{point:?}"); - assert_clean(&bench); - } - let failures = [ - ( - Emulation::guest_failure(), - ErrorKind::Exited, - "records the outcome guest-exit with status 3 of the operation capture", - ), - // OpenVMM may fail once it has published the snapshot, as when it cannot make the - // publication durable. - ( - Emulation { - outcome: Some(("capture", "vmm-failure", 1)), - ..Emulation::capture() - }, - ErrorKind::Exited, - "records the outcome vmm-failure with status 1", - ), - ( - Emulation { - outcome: None, - ..Emulation::capture() - }, - ErrorKind::Exited, - "ended without the outcome report", - ), - ( - Emulation { - outcome: Some(("run", "success", 0)), - ..Emulation::capture() - }, - ErrorKind::Exited, - "of the operation run", - ), - ( - Emulation { - manifest: None, - ..Emulation::capture() - }, - ErrorKind::Io, - "cannot open the snapshot of the capture", - ), - ( - Emulation { - missing: Some(MANIFEST_NAME), - ..Emulation::capture() - }, - ErrorKind::Io, - "cannot read the snapshot manifest", - ), - ( - Emulation { - missing: Some(MEMORY_NAME), - ..Emulation::capture() - }, - ErrorKind::Io, - "holds manifest.bin, state.bin, not exactly the manifest", - ), - ( - Emulation { - manifest: Some(ManifestSpec::with_network()), - ..Emulation::capture() - }, - ErrorKind::Io, - "the capture does not match its class", - ), - ( - Emulation { - extra: Some("scratch.img"), - ..Emulation::capture() - }, - ErrorKind::Io, - "manifest.bin, memory.bin, scratch.img, state.bin, not exactly the manifest", - ), - ( - Emulation { - state_len: Some(ManifestSpec::template().state_size - 512), - ..Emulation::capture() - }, - ErrorKind::Io, - "bytes that its manifest records", - ), - ]; - for (emulation, kind, message) in failures { - let error = bench.refresh(&emulation).unwrap_err(); - assert_eq!(error.kind(), kind, "{error}"); - assert!(error.message().contains(message), "{error}"); - assert_eq!(bench.runs.replace(0), 1); - assert_clean(&bench); - // The slot keeps the outcome report of the latest capture, and none of an earlier one. - assert_eq!( - bench.slot().join(OUTCOME_NAME).exists(), - emulation.outcome.is_some() - ); - } - assert!(bench.indexed().is_empty()); -} - -#[test] -fn what_openvmm_leaves_beside_the_snapshot_is_removed() { - let bench = Bench::new(); - let emulation = Emulation { - leftovers: true, - ..Emulation::capture() - }; - let generation = bench.refresh(&emulation).unwrap(); - assert_eq!( - sorted(&bench.store.payload_dir(generation.generation())), - [CAPTURE_RECORD_NAME, KEY_NAME, SNAPSHOT_NAME] - ); - assert_clean(&bench); -} - -#[test] -fn checked_transactions_outlive_failures_until_recovery_publishes_them() { - let bench = Bench::new(); - bench.interrupted(Point::Checked, false); - assert_eq!( - sorted(&bench.transaction()), - [CAPTURE_RECORD_NAME, KEY_NAME, SNAPSHOT_NAME] - ); - assert!(bench.indexed().is_empty()); - let published = bench.recover().unwrap(); - assert_eq!(published.len(), 1); - assert_eq!(bench.store.index().unwrap().generations(), published); - assert_clean(&bench); - // The next capture of the class publishes it too, and then serves it. - let bench = Bench::new(); - bench.interrupted(Point::Checked, false); - let Created::Existing(generation) = bench.create(&Emulation::capture()).unwrap() else { - panic!("a creation serves the generation that it recovered"); - }; - assert_eq!(generation.generation(), store_generation(1)); - assert_eq!(bench.runs.get(), 1); - assert_clean(&bench); -} - -#[test] -fn publications_interrupted_once_they_renamed_their_envelope_are_completed() { - let bench = Bench::new(); - bench.interrupted(Point::Renamed, false); - assert!(bench.indexed().is_empty()); - assert_eq!( - sorted(&bench.store.payload_dir(store_generation(1))), - [CAPTURE_RECORD_NAME, KEY_NAME, SNAPSHOT_NAME] - ); - // The next capture completes it first. - let generation = bench.refresh(&Emulation::capture()).unwrap(); - assert_eq!(generation.generation(), store_generation(2)); - assert_eq!(bench.indexed(), [1, 2]); - // So does a recovery. - bench.interrupted(Point::Renamed, false); - let published = bench.recover().unwrap(); - assert_eq!(published.len(), 1); - assert_eq!(published[0].generation(), store_generation(3)); - assert_eq!(bench.indexed(), [1, 2, 3]); - // A recovery that completes one before it publishes a checked transaction returns both. - bench.interrupted(Point::Checked, false); - bench.unindex_last(); - assert_eq!(bench.indexed(), [1, 2]); - assert_eq!(numbers(&bench.recover().unwrap()), [3, 4]); - assert_eq!(bench.indexed(), [1, 2, 3, 4]); - assert_clean(&bench); -} - -#[test] -fn recoveries_make_what_they_publish_durable_before_they_index_it() { - let bench = Bench::new(); - bench.refresh(&Emulation::capture()).unwrap(); - bench.interrupted(Point::Checked, false); - bench.unindex_last(); - // Each directory that the recovery syncs, with what the index lists as it does. - let synced = RefCell::new(Vec::new()); - let sync = |dir: &Path| -> Result<()> { - synced - .borrow_mut() - .push((dir.to_path_buf(), bench.indexed())); - sync_dir(dir) - }; - assert_eq!(numbers(&bench.recover_syncing(&sync).unwrap()), [1, 2]); - let generations = bench.store.dir.join(GENERATIONS_NAME); - let parent = bench.store.dir.parent().unwrap().to_path_buf(); - assert_eq!( - synced.into_inner(), - [ - // The transaction that the crash left checked. - (bench.transaction(), vec![]), - // The publication that the crash interrupted once it had renamed its envelope. - (bench.store.payload_dir(store_generation(1)), vec![]), - (generations.clone(), vec![]), - (bench.slot(), vec![]), - // The transaction's publication. - (generations, vec![]), - (bench.slot(), vec![]), - (bench.store.dir.clone(), vec![1, 2]), - (parent, vec![1, 2]), - ] - ); - assert_clean(&bench); -} - -#[test] -fn captures_make_the_store_durable_before_they_report_a_generation() { - let bench = Bench::new(); - // Each directory that the capture syncs, with what the index lists as it does. - let synced = RefCell::new(Vec::new()); - let sync = |dir: &Path| -> Result<()> { - synced - .borrow_mut() - .push((dir.to_path_buf(), bench.indexed())); - sync_dir(dir) - }; - let run = |locked: &mut LockedStore, vm: &Vm<'_>| Emulation::capture().run(locked, vm); - let hooks = Hooks { - sync: &sync, - run: &run, - interrupt: &uninterrupted, - crash: false, - }; - let created = bench - .store - .capture(&bench.capture, None, deadline(), &hooks) - .unwrap(); - assert_eq!(created.generation().generation(), store_generation(1)); - let parent = bench.store.dir.parent().unwrap().to_path_buf(); - assert_eq!( - synced.into_inner(), - [ - // The checked transaction, and every directory above it, up to the store's parent, - // which the store's creation did not sync. - (bench.transaction(), vec![]), - (bench.slot(), vec![]), - (bench.store.captures_dir(), vec![]), - (bench.store.dir.clone(), vec![]), - (parent.clone(), vec![]), - // The rename into the store. - (bench.store.dir.join(GENERATIONS_NAME), vec![]), - (bench.slot(), vec![]), - // The index. - (bench.store.dir.clone(), vec![1]), - (parent, vec![1]), - ] - ); -} - -#[test] -fn creations_recover_the_index_before_they_look_for_a_generation() { - // A creation completes a publication that a crash interrupted, and serves its generation, - // the class's latest. - let bench = Bench::new(); - bench.refresh(&Emulation::capture()).unwrap(); - bench.interrupted(Point::Renamed, false); - assert_eq!(bench.indexed(), [1]); - let Created::Existing(generation) = bench.create(&Emulation::capture()).unwrap() else { - panic!("a creation serves the generation whose publication it completed"); - }; - assert_eq!(generation.generation(), store_generation(2)); - assert_eq!(bench.indexed(), [1, 2]); - assert_eq!(bench.runs.get(), 2); - // It serves no generation whose payload a crash lost, and captures one in its place. - let bench = Bench::new(); - bench.refresh(&Emulation::capture()).unwrap(); - fs::remove_dir_all(bench.store.payload_dir(store_generation(1))).unwrap(); - let Created::Captured(generation) = bench.create(&Emulation::capture()).unwrap() else { - panic!("a creation captures in place of a generation whose payload was lost"); - }; - assert_eq!(generation.generation(), store_generation(2)); - assert_eq!(bench.indexed(), [2]); - // A store whose index is full of generations whose payloads a crash lost still captures. - let bench = Bench::new(); - let first = bench.refresh(&Emulation::capture()).unwrap(); - let last = MAX_GENERATIONS as u64; - let mut index = bench.store.index().unwrap(); - index - .generations - .extend((2..=last).map(|generation| Generation { - generation: store_generation(generation), - ..first.clone() - })); - index.next = store_generation(last + 1); - bench.store.write_index(&index).unwrap(); - assert_eq!(bench.indexed().len(), MAX_GENERATIONS); - let generation = bench.refresh(&Emulation::capture()).unwrap(); - assert_eq!(generation.generation(), store_generation(last + 1)); - assert_eq!(bench.indexed(), [1, last + 1]); - assert_clean(&bench); -} - -#[test] -fn recoveries_drop_lost_generations_before_they_index_another() { - // A store whose index is full of generations that a crash lost, but one, beside a publication - // that a crash interrupted once it had renamed its envelope. - let bench = Bench::new(); - let first = bench.refresh(&Emulation::capture()).unwrap(); - bench.interrupted(Point::Renamed, false); - let last = MAX_GENERATIONS as u64; - fs::rename( - bench.store.payload_dir(store_generation(2)), - bench.store.payload_dir(store_generation(last + 1)), - ) - .unwrap(); - let mut index = bench.store.index().unwrap(); - index - .generations - .extend((2..=last).map(|generation| Generation { - generation: store_generation(generation), - ..first.clone() - })); - index.next = store_generation(last + 1); - bench.store.write_index(&index).unwrap(); - assert_eq!(bench.indexed().len(), MAX_GENERATIONS); - // The lost generations take none of the store's capacity from the publication. - assert_eq!(numbers(&bench.recover().unwrap()), [last + 1]); - assert_eq!(bench.indexed(), [1, last + 1]); - assert_clean(&bench); -} - -#[test] -fn creations_make_the_store_durable_before_they_serve_a_generation() { - let bench = Bench::new(); - let parent = bench.store.dir.parent().unwrap().to_path_buf(); - let run = |locked: &mut LockedStore, vm: &Vm<'_>| Emulation::capture().run(locked, vm); - // A capture whose host crashes as it syncs the store's parent, once the index lists the - // capture's generation. - let crashing = |dir: &Path| -> Result<()> { - if dir == parent && !bench.indexed().is_empty() { - return Err(Error::other("the host crashed")); - } - sync_dir(dir) - }; - let hooks = Hooks { - sync: &crashing, - run: &run, - interrupt: &uninterrupted, - crash: true, - }; - bench - .store - .capture(&bench.capture, None, deadline(), &hooks) - .unwrap_err(); - assert_eq!(bench.indexed(), [1]); - // A creation serves the generation once it has made the index, and the store's own entry in - // its parent, durable, although no recovery changed the index. - let synced = RefCell::new(Vec::new()); - let sync = |dir: &Path| -> Result<()> { - synced.borrow_mut().push(dir.to_path_buf()); - sync_dir(dir) - }; - let hooks = Hooks { - sync: &sync, - run: &run, - interrupt: &uninterrupted, - crash: false, - }; - let now = || -> Result { Ok(after(1)) }; - let Created::Existing(generation) = bench - .store - .capture(&bench.capture, Some(&now), deadline(), &hooks) - .unwrap() - else { - panic!("a creation serves the generation that the index lists"); - }; - assert_eq!(generation.generation(), store_generation(1)); - assert_eq!(synced.into_inner(), [bench.store.dir.clone(), parent]); - assert_clean(&bench); -} - -#[test] -fn recovery_ends_the_captures_that_crashes_interrupt() { - for (point, left, indexed) in [ - ( - Point::Prepared, - vec![CAPTURE_RECORD_NAME, LOCK_NAME, SCRATCH_NAME], - vec![], - ), - ( - Point::Exited, - vec![CAPTURE_RECORD_NAME, LOCK_NAME, SCRATCH_NAME, SNAPSHOT_NAME], - vec![], - ), - ( - Point::Checked, - vec![CAPTURE_RECORD_NAME, KEY_NAME, SNAPSHOT_NAME], - vec![1], - ), - (Point::Renamed, vec![], vec![1]), - (Point::Indexed, vec![], vec![1]), - ] { - let bench = Bench::new(); - bench.interrupted(point, true); - if left.is_empty() { - assert!(!bench.transaction().exists(), "{point:?}"); - } else { - assert_eq!(sorted(&bench.transaction()), left, "{point:?}"); - } - let published = bench.recover().unwrap(); - let recovered: Vec = published - .iter() - .map(|generation| generation.generation().get().get()) - .collect(); - let expected = if matches!(point, Point::Checked | Point::Renamed) { - vec![1] - } else { - vec![] - }; - assert_eq!(recovered, expected, "{point:?}"); - assert_eq!(bench.indexed(), indexed, "{point:?}"); - assert_clean(&bench); - // The slot captures again. - bench.refresh(&Emulation::capture()).unwrap(); - } -} - -#[test] -fn generations_whose_payloads_are_lost_are_dropped_from_the_index() { - let bench = Bench::new(); - let first = bench.refresh(&Emulation::capture()).unwrap(); - bench.refresh(&Emulation::capture()).unwrap(); - fs::remove_dir_all(bench.store.payload_dir(first.generation())).unwrap(); - bench.refresh(&Emulation::capture()).unwrap(); - assert_eq!(bench.indexed(), [2, 3]); - fs::remove_dir_all(bench.store.payload_dir(store_generation(3))).unwrap(); - assert!(bench.recover().unwrap().is_empty()); - assert_eq!(bench.indexed(), [2]); - // The index never assigns a store generation twice. - assert_eq!( - bench.store.index().unwrap().next_generation(), - store_generation(4) - ); -} - -#[test] -fn a_transaction_stays_while_it_records_a_launch() { - let bench = Bench::new(); - // A source that fails while its launch remains, as one whose VMM may still run: dropping a - // launch leaves its journal, as a crash would. - let failed = |locked: &mut LockedStore, vm: &Vm<'_>| -> Result<()> { - let capability = [7; crate::protocol::CAPABILITY_LEN]; - let launch = LaunchId::generate()?; - drop(locked.begin_launch(&launch, &vm.capture.class.runtime, &capability)?); - Err(Error::other("the VMM may still run")) - }; - let error = bench - .serve_with(None, &failed, &uninterrupted, false) - .unwrap_err(); - assert_eq!(error.message(), "the VMM may still run"); - // The transaction's sandbox store records the launch, and the slot none. - assert!(bench.transaction().join(JOURNAL_NAME).exists()); - assert!(!bench.slot().join(JOURNAL_NAME).exists()); - let left = sorted(&bench.transaction()); - for name in [CAPTURE_RECORD_NAME, SCRATCH_NAME, LOCK_NAME, LOG_NAME] { - assert!(left.iter().any(|left| left == name), "{name}: {left:?}"); - } - // The next capture ends the launch, which nothing holds, retires the transaction, and - // captures. - bench.refresh(&Emulation::capture()).unwrap(); - assert_clean(&bench); - assert_eq!(bench.indexed(), [1]); -} - -/// Returns whether this host lets the tests create a symbolic link to a file: Windows lets only -/// Developer Mode or a privilege create one. -fn can_link_files() -> bool { - let directory = tempfile::tempdir().unwrap(); - let target = directory.path().join("target"); - fs::write(&target, b"").unwrap(); - link_file(&target, &directory.path().join("link")) -} - -/// Asserts that `result` failed with `kind`, with a message that holds `message`, and that its -/// capture ran no VM, and left no transaction. -fn assert_refused(bench: &Bench, result: Result, kind: ErrorKind, message: &str) { - let error = result.unwrap_err(); - assert_eq!(error.kind(), kind, "{error}"); - assert!( - error.message().contains(message) || cause(&error).contains(message), - "{error}: {}", - cause(&error) - ); - assert_eq!(bench.runs.replace(0), 0); - assert!(bench.indexed().is_empty()); -} - -#[test] -fn links_and_unsafe_ownership_are_refused_before_the_vm_starts() { - // A placeholder that is a link, which OpenVMM would not follow, though the check of its - // binding follows it. - if can_link_files() { - let bench = Bench::with(|image| { - let link = image.with_file_name("link.erofs"); - assert!(link_file(image, &link)); - link - }); - let result = bench.create(&Emulation::capture()); - assert_refused( - &bench, - result, - ErrorKind::Unavailable, - "it is a link, or not a regular file", - ); - assert_clean(&bench); - } - // A placeholder that others may change. - let bench = Bench::with(|image| { - let_anyone_write(image); - image.to_path_buf() - }); - let result = bench.create(&Emulation::capture()); - let message = if cfg!(windows) { - "allows other users to change it" - } else { - "is writable by other users" - }; - assert_refused(&bench, result, ErrorKind::Unavailable, message); - assert_clean(&bench); - // A scratch that a link replaced, or that others may change. - let bench = Bench::new(); - let scratch = bench.transaction().join(SCRATCH_NAME); - let replaced = |point: Point| -> Result<()> { - if point == Point::Prepared { - let target = bench.slot().join("elsewhere.raw"); - sparse(&target, class().scratch.length); - fs::remove_file(&scratch).unwrap(); - assert!(link_file(&target, &scratch)); - } - Ok(()) - }; - if can_link_files() { - let result = bench.serve(None, &Emulation::capture(), &replaced, false); - assert_refused( - &bench, - result, - ErrorKind::Io, - "it is a link, or not a regular file", - ); - assert_clean(&bench); - } - let writable = |point: Point| -> Result<()> { - if point == Point::Prepared { - let_anyone_write(&scratch); - } - Ok(()) - }; - let result = bench.serve(None, &Emulation::capture(), &writable, false); - assert_refused(&bench, result, ErrorKind::Io, message); - assert_clean(&bench); - // A transaction that others may access, which the capture keeps rather than retire. - let exposed_message = if cfg!(windows) { - "allows other users" - } else { - "is accessible to other users" - }; - let transaction = bench.transaction(); - let exposed = |point: Point| -> Result<()> { - if point == Point::Prepared { - let_anyone_write(&transaction); - } - Ok(()) - }; - let result = bench.serve(None, &Emulation::capture(), &exposed, false); - assert_refused(&bench, result, ErrorKind::Io, exposed_message); - assert!(transaction.exists()); - let result = bench.create(&Emulation::capture()); - assert_refused(&bench, result, ErrorKind::Io, exposed_message); - fs::remove_dir_all(&transaction).unwrap(); - // A transaction that is a link. - let elsewhere = bench.slot().join("elsewhere"); - private_dir(&elsewhere); - link_dir(&elsewhere, &transaction); - let result = bench.create(&Emulation::capture()); - assert_refused(&bench, result, ErrorKind::Io, "is not a plain directory"); - assert!(fs::symlink_metadata(&transaction).is_ok()); -} - -#[test] -fn scratches_are_the_files_that_their_captures_created() { - // Another file of the scratch's length, renamed over the scratch. It is created while the - // scratch exists, so that the host cannot give it the scratch's identity again. - let bench = Bench::new(); - let scratch = bench.transaction().join(SCRATCH_NAME); - let replaced = |point: Point| -> Result<()> { - if point == Point::Prepared { - let other = bench.slot().join("other.raw"); - sparse(&other, class().scratch.length); - fs::rename(&other, &scratch).unwrap(); - } - Ok(()) - }; - let result = bench.serve(None, &Emulation::capture(), &replaced, false); - assert_refused( - &bench, - result, - ErrorKind::Io, - "is not the file that the capture created", - ); - assert_clean(&bench); - // A file whose ID differs from the scratch's only above its low 64 bits, as the 128-bit IDs - // of two files of a file system such as ReFS may: the check compares the whole ID. - let inputs = Inputs::check(&bench.capture).unwrap(); - let template = TemplatePlan::new( - &bench.capture.adapter, - &bench.capture.binding, - bench.capture.class.scratch.length, - inputs.resources, - &bench.transaction(), - StorageGeneration::generate().unwrap(), - ) - .unwrap(); - private_dir(&bench.transaction()); - let created = create_scratch(template.scratch_path(), template.scratch_length()).unwrap(); - check_scratch(&template, created).unwrap(); - let (device, file_id) = created; - let error = check_scratch(&template, (device, file_id ^ (1 << 64))).unwrap_err(); - assert!( - error - .message() - .contains("is not the file that the capture created"), - "{error}" - ); -} - -/// Returns whether this process holds the file at `path` open for writing. -#[cfg(target_os = "linux")] -fn held_for_writing(path: &Path) -> bool { - let target = fs::canonicalize(path).unwrap(); - fs::read_dir("/proc/self/fd") - .unwrap() - .filter_map(std::result::Result::ok) - .any(|entry| { - let flags = || { - let info = - fs::read_to_string(Path::new("/proc/self/fdinfo").join(entry.file_name())) - .ok()?; - let flags = info.lines().find_map(|line| line.strip_prefix("flags:"))?; - u32::from_str_radix(flags.trim(), 8).ok() - }; - fs::read_link(entry.path()).is_ok_and(|linked| linked == target) - && flags().is_some_and(|flags| flags & 0o3 != 0) - }) -} - -/// Opens the file at `path` as OpenVMM opens a block device of a capture on Windows: without -/// following a reparse point, for reading, and for writing if `write`, sharing reads and -/// deletion alone, which fails while another handle may write it. -#[cfg(windows)] -fn open_as_openvmm(path: &Path, write: bool) -> io::Result { - use std::os::windows::fs::OpenOptionsExt; - - /// `FILE_SHARE_READ | FILE_SHARE_DELETE`. - const SHARE_READ_DELETE: u32 = 0x1 | 0x4; - /// `FILE_FLAG_OPEN_REPARSE_POINT`. - const OPEN_REPARSE_POINT: u32 = 0x0020_0000; - OpenOptions::new() - .read(true) - .write(write) - .share_mode(SHARE_READ_DELETE) - .custom_flags(OPEN_REPARSE_POINT) - .open(path) -} - -#[test] -fn no_handle_through_which_the_host_could_write_the_block_devices_is_open_as_the_vm_starts() { - let bench = Bench::new(); - let checked = |locked: &mut LockedStore, vm: &Vm<'_>| -> Result<()> { - #[cfg(any(target_os = "linux", windows))] - let (placeholder, scratch) = ( - vm.template.plan().distro().path(), - vm.template.scratch_path(), - ); - #[cfg(target_os = "linux")] - { - assert!(!held_for_writing(placeholder)); - assert!(!held_for_writing(scratch)); - } - #[cfg(windows)] - { - open_as_openvmm(placeholder, false).unwrap(); - open_as_openvmm(scratch, true).unwrap(); - // The checked placeholder stays held, so that nothing writes it meanwhile. - let error = OpenOptions::new() - .write(true) - .open(placeholder) - .unwrap_err(); - assert_eq!(error.raw_os_error(), Some(32), "{error}"); - } - Emulation::capture().run(locked, vm) - }; - bench - .serve_with(None, &checked, &uninterrupted, false) - .unwrap(); - assert_eq!(bench.runs.get(), 1); - // The placeholder is released once the VM has ended. - OpenOptions::new() - .write(true) - .open(bench.capture.adapter.distro().path()) - .unwrap(); -} - -#[test] -fn captures_of_one_class_are_serialized() { - let active = AtomicUsize::new(0); - let most = AtomicUsize::new(0); - let runs = AtomicUsize::new(0); - let run = |locked: &mut LockedStore, vm: &Vm<'_>| -> Result<()> { - let now = active.fetch_add(1, Ordering::SeqCst) + 1; - most.fetch_max(now, Ordering::SeqCst); - runs.fetch_add(1, Ordering::SeqCst); - thread::sleep(Duration::from_millis(100)); - let ran = Emulation::capture().run(locked, vm); - active.fetch_sub(1, Ordering::SeqCst); - ran - }; - // Serves the capture of `bench` twice at once, as creations at the host's time that `now` - // returns, or as refreshes. - let race = - |bench: &Bench, now: Option<&(dyn Fn() -> Result + Sync)>| -> Vec { - let (store, capture) = (&bench.store, &bench.capture); - let barrier = Arc::new(Barrier::new(2)); - let serve = || { - let hooks = Hooks { - sync: &sync_dir, - run: &run, - interrupt: &uninterrupted, - crash: false, - }; - let now: Option<&dyn Fn() -> Result> = match now { - Some(now) => Some(now), - None => None, - }; - barrier.wait(); - store.capture(capture, now, deadline(), &hooks).unwrap() - }; - thread::scope(|scope| { - let threads: Vec<_> = (0..2).map(|_| scope.spawn(serve)).collect(); - threads - .into_iter() - .map(|thread| thread.join().unwrap()) - .collect() - }) - }; - // Refreshes capture one after the other. - let bench = Bench::new(); - let refreshed = race(&bench, None); - assert!( - refreshed - .iter() - .all(|created| matches!(created, Created::Captured(_))) - ); - assert_eq!(runs.swap(0, Ordering::SeqCst), 2); - assert_eq!(bench.indexed(), [1, 2]); - // Creations that race make one generation between them, although both start before the - // capture's anchor: the host's time is before it until a capture is published. - let bench = Bench::new(); - let store = &bench.store; - let now = || -> Result { - Ok(if store.index()?.generations().is_empty() { - before(1) - } else { - after(1) - }) - }; - assert_eq!(now().unwrap(), before(1)); - let created = race(&bench, Some(&now)); - assert_eq!(runs.load(Ordering::SeqCst), 1); - assert_eq!(created[0].generation(), created[1].generation()); - assert_eq!( - created - .iter() - .filter(|created| matches!(created, Created::Captured(_))) - .count(), - 1 - ); - assert_eq!(most.load(Ordering::SeqCst), 1); -} - -#[test] -fn recoveries_refuse_entries_of_the_captures_that_are_not_slots() { - let bench = Bench::new(); - assert!(bench.recover().unwrap().is_empty()); - bench.refresh(&Emulation::capture()).unwrap(); - fs::write(bench.store.captures_dir().join("notes.txt"), b"").unwrap(); - let error = bench.recover().unwrap_err(); - assert_eq!(error.code(), ErrorCode::ForeignFiles, "{error}"); - assert!(error.message().ends_with(": notes.txt"), "{error}"); -} - -#[cfg(target_os = "linux")] -#[test] -fn stores_too_deep_for_the_control_socket_of_a_capture_are_refused() { - let bench = Bench::new(); - let parent = bench._directory.path().join("d".repeat(80)); - fs::create_dir(&parent).unwrap(); - let bench = Bench { - store: Store::create(parent.join("templates")).unwrap(), - ..bench - }; - let error = bench.refresh(&Emulation::capture()).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported, "{error}"); - assert!( - error.message().contains("too deep for the control socket"), - "{error}" - ); - assert_eq!(bench.runs.get(), 0); -} - -#[cfg(windows)] -#[test] -fn working_directory_limits_are_checked_before_captures() { - use std::os::windows::ffi::OsStrExt; - - let bench = Bench::new(); - let mut parent = bench._directory.path().to_path_buf(); - while parent.as_os_str().encode_wide().count() < 300 { - parent.push("long-state-root-component"); - } - fs::create_dir_all(&parent).unwrap(); - let bench = Bench { - store: Store::create(parent.join("templates")).unwrap(), - ..bench - }; - let error = bench.refresh(&Emulation::capture()).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported); - assert!(error.message().contains("working directory")); - assert!(error.message().contains("258")); - assert_eq!(bench.runs.get(), 0); - assert!(!bench.store.captures_dir().exists()); -} diff --git a/nvx-rs/src/host/template/store/leases.rs b/nvx-rs/src/host/template/store/leases.rs deleted file mode 100644 index 45e249530..000000000 --- a/nvx-rs/src/host/template/store/leases.rs +++ /dev/null @@ -1,1075 +0,0 @@ -//! The leases of a store's generations, the bindings of clones to them, the tombstones that -//! quarantine generations, and the retirement of generations whose leases have drained; see -//! [the store's documentation](Store#leases-tombstones-and-retirement). - -#[cfg(test)] -mod tests; - -use std::fs; -use std::io; -use std::num::NonZeroU64; -use std::path::{Path, PathBuf}; -use std::sync::{Mutex, MutexGuard, PoisonError}; -use std::time::Instant; - -use serde::{Deserialize, Serialize}; - -use super::{ - GENERATIONS_NAME, Generation, STORE_FORMAT, Store, StoreId, entries, random_id, read_record, - refuse_problem, store_error, take_lock, write_record, -}; -use crate::activation::{Identity, StoreGeneration}; -use crate::error::{Error, ErrorCode, ErrorKind, Result}; -use crate::host::filesystem; -use crate::host::platform; -use crate::host::store::files::{remove_if_present, sync_dir}; -use crate::host::store::lease::{ - Lease, LeaseId, LeaseRecord, MAX_LEASE_RECORD_LEN, clone_socket_name, open_payload, - read_binding, read_binding_in, read_lease, -}; -use crate::host::store::regular::{open_regular_in, was_removed}; -use crate::host::store::{ - CLONE_BINDING_NAME, CallerFiles, JOURNAL_NAME, LaunchId, LockedStore, RUNTIME_NAME, - Reservation, SandboxStore, -}; -use crate::host::template::eligibility::HostTime; -use crate::host::template::key::KeyIdentity; - -/// Name of the directory of the records of the leases that a store granted. -pub const LEASES_NAME: &str = "leases"; -/// Name of the directory of the tombstones of a store's generations. -pub const TOMBSTONES_NAME: &str = "tombstones"; - -/// Syncs a directory, as [`sync_dir`] does. The transactions of leases take it as a parameter, so -/// that tests observe the order of their syncs, and fail them. -type SyncDir<'a> = &'a dyn Fn(&Path) -> Result<()>; - -/// The directories of leases and tombstones whose entries in their stores this process made -/// durable, by the directory and the identity of the store, and the name of the directory. A store -/// creates each once, and never removes it, so one sync of the store's directory, and of its -/// parent, once a process sees one, keeps its entry, and the store's, durable, even if the -/// process that created them did not finish its own syncs. -static DURABLE_RECORD_DIRS: Mutex> = Mutex::new(Vec::new()); - -/// An entry of [`DURABLE_RECORD_DIRS`]: the directory and the identity of a store, and the name -/// of one of its directories. -type DurableRecordDir = (PathBuf, StoreId, &'static str); - -/// Returns [`DURABLE_RECORD_DIRS`], locked. -fn durable_record_dirs() -> MutexGuard<'static, Vec> { - DURABLE_RECORD_DIRS - .lock() - .unwrap_or_else(PoisonError::into_inner) -} - -/// The directory of a store's leases, in messages. -const LEASES_WHAT: &str = "the directory of the leases of the template store"; -/// The directory of a store's tombstones, in messages. -const TOMBSTONES_WHAT: &str = "the directory of the tombstones of the template store"; -/// The record of a lease in its store, in messages. -const LEASE_DESCRIPTION: &str = "lease of the template store"; -/// A tombstone, in messages. -const TOMBSTONE_DESCRIPTION: &str = "tombstone of the template store"; - -// A lease is the sandbox's store's, which reads it from the clone binding, and which holds its key -// as the identity by which the clone's activation record names it: the key's identity is a type of -// this module. -impl Lease { - /// Returns the identity of the leased generation's key. - pub fn key(&self) -> KeyIdentity { - KeyIdentity::from(self.template().key) - } -} - -/// A lease that [`Store::acquire`] granted, with the lock of the sandbox that holds it. -#[derive(Debug)] -pub struct Acquired { - /// The lease. - pub lease: Lease, - /// The lock of the sandbox, which names it by the lease's path, the only one under which the - /// caller launches its clone, as the launch that the lease reserves. Dropping it releases the - /// lock, and keeps the lease, whose launch no later lock begins: a release ends the lease, and - /// another acquisition reserves another launch. - pub locked: LockedStore, -} - -/// What [`Store::retire`] found of a generation. -#[derive(Debug, Clone, PartialEq, Eq)] -pub enum Retirement { - /// The generation is gone, with its payload, its entry in the index, and its tombstone, or was - /// gone already. - Retired, - /// A tombstone quarantines the generation, which keeps its payload and its entry in the index - /// for its leases, which this lists in the order of their identities, until their clones - /// release them. - Draining(Vec), -} - -impl Store { - /// Selects the generation of the key `key` from which a clone restores at the host's time - /// `now`: the latest that the store indexes, that no tombstone quarantines, and that is - /// eligible at `now`, as [`Generation::eligibility`] decides it, if one is. - /// - /// The selection reads the index and the tombstones without the store's lock, as - /// [`index`](Self::index) does, so a selected generation may be tombstoned or retired before - /// the caller leases it, which [`acquire`](Self::acquire) refuses. - pub fn select(&self, key: KeyIdentity, now: &HostTime) -> Result> { - let index = self.index()?; - let tombstoned = self.tombstones()?; - Ok(index - .generations - .iter() - .rev() - .find(|generation| { - generation.key == key - && !tombstoned.contains(&generation.generation) - && generation.eligibility(now).is_eligible() - }) - .cloned()) - } - - /// Leases the generation `generation` to the clone of the sandbox in the directory `sandbox`, - /// whose caller keeps `files` there, at the host's time `now`, as [`HostTime::now`] samples - /// it, waiting for the store's lock and the sandbox's until `deadline`, or for as long as it - /// takes without one. - /// - /// In one transaction, under the store's lock and then the sandbox's - /// [lifecycle lock](crate::host::store#lifecycle-locks), it checks that the store indexes the - /// generation, that no tombstone quarantines it, that it is eligible at `now`, as - /// [`Generation::eligibility`] decides it, that its payload, on Linux, is one that a launch of - /// the clone and a release of its lease can use, and that the sandbox is provisioned, records - /// no VM and no launch, and holds no lease; and it writes the lease's record in the store, and - /// then the same record as the sandbox's clone binding, beside its launch journal, each - /// durable before the next step, so that no binding outlasts, through a crash of the host, the - /// record that keeps its generation in the store. It resolves the links of the sandbox's path - /// before it takes the sandbox's lock, and the lease, the binding, and the lock that it - /// returns all name the sandbox by that path, as [`Lease::sandbox`] returns it, which no later - /// change of a link redirects to another directory. It returns the lease, with the sandbox's - /// lock, the only one under which the caller launches the clone, once, as the launch that the - /// lease reserves, [`Lease::launch`], and releases the store's lock. - /// - /// The launch that the lease reserves restores the generation's snapshot in place, and its VMM - /// listens beside the snapshot, as OpenVMM requires of a restore on Linux: at the control - /// socket `.sock`, in the generation's payload, after the launch's identity, rather - /// than at the sandbox's own socket. So a payload holds the sockets of the clones of its - /// generation, each named after a launch, while they lease it, and none once its leases have - /// drained: the VMM removes its socket as it exits, a stop or a recovery of the sandbox - /// removes the one that an abrupt exit leaves, and a [release](Self::release) makes the - /// socket's absence durable before the lease ends. On Windows, the VMM listens at a named - /// pipe, as every launch does. - /// - /// Fails, and leases nothing: - /// - /// - with [`ErrorKind::NotFound`] if the store does not index the generation; - /// - with [`ErrorCode::TemplateTombstoned`] if a tombstone quarantines it; - /// - with [`ErrorCode::TemplateIneligible`] if it is not eligible at `now`; - /// - on Linux, with [`ErrorKind::Io`] if the generation's payload is not a directory of the - /// current user with mode 0700, which a launch of the clone requires, as OpenVMM does of the - /// directory of a control socket, or if a release could not open it as it opens it, - /// following no link at the store's directory, at the directory of the generations, or at - /// the payload: a release opens the payload to remove the clone's socket from it and make - /// its absence durable; - /// - with [`ErrorCode::SandboxNotProvisioned`] if the directory does not exist, or holds no - /// configuration record of the caller, as `files` declares it; - /// - with [`ErrorCode::SandboxRunning`] if the sandbox records a VM, or a launch, which may - /// still run; - /// - with [`ErrorCode::SandboxLeased`] if the sandbox holds a lease already; - /// - with [`ErrorKind::Unsupported`] if the sandbox's path, with its links resolved, is not - /// valid Unicode, which its record cannot hold, or, on Windows, is a network path, or one - /// that names the directory only in its verbatim form; - /// - with [`ErrorKind::TimedOut`] if the deadline passes before it takes the store's lock, and - /// as [`SandboxStore::lock_existing`] fails before it takes the sandbox's. - /// - /// A failure to write the binding, or to make it durable, leases nothing either, unless the - /// binding's removal cannot be made durable: the lease's record then stays, and the error says - /// that the sandbox may hold the lease still, which a [release](Self::release) ends. - pub fn acquire( - &self, - generation: StoreGeneration, - sandbox: impl Into, - files: &CallerFiles, - now: &HostTime, - deadline: Option, - ) -> Result { - self.acquire_with(generation, sandbox.into(), files, now, deadline, &sync_dir) - } - - /// Leases a generation as [`acquire`](Self::acquire) does, making its records durable with - /// `sync`. - fn acquire_with( - &self, - generation: StoreGeneration, - sandbox: PathBuf, - files: &CallerFiles, - now: &HostTime, - deadline: Option, - sync: SyncDir<'_>, - ) -> Result { - let lock = take_lock(&self.dir, deadline)?; - let index = self.index()?; - let indexed = index - .get(generation) - .ok_or_else(|| self.not_indexed(generation))?; - if self.is_tombstoned(generation)? { - return Err(Error::new( - ErrorCode::TemplateTombstoned, - format!( - "a tombstone quarantines the generation {generation} of the template store \ - {}, which takes no new clones", - self.dir.display() - ), - )); - } - let eligibility = indexed.eligibility(now); - if !eligibility.is_eligible() { - return Err(Error::new( - ErrorCode::TemplateIneligible, - format!( - "the generation {generation} of the template store {} is {eligibility}", - self.dir.display() - ), - )); - } - // A release removes the socket of the clone's VMM through a handle of the payload, which - // it opens for reading, following no link from the store's directory on, so that it can - // make the removal durable: a payload that it could not open, or remove an entry from, - // would keep the lease and its binding past it. - #[cfg(target_os = "linux")] - self.check_payload(generation)?; - let not_provisioned = || { - Error::new( - ErrorCode::SandboxNotProvisioned, - format!("no sandbox is provisioned in {}", sandbox.display()), - ) - }; - // The lease, the binding, and the lock name the sandbox by its path with its links - // resolved, which no later change of a link redirects to another directory. - let resolved = match filesystem::canonicalize(&sandbox) { - Ok(resolved) => resolved, - Err(error) if error.kind() == io::ErrorKind::NotFound => { - return Err(not_provisioned().with_source(error)); - } - Err(error) => { - let message = format!( - "cannot resolve the path of the sandbox in {}", - sandbox.display() - ); - return Err(if error.kind() == io::ErrorKind::Unsupported { - Error::unsupported(message).with_source(error) - } else { - store_error(message)(error) - }); - } - }; - let mut locked = match SandboxStore::lock_existing(&resolved, files, deadline) { - Ok(locked) => locked, - Err(error) if error.kind() == ErrorKind::NotFound => { - return Err(not_provisioned().with_source(error)); - } - Err(error) => return Err(error), - }; - if let Some(config) = files.config() { - let path = locked.dir().join(config); - match fs::symlink_metadata(&path) { - Ok(_) => {} - Err(error) if error.kind() == io::ErrorKind::NotFound => { - return Err(not_provisioned()); - } - Err(error) => { - return Err(store_error(format!("cannot examine {}", path.display()))( - error, - )); - } - } - } - if locked.journal()?.is_some() || locked.runtime()?.is_some() { - return Err(may_run(locked.dir())); - } - if let Some(held) = read_binding(locked.dir())? { - return Err(Error::new( - ErrorCode::SandboxLeased, - format!( - "the sandbox in {} holds the lease {} of the generation {} of the template \ - store {} already", - locked.dir().display(), - held.id(), - held.generation(), - held.store() - ), - )); - } - if locked.dir().to_str().is_none() { - return Err(Error::unsupported(format!( - "the path of the sandbox {} is not valid Unicode, which the record of a lease \ - cannot hold", - locked.dir().display() - ))); - } - let lease = Lease::new( - LeaseId::from_bytes(random_id("a lease's identity")?), - self.id, - generation, - Identity::from(indexed.key), - locked.dir().to_owned(), - LaunchId::generate()?, - ); - let record = LeaseRecord::new(&lease); - self.create_records_dir(LEASES_NAME, LEASES_WHAT, sync)?; - let path = self.lease_path(lease.id()); - write_record(&path, &record, "the lease of the template store")?; - // A retirement deletes a generation that no record of the store leases, so the record - // must be durable before a binding names it. - if let Err(error) = sync(&self.dir.join(LEASES_NAME)) { - let _ = remove_if_present(&path); - return Err(error); - } - if let Err(error) = bind(locked.dir(), &record, sync) { - // The record goes only once the binding's removal is durable. Until then, it keeps - // the generation for a binding that may outlast a crash, and a retirement removes it - // once no binding names it. - if unbind(locked.dir(), sync).is_err() { - return Err(held_still(error, locked.dir(), lease.id())); - } - let _ = remove_if_present(&path); - return Err(error); - } - drop(lock); - locked.reserve(Some(Reservation { - socket: self.clone_socket_path(&lease), - launch: lease.launch().clone(), - })); - Ok(Acquired { lease, locked }) - } - - /// Returns the path of the Linux socket at which the VMM of the clone that holds `lease` - /// listens, its control endpoint: `.sock`, after the launch that the lease reserves, - /// in the payload of the leased generation, beside the snapshot that the VMM restores in - /// place, since OpenVMM requires a restore's control socket in the snapshot's parent. - /// - /// A payload holds the sockets of the clones of its generation only while they lease it: the - /// VMM removes its socket as it exits, a stop or a recovery removes what an abrupt exit - /// leaves, and a [release](Self::release) makes the socket's absence durable before the lease - /// ends. A [retirement](Self::retire) removes the payload with whatever a crash left in it. - pub(crate) fn clone_socket_path(&self, lease: &Lease) -> PathBuf { - self.payload_dir(lease.generation()) - .join(clone_socket_name(lease.launch())) - } - - /// Releases the lease that the sandbox of `locked` holds, and returns it, or returns `None` if - /// the sandbox holds none. - /// - /// A lease lasts until its clone's VM has exited: the release refuses with - /// [`ErrorCode::SandboxRunning`] while the sandbox records a VM or a launch, which may still - /// run. The sandbox's store clears those records only once it has verified that the VM's - /// process exited, through its recorded [`ProcessRef`](crate::host::process::ProcessRef), as - /// a stop or a [`reattach`](crate::host::supervisor::reattach) does. The release first ends - /// the reservation of the launch that `locked` holds, if it acquired the lease, so that one - /// that fails part of the way, once it removed files that a launch of the clone's plan needs, - /// leaves no launch that the lock may begin. It then removes what the clone kept of its - /// launch: the file of the scratch device that an earlier version created for a clone in RAM, - /// [`CLONE_SCRATCH_NAME`](crate::host::store::CLONE_SCRATCH_NAME), if one remains, the - /// clone's activation record, - /// [`CLONE_ACTIVATION_NAME`](crate::host::store::CLONE_ACTIVATION_NAME), the socket of its - /// restore-ready event, [`RESTORE_READY_NAME`](crate::host::store::RESTORE_READY_NAME), and - /// its records, which last as long as the lease, its progress log, - /// [`CLONE_PROGRESS_NAME`](crate::host::store::CLONE_PROGRESS_NAME), and the record of its - /// failure, [`CLONE_FAILURE_NAME`](crate::host::store::CLONE_FAILURE_NAME); - /// then, on Linux, the control socket of the clone's VMM, `.sock` in the leased - /// generation's payload, after the launch that the lease reserved, whose absence it makes - /// durable; then the clone binding, and once their removal is durable, the lease's record in - /// the store, so that an interrupted release leaves a record that no binding names, which a - /// [retirement](Self::retire) removes, and never a binding without its record. It takes no - /// lock of the store. - /// - /// The caller takes the sandbox's lock through the lease's path, as [`Lease::sandbox`] - /// returns it, and as [`acquire`](Self::acquire) takes it, with its links resolved. The - /// release names the entries of the sandbox's directory through one handle of the directory - /// that the path names, once it checked through that handle that the directory holds the - /// lock's file: through it, it checks that the sandbox records no VM and no launch, reads the - /// binding, removes the clone's files and the binding, and makes their absence durable. No - /// change of the path as the release runs, of a link in it or of one of its directories, then - /// makes it check or remove the files of another sandbox, whose lock it does not hold. - /// - /// Likewise, on Linux, the release opens the leased generation's payload before it removes - /// anything, through a handle of the store's directory: following no link at the store's - /// directory, at its directory of the generations, or at the payload, and once it checked - /// through that handle that the directory holds this store. It removes the clone's socket and - /// makes its absence durable through the payload's handle, so that neither a link that takes - /// the place of one of them nor a directory that takes the place of the store ever leads it to - /// remove the files of another directory. A directory that holds another store, or none, holds - /// no payload of this store's, so no socket goes. - /// - /// Fails with [`ErrorKind::Other`] if the sandbox holds a lease of another store, if the lock - /// does not name the sandbox by its lease's path, or if the directory that the path names does - /// not hold the lock's file, and with [`ErrorKind::Io`] if the store's directory, its - /// directory of the generations, or the payload is a link, or not a directory, and then - /// removes nothing. - pub fn release(&self, locked: &mut LockedStore) -> Result> { - self.release_with( - locked, - &|| {}, - &platform::Directory::sync, - &platform::Directory::sync, - ) - } - - /// Releases a lease as [`release`](Self::release) does, running `checked` once its checks - /// pass, before it removes anything, and making the removal of the clone's socket durable with - /// `sync_payload`, through the handle of the leased generation's payload, and that of the - /// binding with `sync_sandbox`, through the handle of the sandbox's directory. - fn release_with( - &self, - locked: &mut LockedStore, - checked: &dyn Fn(), - sync_payload: &dyn Fn(&platform::Directory) -> io::Result<()>, - sync_sandbox: &dyn Fn(&platform::Directory) -> io::Result<()>, - ) -> Result> { - let directory = locked.open_dir()?; - // A record whose deletion is pending, on Windows, is removed, as its readers take it. - for name in [JOURNAL_NAME, RUNTIME_NAME] { - let present = open_regular_in(&directory, name).map_err(store_error(format!( - "cannot examine {}", - locked.dir().join(name).display() - )))?; - if present.is_some() { - return Err(may_run(locked.dir())); - } - } - let Some(lease) = read_binding_in(&directory, locked.dir())? else { - return Ok(None); - }; - if lease.store() != self.id { - return Err(Error::other(format!( - "the sandbox in {} holds a lease of the template store {}, not of this one, {}", - locked.dir().display(), - lease.store(), - self.id - ))); - } - if locked.dir() != lease.sandbox() { - return Err(Error::other(format!( - "the lock of the sandbox in {} does not name it by the path of its lease, {}, \ - through which a release takes it", - locked.dir().display(), - lease.sandbox().display() - ))); - } - let recorded = self.check_records_dir(LEASES_NAME, LEASES_WHAT)?; - // The payload opens before anything goes, so that a link in its place refuses the release. - let payload = self.open_clone_payload(&lease)?; - checked(); - // A plan of the clone launches under the reservation that the lease's acquisition left in - // the lock, which the release ends before it removes anything, so that a release that - // fails once it removed the clone's files, such as the record of its failure, which ends - // the clone's launches until the release, leaves no launch that the lock may begin. - locked.reserve(None); - // The clone's VM has exited, and the sandbox launches nothing else under the lease, so no - // VM listens at the clone's control socket. The binding's removal makes the removal of the - // clone's files durable too, since - // all are entries of the sandbox's directory, but not that of the socket, an entry of the - // leased generation's payload, which must be durable before the lease ends, so that a - // generation whose leases have drained holds no clone's socket. - locked.remove_clone_files(&directory)?; - if let Some(payload) = &payload { - self.remove_clone_socket(&lease, payload, sync_payload)?; - } - let binding = locked.dir().join(CLONE_BINDING_NAME); - directory - .remove_file(CLONE_BINDING_NAME) - .map_err(store_error(format!("cannot remove {}", binding.display())))?; - // The binding's absence is durable before its record goes: a binding that outlasted the - // record through a crash would let a retirement delete the generation that it names. - sync_sandbox(&directory).map_err(store_error(format!( - "cannot sync the directory {}", - locked.dir().display() - )))?; - if recorded { - remove_lease_record(&self.lease_path(lease.id()))?; - } - Ok(Some(lease)) - } - - /// Refuses the payload of the generation `generation` unless a release of a clone's lease can - /// open it, as [`open_payload`] opens it, following no link at the store's directory, at the - /// directory of the generations, or at the payload, through a directory that holds this - /// store, and it is a directory of the current user with mode 0700. A launch of a clone - /// requires that mode, OpenVMM requiring it of the directory of a control socket, and so does - /// the release: it opens the payload for reading, and removes the clone's socket from it. - #[cfg(target_os = "linux")] - fn check_payload(&self, generation: StoreGeneration) -> Result<()> { - let payload = self.payload_dir(generation); - let problem = match open_payload(self.dir(), self.id, generation)? { - Some(directory) => directory.store_dir_problem().map_err(store_error(format!( - "cannot examine the payload of the generation {}", - payload.display() - )))?, - None => { - Some("is gone, or the store's directory holds another store, or none".to_owned()) - } - }; - match problem { - Some(problem) => Err(Error::io(format!( - "the payload of the generation {} {problem}", - payload.display() - ))), - None => Ok(()), - } - } - - /// Opens the payload of the generation that `lease` leases, which on Linux holds the control - /// socket of the clone's VMM, through the store's directory, following no link, as - /// [`open_payload`] opens it, or returns `None` elsewhere, where the VMM listens at a named - /// pipe, or if the payload is gone, or the store's directory holds another store, or none, - /// since neither holds a socket of this store's clones. - fn open_clone_payload(&self, lease: &Lease) -> Result> { - if !cfg!(unix) { - return Ok(None); - } - open_payload(self.dir(), self.id, lease.generation()) - } - - /// Removes the control socket of the VMM of the clone that holds `lease`, beside the snapshot - /// that the VMM restored, from `payload`, the handle of the leased generation's payload, if it - /// is there, and makes its absence durable with `sync`, through that handle. The VMM removes - /// its socket as it exits, and a stop or a recovery removes the one that an abrupt exit - /// leaves, but neither makes the removal durable. - fn remove_clone_socket( - &self, - lease: &Lease, - payload: &platform::Directory, - sync: &dyn Fn(&platform::Directory) -> io::Result<()>, - ) -> Result<()> { - payload - .remove_file(&clone_socket_name(lease.launch())) - .map_err(store_error(format!( - "cannot remove {}", - self.clone_socket_path(lease).display() - )))?; - sync(payload).map_err(store_error(format!( - "cannot sync the directory {}", - self.payload_dir(lease.generation()).display() - ))) - } - - /// Returns the leases of the generation `generation` that the store's records list, in the - /// order of their identities, including those that an interrupted release or removal of - /// their sandboxes left, which a [retirement](Self::retire) removes. - /// - /// Fails with [`ErrorCode::ForeignFiles`] if the directory of the leases holds entries that - /// are not the store's. - pub fn leases(&self, generation: StoreGeneration) -> Result> { - let mut leases = Vec::new(); - for id in self.records(LEASES_NAME, LEASES_WHAT, lease_name)?.records { - // A release may remove the record after the listing. - if let Some(lease) = self.read_lease(id)? - && lease.generation() == generation - { - leases.push(lease); - } - } - leases.sort_by_key(|lease| lease.id()); - Ok(leases) - } - - /// Quarantines the generation `generation` with a tombstone, waiting for the store's lock - /// until `deadline`, or for as long as it takes without one: the generation takes no new - /// lease, and [`select`](Self::select) skips it, while its leases keep working. A tombstone - /// is durable once this returns, and lasts until the generation's - /// [retirement](Self::retire). - /// - /// Tombstoning a generation again does nothing. Fails with [`ErrorKind::NotFound`] if the - /// store does not index the generation. - pub fn tombstone(&self, generation: StoreGeneration, deadline: Option) -> Result<()> { - let _lock = take_lock(&self.dir, deadline)?; - if self.index()?.get(generation).is_none() { - return Err(self.not_indexed(generation)); - } - self.write_tombstone(generation, &sync_dir) - } - - /// Returns the store generations that tombstones quarantine, in order. - /// - /// Each tombstone is read and checked, as an acquisition checks it: a tombstone that cannot be - /// read, or is malformed, fails with [`ErrorKind::Io`], and one of another format with - /// [`ErrorCode::FormatUnsupported`], rather than quarantining its generation. Fails with - /// [`ErrorCode::ForeignFiles`] if the directory of the tombstones holds entries that are not - /// the store's. - pub fn tombstones(&self) -> Result> { - let mut tombstoned = Vec::new(); - for generation in self - .records(TOMBSTONES_NAME, TOMBSTONES_WHAT, tombstone_name)? - .records - { - // A retirement may remove the tombstone after the listing. - if self.read_tombstone(generation)? { - tombstoned.push(generation); - } - } - tombstoned.sort(); - Ok(tombstoned) - } - - /// Retires the generation `generation`, waiting for the store's lock until `deadline`, or for - /// as long as it takes without one: it tombstones the generation, and deletes it once its - /// leases have drained. - /// - /// Under the store's lock, it tombstones the generation, and removes the records of its - /// leases whose sandboxes no longer hold them: a sandbox that is gone, that holds no clone - /// binding, or one of another lease, as an interrupted release, or a removal of the sandbox, - /// leaves it. It makes the binding's absence durable first, by syncing the sandbox's - /// directory, if it is one, and the directory that holds the sandbox's entry, the nearest of - /// its ancestors that remains, through the links that the sandbox's path names, as a release - /// that a crash or a failed sync interrupted may not have. A [removal](LockedStore::remove) of - /// the sandbox made it durable before the sandbox went; for a sandbox that went otherwise, - /// these syncs make it durable only while the directories that held the sandbox remain. If - /// any lease remains, the generation keeps its payload and its entry in the index, and the - /// retirement returns [`Retirement::Draining`] with the leases, whose clones - /// [release](Self::release) them; a later retirement deletes it. Otherwise it removes the - /// generation's payload, then its entry in the index, and then its tombstone, each durable - /// before the next, as the tombstone is before the payload goes, so that a retirement that a - /// crash interrupts, even one of the host, leaves a tombstoned generation, which retiring it - /// again deletes, and returns [`Retirement::Retired`]. A generation that the store indexed - /// once, and no longer indexes, is retired already. Its store generation is never assigned - /// again. - /// - /// It also removes the temporary files that interrupted replacements of the records of leases - /// and tombstones left, since the store writes those only under its lock. - /// - /// Fails with [`ErrorKind::NotFound`] for a store generation that the store never assigned, - /// and with [`ErrorCode::ForeignFiles`] if the directory of the leases, or of the tombstones, - /// holds entries that are not the store's. A clone binding that cannot be read, or whose - /// absence cannot be made durable, fails the retirement, which then deletes no generation. - pub fn retire( - &self, - generation: StoreGeneration, - deadline: Option, - ) -> Result { - self.retire_with(generation, deadline, &sync_dir) - } - - /// Retires a generation as [`retire`](Self::retire) does, making each step durable with `sync` - /// before the next. - fn retire_with( - &self, - generation: StoreGeneration, - deadline: Option, - sync: SyncDir<'_>, - ) -> Result { - let _lock = take_lock(&self.dir, deadline)?; - let mut index = self.index()?; - if index.get(generation).is_none() { - if generation >= index.next { - return Err(self.not_indexed(generation)); - } - // A retirement that was interrupted once it had written the index may not have made - // it durable, which must be before the tombstone goes. - sync(&self.dir)?; - self.remove_tombstone(generation)?; - return Ok(Retirement::Retired); - } - let leases = self.records(LEASES_NAME, LEASES_WHAT, lease_name)?; - let tombstones = self.records(TOMBSTONES_NAME, TOMBSTONES_WHAT, tombstone_name)?; - self.write_tombstone(generation, sync)?; - for temporary in leases.temporaries.iter().chain(&tombstones.temporaries) { - remove_if_present(temporary)?; - } - let mut live = Vec::new(); - for id in leases.records { - let Some(lease) = self.read_lease(id)? else { - continue; - }; - if lease.generation() != generation { - continue; - } - if is_bound(&lease)? { - live.push(lease); - } else { - // A release may have removed the binding without making that durable yet, so the - // binding's absence must be durable before its record goes. - for dir in binding_dirs(lease.sandbox())? { - sync(dir)?; - } - remove_lease_record(&self.lease_path(id))?; - } - } - if !live.is_empty() { - live.sort_by_key(|lease| lease.id()); - return Ok(Retirement::Draining(live)); - } - self.remove_payload(generation)?; - sync(&self.dir.join(GENERATIONS_NAME))?; - index - .generations - .retain(|indexed| indexed.generation != generation); - self.write_index(&index)?; - sync(&self.dir)?; - self.remove_tombstone(generation)?; - Ok(Retirement::Retired) - } - - /// Returns the error of the generation `generation`, which the store does not index. - fn not_indexed(&self, generation: StoreGeneration) -> Error { - Error::new( - ErrorKind::NotFound, - format!( - "the template store {} holds no generation {generation}", - self.dir.display() - ), - ) - } - - /// Returns the path of the record of the lease `id`. - fn lease_path(&self, id: LeaseId) -> PathBuf { - self.dir.join(LEASES_NAME).join(format!("{id}.json")) - } - - /// Returns the path of the tombstone of the store generation `generation`. - fn tombstone_path(&self, generation: StoreGeneration) -> PathBuf { - self.dir - .join(TOMBSTONES_NAME) - .join(format!("{generation}.json")) - } - - /// Creates the store's directory `name`, which `what` names, private to the current user, - /// unless it exists, under the store's lock, and makes its entry in the store durable with - /// `sync`, and the store's own entry in its parent: it syncs the store's directory, and then - /// the store's parent, once it creates the directory, and once it finds one that this process - /// has not synced yet, whose creation, or the store's, may not have finished its sync. A - /// binding then never names a record of a store whose own entry a crash of the host could - /// lose. - fn create_records_dir(&self, name: &'static str, what: &str, sync: SyncDir<'_>) -> Result<()> { - let key = (self.dir.clone(), self.id, name); - if !self.check_records_dir(name, what)? { - let dir = self.dir.join(name); - let problem = platform::create_owner_only_dir(&dir).map_err(store_error(format!( - "cannot create {what} {}", - dir.display() - )))?; - refuse_problem(&dir, what, problem)?; - } else if durable_record_dirs().contains(&key) { - return Ok(()); - } - sync(&self.dir)?; - if let Some(parent) = self.dir.parent() { - sync(parent)?; - } - let mut durable = durable_record_dirs(); - if !durable.contains(&key) { - durable.push(key); - } - Ok(()) - } - - /// Checks that the store's directory `name`, which `what` names, is private to the current - /// user, and not a link, if it exists, and returns whether it exists. - fn check_records_dir(&self, name: &str, what: &str) -> Result { - let dir = self.dir.join(name); - match platform::check_owner_only_dir(&dir) { - Err(error) if error.kind() == io::ErrorKind::NotFound => Ok(false), - checked => { - let problem = checked - .map_err(store_error(format!("cannot open {what} {}", dir.display())))?; - refuse_problem(&dir, what, problem)?; - Ok(true) - } - } - } - - /// Lists the store's directory `name`, which `what` names, whose records `parse` names: the - /// records, and the temporary files that interrupted replacements of them left. A directory - /// that does not exist holds none. - /// - /// Fails with [`ErrorCode::ForeignFiles`] for any other entry. - fn records( - &self, - name: &str, - what: &str, - parse: impl Fn(&str) -> Option, - ) -> Result> { - let mut listed = Records { - records: Vec::new(), - temporaries: Vec::new(), - }; - if !self.check_records_dir(name, what)? { - return Ok(listed); - } - let dir = self.dir.join(name); - let mut foreign = Vec::new(); - for entry in entries(&dir)? { - if let Some(record) = parse(&entry) { - listed.records.push(record); - } else if is_temporary_of(&entry, |file| parse(file).is_some()) { - listed.temporaries.push(dir.join(entry)); - } else { - foreign.push(entry); - } - } - if foreign.is_empty() { - return Ok(listed); - } - foreign.sort(); - Err(Error::new( - ErrorCode::ForeignFiles, - format!( - "{what} {} holds entries that are not the store's: {}", - dir.display(), - foreign.join(", ") - ), - )) - } - - /// Reads the record of the lease `id`, or returns `None` if it does not exist. - fn read_lease(&self, id: LeaseId) -> Result> { - let path = self.lease_path(id); - let Some(lease) = read_lease(&path, LEASE_DESCRIPTION)? else { - return Ok(None); - }; - if lease.id() != id || lease.store() != self.id { - return Err(Error::io(format!( - "the {LEASE_DESCRIPTION} {} is malformed: it is the lease {} of the template \ - store {}", - path.display(), - lease.id(), - lease.store() - ))); - } - Ok(Some(lease)) - } - - /// Returns whether a tombstone quarantines the generation `generation`. - fn is_tombstoned(&self, generation: StoreGeneration) -> Result { - if !self.check_records_dir(TOMBSTONES_NAME, TOMBSTONES_WHAT)? { - return Ok(false); - } - self.read_tombstone(generation) - } - - /// Reads the tombstone of the generation `generation`, in the directory of the tombstones, - /// which exists, and returns whether it exists: a tombstone that cannot be read, is malformed, - /// or is of another format fails. - fn read_tombstone(&self, generation: StoreGeneration) -> Result { - let path = self.tombstone_path(generation); - let Some(record) = - read_record::(&path, TOMBSTONE_DESCRIPTION, MAX_LEASE_RECORD_LEN)? - else { - return Ok(false); - }; - if record.store != self.id.to_string() || record.generation != generation.get().get() { - return Err(Error::io(format!( - "the {TOMBSTONE_DESCRIPTION} {} is malformed: it is the tombstone of the \ - generation {} of the template store {}", - path.display(), - record.generation, - record.store - ))); - } - Ok(true) - } - - /// Writes the tombstone of the generation `generation`, unless it has one, under the store's - /// lock, and makes it durable with `sync`, even one that an earlier transaction wrote, whose - /// sync may have failed. - fn write_tombstone(&self, generation: StoreGeneration, sync: SyncDir<'_>) -> Result<()> { - if !self.is_tombstoned(generation)? { - self.create_records_dir(TOMBSTONES_NAME, TOMBSTONES_WHAT, sync)?; - write_record( - &self.tombstone_path(generation), - &TombstoneRecord { - format: STORE_FORMAT, - store: self.id.to_string(), - generation: generation.get().get(), - }, - "the tombstone of the template store", - )?; - } - sync(&self.dir.join(TOMBSTONES_NAME)) - } - - /// Removes the tombstone of the generation `generation`, if it has one, under the store's - /// lock, once it has read and checked it: a tombstone that is not the store's tombstone of - /// that generation, or that is of another format, fails, and is kept. - fn remove_tombstone(&self, generation: StoreGeneration) -> Result<()> { - if self.is_tombstoned(generation)? { - remove_if_present(&self.tombstone_path(generation))?; - } - Ok(()) - } - - /// Removes the payload of the generation `generation`, if it exists, under the store's lock. - /// A payload that is a link, or not private to the current user, is refused, and kept. - fn remove_payload(&self, generation: StoreGeneration) -> Result<()> { - self.check_generations_dir()?; - let payload = self.payload_dir(generation); - let what = "the payload of the generation"; - match platform::check_owner_only_dir(&payload) { - Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(()), - checked => { - let problem = checked.map_err(store_error(format!( - "cannot open {what} {}", - payload.display() - )))?; - refuse_problem(&payload, what, problem)?; - } - } - match fs::remove_dir_all(&payload) { - Err(error) if error.kind() != io::ErrorKind::NotFound => Err(store_error(format!( - "cannot remove {what} {}", - payload.display() - ))(error)), - _ => Ok(()), - } - } -} - -/// Returns the error of a lease that the sandbox in `dir` cannot take or end while it records a -/// VM or a launch, [`ErrorCode::SandboxRunning`]. -fn may_run(dir: &Path) -> Error { - Error::new( - ErrorCode::SandboxRunning, - format!( - "a VM of the sandbox in {}, or a launch of it, may still run", - dir.display() - ), - ) -} - -/// Returns whether the clone binding of the sandbox of `lease` names it still: unless the sandbox -/// is gone, holds no clone binding, or holds one of another lease. -fn is_bound(lease: &Lease) -> Result { - Ok(read_binding(lease.sandbox())? - .is_some_and(|binding| binding.id() == lease.id() && binding.store() == lease.store())) -} - -/// Returns the directories whose entries changed as the clone binding of the sandbox in `sandbox` -/// went: the sandbox's own directory, if it is one, and the directory that holds the sandbox's -/// entry, the nearest of its ancestors that is a directory, following the links that its path -/// names, as the removal of the sandbox followed them. A sandbox that was removed, and provisioned -/// again at its path, is a new directory, whose entries never held the binding: the binding went -/// with the directory that the entry of the sandbox named before. -/// -/// Fails for an ancestor that cannot be examined, or that is a link to nothing, whose target, the -/// directory that held the sandbox, cannot be known. -fn binding_dirs(sandbox: &Path) -> Result> { - let mut dirs = Vec::with_capacity(2); - if fs::symlink_metadata(sandbox).is_ok_and(|metadata| metadata.is_dir()) { - dirs.push(sandbox); - } - for ancestor in sandbox.ancestors().skip(1) { - let unknown = |error: io::Error| { - Error::io(format!( - "cannot find the directory from which the sandbox in {} went, through {}", - sandbox.display(), - ancestor.display() - )) - .with_source(error) - }; - match fs::metadata(ancestor) { - Ok(metadata) if metadata.is_dir() => { - dirs.push(ancestor); - return Ok(dirs); - } - // A file replaced the directory, in the entries of its parent. - Ok(_) => {} - Err(error) if error.kind() == io::ErrorKind::NotFound => { - if fs::symlink_metadata(ancestor).is_ok() { - return Err(unknown(error)); - } - } - Err(error) => return Err(unknown(error)), - } - } - Err(Error::io(format!( - "no directory holds the sandbox in {}", - sandbox.display() - ))) -} - -/// Writes `record` as the clone binding of the sandbox in `dir`, and makes it durable with `sync`. -fn bind(dir: &Path, record: &LeaseRecord, sync: SyncDir<'_>) -> Result<()> { - write_record(&dir.join(CLONE_BINDING_NAME), record, "the clone binding")?; - sync(dir) -} - -/// Removes the clone binding of the sandbox in `dir`, if it has one, and makes its removal durable -/// with `sync`, before a record of its lease goes: a binding that outlasted the record through a -/// crash would let a retirement delete the generation that it names. -fn unbind(dir: &Path, sync: SyncDir<'_>) -> Result<()> { - remove_if_present(&dir.join(CLONE_BINDING_NAME))?; - sync(dir) -} - -/// Removes the record of a lease at `path`, if it exists, which a release and a retirement, in -/// other processes, may both remove at once: a record that another removal removed as this tried, -/// as [`was_removed`] tells, is gone already. -fn remove_lease_record(path: &Path) -> Result<()> { - match fs::remove_file(path) { - Err(error) if error.kind() != io::ErrorKind::NotFound && !was_removed(path, &error) => Err( - store_error(format!("cannot remove {}", path.display()))(error), - ), - _ => Ok(()), - } -} - -/// Returns `error`, which ended an acquisition of the lease `id` whose binding the sandbox in -/// `dir` may hold still, with the error's code and source, saying so. -fn held_still(error: Error, dir: &Path, id: LeaseId) -> Error { - let code = error.code(); - let (_, message, source) = error.into_parts(); - let error = Error::new( - code, - format!( - "{message}; the sandbox in {} may hold the lease {id} still, which a release ends", - dir.display() - ), - ); - match source { - Some(source) => error.with_source(source), - None => error, - } -} - -/// Parses the name of the record of a lease, `ID.json`, in the form in which the store writes -/// it. -fn lease_name(name: &str) -> Option { - let stem = name.strip_suffix(".json")?; - let id: LeaseId = stem.parse().ok()?; - (id.to_string() == stem).then_some(id) -} - -/// Parses the name of a tombstone, `G.json`, in the form in which the store writes it. -fn tombstone_name(name: &str) -> Option { - let stem = name.strip_suffix(".json")?; - let generation = NonZeroU64::new(stem.parse().ok()?).map(StoreGeneration::new)?; - (generation.to_string() == stem).then_some(generation) -} - -/// Returns whether `name` is a temporary file that an interrupted replacement of a record whose -/// name `record` accepts leaves beside it: `.NAME.NUMBER.tmp`. -fn is_temporary_of(name: &str, record: impl Fn(&str) -> bool) -> bool { - name.strip_prefix('.') - .and_then(|name| name.strip_suffix(".tmp")) - .and_then(|name| name.rsplit_once('.')) - .is_some_and(|(file, number)| { - record(file) && !number.is_empty() && number.bytes().all(|byte| byte.is_ascii_digit()) - }) -} - -/// The entries of the directory of a store's leases or tombstones. -struct Records { - /// What the names of the records name. - records: Vec, - /// The temporary files that interrupted replacements of records left. - temporaries: Vec, -} - -/// A tombstone, as JSON. -#[derive(Debug, Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -struct TombstoneRecord { - /// [`STORE_FORMAT`]. - format: u32, - /// The identity of the store. - store: String, - /// The store generation of the tombstoned generation. - generation: u64, -} diff --git a/nvx-rs/src/host/template/store/leases/tests.rs b/nvx-rs/src/host/template/store/leases/tests.rs deleted file mode 100644 index 2a8b6898d..000000000 --- a/nvx-rs/src/host/template/store/leases/tests.rs +++ /dev/null @@ -1,2956 +0,0 @@ -//! Tests of leases, clone bindings, tombstones, and retirement, by one process, and by -//! transactions in processes of their own, which [`transaction_role`] runs; and of the crashes of -//! a clone's orchestrator, which [`clone_role`] runs, with a [`vmm_role`] as the clone's VMM. - -use std::cell::{Cell, RefCell}; -use std::convert::Infallible; -use std::env; -use std::ffi::OsString; -use std::io::{BufRead, BufReader, Read, Write}; -use std::process::{Child, ChildStdin, ChildStdout, Command, Stdio}; -use std::thread; -use std::time::Duration; - -use serde_json::{Value, json}; - -use super::*; -use crate::activation::TemplateReference; -use crate::host::process::{Opened, Process, ProcessRef}; -use crate::host::snapshot::tests::{ManifestSpec, artifact, network_class}; -use crate::host::store::files::write_private; -use crate::host::store::{ - CLONE_ACTIVATION_NAME, CLONE_FAILURE_NAME, CLONE_PROGRESS_NAME, CLONE_SCRATCH_NAME, FORMAT, - LOCK_NAME, LaunchState, RESTORE_READY_NAME, RUNTIME_NAME, Recovery, RuntimeIdentity, -}; -use crate::host::supervisor::{self, Reattached, StopOutcome, VmmCommand}; -use crate::host::template::eligibility::MAX_DOWNTIME_NS; -use crate::host::template::store::SNAPSHOT_NAME; -use crate::host::template::store::tests::{ - cause, link_dir, link_file, publish, publish_with, remove, storage, store as new_store, - unlink_dir, -}; -use crate::host::tests::released; -use crate::protocol::CAPABILITY_LEN; - -/// A change to a record, as JSON. -type Change = fn(&mut Value); - -/// Nanoseconds in a second. -const SECOND: u64 = 1_000_000_000; -/// Nanoseconds in a day. -const DAY: u64 = 86_400 * SECOND; -/// The configuration record of these tests' sandboxes. -const CONFIG: &str = "config.json"; -/// The directory of the store in which [`transaction_role`] runs its transaction. -const ROLE_STORE: &str = "NVX_TEMPLATE_LEASE_TEST_STORE"; -/// The transaction that [`transaction_role`] runs: `acquire`, `release`, `tombstone`, or -/// `retire`. -const ROLE_ACTION: &str = "NVX_TEMPLATE_LEASE_TEST_ACTION"; -/// The store generation of the transaction of [`transaction_role`]. -const ROLE_GENERATION: &str = "NVX_TEMPLATE_LEASE_TEST_GENERATION"; -/// The directory of the sandbox of the transaction of [`transaction_role`], if it has one. -const ROLE_SANDBOX: &str = "NVX_TEMPLATE_LEASE_TEST_SANDBOX"; -/// The line that [`transaction_role`] prints once it is ready, before it waits for a line on its -/// standard input. -const READY: &str = "nvx-template-lease-test-ready"; -/// The start of the line in which [`transaction_role`] prints how its transaction ended. -const OUTCOME: &str = "nvx-template-lease-test-outcome "; -/// The step after which [`clone_role`] crashes: `after-record`, `after-binding`, or `acquired`, of -/// its acquisition; `before-spawn`, `after-spawn`, `after-identity`, or `committed`, of its -/// clone's launch; or `releasing`, of a release. -const ROLE_CRASH: &str = "NVX_TEMPLATE_LEASE_TEST_CRASH"; -/// The working directory of the VMM that [`clone_role`] launches. -const ROLE_VMM: &str = "NVX_TEMPLATE_LEASE_TEST_VMM"; -/// The start of the line in which [`clone_role`] prints the reference of the VMM that it spawned, -/// as JSON. -const REFERENCE: &str = "nvx-template-lease-test-vmm "; -/// The start of the line that [`clone_role`] prints as it crashes, before the step after which it -/// crashes. -const CRASHED: &str = "nvx-template-lease-test-crashed "; -/// File whose presence in its working directory marks a [`vmm_role`]. Without it, the role -/// returns at once, as it does when `cargo test -- --ignored` runs it. -const VMM_MARKER: &str = "vmm-role"; -/// File whose appearance in its working directory ends a [`vmm_role`]. -const EXIT_MARKER: &str = "exit"; -/// The capability of the VMMs of these tests' clones. -const CAPABILITY: [u8; CAPABILITY_LEN] = [7; CAPABILITY_LEN]; -/// Longest wait for a VMM that is expected to exit, for a sandbox's lock, or for a reattach to -/// become determinate. -const TIMEOUT: Duration = Duration::from_secs(60); -/// Longest life of a [`vmm_role`] that nothing ends, so that a failed test leaves no process. -const VMM_LIFETIME: Duration = Duration::from_secs(180); - -/// Returns the files that these tests' callers keep in their sandboxes. -fn files() -> CallerFiles { - CallerFiles::new(CONFIG).unwrap() -} - -/// Provisions the sandbox `name` in `parent`, and returns its directory, with its links resolved, -/// as a lease names it. -fn provision(parent: &Path, name: &str) -> PathBuf { - let dir = parent.join(name); - let locked = SandboxStore::lock_for_provision(&dir, &files(), None).unwrap(); - fs::write(dir.join(CONFIG), b"{}").unwrap(); - drop(locked); - filesystem::canonicalize(&dir).unwrap() -} - -/// Returns the time of the host that captured `generation`, during the boot of the capture, -/// `seconds` after it. -fn after(generation: &Generation, seconds: u64) -> HostTime { - let time = generation.time(); - HostTime { - host_id: time.host_id, - boot_id: time.boot_id, - clock: time.host_clock, - monotonic_ns: time.capture_monotonic_ns + seconds * SECOND, - utc_ns: time.capture_utc_ns + seconds * SECOND, - } -} - -/// Leases `generation` of `store` to the sandbox in `sandbox`, a second after its capture, and -/// releases the sandbox's lock. -fn leased(store: &Store, generation: &Generation, sandbox: &Path) -> Lease { - store - .acquire( - generation.generation(), - sandbox, - &files(), - &after(generation, 1), - None, - ) - .unwrap() - .lease -} - -/// Returns the error with which leasing `generation` of `store` to the sandbox in `sandbox` at -/// `now` fails. -fn refusal(store: &Store, generation: StoreGeneration, sandbox: &Path, now: &HostTime) -> Error { - store - .acquire(generation, sandbox, &files(), now, None) - .unwrap_err() -} - -/// Releases the lease of the sandbox in `sandbox` from `store`, under the sandbox's lock. -fn release(store: &Store, sandbox: &Path) -> Result> { - let mut locked = SandboxStore::lock_existing(sandbox, &files(), None).unwrap(); - store.release(&mut locked) -} - -/// Returns the lease that the clone binding of the sandbox in `sandbox` records, if any. -fn binding(sandbox: &Path) -> Option { - Lease::of(&SandboxStore::open(sandbox).unwrap()).unwrap() -} - -/// Returns the store generation `generation`, which must not be 0. -fn store_generation(generation: u64) -> StoreGeneration { - StoreGeneration::new(NonZeroU64::new(generation).unwrap()) -} - -/// Returns a runtime identity under this crate's contracts. -fn runtime() -> RuntimeIdentity { - RuntimeIdentity::new(artifact('a'), artifact('b'), artifact('c')) -} - -/// Records a VM of the sandbox in `sandbox`, as the commit of its launch does. -fn record_vm(sandbox: &Path) { - let record = json!({ - "format": FORMAT, - "launch": "0123456789abcdef0123456789abcdef", - "process": { "pid": 42, "startTime": 4242, "boot": "ab".repeat(16) }, - "endpoint": "/state/sandbox/control.sock", - "runtime": serde_json::to_value(runtime()).unwrap(), - }); - write_private(&sandbox.join(RUNTIME_NAME), record.to_string().as_bytes()).unwrap(); -} - -/// Ends, through `locked`, a launch that spawned nothing, and whose log is closed. -/// -/// A process that a concurrent test spawned may still hold the log, until it executes its -/// program, and a later recovery then ends the launch. -fn recover_unspawned(locked: &mut LockedStore) { - assert!(released(|| locked.recover().unwrap() == Recovery::Exited)); -} - -/// Returns the file at `path` as JSON. -fn json_of(path: &Path) -> Value { - serde_json::from_slice(&fs::read(path).unwrap()).unwrap() -} - -/// Returns the identities of `leases`, sorted, and separated by spaces. -fn ids(leases: &[Lease]) -> String { - let mut ids: Vec = leases.iter().map(|lease| lease.id().to_string()).collect(); - ids.sort(); - ids.join(" ") -} - -/// Returns the arguments that run only the role `name` of this test executable, on libtest's main -/// thread, so that libtest prints the role's name before the role's own output. -fn role(name: &str) -> Vec { - [ - "--exact", - &format!("host::template::store::leases::tests::{name}"), - "--ignored", - "--nocapture", - "--test-threads=1", - ] - .map(OsString::from) - .to_vec() -} - -/// Returns the error with which `locked` refuses to begin the launch `launch`. -fn launch_refusal(locked: &mut LockedStore, launch: &LaunchId) -> Error { - match locked.begin_launch(launch, &runtime(), &CAPABILITY) { - Ok(_) => panic!("the launch {launch} began"), - Err(error) => error, - } -} - -#[test] -fn leases_bind_clones_to_the_generations_that_they_restore() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let first = provision(directory.path(), "first"); - let acquired = store - .acquire( - published.generation(), - &first, - &files(), - &after(&published, 1), - None, - ) - .unwrap(); - let lease = acquired.lease.clone(); - assert_eq!(lease.store(), store.id()); - assert_eq!(lease.generation(), published.generation()); - assert_eq!(lease.key(), published.key()); - assert_eq!(lease.sandbox(), first); - assert_eq!( - lease.template(), - TemplateReference { - generation: published.generation(), - key: Identity::from(published.key()), - } - ); - // The caller keeps the sandbox's lock, under which it launches the clone, until it drops it. - assert_eq!(acquired.locked.dir(), first); - let busy = SandboxStore::lock_existing(&first, &files(), Some(Instant::now())).unwrap_err(); - assert_eq!(busy.code(), ErrorCode::SandboxBusy, "{busy}"); - drop(acquired); - // The clone binding in the sandbox and the record in the store are the same lease. - assert_eq!(binding(&first), Some(lease.clone())); - assert_eq!( - store.leases(published.generation()).unwrap(), - std::slice::from_ref(&lease) - ); - // A sandbox holds one lease at a time. - let error = refusal( - &store, - published.generation(), - &first, - &after(&published, 2), - ); - assert_eq!(error.code(), ErrorCode::SandboxLeased, "{error}"); - assert!(error.message().contains(&lease.id().to_string()), "{error}"); - // Other sandboxes lease the generation too, and other generations. - let second = provision(directory.path(), "second"); - let mut leases = vec![lease, leased(&store, &published, &second)]; - leases.sort_by_key(Lease::id); - let other = publish(&store, &ManifestSpec::template(), storage(2)); - let third = provision(directory.path(), "third"); - let other_lease = leased(&store, &other, &third); - assert_eq!(store.leases(published.generation()).unwrap(), leases); - assert_eq!(store.leases(other.generation()).unwrap(), [other_lease]); -} - -#[test] -fn acquisitions_refuse_what_they_cannot_lease_and_lease_nothing() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let now = after(&published, 1); - // A generation that the store does not index. - let error = refusal(&store, store_generation(9), &sandbox, &now); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); - // One that is not eligible at the host's time: no longer, or not yet. - let expired = after(&published, MAX_DOWNTIME_NS / SECOND + 1); - let mut early = after(&published, 0); - early.monotonic_ns -= 1; - for (now, condition) in [ - (expired, "is no longer eligible"), - (early, "is not eligible yet"), - ] { - let error = refusal(&store, generation, &sandbox, &now); - assert_eq!(error.code(), ErrorCode::TemplateIneligible, "{error}"); - assert!(error.message().contains(condition), "{error}"); - } - // On Linux, a payload that is not a directory of the current user with mode 0700, which a - // launch of the clone requires, and through which a release ends the lease: it opens the - // payload for reading, and removes the clone's socket from it. A user other than root cannot - // open a payload of mode 0300 for reading at all. - #[cfg(target_os = "linux")] - { - use std::os::unix::fs::PermissionsExt; - - let payload = store.payload_dir(generation); - let unopened = format!( - "cannot open the payload of the generation {}", - payload.display() - ); - for mode in [0o300, 0o500] { - fs::set_permissions(&payload, fs::Permissions::from_mode(mode)).unwrap(); - let error = refusal(&store, generation, &sandbox, &now); - fs::set_permissions(&payload, fs::Permissions::from_mode(0o700)).unwrap(); - assert_eq!(error.kind(), ErrorKind::Io, "{error}"); - let refused = format!( - "the payload of the generation {} has mode {mode:o}, not the mode 700 that the \ - directory of a control socket must have", - payload.display() - ); - assert!( - error.message() == refused || (mode == 0o300 && error.message() == unopened), - "{error}" - ); - } - // A link in the place of the store's directory, of the directory of the generations, or of - // the payload, even to a directory that holds the same files: a release, which follows - // none of them, could not open the payload. - let root = store.dir().to_owned(); - let generations = payload.parent().unwrap().to_owned(); - let elsewhere = directory.path().join("elsewhere"); - for linked in [&root, &generations, &payload] { - fs::rename(linked, &elsewhere).unwrap(); - std::os::unix::fs::symlink(&elsewhere, linked).unwrap(); - let error = refusal(&store, generation, &sandbox, &now); - fs::remove_file(linked).unwrap(); - fs::rename(&elsewhere, linked).unwrap(); - assert_eq!(error.kind(), ErrorKind::Io, "{error}"); - assert_eq!(error.message(), unopened, "{error}"); - let source = std::error::Error::source(&error).unwrap().to_string(); - assert!( - source.ends_with("is a link, or not a directory"), - "{source}" - ); - } - } - // A sandbox that is not provisioned: its directory is missing, or holds no configuration. - let unprovisioned = directory.path().join("unprovisioned"); - drop(SandboxStore::lock_for_provision(&unprovisioned, &files(), None).unwrap()); - for dir in [directory.path().join("missing"), unprovisioned] { - let error = refusal(&store, generation, &dir, &now); - assert_eq!(error.code(), ErrorCode::SandboxNotProvisioned, "{error}"); - } - // A sandbox that records a launch, or a VM, which may still run. - let mut locked = SandboxStore::lock_existing(&sandbox, &files(), None).unwrap(); - drop( - locked - .begin_launch( - &LaunchId::generate().unwrap(), - &runtime(), - &[7; CAPABILITY_LEN], - ) - .unwrap(), - ); - drop(locked); - let error = refusal(&store, generation, &sandbox, &now); - assert_eq!(error.code(), ErrorCode::SandboxRunning, "{error}"); - let mut locked = SandboxStore::lock_existing(&sandbox, &files(), None).unwrap(); - recover_unspawned(&mut locked); - record_vm(&sandbox); - drop(locked); - let error = refusal(&store, generation, &sandbox, &now); - assert_eq!(error.code(), ErrorCode::SandboxRunning, "{error}"); - let mut locked = SandboxStore::lock_existing(&sandbox, &files(), None).unwrap(); - locked.clear_runtime().unwrap(); - drop(locked); - // A generation that a tombstone quarantines. - store.tombstone(generation, None).unwrap(); - let error = refusal(&store, generation, &sandbox, &now); - assert_eq!(error.code(), ErrorCode::TemplateTombstoned, "{error}"); - // None of them leased anything. - assert!(store.leases(generation).unwrap().is_empty()); - assert_eq!(binding(&sandbox), None); - assert!(!store.dir().join(LEASES_NAME).exists()); -} - -#[test] -fn releases_end_leases_once_their_clones_have_exited() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let mut acquired = store - .acquire(generation, &sandbox, &files(), &after(&published, 1), None) - .unwrap(); - let lease = acquired.lease.clone(); - // While a launch of the clone, or its VM, may run, its lease stays. - drop( - acquired - .locked - .begin_launch(lease.launch(), &runtime(), &[7; CAPABILITY_LEN]) - .unwrap(), - ); - let error = store.release(&mut acquired.locked).unwrap_err(); - assert_eq!(error.code(), ErrorCode::SandboxRunning, "{error}"); - recover_unspawned(&mut acquired.locked); - record_vm(&sandbox); - let error = store.release(&mut acquired.locked).unwrap_err(); - assert_eq!(error.code(), ErrorCode::SandboxRunning, "{error}"); - assert_eq!(binding(&sandbox), Some(lease.clone())); - assert_eq!( - store.leases(generation).unwrap(), - std::slice::from_ref(&lease) - ); - // Once its VM has exited, the release removes the binding, and then the store's record. - acquired.locked.clear_runtime().unwrap(); - assert_eq!( - store.release(&mut acquired.locked).unwrap(), - Some(lease.clone()) - ); - drop(acquired); - assert_eq!(binding(&sandbox), None); - assert!(store.leases(generation).unwrap().is_empty()); - assert_eq!(release(&store, &sandbox).unwrap(), None); - // The sandbox may lease the generation again, under another identity. - let again = leased(&store, &published, &sandbox); - assert_ne!(again.id(), lease.id()); - // Another store's release keeps the lease. - let (_other_directory, other) = new_store(); - let error = release(&other, &sandbox).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Other, "{error}"); - assert_eq!(binding(&sandbox), Some(again.clone())); - assert_eq!(store.leases(generation).unwrap(), [again]); -} - -#[test] -fn a_lease_allows_one_launch_of_its_clone_under_the_lock_that_acquired_it() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let sandbox = provision(directory.path(), "sandbox"); - let mut acquired = store - .acquire( - published.generation(), - &sandbox, - &files(), - &after(&published, 1), - None, - ) - .unwrap(); - let reserved = acquired.lease.launch().clone(); - let once = - format!("whose launch {reserved} begins once, under the lock that acquired the lease"); - // No launch but the reserved one. - let other = LaunchId::generate().unwrap(); - let error = launch_refusal(&mut acquired.locked, &other); - assert_eq!(error.code(), ErrorCode::SandboxLeased, "{error}"); - assert!( - error.message().ends_with(&format!( - "which reserves the launch {reserved}, not {other}" - )), - "{error}" - ); - // The reserved launch, which its journal names, once. - drop( - acquired - .locked - .begin_launch(&reserved, &runtime(), &CAPABILITY) - .unwrap(), - ); - assert_eq!( - acquired.locked.journal().unwrap().unwrap().launch(), - &reserved - ); - recover_unspawned(&mut acquired.locked); - for launch in [&reserved, &other] { - let error = launch_refusal(&mut acquired.locked, launch); - assert_eq!(error.code(), ErrorCode::SandboxLeased, "{error}"); - } - let error = launch_refusal(&mut acquired.locked, &reserved); - assert!(error.message().ends_with(&once), "{error}"); - // Once a release ends the lease, the sandbox launches as any launch. - assert_eq!( - store.release(&mut acquired.locked).unwrap(), - Some(acquired.lease.clone()) - ); - drop( - acquired - .locked - .begin_launch(&other, &runtime(), &CAPABILITY) - .unwrap(), - ); - recover_unspawned(&mut acquired.locked); - drop(acquired); - // The launch of a lease whose acquisition's lock is gone, as a crash leaves it, begins under - // no other lock. - let lease = leased(&store, &published, &sandbox); - let mut locked = SandboxStore::lock_existing(&sandbox, &files(), None).unwrap(); - let error = launch_refusal(&mut locked, lease.launch()); - assert_eq!(error.code(), ErrorCode::SandboxLeased, "{error}"); - assert!( - error.message().ends_with(&format!( - "whose launch {} begins once, under the lock that acquired the lease", - lease.launch() - )), - "{error}" - ); -} - -#[test] -fn tombstones_quarantine_generations_and_keep_their_leases() { - let (directory, store) = new_store(); - let first = publish(&store, &ManifestSpec::template(), storage(1)); - let second = publish(&store, &ManifestSpec::template(), storage(2)); - let leasing = provision(directory.path(), "leasing"); - let lease = leased(&store, &second, &leasing); - let now = after(&second, 1); - assert_eq!( - store.select(second.key(), &now).unwrap(), - Some(second.clone()) - ); - assert!(store.tombstones().unwrap().is_empty()); - // A tombstone stops new leases, and selection skips the generation, whose leases stay. - store.tombstone(second.generation(), None).unwrap(); - store.tombstone(second.generation(), None).unwrap(); - assert_eq!(store.tombstones().unwrap(), [second.generation()]); - let sandbox = provision(directory.path(), "sandbox"); - let error = refusal(&store, second.generation(), &sandbox, &now); - assert_eq!(error.code(), ErrorCode::TemplateTombstoned, "{error}"); - assert_eq!(binding(&leasing), Some(lease.clone())); - assert_eq!(store.leases(second.generation()).unwrap(), [lease]); - assert_eq!( - store.select(second.key(), &now).unwrap(), - Some(first.clone()) - ); - leased(&store, &first, &sandbox); - // A tombstone needs a generation that the store indexes. - let error = store.tombstone(store_generation(9), None).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); - assert_eq!(store.tombstones().unwrap(), [second.generation()]); -} - -#[test] -fn selection_takes_the_latest_eligible_generation_of_a_key() { - let (_directory, store) = new_store(); - let first = publish(&store, &ManifestSpec::template(), storage(1)); - let mut spec = ManifestSpec::template(); - spec.time().capture_utc_ns += 10 * DAY; - spec.time().capture_monotonic_ns += 10 * DAY; - let second = publish(&store, &spec, storage(2)); - let other = publish_with( - &store, - &network_class(), - &ManifestSpec::with_network(), - storage(3), - ); - assert_eq!(second.key(), first.key()); - assert_ne!(other.key(), first.key()); - // The host's time `days` after the first capture, during its boot. - let at = |days: u64| HostTime { - monotonic_ns: first.time().capture_monotonic_ns + days * DAY, - utc_ns: first.time().capture_utc_ns + days * DAY, - ..after(&first, 0) - }; - let select = |now: &HostTime| store.select(first.key(), now).unwrap(); - // Before the second capture, only the first generation is eligible; then both are, and the - // latest is selected; after the first's window, only the second is, and after both, none. - assert_eq!(select(&at(1)), Some(first.clone())); - assert_eq!(select(&at(11)), Some(second.clone())); - assert_eq!(select(&at(35)), Some(second.clone())); - assert_eq!(select(&at(45)), None); - // A quarantined generation is skipped. - store.tombstone(second.generation(), None).unwrap(); - assert_eq!(select(&at(11)), Some(first.clone())); - assert_eq!(select(&at(35)), None); - // Each key selects among its own generations. - assert_eq!(store.select(other.key(), &at(1)).unwrap(), Some(other)); - assert_eq!( - store - .select(KeyIdentity::from_bytes([0; 32]), &at(1)) - .unwrap(), - None - ); -} - -#[test] -fn retirement_deletes_a_generation_once_its_leases_drain() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - // The generation stays, quarantined, while its lease lasts. - for _ in 0..2 { - assert_eq!( - store.retire(generation, None).unwrap(), - Retirement::Draining(vec![lease.clone()]) - ); - } - assert_eq!(store.tombstones().unwrap(), [generation]); - assert_eq!(store.index().unwrap().get(generation), Some(&published)); - assert!(store.snapshot_dir(&published).is_ok()); - let other = provision(directory.path(), "other"); - let error = refusal(&store, generation, &other, &after(&published, 1)); - assert_eq!(error.code(), ErrorCode::TemplateTombstoned, "{error}"); - // Once its clone releases it, the generation goes: its payload, its entry, and its tombstone. - assert_eq!(release(&store, &sandbox).unwrap(), Some(lease)); - assert_eq!(store.retire(generation, None).unwrap(), Retirement::Retired); - assert!(!store.payload_dir(generation).exists()); - assert_eq!(store.index().unwrap().get(generation), None); - assert!(store.tombstones().unwrap().is_empty()); - assert!(store.leases(generation).unwrap().is_empty()); - // Retiring it again finds it retired, and its store generation is never assigned again. - assert_eq!(store.retire(generation, None).unwrap(), Retirement::Retired); - let next = publish(&store, &ManifestSpec::template(), storage(1)); - assert!(next.generation() > generation); - let unassigned = store_generation(next.generation().get().get() + 1); - let error = store.retire(unassigned, None).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); - let error = refusal(&store, generation, &other, &after(&published, 1)); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); -} - -#[test] -fn retirement_removes_the_leases_that_no_clone_binds() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandboxes: Vec = ["released", "removed", "leased-again", "bound"] - .into_iter() - .map(|name| provision(directory.path(), name)) - .collect(); - let leases: Vec = sandboxes - .iter() - .map(|sandbox| leased(&store, &published, sandbox)) - .collect(); - // A release that was interrupted once it had removed the clone binding. - fs::remove_file(sandboxes[0].join(CLONE_BINDING_NAME)).unwrap(); - // A sandbox whose removal removed its binding with the rest. - let locked = SandboxStore::lock_existing(&sandboxes[1], &files(), None).unwrap(); - assert!(locked.remove().unwrap().is_none()); - assert!(!sandboxes[1].exists()); - // A sandbox that leased the generation again after such a release. - fs::remove_file(sandboxes[2].join(CLONE_BINDING_NAME)).unwrap(); - let again = leased(&store, &published, &sandboxes[2]); - let mut listed = leases.clone(); - listed.push(again.clone()); - listed.sort_by_key(Lease::id); - assert_eq!(store.leases(generation).unwrap(), listed); - // The retirement removes the records of the first three, and drains the bound ones. - let mut live = vec![again, leases[3].clone()]; - live.sort_by_key(Lease::id); - assert_eq!( - store.retire(generation, None).unwrap(), - Retirement::Draining(live.clone()) - ); - assert_eq!(store.leases(generation).unwrap(), live); - for sandbox in &sandboxes[2..] { - assert!(release(&store, sandbox).unwrap().is_some()); - } - assert_eq!(store.retire(generation, None).unwrap(), Retirement::Retired); -} - -#[test] -fn interrupted_retirements_finish() { - let (directory, store) = new_store(); - let first = publish(&store, &ManifestSpec::template(), storage(1)); - let second = publish(&store, &ManifestSpec::template(), storage(2)); - let third = publish(&store, &ManifestSpec::template(), storage(3)); - // One that was interrupted once it had removed the payload leaves the generation indexed and - // quarantined. - store.tombstone(first.generation(), None).unwrap(); - fs::remove_dir_all(store.payload_dir(first.generation())).unwrap(); - assert_eq!( - store.retire(first.generation(), None).unwrap(), - Retirement::Retired - ); - assert_eq!(store.index().unwrap().get(first.generation()), None); - // One that was interrupted once it had removed the entry from the index leaves the tombstone. - store.tombstone(second.generation(), None).unwrap(); - fs::remove_dir_all(store.payload_dir(second.generation())).unwrap(); - remove(&store, second.generation()); - assert_eq!(store.tombstones().unwrap(), [second.generation()]); - assert_eq!( - store.retire(second.generation(), None).unwrap(), - Retirement::Retired - ); - assert!(store.tombstones().unwrap().is_empty()); - // Interrupted replacements of records leave temporary files, which listings skip, and which - // a retirement removes under the store's lock. - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &third, &sandbox); - store.tombstone(third.generation(), None).unwrap(); - let temporaries = [ - store - .dir() - .join(LEASES_NAME) - .join(format!(".{}.json.42.tmp", lease.id())), - store.dir().join(TOMBSTONES_NAME).join(".7.json.1.tmp"), - ]; - for temporary in &temporaries { - fs::write(temporary, b"{").unwrap(); - } - assert_eq!( - store.leases(third.generation()).unwrap(), - std::slice::from_ref(&lease) - ); - assert_eq!(store.tombstones().unwrap(), [third.generation()]); - assert_eq!( - store.retire(third.generation(), None).unwrap(), - Retirement::Draining(vec![lease]) - ); - for temporary in &temporaries { - assert!(!temporary.exists(), "{}", temporary.display()); - } -} - -#[test] -fn acquisitions_make_their_records_durable_before_their_bindings_and_both_before_they_return() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let leases_dir = store.dir().join(LEASES_NAME); - let first = provision(directory.path(), "first"); - let second = provision(directory.path(), "second"); - // Each sync notes its directory, the records of leases in the store, and the bindings of the - // sandboxes. - let synced = RefCell::new(Vec::new()); - let sync = |dir: &Path| -> Result<()> { - let records = fs::read_dir(&leases_dir).map_or(0, Iterator::count); - let bindings = [&first, &second] - .into_iter() - .filter(|sandbox| sandbox.join(CLONE_BINDING_NAME).exists()) - .count(); - synced - .borrow_mut() - .push((dir.to_owned(), records, bindings)); - Ok(()) - }; - // An acquisition that creates the directory of the leases, but cannot make its entry in the - // store durable, leases nothing, and the next acquisition syncs the store's directory again. - let now = after(&published, 1); - let unsynced_store = |dir: &Path| { - if dir == store.dir() { - Err(Error::io("the disk failed")) - } else { - Ok(()) - } - }; - let error = store - .acquire_with( - published.generation(), - first.clone(), - &files(), - &now, - None, - &unsynced_store, - ) - .unwrap_err(); - assert_eq!(error.message(), "the disk failed"); - assert!(leases_dir.is_dir()); - assert_eq!(binding(&first), None); - for sandbox in [&first, &second] { - let acquired = store - .acquire_with( - published.generation(), - sandbox.clone(), - &files(), - &now, - None, - &sync, - ) - .unwrap(); - drop(acquired); - } - assert_eq!( - *synced.borrow(), - [ - // The first makes the entry of the directory of the leases durable, which no - // acquisition of this process did before, ahead of any record in it, and then the - // store's own entry. - (store.dir().to_owned(), 0, 0), - (store.dir().parent().unwrap().to_owned(), 0, 0), - // Each makes its record durable before its binding, and its binding before it - // returns. - (leases_dir.clone(), 1, 0), - (first.clone(), 1, 1), - (leases_dir.clone(), 2, 1), - (second.clone(), 2, 2), - ] - ); -} - -#[test] -fn an_acquisition_keeps_its_record_while_its_binding_may_outlast_a_crash() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let now = after(&published, 1); - let sandbox = provision(directory.path(), "sandbox"); - let leases_dir = store.dir().join(LEASES_NAME); - let failed = || Err(Error::io("the disk failed")); - let acquire = |sync: SyncDir<'_>| { - store - .acquire_with(generation, sandbox.clone(), &files(), &now, None, sync) - .unwrap_err() - }; - // A record that cannot be made durable goes before any binding names it. - let error = acquire(&|dir: &Path| if dir == leases_dir { failed() } else { Ok(()) }); - assert_eq!(error.message(), "the disk failed"); - assert_eq!(binding(&sandbox), None); - assert!(store.leases(generation).unwrap().is_empty()); - // A binding that cannot be made durable goes, and its record once the binding's removal is - // durable. - let first = Cell::new(true); - let error = acquire(&|dir: &Path| { - if dir == sandbox && first.replace(false) { - failed() - } else { - Ok(()) - } - }); - assert_eq!(error.message(), "the disk failed"); - assert_eq!(binding(&sandbox), None); - assert!(store.leases(generation).unwrap().is_empty()); - // A binding whose removal cannot be made durable either may outlast a crash, so its record - // stays, and keeps the generation, until a retirement finds that no binding names it. - let error = acquire(&|dir: &Path| if dir == sandbox { failed() } else { Ok(()) }); - let [kept]: [Lease; 1] = store.leases(generation).unwrap().try_into().unwrap(); - assert_eq!( - error.message(), - format!( - "the disk failed; the sandbox in {} may hold the lease {} still, which a release ends", - sandbox.display(), - kept.id() - ) - ); - assert_eq!(binding(&sandbox), None); - assert_eq!(store.retire(generation, None).unwrap(), Retirement::Retired); -} - -#[test] -fn a_release_makes_the_removal_of_its_binding_durable_before_it_removes_its_record() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - let record = store - .dir() - .join(LEASES_NAME) - .join(format!("{}.json", lease.id())); - let payload = store.payload_dir(generation); - let socket = store.clone_socket_path(&lease); - // Each sync notes its directory, and whether any file that the clone kept of its launch, the - // control socket of its VMM and the scratch file that an earlier version left for a clone in - // RAM included, the binding, and the record existed then. - let kept = [ - CLONE_SCRATCH_NAME, - CLONE_ACTIVATION_NAME, - CLONE_PROGRESS_NAME, - CLONE_FAILURE_NAME, - RESTORE_READY_NAME, - ]; - let synced = RefCell::new(Vec::new()); - let note = |dir: &Path| { - let clone = kept - .iter() - .any(|name| fs::symlink_metadata(sandbox.join(name)).is_ok()) - || fs::symlink_metadata(&socket).is_ok(); - let bound = sandbox.join(CLONE_BINDING_NAME).exists(); - synced - .borrow_mut() - .push((dir.to_owned(), clone, bound, record.exists())); - }; - // The release syncs the payload through its handle, which holds the generation's snapshot. - let snapshot = platform::file_id(&payload.join(SNAPSHOT_NAME)).unwrap(); - let sync_payload = |directory: &platform::Directory| -> io::Result<()> { - assert_eq!(directory.entry_id(SNAPSHOT_NAME).unwrap(), Some(snapshot)); - note(&payload); - Ok(()) - }; - // The release syncs the sandbox's directory through its handle, which holds the lock's file. - let lock = platform::file_id(&sandbox.join(LOCK_NAME)).unwrap(); - let sync_sandbox = |directory: &platform::Directory| -> io::Result<()> { - assert_eq!(directory.entry_id(LOCK_NAME).unwrap(), Some(lock)); - note(&sandbox); - Ok(()) - }; - for name in kept { - fs::write(sandbox.join(name), b"").unwrap(); - } - // A socket that a VMM on Linux left in the payload, which a file stands for. - if cfg!(unix) { - fs::write(&socket, b"").unwrap(); - } - let mut locked = SandboxStore::lock_existing(&sandbox, &files(), None).unwrap(); - assert_eq!( - store - .release_with(&mut locked, &|| {}, &sync_payload, &sync_sandbox) - .unwrap(), - Some(lease) - ); - drop(locked); - let mut expected = vec![(sandbox.clone(), false, false, true)]; - if cfg!(unix) { - expected.insert(0, (payload.clone(), false, true, true)); - } - assert_eq!(*synced.borrow(), expected); - assert!(!record.exists()); - assert!(fs::symlink_metadata(&socket).is_err()); - // A release under the lock that acquired the lease ends the lock's reservation before it - // removes the clone's files, such as the record of its failure, which ends the clone's - // launches until the release, so that one that fails after their removal leaves no launch to - // begin. - let Acquired { - lease: again, - mut locked, - } = store - .acquire( - generation, - sandbox.as_path(), - &files(), - &after(&published, 1), - None, - ) - .unwrap(); - assert_eq!(locked.reserved(), Some(again.launch())); - let failure = sandbox.join(CLONE_FAILURE_NAME); - fs::write(&failure, b"").unwrap(); - // A release that cannot make the socket's removal durable keeps the binding, and the lease. - if cfg!(unix) { - let error = store - .release_with( - &mut locked, - &|| {}, - &|_| Err(io::Error::other("the disk failed")), - &|_| Ok(()), - ) - .unwrap_err(); - assert_eq!( - error.message(), - format!("cannot sync the directory {}", payload.display()) - ); - assert_eq!(cause(&error), "the disk failed"); - assert_eq!(binding(&sandbox), Some(again.clone())); - assert_eq!(locked.reserved(), None); - assert!(fs::symlink_metadata(&failure).is_err()); - } - // A release that cannot make the binding's removal durable keeps the record, which a - // retirement removes once no binding names it. - let error = store - .release_with(&mut locked, &|| {}, &|_| Ok(()), &|_| { - Err(io::Error::other("the disk failed")) - }) - .unwrap_err(); - assert_eq!(locked.reserved(), None); - assert!(fs::symlink_metadata(&failure).is_err()); - drop(locked); - assert_eq!( - error.message(), - format!("cannot sync the directory {}", sandbox.display()) - ); - assert_eq!(cause(&error), "the disk failed"); - assert_eq!(binding(&sandbox), None); - assert_eq!(store.leases(generation).unwrap(), [again]); - assert_eq!(store.retire(generation, None).unwrap(), Retirement::Retired); -} - -#[test] -fn a_retirement_makes_each_step_durable_before_the_next() { - let (_directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let tombstones = store.dir().join(TOMBSTONES_NAME); - let tombstone = tombstones.join(format!("{generation}.json")); - let payload = store.payload_dir(generation); - // Each sync notes its directory, and whether the tombstone, the payload, and the - // generation's entry in the index existed then. - let synced = RefCell::new(Vec::new()); - let sync = |dir: &Path| -> Result<()> { - let indexed = store.index()?.get(generation).is_some(); - synced.borrow_mut().push(( - dir.to_owned(), - tombstone.exists(), - payload.exists(), - indexed, - )); - Ok(()) - }; - assert_eq!( - store.retire_with(generation, None, &sync).unwrap(), - Retirement::Retired - ); - assert_eq!( - *synced.borrow(), - [ - // The new directory of the tombstones, and the store's own entry, before the - // tombstone in it. - (store.dir().to_owned(), false, true, true), - (store.dir().parent().unwrap().to_owned(), false, true, true), - // The tombstone, before anything goes. - (tombstones.clone(), true, true, true), - // The payload's removal, before the index changes. - (store.dir().join(GENERATIONS_NAME), true, false, true), - // The index, before the tombstone goes. - (store.dir().to_owned(), true, false, false), - ] - ); - assert!(!tombstone.exists()); -} - -#[test] -fn a_retirement_makes_the_absence_of_a_binding_durable_before_it_removes_its_record() { - let (directory, store) = new_store(); - let root = filesystem::canonicalize(directory.path()).unwrap(); - let first = publish(&store, &ManifestSpec::template(), storage(1)); - let second = publish(&store, &ManifestSpec::template(), storage(2)); - let third = publish(&store, &ManifestSpec::template(), storage(3)); - let fourth = publish(&store, &ManifestSpec::template(), storage(4)); - let released = provision(&root, "released"); - let removed = provision(&root, "removed"); - let reprovisioned = provision(&root, "reprovisioned"); - let sandboxes = root.join("sandboxes"); - fs::create_dir(&sandboxes).unwrap(); - let removed_through_link = provision(&sandboxes, "removed"); - let released_lease = leased(&store, &first, &released); - let removed_lease = leased(&store, &second, &removed); - let linked_lease = leased(&store, &third, &removed_through_link); - let reprovisioned_lease = leased(&store, &fourth, &reprovisioned); - // The directory that holds a sandbox moves, and a link at its path names it, so the path that - // names the sandbox's lease runs through the link. - let moved = root.join("moved"); - fs::rename(&sandboxes, &moved).unwrap(); - link_dir(&moved, &sandboxes); - // A release that removed its binding, but did not make that durable, and removals of - // sandboxes, one of which is provisioned again at its path. - fs::remove_file(released.join(CLONE_BINDING_NAME)).unwrap(); - for sandbox in [&removed, &removed_through_link, &reprovisioned] { - let locked = SandboxStore::lock_existing(sandbox, &files(), None).unwrap(); - assert!(locked.remove().unwrap().is_none()); - } - assert_eq!(provision(&root, "reprovisioned"), reprovisioned); - let record = |lease: &Lease| { - store - .dir() - .join(LEASES_NAME) - .join(format!("{}.json", lease.id())) - }; - // A retirement that cannot make the binding's absence durable keeps its record, and the - // generation. - let unsynced_sandbox = |dir: &Path| { - if dir == released { - Err(Error::io("the disk failed")) - } else { - Ok(()) - } - }; - let error = store - .retire_with(first.generation(), None, &unsynced_sandbox) - .unwrap_err(); - assert_eq!(error.message(), "the disk failed"); - assert!(record(&released_lease).exists()); - assert!(store.snapshot_dir(&first).is_ok()); - // A record goes once the binding's absence is durable: in the sandbox's directory, if it is - // one, and in the directory that holds its entry, the nearest of its ancestors that remains, - // through the links that its path names. The directory of a sandbox that was provisioned - // again never held the binding, which went with the directory that its entry named before. - for (generation, lease, dirs) in [ - ( - &first, - &released_lease, - vec![released.as_path(), root.as_path()], - ), - (&second, &removed_lease, vec![root.as_path()]), - (&third, &linked_lease, vec![sandboxes.as_path()]), - ( - &fourth, - &reprovisioned_lease, - vec![reprovisioned.as_path(), root.as_path()], - ), - ] { - let path = record(lease); - let synced = RefCell::new(Vec::new()); - let sync = |synced_dir: &Path| -> Result<()> { - synced - .borrow_mut() - .push((synced_dir.to_owned(), path.exists())); - Ok(()) - }; - assert_eq!( - store - .retire_with(generation.generation(), None, &sync) - .unwrap(), - Retirement::Retired - ); - let mut expected = vec![(store.dir().join(TOMBSTONES_NAME), true)]; - expected.extend(dirs.into_iter().map(|dir| (dir.to_owned(), true))); - expected.extend([ - (store.dir().join(GENERATIONS_NAME), false), - (store.dir().to_owned(), false), - ]); - assert_eq!(*synced.borrow(), expected); - } - // A link to nothing hides the directory that held a sandbox, so the record stays. - let fifth = publish(&store, &ManifestSpec::template(), storage(5)); - let parked = root.join("parked"); - fs::create_dir(&parked).unwrap(); - let dangling = provision(&parked, "dangling"); - let dangling_lease = leased(&store, &fifth, &dangling); - let locked = SandboxStore::lock_existing(&dangling, &files(), None).unwrap(); - assert!(locked.remove().unwrap().is_none()); - let away = root.join("away"); - fs::rename(&parked, &away).unwrap(); - link_dir(&away, &parked); - fs::remove_dir(&away).unwrap(); - let error = store.retire(fifth.generation(), None).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Io, "{error}"); - assert!( - error.message().contains("cannot find the directory"), - "{error}" - ); - assert!(record(&dangling_lease).exists()); - assert!(store.snapshot_dir(&fifth).is_ok()); -} - -#[test] -fn a_lease_names_its_sandbox_by_its_path_with_its_links_resolved() { - let (directory, store) = new_store(); - let root = filesystem::canonicalize(directory.path()).unwrap(); - let first = publish(&store, &ManifestSpec::template(), storage(1)); - let second = publish(&store, &ManifestSpec::template(), storage(2)); - let sandboxes = root.join("sandboxes"); - fs::create_dir(&sandboxes).unwrap(); - let parent = provision(&sandboxes, "parent"); - let linked = provision(&sandboxes, "linked"); - // Paths through a directory that a `..` component passes through, and through a link, both of - // which go while the clones run. - let helper = root.join("helper"); - fs::create_dir(&helper).unwrap(); - let link = root.join("link"); - link_dir(&sandboxes, &link); - let mut leases = Vec::new(); - for (generation, sandbox, path) in [ - ( - &first, - &parent, - helper.join("..").join("sandboxes").join("parent"), - ), - (&second, &linked, link.join("linked")), - ] { - let acquired = store - .acquire( - generation.generation(), - &path, - &files(), - &after(generation, 1), - None, - ) - .unwrap(); - assert_eq!(acquired.lease.sandbox(), sandbox); - assert_eq!(acquired.locked.dir(), sandbox); - drop(acquired.locked); - assert_eq!(binding(sandbox), Some(acquired.lease.clone())); - leases.push(acquired.lease); - } - fs::remove_dir(&helper).unwrap(); - unlink_dir(&link); - // A retirement still finds both bindings, and keeps the generations. - for (generation, lease) in [&first, &second].into_iter().zip(leases) { - assert_eq!( - store.retire(generation.generation(), None).unwrap(), - Retirement::Draining(vec![lease]) - ); - } -} - -#[test] -fn an_acquisition_binds_the_sandbox_that_it_resolved_whatever_its_path_names_meanwhile() { - let (directory, store) = new_store(); - let root = filesystem::canonicalize(directory.path()).unwrap(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let (first, second) = (root.join("first"), root.join("second")); - fs::create_dir(&first).unwrap(); - fs::create_dir(&second).unwrap(); - let leased_sandbox = provision(&first, "sandbox"); - let other = provision(&second, "sandbox"); - let link = root.join("link"); - link_dir(&first, &link); - // The link names the other sandbox's directory once the acquisition has resolved its path, as - // it makes the lease's record durable. - let retargeted = Cell::new(false); - let retarget = |dir: &Path| -> Result<()> { - if dir == store.dir().join(LEASES_NAME) && !retargeted.replace(true) { - unlink_dir(&link); - link_dir(&second, &link); - } - Ok(()) - }; - let acquired = store - .acquire_with( - published.generation(), - link.join("sandbox"), - &files(), - &after(&published, 1), - None, - &retarget, - ) - .unwrap(); - assert!(retargeted.get()); - // The binding, and the lock, are those of the sandbox that the lease names. - assert_eq!(acquired.lease.sandbox(), leased_sandbox); - assert_eq!(acquired.locked.dir(), leased_sandbox); - drop(acquired.locked); - assert_eq!(binding(&leased_sandbox), Some(acquired.lease.clone())); - assert_eq!(binding(&other), None); - assert_eq!( - store.retire(published.generation(), None).unwrap(), - Retirement::Draining(vec![acquired.lease]) - ); -} - -#[test] -fn a_release_takes_the_lock_of_its_sandbox_through_the_path_of_its_lease() { - let (directory, store) = new_store(); - let root = filesystem::canonicalize(directory.path()).unwrap(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let (first, second) = (root.join("first"), root.join("second")); - fs::create_dir(&first).unwrap(); - fs::create_dir(&second).unwrap(); - let leased_sandbox = provision(&first, "sandbox"); - let other = provision(&second, "sandbox"); - let lease = leased(&store, &published, &leased_sandbox); - let other_lease = leased(&store, &published, &other); - let mut both = vec![lease.clone(), other_lease.clone()]; - both.sort_by_key(Lease::id); - let link = root.join("link"); - link_dir(&first, &link); - // A lock taken through a link refuses a release, which removes nothing, also once the link - // names another sandbox, whose lease a release would otherwise end without its lock: the - // directory that the link names then holds another lock file than the one locked. - let mut locked = SandboxStore::lock_existing(link.join("sandbox"), &files(), None).unwrap(); - for (retargeted, refusal) in [ - (false, "does not name it by the path of its lease"), - (true, "does not hold the lock of its sandbox"), - ] { - if retargeted { - unlink_dir(&link); - link_dir(&second, &link); - } - let error = store.release(&mut locked).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Other, "{error}"); - assert!(error.message().contains(refusal), "{error}"); - assert_eq!(binding(&leased_sandbox), Some(lease.clone())); - assert_eq!(binding(&other), Some(other_lease.clone())); - assert_eq!(store.leases(generation).unwrap(), both); - } - drop(locked); - // A lock taken through the lease's path ends it. - assert_eq!(release(&store, lease.sandbox()).unwrap(), Some(lease)); - assert_eq!(binding(&leased_sandbox), None); - assert_eq!(store.leases(generation).unwrap(), [other_lease]); -} - -/// A release whose path comes to name another sandbox, once a directory in the path is renamed, -/// and a link to the other sandbox's parent takes its place: only on Unix, since Windows refuses -/// to rename a directory that holds an open file, such as the lock's. -#[cfg(unix)] -#[test] -fn a_release_ends_the_lease_of_the_sandbox_whose_lock_it_holds_whatever_its_path_names_meanwhile() { - let (directory, store) = new_store(); - let root = filesystem::canonicalize(directory.path()).unwrap(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let (first, second) = (root.join("first"), root.join("second")); - fs::create_dir(&first).unwrap(); - fs::create_dir(&second).unwrap(); - let leased_sandbox = provision(&first, "sandbox"); - let other = provision(&second, "sandbox"); - let lease = leased(&store, &published, &leased_sandbox); - let other_lease = leased(&store, &published, &other); - let kept = [ - CLONE_SCRATCH_NAME, - CLONE_ACTIVATION_NAME, - CLONE_PROGRESS_NAME, - CLONE_FAILURE_NAME, - RESTORE_READY_NAME, - ]; - for sandbox in [&leased_sandbox, &other] { - for name in kept { - fs::write(sandbox.join(name), b"").unwrap(); - } - } - // Moves `parent` to `to`, and links `parent` to the other sandbox's parent. - let swap = |parent: &Path, to: &Path| { - fs::rename(parent, to).unwrap(); - link_dir(&second, parent); - }; - // The other sandbox's files, and its lease, stay. - let other_stays = || { - for name in kept { - assert!(other.join(name).exists(), "{name}"); - } - assert_eq!(binding(&other), Some(other_lease.clone())); - }; - // Once the release has checked the sandbox, its path comes to name the other sandbox, from - // the removal of its files on. - let moved = root.join("moved"); - let mut locked = SandboxStore::lock_existing(&leased_sandbox, &files(), None).unwrap(); - assert_eq!( - store - .release_with( - &mut locked, - &|| swap(&first, &moved), - &platform::Directory::sync, - &platform::Directory::sync - ) - .unwrap(), - Some(lease) - ); - drop(locked); - // The release removed the files and the binding of the sandbox whose lock it held. - let released = moved.join("sandbox"); - for name in kept { - assert!(fs::symlink_metadata(released.join(name)).is_err(), "{name}"); - } - assert_eq!(binding(&released), None); - other_stays(); - assert_eq!( - store.leases(generation).unwrap(), - std::slice::from_ref(&other_lease) - ); - // A release whose path names the other sandbox once the lock is taken, before the release - // begins, refuses, and removes nothing. - let again = leased(&store, &published, &released); - for name in kept { - fs::write(released.join(name), b"").unwrap(); - } - let mut locked = SandboxStore::lock_existing(&released, &files(), None).unwrap(); - let away = root.join("away"); - swap(&moved, &away); - let error = store.release(&mut locked).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Other, "{error}"); - assert!( - error - .message() - .contains("does not hold the lock of its sandbox"), - "{error}" - ); - drop(locked); - let kept_sandbox = away.join("sandbox"); - for name in kept { - assert!(kept_sandbox.join(name).exists(), "{name}"); - } - assert_eq!(binding(&kept_sandbox), Some(again.clone())); - other_stays(); - let mut both = vec![again, other_lease]; - both.sort_by_key(Lease::id); - assert_eq!(store.leases(generation).unwrap(), both); -} - -/// The payload of a generation, which holds the control sockets of the VMMs of its clones on -/// Linux, opens through the store's directory, following no link, or junction on Windows, at the -/// store's directory, at its directory of the generations, or at the payload, once it has found -/// that the store's directory holds the store of the caller's identity. On Unix, a store's -/// directory and its directory of the generations need only the right to be searched, which an -/// owner-only directory may grant without the right to read it. -#[test] -fn the_payload_of_a_generation_opens_through_no_link() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let id = store.id(); - let payload = store.payload_dir(generation); - let snapshot = platform::file_id(&payload.join(SNAPSHOT_NAME)).unwrap(); - let opened = open_payload(store.dir(), id, generation).unwrap().unwrap(); - assert_eq!(opened.entry_id(SNAPSHOT_NAME).unwrap(), Some(snapshot)); - drop(opened); - // A generation without a payload has none, nor has a store that is gone, or one of another - // identity. - assert!( - open_payload(store.dir(), id, store_generation(9)) - .unwrap() - .is_none() - ); - let missing = directory.path().join("missing"); - assert!(open_payload(&missing, id, generation).unwrap().is_none()); - let foreign = StoreId::from_bytes([0x5a; 16]); - assert!( - open_payload(store.dir(), foreign, generation) - .unwrap() - .is_none() - ); - // Another store in the store's place, whose generation of the same number has a payload, has - // none of the store's, nor has a directory that holds no store. - let other = Store::create(directory.path().join("other")).unwrap(); - fs::create_dir(other.payload_dir(generation)).unwrap(); - let moved = directory.path().join("moved"); - fs::rename(store.dir(), &moved).unwrap(); - fs::rename(other.dir(), store.dir()).unwrap(); - assert!(open_payload(store.dir(), id, generation).unwrap().is_none()); - assert!( - open_payload(store.dir(), other.id(), generation) - .unwrap() - .is_some() - ); - fs::rename(store.dir(), other.dir()).unwrap(); - fs::create_dir(store.dir()).unwrap(); - assert!(open_payload(store.dir(), id, generation).unwrap().is_none()); - fs::remove_dir(store.dir()).unwrap(); - // A link in the place of the store's directory, even to the store, of the directory of the - // generations, or of the payload, and a file in the place of the store's directory or of the - // payload, are refused. - let generations = store.dir().join(GENERATIONS_NAME); - let refused = |entry: &dyn std::fmt::Display| { - let error = open_payload(store.dir(), id, generation).err().unwrap(); - assert_eq!(error.kind(), ErrorKind::Io, "{error}"); - assert_eq!( - error.message(), - format!( - "cannot open the payload of the generation {}", - payload.display() - ) - ); - assert_eq!( - cause(&error), - format!("{entry} is a link, or not a directory") - ); - }; - link_dir(&moved, store.dir()); - refused(&store.dir().display()); - unlink_dir(store.dir()); - fs::write(store.dir(), b"").unwrap(); - refused(&store.dir().display()); - fs::remove_file(store.dir()).unwrap(); - fs::rename(&moved, store.dir()).unwrap(); - for (path, name, link) in [ - (&payload, generation.to_string(), true), - (&generations, GENERATIONS_NAME.to_owned(), true), - (&payload, generation.to_string(), false), - ] { - fs::rename(path, &moved).unwrap(); - if link { - link_dir(&moved, path); - } else { - fs::write(path, b"").unwrap(); - } - refused(&name); - if link { - unlink_dir(path); - } else { - fs::remove_file(path).unwrap(); - } - fs::rename(&moved, path).unwrap(); - } - assert!(open_payload(store.dir(), id, generation).unwrap().is_some()); - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - - let mode = |dir: &Path, mode| { - fs::set_permissions(dir, fs::Permissions::from_mode(mode)).unwrap(); - }; - for dir in [&generations, store.dir()] { - mode(dir, 0o300); - } - let opened = open_payload(store.dir(), id, generation); - for dir in [&generations, store.dir()] { - mode(dir, 0o700); - } - let opened = opened.unwrap().unwrap(); - assert_eq!(opened.entry_id(SNAPSHOT_NAME).unwrap(), Some(snapshot)); - opened.sync().unwrap(); - } -} - -/// A release whose generation's payload, or store's directory, a link takes the place of: before -/// the release begins, it refuses, and removes nothing; once it has opened the payload, it removes -/// the socket from the payload that it opened, and the link's target keeps its file. Only on Unix, -/// where the payload holds the socket of the clone's VMM. -#[cfg(unix)] -#[test] -fn a_release_removes_the_socket_from_the_payload_that_it_opened_whatever_its_path_names_meanwhile() -{ - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - let payload = store.payload_dir(generation); - let name = clone_socket_name(lease.launch()); - // A socket that the VMM left in the payload, which a file stands for, and a directory that - // holds a file of its name, to which a link takes the payload's place. - fs::write(payload.join(&name), b"").unwrap(); - let decoy = directory.path().join("decoy"); - fs::create_dir(&decoy).unwrap(); - fs::write(decoy.join(&name), b"").unwrap(); - // A directory of the store's layout, whose generation of the same number holds a file of the - // socket's name, to which a link takes the store's place. - let decoy_store = directory.path().join("decoy-store"); - let decoy_socket = decoy_store - .join(GENERATIONS_NAME) - .join(generation.to_string()) - .join(&name); - fs::create_dir_all(decoy_socket.parent().unwrap()).unwrap(); - fs::write(&decoy_socket, b"").unwrap(); - let moved = directory.path().join("moved"); - let swap = || { - fs::rename(&payload, &moved).unwrap(); - link_dir(&decoy, &payload); - }; - let restore = || { - unlink_dir(&payload); - fs::rename(&moved, &payload).unwrap(); - }; - let mut locked = SandboxStore::lock_existing(&sandbox, &files(), None).unwrap(); - let refused = |locked: &mut LockedStore, entry: &dyn std::fmt::Display| { - let error = store.release(locked).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Io, "{error}"); - assert_eq!( - error.message(), - format!( - "cannot open the payload of the generation {}", - payload.display() - ) - ); - assert_eq!( - cause(&error), - format!("{entry} is a link, or not a directory") - ); - assert!(decoy.join(&name).exists()); - assert!(decoy_socket.exists()); - assert_eq!(binding(&sandbox), Some(lease.clone())); - }; - swap(); - refused(&mut locked, &generation); - assert!(moved.join(&name).exists()); - restore(); - fs::rename(store.dir(), &moved).unwrap(); - link_dir(&decoy_store, store.dir()); - refused(&mut locked, &store.dir().display()); - unlink_dir(store.dir()); - fs::rename(&moved, store.dir()).unwrap(); - assert!(payload.join(&name).exists()); - assert_eq!( - store - .release_with( - &mut locked, - &swap, - &platform::Directory::sync, - &platform::Directory::sync - ) - .unwrap(), - Some(lease) - ); - drop(locked); - assert!(fs::symlink_metadata(moved.join(&name)).is_err()); - assert!(decoy.join(&name).exists()); - assert!(decoy_socket.exists()); - assert_eq!(binding(&sandbox), None); - restore(); - assert!(store.leases(generation).unwrap().is_empty()); -} - -/// A sandbox's store that clears the runtime record of its clone's VM removes the VMM's control -/// socket through the payload of its generation, as a recovery of its launch does: it refuses a -/// link in the place of the payload or of the template store's directory, keeping the record, and -/// leaves the files of another store in the template store's place. Only on Unix, where the socket -/// is a file. -#[cfg(unix)] -#[test] -fn a_sandbox_s_store_removes_the_socket_of_its_clone_through_no_link() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - let payload = store.payload_dir(generation); - let socket = store.clone_socket_path(&lease); - let name = clone_socket_name(lease.launch()); - fs::write(&socket, b"").unwrap(); - let decoy = directory.path().join("decoy"); - fs::create_dir(&decoy).unwrap(); - fs::write(decoy.join(&name), b"").unwrap(); - // Another store, whose generation of the same number holds a file of the socket's name. - let other = Store::create(directory.path().join("other")).unwrap(); - fs::create_dir(other.payload_dir(generation)).unwrap(); - let other_socket = other.payload_dir(generation).join(&name); - fs::write(&other_socket, b"").unwrap(); - // The runtime record of the clone's VM, whose control endpoint is the socket. - let record = json!({ - "format": FORMAT, - "launch": lease.launch().to_string(), - "process": { "pid": 42, "startTime": 4242, "boot": "ab".repeat(16) }, - "endpoint": socket.to_str().unwrap(), - "runtime": serde_json::to_value(runtime()).unwrap(), - }); - let write_record = || { - write_private(&sandbox.join(RUNTIME_NAME), record.to_string().as_bytes()).unwrap(); - }; - write_record(); - let mut locked = SandboxStore::lock_existing(&sandbox, &files(), None).unwrap(); - let refused = |locked: &mut LockedStore, entry: &dyn std::fmt::Display| { - let error = locked.clear_runtime().unwrap_err(); - assert_eq!( - error.message(), - format!( - "cannot open the payload of the generation {}", - payload.display() - ) - ); - assert_eq!( - cause(&error), - format!("{entry} is a link, or not a directory") - ); - assert!(decoy.join(&name).exists()); - assert!(other_socket.exists()); - assert!(sandbox.join(RUNTIME_NAME).exists()); - }; - let moved = directory.path().join("moved"); - fs::rename(&payload, &moved).unwrap(); - link_dir(&decoy, &payload); - refused(&mut locked, &generation); - assert!(moved.join(&name).exists()); - unlink_dir(&payload); - fs::rename(&moved, &payload).unwrap(); - fs::rename(store.dir(), &moved).unwrap(); - link_dir(other.dir(), store.dir()); - refused(&mut locked, &store.dir().display()); - unlink_dir(store.dir()); - // Another store in the store's place holds no socket of the clone's, so the record goes, and - // the other store's file stays, as does the socket in the store's own directory. - fs::rename(other.dir(), store.dir()).unwrap(); - locked.clear_runtime().unwrap(); - assert!(!sandbox.join(RUNTIME_NAME).exists()); - fs::rename(store.dir(), other.dir()).unwrap(); - fs::rename(&moved, store.dir()).unwrap(); - assert!(other_socket.exists()); - assert!(socket.exists()); - write_record(); - locked.clear_runtime().unwrap(); - assert!(fs::symlink_metadata(&socket).is_err()); - assert!(!sandbox.join(RUNTIME_NAME).exists()); - assert!(decoy.join(&name).exists()); - assert!(other_socket.exists()); - drop(locked); - assert_eq!(release(&store, &sandbox).unwrap(), Some(lease)); -} - -#[test] -fn a_retirement_that_finds_its_generation_unindexed_makes_the_index_durable_first() { - let (_directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - // A retirement that a crash interrupted once it had written the index, which it may not have - // made durable, leaves the tombstone. - store.tombstone(generation, None).unwrap(); - fs::remove_dir_all(store.payload_dir(generation)).unwrap(); - remove(&store, generation); - let tombstone = store - .dir() - .join(TOMBSTONES_NAME) - .join(format!("{generation}.json")); - let synced = RefCell::new(Vec::new()); - let sync = |dir: &Path| -> Result<()> { - synced - .borrow_mut() - .push((dir.to_owned(), tombstone.exists())); - Ok(()) - }; - assert_eq!( - store.retire_with(generation, None, &sync).unwrap(), - Retirement::Retired - ); - assert_eq!(*synced.borrow(), [(store.dir().to_owned(), true)]); - assert!(!tombstone.exists()); -} - -#[test] -fn a_retirement_keeps_a_tombstone_that_is_not_the_stores() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - // A retirement that a crash interrupted once it had written the index leaves the tombstone, - // which retiring the generation again removes only once it has checked it. - store.tombstone(generation, None).unwrap(); - fs::remove_dir_all(store.payload_dir(generation)).unwrap(); - remove(&store, generation); - let tombstone = store - .dir() - .join(TOMBSTONES_NAME) - .join(format!("{generation}.json")); - let valid = json_of(&tombstone); - let foreign: [(Change, ErrorCode); 2] = [ - ( - |value| value["store"] = json!("0".repeat(32)), - ErrorCode::Io, - ), - ( - |value| value["format"] = json!(2), - ErrorCode::FormatUnsupported, - ), - ]; - for (change, code) in foreign { - let mut value = valid.clone(); - change(&mut value); - fs::write(&tombstone, value.to_string()).unwrap(); - let error = store.retire(generation, None).unwrap_err(); - assert_eq!(error.code(), code, "{error}"); - assert_eq!(json_of(&tombstone), value); - } - // Nor does it remove a link in place of the tombstone. - fs::write(&tombstone, valid.to_string()).unwrap(); - let moved = directory.path().join("tombstone.json"); - fs::rename(&tombstone, &moved).unwrap(); - if link_file(&moved, &tombstone) { - let error = store.retire(generation, None).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Io, "{error}"); - assert!(fs::symlink_metadata(&tombstone).unwrap().is_symlink()); - fs::remove_file(&tombstone).unwrap(); - } - fs::rename(&moved, &tombstone).unwrap(); - assert_eq!(store.retire(generation, None).unwrap(), Retirement::Retired); - assert!(!tombstone.exists()); -} - -#[test] -fn directories_are_synced_where_the_platform_can() { - let directory = tempfile::tempdir().unwrap(); - sync_dir(directory.path()).unwrap(); - // On Unix, a sync opens the directory, which must exist. - #[cfg(unix)] - { - let error = sync_dir(&directory.path().join("missing")).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Io, "{error}"); - } -} - -#[test] -fn records_keep_their_json_form() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - let expected = json!({ - "format": 1, - "store": store.id().to_string(), - "lease": lease.id().to_string(), - "generation": generation.get().get(), - "key": published.key().to_string(), - "sandbox": sandbox.to_str().unwrap(), - "launch": lease.launch().as_str(), - }); - let record = store - .dir() - .join(LEASES_NAME) - .join(format!("{}.json", lease.id())); - assert_eq!(json_of(&record), expected); - assert_eq!(json_of(&sandbox.join(CLONE_BINDING_NAME)), expected); - store.tombstone(generation, None).unwrap(); - assert_eq!( - json_of( - &store - .dir() - .join(TOMBSTONES_NAME) - .join(format!("{generation}.json")) - ), - json!({ - "format": 1, - "store": store.id().to_string(), - "generation": generation.get().get(), - }) - ); - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - - for name in [LEASES_NAME, TOMBSTONES_NAME] { - let mode = fs::metadata(store.dir().join(name)) - .unwrap() - .permissions() - .mode(); - assert_eq!(mode & 0o777, 0o700, "{name}"); - } - } - // A lease's identity is its text form. - let text = lease.id().to_string(); - assert_eq!(text.len(), 32); - assert_eq!(text.parse::().unwrap(), lease.id()); - assert_eq!(text.to_uppercase().parse::().unwrap(), lease.id()); - assert_eq!(LeaseId::from_bytes(*lease.id().as_bytes()), lease.id()); - for invalid in ["", "0123", &"g".repeat(32)] { - let error = invalid.parse::().unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Unsupported, "{invalid}"); - } -} - -#[test] -fn malformed_records_and_foreign_entries_fail_clearly() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - let record = store - .dir() - .join(LEASES_NAME) - .join(format!("{}.json", lease.id())); - let bound = sandbox.join(CLONE_BINDING_NAME); - let valid = json_of(&record); - let rewrite = |path: &Path, change: Change| { - let mut value = valid.clone(); - change(&mut value); - fs::write(path, value.to_string()).unwrap(); - }; - // A record of another format is neither read nor migrated. - rewrite(&record, |value| value["format"] = json!(2)); - let error = store.leases(generation).unwrap_err(); - assert_eq!(error.code(), ErrorCode::FormatUnsupported, "{error}"); - // Records that are not leases. - let malformed: [(Change, &str); 7] = [ - ( - |value| value["generation"] = json!(0), - "it leases store generation 0", - ), - ( - |value| value["sandbox"] = json!("relative"), - "is not an absolute path", - ), - (|value| value["key"] = json!("00"), "its key is invalid"), - ( - |value| value["lease"] = json!("00"), - "its identity is invalid", - ), - ( - |value| value["launch"] = json!("00"), - "its launch is invalid", - ), - // A lease reserves its clone's launch. - ( - |value| { - value.as_object_mut().unwrap().remove("launch"); - }, - "is malformed", - ), - (|value| value["extra"] = json!(1), "is malformed"), - ]; - for (change, problem) in malformed { - rewrite(&bound, change); - let error = Lease::of(&SandboxStore::open(&sandbox).unwrap()).unwrap_err(); - assert_eq!(error.code(), ErrorCode::Io, "{error}"); - assert!(error.message().contains("is malformed"), "{error}"); - assert!(error.message().contains(problem), "{error}"); - } - // A key that is not a key's identity fails as its parse does. - rewrite(&bound, |value| value["key"] = json!("00")); - let error = Lease::of(&SandboxStore::open(&sandbox).unwrap()).unwrap_err(); - let parse = "00".parse::().unwrap_err(); - assert!( - error - .message() - .ends_with(&format!("its key is invalid: {}", parse.message())), - "{error}" - ); - fs::write(&bound, valid.to_string()).unwrap(); - // A record of another lease than its name names, or of another store. - let other_lease: Change = |value| value["lease"] = json!("0".repeat(32)); - let other_store: Change = |value| value["store"] = json!("0".repeat(32)); - for change in [other_lease, other_store] { - rewrite(&record, change); - let error = store.leases(generation).unwrap_err(); - assert_eq!(error.code(), ErrorCode::Io, "{error}"); - assert!(error.message().contains("is malformed"), "{error}"); - } - fs::write(&record, valid.to_string()).unwrap(); - assert_eq!(store.leases(generation).unwrap(), [lease]); - // Entries that are not the store's, which a retirement refuses before it changes anything. - let notes = store.dir().join(LEASES_NAME).join("notes.txt"); - fs::write(¬es, b"").unwrap(); - for error in [ - store.leases(generation).unwrap_err(), - store.retire(generation, None).unwrap_err(), - ] { - assert_eq!(error.code(), ErrorCode::ForeignFiles, "{error}"); - assert!( - error - .message() - .ends_with("holds entries that are not the store's: notes.txt"), - "{error}" - ); - } - assert!(store.tombstones().unwrap().is_empty()); - fs::remove_file(¬es).unwrap(); - // A tombstone of another store, or of another format, which neither an acquisition nor a - // listing or a selection takes for a quarantine, and entries that are not tombstones. - store.tombstone(generation, None).unwrap(); - let tombstones = store.dir().join(TOMBSTONES_NAME); - let tombstone = tombstones.join(format!("{generation}.json")); - let valid_tombstone = json_of(&tombstone); - let other = provision(directory.path(), "other"); - let foreign_tombstones: [(Change, ErrorCode); 2] = [ - ( - |value| value["store"] = json!("0".repeat(32)), - ErrorCode::Io, - ), - ( - |value| value["format"] = json!(2), - ErrorCode::FormatUnsupported, - ), - ]; - for (change, code) in foreign_tombstones { - let mut value = valid_tombstone.clone(); - change(&mut value); - fs::write(&tombstone, value.to_string()).unwrap(); - for error in [ - refusal(&store, generation, &other, &after(&published, 1)), - store.tombstones().unwrap_err(), - store - .select(published.key(), &after(&published, 1)) - .unwrap_err(), - ] { - assert_eq!(error.code(), code, "{error}"); - } - } - fs::write(tombstones.join("+1.json"), b"").unwrap(); - for error in [ - store.tombstones().unwrap_err(), - store - .select(published.key(), &after(&published, 1)) - .unwrap_err(), - ] { - assert_eq!(error.code(), ErrorCode::ForeignFiles, "{error}"); - assert!(error.message().ends_with(": +1.json"), "{error}"); - } -} - -#[test] -fn the_directories_of_leases_and_tombstones_refuse_links() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - store.tombstone(generation, None).unwrap(); - for name in [LEASES_NAME, TOMBSTONES_NAME] { - let path = store.dir().join(name); - let moved = path.with_extension("moved"); - fs::rename(&path, &moved).unwrap(); - link_dir(&moved, &path); - let mut errors = vec![store.retire(generation, None).unwrap_err()]; - if name == LEASES_NAME { - errors.push(store.leases(generation).unwrap_err()); - errors.push(release(&store, &sandbox).unwrap_err()); - } else { - errors.push(store.tombstones().unwrap_err()); - errors.push( - store - .select(published.key(), &after(&published, 1)) - .unwrap_err(), - ); - } - for error in errors { - assert!( - error.message().ends_with("is not a plain directory"), - "{name}: {error}" - ); - } - unlink_dir(&path); - fs::rename(&moved, &path).unwrap(); - } - // Nothing changed meanwhile. - assert_eq!(binding(&sandbox), Some(lease.clone())); - assert_eq!(store.leases(generation).unwrap(), [lease]); - assert_eq!(store.tombstones().unwrap(), [generation]); -} - -/// A transaction in a process of its own, which [`transaction_role`] runs. -struct Transaction { - /// The process. - child: Child, - /// Its standard input, a line on which starts the transaction. - input: ChildStdin, - /// Its standard output. - output: BufReader, -} - -impl Transaction { - /// Starts the transaction `action` of the store generation `generation` of the store in - /// `store`, for the sandbox in `sandbox`, if any, and returns once it is ready to run. - fn start( - store: &Path, - action: &str, - generation: StoreGeneration, - sandbox: Option<&Path>, - ) -> Self { - let mut command = Command::new(env::current_exe().unwrap()); - command - .args(role("transaction_role")) - .env(ROLE_STORE, store) - .env(ROLE_ACTION, action) - .env(ROLE_GENERATION, generation.to_string()) - .stdin(Stdio::piped()) - .stdout(Stdio::piped()); - if let Some(sandbox) = sandbox { - command.env(ROLE_SANDBOX, sandbox); - } - let mut child = supervisor::spawn_command(&mut command).unwrap(); - let input = child.stdin.take().unwrap(); - let mut output = BufReader::new(child.stdout.take().unwrap()); - let mut line = String::new(); - while line.trim_end() != READY { - line.clear(); - let read = output.read_line(&mut line).unwrap(); - assert_ne!(read, 0, "the transaction ended before it was ready"); - } - Self { - child, - input, - output, - } - } - - /// Lets the transaction run. - fn run(&mut self) { - self.input.write_all(b"run\n").unwrap(); - self.input.flush().unwrap(); - } - - /// Waits for the transaction to end, and returns how it ended, as [`transaction_role`] - /// prints it. - fn outcome(mut self) -> String { - let mut rest = String::new(); - self.output.read_to_string(&mut rest).unwrap(); - assert!(self.child.wait().unwrap().success(), "{rest}"); - rest.lines() - .find_map(|line| line.strip_prefix(OUTCOME)) - .expect("the transaction printed its outcome") - .to_owned() - } -} - -impl Drop for Transaction { - fn drop(&mut self) { - let _ = self.child.kill(); - let _ = self.child.wait(); - } -} - -/// Runs `transactions` at once, with `other` amid them, and returns how each of `transactions` -/// ended, in their order, and how `other` ended. -fn race(mut transactions: Vec, mut other: Transaction) -> (Vec, String) { - let middle = transactions.len() / 2; - for (index, transaction) in transactions.iter_mut().enumerate() { - if index == middle { - other.run(); - } - transaction.run(); - } - let outcomes = transactions.into_iter().map(Transaction::outcome).collect(); - (outcomes, other.outcome()) -} - -/// Returns the identities of the leases that `outcomes` report as acquired, sorted, and separated -/// by spaces. -fn acquired(outcomes: &[String]) -> String { - let mut ids: Vec<&str> = outcomes - .iter() - .filter_map(|outcome| outcome.strip_prefix("acquired ")) - .collect(); - ids.sort_unstable(); - ids.join(" ") -} - -/// A transaction in a process of its own, which [`Transaction::start`] runs: once a line arrives on -/// its standard input, it runs the transaction that [`ROLE_ACTION`] names, of the store generation -/// [`ROLE_GENERATION`] of the store in [`ROLE_STORE`], for the sandbox in [`ROLE_SANDBOX`], and -/// prints how it ended: `acquired ID`, `released ID` or `none`, `tombstoned`, `retired` or -/// `draining ID...`, or the code of its error. -#[test] -#[ignore = "a transaction that the lease tests run in a process of its own"] -fn transaction_role() { - let (Some(dir), Ok(action), Ok(generation)) = ( - env::var_os(ROLE_STORE), - env::var(ROLE_ACTION), - env::var(ROLE_GENERATION), - ) else { - return; - }; - let store = Store::open(dir).unwrap(); - let generation = StoreGeneration::new(generation.parse().unwrap()); - let sandbox = env::var_os(ROLE_SANDBOX).map(PathBuf::from); - // A second after the capture of the generation, on the host that captured it. - let now = store - .index() - .unwrap() - .get(generation) - .map(|indexed| after(indexed, 1)); - // The test harness has printed the test's name without ending the line. - println!("\n{READY}"); - let mut line = String::new(); - std::io::stdin().read_line(&mut line).unwrap(); - let deadline = Some(Instant::now() + Duration::from_secs(30)); - let code = |error: Error| error.code().to_string(); - let outcome = match action.as_str() { - "acquire" => store - .acquire( - generation, - sandbox.unwrap(), - &files(), - &now.unwrap(), - deadline, - ) - .map_or_else(code, |acquired| format!("acquired {}", acquired.lease.id())), - "release" => { - let mut locked = - SandboxStore::lock_existing(sandbox.unwrap(), &files(), deadline).unwrap(); - match store.release(&mut locked) { - Ok(Some(lease)) => format!("released {}", lease.id()), - Ok(None) => "none".to_owned(), - Err(error) => code(error), - } - } - "tombstone" => store - .tombstone(generation, deadline) - .map_or_else(code, |()| "tombstoned".to_owned()), - "retire" => match store.retire(generation, deadline) { - Ok(Retirement::Retired) => "retired".to_owned(), - Ok(Retirement::Draining(leases)) => format!("draining {}", ids(&leases)), - Err(error) => code(error), - }, - _ => panic!("unknown transaction {action}"), - }; - println!("{OUTCOME}{outcome}"); -} - -#[test] -fn processes_that_race_to_lease_one_sandbox_lease_it_once() { - /// The processes that race. - const PROCESSES: usize = 6; - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let mut acquisitions: Vec = (0..PROCESSES) - .map(|_| Transaction::start(store.dir(), "acquire", generation, Some(&sandbox))) - .collect(); - for acquisition in &mut acquisitions { - acquisition.run(); - } - let outcomes: Vec = acquisitions.into_iter().map(Transaction::outcome).collect(); - let leases = store.leases(generation).unwrap(); - assert_eq!(leases.len(), 1, "{outcomes:?}"); - assert_eq!(acquired(&outcomes), leases[0].id().to_string()); - assert_eq!( - outcomes - .iter() - .filter(|outcome| *outcome == "sandbox-leased") - .count(), - PROCESSES - 1, - "{outcomes:?}" - ); - assert_eq!(binding(&sandbox), Some(leases[0].clone())); -} - -#[test] -fn a_tombstone_that_races_acquisitions_in_other_processes_keeps_the_leases_before_it() { - /// The clones whose acquisitions race the tombstone. - const CLONES: usize = 6; - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandboxes: Vec = (0..CLONES) - .map(|clone| provision(directory.path(), &format!("clone-{clone}"))) - .collect(); - let acquisitions = sandboxes - .iter() - .map(|sandbox| Transaction::start(store.dir(), "acquire", generation, Some(sandbox))) - .collect(); - let tombstone = Transaction::start(store.dir(), "tombstone", generation, None); - let (outcomes, tombstoned) = race(acquisitions, tombstone); - assert_eq!(tombstoned, "tombstoned"); - // Each acquisition leased the generation before the tombstone, or found it quarantined after. - for (sandbox, outcome) in sandboxes.iter().zip(&outcomes) { - match outcome.strip_prefix("acquired ") { - Some(id) => assert_eq!(binding(sandbox).unwrap().id().to_string(), id), - None => { - assert_eq!(outcome, "template-tombstoned"); - assert_eq!(binding(sandbox), None); - let error = refusal(&store, generation, sandbox, &after(&published, 1)); - assert_eq!(error.code(), ErrorCode::TemplateTombstoned, "{error}"); - } - } - } - // The leases granted before it keep working. - assert_eq!(ids(&store.leases(generation).unwrap()), acquired(&outcomes)); - assert_eq!(store.tombstones().unwrap(), [generation]); -} - -#[test] -fn a_retirement_that_races_acquisitions_and_releases_in_other_processes_waits_for_its_leases() { - /// The clones whose acquisitions and releases race the retirements. - const CLONES: usize = 6; - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandboxes: Vec = (0..CLONES) - .map(|clone| provision(directory.path(), &format!("clone-{clone}"))) - .collect(); - let start = |action: &str, sandbox: Option<&Path>| { - Transaction::start(store.dir(), action, generation, sandbox) - }; - let acquisitions = sandboxes - .iter() - .map(|sandbox| start("acquire", Some(sandbox))) - .collect(); - let (outcomes, retired) = race(acquisitions, start("retire", None)); - // The retirement drains the leases of the acquisitions before it, and those after it find - // the generation quarantined, or gone if none leased it before. - let granted = acquired(&outcomes); - if granted.is_empty() { - assert_eq!(retired, "retired"); - assert!( - outcomes.iter().all(|outcome| outcome == "not-found"), - "{outcomes:?}" - ); - } else { - assert_eq!(retired, format!("draining {granted}")); - assert!( - outcomes.iter().all(|outcome| outcome.starts_with("acquired ") - || outcome == "template-tombstoned"), - "{outcomes:?}" - ); - assert!(store.snapshot_dir(&published).is_ok()); - } - assert_eq!(ids(&store.leases(generation).unwrap()), granted); - // The clones release their leases while another retirement runs, which deletes the - // generation only once none remains. - let releases = sandboxes - .iter() - .map(|sandbox| start("release", Some(sandbox))) - .collect(); - let (released, retired) = race(releases, start("retire", None)); - for (outcome, acquisition) in released.iter().zip(&outcomes) { - match acquisition.strip_prefix("acquired ") { - Some(id) => assert_eq!(outcome, &format!("released {id}")), - None => assert_eq!(outcome, "none"), - } - } - assert!( - retired == "retired" || retired.starts_with("draining "), - "{retired}" - ); - assert_eq!(store.retire(generation, None).unwrap(), Retirement::Retired); - assert!(!store.payload_dir(generation).exists()); - assert_eq!(store.index().unwrap().get(generation), None); - assert!(store.leases(generation).unwrap().is_empty()); - assert!(store.tombstones().unwrap().is_empty()); -} - -/// A clone's orchestrator in a process of its own, which [`run_clone_role`] runs: it leases the -/// store generation [`ROLE_GENERATION`] of the store in [`ROLE_STORE`] to the sandbox in -/// [`ROLE_SANDBOX`], and launches the clone with a [`vmm_role`] in [`ROLE_VMM`] as its VMM, or it -/// releases the sandbox's lease; and it exits without unwinding after the step that [`ROLE_CRASH`] -/// names, as a crash would, once it has printed the reference of the VMM that it spawned, if any. -#[test] -#[ignore = "a clone's orchestrator that the lease tests run in a process of its own"] -fn clone_role() { - let (Some(dir), Ok(generation), Some(sandbox), Ok(step), Some(vmm)) = ( - env::var_os(ROLE_STORE), - env::var(ROLE_GENERATION), - env::var_os(ROLE_SANDBOX).map(PathBuf::from), - env::var(ROLE_CRASH), - env::var_os(ROLE_VMM).map(PathBuf::from), - ) else { - return; - }; - let store = Store::open(dir).unwrap(); - let generation = StoreGeneration::new(generation.parse().unwrap()); - // The step's transaction crashes as it makes its write in `target` durable, or, for a - // release, the removal of the binding, in the sandbox's directory. - let target = match step.as_str() { - "after-record" => Some(store.dir().join(LEASES_NAME)), - "after-binding" => Some(sandbox.clone()), - _ => None, - }; - let sync = |dir: &Path| -> Result<()> { - if target.as_deref() == Some(dir) { - crash(&step); - } - sync_dir(dir) - }; - if step == "releasing" { - let mut locked = SandboxStore::lock_existing(&sandbox, &files(), None).unwrap(); - store - .release_with(&mut locked, &|| {}, &platform::Directory::sync, &|_| { - crash(&step) - }) - .unwrap(); - panic!("the release did not make the removal of its binding durable"); - } - let now = after(store.index().unwrap().get(generation).unwrap(), 1); - let mut acquired = store - .acquire_with(generation, sandbox.clone(), &files(), &now, None, &sync) - .unwrap(); - let reserved = acquired.lease.launch().clone(); - let command = |_: &str| -> Result> { - Ok(VmmCommand::new( - env::current_exe().unwrap(), - role("vmm_role"), - &vmm, - )) - }; - match step.as_str() { - "acquired" => crash(&step), - "before-spawn" => { - let _ = supervisor::launch( - &mut acquired.locked, - &reserved, - &runtime(), - &CAPABILITY, - |_| crash(&step), - ); - } - "after-spawn" => { - // The steps of `supervisor::launch`, up to the spawn. - let mut launch = acquired - .locked - .begin_launch(&reserved, &runtime(), &CAPABILITY) - .unwrap(); - let command = command(launch.endpoint()).unwrap(); - let log = launch.take_log().unwrap(); - let launched = supervisor::spawn(&command, &CAPABILITY, log).unwrap(); - print_reference(launched.process().unwrap().reference()); - crash(&step); - } - "after-identity" => { - let starting = supervisor::launch( - &mut acquired.locked, - &reserved, - &runtime(), - &CAPABILITY, - command, - ) - .unwrap(); - print_reference(starting.process().reference()); - crash(&step); - } - "committed" => { - let starting = supervisor::launch( - &mut acquired.locked, - &reserved, - &runtime(), - &CAPABILITY, - command, - ) - .unwrap(); - let running = starting.commit().unwrap(); - print_reference(running.record().process()); - crash(&step); - } - _ => {} - } - panic!("the clone's orchestrator did not crash after {step}"); -} - -/// The VMM of a clone of [`clone_role`]: reads its capability as OpenVMM does, then runs until a -/// file named [`EXIT_MARKER`] appears in its working directory, which [`VMM_MARKER`] marks, or for -/// [`VMM_LIFETIME`] at most. -#[test] -#[ignore = "a VMM that the lease tests run in a process of its own"] -fn vmm_role() { - let directory = env::current_dir().unwrap(); - if !directory.join(VMM_MARKER).exists() { - return; - } - let mut capability = [0; CAPABILITY_LEN]; - std::io::stdin().read_exact(&mut capability).unwrap(); - assert_eq!(capability, CAPABILITY); - let started = Instant::now(); - while !directory.join(EXIT_MARKER).exists() && started.elapsed() < VMM_LIFETIME { - thread::sleep(Duration::from_millis(10)); - } -} - -/// Prints `reference` as JSON, on a line of its own, after [`REFERENCE`]. -fn print_reference(reference: &ProcessRef) { - println!("\n{REFERENCE}{}", serde_json::to_string(reference).unwrap()); -} - -/// Ends this process at once, once it has printed that it crashes after `step`, without unwinding -/// or running destructors, as a crash would. -fn crash(step: &str) -> ! { - println!("\n{CRASHED}{step}"); - std::io::stdout().flush().unwrap(); - std::process::exit(0) -} - -/// Runs [`clone_role`] for the generation `generation` of `store`, the sandbox in `sandbox`, and a -/// VMM in `vmm`, until it crashes after `step`, and returns the reference of the VMM that it -/// spawned, if it spawned one. -fn run_clone_role( - step: &str, - store: &Store, - generation: StoreGeneration, - sandbox: &Path, - vmm: &Path, -) -> Option { - let mut command = Command::new(env::current_exe().unwrap()); - command - .args(role("clone_role")) - .env(ROLE_STORE, store.dir()) - .env(ROLE_GENERATION, generation.to_string()) - .env(ROLE_SANDBOX, sandbox) - .env(ROLE_CRASH, step) - .env(ROLE_VMM, vmm) - .stdin(Stdio::null()) - .stdout(Stdio::piped()) - .stderr(Stdio::piped()); - let output = supervisor::spawn_command(&mut command) - .unwrap() - .wait_with_output() - .unwrap(); - let stdout = String::from_utf8_lossy(&output.stdout); - let crashed = format!("{CRASHED}{step}"); - assert!( - output.status.success() && stdout.lines().any(|line| line == crashed), - "{}: {stdout}{}", - output.status, - String::from_utf8_lossy(&output.stderr) - ); - stdout - .lines() - .find_map(|line| line.strip_prefix(REFERENCE)) - .map(|reference| serde_json::from_str(reference).unwrap()) -} - -/// The working directory of a [`vmm_role`], which it marks. Dropping it ends the VMM, and waits -/// for it to exit, so that a failed test leaves no process behind. -struct VmmDir { - /// The reference of the VMM, once the test knows it. - reference: Option, - /// The directory, which is removed after the VMM exits. - dir: tempfile::TempDir, -} - -impl VmmDir { - /// Creates the directory, and marks it. - fn new() -> Self { - let dir = tempfile::tempdir().unwrap(); - fs::write(dir.path().join(VMM_MARKER), b"").unwrap(); - Self { - reference: None, - dir, - } - } - - /// Returns the path of the directory. - fn path(&self) -> &Path { - self.dir.path() - } - - /// Asks the VMM to exit. - fn end(&self) { - fs::write(self.path().join(EXIT_MARKER), b"").unwrap(); - } -} - -impl Drop for VmmDir { - fn drop(&mut self) { - let _ = fs::write(self.path().join(EXIT_MARKER), b""); - if let Some(reference) = &self.reference - && let Ok(Opened::Running(process)) = Process::open(reference) - && !process - .wait(Instant::now() + TIMEOUT) - .is_ok_and(|exited| exited.is_some()) - { - let _ = process.kill(); - let _ = process.wait(Instant::now() + TIMEOUT); - } - } -} - -/// A clone whose orchestrator, [`clone_role`], crashed after a step of the clone's launch, in a -/// fresh temporary directory: its VMM's working directory, its template store and sandbox, and -/// the lease that the sandbox holds. -struct Crashed { - /// The working directory of the clone's VMM, which ends the VMM as it drops, before the rest. - vmm: VmmDir, - /// The template store. - store: Store, - /// The sandbox's directory. - sandbox: PathBuf, - /// The lease that the sandbox holds. - lease: Lease, - /// The temporary directory of the store and the sandbox. - _directory: tempfile::TempDir, -} - -impl Crashed { - /// Publishes a generation in a new store, and leases it to a new sandbox, whose clone - /// [`clone_role`] launches until it crashes after `step`. - fn after(step: &str) -> Self { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let sandbox = provision(directory.path(), "sandbox"); - let mut vmm = VmmDir::new(); - vmm.reference = run_clone_role(step, &store, published.generation(), &sandbox, vmm.path()); - let lease = binding(&sandbox).expect("the crash kept the clone's lease"); - Self { - vmm, - store, - sandbox, - lease, - _directory: directory, - } - } - - /// Takes the lock of the clone's sandbox, as its restarted orchestrator does. - fn lock(&self) -> LockedStore { - SandboxStore::lock_existing(&self.sandbox, &files(), Some(Instant::now() + TIMEOUT)) - .unwrap() - } - - /// Checks that the store refuses to release the lease of the sandbox of `locked`, whose VM - /// may still run, and that the sandbox keeps it. - fn refuses_release(&self, locked: &mut LockedStore) { - let error = self.store.release(locked).unwrap_err(); - assert_eq!(error.code(), ErrorCode::SandboxRunning, "{error}"); - assert_eq!(binding(&self.sandbox), Some(self.lease.clone())); - } - - /// Checks that the store releases the lease of the sandbox of `locked`, and that the lease is - /// gone. - fn releases(&self, locked: &mut LockedStore) { - assert_eq!( - self.store.release(locked).unwrap(), - Some(self.lease.clone()) - ); - assert_eq!(binding(&self.sandbox), None); - assert!( - self.store - .leases(self.lease.generation()) - .unwrap() - .is_empty() - ); - } -} - -/// Reattaches to the VM of the sandbox of `locked` until the result is determinate, as a caller -/// retries an indeterminate reattach, and returns it. -/// -/// On Windows, a scanner that briefly opens the log of a launch can keep its claim held for a -/// moment. -fn reattach_until_determinate(locked: &mut LockedStore) -> Reattached { - let deadline = Instant::now() + TIMEOUT; - loop { - match supervisor::reattach(locked, &runtime()).unwrap() { - Reattached::Indeterminate(reason) if Instant::now() < deadline => { - eprintln!("reattach is indeterminate: {reason}"); - thread::sleep(Duration::from_millis(100)); - } - reattached => return reattached, - } - } -} - -#[test] -fn a_crash_during_an_acquisition_leaves_no_lease_or_one_that_launches_nothing() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - // No VMM runs: the orchestrator crashes before any launch. - let vmm = directory.path(); - // After the record, before the binding: the sandbox holds no lease, and leases the generation - // again, while the record, which no binding names, keeps the generation. - let unbound = provision(directory.path(), "unbound"); - assert_eq!( - run_clone_role("after-record", &store, generation, &unbound, vmm), - None - ); - assert_eq!(binding(&unbound), None); - let [orphan]: [Lease; 1] = store.leases(generation).unwrap().try_into().unwrap(); - assert_eq!(orphan.sandbox(), unbound); - let mut leases = vec![leased(&store, &published, &unbound)]; - // After the binding, before or after the acquisition returned: the sandbox holds the lease, - // whose launch no later lock begins, and which a release ends. Another acquisition then - // reserves another launch. - for step in ["after-binding", "acquired"] { - let sandbox = provision(directory.path(), step); - assert_eq!( - run_clone_role(step, &store, generation, &sandbox, vmm), - None - ); - let lease = binding(&sandbox).unwrap(); - assert!(store.leases(generation).unwrap().contains(&lease), "{step}"); - let mut locked = SandboxStore::lock_existing(&sandbox, &files(), None).unwrap(); - let error = launch_refusal(&mut locked, lease.launch()); - assert_eq!(error.code(), ErrorCode::SandboxLeased, "{step}: {error}"); - assert_eq!(store.release(&mut locked).unwrap(), Some(lease.clone())); - drop(locked); - let again = leased(&store, &published, &sandbox); - assert_ne!(again.launch(), lease.launch()); - leases.push(again); - } - // A retirement removes the record that no binding names, and waits for the leases. - leases.sort_by_key(Lease::id); - assert_eq!( - store.retire(generation, None).unwrap(), - Retirement::Draining(leases.clone()) - ); - assert_eq!(store.leases(generation).unwrap(), leases); -} - -#[test] -fn a_clone_that_crashes_before_its_spawn_keeps_its_lease_until_its_launch_ends() { - let crashed = Crashed::after("before-spawn"); - assert_eq!(crashed.vmm.reference, None); - let mut locked = crashed.lock(); - // The journal names the launch that the lease reserved, which spawned nothing. - let journal = locked.journal().unwrap().unwrap(); - assert_eq!(journal.launch(), crashed.lease.launch()); - assert_eq!(journal.state(), LaunchState::Prepared); - crashed.refuses_release(&mut locked); - match reattach_until_determinate(&mut locked) { - Reattached::Exited { - record: None, - recovery: Recovery::Exited, - } => {} - other => panic!("unexpected reattach: {other:?}"), - } - crashed.releases(&mut locked); -} - -#[test] -fn a_clone_that_crashes_after_its_spawn_keeps_its_lease_until_its_unrecorded_vmm_exits() { - let crashed = Crashed::after("after-spawn"); - let reference = crashed.vmm.reference.clone().unwrap(); - let Opened::Running(process) = Process::open(&reference).unwrap() else { - panic!("the unrecorded VMM runs"); - }; - let mut locked = crashed.lock(); - assert_eq!( - locked.journal().unwrap().unwrap().state(), - LaunchState::Prepared - ); - // Reattach can neither prove that the VMM, which no record names, exited, nor verify it: it - // signals nothing, and the lease stays. - let Reattached::Indeterminate(_) = supervisor::reattach(&mut locked, &runtime()).unwrap() - else { - panic!("reattach ended a launch whose VMM runs unverified"); - }; - assert!(!process.has_exited().unwrap()); - crashed.refuses_release(&mut locked); - // Once the VMM exits, the claim that it released proves it. - crashed.vmm.end(); - assert!(process.wait(Instant::now() + TIMEOUT).unwrap().is_some()); - match reattach_until_determinate(&mut locked) { - Reattached::Exited { - record: None, - recovery: Recovery::Exited, - } => {} - other => panic!("unexpected reattach: {other:?}"), - } - crashed.releases(&mut locked); -} - -#[test] -fn a_clone_that_crashes_after_recording_its_vmm_keeps_its_lease_until_reattach_ends_it() { - let crashed = Crashed::after("after-identity"); - let reference = crashed.vmm.reference.clone().unwrap(); - let Opened::Running(process) = Process::open(&reference).unwrap() else { - panic!("the recorded VMM runs"); - }; - let mut locked = crashed.lock(); - let journal = locked.journal().unwrap().unwrap(); - assert_eq!(journal.launch(), crashed.lease.launch()); - assert_eq!(journal.state(), LaunchState::Spawned); - assert_eq!(journal.process(), Some(&reference)); - crashed.refuses_release(&mut locked); - // The launch never committed, so reattach ends its VMM through its verified reference. - match supervisor::reattach(&mut locked, &runtime()).unwrap() { - Reattached::Exited { - record: None, - recovery: Recovery::Terminated(terminated), - } => assert_eq!(terminated, reference), - other => panic!("unexpected reattach: {other:?}"), - } - assert!(process.wait(Instant::now() + TIMEOUT).unwrap().is_some()); - crashed.releases(&mut locked); -} - -#[test] -fn a_committed_clone_keeps_its_lease_through_a_crash_until_its_vm_stops() { - let crashed = Crashed::after("committed"); - let reference = crashed.vmm.reference.clone().unwrap(); - let mut locked = crashed.lock(); - crashed.refuses_release(&mut locked); - // The restarted orchestrator holds the clone's VM again, and its lease lasts while it runs. - let Reattached::Live(running) = supervisor::reattach(&mut locked, &runtime()).unwrap() else { - panic!("the committed clone was not reattached"); - }; - assert_eq!(running.record().launch(), crashed.lease.launch()); - assert_eq!(running.record().process(), &reference); - crashed.refuses_release(&mut locked); - let stopped = running - .stop( - &mut locked, - Instant::now() + TIMEOUT, - None, - |_running, _deadline| { - crashed.vmm.end(); - Ok::<(), Infallible>(()) - }, - ) - .unwrap(); - assert_eq!(stopped.outcome, StopOutcome::Graceful); - crashed.releases(&mut locked); -} - -#[test] -fn a_crash_during_a_release_leaves_a_record_that_a_retirement_removes() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - assert_eq!( - run_clone_role("releasing", &store, generation, &sandbox, directory.path()), - None - ); - // The binding went first: the sandbox holds no lease, while the record, which no binding - // names, keeps the generation until a retirement removes it. - assert_eq!(binding(&sandbox), None); - assert_eq!(store.leases(generation).unwrap(), [lease]); - assert_eq!(release(&store, &sandbox).unwrap(), None); - assert_eq!(store.retire(generation, None).unwrap(), Retirement::Retired); - assert!(store.leases(generation).unwrap().is_empty()); -} - -/// Tests of the records that a Windows file system without POSIX deletion keeps under their names -/// while other handles hold them, after a transaction in another process removed them, and of the -/// records whose access control lists refuse access, which are not removed ones. -#[cfg(windows)] -#[allow(unsafe_code)] -mod windows { - use std::os::windows::fs::OpenOptionsExt; - use std::os::windows::io::AsRawHandle; - - use windows_sys::Win32::Foundation::HANDLE; - use windows_sys::Win32::Storage::FileSystem::{ - DELETE, FILE_DISPOSITION_INFO, FILE_SHARE_DELETE, FILE_SHARE_READ, FILE_SHARE_WRITE, - FileDispositionInfo, SetFileInformationByHandle, - }; - - use super::*; - use crate::host::store::lease::BINDING_DESCRIPTION; - - /// Deletes the file `path` as a file system without POSIX deletion deletes a file that other - /// handles hold: its name stays, and opening it fails, until the last handle closes, of which - /// the returned one is the last unless others hold the file. - fn delete_pending(path: &Path) -> fs::File { - let file = fs::OpenOptions::new() - .access_mode(DELETE) - .share_mode(FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE) - .open(path) - .unwrap(); - let disposition = FILE_DISPOSITION_INFO { DeleteFile: true }; - // SAFETY: the handle is open, and the information is a FILE_DISPOSITION_INFO of its size. - let set = unsafe { - SetFileInformationByHandle( - file.as_raw_handle() as HANDLE, - FileDispositionInfo, - (&raw const disposition).cast(), - size_of::() as u32, - ) - }; - assert_ne!(set, 0, "{}", io::Error::last_os_error()); - file - } - - /// The access control list of a file that allows no one anything, which its owner may still - /// change: every open of the file fails, also one without access rights, unless the process's - /// privileges bypass the list. - const NO_ACCESS: &str = "D:P"; - - /// The access control list of a file that allows its owner alone, as the store's files do. - const OWNER_ONLY: &str = "D:P(A;;FA;;;OW)"; - - #[test] - fn only_a_file_that_is_gone_or_whose_deletion_is_pending_was_removed() { - let directory = tempfile::tempdir().unwrap(); - let path = directory.path().join("record.json"); - fs::write(&path, b"{}").unwrap(); - let denied = io::Error::from(io::ErrorKind::PermissionDenied); - assert!(!was_removed(&path, &denied)); - // A file whose list refuses every open, also one without access rights, is not removed. - // Nor is it where the process's privileges bypass the list, as the backup privilege lets - // the open of `file_identity`, which has backup semantics, through. - platform::tests::set_access(&path, NO_ACCESS).unwrap(); - if let Err(error) = platform::file_identity(&path) { - assert_eq!(error.kind(), io::ErrorKind::PermissionDenied, "{error}"); - } - assert!(!was_removed(&path, &denied)); - platform::tests::set_access(&path, OWNER_ONLY).unwrap(); - // One whose deletion is pending is, as is one that is gone, but only a denied access tells - // so. - let held = delete_pending(&path); - assert!(was_removed(&path, &denied)); - assert!(!was_removed(&path, &io::Error::other("the open failed"))); - drop(held); - assert!(was_removed(&path, &denied)); - // So is a file whose directory is gone. - let gone = directory.path().join("gone").join("record.json"); - assert!(was_removed(&gone, &denied)); - } - - #[test] - fn records_that_refuse_access_are_not_taken_for_removed_ones() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - let record = store - .dir() - .join(LEASES_NAME) - .join(format!("{}.json", lease.id())); - // A retirement that cannot read the record of a lease, or its clone binding, cannot tell - // whether the lease is live, and fails, keeping the generation. - for (path, description) in [ - (record, LEASE_DESCRIPTION), - (sandbox.join(CLONE_BINDING_NAME), BINDING_DESCRIPTION), - ] { - platform::tests::set_access(&path, NO_ACCESS).unwrap(); - let error = store.retire(generation, None).unwrap_err(); - assert!( - error - .message() - .starts_with(&format!("cannot read the {description} ")), - "{error}" - ); - platform::tests::set_access(&path, OWNER_ONLY).unwrap(); - } - assert_eq!( - store.retire(generation, None).unwrap(), - Retirement::Draining(vec![lease.clone()]) - ); - assert_eq!(binding(&sandbox), Some(lease)); - // A tombstone that refuses access quarantines its generation still. - let other = publish(&store, &ManifestSpec::template(), storage(2)); - store.tombstone(other.generation(), None).unwrap(); - let tombstone = store - .dir() - .join(TOMBSTONES_NAME) - .join(format!("{}.json", other.generation())); - platform::tests::set_access(&tombstone, NO_ACCESS).unwrap(); - let now = after(&other, 1); - let leasing = provision(directory.path(), "leasing"); - for error in [ - store.tombstones().unwrap_err(), - store.select(other.key(), &now).unwrap_err(), - refusal(&store, other.generation(), &leasing, &now), - ] { - assert!( - error - .message() - .starts_with(&format!("cannot read the {TOMBSTONE_DESCRIPTION} ")), - "{error}" - ); - } - assert_eq!(binding(&leasing), None); - platform::tests::set_access(&tombstone, OWNER_ONLY).unwrap(); - assert_eq!( - store.tombstones().unwrap(), - [generation, other.generation()] - ); - } - - #[test] - fn records_that_another_transaction_removes_are_gone_for_their_readers() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - let record = store - .dir() - .join(LEASES_NAME) - .join(format!("{}.json", lease.id())); - // A release in another process removed the binding, and is removing the record, which a - // scanner, say, holds open. - let held_binding = delete_pending(&sandbox.join(CLONE_BINDING_NAME)); - let held_record = delete_pending(&record); - assert_eq!(binding(&sandbox), None); - assert!(store.leases(generation).unwrap().is_empty()); - // A tombstone that a retirement removes, likewise. - let other = publish(&store, &ManifestSpec::template(), storage(2)); - store.tombstone(other.generation(), None).unwrap(); - let tombstone = store - .dir() - .join(TOMBSTONES_NAME) - .join(format!("{}.json", other.generation())); - let held_tombstone = delete_pending(&tombstone); - assert!(store.tombstones().unwrap().is_empty()); - assert_eq!( - store.select(other.key(), &after(&other, 1)).unwrap(), - Some(other.clone()) - ); - // A retirement finds no lease of the generation. - assert_eq!(store.retire(generation, None).unwrap(), Retirement::Retired); - drop((held_binding, held_record, held_tombstone)); - assert!(!record.exists()); - assert!(!tombstone.exists()); - } - - #[test] - fn a_release_whose_record_another_transaction_removes_ends_its_lease() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - let record = store - .dir() - .join(LEASES_NAME) - .join(format!("{}.json", lease.id())); - // A retirement in another process is removing the record, as the release does. - let held_record = delete_pending(&record); - assert_eq!(release(&store, &sandbox).unwrap(), Some(lease)); - assert_eq!(binding(&sandbox), None); - drop(held_record); - assert!(store.leases(generation).unwrap().is_empty()); - } - - #[test] - fn a_release_takes_the_files_of_its_sandbox_whose_deletion_is_pending_for_removed() { - let (directory, store) = new_store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let generation = published.generation(); - let sandbox = provision(directory.path(), "sandbox"); - let lease = leased(&store, &published, &sandbox); - // A stop removed the runtime record of the clone's VM, and the journal of its launch, - // which a scanner, say, holds: the release, which reads them through its handle of the - // sandbox's directory, finds that the sandbox records no VM and no launch. - let held: Vec = [JOURNAL_NAME, RUNTIME_NAME] - .into_iter() - .map(|name| { - fs::write(sandbox.join(name), b"{}").unwrap(); - delete_pending(&sandbox.join(name)) - }) - .collect(); - assert_eq!(release(&store, &sandbox).unwrap(), Some(lease)); - assert_eq!(binding(&sandbox), None); - assert!(store.leases(generation).unwrap().is_empty()); - drop(held); - // A release that removed the binding, which a reader holds, and failed before its record - // went, leaves a binding whose deletion is pending, which a release retried finds removed. - let again = leased(&store, &published, &sandbox); - let held_binding = delete_pending(&sandbox.join(CLONE_BINDING_NAME)); - assert_eq!(release(&store, &sandbox).unwrap(), None); - drop(held_binding); - assert_eq!(binding(&sandbox), None); - assert_eq!(store.leases(generation).unwrap(), [again]); - assert_eq!(store.retire(generation, None).unwrap(), Retirement::Retired); - } -} diff --git a/nvx-rs/src/host/template/store/tests.rs b/nvx-rs/src/host/template/store/tests.rs deleted file mode 100644 index 45467d334..000000000 --- a/nvx-rs/src/host/template/store/tests.rs +++ /dev/null @@ -1,988 +0,0 @@ -//! Tests of the template store: its creation and opening, its records and their formats, the -//! namespaces of its generations, its lock, and its refusal of directories and files that are -//! not private to the current user, or that are links. - -use std::sync::{Arc, Barrier, mpsc}; - -use serde_json::{Value, json}; - -use super::*; -use crate::host::snapshot::tests::{ManifestSpec, class}; -use crate::host::template::MANIFEST_NAME; -use crate::host::template::key::CaptureClass; -use crate::host::template::manifest::TemplateManifest; - -impl Store { - /// Publishes a generation of a template, whose fixed time contract names a host and boot of - /// its own, for the tests of other modules, and returns it. - pub(crate) fn publish_for_tests(&self) -> Generation { - tests::publish( - self, - &crate::host::snapshot::tests::ManifestSpec::template(), - tests::storage(1), - ) - } -} - -/// A change to a record, as JSON. -type Change = fn(&mut Value); - -/// Creates a store in a fresh temporary directory, and returns the directory and the store. -pub(super) fn store() -> (tempfile::TempDir, Store) { - let directory = tempfile::tempdir().unwrap(); - let store = Store::create(directory.path().join("templates")).unwrap(); - (directory, store) -} - -/// Returns the storage generation whose bytes are all `byte`, which must not be 0. -pub(super) fn storage(byte: u8) -> StorageGeneration { - StorageGeneration::from_bytes([byte; ID_LEN]).unwrap() -} - -/// Creates the directory `path` private to the current user, as the store creates its own. -pub(super) fn private_dir(path: &Path) { - assert_eq!(platform::create_owner_only_dir(path).unwrap(), None); -} - -/// Publishes a capture of `class` with the storage generation `storage`, whose manifest `spec` -/// encodes, as a publication does, under the store's lock: its snapshot at the place of the next -/// store generation, its key beside it, and then the index, which lists it. -pub(super) fn publish_with( - store: &Store, - class: &CaptureClass, - spec: &ManifestSpec, - storage: StorageGeneration, -) -> Generation { - let _lock = take_lock(store.dir(), None).unwrap(); - let mut index = store.index().unwrap(); - let generation = index.next_generation(); - let payload = store.payload_dir(generation); - let snapshot = payload.join(SNAPSHOT_NAME); - private_dir(&payload); - private_dir(&snapshot); - fs::write(snapshot.join(MANIFEST_NAME), spec.encode()).unwrap(); - let manifest = TemplateManifest::read(&snapshot).unwrap(); - let key = TemplateKey::new(class, &manifest).unwrap(); - write_private(&payload.join(KEY_NAME), &key.to_bytes()).unwrap(); - let published = Generation { - generation, - key: key.identity(), - class: class.identity().unwrap(), - storage, - created_at: manifest.created_at(), - time: manifest.time().clone(), - state_len: manifest.state_len(), - memory_len: manifest.memory_len(), - }; - index.generations.push(published.clone()); - index.next = StoreGeneration::new(generation.get().checked_add(1).unwrap()); - store.write_index(&index).unwrap(); - published -} - -/// Publishes a capture of the class of [`ManifestSpec::template`], as [`publish_with`] does. -pub(super) fn publish( - store: &Store, - spec: &ManifestSpec, - storage: StorageGeneration, -) -> Generation { - publish_with(store, &class(), spec, storage) -} - -/// Removes the generation of store generation `generation` from the index of `store`, under its -/// lock, as a retirement does. -pub(super) fn remove(store: &Store, generation: StoreGeneration) { - let _lock = take_lock(store.dir(), None).unwrap(); - let mut index = store.index().unwrap(); - index - .generations - .retain(|indexed| indexed.generation != generation); - store.write_index(&index).unwrap(); -} - -/// Returns the store's record `name` as JSON. -fn record(store: &Store, name: &str) -> Value { - serde_json::from_slice(&fs::read(store.dir().join(name)).unwrap()).unwrap() -} - -/// Replaces the store's record `name` with `value`. -fn rewrite(store: &Store, name: &str, value: &Value) { - write_private(&store.dir().join(name), value.to_string().as_bytes()).unwrap(); -} - -/// Returns the sorted names of the entries of `dir`. -fn sorted_entries(dir: &Path) -> Vec { - let mut names = entries(dir).unwrap(); - names.sort(); - names -} - -/// Returns the message of the cause of `error`. -pub(super) fn cause(error: &Error) -> String { - std::error::Error::source(error) - .map(ToString::to_string) - .unwrap_or_default() -} - -/// Links `link` to the directory `target`: a symbolic link on Unix, and a junction on Windows, -/// which any user may create. -pub(super) fn link_dir(target: &Path, link: &Path) { - #[cfg(unix)] - std::os::unix::fs::symlink(target, link).unwrap(); - #[cfg(windows)] - crate::host::tests::junction(link, target); -} - -/// Links `link` to the file `target` with a symbolic link, and returns whether it could: Windows -/// lets only Developer Mode or a privilege create one. -pub(super) fn link_file(target: &Path, link: &Path) -> bool { - #[cfg(unix)] - let linked = std::os::unix::fs::symlink(target, link); - #[cfg(windows)] - let linked = std::os::windows::fs::symlink_file(target, link); - if linked.is_err() { - eprintln!("skipped: this host does not let the test create a symbolic link"); - } - linked.is_ok() -} - -/// Removes the link `link` to a directory, which leaves its target as it is. -pub(super) fn unlink_dir(link: &Path) { - #[cfg(unix)] - fs::remove_file(link).unwrap(); - #[cfg(windows)] - fs::remove_dir(link).unwrap(); -} - -#[test] -fn stores_are_created_opened_and_reopened() { - let directory = tempfile::tempdir().unwrap(); - let dir = directory.path().join("templates"); - let error = Store::open(&dir).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); - let store = Store::create(&dir).unwrap(); - assert_eq!(store.dir(), dir); - assert_eq!( - sorted_entries(&dir), - [ - GENERATIONS_NAME, - INDEX_NAME, - INDEX_LOCK_NAME, - STORE_RECORD_NAME - ] - ); - assert!(entries(&dir.join(GENERATIONS_NAME)).unwrap().is_empty()); - let index = store.index().unwrap(); - assert!(index.generations().is_empty()); - assert_eq!(index.next_generation(), FIRST_GENERATION); - // Opening it, creating it again, and reopening it find the same store, with its index. - let published = publish(&store, &ManifestSpec::template(), storage(1)); - assert_eq!(Store::open(&dir).unwrap(), store); - assert_eq!(Store::create(&dir).unwrap(), store); - let id = store.id(); - drop(store); - let reopened = Store::open(&dir).unwrap(); - assert_eq!(reopened.id(), id); - assert_eq!(reopened.index().unwrap().generations(), [published]); - // Another store has an identity of its own. - let other = Store::create(directory.path().join("other")).unwrap(); - assert_ne!(other.id(), id); - // A directory without a store holds none. - let empty = directory.path().join("empty"); - private_dir(&empty); - let error = Store::open(&empty).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); - assert!( - error.message().ends_with("holds no template store"), - "{error}" - ); - // A store needs its parent. - let error = Store::create(directory.path().join("missing").join("templates")).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); -} - -#[test] -fn a_creation_replaces_an_interrupted_one_and_refuses_foreign_entries() { - let directory = tempfile::tempdir().unwrap(); - // What a creation leaves before it writes the identity record is replaced. - let interrupted = directory.path().join("interrupted"); - private_dir(&interrupted); - private_dir(&interrupted.join(GENERATIONS_NAME)); - for name in [ - INDEX_NAME, - INDEX_LOCK_NAME, - ".index.json.42.tmp", - ".store.json.7.tmp", - ] { - fs::write(interrupted.join(name), b"{").unwrap(); - } - let store = Store::create(&interrupted).unwrap(); - assert!(store.index().unwrap().generations().is_empty()); - assert_eq!(Store::open(&interrupted).unwrap(), store); - // Anything else is refused, and kept as it is: other files, and the payloads of a store whose - // identity record is gone. - let foreign = directory.path().join("foreign"); - private_dir(&foreign); - for name in ["notes.txt", ".index.json.1a.tmp", ".config.json.1.tmp"] { - fs::write(foreign.join(name), b"").unwrap(); - } - private_dir(&foreign.join(GENERATIONS_NAME)); - private_dir(&foreign.join(GENERATIONS_NAME).join("7")); - let before = sorted_entries(&foreign); - let error = Store::create(&foreign).unwrap_err(); - assert_eq!(error.code(), ErrorCode::ForeignFiles, "{error}"); - assert!( - error.message().ends_with( - "holds no template store, but entries that are not a store's: .config.json.1.tmp, \ - .index.json.1a.tmp, generations/7, notes.txt" - ), - "{error}" - ); - assert_eq!(sorted_entries(&foreign), before); - let error = Store::open(&foreign).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); -} - -#[test] -fn concurrent_creations_make_one_store() { - /// The creations that race. - const CREATIONS: usize = 8; - let directory = tempfile::tempdir().unwrap(); - let dir = directory.path().join("templates"); - let barrier = Arc::new(Barrier::new(CREATIONS)); - let creations: Vec<_> = (0..CREATIONS) - .map(|_| { - let (dir, barrier) = (dir.clone(), Arc::clone(&barrier)); - thread::spawn(move || { - barrier.wait(); - Store::create(&dir).unwrap().id() - }) - }) - .collect(); - let ids: Vec = creations - .into_iter() - .map(|creation| creation.join().unwrap()) - .collect(); - assert!(ids.iter().all(|id| *id == ids[0]), "{ids:?}"); - assert_eq!(Store::open(&dir).unwrap().id(), ids[0]); -} - -#[test] -fn a_creation_that_another_overtakes_opens_its_store() { - let directory = tempfile::tempdir().unwrap(); - // A creation that finds no store, and then the entries of one that another creation made - // meanwhile, even with a generation, opens that store. - let dir = directory.path().join("templates"); - let mut overtaking = None; - let store = Store::create_racing(dir.clone(), || { - let store = Store::create(&dir).unwrap(); - publish(&store, &ManifestSpec::template(), storage(1)); - overtaking = Some(store); - }) - .unwrap(); - assert_eq!(Some(store), overtaking); - // Entries that no store explains are still refused. - let foreign = directory.path().join("foreign"); - let error = Store::create_racing(foreign.clone(), || { - fs::write(foreign.join("notes.txt"), b"").unwrap(); - }) - .unwrap_err(); - assert_eq!(error.code(), ErrorCode::ForeignFiles, "{error}"); -} - -#[test] -fn records_keep_their_json_form() { - let (_directory, store) = store(); - let mut spec = ManifestSpec::template(); - // Windows keeps times in units of 100 ns. - spec.created_at = (1_700_000_000, 123_456_700); - spec.time().cut_to_anchor_ns = None; - let published = publish(&store, &spec, storage(9)); - let manifest = TemplateManifest::parse(&spec.encode()).unwrap(); - let key = TemplateKey::new(&class(), &manifest).unwrap(); - let time = manifest.time(); - assert_eq!( - record(&store, STORE_RECORD_NAME), - json!({ "format": 1, "store": store.id().to_string() }) - ); - // The index holds no VM generation ID, which only the clones' guests know. - assert_eq!( - record(&store, INDEX_NAME), - json!({ - "format": 1, - "store": store.id().to_string(), - "nextGeneration": 2, - "generations": [{ - "generation": 1, - "key": key.identity().to_string(), - "class": class().identity().unwrap().to_string(), - "storageGeneration": "09090909090909090909090909090909", - "createdAt": { "seconds": 1_700_000_000, "nanos": 123_456_700 }, - "time": { - "tscFrequencyHz": time.tsc_frequency_hz, - "tscTolerancePpm": time.tsc_tolerance_ppm, - "apicFrequencyHz": time.apic_frequency_hz, - "captureTsc": time.capture_tsc, - "captureUtcNs": time.capture_utc_ns, - "captureMonotonicNs": time.capture_monotonic_ns, - "hostClock": "linux-boottime", - "hostId": hex(&time.host_id), - "bootId": time.boot_id.to_string(), - "lineage": time.lineage, - "cutToAnchorNs": null, - }, - "stateLen": manifest.state_len(), - "memoryLen": manifest.memory_len(), - }], - }) - ); - // Each generation keeps its own metadata, which its manifest recorded, and its key. - let index = Store::open(store.dir()).unwrap().index().unwrap(); - assert_eq!(index.generations(), std::slice::from_ref(&published)); - assert_eq!(index.get(FIRST_GENERATION), Some(&published)); - assert_eq!(published.generation(), FIRST_GENERATION); - assert_eq!(published.key(), key.identity()); - assert_eq!(published.class(), class().identity().unwrap()); - assert_eq!(published.storage_generation(), storage(9)); - assert_eq!(published.created_at(), manifest.created_at()); - assert_eq!(published.time(), manifest.time()); - assert_eq!(published.state_len(), manifest.state_len()); - assert_eq!(published.memory_len(), manifest.memory_len()); - assert_eq!(store.read_key(&published).unwrap(), key); - assert_eq!( - store.snapshot_dir(&published).unwrap(), - store - .dir() - .join(GENERATIONS_NAME) - .join("1") - .join(SNAPSHOT_NAME) - ); -} - -#[test] -fn records_of_other_formats_fail_clearly() { - let (_directory, store) = store(); - publish(&store, &ManifestSpec::template(), storage(1)); - let valid = record(&store, INDEX_NAME); - for format in [0, 2] { - let mut index = valid.clone(); - index["format"] = format.into(); - rewrite(&store, INDEX_NAME, &index); - let error = store.index().unwrap_err(); - assert_eq!(error.code(), ErrorCode::FormatUnsupported, "{error}"); - assert!( - error.message().contains(&format!( - "has the unsupported format {format}; this build reads format 1 only and \ - migrates no other" - )), - "{error}" - ); - } - let mut index = valid.clone(); - index.as_object_mut().unwrap().remove("format"); - rewrite(&store, INDEX_NAME, &index); - let error = store.index().unwrap_err(); - assert_eq!(error.code(), ErrorCode::Io, "{error}"); - assert!(error.message().ends_with("it has no format"), "{error}"); - rewrite(&store, INDEX_NAME, &valid); - assert_eq!(store.index().unwrap().generations().len(), 1); - // A store whose identity record is of another format is opened by no build of this one. - rewrite( - &store, - STORE_RECORD_NAME, - &json!({ "format": 2, "store": store.id().to_string() }), - ); - for error in [ - Store::open(store.dir()).unwrap_err(), - Store::create(store.dir()).unwrap_err(), - ] { - assert_eq!(error.code(), ErrorCode::FormatUnsupported, "{error}"); - } -} - -#[test] -fn malformed_indexes_and_those_of_other_stores_are_refused() { - let (_directory, store) = store(); - publish(&store, &ManifestSpec::template(), storage(1)); - publish(&store, &ManifestSpec::template(), storage(2)); - let valid = record(&store, INDEX_NAME); - let cases: [(&str, Change); 11] = [ - ("its next store generation is 0", |index| { - index["nextGeneration"] = 0.into(); - }), - ( - "it lists store generation 2, which its next store generation, 2, does not follow", - |index| index["nextGeneration"] = 2.into(), - ), - ( - "it lists store generation 1 out of order, or twice", - |index| { - index["generations"][1]["generation"] = 1.into(); - }, - ), - ( - "it lists store generation 0, which no generation has", - |index| index["generations"][0]["generation"] = 0.into(), - ), - ("the key of store generation 1 is invalid", |index| { - index["generations"][0]["key"] = "00".into(); - }), - ("the class of store generation 1 is invalid", |index| { - index["generations"][0]["class"] = "".into(); - }), - ( - "the storage generation of store generation 1 is invalid", - |index| index["generations"][0]["storageGeneration"] = "0".repeat(32).into(), - ), - ( - "the creation time of store generation 1 is invalid", - |index| { - index["generations"][0]["createdAt"]["nanos"] = 1_000_000_000.into(); - }, - ), - ( - "the time contract of store generation 1 has the unknown host clock \"tsc\"", - |index| index["generations"][0]["time"]["hostClock"] = "tsc".into(), - ), - ( - "the time contract of store generation 1 has the invalid host identity \"01\"", - |index| index["generations"][0]["time"]["hostId"] = "01".into(), - ), - ( - "it lists 1025 generations, more than the 1024 that a store holds", - |index| { - let entry = index["generations"][0].clone(); - index["generations"] = (1..=MAX_GENERATIONS + 1) - .map(|generation| { - let mut entry = entry.clone(); - entry["generation"] = generation.into(); - entry - }) - .collect(); - index["nextGeneration"] = (MAX_GENERATIONS + 2).into(); - }, - ), - ]; - for (problem, change) in cases { - let mut index = valid.clone(); - change(&mut index); - rewrite(&store, INDEX_NAME, &index); - let error = store.index().unwrap_err(); - assert_eq!(error.code(), ErrorCode::Io, "{error}"); - assert!( - error - .message() - .contains(&format!("is malformed: {problem}")), - "{problem}: {error}" - ); - } - // Fields that the format does not have are refused. - let mut index = valid.clone(); - index["generations"][0]["generationId"] = "00".repeat(16).into(); - rewrite(&store, INDEX_NAME, &index); - let error = store.index().unwrap_err(); - assert!(error.message().ends_with("is malformed"), "{error}"); - assert!( - cause(&error).starts_with("unknown field `generationId`"), - "{error}" - ); - // So is the index of another store. - let mut index = valid.clone(); - index["store"] = "07".repeat(16).into(); - rewrite(&store, INDEX_NAME, &index); - let error = store.index().unwrap_err(); - assert!( - error.message().ends_with(&format!( - "belongs to the template store {}, not to this one, {}", - "07".repeat(16), - store.id() - )), - "{error}" - ); - // And one longer than an index may be, before anything of it is parsed. - let path = store.dir().join(INDEX_NAME); - write_private(&path, &vec![b' '; MAX_INDEX_LEN as usize + 1]).unwrap(); - let error = store.index().unwrap_err(); - assert_eq!( - cause(&error), - format!( - "it holds more than the {MAX_INDEX_LEN} bytes that the index of the template store \ - may hold" - ) - ); - // A missing index is an error of the store, not an empty index. - fs::remove_file(&path).unwrap(); - let error = store.index().unwrap_err(); - assert_eq!(error.code(), ErrorCode::Io, "{error}"); - assert!(error.message().ends_with("has no index"), "{error}"); - rewrite(&store, INDEX_NAME, &valid); - assert_eq!(store.index().unwrap().generations().len(), 2); -} - -#[test] -fn the_four_namespaces_stay_apart() { - let (_directory, store) = store(); - // Store generations follow the order of publication alone: neither the time lineage counters, - // nor the storage generations, which captures may share, and which may look like a store - // generation. - let one = StorageGeneration::from_bytes(1u128.to_be_bytes()).unwrap(); - let mut published = Vec::new(); - for (lineage, storage) in [(7, one), (7, one), (1, storage(0xff))] { - let mut spec = ManifestSpec::template(); - spec.time().lineage = lineage; - let generation = publish(&store, &spec, storage); - published.push(( - generation.generation().get().get(), - generation.time().lineage, - generation.storage_generation(), - )); - } - assert_eq!(published, [(1, 7, one), (2, 7, one), (3, 1, storage(0xff))]); - // A store generation is never assigned again, even once its generation is gone. - let third = StoreGeneration::new(NonZeroU64::new(3).unwrap()); - remove(&store, third); - let index = store.index().unwrap(); - assert_eq!(index.get(third), None); - assert_eq!(index.next_generation().get().get(), 4); - let fourth = publish(&store, &ManifestSpec::template(), one); - assert_eq!(fourth.generation().get().get(), 4); - // The generations of one key, whatever their lineage counters and storage generations. - let of_key: Vec = store - .index() - .unwrap() - .of_key(fourth.key()) - .map(|generation| generation.generation().get().get()) - .collect(); - assert_eq!(of_key, [1, 2, 4]); - // The text form of each names nothing of another's. - assert_eq!(one.to_string(), "00000000000000000000000000000001"); - assert_eq!(fourth.generation().to_string(), "4"); - assert!("1".parse::().is_err()); - assert!("4".parse::().is_err()); - assert_eq!(StorageGeneration::from_bytes([0; ID_LEN]), None); - assert!("0".repeat(32).parse::().is_err()); - assert_eq!( - "00000000000000000000000000000001" - .parse::() - .unwrap(), - one - ); - assert_eq!( - "FF".repeat(16).parse::().unwrap(), - storage(0xff) - ); - assert_eq!(format!("{one:?}"), format!("StorageGeneration({one})")); - let drawn = StorageGeneration::generate().unwrap(); - assert_ne!(drawn, StorageGeneration::generate().unwrap()); - // A store's identity is none of them either. - let id = store.id(); - assert_eq!(id.to_string().parse::().unwrap(), id); - assert_eq!( - id.to_string().to_uppercase().parse::().unwrap(), - id - ); - assert_eq!(format!("{id:?}"), format!("StoreId({id})")); - assert!("1".parse::().is_err()); -} - -#[test] -fn payloads_are_checked_as_they_are_read() { - let (_directory, store) = store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let payload = store.payload_dir(published.generation()); - let key = payload.join(KEY_NAME); - let bytes = fs::read(&key).unwrap(); - // A key that is not a key's canonical encoding is refused as such. - write_private(&key, &[2, 0]).unwrap(); - let error = store.read_key(&published).unwrap_err(); - assert_eq!(error.code(), ErrorCode::FormatUnsupported, "{error}"); - assert!( - error - .message() - .starts_with(&format!("cannot read {}: ", key.display())), - "{error}" - ); - write_private(&key, &bytes[..bytes.len() - 1]).unwrap(); - let error = store.read_key(&published).unwrap_err(); - assert_eq!(error.code(), ErrorCode::Io, "{error}"); - write_private(&key, &vec![0; TemplateKey::MAX_LEN + 1]).unwrap(); - let error = store.read_key(&published).unwrap_err(); - assert_eq!( - cause(&error), - format!( - "it holds more than the {} bytes that the template key may hold", - TemplateKey::MAX_LEN - ) - ); - // A missing key, snapshot, or payload, which a crash may lose, is missing. - fs::remove_file(&key).unwrap(); - let error = store.read_key(&published).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); - fs::remove_dir_all(payload.join(SNAPSHOT_NAME)).unwrap(); - let error = store.snapshot_dir(&published).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); - fs::remove_dir_all(&payload).unwrap(); - for error in [ - store.snapshot_dir(&published).unwrap_err(), - store.read_key(&published).unwrap_err(), - ] { - assert_eq!(error.kind(), ErrorKind::NotFound, "{error}"); - } -} - -#[test] -fn the_index_lock_excludes_other_holders() { - let (_directory, store) = store(); - let held = take_lock(store.dir(), None).unwrap(); - let started = Instant::now(); - let deadline = Some(Instant::now() + Duration::from_millis(100)); - let error = take_lock(store.dir(), deadline).unwrap_err(); - assert_eq!(error.kind(), ErrorKind::TimedOut, "{error}"); - assert!(error.message().ends_with("is still in progress"), "{error}"); - assert!(started.elapsed() >= Duration::from_millis(100)); - // A creation takes the lock too, and waits for its holder to release it. - let (sender, receiver) = mpsc::channel(); - let dir = store.dir().to_owned(); - let creation = thread::spawn(move || { - let created = Store::create(&dir).map(|store| store.id()); - sender.send(()).unwrap(); - created - }); - assert!(receiver.recv_timeout(Duration::from_millis(200)).is_err()); - drop(held); - receiver.recv_timeout(Duration::from_secs(30)).unwrap(); - assert_eq!(creation.join().unwrap().unwrap(), store.id()); -} - -#[test] -fn links_are_refused() { - let (directory, store) = store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - // A path that a separator ends names the store's own directory. - assert_eq!(Store::open(store.dir().join("")).unwrap(), store); - assert_eq!(Store::create(store.dir().join("")).unwrap(), store); - // A link to a store is no store, however its path ends: Linux follows a link at a name that a - // separator ends, even when it follows none at the final name. - let link = directory.path().join("link"); - link_dir(store.dir(), &link); - for path in [link.clone(), link.join(""), link.join(".")] { - for error in [ - Store::create(&path).unwrap_err(), - Store::open(&path).unwrap_err(), - ] { - assert_eq!(error.code(), ErrorCode::Io, "{}: {error}", path.display()); - assert!( - error.message().ends_with("is not a plain directory"), - "{}: {error}", - path.display() - ); - } - } - // Nor is a link to the generations, to a payload, or to a snapshot followed. - let payload = store.payload_dir(published.generation()); - for path in [ - store.dir().join(GENERATIONS_NAME), - payload.clone(), - payload.join(SNAPSHOT_NAME), - ] { - let moved = path.with_extension("moved"); - fs::rename(&path, &moved).unwrap(); - link_dir(&moved, &path); - let mut errors = vec![store.snapshot_dir(&published).unwrap_err()]; - if path != payload.join(SNAPSHOT_NAME) { - errors.push(store.read_key(&published).unwrap_err()); - } - if path.ends_with(GENERATIONS_NAME) { - errors.push(Store::open(store.dir()).unwrap_err()); - } - for error in errors { - assert!( - error.message().ends_with("is not a plain directory"), - "{}: {error}", - path.display() - ); - } - unlink_dir(&path); - fs::rename(&moved, &path).unwrap(); - } - assert_eq!( - store.read_key(&published).unwrap().identity(), - published.key() - ); - // Nor is a link to a record, or to a key. - for path in [ - store.dir().join(STORE_RECORD_NAME), - store.dir().join(INDEX_NAME), - payload.join(KEY_NAME), - ] { - let moved = path.with_extension("moved"); - fs::rename(&path, &moved).unwrap(); - if !link_file(&moved, &path) { - fs::rename(&moved, &path).unwrap(); - return; - } - let error = if path.ends_with(STORE_RECORD_NAME) { - Store::open(store.dir()).unwrap_err() - } else if path.ends_with(INDEX_NAME) { - store.index().unwrap_err() - } else { - store.read_key(&published).unwrap_err() - }; - assert_eq!( - cause(&error), - "it is a link, or not a regular file", - "{}: {error}", - path.display() - ); - fs::remove_file(&path).unwrap(); - fs::rename(&moved, &path).unwrap(); - } - // Nor a link to the lock file. - let lock = store.dir().join(INDEX_LOCK_NAME); - let moved = lock.with_extension("moved"); - fs::rename(&lock, &moved).unwrap(); - assert!(link_file(&moved, &lock)); - let error = take_lock(store.dir(), None).unwrap_err(); - assert_eq!(cause(&error), "is not a regular file", "{error}"); -} - -#[cfg(unix)] -#[test] -fn records_that_are_not_regular_files_fail_at_once() { - use nix::sys::stat::Mode; - - let (_directory, store) = store(); - for name in [STORE_RECORD_NAME, INDEX_NAME, INDEX_LOCK_NAME] { - let path = store.dir().join(name); - let moved = path.with_extension("moved"); - fs::rename(&path, &moved).unwrap(); - nix::unistd::mkfifo(&path, Mode::S_IRUSR | Mode::S_IWUSR).unwrap(); - // A read that waited for a writer would never end, so it runs on a thread that the test - // can leave behind. - let (sender, receiver) = mpsc::channel(); - let opened = store.clone(); - thread::spawn(move || { - let result = match name { - STORE_RECORD_NAME => Store::open(opened.dir()).map(drop), - INDEX_NAME => opened.index().map(drop), - _ => take_lock(opened.dir(), None).map(drop), - }; - sender.send(result).unwrap(); - }); - let error = receiver - .recv_timeout(Duration::from_secs(10)) - .expect("the read does not wait for a writer") - .unwrap_err(); - assert!( - [ - "it is a link, or not a regular file", - "is not a regular file" - ] - .contains(&cause(&error).as_str()), - "{name}: {error}" - ); - fs::remove_file(&path).unwrap(); - fs::rename(&moved, &path).unwrap(); - } -} - -#[cfg(target_os = "linux")] -#[test] -fn the_store_is_private_to_the_current_user() { - use std::os::unix::fs::PermissionsExt; - - let (directory, store) = store(); - let mode = |path: &Path| fs::metadata(path).unwrap().permissions().mode() & 0o7777; - let set_mode = |path: &Path, mode: u32| { - fs::set_permissions(path, fs::Permissions::from_mode(mode)).unwrap(); - }; - // The store's directories and records are the current user's alone. - for path in [store.dir().to_owned(), store.dir().join(GENERATIONS_NAME)] { - assert_eq!(mode(&path), 0o700, "{}", path.display()); - } - for name in [STORE_RECORD_NAME, INDEX_NAME, INDEX_LOCK_NAME] { - assert_eq!(mode(&store.dir().join(name)), 0o600, "{name}"); - } - // A directory that others could access is refused, and left as it is, rather than - // restricted: another user could hold its files open. - let shared = directory.path().join("shared"); - fs::create_dir(&shared).unwrap(); - for shared_mode in [0o750, 0o705, 0o755, 0o777] { - set_mode(&shared, shared_mode); - for error in [ - Store::create(&shared).unwrap_err(), - Store::open(&shared).unwrap_err(), - ] { - let expected = format!("is accessible to other users (mode {shared_mode:03o})"); - assert!(error.message().ends_with(&expected), "{error}"); - } - assert_eq!(mode(&shared), shared_mode); - } - assert!(entries(&shared).unwrap().is_empty()); - // So are a store whose generations others could access, and its payloads. - let generations = store.dir().join(GENERATIONS_NAME); - set_mode(&generations, 0o750); - let error = Store::open(store.dir()).unwrap_err(); - assert!( - error - .message() - .ends_with("is accessible to other users (mode 750)"), - "{error}" - ); - set_mode(&generations, 0o700); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let snapshot = store - .payload_dir(published.generation()) - .join(SNAPSHOT_NAME); - set_mode(&snapshot, 0o755); - let error = store.snapshot_dir(&published).unwrap_err(); - assert!( - error - .message() - .ends_with("is accessible to other users (mode 755)"), - "{error}" - ); - // A store created under a setgid directory drops the bit for mode 0700, and an existing - // directory that grants no one else access is private, whatever else its mode holds. - let setgid = directory.path().join("setgid"); - fs::create_dir(&setgid).unwrap(); - set_mode(&setgid, 0o2700); - let nested = Store::create(setgid.join("templates")).unwrap(); - assert_eq!(mode(nested.dir()), 0o700); - let existing = directory.path().join("existing"); - fs::create_dir(&existing).unwrap(); - set_mode(&existing, 0o2700); - Store::create(&existing).unwrap(); - assert_eq!(mode(&existing), 0o2700); -} - -#[cfg(target_os = "linux")] -#[test] -fn private_directories_belong_to_the_current_user_and_grant_others_nothing() { - use nix::sys::stat::Mode; - use nix::unistd::Uid; - - // The check takes the owner and the current user as user IDs, and the permission bits of the - // directory's mode, without the bits of its type, as the store reads them from its metadata. - let owner_only_problem = |uid, mode, euid| { - platform::tests::owner_only_problem( - Uid::from_raw(uid), - Mode::from_bits_truncate(mode), - Uid::from_raw(euid), - ) - }; - // Another user's directory is refused whatever its mode, since its owner may change it. - assert_eq!( - owner_only_problem(1001, 0o40700, 1000), - Some("belongs to user 1001, not to the current user 1000".to_owned()) - ); - for mode in [0o40700, 0o42700, 0o41700, 0o40500, 0o40300] { - assert_eq!(owner_only_problem(1000, mode, 1000), None, "{mode:o}"); - } - for mode in [0o40750, 0o40705, 0o40770, 0o40701, 0o40710] { - let problem = owner_only_problem(1000, mode, 1000).unwrap(); - assert!( - problem.starts_with("is accessible to other users"), - "{problem}" - ); - } -} - -#[cfg(windows)] -#[test] -fn the_store_is_private_to_the_current_user() { - let directory = tempfile::tempdir().unwrap(); - platform::tests::set_access(directory.path(), "D:(A;OICI;FA;;;WD)").unwrap(); - // A directory that every user may change holds no store, nor does one created in it. - let error = Store::open(directory.path()).unwrap_err(); - assert!(error.message().ends_with("allows other users"), "{error}"); - let shared = directory.path().join("shared"); - fs::create_dir(&shared).unwrap(); - let error = Store::create(&shared).unwrap_err(); - assert!(error.message().ends_with("allows other users"), "{error}"); - assert!(entries(&shared).unwrap().is_empty()); - // Nor does one that allows its owner alone, but passes nothing on to the files created in it, - // which would take their creator's default list. - platform::tests::set_access(&shared, "D:P(A;;FA;;;OW)").unwrap(); - for error in [ - Store::create(&shared).unwrap_err(), - Store::open(&shared).unwrap_err(), - ] { - assert!( - error - .message() - .ends_with("passes no access control entries on to the files created in it"), - "{error}" - ); - } - // A store created there allows the current user alone, and so do its records. - let store = Store::create(directory.path().join("templates")).unwrap(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let payload = store.payload_dir(published.generation()); - let problem = |path: &Path| { - let file = File::open(path).unwrap(); - platform::private_file_problem(&file).unwrap() - }; - for path in [ - store.dir().join(STORE_RECORD_NAME), - store.dir().join(INDEX_NAME), - store.dir().join(INDEX_LOCK_NAME), - payload.join(KEY_NAME), - ] { - assert_eq!(problem(&path), None, "{}", path.display()); - } - // A payload that every user may change is refused. - let snapshot = payload.join(SNAPSHOT_NAME); - platform::tests::set_access(&snapshot, "D:(A;OICI;FA;;;WD)").unwrap(); - let error = store.snapshot_dir(&published).unwrap_err(); - assert!(error.message().ends_with("allows other users"), "{error}"); - // So is a record or a key whose own list allows others, who could have written it, whatever - // the store's directory allows, and a lock file that others could hold. - for name in [KEY_NAME, INDEX_NAME, INDEX_LOCK_NAME, STORE_RECORD_NAME] { - let path = if name == KEY_NAME { - payload.join(name) - } else { - store.dir().join(name) - }; - platform::tests::set_access(&path, "D:P(A;;FA;;;WD)").unwrap(); - let error = match name { - KEY_NAME => store.read_key(&published).unwrap_err(), - INDEX_NAME => store.index().unwrap_err(), - INDEX_LOCK_NAME => take_lock(store.dir(), None).unwrap_err(), - _ => Store::open(store.dir()).unwrap_err(), - }; - assert!( - error.message().ends_with("allows other users"), - "{name}: {error}" - ); - } -} - -#[cfg(windows)] -#[test] -fn snapshots_that_openvmm_published_are_private_without_entries_for_their_files() { - let (_directory, store) = store(); - let published = publish(&store, &ManifestSpec::template(), storage(1)); - let payload = store.payload_dir(published.generation()); - // OpenVMM gives a snapshot's directory, and each of its files, a protected list of their own, - // which allows SYSTEM and the owner, and passes nothing on to the files created in it. - let openvmm = "D:P(A;;FA;;;SY)(A;;FA;;;OW)"; - let snapshot = payload.join(SNAPSHOT_NAME); - platform::tests::set_access(&snapshot.join(MANIFEST_NAME), openvmm).unwrap(); - platform::tests::set_access(&snapshot, openvmm).unwrap(); - assert_eq!(store.snapshot_dir(&published).unwrap(), snapshot); - // The host's own directories still pass entries on to the files created in them. - platform::tests::set_access(&payload, openvmm).unwrap(); - let error = store.snapshot_dir(&published).unwrap_err(); - assert!( - error - .message() - .ends_with("passes no access control entries on to the files created in it"), - "{error}" - ); - // A snapshot whose list allows anyone else is refused. - platform::tests::set_access(&payload, "D:P(A;OICI;FA;;;OW)").unwrap(); - platform::tests::set_access(&snapshot, "D:P(A;;FA;;;SY)(A;;FA;;;OW)(A;;FR;;;WD)").unwrap(); - let error = store.snapshot_dir(&published).unwrap_err(); - assert!(error.message().ends_with("allows other users"), "{error}"); -} diff --git a/nvx-rs/src/host/vm_directory/tests.rs b/nvx-rs/src/host/vm_directory/tests.rs index 1a284542b..15d7e1da4 100644 --- a/nvx-rs/src/host/vm_directory/tests.rs +++ b/nvx-rs/src/host/vm_directory/tests.rs @@ -103,7 +103,6 @@ fn an_existing_directory_that_is_not_private_is_refused_and_left_unchanged() { ); assert_eq!(mode(&path), other); } - assert_eq!(platform::check_owner_only_dir(&path).unwrap(), None); assert_eq!( platform::tests::store_dir_problem(Uid::from_raw(1001), Mode::S_IRWXU, Uid::from_raw(1000)), Some("belongs to user 1001, not to the current user 1000".to_owned()) diff --git a/nvx-rs/src/lib.rs b/nvx-rs/src/lib.rs index 90c2b4fdf..5b635d012 100644 --- a/nvx-rs/src/lib.rs +++ b/nvx-rs/src/lib.rs @@ -35,7 +35,7 @@ //! network and filesystem policies, the host's NVXS connection, control sessions, process and //! platform primitives, the supervision of VM instances: starting, reattaching, executing //! workloads, and stopping, the store of a sandbox's runtime records and launch journal, and the -//! manifests, compatibility keys, store, and leases of templates. +//! manifests and compatibility keys of templates. //! - `guest` (feature `guest`, Linux): the programs that run in NVX guests, such as //! `nvx-console-pending`, the guest's NVXS connection, the libraries through which any guest //! init or agent opens the terminal on which it serves its session (`guest::terminal`), starts diff --git a/nvx-rs/tests/openvmm_vectors.rs b/nvx-rs/tests/openvmm_vectors.rs index 803910b40..9febc1b6f 100644 --- a/nvx-rs/tests/openvmm_vectors.rs +++ b/nvx-rs/tests/openvmm_vectors.rs @@ -2,10 +2,10 @@ //! protocol vectors and the excerpts of OpenVMM's sources in `tests/vectors/openvmm/`, and the //! versions of OpenVMM's contracts in `nvx::contract`. //! -//! The protocol's tests read the copies, and the unit tests of `nvx::host::template` compile or -//! read the excerpts, because the `nvx-rs` job cannot check out the submodule, whose deploy key -//! fork pull requests lack. This test is ignored by default: the NVX CLI jobs, which check out the -//! submodule, run it, and so does an OpenVMM pin promotion. +//! The protocol's tests read the copies, and the unit tests of `nvx::host::config` compile the +//! excerpts of OpenVMM's allowlist, because the `nvx-rs` job cannot check out the submodule, whose +//! deploy key fork pull requests lack. This test is ignored by default: the NVX CLI jobs, which +//! check out the submodule, run it, and so does an OpenVMM pin promotion. use std::fs; use std::path::Path; diff --git a/nvx-rs/tests/template_capture.rs b/nvx-rs/tests/template_capture.rs deleted file mode 100644 index d03074966..000000000 --- a/nvx-rs/tests/template_capture.rs +++ /dev/null @@ -1,697 +0,0 @@ -//! Captures of templates through the supervisor, with the fake OpenVMM as their VMM: the block -//! devices that the VMM opens, which no handle of the host may write as it starts, the -//! generations that captures publish and that creations then serve, VMs that end without a -//! capture or never end, captures of one class that race, and an adapter that crashes while its -//! capture's VM runs, whose VMM a recovery ends. -//! -//! The fake OpenVMM publishes the snapshot whose manifest OpenVMM wrote for the fixture -//! `tests/fixtures/template/platform-kvm`, or fails, as the scenario next to the kernel asks; see -//! `nvx::testing::fake_openvmm`. The crash is real: a child process of this test binary, the -//! adapter, captures in the store, and the test kills it while its capture's VM runs. -#![cfg(all( - feature = "host", - feature = "testing", - any(target_os = "linux", windows) -))] - -use std::env; -use std::fs; -use std::path::{Path, PathBuf}; -use std::process::{Command, Stdio}; -use std::sync::{Mutex, MutexGuard, PoisonError}; -use std::thread; -use std::time::{Duration, Instant}; - -use nvx::host::platform; -use nvx::host::process::{BootId, Opened, Process, ProcessRef}; -use nvx::host::sandbox::distro::{DISTRO_POLICY_REVISION, DistroBinding, PolicyVerdict}; -use nvx::host::sandbox::{LayerRole, SandboxDistro, SandboxLayer, SandboxPlan, SandboxUpper}; -#[cfg(target_os = "linux")] -use nvx::host::store::SOCKET_NAME; -use nvx::host::store::{ArtifactIdentity, OUTCOME_NAME, RuntimeIdentity, SandboxStore}; -use nvx::host::template::{ - BlockGeometry, CAPTURES_NAME, Capture, CaptureClass, Created, GENERATIONS_NAME, Generation, - HostTime, Invariants, SCRATCH_NAME, SNAPSHOT_NAME, Store, TRANSACTION_NAME, TemplateKey, - TemplateManifest, -}; -use nvx::protocol::root::RootImage; -use nvx::uuid::Uuid; -use nvx::{Artifacts, ErrorCode, ErrorKind, Hypervisor}; -use serde_json::{Value, json}; - -/// Bytes in a MiB. -const MIB: u64 = 1024 * 1024; -/// Nanoseconds in a second. -const SECOND_NS: u64 = 1_000_000_000; -/// The identity of the placeholder. -const IDENTITY: &str = "11111111-1111-1111-1111-111111111111"; -/// Name of the scenario of the fake OpenVMM's captures, next to the kernel. -const SCENARIO_NAME: &str = "fake-openvmm-capture.json"; -/// Name of the marker that the fake OpenVMM holds while a capture's VM runs, in the platform's -/// directory. -const ACTIVE_NAME: &str = "active"; -/// The longest wait of these tests for a capture, a recovery, or a process. -const TIMEOUT: Duration = Duration::from_secs(60); -/// Environment variable that marks the adapter that [`adapter_role`] runs, and names the -/// directory of its bench. -const ADAPTER_BENCH: &str = "NVX_TEMPLATE_CAPTURE_TEST_BENCH"; - -/// Serializes the tests that spawn processes: a process that one test spawns while another test's -/// log is inheritable for its own spawn could hold that log's claim for a while. -static SPAWNS: Mutex<()> = Mutex::new(()); - -/// Takes [`SPAWNS`]. -fn serialized() -> MutexGuard<'static, ()> { - SPAWNS.lock().unwrap_or_else(PoisonError::into_inner) -} - -/// Returns the directory of the fixture whose manifest OpenVMM wrote for a capture of [`class`]. -fn fixture() -> PathBuf { - Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures/template/platform-kvm") -} - -/// Returns the manifest of [`fixture`]. -fn manifest() -> TemplateManifest { - TemplateManifest::read(&fixture()).unwrap() -} - -/// Returns the identity `sha256:` and 64 copies of `digit`. -fn artifact(digit: char) -> ArtifactIdentity { - ArtifactIdentity::sha256(&digit.to_string().repeat(64)).unwrap() -} - -/// Returns the class of the capture of [`fixture`]. -fn class() -> CaptureClass { - CaptureClass { - runtime: RuntimeIdentity::new(artifact('a'), artifact('b'), artifact('c')), - guest_build: ArtifactIdentity::new("guest-build-1").unwrap(), - hypervisor: Hypervisor::Kvm, - cpu_profile: "auto".to_owned(), - processors: 2, - boot_processors: 2, - memory: 512 * MIB, - memory_capacity: 2048 * MIB, - distro: BlockGeometry { - length: 1024 * MIB, - logical_block_size: 512, - physical_block_size: 4096, - }, - scratch: BlockGeometry { - length: 64 * MIB, - logical_block_size: 512, - physical_block_size: 512, - }, - network: None, - invariants: Invariants::new(), - } -} - -/// Returns the host's time `seconds` after the capture of [`fixture`], on the host and in the -/// boot that captured it. -fn after(seconds: u64) -> HostTime { - let manifest = manifest(); - let time = manifest.time(); - HostTime { - host_id: time.host_id, - boot_id: time.boot_id, - clock: time.host_clock, - monotonic_ns: time.capture_monotonic_ns + seconds * SECOND_NS, - utc_ns: time.capture_utc_ns + seconds * SECOND_NS, - } -} - -/// Returns a deadline that no capture of these tests reaches. -fn deadline() -> Instant { - Instant::now() + TIMEOUT -} - -/// Creates the directory `path`, private to the current user: on Windows, with a protected list -/// that allows the owner and SYSTEM alone, which the files created in it inherit. -fn private_dir(path: &Path) { - fs::create_dir(path).unwrap(); - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - - fs::set_permissions(path, fs::Permissions::from_mode(0o700)).unwrap(); - } - #[cfg(windows)] - windows::set_access(path, "D:P(A;OICI;FA;;;OW)(A;OICI;FA;;;SY)"); -} - -/// Creates the sparse file `path` of `length` bytes, which no one else may write, whatever the -/// umask. -fn sparse(path: &Path, length: u64) { - let mut options = fs::OpenOptions::new(); - options.write(true).create_new(true); - #[cfg(unix)] - std::os::unix::fs::OpenOptionsExt::mode(&mut options, 0o600); - let file = options.open(path).unwrap(); - #[cfg(windows)] - windows::set_sparse(&file); - file.set_len(length).unwrap(); -} - -/// Returns the file at `path` as JSON. -fn json_of(path: &Path) -> Value { - serde_json::from_slice(&fs::read(path).unwrap()).unwrap() -} - -/// Returns the names of the entries of `dir`, sorted. -fn sorted(dir: &Path) -> Vec { - let mut names: Vec = fs::read_dir(dir) - .unwrap() - .map(|entry| entry.unwrap().file_name().into_string().unwrap()) - .collect(); - names.sort(); - names -} - -/// Returns the store generation of `generation`, as a number. -fn number(generation: &Generation) -> u64 { - generation.generation().get().get() -} - -/// A store, and the inputs of captures of [`class`] into it, in a temporary directory: the -/// platform's files in `platform`, a placeholder, a kernel, an initramfs, and the fake OpenVMM's -/// scenario, which a capture's VM follows, and the store in `store`. -struct Bench { - /// The temporary directory. - directory: tempfile::TempDir, - /// The store. - store: Store, - /// The inputs of a capture. - capture: Capture, -} - -impl Bench { - /// Returns a bench whose VMs publish the fixture's snapshot, as `scenario` modifies the - /// default scenario. - fn new(scenario: &Value) -> Self { - let directory = tempfile::tempdir().unwrap(); - let platform = directory.path().join("platform"); - private_dir(&platform); - sparse(&platform.join("placeholder.erofs"), class().distro.length); - fs::write(platform.join("vmlinux"), b"kernel").unwrap(); - fs::write(platform.join("initramfs.cpio.gz"), b"initrd").unwrap(); - let store = Store::create(directory.path().join("store")).unwrap(); - let bench = Self { - store, - capture: capture_in(&platform), - directory, - }; - bench.follow(scenario); - bench - } - - /// Returns the bench in `dir`, as another process made it. - fn open(dir: &Path) -> (Store, Capture) { - ( - Store::open(dir.join("store")).unwrap(), - capture_in(&dir.join("platform")), - ) - } - - /// Returns the platform's directory. - fn platform(&self) -> PathBuf { - self.directory.path().join("platform") - } - - /// Lets the next VMs follow the default scenario, as `fields` modify it: the fake OpenVMM - /// publishes the fixture's snapshot. - fn follow(&self, fields: &Value) { - let mut scenario = json!({ - "manifest": fixture().join("manifest.bin"), - "stateLen": manifest().state_len(), - }); - for (name, value) in fields.as_object().unwrap() { - scenario[name] = value.clone(); - } - fs::write(self.platform().join(SCENARIO_NAME), scenario.to_string()).unwrap(); - } - - /// Returns the path of the marker of an active VM. - fn active(&self) -> PathBuf { - self.platform().join(ACTIVE_NAME) - } - - /// Returns the slot of the class, the only one. - fn slot(&self) -> PathBuf { - let captures = self.store.dir().join(CAPTURES_NAME); - let slots = sorted(&captures); - assert_eq!(slots.len(), 1, "{slots:?}"); - captures.join(&slots[0]) - } - - /// Returns the path of the file `suffix` that the fake OpenVMM wrote of the slot's last VM, - /// which it names after the directory of the VM's outcome report: the capture's transaction, - /// from whose sandbox store the VM launched. - fn dump(&self, suffix: &str) -> PathBuf { - self.platform() - .join(format!("fake-openvmm-{TRANSACTION_NAME}{suffix}.json")) - } - - /// Creates a template at a second after the fixture's capture. - fn create(&self) -> nvx::Result { - self.store - .create_template(&self.capture, || Ok(after(1)), deadline()) - } - - /// Refreshes the template. - fn refresh(&self) -> nvx::Result { - self.store.refresh_template(&self.capture, deadline()) - } - - /// Returns the store generations that the index lists. - fn indexed(&self) -> Vec { - self.store - .index() - .unwrap() - .generations() - .iter() - .map(number) - .collect() - } - - /// Asserts that no capture of the class runs or remains: its slot holds no transaction and no - /// launch. - fn assert_idle(&self) { - let slot = self.slot(); - assert!(!slot.join(TRANSACTION_NAME).exists()); - assert!( - SandboxStore::open(&slot) - .unwrap() - .journal() - .unwrap() - .is_none() - ); - } -} - -/// Returns the inputs of a capture of [`class`] whose placeholder, kernel, initramfs, and -/// scenario are in `platform`, and whose VMM is the fake OpenVMM. -fn capture_in(platform: &Path) -> Capture { - let placeholder = platform.join("placeholder.erofs"); - let path = placeholder.to_str().unwrap(); - let layer = SandboxLayer::new(LayerRole::Distro, path, IDENTITY); - let binding = DistroBinding::import(&layer, PolicyVerdict::ACCEPTED).unwrap(); - let image = RootImage::erofs(Some(Uuid::parse(IDENTITY).unwrap())); - let upper = SandboxUpper::scratch(platform.join("clone.ext4").to_str().unwrap()); - Capture { - class: class(), - adapter: SandboxPlan::adapter(SandboxDistro::new(path, image), upper), - binding, - policy_revision: DISTRO_POLICY_REVISION, - artifacts: Artifacts::new( - env!("CARGO_BIN_EXE_nvx-fake-openvmm"), - platform.join("vmlinux"), - platform.join("initramfs.cpio.gz"), - ), - } -} - -/// Returns the value of `option` in the command line `arguments`. -fn value_of<'a>(arguments: &'a [String], option: &str) -> &'a str { - let at = arguments - .iter() - .position(|argument| argument == option) - .unwrap_or_else(|| panic!("{option} is missing from {arguments:?}")); - &arguments[at + 1] -} - -#[test] -fn captures_publish_generations_that_creations_then_serve() { - let _spawns = serialized(); - let bench = Bench::new(&json!({})); - let Created::Captured(generation) = bench.create().unwrap() else { - panic!("the first creation captures"); - }; - assert_eq!(bench.indexed(), [1]); - assert_eq!(number(&generation), 1); - // The VMM received the template's command line, with the generation's storage generation as - // its block devices' identity. - let arguments: Vec = serde_json::from_value(json_of(&bench.dump(""))).unwrap(); - assert_eq!(value_of(&arguments, "--cmdline"), "nvx_sandbox=1"); - assert_eq!( - value_of(&arguments, "--snapshot-generation-id"), - generation.storage_generation().to_string() - ); - let transaction = bench.slot().join(TRANSACTION_NAME); - assert_eq!( - Path::new(value_of(&arguments, "--snapshot-destination")), - transaction.join(SNAPSHOT_NAME) - ); - // The VM launched from the transaction's sandbox store, so that its control socket, on - // Linux, is beside the snapshot's destination, as OpenVMM requires, and its outcome report - // too, which the slot then keeps. - assert_eq!( - Path::new(value_of(&arguments, "--microvm-report")), - transaction.join(OUTCOME_NAME) - ); - #[cfg(target_os = "linux")] - assert_eq!( - value_of(&arguments, "--microvm-control-console"), - format!("listen={}", transaction.join(SOCKET_NAME).display()) - ); - assert!(bench.slot().join(OUTCOME_NAME).is_file()); - // The VMM opened the placeholder and the scratch at their paths, and found no handle that - // could write them: on Windows, its open would have failed, and on Linux, its read lease - // finds one, unless the file system takes no lease. - let blocks = json_of(&bench.dump("-blocks")); - let opened: Vec<(&str, PathBuf, u64)> = blocks - .as_array() - .unwrap() - .iter() - .map(|block| { - assert_ne!(block["writers"], true, "{block}"); - ( - block["role"].as_str().unwrap(), - PathBuf::from(block["path"].as_str().unwrap()), - block["length"].as_u64().unwrap(), - ) - }) - .collect(); - assert_eq!( - opened, - [ - ( - "distro", - bench.platform().join("placeholder.erofs"), - 1024 * MIB - ), - ("scratch", transaction.join(SCRATCH_NAME), 64 * MIB), - ] - ); - // The generation's payload holds the snapshot that the VMM published, with its key, and - // the slot keeps nothing of the capture. - let snapshot = bench.store.snapshot_dir(&generation).unwrap(); - assert_eq!( - sorted(&snapshot), - ["manifest.bin", "memory.bin", "state.bin"] - ); - let key = TemplateKey::new(&bench.capture.class, &manifest()).unwrap(); - assert_eq!(bench.store.read_key(&generation).unwrap(), key); - assert_eq!(generation.key(), key.identity()); - bench.assert_idle(); - // A second creation serves the generation without a VM. - fs::remove_file(bench.dump("")).unwrap(); - assert_eq!(bench.create().unwrap(), Created::Existing(generation)); - assert!(!bench.dump("").exists()); -} - -#[test] -fn refreshes_publish_new_generations_and_leave_the_older_ones() { - let _spawns = serialized(); - let bench = Bench::new(&json!({})); - let first = bench.create().unwrap().into_generation(); - // A VMM that ends abruptly once it has published leaves the file that backed the RAM, which - // the publication does not take. - bench.follow(&json!({ "keepMemory": true })); - let second = bench.refresh().unwrap(); - assert_eq!(number(&second), 2); - assert_ne!(second.storage_generation(), first.storage_generation()); - assert_eq!(bench.indexed(), [1, 2]); - for generation in [&first, &second] { - let snapshot = bench.store.snapshot_dir(generation).unwrap(); - assert_eq!( - sorted(&snapshot), - ["manifest.bin", "memory.bin", "state.bin"] - ); - } - assert_eq!( - sorted(&bench.store.dir().join(GENERATIONS_NAME)), - ["1", "2"] - ); - bench.assert_idle(); - assert_eq!(bench.create().unwrap(), Created::Existing(second)); -} - -#[test] -fn vms_that_end_without_a_capture_publish_nothing() { - let _spawns = serialized(); - let bench = Bench::new(&json!({})); - for scenario in [ - json!({ "outcome": "fail", "status": 3 }), - // The snapshot that the VMM published before it failed is not the capture's. - json!({ "outcome": "publish-then-fail" }), - json!({ "outcome": "crash" }), - ] { - bench.follow(&scenario); - let error = bench.refresh().unwrap_err(); - assert_eq!(error.kind(), ErrorKind::Exited, "{scenario}: {error}"); - assert!(bench.indexed().is_empty(), "{scenario}"); - bench.assert_idle(); - } - // The slot serves the next capture. - bench.follow(&json!({})); - bench.refresh().unwrap(); - assert_eq!(bench.indexed(), [1]); -} - -/// Returns the process ID that the marker of an active VM at `path` holds. -fn active_pid(path: &Path) -> u32 { - fs::read_to_string(path).unwrap().parse().unwrap() -} - -#[test] -fn vms_that_never_end_are_torn_down_at_the_deadline() { - let _spawns = serialized(); - let bench = Bench::new(&json!({})); - for outcome in ["hang", "publish-then-hang"] { - bench.follow(&json!({ "outcome": outcome, "active": bench.active() })); - let deadline = Instant::now() + Duration::from_secs(2); - let error = bench - .store - .refresh_template(&bench.capture, deadline) - .unwrap_err(); - assert_eq!(error.kind(), ErrorKind::TimedOut, "{outcome}: {error}"); - // The VMM, which was killed, left its marker. - let pid = active_pid(&bench.active()); - assert_eq!( - platform::process_start_time(pid).unwrap(), - None, - "{outcome}" - ); - fs::remove_file(bench.active()).unwrap(); - assert!(bench.indexed().is_empty(), "{outcome}"); - bench.assert_idle(); - } -} - -#[test] -fn captures_of_one_class_are_serialized() { - let _spawns = serialized(); - // A VM that started while another ran would fail to mark itself active. - let bench = Bench::new(&json!({})); - bench.follow(&json!({ "active": bench.active(), "delayMs": 300 })); - let (store, capture) = (&bench.store, &bench.capture); - let generations: Vec = thread::scope(|scope| { - let refreshes: Vec<_> = (0..2) - .map(|_| scope.spawn(|| store.refresh_template(capture, deadline()))) - .collect(); - refreshes - .into_iter() - .map(|refresh| refresh.join().unwrap().unwrap()) - .collect() - }); - let mut published: Vec = generations.iter().map(number).collect(); - published.sort_unstable(); - assert_eq!(published, [1, 2]); - assert_eq!(bench.indexed(), [1, 2]); - assert!(!bench.active().exists()); - bench.assert_idle(); -} - -/// The adapter that [`a_vm_that_outlives_its_crashed_adapter_is_ended_by_recovery`] runs in a -/// process of its own: it refreshes the template of the bench that [`ADAPTER_BENCH`] names, until -/// the test kills it. -#[test] -#[ignore = "an adapter that the capture tests run in a process of its own"] -fn adapter_role() { - let Some(dir) = env::var_os(ADAPTER_BENCH) else { - return; - }; - let (store, capture) = Bench::open(Path::new(&dir)); - let result = store.refresh_template(&capture, deadline()); - panic!("the capture ended before the test killed its adapter: {result:?}"); -} - -/// A VMM that a crashed adapter left behind, which is killed when the test ends if it still runs. -struct Orphan(ProcessRef); - -impl Orphan { - /// Returns whether the VMM still runs. - fn runs(&self) -> bool { - platform::process_start_time(self.0.pid).unwrap() == Some(self.0.start_time) - } -} - -impl Drop for Orphan { - fn drop(&mut self) { - if let Ok(Opened::Running(process)) = Process::open(&self.0) { - let _ = process.kill(); - let _ = process.wait(Instant::now() + TIMEOUT); - } - } -} - -/// Runs [`adapter_role`] on `bench` until its capture's VM is active, the journal of the -/// transaction's sandbox store records the VMM, and, if `published`, the VM has published its -/// snapshot, then kills the adapter, as a crash would end it, and returns the VMM, which still -/// runs. -fn crash_adapter(bench: &Bench, published: bool) -> Orphan { - let mut adapter = Command::new(env::current_exe().unwrap()) - .args([ - "--exact", - "adapter_role", - "--ignored", - "--nocapture", - "--test-threads=1", - ]) - .env(ADAPTER_BENCH, bench.directory.path()) - .stdin(Stdio::null()) - .stdout(Stdio::null()) - .spawn() - .unwrap(); - let started = Instant::now(); - let recorded = loop { - assert!( - adapter.try_wait().unwrap().is_none(), - "the adapter ended before the test killed it" - ); - // The VM marks itself active before it publishes its snapshot. - let recorded = bench.active().exists().then(|| { - let transaction = bench.slot().join(TRANSACTION_NAME); - if published && !transaction.join(SNAPSHOT_NAME).is_dir() { - return None; - } - SandboxStore::open(transaction) - .ok()? - .journal() - .ok()?? - .process() - .cloned() - }); - if let Some(Some(process)) = recorded { - break process; - } - assert!( - started.elapsed() < TIMEOUT, - "the capture's VM did not start" - ); - thread::sleep(Duration::from_millis(20)); - }; - adapter.kill().unwrap(); - adapter.wait().unwrap(); - assert_eq!(recorded.pid, active_pid(&bench.active())); - assert_eq!(recorded.boot, BootId::current().unwrap()); - let orphan = Orphan(recorded); - assert!(orphan.runs()); - orphan -} - -#[test] -fn a_vm_that_outlives_its_crashed_adapter_is_ended_by_recovery() { - let _spawns = serialized(); - let bench = Bench::new(&json!({})); - // A VM that published its snapshot, but whose exit no capture verified, is not published - // either. - for outcome in ["hang", "publish-then-hang"] { - bench.follow(&json!({ "outcome": outcome, "active": bench.active() })); - let orphan = crash_adapter(&bench, outcome == "publish-then-hang"); - let deadline = deadline(); - // A scanner that briefly opens the launch's log can keep its claim held for a moment. - let recovered = loop { - match bench.store.recover_captures(deadline) { - Err(error) - if error.code() == ErrorCode::VmmIndeterminate && Instant::now() < deadline => - { - eprintln!("the recovery is indeterminate: {error}"); - thread::sleep(Duration::from_millis(100)); - } - recovered => break recovered.unwrap(), - } - }; - assert!(recovered.is_empty(), "{outcome}"); - assert!(!orphan.runs(), "{outcome}"); - assert!(bench.indexed().is_empty(), "{outcome}"); - bench.assert_idle(); - fs::remove_file(bench.active()).unwrap(); - } - // The slot serves the next capture. - bench.follow(&json!({})); - bench.refresh().unwrap(); - assert_eq!(bench.indexed(), [1]); -} - -/// What the tests need of Windows that the crate does not export. -#[cfg(windows)] -mod windows { - use std::fs::File; - use std::os::windows::ffi::OsStrExt; - use std::os::windows::io::AsRawHandle; - use std::path::Path; - use std::{io, ptr}; - - use windows_sys::Win32::Foundation::{HANDLE, LocalFree}; - use windows_sys::Win32::Security::Authorization::{ - ConvertStringSecurityDescriptorToSecurityDescriptorW, SDDL_REVISION_1, - }; - use windows_sys::Win32::Security::{ - DACL_SECURITY_INFORMATION, PROTECTED_DACL_SECURITY_INFORMATION, SetFileSecurityW, - }; - use windows_sys::Win32::System::IO::DeviceIoControl; - - /// Returns `text` as a NUL-terminated UTF-16 string. - fn wide(text: impl AsRef) -> Vec { - text.as_ref().encode_wide().chain([0]).collect() - } - - /// Replaces the access control list of the file or directory `path` with the protected list - /// of the descriptor `sddl`, in the security descriptor definition language. - pub(super) fn set_access(path: &Path, sddl: &str) { - let sddl = wide(sddl); - let name = wide(path); - let mut descriptor = ptr::null_mut(); - // SAFETY: the string is NUL-terminated, and the output pointer is valid. - let converted = unsafe { - ConvertStringSecurityDescriptorToSecurityDescriptorW( - sddl.as_ptr(), - SDDL_REVISION_1, - &mut descriptor, - ptr::null_mut(), - ) - }; - assert_ne!(converted, 0, "{}", io::Error::last_os_error()); - let information = DACL_SECURITY_INFORMATION | PROTECTED_DACL_SECURITY_INFORMATION; - // SAFETY: the name is NUL-terminated, and the descriptor is valid. - let set = unsafe { SetFileSecurityW(name.as_ptr(), information, descriptor) }; - let error = io::Error::last_os_error(); - // SAFETY: the conversion allocated the descriptor, which nothing uses any more. - unsafe { LocalFree(descriptor) }; - assert_ne!(set, 0, "{error}"); - } - - /// Marks the file that `file` holds open for writing sparse, which NTFS otherwise allocates - /// whole as the file is extended. - pub(super) fn set_sparse(file: &File) { - /// `FSCTL_SET_SPARSE`, in `winioctl.h`, which marks a file sparse without a buffer. - const FSCTL_SET_SPARSE: u32 = 0x0009_00c4; - let mut returned = 0; - // SAFETY: the handle is open for as long as `file` is borrowed, the control takes no - // input or output buffer, and the count's pointer is valid. - let set = unsafe { - DeviceIoControl( - file.as_raw_handle() as HANDLE, - FSCTL_SET_SPARSE, - ptr::null(), - 0, - ptr::null_mut(), - 0, - &mut returned, - ptr::null_mut(), - ) - }; - assert_ne!(set, 0, "{}", io::Error::last_os_error()); - } -} diff --git a/nvx-rs/tests/vectors/README.md b/nvx-rs/tests/vectors/README.md index b53ae6063..547866e3d 100644 --- a/nvx-rs/tests/vectors/README.md +++ b/nvx-rs/tests/vectors/README.md @@ -70,10 +70,10 @@ the key that the vector's bytes decode to. `CLASS|NETWORK` of `snapshot-key-v1.txt` came to encode its network in place as the template captures it, under the complete deny-all policy: the encodings of a key and - of a class have not been released, so a template store that an earlier build - created is not supported, and is deleted and created again, since nothing - migrates it. The snapshot key also drops the obsolete guest-build field and - uses the `nvx snapshot key` hash domain in place, and now appends the full + of a class have not been released, so no key or class that an earlier build + encoded is supported, since nothing migrates it. The snapshot key also drops + the obsolete guest-build field and uses the `nvx snapshot key` hash domain in + place, and now appends the full capture-time contract while its compatibility identity still digests only the preceding compatibility fields: no snapshot outlives a deployment, and the prior key encoding is not supported. @@ -98,15 +98,15 @@ of `openvmm/openvmm_helpers/src/snapshot/microvm.rs` that admits the kernel command-line tokens of a capture at OpenVMM's platform tier, and `microvm_control_tty_command_line.rs`, the constant of `openvmm/openvmm_defs/src/microvm.rs` that names the control tty's token, which -the function reads. The tests of `nvx::host::template` compile them, and check -each token of a template's kernel command line with OpenVMM's own function. The -third, `port_forward_socket_creation.rs`, is the variant of the error of -`vm/devices/net/net_consomme/src/resolver.rs` with which OpenVMM reports a port -forward whose socket on the host it cannot bind, as when a clone's VMM restores -its NIC with a forward whose port another socket holds. OpenVMM gives that -error no code, so the launch of a clone attributes it to the host by its text, -and the tests of `nvx::host::template` read the excerpt, and check that the -text is OpenVMM's. +the function reads. The tests of `nvx::host::config` compile them, and check +that its allowlist of a capture's kernel command-line tokens admits each token +as OpenVMM's own function does. The third, `port_forward_socket_creation.rs`, +is the variant of the error of `vm/devices/net/net_consomme/src/resolver.rs` +with which OpenVMM reports a port forward whose socket on the host it cannot +bind, as when a clone's VMM restores its NIC with a forward whose port another +socket holds. OpenVMM gives that error no code, so the launch of a clone, which +#1121 deleted, attributed it to the host by its text; only the comparison below +reads the excerpt now. The tests cannot call OpenVMM's broker or its allowlist: OpenVMM's workspace needs a later Rust release than `nvx`'s minimum, and the `nvx-rs` job cannot