Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/run-nvx-microvm-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,7 @@ jobs:
--scenario vm-lifecycle
--scenario snapshot-core
--scenario snapshot-tiers
--scenario untiered-network-policy
--output-dir "build/test-results/public-exec-${{ inputs.backend }}"

- name: Run public managed sandbox acceptance on Windows
Expand All @@ -238,6 +239,7 @@ jobs:
--scenario vm-lifecycle
--scenario snapshot-core
--scenario snapshot-tiers
--scenario untiered-network-policy
--output-dir "build/test-results/public-exec-${{ inputs.backend }}"

- name: Run Ubuntu guest smoke tests on Linux
Expand Down
27 changes: 15 additions & 12 deletions doc/acceptance-case-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,9 @@ vm-lifecycle` creates, spawns, reattaches to and stops a VM through the public A
as [Cold boots](#cold-boots) lists. SL5.1c (#1118) added those of snapshots, which capture a
snapshot, open it and restore it with nvx-init as the guest, run concurrent restores of one
snapshot under the CLI's lock, and check that a failed restore reports its attribution, as
[Snapshots](#snapshots) lists.
[Snapshots](#snapshots) lists. SL3.7c (#1114) let a named snapshot keep a NIC, and runs the
network policies of its restores on real VMs with
`test-microvm --scenario untiered-network-policy`, as [Snapshots](#snapshots) lists too.

The inventory is fixed: it lists the tests as they are where this matrix was added, from
`tests/runtime-adapter/scenarios.txt` and the adapter's sources, `MICROVM_TEST_SCENARIOS` and
Expand All @@ -30,7 +32,7 @@ its own in turn.
| --- | ---: | ---: | ---: | ---: |
| [The adapter's scenarios](#the-adapters-scenarios) | 157 | 0 | 114 | 43 |
| [The adapter's other tests](#the-adapters-other-tests) | 169 | 0 | 9 | 160 |
| [MicroVM scenarios](#microvm-scenarios) | 54 | 51 | 2 | 1 |
| [MicroVM scenarios](#microvm-scenarios) | 54 | 50 | 3 | 1 |
| [CI](#ci) | 69 | 43 | 3 | 23 |
| [Docs](#docs) | 36 | 28 | 2 | 6 |

Expand Down Expand Up @@ -212,13 +214,13 @@ The adapter's README said what each proved.
| `a_removal_reports_busy_while_leases_keep_the_generation` | retired | Removals, leases, tombstones and generations leave with the old clone path (#1121) |
| `a_template_whose_processors_are_all_online_at_boot_limits_none` | rewritten | `nvx-rs/src/host/snapshot/tests.rs::snapshot_accepts_treats_zero_boot_processors_as_all_capacity_online`; `nvx-rs/src/host/config/tests.rs::fresh_captures_pass_with_supported_processors_and_aligned_memory`; `test-microvm --scenario smp-snapshot` |
| `a_template_without_a_network_has_no_nic` | rewritten | `nvx-rs/src/host/vm/tests.rs::capture_creation_records_sparse_disks_devices_and_a_complete_serde_context`; `nvx-rs/src/host/snapshot/tests.rs::snapshot_accepts_refuses_both_nic_presence_mismatches_and_each_named_address_mismatch`. A store that serves it again retires (#1121) |
| `the_networks_of_every_policy_at_one_guest_s_address_share_one_template` | rewritten | `nvx-rs/src/host/network/tests.rs::templates_attach_their_nic_under_the_complete_deny_all_policy`; `test-microvm --scenario untiered-network-policy`, which SL3.7c (#1114) rewrites on a strict snapshot with a NIC. A cached template retires (#1121) |
| `the_networks_of_every_policy_at_one_guest_s_address_share_one_template` | rewritten | `nvx-rs/src/host/network/tests.rs::templates_attach_their_nic_under_the_complete_deny_all_policy`; `test-microvm --scenario untiered-network-policy`, whose four restores of one named snapshot with a NIC each install a network policy of their own. A cached template retires (#1121) |

### `vm_clone` (36)

| Scenario | Fate | Coverage, or why it retires |
| --- | --- | --- |
| `a_clone_binds_the_resolver_of_its_policy_s_dns_server_read_only` | rewritten | `test-microvm --scenario untiered-network-policy`, which SL3.7c (#1114) rewrites on a strict snapshot with a NIC; `test-microvm --scenario network-snapshot`; `nvx-rs/src/guest/root/tests/view.rs::actual_resolver_failures_keep_the_network_class_and_root_owned_bind` |
| `a_clone_binds_the_resolver_of_its_policy_s_dns_server_read_only` | rewritten | `test-microvm --scenario untiered-network-policy`, whose restored guests' resolvers name exactly the DNS server of their policies; `test-microvm --scenario network-snapshot`; `nvx-rs/src/guest/root/tests/view.rs::actual_resolver_failures_keep_the_network_class_and_root_owned_bind` |
| `a_clone_maps_an_aggregate_of_directories_and_files_beneath_its_root_before_its_sandbox_starts` | rewritten | `test-microvm --scenario platform-aggregate-export`; `nvx-rs/src/guest/mappings/table/tests.rs::mappings_keep_the_paths_and_the_access_of_their_entries` |
| `a_clone_of_a_template_with_a_nic_maps_its_export_beside_its_resolver` | rewritten | `nvx-rs/src/guest/activation/state/apply/tests.rs::the_record_s_view_is_applied_once_with_its_own_dns_and_returned_machine_id`; `test-microvm --scenario filesystem-snapshot` |
| `a_clone_reads_no_more_of_its_disks_than_its_root_needs` | rewritten | A restore reads no content of its disks or snapshot: `nvx-rs/src/host/snapshot/tests.rs::snapshot_accepts_never_reopens_snapshot_paths_or_reads_runtime_artifacts`; `nvx-rs/src/host/sandbox/distro/tests.rs::the_staged_check_reads_no_content`; `nvx-rs/src/host/sandbox/scratch/tests.rs::checks_claim_the_image_without_reading_it`. The adapter's in-guest read statistics retire |
Expand All @@ -234,7 +236,7 @@ The adapter's README said what each proved.
| `a_generation_is_eligible_by_the_clock_of_its_capture` | rewritten | `nvx-rs/src/host/snapshot/eligibility/tests.rs::within_one_boot_the_downtime_is_the_host_monotonic_clocks`; `nvx-rs/src/host/snapshot/tests.rs::snapshot_accepts_reports_early_and_expired_time_as_unavailable` |
| `a_generation_refuses_clones_of_fewer_processors_than_its_template_booted_with` | rewritten | `nvx-rs/src/host/snapshot/tests.rs::snapshot_accepts_refuses_a_blockless_root_and_each_processor_mismatch`; `test-microvm --scenario restore-processors` |
| `a_host_program_of_another_runtime_ends_the_clone_through_its_verified_identity` | rewritten | `nvx-rs/tests/security_checklist.rs::a_saved_vm_runs_until_its_process_exits_and_another_runtime_is_refused_without_a_signal`; `nvx-rs/src/host/vm/tests.rs::stop_forces_only_its_verified_process_after_grace`. Leases retire (#1121) |
| `a_run_in_a_template_of_a_network_finds_its_policy_s_proxy` | rewritten | `test-microvm --scenario platform-network-policy`; `test-microvm --scenario snapshot-core`, whose runs restore a snapshot without a NIC until SL3.7c (#1114). The template's `run` retires (#1121) |
| `a_run_in_a_template_of_a_network_finds_its_policy_s_proxy` | rewritten | `test-microvm --scenario platform-network-policy`; `test-microvm --scenario untiered-network-policy`, whose runs restore a named snapshot with a NIC under a network policy of their own, though with no proxy. The template's `run` retires (#1121) |
| `a_run_runs_one_execution_in_its_clone_then_tears_the_clone_down` | rewritten | `test-microvm --scenario snapshot-core`, whose `run --restore-snapshot` runs one workload in its restored VM and removes its sandbox. The template's `run` and leases retire (#1121) |
| `a_run_s_execution_finds_its_shares_beneath_the_root` | rewritten | `test-microvm --scenario platform-aggregate-export` |
| `a_run_whose_clone_does_not_start_runs_nothing` | retired | Template selections and the template's `run` leave with the old clone path (#1121) |
Expand All @@ -250,7 +252,7 @@ The adapter's README said what each proved.
| `one_generation_serves_clones_of_each_processor_count_from_its_boot_online_count_to_its_capacity` | rewritten | `test-microvm --scenario restore-processors`; `test-microvm --scenario platform-restore-processors` |
| `one_generation_serves_clones_of_each_upper_and_memory_target` | rewritten | `test-microvm --scenario scratch-snapshot`; `test-microvm --scenario restore-memory`. Leases retire (#1121) |
| `one_generation_serves_clones_of_either_format_over_either_upper` | rewritten | `test-microvm --scenario snapshot-tiers`; `nvx-rs/src/host/snapshot/tests.rs::snapshot_accepts_matching_real_disk_geometry_and_every_resource_boundary` |
| `one_template_serves_the_clones_of_every_network_policy_at_its_guest_s_address` | rewritten | `test-microvm --scenario untiered-network-policy`, which SL3.7c (#1114) rewrites on a strict snapshot with a NIC; `test-microvm --scenario platform-network-policy` |
| `one_template_serves_the_clones_of_every_network_policy_at_its_guest_s_address` | rewritten | `test-microvm --scenario untiered-network-policy`, whose four restores of one named snapshot with a NIC each install a network policy of their own at the NIC's address; `test-microvm --scenario platform-network-policy` |
| `openvmm_reads_none_of_the_host_s_input_until_the_repair_acknowledges_the_restore` | retired | It watches OpenVMM's own gating of the host's input through the repair delay of the adapter's templates, which leaves with them. NVX's side stays: `nvx-rs/src/guest/activation/state/tests.rs::the_deadline_starts_at_the_acknowledgement_and_retries_never_extend_it`; `nvx-rs/src/guest/init/boot/tests.rs::restored_sessions_keep_the_deadline_of_the_actual_repair_acknowledgement` |
| `restore_exclusion::hard_linked_restore_views_obey_the_callers_exclusion` (linux) | rewritten | `test-microvm --scenario snapshot-core`, whose concurrent restores of one snapshot take turns at the CLI's lock and leave no link to its files. The view that a restore hard-links, which `vm::create` documents: `nvx-rs/src/host/snapshot/tests.rs::a_restore_view_hard_links_only_the_fixed_openvmm_inventory`; `nvx-rs/src/host/vm/tests.rs::snapshot_creation_owns_the_fixed_view_and_uses_captured_nic_with_requested_policy`; `nvx-rs/src/host/vm/tests.rs::snapshot_creation_refuses_a_true_cross_filesystem_view_without_copying`; `nvx-rs/src/host/vm/tests.rs::snapshot_creation_never_replaces_existing_outputs_views_or_listeners`; `nvx-rs/src/host/vm/tests.rs::the_same_prebound_listener_keeps_its_queue_through_both_launch_paths_without_inheritance`. OpenVMM's own checks before its restore-ready connection, which fail a restore whose view changed or whose caller died, and which `vm::create` documents too, leave NVX's tests |
| `the_lease_of_a_start_that_ended_before_its_record_is_released_through_its_template_store` | retired | Leases and clone records leave with the old clone path (#1121) |
Expand Down Expand Up @@ -298,7 +300,8 @@ checked the adapter's own code, and CI ran in its configurations of the adapter.
suite in its shards, and the public sandbox scenarios in its guest and sandbox jobs. The public
`sandbox-gpt-layer` and `vm-lifecycle` scenarios came with SL5.1b, after this matrix;
[Cold boots](#cold-boots) says what they check. SL5.1c rewrote `snapshot-core` and
`snapshot-tiers` as public scenarios; [Snapshots](#snapshots) says what they check.
`snapshot-tiers` as public scenarios, and SL3.7c `untiered-network-policy`;
[Snapshots](#snapshots) says what they check.

| Scenario | Fate | Note |
| --- | --- | --- |
Expand Down Expand Up @@ -351,7 +354,7 @@ suite in its shards, and the public sandbox scenarios in its guest and sandbox j
| `structured-outcome` | kept | |
| `time-abi-conformance` | kept | |
| `time-host-correction` | kept | |
| `untiered-network-policy` | kept | On a raw snapshot, until a strict snapshot can carry a NIC: SL3.7c (#1114) rewrites it on one. It keeps the coverage of a restore's DNS, egress, forwards and listener publication until then |
| `untiered-network-policy` | rewritten | On named snapshots with a NIC, which `snapshot capture --net` captures, as [Snapshots](#snapshots) says, keeping its checks of a restore's DNS, egress, forwards and listener publication. Its raw-snapshot parts retire: the raw capture and restores, the guest's repair marker, the console release, and OpenVMM's refusals of raw restores that the CLI cannot issue, in whose place the CLI must refuse a restore without `--network-profile`, and one with `--net` |
| `virtio-features` | kept | |
| `virtio-net` | kept | |
| `vm-lifecycle` | kept | A [cold boot](#cold-boots) through the public API: `nvx-rs/tests/vm_lifecycle.rs::a_cold_vm_is_reattached_from_saved_values_and_stopped` |
Expand Down Expand Up @@ -396,13 +399,13 @@ capture`, whose VM nvx-init runs, and restore it with `nvx-rs snapshot restore`
| A real VM is captured, opened as a strict snapshot and restored with nvx-init | `snapshot-core`: two one-off restores and three managed ones, whose guests keep the snapshot's uptime, read the host's clock, and take fresh random values and machine IDs. `snapshot-tiers`: a managed restore of each image format, EROFS and GPT, over each upper, scratch and RAM |
| Concurrent restores of one snapshot are serialized by the CLI's lock | `snapshot-core`: three restores at once, whose workloads end only when the scenario releases them. While one holds the lock, no other links its view of the snapshot, and a restore that cannot wait for the lock fails with `sandbox-busy` before its workload runs. The three then run at once, each with its own share, upper and machine ID |
| The selected failed restore reports its attribution | `snapshot-core`: a scratch image that the snapshot cannot serve is refused before any VM starts, with an error that names the lengths of the requested scratch and of the snapshot's, which attributes the failure to the request; the managed sandbox of the refused restore holds its configuration and its lock alone, with no trace of a launch. A restore that fails in its guest ends its VM, frees the lock, and reports `guest-exit status 125` |
| A restore of a snapshot with a NIC installs a network policy of its own | `untiered-network-policy`: a snapshot that `snapshot capture --net` captures under the complete deny-all policy, restored four times with `snapshot restore` and `run --restore-snapshot`. Each guest names exactly its policy's DNS server and runs the announcer of its listeners only when its policy publishes them, reaches exactly the host services that its egress admits, and serves the host through a forward and through the port that its policy publishes, on host loopback alone or to the host's private networks. The CLI refuses a restore without `--network-profile`, and one with `--net`, before it provisions anything |

The CLI prints no attribution of a restore that fails in its guest beyond the guest's exit
status: nvx-init's status for each class of failure, and the CLI's report of the restore's
attribution, are a follow-up on #1118. A snapshot that `snapshot capture` captures has no NIC, so
its restores take no network option: `untiered-network-policy` keeps the coverage of a restore's
DNS, egress, forwards and listener publication on raw snapshots until SL3.7c (#1114) lets a
capture keep a NIC.
attribution, are a follow-up on #1118. A snapshot that `snapshot capture` captures without
`--net` has no NIC, so its restores take no network option; one captured with `--net` keeps its
NIC, and each of its restores installs the network policy of its own options on it.

## CI

Expand Down
57 changes: 34 additions & 23 deletions doc/ci.md
Original file line number Diff line number Diff line change
Expand Up @@ -275,28 +275,36 @@ forward whose host port another socket holds, the option on a platform
snapshot captured with plain deny-all egress, which leaves host loopback
allowed, and the option on a cold boot, and no restore may change the
template.
The `untiered-network-policy` scenario covers a NIC template without a tier, a
snapshot without sandbox blocks, as the non-sandbox mode captures one, whose
NIC was captured with the complete deny-all policy. It restores the template
with `--restore-network-policy` four times: under the template's own deny-all,
under egress rules that admit DNS over IPv4 and one host service, with
explicit host loopback and a forward under egress rules that admit DNS over
IPv6 alone, and with private-network ingress and allowed egress. The
template's guest learned no DNS server and runs no announcer of its listeners,
so each restore's repair applies those of the restore's policy, which the
restore packet's network settings carry, before it acknowledges the restore:
the guest's resolver must name exactly the policy's DNS server, and no gateway
without one, and the announcer must run exactly when the policy publishes
the guest's listeners. Each restore must reach through its gateway exactly the
host-loopback services that its policy admits, which must receive no other
request, and the host must reach the guest's server through the forward and
through the port that the policy publishes: on host loopback alone, which no
other host address reaches, or to the host's private networks, through a host
address other than loopback. It must find the server on no host loopback port
without a publication. Before the guest runs, OpenVMM must refuse a restore
that keeps the template's policy but passes another, and one that passes
private-network ingress without the option, and no restore may change the
template.
The `untiered-network-policy` scenario, one of the public sandbox scenarios
that the guest and sandbox jobs run (see below), covers a NIC template without
a tier: a named snapshot that `nvx-rs snapshot capture --net` captures, of a
blockless VM whose NIC has the complete deny-all policy. It restores the
template four times, with the network options of a sandbox of the Ubuntu
layer over a RAM upper, at the NIC's captured address, as the restore's own
policy, and runs the scenario's checks as the restored workload: through
`nvx-rs snapshot restore`, under the template's own deny-all and with
private-network ingress and allowed egress, and through
`nvx-rs run --restore-snapshot`, under egress rules that admit DNS over IPv4
and one host service, and with explicit host loopback and a forward under
egress rules that admit DNS over IPv6 alone. The template's guest learned no
DNS server and runs no announcer of its listeners, so each restore's
activation applies those of the restore's policy: the guest's resolver must
name exactly the policy's DNS server, and no gateway without one, and the
announcer must run exactly when the policy publishes the guest's listeners.
Each restore must reach through its gateway exactly the host-loopback services
that its policy admits, which must receive no other request, and the host must
reach the guest's server through the forward and through the port that the
policy publishes: on host loopback alone, which no other host address reaches,
or to the host's private networks, through a host address other than loopback.
It must find the server on no host loopback port without a publication, while
the guest holds the server's port, which the host cannot release: it must
check within two seconds of seeing the server listen, and see the six-second
hold end at least two seconds after its check, so that a check that a late
marker could have moved past the hold fails instead of passing. Before it
provisions anything, the CLI must refuse a restore of the template without
`--network-profile`, and one that names the NIC's address with `--net`. No
restore may change the template, or leave its sandbox or an OpenVMM process
behind; a failed run keeps the VM log of each sandbox that it leaves.
The `platform-filesystem` scenario covers a share that a platform restore
attaches to the filesystem slot that the capture left dormant, as a clone's
export does. It captures a platform snapshot without `--mount` and restores it
Expand Down Expand Up @@ -565,7 +573,10 @@ a VM through the public API of `nvx`. The public `snapshot-core` and
[Run](run.md#named-snapshots)): restores of each image format over each upper,
repeated restores with fresh entropy and machine IDs, clocks that follow the
host's across the downtime, concurrent restores that take turns at the
snapshot's lock, failed restores, and snapshots that no restore changes.
snapshot's lock, failed restores, and snapshots that no restore changes. The
public `untiered-network-policy` scenario restores a named snapshot with a NIC,
which `snapshot capture --net` captures, under four network policies, as
described above.
Their bounded evidence is part of the backend's failure logs. The same
entrypoint then verifies a live virtio-fs
share inside the container: a read-write `/workspace` share with a denied
Expand Down
Loading
Loading