restore: report the attribution of a restore that fails in its guest (SL5.1d)

doc/acceptance-case-matrix.md

@@ -228,7 +228,7 @@ The adapter's README said what each proved.
 | `a_clone_runs_the_executions_that_name_their_identity_once_it_is_ready` | retired | Executions that name their identities leave with the exec engine (#1087) |
 | `a_clone_s_executions_find_the_proxy_of_its_policy` | rewritten | `nvx-rs/src/guest/managed_agent/launch/tests.rs::restored_proxy_defaults_replace_template_values_before_exec_overrides`; `test-microvm --scenario platform-network-policy` |
 | `a_clone_s_time_settles_after_it_is_ready` | rewritten | `test-microvm --scenario time-host-correction`; `test-microvm --scenario restore-downtime` |
-| `a_clone_whose_activation_fails_ends_before_it_is_ready` | rewritten | `nvx-rs/src/host/restore/tests.rs::restore_failures_attribute_the_existing_policy_free_evidence_classes`; `nvx-rs/src/host/restore/tests.rs::restore_exit_reports_keep_guest_status_classification_and_actual_details`; `test-microvm --scenario snapshot-core`, whose restore over a zeroed scratch image ends its VM before its workload runs. Quarantine and leases retire (#1121) |
+| `a_clone_whose_activation_fails_ends_before_it_is_ready` | rewritten | `nvx-rs/src/host/restore/tests.rs::restore_failures_attribute_the_existing_policy_free_evidence_classes`; `nvx-rs/src/host/restore/tests.rs::restore_exit_reports_keep_guest_status_classification_and_actual_details`; `nvx-rs/src/guest/init/boot/tests.rs::a_failed_activation_ends_the_vm_with_the_status_of_its_class`; `nvx-rs/src/guest/managed_agent/session/tests.rs::a_failed_restored_activation_gives_init_the_status_of_its_class`; `nvx-rs/src/host/cli/sandbox/lifecycle/start/tests/restore.rs::a_restore_that_fails_before_its_guest_is_ready_reports_its_attribution`; `test-microvm --scenario snapshot-core`, whose restore over a zeroed scratch image ends its VM before its workload runs, with nvx-init's status of an unusable root, 54, which the CLI attributes to the request. Quarantine and leases retire (#1121) |
 | `a_clone_whose_mapping_fails_ends_before_its_sandbox_starts` | rewritten | `nvx-rs/src/guest/mappings/mapper/tests.rs::a_failure_at_any_step_leaves_nothing_mounted`; `nvx-rs/src/guest/mappings/table/tests.rs::a_failed_mapping_keeps_those_before_it_and_latches_the_table` |
 | `a_clone_whose_view_cannot_be_set_ends_before_it_is_ready` | rewritten | `nvx-rs/src/guest/activation/state/apply/tests.rs::the_record_s_view_is_applied_once_with_its_own_dns_and_returned_machine_id`; `nvx-rs/src/guest/root/tests/view.rs::a_later_bind_failure_stays_owned_and_its_teardown_removes_the_earlier_bind` |
 | `a_clone_whose_vm_exits_once_a_stop_found_it_fails_before_its_lease_is_released` | retired | Clone records and leases leave with the old clone path (#1121). A stop that finds an exit: `nvx-rs/src/host/vm/tests.rs::stop_preserves_report_results_after_a_real_self_exit` |
@@ -275,7 +275,7 @@ checked the adapter's own code, and CI ran in its configurations of the adapter.
 | `src/template/tests.rs` | `template_mode_refuses_a_gateway_that_openvmm_would_not_render` | retired | The adapter's template mode; a NIC's address is checked as `NicConfig` is built |
 | `src/template/tests.rs` | `template_mode_refuses_quotes_and_the_delimiter` | rewritten | `nvx-rs/src/host/config/tests.rs::cold_boot_command_lines_refuse_native_control_console_syntax` |
 | `src/template/tests.rs` | `a_template_s_statuses_are_its_own`, `a_template_s_repair_delay_is_positive_milliseconds_of_at_most_half_a_minute` | retired | The adapter's template statuses and its clones' repair delay |
-| `src/worker/activation/tests.rs` | `each_class_of_failure_ends_the_clone_with_its_own_status`, `the_activation_s_timeout_covers_the_wait_for_both_devices`, `the_activation_probe_reports_the_clone_s_events_after_the_acknowledgement` | retired | The adapter's guest worker. NVX's activation: `nvx-rs/src/guest/activation/state/tests.rs` and nvx-init's `nvx-rs/src/guest/init/tests/restore.rs` |
+| `src/worker/activation/tests.rs` | `each_class_of_failure_ends_the_clone_with_its_own_status`, `the_activation_s_timeout_covers_the_wait_for_both_devices`, `the_activation_probe_reports_the_clone_s_events_after_the_acknowledgement` | retired | The adapter's guest worker. NVX's activation: `nvx-rs/src/guest/activation/state/tests.rs` and nvx-init's `nvx-rs/src/guest/init/tests/restore.rs`, and nvx-init's status of each class of failure: `nvx-rs/src/guest/init/boot/tests.rs::a_failed_activation_ends_the_vm_with_the_status_of_its_class` |
 | `src/worker/capture/tests.rs` | `closing_the_fence_acquires_it_until_it_opens`, `each_stage_of_a_capture_ends_the_worker_with_a_status_of_its_own`, `a_request_that_the_time_abi_could_not_repair_ends_with_the_violation_s_status`, `a_template_s_devices_are_those_of_a_root_and_the_host_filesystem`, `a_clone_assembles_its_root_with_the_template_s_configuration`, `the_worker_acknowledges_the_broker_s_reset_before_the_capture_and_reads_nothing_more`, `no_capture_runs_without_the_session_s_handshake` | retired | The adapter's guest worker and its admission fence, which leaves (#1087). NVX's capture: `nvx-rs/src/guest/snapshot/tests.rs` and nvx-init's `nvx-rs/src/guest/init/tests/capture.rs` |
 | `src/worker/entries/tests.rs` | `capabilities_are_named_without_their_prefix`, `status_fields_are_found_by_name`, `outcomes_name_the_error`, `refusals_name_what_refused_the_identity`, `unknown_probes_and_arguments_are_refused`, `absolute_paths_lose_their_leading_slash` | retired | The adapter's guest worker |
 | `src/worker/executions/tests.rs` | `a_cancellation_after_the_timeout_leaves_a_program_that_ended_before_it_its_own_end` | retired | The adapter's guest worker, over the exec engine that leaves (#1087) |
@@ -398,14 +398,12 @@ capture`, whose VM nvx-init runs, and restore it with `nvx-rs snapshot restore`
 | --- | --- |
 | A real VM is captured, opened as a strict snapshot and restored with nvx-init | `snapshot-core`: two one-off restores and three managed ones, whose guests keep the snapshot's uptime, read the host's clock, and take fresh random values and machine IDs. `snapshot-tiers`: a managed restore of each image format, EROFS and GPT, over each upper, scratch and RAM |
 | Concurrent restores of one snapshot are serialized by the CLI's lock | `snapshot-core`: three restores at once, whose workloads end only when the scenario releases them. While one holds the lock, no other links its view of the snapshot, and a restore that cannot wait for the lock fails with `sandbox-busy` before its workload runs. The three then run at once, each with its own share, upper and machine ID |
-| The selected failed restore reports its attribution | `snapshot-core`: a scratch image that the snapshot cannot serve is refused before any VM starts, with an error that names the lengths of the requested scratch and of the snapshot's, which attributes the failure to the request; the managed sandbox of the refused restore holds its configuration and its lock alone, with no trace of a launch. A restore that fails in its guest ends its VM, frees the lock, and reports `guest-exit status 125` |
+| The selected failed restore reports its attribution | `snapshot-core`: a scratch image that the snapshot cannot serve is refused before any VM starts, with an error that names the lengths of the requested scratch and of the snapshot's, which attributes the failure to the request; the managed sandbox of the refused restore holds its configuration and its lock alone, with no trace of a launch. A restore over a zeroed scratch image fails in its guest, which cannot mount the image as its upper: its VM ends with nvx-init's status of an unusable root, 54, and frees the lock, and the CLI reports `guest-exit status 54` with the restore's evidence, `UnusableRoot`, which it attributes to the request |
 | A restore of a snapshot with a NIC installs a network policy of its own | `untiered-network-policy`: a snapshot that `snapshot capture --net` captures under the complete deny-all policy, restored four times with `snapshot restore` and `run --restore-snapshot`. Each guest names exactly its policy's DNS server and runs the announcer of its listeners only when its policy publishes them, reaches exactly the host services that its egress admits, and serves the host through a forward and through the port that its policy publishes, on host loopback alone or to the host's private networks. The CLI refuses a restore without `--network-profile`, and one with `--net`, before it provisions anything |
 
-The CLI prints no attribution of a restore that fails in its guest beyond the guest's exit
-status: nvx-init's status for each class of failure, and the CLI's report of the restore's
-attribution, are a follow-up on #1118. A snapshot that `snapshot capture` captures without
-`--net` has no NIC, so its restores take no network option; one captured with `--net` keeps its
-NIC, and each of its restores installs the network policy of its own options on it.
+A snapshot that `snapshot capture` captures without `--net` has no NIC, so its restores take no
+network option; one captured with `--net` keeps its NIC, and each of its restores installs the
+network policy of its own options on it.
 
 ## CI
 

doc/guest-agent-guidelines.md

@@ -430,7 +430,10 @@ gives the restore its network.
   console, then syncs and powers off with the status.
 - A failed activation, wholly or partly applied, never reports Ready: the init
   reports the failure and ends the VM. nvx-init writes it to the console,
-  after `NVX-INIT-ACTIVATION:`, and powers off with status 125.
+  after `NVX-INIT-ACTIVATION:`, and powers off with the status of the
+  failure's class, which `nvx::protocol::status` defines, such as 54,
+  `ACTIVATION_ROOT`, for a root that does not assemble over its upper, from
+  which the host attributes the failure.
 
 ## Replay
 

doc/run.md

@@ -1154,8 +1154,11 @@ and the view of the snapshot, until `sandbox deprovision` removes it. A
 `run --restore-snapshot` over a zeroed scratch image of the dormant scratch's
 length, which the guest cannot mount as its upper, ends its VM before its
 workload runs and frees the snapshot's lock, which the restores that follow
-take, and the run reports the guest's exit:
-`OpenVMM exited during startup: guest-exit status 125`.
+take: nvx-init ends the VM with the status of a root that does not assemble,
+54, and the run reports the guest's exit with the restore's evidence and what
+it attributes the failure to, the request, such as
+`OpenVMM exited during startup: guest-exit status 54; restore failed during
+Activate: UnusableRoot, attributed to the request`.
 
 Then three `snapshot restore` restores start at once, each with a host name
 and a read-only host share of its own, the last over a scratch upper and the

doc/usage.md

@@ -494,6 +494,16 @@ is ready never completes, and the next command of the sandbox ends its VM.
 [`run --restore-snapshot`](#run) restores a snapshot once, in a sandbox that it
 removes.
 
+A restore that fails before its guest is ready, in `snapshot restore`, a later
+`sandbox start`, or `run --restore-snapshot`, fails with the restore's evidence
+and what it attributes the failure to: the request, the host, the snapshot, the
+guest's repair, or nothing that it establishes. A guest that cannot activate
+the restore, such as one that cannot mount its scratch image as its upper, ends
+its VM with the status of the failure's class, which `nvx::protocol::status`
+defines, and the command reports the guest's exit with that evidence, such as
+`OpenVMM exited during startup: guest-exit status 54; restore failed during
+Activate: UnusableRoot, attributed to the request`.
+
 ### `download`
 
 ```text

nvx-rs/README.md

@@ -2729,7 +2729,7 @@ disk's block device.
 | `nvx-ingress [inherit\|start] GATEWAY_IPV4` | Announces reachable TCP listeners and settled UDP sockets to the portable-network gateway, from a privileged source port that it binds, or, with `inherit`, from its standard input, a socket that the caller bound to one after the clone's process-free application; `start` binds such a socket and starts `inherit` on it in a session of its own, then exits once its program runs, or with status 1 if it does not start, for a guest agent that cannot hold the announcer; `test DIRECTORY` runs synthetic procfs snapshots |
 | `nvx-time [--report-only] COMMAND` | Enforces NVX time ABI v1: runs its conformance checks, repairs the wall clock after a restore, publishes the time state, and runs the discipline and violation-watcher daemon; `test NAME ...` runs its decoder and process test hooks (x86_64) |
 | `nvx-time-probe latency\|warp\|clock ...` | Diagnoses clock-read latency, cross-CPU TSC warp and offset, the clocks, CPUID, clock events, kernel messages, and optionally MSRs (x86_64) |
-| `nvx-init TIME_DAEMON [ANNOUNCER]` | Takes PID 1 over from a layered root's early boot, with its time daemon and optional announcer: gives itself and them an out-of-memory score adjustment of -1000 and a nice value of -10, collects every child's status, and, after `sync`, ends the VM with status 125 when a helper ends or 0 on `SIGTERM`; elsewhere it exits with 2 (x86_64) |
+| `nvx-init TIME_DAEMON [ANNOUNCER]` | Takes PID 1 over from a layered root's early boot, with its time daemon and optional announcer: gives itself and them an out-of-memory score adjustment of -1000 and a nice value of -10, collects every child's status, and, after `sync`, ends the VM with status 125 when a helper ends, 0 on `SIGTERM`, or the status of a failed restored activation's class; elsewhere it exits with 2 (x86_64) |
 
 Each binary's command line, exit statuses, and byte output are compatibility
 contracts; the [`nvx::guest`](src/guest/mod.rs) documentation describes each
@@ -2741,7 +2741,11 @@ then completes and reports Ready. The host supplies the complete mapping table
 before the first valid EXEC starts the restored workload; later executions and
 byte-identical activation retries, also those that arrive while a workload
 runs, reuse the retained state, and an activation with other bytes ends the VM
-as a failure.
+as a failure. A failed activation, which a deadline that passes before Ready
+and a refused activate request also are, ends the VM with the status of the
+failure's class, which [`nvx::protocol::status`](#nvxprotocolstatus) defines,
+such as 54, `ACTIVATION_ROOT`, for a root that does not assemble over its
+upper, or 51, `ACTIVATION_PROTOCOL`, for an activation with other bytes.
 The accepted proxy replaces the four HTTP(S) proxy defaults, or clears them when
 the restore has no proxy. Per-EXEC layered entries still win, and exact
 environments still replace every default.

nvx-rs/src/guest/init.rs

@@ -30,8 +30,13 @@
 //!   nothing from it;
 //! - ends the VM itself, without another guest program: it writes every file system's modified
 //!   data to its storage, with `sync`, and then stops the VM through OpenVMM's control port with a
-//!   status: 0 after a host `STOP` or `SIGTERM`, or 125, as `nvx-init-agent`'s failures do, after
-//!   a failure, which an `NVX-INIT-ERROR:` line on the console names first.
+//!   status: 0 after a host `STOP` or `SIGTERM`; after its restored activation failed, which a
+//!   deadline that passed before Ready and an activate request that it refused also are, the
+//!   status of `nvx::protocol::status` of the failure's class, such as 54, `ACTIVATION_ROOT`, for
+//!   a root that does not assemble over its upper, or 51, `ACTIVATION_PROTOCOL`, for an activate
+//!   request that changes the accepted one, which an `NVX-INIT-ACTIVATION:` line on the console
+//!   names first; or 125, as `nvx-init-agent`'s failures do, after another failure, which an
+//!   `NVX-INIT-ERROR:` line on the console names first.
 //!
 //! Outside PID 1, `nvx-init` changes nothing: it fails with status 2 and a message.
 
@@ -62,8 +67,8 @@ pub(crate) use boot::{RestoreHandoff, Restored};
 
 /// The process ID of the init process.
 const INIT_PROCESS: i32 = 1;
-/// The VM's exit status after a failure of PID 1 or the end of a helper: that of
-/// `nvx-init-agent`'s failures.
+/// The VM's exit status after a failure of PID 1 or the end of a helper, but for a failed restored
+/// activation's: that of `nvx-init-agent`'s failures.
 const FAILURE: u8 = 125;
 /// The out-of-memory score adjustment of PID 1 and its helpers, Linux's `OOM_SCORE_ADJ_MIN`,
 /// which exempts them from the out-of-memory killer.