Skip to content

doc: describe the final runtime commands and fresh state on upgrade (SL6.10d) - #1556

Merged
Pedro Henrique Penna (ppenna) merged 2 commits into
feature/nvx-refactorfrom
agents/slim-sl6-10d-cli-reference
Nov 13, 2026
Merged

Pedro Henrique Penna (ppenna) merged 2 commits into
feature/nvx-refactorfrom
agents/slim-sl6-10d-cli-reference

Conversation

@ppenna

@ppenna Pedro Henrique Penna (ppenna) commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Part of #1131 (SL6.10d, "Runtime users have one CLI reference"). Parent #1090, umbrella #1076.

What and why

doc/run.md and the nvx-rs chapter of doc/usage.md now describe the final run, sandbox, snapshot capture and snapshot restore commands as the code on this base runs them, and present no removed raw-restore, buffered-exec or upgrade-compatibility behavior as supported. It changes only those two documents, and only their runtime sections.

The final commands (doc: describe the final run, sandbox, and snapshot commands):

  • run's cold boot. The docs said that --mount passes through unchanged, with an aggregate at /run/nvx/shares only for several mappings, named by a digest of their targets; that --cmdline reserves every nvx_* and tsc= token; and that the terminal attaches with --virtio-console console. nvx-rs run now exports the outermost mapped directories, and the mapped files outside them, as the children of one aggregate at /run/nvx/hostfs/root, numbered from 0, and binds each --mount from its child with an nvx_share=SOURCE,TARGET,MODE[,file] token whose source may be a path inside the child, such as 0/single.txt, refusing only a read-only mapping inside a writable part; resolves policy paths when it plans them, as sandbox does; refuses only the tokens that NVX renders itself (nvx_hostname, nvx_machine_id, nvx_maps, the root's, virtfs_*), linking design 5.1.2, Effective Command Line; and connects the terminal to OpenVMM's console listener. run.md's translation table and virtio-fs section say so.
  • Raw restores. run --restore-snapshot names only a snapshot of its store, so it refuses a raw OpenVMM snapshot without reading it. The text that described restoring raw snapshots through nvx-rs run (the dormant slot's single --mount and its explicit mount in the guest, --guest ubuntu at restore, the snapshot rules of mappings) is gone; run's network options no longer say that "snapshot capture rejects" what a restore installs. OpenVMM's own restore instructions stay in Run OpenVMM directly.
  • A restore's own network policy, explained once in usage.md's snapshot restore and linked from run, run.md's intro, Networked snapshots and Proxy-only networking: OpenVMM installs its egress, ingress, host-loopback access and forwards, binding their ports, before any vCPU runs; the guest applies the rest as it activates, before its Ready, so before the host sends the mapping table and starts the workload: the DNS server that a cold boot under the policy would name, in /etc/resolv.conf; the announcer of its listeners when the policy publishes them; and a proxy-only policy's proxy as http_proxy, https_proxy, HTTP_PROXY and HTTPS_PROXY in the workloads' default environment, which a cold boot leaves to the caller. A failure ends the VM and fails the restore. Named snapshots now describes untiered-network-policy, which accepts it, in place of saying that captures have no NIC.
  • Streamed execs. An exec and the workload of a sandbox run stream stdin, stdout and stderr without a size limit; the null device gives no input; the workload must end within --timeout, which bounds the whole exchange, or the command fails with timed-out; Ctrl-C asks the guest to end the workload (CANCEL) and exits with 130 without waiting. A restore's workload reads no standard input. sandbox run no longer "prints the workload's standard output and error" at the end, and sandbox-lifecycle's 2 MiB request no longer cites NVXC version 1.
  • The exec lock: held until the competing exec command ends, which after Ctrl-C or --timeout may precede the workload's end; --wait waits for its release. sandbox-busy also covers the locks of a snapshot and of a store's captures.
  • Snapshot restore readiness: OpenVMM writes OPENVMM_RESTORE_READY_V1 once it has started the VM's workers and before it releases a restored vCPU, so before the guest's repair, which onlines a processor or memory target while a gated restore keeps host input gated until its acknowledgement. The section had the order of OpenVMM's Guide, whose two passages on it are wrong (OpenVMM's start_state_units at the pin, and design 5.5.1's steps 20 to 22).
  • The RAM upper's cap: a restore takes --upper and --ram-upper-percent as provision does and hands the cap to its guest in its activation; OpenVMM's refusal of nvx_overlay_upper=ramfs concerns a cold boot's token, while a restore over a RAM upper uses --restore-absent-scratch.

Fresh state on upgrade (doc: tell runtime users to upgrade with fresh state): run.md gains ## Upgrades (anchor run.md#upgrades, for SL6.10h's migration guide to link): NVX keeps and migrates no state across versions; a version may refuse another's configuration with format-unsupported or fail to read its records; a snapshot restores only under the runtime identity that captured it. To upgrade: stop and deprovision each sandbox with the version that started it, switch, and capture the snapshots again into new stores. The scattered compatibility notes (configurations "as older releases wrote", memory that "earlier releases" recorded, the proxy section's list of what "earlier releases" did, the store's version advice, and the old NVX commit that restored snapshots of an earlier OpenVMM as templates) now state current refusals or link to it, and usage.md links to it from the nvx-rs chapter, --ram-upper-percent and snapshot restore.

On "before the host releases its input". In a CLI restore, which restores a snapshot of the platform tier, the guest applies the policy's DNS server and starts its announcer during the activation, before Ready (guest::init::boot's restore order; host::restore's live test), and the host sends MAPS and Start only after Ready (host::restore_driver). OpenVMM's own input gate of a tiered restore lifts earlier, at the repair's acknowledgement, before the activation, so the docs say "before Ready, so before the host sends the mapping table and starts the workload" rather than claim the guest acts before OpenVMM releases its input.

Coordination. Written against 55f44aa4607, and rebased at its merge turn onto 6975b008fe1, which holds the host chain (SL6.6c, SL6.7b, SL6.6d, SL6.5a2, SL6.5b and SL6.8c) and the docs of SL6.10f, SL6.10e, SL6.10c, SL6.10b and SL6.10a. The rebase met no conflict: SL6.8c's deletion of the three error-code rows and SL6.5b's "format 10" paragraph of Live host shares stay as they merged, and git range-diff shows no change in this PR's two commits. It links only anchors that exist on this base, and adds none to the migration guide, which SL6.10h writes.

Noticed, out of scope. run --cmdline passes nvx_mode and nvx_identity on to the blockless guest, as design 5.1.2, Effective Command Line, describes for launchers; the reference doesn't document them as options. host::sandbox::mount::AGGREGATE_MOUNT_TARGET still reserves /run/nvx/shares for cold targets, which /run's reservation covers.

Size

Core lines Counted files
Estimate (#1131) 0 (docs) none
Measured 0 0

Only Markdown changes: doc/run.md and doc/usage.md.

Old tests of the area

None changes: the PR changes no test, CI step or doc test, and the case matrix keeps every row.

Test changes

python scripts\check_test_changes.py --base origin/feature/nvx-refactor: 0 change(s) to existing tests, so no tests-modified.

Validation

local_gates.py --files doc/run.md doc/usage.md (from dev, since this branch has no copy) selects test-changes, markdown-links, test-layout and nvx-cli (validate-nvx). I ran them from this branch's definitions on Windows. Every claim was checked against the code and the help and dry-run output of nvx-rs built from this branch with cargo build --release --locked --features cli --manifest-path nvx-rs/Cargo.toml, and a code-review subagent checked the diff the same way: its one finding, that the exec lock is released when the competing command ends rather than when its workload does, is fixed in the first commit. So are the two findings since: Copilot's, that the cold run groups nested and repeated --mount host paths into one child, which the virtio-fs section, the translation table and usage.md's run --mount row now describe, as run --dry-run confirms; and the restore-ready order, which the readiness section had taken from OpenVMM's Guide. After them, the link check passed at each commit, and the usage doc tests passed again. On 6975b008fe1, the claims were checked again: the --help of every command of nvx-rs built from the rebased branch is byte for byte that of the old base, the run --dry-run mount cases and a policy path through a junction plan as the docs say, and the base's new code in the documented areas, the export's pins, the CLI-private planner, policy reader and mount syntax, and the deleted error codes, changes no documented behavior.

Check Result
python scripts\nvx.py verify passed
python scripts\check_markdown_links.py passed, at each commit
python scripts\check_test_layout.py passed
python scripts\check_test_changes.py --base origin/feature/nvx-refactor 0 changes
SL5.2's check_matrix.py <worktree> 99d2022085f passed
The doc tests of usage.md with NVX_RS set to the built nvx-rs.exe: python -m unittest -k test_usage scripts/test_nvx_tools.py (test_usage_documents_every_option_of_nvx_rs, ..._every_error_code_of_nvx_rs, ..._every_command_and_option, test_usage_states_each_guest_memory_default) passed, at each commit
validate-nvx's Windows commands, with NVX_RS set: compileall, its unittest list, test_adversarial.py, test_hosts.py and nvx.py's --help checks passed on 6975b008fe1: 1803 tests, 70 skipped; 84, 11 skipped; 4; all --help
nvx-rs run, sandbox, snapshot capture and snapshot restore --help, and run --dry-run with placeholder artifacts, --mount, mount policies, --cmdline probes and network options match the documented options, OpenVMM arguments and refusals

No microVM run: the change is documentation only, and the runtime orders that it documents come from the code and its tests.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The mount documentation conflates export children with guest mappings and needs to clarify outermost-root grouping, child-relative sources, accepted nested paths, and the scope of cross-child and standalone-file guarantees.

1 open issue
What changed in this PR

This documentation-only PR advances #1131 by aligning NVX’s runtime guide and CLI reference with the final commands and fresh-state upgrade model.

Changes:

  • Updates cold-boot and named-snapshot workflows, raw-restore refusal, and restore-specific networking.
  • Clarifies streamed execution, timeouts, exec locking, and RAM-upper configuration.
  • Consolidates upgrade guidance around stopping existing sandboxes and recreating state with the new version.
File Description
doc/​usage.md Updates runtime command semantics and links to fresh-state upgrade guidance.
doc/​run.md Revises runtime workflows, host-share descriptions, snapshot behavior, and upgrade steps.

Review effort: Balanced

Comment thread doc/run.md Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the documentation-only updates to runtime commands and fresh-state upgrade guidance.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

doc/run.md and the nvx-rs chapter of doc/usage.md still described parts
of the runtime CLI that the workstream replaced:

- run's cold boot passed --mount through unchanged, with an aggregate
  at /run/nvx/shares only for several mappings, named by a digest of
  their targets, refused host paths that equal or contain one another,
  and passed policy paths to OpenVMM unresolved; reserved every nvx_*
  and tsc= token of --cmdline; and attached the terminal with
  --virtio-console console.
- run's restores still read as raw ones: --restore-snapshot "no longer"
  took a raw snapshot, run's network options said that snapshot capture
  rejects what a restore now installs, and the virtio-fs section
  described restoring a raw snapshot through nvx-rs run.
- A snapshot that snapshot capture captures "has no NIC", and the raw
  untiered-network-policy scenario kept the coverage of a restore's
  DNS and listeners.
- sandbox run "prints the workload's standard output and error" once
  the workload has ended.
- Snapshot restore readiness had OpenVMM write its restore-ready event
  after the guest's repair and the release of host input, and the guest
  online its processors and memory before the event, as OpenVMM's Guide
  says.

The documents now describe what the commands do:

- A cold run exports the outermost mapped directories, and the mapped
  files outside them, as the children of one aggregate at
  /run/nvx/hostfs/root, numbered from 0, and binds each --mount from
  its child with an nvx_share= token, whose source may be a path inside
  the child, such as 0/single.txt. It refuses only a read-only mapping
  inside a writable part, and resolves policy paths as sandbox does. It
  refuses only the kernel tokens that NVX renders itself, and connects
  the terminal to OpenVMM's console listener. nvx-rs snapshots no VM
  with mappings.
- run --restore-snapshot names only a snapshot of its store, so it
  refuses a raw snapshot without reading it, and its workload, like
  that of every restore, reads no standard input.
- A restore of a snapshot with a NIC installs its network options as
  its own policy: OpenVMM its egress, ingress, host-loopback access and
  forwards before any vCPU runs, and the guest, as it activates and
  before its Ready, so before the host sends the mapping table and
  starts the workload, its DNS server, the announcer of its listeners,
  and a proxy-only policy's proxy in its workloads' default
  environment. usage.md's snapshot restore holds that explanation, and
  run.md links to it; Named snapshots describes the scenario that
  accepts it.
- An exec, and the workload of a sandbox run, stream their standard
  input, output, and error without a limit on their size, and must end
  within --timeout, which bounds the whole exchange. Ctrl-C asks the
  guest to end the workload. The exec lock serializes execs until the
  competing command ends, and the locks of a snapshot and of a store's
  captures fail with sandbox-busy too.
- A restore hands its guest the RAM upper's cap in its activation,
  since the snapshot keeps the kernel command line of its capture.
- OpenVMM writes the restore-ready event once it has started the VM's
  workers and before it releases a restored vCPU, so before the guest's
  repair, which onlines the requested processors and memory while a
  gated restore keeps host input gated, until its acknowledgement.

Evidence: nvx-rs run --dry-run, with placeholder artifacts, prints
--mount-aggregate /run/nvx/hostfs/root and --virtio-console
listen=ENDPOINT. Two --mount of one directory, or of a directory and a
file in it, give one --mount-child and nvx_share=0 and 0/single.txt
sources; a read-write --mount inside a read-only one adds --mount-write;
a read-only one inside a read-write one is refused unless --mount-write
leaves it outside the writable parts; and a --mount-deny through a
junction reaches OpenVMM resolved. run accepts --cmdline tsc=reliable
and nvx_foo=1 and refuses nvx_hostname=, nvx_maps= and virtfs_dir=.
OpenVMM's start_state_units, in
openvmm_core/src/worker/dispatch/restore.rs at the pin 14dc4116ec7,
gates input, starts the units, writes the event, and only then releases
the vCPUs, as design 5.5.1's steps 20 to 22 say; the Guide's two
passages that order it otherwise are wrong. The rest follows
host::filesystem's planner, host::cli::session's exec deadline,
host::restore_driver, guest::init::boot's restore order,
guest::managed_agent::launch's restored proxy, and the scenarios of
scripts/nvx_tools/microvm_tests.py.

Part of #1131 (SL6.10d).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The runtime docs presented upgrades as a compatibility matter, in notes
scattered across doc/run.md: start refused configurations "as older
releases wrote" and memory that "nvx.py and earlier releases" recorded,
the proxy section listed what "earlier releases" passed to OpenVMM, the
store asked to stop sandboxes before switching versions, and snapshots
of an earlier OpenVMM pointed to an old NVX commit that restores them,
as templates. Nothing in NVX reads or migrates the state of another
version, and a snapshot restores only under the runtime identity that
captured it, so after an upgrade every sandbox and snapshot starts
fresh.

run.md gains Upgrades, the one explanation, which the migration guide
can link: NVX keeps no state from one version to the next and migrates
none; a version may refuse another's sandbox configuration with
format-unsupported, or fail to read its records; and a snapshot of
other artifacts or contracts is refused. To upgrade, stop and
deprovision each sandbox with the version that started it, switch, and
capture the snapshots again into new stores. The scattered notes now
state the current refusals and link to it, the proxy section loses its
list of what earlier releases did, which Sandbox networking and
Proxy-only networking state as current rules, and usage.md links to it
from the nvx-rs chapter, --ram-upper-percent, and snapshot restore,
which also says that its snapshot's key records the runtime identity
that captured it.

Evidence: host::cli::sandbox::state reads only format 10 and refuses
other configurations with format-unsupported; host::cli::store reads
its records with deny_unknown_fields; host::snapshot's
Snapshot::accepts refuses a runtime identity other than its key's.

Part of #1131 (SL6.10d).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ppenna
Pedro Henrique Penna (ppenna) force-pushed the agents/slim-sl6-10d-cli-reference branch from 3ef56e7 to 7a95d13 Compare November 13, 2026 02:29
@ppenna
Pedro Henrique Penna (ppenna) marked this pull request as ready for review November 13, 2026 02:29
@ppenna
Pedro Henrique Penna (ppenna) merged commit 7e5440a into feature/nvx-refactor Nov 13, 2026
82 checks passed
@ppenna
Pedro Henrique Penna (ppenna) deleted the agents/slim-sl6-10d-cli-reference branch November 13, 2026 02:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants