Skip to content

guest: hand layered init control to the managed runtime - #1421

Merged
Pedro Henrique Penna (ppenna) merged 17 commits into
feature/nvx-refactorfrom
esaurez/slim-3.3.2-sl7.6b-20261109
Nov 10, 2026
Merged

Pedro Henrique Penna (ppenna) merged 17 commits into
feature/nvx-refactorfrom
esaurez/slim-3.3.2-sl7.6b-20261109

Conversation

@esaurez

@esaurez Enrique Saurez (esaurez) commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Refs #1116

Hand the real layered init's retained root, control endpoint, CLI identity,
account defaults, mapper, and optional source root to the existing managed
dispatcher and runtime. Retain init-owned helper failure and shutdown behavior,
prelaunch ownership, cancellation, and the legacy sandbox path.

All twelve qualified SL7.6b commits were replayed exactly once directly onto
#1397 at 29fca59317ffb67d8a3c89061faddf224c086051, without conflicts,
ancestor commits, or merges. Original messages, trailers, authors, patch
operations, modes and unsigned signature states are preserved. Their only
range-diff difference is unchanged allowlist hunk context from the new parent.
The OpenVMM pin remains 14dc4116.

The independent review confirmed Copilot's blocking-output finding. The coupled
fix makes only runtime-backed control descriptors nonblocking and polls writable
readiness alongside the same Runtime's signal descriptor. It retains statuses
collected during writes, hands them to the execution driver before each event,
and preserves orderly termination during ATTACH and RESET acknowledgement.
Legacy sessions remain unchanged; no timer-retry workaround, second reaper,
public API or wire-format change was introduced.

The first fix commit accidentally placed the new poll regression inside an
assertion's token tree. The independent review caught this; the append-only
correction 9445f16b places it at module scope without changing its assertions.
Actual Linux compilation then caught a slice/array mismatch in the new Runtime
test's frame decoder. c7e97160 uses the existing session-test conversion to the
decoder's fixed-size header type. Production and framing assertions are unchanged.
Both introduced test defects and their superseded results remain historical.

The full actual-base delta is 313 CORE, seven counted paths, fifteen raw paths:
twelve preserved commits plus three coupled fix commits. The user approved the
existing poll helper/test extension and necessary additional lines for this task.
All JSON and correction lines are counted, without extraction/category discounts.

Existing tests

The full structured comparison flags eight existing assets, relocates three
unchanged, adds 52 assets and extends one. The original seven flagged assets remain:

  • init::tests::{init,serve} keep the old helper/signal scripts in test-only
    drivers of production preparation, collection and shutdown.
  • Harness::spawn_at delegates to a shared constructor for the same dispatcher.
  • Harness::read_outer delegates to the same extracted frame reader.
  • isolate_process_test includes ignored tests only in its exact named child
    invocation, so privileged regression cases are actually runnable.
  • The existing managed lifecycle test additionally checks real layered init with
    zero/two mappings, identity/defaults, reconnects and cancellation; its original
    workload/shutdown checks remain.
  • The obsolete Runtime::wake hook disappears with the idle loop; the existing
    init hook allowance tightens from eight to seven.

The eighth flagged asset is the existing poll-order test: its expected readable
event also asserts writable == false. New focused regressions cover writable
readiness/backpressure, SIGTERM and helper-failure precedence while output is
blocked, pending termination before ATTACH, and correct delivery of a status
collected during a write. No assertion, deadline or existing conditional skip was
relaxed. The new Runtime cases retain their real CAP_SYS_NICE/CAP_SYS_RESOURCE
prerequisite.

Validation

Current head: c7e971607a0ea3b638800b4d90a3fd549685c42b.

  • Full current-head CI run 42697389932, attempt 1: success. Current Copilot
    review 5832613919: COMMENTED, with zero unresolved review threads at the
    final inspection. No formal approval is claimed.
  • Independent whole-owned source review: CLEAR, actual Opus 5.5/MAX/LONG.
    The user approved the retained reviewer's audited read-only path; actual calls
    and source/native/index immutability were checked. There was no technical
    tool-deny fence, and this source-only review did not substitute for execution.
  • Windows: all 29 Rust command rows passed in a fresh whole confirmation with
    strict guard exit0 and no owned descendants. Standalone Python/quality passed:
    1823 unit tests, 70 conditional skips, plus applicable lint/type/format,
    layout, kernel-inventory and PowerShell checks, strict guard exit0.
  • Linux: format, five clippy variants and the focused poll regressions passed.
    The full all-feature library run passed 2091 tests, 51 ignored, with
    default test parallelism. All 33 remaining Rust command rows passed, including
    applicable MSRV/cross-target, public API, guest and adapter matrices.
    CLI/quality passed 1823 unit tests, 30 conditional skips, Specula's
    25 tests, and applicable shell/type/lint/layout/kernel/PowerShell checks.
  • All ten selected privileged Runtime cases actually executed and passed,
    including the four new regressions: one passed, zero failed, zero ignored
    per invocation. These were current frozen-binary runs under narrowly approved
    private-namespace privileges, not early-return skips or old-head evidence.
  • Fresh current Alpine/Ubuntu guests, distro, real layered-init fixture and
    runtime-adapter guest were assembled and input/provenance checked. Native
    OpenVMM/kernel inputs were reused only after matching provenance validation.
    Eighteen host runners per platform were prebuilt and hash-bound; hardware
    execution used a strict compiler-free bridge, with no own compiler/VM overlap.

Fresh hardware acceptance completed on Windows/WHP (user-declared bare metal)
and Pedro/MSHV (configured virtual machine):

Acceptance WHP MSHV
Real layered init, maps0 and maps2 Each 1 passed, 0 ignored Each 1 passed, 0 ignored
Whole default microVM suite 57 passed records, 51 distinct scenarios 57 passed records, 51 distinct scenarios
Public execution/lifecycle/proxy/RAM-upper checks 27 / 41 / 12 / 14 27 / 41 / 12 / 14
Canonical runtime-adapter suite 156 cases, 11 targets passed 156 cases, 11 targets passed

Both backends ended with no owned VMMs. MSHV VMMs ran after dropping to UID1001,
not as root. The full MSHV operation and both WHP layered cases passed their
unchanged resource guards. The user explicitly authorized bypassing resource
guard/admission for the remaining WHP default and canonical tests. Those are
actual test passes under that exception, not resource-guard passes.
Original
test assertions/deadlines, bounded execution, source/artifact checks, owned Job
containment and teardown checks remained in force; no global guard was changed.

Historical failures remain recorded, not waived or relabelled:

  • Current-head initial GNU runs exposed long Unix-socket paths (77 failures),
    then an unmapped inherited process group (one failure). User-approved private
    /tmp binding and an own session/group before Bash PID1 corrected only the QA
    environment; the unchanged-source, default-parallel whole run then passed.
  • An earlier current-head Windows Rust attempt completed command rows but failed
    its guard with exit125. The separate whole confirmation passed the unchanged
    guard; the earlier cause remains unproven. Superseded 9445f16b also retains
    its E0308 test-helper compilation failure, Windows guard125 and telemetry
    timeout; none qualifies this head.
  • First candidate 9698af55 passed its CI but retained the actionable blocking
    write finding. Its first Windows Python run had 63 failures, 29 errors and 71
    skips, including DLL-initialization errors; their cause remains unproven.
    The superseded 17ca29e3 interruption and the original source's ETXTBSY
    history, conditional skips and limitations remain historical.

A separate red-baseline executable was not built or run. No performance
comparison is inferred from these acceptance results.

This PR remains Draft. Its current head received exactly one authorized
temporary Ready/full-CI/Draft cycle; no additional CI rerun was used for local
qualification. Final Ready and merge remain the user's decision.

Enrique Saurez and others added 12 commits November 9, 2026 10:36
Keep prepared mapping handles and the host table intact when handing them to the shared dispatcher. Read only the CLI-owned workload identity while retaining the resolved account defaults.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4f77d672-f979-4d98-9305-63a5f57ae08c
Share the init token grammar so carriage returns and form feeds cannot create extra identities. Cover those cases and check that CLI IDs reach the native execution request.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4f77d672-f979-4d98-9305-63a5f57ae08c
Adopt prepared layered mappings and the init-owned collector through the existing control dispatcher. Keep framing and legacy sandbox policy in place.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4f77d672-f979-4d98-9305-63a5f57ae08c
Cover fragmented control input, helper failure versus SIGTERM and terminal status delivery before native driver events. Reuse the existing outer-record reader and run explicitly selected ignored proofs in isolated subprocesses.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4f77d672-f979-4d98-9305-63a5f57ae08c
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4f77d672-f979-4d98-9305-63a5f57ae08c
Keep helper failure policy and the first launch and cleanup diagnostics while the existing dispatcher settles an owned failed child.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4f77d672-f979-4d98-9305-63a5f57ae08c
Accept a zero-mapping root without an export, reject a positive table without its retained export, and keep unexpected MAPS on the existing accounting path.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4f77d672-f979-4d98-9305-63a5f57ae08c
Keep direct prelaunch collection outside the launch-error conversion so a pending SIGTERM or helper failure ends the session without a false launch-failed response. Exercise both after receiving and crediting a complete EXEC, and keep the helper-failure host test on the existing fake diagnostic boundary.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4f77d672-f979-4d98-9305-63a5f57ae08c
Keep the switched root and retained mapping handles alive while the existing managed dispatcher serves init control with the same Runtime. Remove the now-used wake and dispatcher hook allowances.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4f77d672-f979-4d98-9305-63a5f57ae08c
The shared control loop now polls Runtime directly. Retain the old helper and signal scripts as test-only waiting fixtures instead of a second unused production wait path.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4f77d672-f979-4d98-9305-63a5f57ae08c
Reuse the native host control driver for a dedicated PID-1 layered boot fixture. Check CLI identity and account defaults, retained zero or ordered mappings across reset, cancellation and clean shutdown without changing the existing default scenario.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4f77d672-f979-4d98-9305-63a5f57ae08c
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 4f77d672-f979-4d98-9305-63a5f57ae08c
@esaurez Enrique Saurez (esaurez) added the tests-modified Changes existing tests, test data, or test wiring label Nov 9, 2026
@esaurez
Enrique Saurez (esaurez) requested a balanced review from Copilot November 9, 2026 18:42
@esaurez
Enrique Saurez (esaurez) marked this pull request as ready for review November 9, 2026 18:42
@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

This pull request changes existing tests

It modifies or removes 12 existing test asset(s), listed below. Review each with care: a change to a test can hide a regression that the test caught. See doc/contribute.md#changes-to-existing-tests.

Test code

Change What Before After
modified helper fn init in guest::init::tests nvx-rs/src/guest/init/tests.rs:26 nvx-rs/src/guest/init/tests.rs:26
modified helper fn serve in guest::init::tests nvx-rs/src/guest/init/tests.rs:31 nvx-rs/src/guest/init/tests.rs:31
modified helper fn spawn_at in guest::managed_agent::dispatch::tests, impl Harness`` nvx-rs/src/guest/managed_agent/dispatch/tests.rs:70 nvx-rs/src/guest/managed_agent/dispatch/tests.rs:70
modified helper fn read_outer in guest::managed_agent::dispatch::tests, impl Harness`` nvx-rs/src/guest/managed_agent/dispatch/tests.rs:210 nvx-rs/src/guest/managed_agent/dispatch/tests.rs:214
modified helper fn isolate_process_test in guest::sys::process::tests nvx-rs/src/guest/sys/process/tests.rs:13 nvx-rs/src/guest/sys/process/tests.rs:13
modified helper fn reach_guest in vm_clone (tests) tests/runtime-adapter/tests/vm_clone.rs:4788 tests/runtime-adapter/tests/vm_clone.rs:4788
modified helper fn reach_guest in vm_layered (tests) tests/runtime-adapter/tests/vm_layered.rs:3019 tests/runtime-adapter/tests/vm_layered.rs:3017
modified imports the imports of vm_clone (tests) tests/runtime-adapter/tests/vm_clone.rs:206 tests/runtime-adapter/tests/vm_clone.rs:206
modified imports the imports of vm_layered (tests) tests/runtime-adapter/tests/vm_layered.rs:41 tests/runtime-adapter/tests/vm_layered.rs:41
modified test fn poll_reports_input_in_the_callers_order in guest::sys::poll::tests nvx-rs/src/guest/sys/poll/tests.rs:7 nvx-rs/src/guest/sys/poll/tests.rs:7
modified test fn a_managed_guest_runs_request_scoped_workloads_until_it_stops in managed_lifecycle (tests) nvx-rs/tests/managed_lifecycle.rs:369 nvx-rs/tests/managed_lifecycle.rs:494
removed hook the test hooks of fn wake in guest::init (impl Runtime) nvx-rs/src/guest/init.rs:860

Relocated unchanged: 3; added: 60. Compared the merge base 29fca59317ff with 061b99e153b5.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A critical blocking control-output write can leave PID 1 unable to handle shutdown or helper failure when the host stops draining output, requiring nonblocking writes and backpressure regression coverage.

1 open issue
What changed in this PR

This PR advances #1116 by routing nvx-init’s prepared layered roots through the managed agent’s existing control dispatcher, sharing execution and mapping handling with blockless VMs while retaining init-owned shutdown and helper supervision.

Changes:

  • Pass retained mapping handles, CLI identity, and account defaults into the managed dispatcher.
  • Integrate init’s status collector and signal handling into control reads and workload execution.
  • Add layered-runtime regressions and lifecycle checks, while moving the obsolete idle-loop driver into test-only code.
File Description
scripts/​test-layout-allowlist.json Tightens init’s test-hook allowance.
nvx-rs/​tests/​managed_lifecycle.rs Adds real layered-init mapping, identity, reconnect, and cancellation checks.
nvx-rs/​src/​guest/​sys/​process/​tests.rs Allows explicitly selected ignored tests in isolated children.
nvx-rs/​src/​guest/​managed_agent/​tests.rs Tests layered CLI identity parsing.
nvx-rs/​src/​guest/​managed_agent/​session.rs Integrates init supervision and status retention into control reads.
nvx-rs/​src/​guest/​managed_agent/​mappings/​tests.rs Tests retained-export requirements.
nvx-rs/​src/​guest/​managed_agent/​mappings.rs Adopts prepared layered mapping state and handles.
nvx-rs/​src/​guest/​managed_agent/​exec.rs Routes workload collection through the shared session.
nvx-rs/​src/​guest/​managed_agent/​dispatch/​tests/​runtime.rs Adds privileged shutdown, helper-failure, and status-ownership regressions.
nvx-rs/​src/​guest/​managed_agent/​dispatch/​tests.rs Shares harness helpers and tests layered dispatch.
nvx-rs/​src/​guest/​managed_agent/​dispatch.rs Adds layered configuration and runtime-backed serving.
nvx-rs/​src/​guest/​init/​tests.rs Keeps idle-loop test drivers outside production code.
nvx-rs/​src/​guest/​init.rs Replaces the production idle loop with managed dispatch.

Review effort: Balanced

Make only runtime-backed control descriptors nonblocking and wait for writable
readiness alongside the existing Runtime signal descriptor. Keep helper failure
and SIGTERM handling live without polling on a timer or changing legacy sessions.

Retain leader statuses collected during writes and hand them to the execution
driver before each event. Preserve orderly termination during ATTACH and reset
acknowledgement as well as workload output.

Add focused readiness, backpressure, helper-precedence, pending-termination and
write-collected-status regressions without relaxing existing checks or hooks.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 190278d5-cbae-4ec6-b47d-ddf469124598

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new test function is declared inside the expected array passed to assert_eq!, preventing Linux guest unit tests from compiling.

1 open issue
1 resolved since last review

Review effort: Balanced

Comment thread nvx-rs/src/guest/sys/poll/tests.rs Resolved
Move the new poll regression out of the existing assertion's token tree so
Linux unit tests compile and the test inventory discovers it independently.
Keep the original ordering assertions and the readiness regression unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 190278d5-cbae-4ec6-b47d-ddf469124598
@esaurez
Enrique Saurez (esaurez) requested a balanced review from Copilot November 9, 2026 20:31
@esaurez
Enrique Saurez (esaurez) marked this pull request as ready for review November 9, 2026 20:32
@esaurez
Enrique Saurez (esaurez) marked this pull request as draft November 9, 2026 20:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Changes to PID-1 control, reaping, and shutdown across switch-root require privileged regression tests and real layered-init boot validation that remain pending and cannot be established by source inspection alone.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Convert the bounded header slice using the existing session-test pattern so
the Linux-only regression compiles against the fixed-size protocol decoder.
Keep all framing assertions and production behavior unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 190278d5-cbae-4ec6-b47d-ddf469124598

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Production PID-1 control, child-status collection, and shutdown changes still await current-head privileged Runtime regressions and unhooked layered-init lifecycle validation on qualified hosts.

No open issues 🎉

Review effort: Balanced

Base automatically changed from esaurez/slim-3.3.1-sl7.5b-20261108 to feature/nvx-refactor November 9, 2026 23:08
@esaurez
Enrique Saurez (esaurez) marked this pull request as ready for review November 10, 2026 00:07
Enrique Saurez and others added 2 commits November 9, 2026 17:18
A Linux loopback connect may choose its destination as its own source port
before the guest publishes its listener. A normal close leaves that port
in TIME_WAIT and blocks the host's later publication.

Share the existing listener client between cold-launch and clone scenarios
and reset only that self-connection before retrying. Keep connect timing,
read deadlines, and greeting assertions unchanged. Cover publication
rebinding with a real self-connected TCP stream and a normal exchange.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 0c16e8c2-e1da-4c5f-9b3b-5535db03b05b
Keep the original retry behavior when an ingress policy resets a connected
client before peer inspection. Prove the race with a real reset socket,
bound the positive listener control, and document the shared test fixture.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 0c16e8c2-e1da-4c5f-9b3b-5535db03b05b

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because caching a ptrace stop can cause the leader’s terminal status to be discarded, producing a lost-status error instead of the actual termination result.

No open issues 🎉

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feat-nvx-refactor tests-modified Changes existing tests, test data, or test wiring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants