Skip to content

nvx-rs: admit nvx-rs sandbox exec through a bounded queue across CLI processes #819

Closed

Description

Parent: #816. Builds on #535's store transactions and OS-held leases (A1-host). Independent of the other children.

Goal. NVX's CLI runs one exec at a time per sandbox, queues a bounded number of waiters across processes, and lets Stop bypass the queue.

Today (dev c5a0dd7):

  • Each nvx-rs sandbox exec connects through OpenVMM's broker, which has a single host slot. Nothing queues CLI processes, so a second exec that arrives while one runs fails at the broker.
  • During an exec, the guest answers every request except the matching CANCEL with BUSY, Stop included (guest/managed_agent/exec.rs:588).

Scope:

  1. Queue: a persistent FIFO, with one active exec and at most 8 waiters, checked before the broker. It is built on the store's short transactions and OS-held leases, tied to the runtime generation and the caller's process identity, on Linux and Windows.
  2. Waiting:
    • The queue wait defaults to 60 s, can be overridden per invocation, and is separate from the workload timeout.
    • A full queue fails at once. An expired, cancelled or dead waiter never starts.
    • A dead active owner goes through recovery, not lease expiry alone.
  3. Stop: publishes a stopping fence, cancels the waiters, asks the active exec's owner to cancel and release the slot, then stops gracefully. If the slot isn't released or the guest doesn't stop in time, it reports the timeout and leaves the VM, as today. It never escalates to forced termination.
  4. Outcomes: queue-full and queue-timeout are distinct, documented outcomes, with stable error codes.

Not in scope: exec concurrency in NVX's adapter; other adapters' admission policy; NVXC v2 (#821).

Acceptance:

  • Separate CLI processes on Linux and Windows test the FIFO and its capacity, queue expiry, a caller's death at each reservation step, a new runtime generation, and Stop with queued and active work.
  • The time of queue admission is recorded for the performance workstream.
  • The cross-process queue part of consumer-contract scenario C6 passes.

Carries over: #623's scope 6 (A5 PR 3).

Activity

  1. esaurez commented on Nov 1, 2026

    @esaurez
    ContributorAuthor

    Rescoped by the nvx-rs slimming workstream (#1076). A per-sandbox OS file lock replaces the bounded queue: sandbox exec waits for the lock up to --wait (60 s by default), then fails with busy. Delivered by #1111.

  2. ppenna commented on Nov 13, 2026

    @ppenna
    Contributor

    Completed on feature/nvx-refactor, as rescoped above for #1076. #1111, which delivers it, has merged and closed as completed, with SL3.9 (#1469) and SL3.4b (#1490).

    A per-sandbox OS file lock replaces the bounded queue. sandbox exec holds the lock of exec.lock, in the sandbox's state directory, from before it finds the running VM until its workload ends. Another exec waits for it up to --wait, 60 s by default, and then fails with sandbox-busy.

    The push CI run at 367f7fe4d0, which contains it, passed, including the microVM jobs on KVM, MSHV and WHP: https://github.com/microsoft/nvx/actions/runs/43363710885. dev receives it when feature/nvx-refactor lands there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions