diff --git a/.github/actions/check-nvx-rs/action.yml b/.github/actions/check-nvx-rs/action.yml index 02d37edb2..9ec2984f8 100644 --- a/.github/actions/check-nvx-rs/action.yml +++ b/.github/actions/check-nvx-rs/action.yml @@ -71,7 +71,7 @@ runs: set -euo pipefail cargo "+${{ steps.toolchains.outputs.toolchain }}" test --locked --all-features cargo "+${{ steps.toolchains.outputs.toolchain }}" test --locked - # Without features, the library still holds the protocol and the execution engine. + # Without features, the library still holds the protocol. cargo "+${{ steps.toolchains.outputs.toolchain }}" build --locked --lib --no-default-features cargo "+${{ steps.toolchains.outputs.toolchain }}" test --locked --lib --no-default-features diff --git a/doc/acceptance-case-matrix.md b/doc/acceptance-case-matrix.md index d04744f7d..bc85a7719 100644 --- a/doc/acceptance-case-matrix.md +++ b/doc/acceptance-case-matrix.md @@ -34,7 +34,7 @@ its own in turn. | [The adapter's other tests](#the-adapters-other-tests) | 169 | 0 | 9 | 160 | | [MicroVM scenarios](#microvm-scenarios) | 54 | 50 | 3 | 1 | | [CI](#ci) | 69 | 43 | 3 | 23 | -| [Docs](#docs) | 36 | 28 | 2 | 6 | +| [Docs](#docs) | 36 | 26 | 3 | 7 | ## The adapter's scenarios @@ -478,18 +478,18 @@ this matrix. The README's own code is no doctest. | Code | Fate | Note | | --- | --- | --- | | `nvx-rs/src/activation/mod.rs - activation (line 130)` | kept | | -| `nvx-rs/src/exec/mod.rs - exec (line 113)` | kept | Its module leaves with the exec engine (#1087), whose PR retires it | +| `nvx-rs/src/exec/mod.rs - exec (line 113)` | retired | The exec engine leaves (SL6.11a) | | `nvx-rs/src/gpt/mod.rs - gpt (line 43)` | kept | | | `nvx-rs/src/guest/activation.rs - guest::activation (line 195)` | kept | Its capture preparation and admission fence leave (#1087), whose PR rewrites it on `guest::snapshot::capture` | | `nvx-rs/src/guest/activation/state.rs - guest::activation::state::Activation (line 56)` | kept | | | `nvx-rs/src/guest/cgroup.rs - guest::cgroup (line 36)` | kept | Its module leaves with the exec engine (#1087), whose PR retires it | | `nvx-rs/src/guest/entry.rs - guest::entry (line 90)` | kept | Its module leaves with the exec engine (#1087), whose PR retires it | -| `nvx-rs/src/guest/exec.rs - guest::exec (line 83)` | kept | Its module leaves with the exec engine (#1087), whose PR retires it | -| `nvx-rs/src/guest/exec.rs - guest::exec::Execution::adopt (line 625)` | kept | Its module leaves with the exec engine (#1087), whose PR retires it | +| `nvx-rs/src/guest/exec.rs - guest::exec (line 80)` | kept | Its module leaves with the exec engine (#1087), whose PR retires it | +| `nvx-rs/src/guest/exec.rs - guest::exec::Execution::adopt (line 620)` | kept | Its module leaves with the exec engine (#1087), whose PR retires it | | `nvx-rs/src/guest/gpt.rs - guest::gpt (line 18)` | kept | The guest's space becomes private (#1127), whose PR moves it to the space's unit tests | | `nvx-rs/src/guest/mappings/mapper.rs - guest::mappings::mapper (line 107)` | kept | The guest's space becomes private (#1127), whose PR moves it to the space's unit tests | | `nvx-rs/src/guest/mappings/table.rs - guest::mappings::table::MappingTable (line 47)` | kept | | -| `nvx-rs/src/guest/mod.rs - guest (line 41)` | kept | The guest-agent guidelines quote it | +| `nvx-rs/src/guest/mod.rs - guest (line 40)` | kept | The guest-agent guidelines quote it | | `nvx-rs/src/guest/nvxs.rs - guest::nvxs (line 22)` | kept | | | `nvx-rs/src/guest/process.rs - guest::process (line 12)` | kept | Its module leaves with the exec engine (#1087), whose PR retires it | | `nvx-rs/src/guest/process.rs - guest::process (line 39)` | kept | Its module leaves with the exec engine (#1087), whose PR retires it | @@ -497,7 +497,7 @@ this matrix. The README's own code is no doctest. | `nvx-rs/src/guest/root.rs - guest::root (line 130)` | kept | The guest's space becomes private (#1127), whose PR moves it to the space's unit tests | | `nvx-rs/src/guest/snapshot.rs - guest::snapshot (line 25)` | kept | The guest-agent guidelines quote it | | `nvx-rs/src/guest/spawn.rs - guest::spawn (line 41)` | kept | Its module leaves with the exec engine (#1087), whose PR retires it | -| `nvx-rs/src/guest/spawn.rs - guest::spawn::LaunchStage::from_code (line 201)` | kept | Its module leaves with the exec engine (#1087), whose PR retires it | +| `nvx-rs/src/guest/spawn.rs - guest::spawn::LaunchStage::from_code (line 201)` | rewritten | `nvx-rs/src/guest/spawn/tests.rs::launch_stages_keep_their_numbers`. The exec engine's launch failure and outcome, which the example read, leave with the engine (SL6.11a) | | `nvx-rs/src/guest/terminal.rs - guest::terminal (line 10)` | kept | | | `nvx-rs/src/guest/time/mod.rs - guest::time (line 15)` | kept | | | `nvx-rs/src/host/sandbox/prepared.rs - host::sandbox::prepared::PreparedLaunch (line 92)` | kept | Its launch type leaves with the old launch paths (#1087), whose PR retires it | @@ -507,9 +507,9 @@ this matrix. The README's own code is no doctest. | `nvx-rs/src/host/template/plan.rs - host::template::plan::TemplatePlan (line 137)` | retired | `TemplatePlan` leaves with the template store (SL6.2a, #1121) | | `nvx-rs/src/host/template/store.rs - host::template::store::Store (line 473)` | retired | The template store leaves (SL6.2a, #1121) | | `nvx-rs/src/host/transport.rs - host::transport (line 22)` | kept | | -| `nvx-rs/src/lib.rs - (line 58)` | rewritten | Its `Instance` and sandbox store leave (SL6.1, #1121): `nvx-rs/src/lib.rs - (line 62)` compiles a cold boot and a stop through `host::vm` | +| `nvx-rs/src/lib.rs - (line 58)` | rewritten | Its `Instance` and sandbox store leave (SL6.1, #1121): `nvx-rs/src/lib.rs - (line 57)` compiles a cold boot and a stop through `host::vm` | | `nvx-rs/src/protocol/nvxs/mod.rs - protocol::nvxs (line 68)` | kept | | | `nvx-rs/src/host/snapshot/capture.rs - host::snapshot::capture::finish_capture (line 48)` | kept | | | `nvx-rs/src/host/store/mod.rs - host::store (line 104)` | retired | The sandbox store leaves (SL6.1, #1121) | -| `nvx-rs/README.md`, its Example: an instance through `LaunchSpec` and the sandbox store | rewritten | Nothing compiles it. `nvx-rs/examples/host-workflows.rs` compiles the documented host workflows, and the library's guide (SL6.10a) replaces the example. SL6.1 (#1121) made it the cold boot that `nvx-rs/src/lib.rs - (line 62)` compiles | +| `nvx-rs/README.md`, its Example: an instance through `LaunchSpec` and the sandbox store | rewritten | Nothing compiles it. `nvx-rs/examples/host-workflows.rs` compiles the documented host workflows, and the library's guide (SL6.10a) replaces the example. SL6.1 (#1121) made it the cold boot that `nvx-rs/src/lib.rs - (line 57)` compiles | | `nvx-rs/examples/external-launch.rs` | kept | Compiled with the crate's targets, and run by hand on a host | diff --git a/doc/guest-agent-guidelines.md b/doc/guest-agent-guidelines.md index 189296142..76740bcaf 100644 --- a/doc/guest-agent-guidelines.md +++ b/doc/guest-agent-guidelines.md @@ -55,7 +55,7 @@ announcer, which is a program, `/sbin/nvx-ingress`: an init runs it itself, or has `nvx-ingress start` run it from a socket that it binds for it. Until the nvx-rs slimming ([microsoft/nvx#1076](https://github.com/microsoft/nvx/issues/1076)) removes -them, the crate also keeps the older execution engine and its process +them, the crate also keeps the older process modules, `guest::{exec, spawn, entry, cgroup, process}`, which NVX's older guest agents use. nvx-init uses none of these modules, and its time daemon runs for the VM's life. diff --git a/doc/modules/README.md b/doc/modules/README.md index 781aedeaf..f29641690 100644 --- a/doc/modules/README.md +++ b/doc/modules/README.md @@ -51,13 +51,6 @@ to a module that it contains. ![The modules that nvx::activation contains](nvx-activation.svg) -### `nvx::exec` - -`nvx::exec`, in [`nvx-rs/src/exec/mod.rs`](../../nvx-rs/src/exec/mod.rs), -contains 6 modules. - -![The modules that nvx::exec contains](nvx-exec.svg) - ### `nvx::gpt` `nvx::gpt`, in [`nvx-rs/src/gpt/mod.rs`](../../nvx-rs/src/gpt/mod.rs), contains diff --git a/doc/modules/nvx-exec.svg b/doc/modules/nvx-exec.svg deleted file mode 100644 index 6bc850016..000000000 --- a/doc/modules/nvx-exec.svg +++ /dev/null @@ -1,112 +0,0 @@ - - - - - - -nvx::exec - - - -nvx::exec->nvx::exec::credit - - - - - -nvx::exec->nvx::exec::peer - - - - - -nvx::exec->nvx::exec::runner - - - - - -nvx::exec::credit->nvx::exec::credit::inbox - - - - - -nvx::exec::credit->nvx::exec::credit::receiver - - - - - -nvx::exec::credit->nvx::exec::credit::sender - - - - - -nvx::exec - - -nvx::exec - - - - - -nvx::exec::credit - - -credit - - - - - -nvx::exec::credit::inbox - - -inbox - - - - - -nvx::exec::credit::receiver - - -receiver - - - - - -nvx::exec::credit::sender - - -sender - - - - - -nvx::exec::peer - - -peer - - - - - -nvx::exec::runner - - -runner - - - - - diff --git a/nvx-rs/README.md b/nvx-rs/README.md index 513d02f27..678cd5bf6 100644 --- a/nvx-rs/README.md +++ b/nvx-rs/README.md @@ -43,8 +43,6 @@ nvx-rs/ │ │ by the host and the guest │ ├── hypervisor.rs │ ├── error.rs nvx::Error, its ErrorKind, and its stable ErrorCode -│ ├── exec/ the execution engine: the streams, credit, and outcome of an execution's two -│ │ sides, without I/O, threads, or features, for any framing and host │ ├── host/ feature `host` (default): the runtime that every launch shares, VM │ │ directories and configurations, artifacts, launch, network and egress policy │ │ files, filesystem, NVXS connection, session, process, platform, templates' @@ -55,12 +53,12 @@ nvx-rs/ │ ├── guest/ feature `guest`: the programs that run in Linux guests, the guest's NVXS │ │ connection, the libraries through which a guest init or agent opens the │ │ terminal on which it serves its session, starts a program, drives its -│ │ execution, alone or through the execution engine, collects its children's -│ │ statuses, settles its executions' cgroups, mounts host mappings beneath the -│ │ descriptors of the roots that it names, assembles and tears down a layered -│ │ root, reads a GPT disk's partition table through its block device, and -│ │ prepares a template's guest for a capture before its root's assembly, and -│ │ the typed calls through which it runs the time ABI's steps and the reseed +│ │ execution, collects its children's statuses, settles its executions' +│ │ cgroups, mounts host mappings beneath the descriptors of the roots that it +│ │ names, assembles and tears down a layered root, reads a GPT disk's partition +│ │ table through its block device, and prepares a template's guest for a +│ │ capture before its root's assembly, and the typed calls through which it +│ │ runs the time ABI's steps and the reseed │ └── bin/ nvx-rs, nvx-fake-openvmm, and the guest binaries ├── tests/ fake-OpenVMM, CLI, NVXS session, process and spawn, checked-handoff, │ host-module layering, and guest-binary tests, the CLI's scenario tests and @@ -115,7 +113,6 @@ changes only when the module's purpose does. | [`nvx::gpt`](#nvxgpt) | Strict reading of GPT disks whose first partition configures an adapter and whose others hold a root's read-only layers | | [`nvx::uuid`](#nvxuuid) | UUIDs, such as those of GPT disks and filesystems, in their canonical text and in GPT's layout | | [`nvx::Hypervisor`](#nvxhypervisor) | KVM, MSHV, or WHP, with its serde form behind the `serde` feature | -| [`nvx::exec`](#nvxexec) | The execution engine: the streams, credit, and outcome of an execution's two sides, without I/O, for any framing | | [`nvx::host::runtime`](#nvxhostruntime) | `Runtime`: what every launch shares, the runtime's files and their identity, the hypervisor and its probe, and the launch options | | [`nvx::host::vm`](#nvxhostvm) | A private VM directory, cold, capture and restore preparation, owned launch commands, verified detached launch, confirmed stop, and caller-saved reattachment | | [`nvx::host::snapshot`](#nvxhostsnapshot) | Strict private snapshots, post-exit capture finishing, verified metadata, VM compatibility, captured NIC addresses, and time eligibility | @@ -142,7 +139,6 @@ changes only when the module's purpose does. | [`nvx::guest::process`](#nvxguestprocess) | `Child`: the status of one child that the caller names, and of no other | | [`nvx::guest::cgroup`](#nvxguestcgroup) | Cgroup v2 handles through which a guest init or agent places, accounts, kills, and settles its executions | | [`nvx::guest::exec`](#nvxguestexec) | `Execution`: drives one program as an execution, for any guest init or agent | -| [`nvx::guest::exec::binding`](#nvxguestexecbinding) | `Binding`: an `nvx::exec::Runner` bound to an `Execution`, the glue that a guest adapter would otherwise write | | [`nvx::guest::time`](#nvxguesttime) | x86_64: the steps of NVX time ABI v1 as typed calls in the caller's process, for any guest init or agent | | [`nvx::guest::snapshot`](#nvxguestsnapshot) | x86_64: bounded Fresh capture with inline restore repair and staged failure dispositions, without execution control | | [`nvx::guest::activation`](#nvxguestactivation) | x86_64: fence-free activation, with the retained template capture and clone repair interface | @@ -393,31 +389,6 @@ dependencies, so the host and the guest share it. KVM, MSHV, or WHP, with its serde form behind the `serde` feature. -### `nvx::exec` - -The execution engine: the stream state, credit, ordering, and outcome of one -execution, for any framing that carries its messages, without I/O, threads, -global state, or features, so that it builds with `--no-default-features`. - -A `Runner` is the side that runs the workload: it reads the workload's output -under the peer's credit, holds the input that the peer sent within its window -until the workload takes it, requests the terminations, discards, input close, -and settlement that its adapter applies, and returns one terminal `Outcome`, -which tells whether the execution is reusable or fenced. - -A `Peer` is the other side: it takes output into windows that its sink empties, -returns credit as the sink takes bytes, sends input within the runner's window, -and ends the execution on a cancellation, a timeout, a drop, a reset, or a -disconnection. - -Both take the current `Instant` from the caller, report their next deadline, and -never wait, so a blocked stream, sink, window, or transport stops no timer, -cancellation, or other execution, and both bound their memory. - -The adapter chooses one byte window per stream, or frames charged against one -pool that stdout and stderr share, and the maximum chunk, in `Limits`; a `Key`'s -generation tells stale messages from foreign ones. - ### `nvx::host::runtime` `Runtime`: what every launch shares, the paths of the OpenVMM executable, guest @@ -1465,29 +1436,6 @@ one. NVX's managed agent is one caller. -### `nvx::guest::exec::binding` - -`Binding`: an `nvx::exec::Runner` bound to an `Execution`, the glue that a guest -adapter would otherwise write. - -The runner's read allowances become grants, its discards discards, the input -that it accepted writes and then a close, its terminations a cancellation, a -disconnection, or a termination, and its settlement a kill of the execution's -cgroup and checks until no process occupies it; without a cgroup, nothing proves -that the program's descendants are gone, so the execution is fenced. - -It feeds the execution's events back to the runner, statuses that a caller who -reaps hands over with `observe` included, and never blocks: the caller polls its -descriptors and its input writer until its deadline, and each call takes a -bounded number of steps, so that no execution holds the caller's loop. - -The caller's own `signal` passes through to the execution, which it does not -end: the runner sees only the program's end, once the program exits. - -A launch that failed reaches the runner with the number of the `LaunchStage` -that failed, `LaunchStage::code`, or with 0, which numbers no stage, if the -binding could not read the launch report. - ### `nvx::guest::time` x86_64: `TimeAbi`, the steps of NVX time ABI v1 as typed calls in the caller's @@ -3111,12 +3059,7 @@ The process and spawn test spawns VMM stand-ins through the low-level spawn, waits for them through their verified references, finds one again after the process that spawned it crashed, and checks that a VMM inherits only its standard streams and the descriptors or handles that it is allowed while -another thread spawns processes. The tests of `nvx::exec` drive a runner and a -peer through a -simulated workload, transport, and sink on a clock of their own, and, on Linux, -the tests of `nvx::guest::exec::binding` drive real programs through them, two -at once in one loop, one of them with a stalled sink; run as root, one of them -also settles a real cgroup. +another thread spawns processes. On Linux, with the `cli` feature, a loopback test drives the host's private `ControlSession` through a broker shim into the real managed agent's session, dispatch, and execution engine, and `nvx.py` also lints and diff --git a/nvx-rs/public-api/linux.txt b/nvx-rs/public-api/linux.txt index 2967ea292..33fdecb62 100644 --- a/nvx-rs/public-api/linux.txt +++ b/nvx-rs/public-api/linux.txt @@ -226,243 +226,6 @@ const nvx::contract::SNAPSHOT_FORMAT_VERSION const nvx::contract::TIME_ABI_VERSION fn nvx::contract::manifest_is_compatible fn nvx::contract::manifest_mismatches -mod nvx::exec -enum nvx::exec::Cause -variant nvx::exec::Cause::Cancelled -variant nvx::exec::Cause::Disconnected -variant nvx::exec::Cause::Dropped -variant nvx::exec::Cause::Reset -variant nvx::exec::Cause::TimedOut -struct nvx::exec::Key -method nvx::exec::Key::generation -method nvx::exec::Key::id -method nvx::exec::Key::new -struct nvx::exec::LaunchFailure -method nvx::exec::LaunchFailure::error -method nvx::exec::LaunchFailure::new -method nvx::exec::LaunchFailure::stage -struct nvx::exec::Limits -method nvx::exec::Limits::chunk -method nvx::exec::Limits::input -method nvx::exec::Limits::is_shared -method nvx::exec::Limits::new -method nvx::exec::Limits::output -method nvx::exec::Limits::shared_output -struct nvx::exec::Outcome -method nvx::exec::Outcome::cause -method nvx::exec::Outcome::is_reusable -method nvx::exec::Outcome::is_truncated -method nvx::exec::Outcome::launch_failure -method nvx::exec::Outcome::new -method nvx::exec::Outcome::status -method nvx::exec::Outcome::with_cause -method nvx::exec::Outcome::with_launch_failure -method nvx::exec::Outcome::with_reusable -method nvx::exec::Outcome::with_truncated -struct nvx::exec::Peer -method nvx::exec::Peer::buffered -method nvx::exec::Peer::cancel -method nvx::exec::Peer::consumed -method nvx::exec::Peer::credit -method nvx::exec::Peer::deadline -method nvx::exec::Peer::disconnect -method nvx::exec::Peer::dropped -method nvx::exec::Peer::end_input -method nvx::exec::Peer::end_output -method nvx::exec::Peer::has_started -method nvx::exec::Peer::high_water -method nvx::exec::Peer::input -method nvx::exec::Peer::input_closed -method nvx::exec::Peer::is_detached -method nvx::exec::Peer::is_finished -method nvx::exec::Peer::key -method nvx::exec::Peer::limits -method nvx::exec::Peer::memory_bound -method nvx::exec::Peer::new -method nvx::exec::Peer::next_event -method nvx::exec::Peer::next_message -method nvx::exec::Peer::output -method nvx::exec::Peer::pending_output -method nvx::exec::Peer::reset -method nvx::exec::Peer::started -method nvx::exec::Peer::terminal -method nvx::exec::Peer::tick -method nvx::exec::Peer::write_allowance -enum nvx::exec::Pool -variant nvx::exec::Pool::Input -variant nvx::exec::Pool::Output -variant nvx::exec::Pool::Stderr -variant nvx::exec::Pool::Stdout -enum nvx::exec::Receipt -variant nvx::exec::Receipt::Applied -variant nvx::exec::Receipt::Late -variant nvx::exec::Receipt::Stale -struct nvx::exec::Runner -method nvx::exec::Runner::awaits_settlement -method nvx::exec::Runner::buffered -method nvx::exec::Runner::cancel -method nvx::exec::Runner::credit -method nvx::exec::Runner::deadline -method nvx::exec::Runner::discarded -method nvx::exec::Runner::disconnect -method nvx::exec::Runner::dropped -method nvx::exec::Runner::end_input -method nvx::exec::Runner::exited -method nvx::exec::Runner::high_water -method nvx::exec::Runner::input -method nvx::exec::Runner::input_closed -method nvx::exec::Runner::input_written -method nvx::exec::Runner::is_detached -method nvx::exec::Runner::is_finished -method nvx::exec::Runner::key -method nvx::exec::Runner::launched -method nvx::exec::Runner::limits -method nvx::exec::Runner::memory_bound -method nvx::exec::Runner::new -method nvx::exec::Runner::next_message -method nvx::exec::Runner::next_request -method nvx::exec::Runner::output -method nvx::exec::Runner::output_closed -method nvx::exec::Runner::output_failed -method nvx::exec::Runner::output_hung_up -method nvx::exec::Runner::pending_input -method nvx::exec::Runner::read_allowance -method nvx::exec::Runner::reset -method nvx::exec::Runner::settled -method nvx::exec::Runner::tick -enum nvx::exec::Settlement -variant nvx::exec::Settlement::Failed -variant nvx::exec::Settlement::Settled -enum nvx::exec::Status -variant nvx::exec::Status::Exited -field nvx::exec::Status::Exited::0 -variant nvx::exec::Status::Signaled -field nvx::exec::Status::Signaled::0 -enum nvx::exec::Stream -const nvx::exec::Stream::ALL -variant nvx::exec::Stream::Stderr -variant nvx::exec::Stream::Stdout -struct nvx::exec::Timers -method nvx::exec::Timers::grace -method nvx::exec::Timers::new -method nvx::exec::Timers::settlement -method nvx::exec::Timers::timeout -method nvx::exec::Timers::timeout_duration -enum nvx::exec::Unit -variant nvx::exec::Unit::Bytes -variant nvx::exec::Unit::Frames -enum nvx::exec::Violation -variant nvx::exec::Violation::Chunk -variant nvx::exec::Violation::Credit -variant nvx::exec::Violation::Ended -variant nvx::exec::Violation::Key -variant nvx::exec::Violation::Order -variant nvx::exec::Violation::Pool -variant nvx::exec::Violation::Window -struct nvx::exec::Window -method nvx::exec::Window::bytes -method nvx::exec::Window::frames -method nvx::exec::Window::size -method nvx::exec::Window::unit -mod nvx::exec::peer -enum nvx::exec::peer::Event -variant nvx::exec::peer::Event::End -field nvx::exec::peer::Event::End::0 -variant nvx::exec::peer::Event::Finished -field nvx::exec::peer::Event::Finished::0 -variant nvx::exec::peer::Event::InputClosed -variant nvx::exec::peer::Event::Started -enum nvx::exec::peer::Message -variant nvx::exec::peer::Message::Cancel -field nvx::exec::peer::Message::Cancel::0 -variant nvx::exec::peer::Message::Credit -field nvx::exec::peer::Message::Credit::pool -field nvx::exec::peer::Message::Credit::units -variant nvx::exec::peer::Message::Input -field nvx::exec::peer::Message::Input::length -variant nvx::exec::peer::Message::InputEnd -struct nvx::exec::peer::Peer -method nvx::exec::peer::Peer::buffered -method nvx::exec::peer::Peer::cancel -method nvx::exec::peer::Peer::consumed -method nvx::exec::peer::Peer::credit -method nvx::exec::peer::Peer::deadline -method nvx::exec::peer::Peer::disconnect -method nvx::exec::peer::Peer::dropped -method nvx::exec::peer::Peer::end_input -method nvx::exec::peer::Peer::end_output -method nvx::exec::peer::Peer::has_started -method nvx::exec::peer::Peer::high_water -method nvx::exec::peer::Peer::input -method nvx::exec::peer::Peer::input_closed -method nvx::exec::peer::Peer::is_detached -method nvx::exec::peer::Peer::is_finished -method nvx::exec::peer::Peer::key -method nvx::exec::peer::Peer::limits -method nvx::exec::peer::Peer::memory_bound -method nvx::exec::peer::Peer::new -method nvx::exec::peer::Peer::next_event -method nvx::exec::peer::Peer::next_message -method nvx::exec::peer::Peer::output -method nvx::exec::peer::Peer::pending_output -method nvx::exec::peer::Peer::reset -method nvx::exec::peer::Peer::started -method nvx::exec::peer::Peer::terminal -method nvx::exec::peer::Peer::tick -method nvx::exec::peer::Peer::write_allowance -mod nvx::exec::runner -enum nvx::exec::runner::Message -variant nvx::exec::runner::Message::Credit -field nvx::exec::runner::Message::Credit::units -variant nvx::exec::runner::Message::End -field nvx::exec::runner::Message::End::0 -variant nvx::exec::runner::Message::InputClosed -variant nvx::exec::runner::Message::Output -field nvx::exec::runner::Message::Output::length -field nvx::exec::runner::Message::Output::stream -variant nvx::exec::runner::Message::Started -variant nvx::exec::runner::Message::Terminal -field nvx::exec::runner::Message::Terminal::0 -enum nvx::exec::runner::Request -variant nvx::exec::runner::Request::CloseInput -variant nvx::exec::runner::Request::Discard -field nvx::exec::runner::Request::Discard::0 -variant nvx::exec::runner::Request::Settle -variant nvx::exec::runner::Request::Terminate -field nvx::exec::runner::Request::Terminate::0 -struct nvx::exec::runner::Runner -method nvx::exec::runner::Runner::awaits_settlement -method nvx::exec::runner::Runner::buffered -method nvx::exec::runner::Runner::cancel -method nvx::exec::runner::Runner::credit -method nvx::exec::runner::Runner::deadline -method nvx::exec::runner::Runner::discarded -method nvx::exec::runner::Runner::disconnect -method nvx::exec::runner::Runner::dropped -method nvx::exec::runner::Runner::end_input -method nvx::exec::runner::Runner::exited -method nvx::exec::runner::Runner::high_water -method nvx::exec::runner::Runner::input -method nvx::exec::runner::Runner::input_closed -method nvx::exec::runner::Runner::input_written -method nvx::exec::runner::Runner::is_detached -method nvx::exec::runner::Runner::is_finished -method nvx::exec::runner::Runner::key -method nvx::exec::runner::Runner::launched -method nvx::exec::runner::Runner::limits -method nvx::exec::runner::Runner::memory_bound -method nvx::exec::runner::Runner::new -method nvx::exec::runner::Runner::next_message -method nvx::exec::runner::Runner::next_request -method nvx::exec::runner::Runner::output -method nvx::exec::runner::Runner::output_closed -method nvx::exec::runner::Runner::output_failed -method nvx::exec::runner::Runner::output_hung_up -method nvx::exec::runner::Runner::pending_input -method nvx::exec::runner::Runner::read_allowance -method nvx::exec::runner::Runner::reset -method nvx::exec::runner::Runner::settled -method nvx::exec::runner::Runner::tick mod nvx::gpt struct nvx::gpt::Disk method nvx::gpt::Disk::first_partition @@ -1228,36 +991,6 @@ field nvx::guest::exec::Source::Output::0 enum nvx::guest::exec::Stream variant nvx::guest::exec::Stream::Stderr variant nvx::guest::exec::Stream::Stdout -mod nvx::guest::exec::binding -struct nvx::guest::exec::binding::Binding -method nvx::guest::exec::binding::Binding::cancel -method nvx::guest::exec::binding::Binding::credit -method nvx::guest::exec::binding::Binding::deadline -method nvx::guest::exec::binding::Binding::descriptors -method nvx::guest::exec::binding::Binding::disconnect -method nvx::guest::exec::binding::Binding::dropped -method nvx::guest::exec::binding::Binding::end_input -method nvx::guest::exec::binding::Binding::execution -method nvx::guest::exec::binding::Binding::id -method nvx::guest::exec::binding::Binding::input -method nvx::guest::exec::binding::Binding::input_writer -method nvx::guest::exec::binding::Binding::is_finished -method nvx::guest::exec::binding::Binding::new -method nvx::guest::exec::binding::Binding::next -method nvx::guest::exec::binding::Binding::observe -method nvx::guest::exec::binding::Binding::reset -method nvx::guest::exec::binding::Binding::runner -method nvx::guest::exec::binding::Binding::signal -enum nvx::guest::exec::binding::Event -variant nvx::guest::exec::binding::Event::InputFailed -field nvx::guest::exec::binding::Event::InputFailed::0 -variant nvx::guest::exec::binding::Event::KillFailed -field nvx::guest::exec::binding::Event::KillFailed::0 -field nvx::guest::exec::binding::Event::KillFailed::1 -variant nvx::guest::exec::binding::Event::Message -field nvx::guest::exec::binding::Event::Message::0 -variant nvx::guest::exec::binding::Event::SettlementFailed -field nvx::guest::exec::binding::Event::SettlementFailed::0 mod nvx::guest::exit (hidden) fn nvx::guest::exit::main (hidden) mod nvx::guest::gpt (hidden) diff --git a/nvx-rs/public-api/windows.txt b/nvx-rs/public-api/windows.txt index de5f1ecc5..526f2453f 100644 --- a/nvx-rs/public-api/windows.txt +++ b/nvx-rs/public-api/windows.txt @@ -226,243 +226,6 @@ const nvx::contract::SNAPSHOT_FORMAT_VERSION const nvx::contract::TIME_ABI_VERSION fn nvx::contract::manifest_is_compatible fn nvx::contract::manifest_mismatches -mod nvx::exec -enum nvx::exec::Cause -variant nvx::exec::Cause::Cancelled -variant nvx::exec::Cause::Disconnected -variant nvx::exec::Cause::Dropped -variant nvx::exec::Cause::Reset -variant nvx::exec::Cause::TimedOut -struct nvx::exec::Key -method nvx::exec::Key::generation -method nvx::exec::Key::id -method nvx::exec::Key::new -struct nvx::exec::LaunchFailure -method nvx::exec::LaunchFailure::error -method nvx::exec::LaunchFailure::new -method nvx::exec::LaunchFailure::stage -struct nvx::exec::Limits -method nvx::exec::Limits::chunk -method nvx::exec::Limits::input -method nvx::exec::Limits::is_shared -method nvx::exec::Limits::new -method nvx::exec::Limits::output -method nvx::exec::Limits::shared_output -struct nvx::exec::Outcome -method nvx::exec::Outcome::cause -method nvx::exec::Outcome::is_reusable -method nvx::exec::Outcome::is_truncated -method nvx::exec::Outcome::launch_failure -method nvx::exec::Outcome::new -method nvx::exec::Outcome::status -method nvx::exec::Outcome::with_cause -method nvx::exec::Outcome::with_launch_failure -method nvx::exec::Outcome::with_reusable -method nvx::exec::Outcome::with_truncated -struct nvx::exec::Peer -method nvx::exec::Peer::buffered -method nvx::exec::Peer::cancel -method nvx::exec::Peer::consumed -method nvx::exec::Peer::credit -method nvx::exec::Peer::deadline -method nvx::exec::Peer::disconnect -method nvx::exec::Peer::dropped -method nvx::exec::Peer::end_input -method nvx::exec::Peer::end_output -method nvx::exec::Peer::has_started -method nvx::exec::Peer::high_water -method nvx::exec::Peer::input -method nvx::exec::Peer::input_closed -method nvx::exec::Peer::is_detached -method nvx::exec::Peer::is_finished -method nvx::exec::Peer::key -method nvx::exec::Peer::limits -method nvx::exec::Peer::memory_bound -method nvx::exec::Peer::new -method nvx::exec::Peer::next_event -method nvx::exec::Peer::next_message -method nvx::exec::Peer::output -method nvx::exec::Peer::pending_output -method nvx::exec::Peer::reset -method nvx::exec::Peer::started -method nvx::exec::Peer::terminal -method nvx::exec::Peer::tick -method nvx::exec::Peer::write_allowance -enum nvx::exec::Pool -variant nvx::exec::Pool::Input -variant nvx::exec::Pool::Output -variant nvx::exec::Pool::Stderr -variant nvx::exec::Pool::Stdout -enum nvx::exec::Receipt -variant nvx::exec::Receipt::Applied -variant nvx::exec::Receipt::Late -variant nvx::exec::Receipt::Stale -struct nvx::exec::Runner -method nvx::exec::Runner::awaits_settlement -method nvx::exec::Runner::buffered -method nvx::exec::Runner::cancel -method nvx::exec::Runner::credit -method nvx::exec::Runner::deadline -method nvx::exec::Runner::discarded -method nvx::exec::Runner::disconnect -method nvx::exec::Runner::dropped -method nvx::exec::Runner::end_input -method nvx::exec::Runner::exited -method nvx::exec::Runner::high_water -method nvx::exec::Runner::input -method nvx::exec::Runner::input_closed -method nvx::exec::Runner::input_written -method nvx::exec::Runner::is_detached -method nvx::exec::Runner::is_finished -method nvx::exec::Runner::key -method nvx::exec::Runner::launched -method nvx::exec::Runner::limits -method nvx::exec::Runner::memory_bound -method nvx::exec::Runner::new -method nvx::exec::Runner::next_message -method nvx::exec::Runner::next_request -method nvx::exec::Runner::output -method nvx::exec::Runner::output_closed -method nvx::exec::Runner::output_failed -method nvx::exec::Runner::output_hung_up -method nvx::exec::Runner::pending_input -method nvx::exec::Runner::read_allowance -method nvx::exec::Runner::reset -method nvx::exec::Runner::settled -method nvx::exec::Runner::tick -enum nvx::exec::Settlement -variant nvx::exec::Settlement::Failed -variant nvx::exec::Settlement::Settled -enum nvx::exec::Status -variant nvx::exec::Status::Exited -field nvx::exec::Status::Exited::0 -variant nvx::exec::Status::Signaled -field nvx::exec::Status::Signaled::0 -enum nvx::exec::Stream -const nvx::exec::Stream::ALL -variant nvx::exec::Stream::Stderr -variant nvx::exec::Stream::Stdout -struct nvx::exec::Timers -method nvx::exec::Timers::grace -method nvx::exec::Timers::new -method nvx::exec::Timers::settlement -method nvx::exec::Timers::timeout -method nvx::exec::Timers::timeout_duration -enum nvx::exec::Unit -variant nvx::exec::Unit::Bytes -variant nvx::exec::Unit::Frames -enum nvx::exec::Violation -variant nvx::exec::Violation::Chunk -variant nvx::exec::Violation::Credit -variant nvx::exec::Violation::Ended -variant nvx::exec::Violation::Key -variant nvx::exec::Violation::Order -variant nvx::exec::Violation::Pool -variant nvx::exec::Violation::Window -struct nvx::exec::Window -method nvx::exec::Window::bytes -method nvx::exec::Window::frames -method nvx::exec::Window::size -method nvx::exec::Window::unit -mod nvx::exec::peer -enum nvx::exec::peer::Event -variant nvx::exec::peer::Event::End -field nvx::exec::peer::Event::End::0 -variant nvx::exec::peer::Event::Finished -field nvx::exec::peer::Event::Finished::0 -variant nvx::exec::peer::Event::InputClosed -variant nvx::exec::peer::Event::Started -enum nvx::exec::peer::Message -variant nvx::exec::peer::Message::Cancel -field nvx::exec::peer::Message::Cancel::0 -variant nvx::exec::peer::Message::Credit -field nvx::exec::peer::Message::Credit::pool -field nvx::exec::peer::Message::Credit::units -variant nvx::exec::peer::Message::Input -field nvx::exec::peer::Message::Input::length -variant nvx::exec::peer::Message::InputEnd -struct nvx::exec::peer::Peer -method nvx::exec::peer::Peer::buffered -method nvx::exec::peer::Peer::cancel -method nvx::exec::peer::Peer::consumed -method nvx::exec::peer::Peer::credit -method nvx::exec::peer::Peer::deadline -method nvx::exec::peer::Peer::disconnect -method nvx::exec::peer::Peer::dropped -method nvx::exec::peer::Peer::end_input -method nvx::exec::peer::Peer::end_output -method nvx::exec::peer::Peer::has_started -method nvx::exec::peer::Peer::high_water -method nvx::exec::peer::Peer::input -method nvx::exec::peer::Peer::input_closed -method nvx::exec::peer::Peer::is_detached -method nvx::exec::peer::Peer::is_finished -method nvx::exec::peer::Peer::key -method nvx::exec::peer::Peer::limits -method nvx::exec::peer::Peer::memory_bound -method nvx::exec::peer::Peer::new -method nvx::exec::peer::Peer::next_event -method nvx::exec::peer::Peer::next_message -method nvx::exec::peer::Peer::output -method nvx::exec::peer::Peer::pending_output -method nvx::exec::peer::Peer::reset -method nvx::exec::peer::Peer::started -method nvx::exec::peer::Peer::terminal -method nvx::exec::peer::Peer::tick -method nvx::exec::peer::Peer::write_allowance -mod nvx::exec::runner -enum nvx::exec::runner::Message -variant nvx::exec::runner::Message::Credit -field nvx::exec::runner::Message::Credit::units -variant nvx::exec::runner::Message::End -field nvx::exec::runner::Message::End::0 -variant nvx::exec::runner::Message::InputClosed -variant nvx::exec::runner::Message::Output -field nvx::exec::runner::Message::Output::length -field nvx::exec::runner::Message::Output::stream -variant nvx::exec::runner::Message::Started -variant nvx::exec::runner::Message::Terminal -field nvx::exec::runner::Message::Terminal::0 -enum nvx::exec::runner::Request -variant nvx::exec::runner::Request::CloseInput -variant nvx::exec::runner::Request::Discard -field nvx::exec::runner::Request::Discard::0 -variant nvx::exec::runner::Request::Settle -variant nvx::exec::runner::Request::Terminate -field nvx::exec::runner::Request::Terminate::0 -struct nvx::exec::runner::Runner -method nvx::exec::runner::Runner::awaits_settlement -method nvx::exec::runner::Runner::buffered -method nvx::exec::runner::Runner::cancel -method nvx::exec::runner::Runner::credit -method nvx::exec::runner::Runner::deadline -method nvx::exec::runner::Runner::discarded -method nvx::exec::runner::Runner::disconnect -method nvx::exec::runner::Runner::dropped -method nvx::exec::runner::Runner::end_input -method nvx::exec::runner::Runner::exited -method nvx::exec::runner::Runner::high_water -method nvx::exec::runner::Runner::input -method nvx::exec::runner::Runner::input_closed -method nvx::exec::runner::Runner::input_written -method nvx::exec::runner::Runner::is_detached -method nvx::exec::runner::Runner::is_finished -method nvx::exec::runner::Runner::key -method nvx::exec::runner::Runner::launched -method nvx::exec::runner::Runner::limits -method nvx::exec::runner::Runner::memory_bound -method nvx::exec::runner::Runner::new -method nvx::exec::runner::Runner::next_message -method nvx::exec::runner::Runner::next_request -method nvx::exec::runner::Runner::output -method nvx::exec::runner::Runner::output_closed -method nvx::exec::runner::Runner::output_failed -method nvx::exec::runner::Runner::output_hung_up -method nvx::exec::runner::Runner::pending_input -method nvx::exec::runner::Runner::read_allowance -method nvx::exec::runner::Runner::reset -method nvx::exec::runner::Runner::settled -method nvx::exec::runner::Runner::tick mod nvx::gpt struct nvx::gpt::Disk method nvx::gpt::Disk::first_partition diff --git a/nvx-rs/src/exec/credit.rs b/nvx-rs/src/exec/credit.rs deleted file mode 100644 index 623418253..000000000 --- a/nvx-rs/src/exec/credit.rs +++ /dev/null @@ -1,35 +0,0 @@ -//! The credit arithmetic of an execution's pools, and the bounded queues of its streams. -//! -//! A sender holds [`Credit`]: the units of a pool that the receiver granted it and that it has -//! not used, which it spends on each data message it sends. A receiver holds [`Granted`]: the -//! units that it granted and that the sender has not used yet, which bound each data message it -//! accepts, and the units that its consumer freed, which it grants again. Both start from the -//! pool's window: the receiver's first grant opens the whole window, and a grant can never open -//! more than the window, so the units in flight never exceed it. - -mod inbox; -mod receiver; -mod sender; - -use std::collections::VecDeque; - -pub(super) use self::inbox::Inbox; -pub(super) use self::receiver::Granted; -pub(super) use self::sender::Credit; - -/// Returns `count` bytes as units of a byte pool, which a chunk's bound keeps within `u32`. -pub(super) fn units(count: usize) -> u32 { - u32::try_from(count).unwrap_or(u32::MAX) -} - -/// Moves the first bytes of `queue` into `buffer`, at most `length` of them, and returns how -/// many it moved. -pub(super) fn take(queue: &mut VecDeque, buffer: &mut [u8], length: usize) -> usize { - let length = length.min(buffer.len()).min(queue.len()); - let (first, second) = queue.as_slices(); - let head = length.min(first.len()); - buffer[..head].copy_from_slice(&first[..head]); - buffer[head..length].copy_from_slice(&second[..length - head]); - queue.drain(..length); - length -} diff --git a/nvx-rs/src/exec/credit/inbox.rs b/nvx-rs/src/exec/credit/inbox.rs deleted file mode 100644 index da8a44c07..000000000 --- a/nvx-rs/src/exec/credit/inbox.rs +++ /dev/null @@ -1,90 +0,0 @@ -//! The receiver's side of a stream: see the [module](super) documentation. - -use std::collections::VecDeque; - -use super::units; -use crate::exec::Unit; - -/// The bytes that one stream's receiver holds for its consumer, in order, with the bounds of the -/// frames that they arrived in when the pool counts frames. -#[derive(Debug)] -pub(in crate::exec) struct Inbox { - /// What the pool counts. - unit: Unit, - /// The bytes, in order. - bytes: VecDeque, - /// The bytes left of each frame that the inbox holds, oldest first, when the pool counts - /// frames. - frames: VecDeque, -} - -impl Inbox { - /// Returns an empty inbox of a pool that counts `unit`. - pub(in crate::exec) fn new(unit: Unit) -> Self { - Self { - unit, - bytes: VecDeque::new(), - frames: VecDeque::new(), - } - } - - /// Returns the number of bytes held. - pub(in crate::exec) fn len(&self) -> usize { - self.bytes.len() - } - - /// Returns whether the inbox holds nothing. - pub(in crate::exec) fn is_empty(&self) -> bool { - self.bytes.is_empty() - } - - /// Appends one data message, which its pool charged. - pub(in crate::exec) fn push(&mut self, bytes: &[u8]) { - self.bytes.extend(bytes); - if self.unit == Unit::Frames { - self.frames.push_back(bytes.len()); - } - } - - /// Returns the bytes at the front, as many as are contiguous: a part of what the inbox holds - /// while it holds anything. - pub(in crate::exec) fn front(&self) -> &[u8] { - self.bytes.as_slices().0 - } - - /// Removes the first `count` bytes, at most as many as the inbox holds, and returns the units - /// that this frees: each byte, or each frame whose last byte it removes. - pub(in crate::exec) fn pop(&mut self, count: usize) -> u32 { - let count = count.min(self.bytes.len()); - self.bytes.drain(..count); - match self.unit { - Unit::Bytes => units(count), - Unit::Frames => { - let mut left = count; - let mut freed = 0; - while left > 0 { - let Some(frame) = self.frames.front_mut() else { - break; - }; - let taken = left.min(*frame); - *frame -= taken; - left -= taken; - if *frame == 0 { - self.frames.pop_front(); - freed += 1; - } - } - freed - } - } - } - - /// Removes everything, releases the storage, and returns how many bytes it removed and the - /// units that this frees. - pub(in crate::exec) fn clear(&mut self) -> (usize, u32) { - let length = self.bytes.len(); - let freed = self.pop(length); - self.bytes = VecDeque::new(); - (length, freed) - } -} diff --git a/nvx-rs/src/exec/credit/receiver.rs b/nvx-rs/src/exec/credit/receiver.rs deleted file mode 100644 index 154269780..000000000 --- a/nvx-rs/src/exec/credit/receiver.rs +++ /dev/null @@ -1,49 +0,0 @@ -//! The receiver's side of a pool: see the [module](super) documentation. - -use crate::exec::{Violation, Window}; - -/// What the receiver of one pool granted, and what its consumer freed. -#[derive(Debug)] -pub(in crate::exec) struct Granted { - /// The pool's window. - window: Window, - /// The units that the sender may still send: granted, and not used yet. - open: u32, - /// The units that the consumer freed, or that the receiver never granted, which it grants - /// next. - returnable: u32, -} - -impl Granted { - /// Returns the window of a pool before its first grant, which opens the whole window. - pub(in crate::exec) fn new(window: Window) -> Self { - Self { - window, - open: 0, - returnable: window.size(), - } - } - - /// Charges a data message of `length` bytes, which the caller checked against the maximum - /// chunk. A message beyond the units that remain open is refused and changes nothing. - pub(in crate::exec) fn charge(&mut self, length: usize) -> Result<(), Violation> { - let cost = self.window.cost(length); - if cost > self.open { - return Err(Violation::Window); - } - self.open -= cost; - Ok(()) - } - - /// Frees `units` that the consumer took, to grant them again. - pub(in crate::exec) fn free(&mut self, units: u32) { - self.returnable += units; - } - - /// Returns the units to grant now, if any, and opens them. - pub(in crate::exec) fn take_grant(&mut self) -> Option { - let units = std::mem::take(&mut self.returnable); - self.open += units; - (units > 0).then_some(units) - } -} diff --git a/nvx-rs/src/exec/credit/sender.rs b/nvx-rs/src/exec/credit/sender.rs deleted file mode 100644 index c30e77864..000000000 --- a/nvx-rs/src/exec/credit/sender.rs +++ /dev/null @@ -1,50 +0,0 @@ -//! The sender's side of a pool: see the [module](super) documentation. - -use crate::exec::{Unit, Violation, Window}; - -/// The credit that a receiver granted the sender of one pool. -#[derive(Debug)] -pub(in crate::exec) struct Credit { - /// The pool's window. - window: Window, - /// The units that the sender may still send. - available: u32, -} - -impl Credit { - /// Returns the credit of a pool with `window`, before the receiver's first grant. - pub(in crate::exec) fn new(window: Window) -> Self { - Self { - window, - available: 0, - } - } - - /// Adds a grant of `units`. A grant of no units, or of more than the units in flight, which - /// would leave the sender more than the window, is refused and changes nothing. - pub(in crate::exec) fn grant(&mut self, units: u32) -> Result<(), Violation> { - if units == 0 - || u64::from(self.available) + u64::from(units) > u64::from(self.window.size()) - { - return Err(Violation::Credit); - } - self.available += units; - Ok(()) - } - - /// Returns how many of the first `length` queued bytes one data message may carry now: as - /// many as the bytes of credit, or all of them while a frame of credit is left. - pub(in crate::exec) fn sendable(&self, length: usize) -> usize { - match self.window.unit() { - Unit::Bytes => length.min(self.available as usize), - Unit::Frames if self.available > 0 => length, - Unit::Frames => 0, - } - } - - /// Spends the credit of one data message of `length` bytes, which - /// [`sendable`](Self::sendable) allowed. - pub(in crate::exec) fn spend(&mut self, length: usize) { - self.available -= self.window.cost(length); - } -} diff --git a/nvx-rs/src/exec/mod.rs b/nvx-rs/src/exec/mod.rs deleted file mode 100644 index 739bf0ece..000000000 --- a/nvx-rs/src/exec/mod.rs +++ /dev/null @@ -1,722 +0,0 @@ -//! The credit and stdio engine of an execution, which every adapter shares, whatever its framing. -//! -//! An execution runs a workload on one side, the [`Runner`], for another side, the [`Peer`], -//! which sends its standard input and takes its standard output and error. The engine owns what -//! both sides must agree on: the state of each stream, the credit arithmetic that bounds the -//! bytes in flight, the order of each stream's data and end, and the execution's outcome. The -//! adapter owns everything else: its framing and transport, which messages it puts in one frame, -//! its admission of executions, and the processes, which a guest's adapter drives through -//! `guest::exec::binding` (Linux, feature `guest`). -//! -//! The engine needs only the standard library: no feature enables it, and it has no wire format, -//! async runtime, threads, global state, admission, or reaper. Each side is a state machine that -//! never blocks and never waits: the adapter hands it each input as it arrives, with the current -//! [`Instant`] where time matters, retrieves what it must do, one bounded item at a time, and -//! [ticks](Runner::tick) it once its [deadline](Runner::deadline) passes. So a stream, a sink, a -//! window, or a transport that blocks holds back only the bytes that wait on it, never a timer, a -//! cancellation, a reset, or another execution, and tests drive both sides deterministically. -//! -//! # Messages -//! -//! The two sides exchange these messages, each of which the adapter frames and tags with the -//! execution's [`Key`]: -//! -//! | From | Message | Meaning | -//! | --- | --- | --- | -//! | Peer | [credit](peer::Message::Credit) for output | the peer's consumer freed output units | -//! | Peer | [input](peer::Message::Input) | bytes of the workload's standard input | -//! | Peer | [end of input](peer::Message::InputEnd) | the input ends after its last byte | -//! | Peer | [cancel](peer::Message::Cancel) | end the workload | -//! | Runner | [credit](runner::Message::Credit) for input | the workload took input units | -//! | Runner | [started](runner::Message::Started) | the workload's program runs | -//! | Runner | [output](runner::Message::Output) | bytes of standard output or error | -//! | Runner | [end](runner::Message::End) | the stream ends after its last byte | -//! | Runner | [input closed](runner::Message::InputClosed) | the workload's input closed early | -//! | Runner | [terminal](runner::Message::Terminal) | the execution's [`Outcome`] | -//! -//! A message for an earlier generation of the execution's key is [stale](Receipt::Stale), and one -//! that the receiving side no longer needs, such as a credit after the execution ended, or any -//! message of the runner's once the peer detached from it or gave up waiting for it, is -//! [late](Receipt::Late): neither changes anything. A message that breaks the rules, such as data -//! beyond the window, is a [`Violation`], which changes nothing either, and after which the -//! adapter decides what to do, such as reset its session. -//! -//! # Credit -//! -//! Each data message carries at most [`Limits::chunk`] bytes, and costs credit in its pool: as -//! many units as its bytes, or one unit whatever its length, as the pool's [`Window`] counts. The -//! adapter chooses the pools: the input's, and either one window for each output stream, or one -//! pool that standard output and error [share](Limits::shared_output). Each receiver grants its -//! whole window first, then each unit again once its consumer took the unit's bytes: the runner -//! once the workload's input took them, the peer once its sink took them. A receiver refuses data -//! beyond what it granted, and a sender refuses a grant that would leave it more than the window, -//! so at most a window of units is ever in flight, and each side's buffers stay bounded, with no -//! limit on a stream's total length: see [`Runner::memory_bound`] and [`Peer::memory_bound`]. -//! -//! A sender queues at most one chunk of each stream ahead of the credit, which the runner reads -//! from the workload as its [read allowance](Runner::read_allowance) lets it, and the peer takes -//! from its source as its [write allowance](Peer::write_allowance) does. Data leaves the queue as -//! the adapter retrieves it, as one message of whatever the queue holds, up to a chunk: an -//! adapter whose transport is busy retrieves less often, and so sends fewer, fuller messages. -//! -//! # Completion -//! -//! - The workload's input ends only after every byte that the runner accepted: the runner -//! requests to [close it](runner::Request::CloseInput) once the peer ended it and the workload -//! took the rest. A workload that closes its input early, or ends, makes the runner tell the -//! peer that the input [closed](runner::Message::InputClosed), and drop what the peer sends -//! after, so that the peer's source never waits. Closing the input never closes the other -//! direction. -//! - Each stream keeps its order, and its end follows its last byte. A stream's end goes ahead -//! of any later output of the other stream, which never holds it back; otherwise, the streams' -//! messages have no order between them. The runner sends the terminal outcome after both -//! streams' ends. An end does not report whether the runner lost bytes; only the terminal -//! outcome does. A peer that detaches or gives up without that outcome conservatively marks -//! both streams as truncated, even if their ends arrived. -//! - A cancellation, a timeout, a drop, a reset, or a disconnection proceeds whatever the streams -//! do. The runner requests to [end](runner::Request::Terminate) the workload, and pending output -//! still flows for [`Timers`]'s grace, then the rest is discarded, and the terminal outcome -//! marks each stream that lost bytes as [truncated](Outcome::is_truncated). A reset, a drop, or -//! a disconnection of the runner leaves no peer to deliver to, so the runner discards at once. -//! A workload that ends on its own, with no such cause, loses no byte, however slowly the peer -//! takes them, if its execution settles, or if no process holds its streams any more, as the -//! adapter tells the runner once they [hung up](Runner::output_hung_up). -//! - The logical outcome, the delivery of the output, and the settlement of the execution are -//! separate. Once the workload's status is known, the runner requests to -//! [settle](runner::Request::Settle) its processes, whose result the adapter reports once its -//! own check shows that the workload's descendants, and every process that holds one of its -//! pipes, are gone; it may stop checking once the runner no longer -//! [awaits](Runner::awaits_settlement) the result. The workload's exit, a signal to its process -//! group, or a peer's acknowledgement never counts. The terminal outcome follows the status, -//! the delivery, and the settlement, and says whether the execution is -//! [reusable](Outcome::is_reusable) or fenced: a failed or late settlement fences it. -//! - Each side bounds every wait: a termination gives the workload its grace and then -//! [`Timers`]'s settlement time to end, close its streams, and settle, after which the runner -//! gives up on an execution whose workload has not, and fences it, while one whose cleanup is -//! complete waits only for the transport to take its last messages; a workload that ended on -//! its own, but whose execution did not settle, leaves its streams the settlement time and then -//! the grace after its end to close or hang up, after which the runner gives up on those still -//! open that a process may hold, sends what it read of them, and marks them truncated; a launch -//! result that has not arrived by then fences the execution, whether or not it settled, as a -//! process may hold what reports it too, and the runner gives up on the result, and on those -//! streams, and sends the workload's output and outcome without it; a peer that ends the -//! execution itself waits as long for the terminal outcome, after which it gives up and fences -//! it too; and a peer that receives a terminal outcome with a cause, without having ended the -//! execution itself, gives its sink the grace from then on to take the output that it holds, -//! and discards the rest. -//! -//! # Example -//! -//! A peer and a runner, connected directly, as an adapter's framing and transport would connect -//! them: -//! -//! ``` -//! use std::time::{Duration, Instant}; -//! -//! use nvx::exec::{ -//! peer, runner, Key, Limits, Peer, Runner, Settlement, Status, Stream, Timers, Window, -//! }; -//! -//! let now = Instant::now(); -//! let key = Key::new(1, 0); -//! let window = Window::bytes(64 * 1024); -//! let limits = Limits::new(16 * 1024, window, window).unwrap(); -//! let timers = Timers::new(Duration::from_secs(1), Duration::from_secs(5)); -//! let mut peer = Peer::new(key, limits, timers, now); -//! let mut runner = Runner::new(key, limits, timers, now); -//! let mut buffer = vec![0u8; limits.chunk()]; -//! -//! // The workload runs, writes to its standard output, and exits; its processes settle. -//! runner.launched(Ok(())); -//! assert_eq!(runner.output(Stream::Stdout, b"hello"), 5); -//! runner.output_closed(Stream::Stdout); -//! runner.output_closed(Stream::Stderr); -//! runner.exited(now, Some(Status::Exited(0))); -//! assert_eq!(runner.next_request(), Some(runner::Request::Settle)); -//! runner.settled(Settlement::Settled); -//! -//! // The adapter carries each side's messages to the other until the peer finishes. -//! let mut stdout = Vec::new(); -//! let outcome = loop { -//! while let Some(message) = peer.next_message(&mut buffer) { -//! match message { -//! peer::Message::Credit { pool, units } => { -//! runner.credit(key, pool, units).unwrap(); -//! } -//! message => unreachable!("{message:?}"), -//! } -//! } -//! while let Some(message) = runner.next_message(&mut buffer) { -//! match message { -//! runner::Message::Credit { units } => { -//! peer.credit(key, units).unwrap(); -//! } -//! runner::Message::Started => { -//! peer.started(key).unwrap(); -//! } -//! runner::Message::Output { stream, length } => { -//! peer.output(key, stream, &buffer[..length]).unwrap(); -//! } -//! runner::Message::End(stream) => { -//! peer.end_output(key, stream).unwrap(); -//! } -//! runner::Message::Terminal(outcome) => { -//! peer.terminal(now, key, outcome).unwrap(); -//! } -//! runner::Message::InputClosed => { -//! peer.input_closed(key).unwrap(); -//! } -//! message => unreachable!("{message:?}"), -//! } -//! } -//! // The peer's sink takes what arrived. -//! let pending = peer.pending_output(Stream::Stdout).to_vec(); -//! stdout.extend_from_slice(&pending); -//! peer.consumed(Stream::Stdout, pending.len()); -//! let last = std::iter::from_fn(|| peer.next_event()).last(); -//! if let Some(peer::Event::Finished(outcome)) = last { -//! break outcome; -//! } -//! }; -//! assert_eq!(stdout, b"hello"); -//! assert_eq!(outcome.status(), Some(Status::Exited(0))); -//! assert!(outcome.is_reusable() && !outcome.is_truncated(Stream::Stdout)); -//! ``` - -use std::fmt; -use std::time::{Duration, Instant}; - -mod credit; -pub mod peer; -pub mod runner; - -#[cfg(test)] -mod tests; - -pub use peer::Peer; -pub use runner::Runner; - -/// The identity of one execution: an identifier that the adapter assigns, such as a request ID or -/// a stream slot, and the generation of that identifier, which tells a later execution with the -/// same identifier, or the same execution in a later session, from an earlier one. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)] -pub struct Key { - /// The execution's identifier. - id: u64, - /// The identifier's generation. - generation: u64, -} - -impl Key { - /// Returns the key of execution `id` in `generation`. - pub const fn new(id: u64, generation: u64) -> Self { - Self { id, generation } - } - - /// Returns the execution's identifier. - pub const fn id(&self) -> u64 { - self.id - } - - /// Returns the identifier's generation. - pub const fn generation(&self) -> u64 { - self.generation - } - - /// Checks the key of a message for this execution: `Ok(true)` for its own generation, - /// `Ok(false)` for an earlier one, whose messages are stale, and [`Violation::Key`] for - /// another identifier or a later generation. - fn admits(self, message: Key) -> Result { - if message.id != self.id { - return Err(Violation::Key); - } - match message.generation.cmp(&self.generation) { - std::cmp::Ordering::Less => Ok(false), - std::cmp::Ordering::Equal => Ok(true), - std::cmp::Ordering::Greater => Err(Violation::Key), - } - } -} - -/// One of the workload's output streams. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] -pub enum Stream { - /// The workload's standard output. - Stdout, - /// The workload's standard error. - Stderr, -} - -impl Stream { - /// Both streams, in the order in which the sides first serve them. - pub const ALL: [Self; 2] = [Self::Stdout, Self::Stderr]; - - /// Returns the stream's position in [`ALL`](Self::ALL). - const fn index(self) -> usize { - match self { - Self::Stdout => 0, - Self::Stderr => 1, - } - } -} - -/// A pool of credit, which a [credit message](peer::Message::Credit) names. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] -pub enum Pool { - /// The workload's standard input. - Input, - /// The workload's standard output, which has a pool of its own unless the output streams - /// [share](Limits::shared_output) one. - Stdout, - /// The workload's standard error, which has a pool of its own unless the output streams share - /// one. - Stderr, - /// The one pool that standard output and error [share](Limits::shared_output). - Output, -} - -/// What a pool's credit counts. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] -pub enum Unit { - /// Data bytes: a data message costs as many units as its bytes. - Bytes, - /// Data messages: a data message costs one unit, whatever its length up to a chunk. - Frames, -} - -/// The size of a pool's window: the most units that may be in flight at once. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] -pub struct Window { - /// What the window counts. - unit: Unit, - /// The units of the window. - size: u32, -} - -impl Window { - /// Returns a window of `size` bytes. - pub const fn bytes(size: u32) -> Self { - Self { - unit: Unit::Bytes, - size, - } - } - - /// Returns a window of `size` data messages. - pub const fn frames(size: u32) -> Self { - Self { - unit: Unit::Frames, - size, - } - } - - /// Returns what the window counts. - pub const fn unit(&self) -> Unit { - self.unit - } - - /// Returns the units of the window. - pub const fn size(&self) -> u32 { - self.size - } - - /// Returns the units that a data message of `length` bytes costs. - fn cost(&self, length: usize) -> u32 { - match self.unit { - Unit::Bytes => credit::units(length), - Unit::Frames => 1, - } - } - - /// Returns the most data bytes that the window holds with chunks of `chunk` bytes. - fn bytes_with(&self, chunk: usize) -> usize { - match self.unit { - Unit::Bytes => self.size as usize, - Unit::Frames => (self.size as usize).saturating_mul(chunk), - } - } -} - -/// The limits on which both sides of an execution agree: the maximum chunk, the input's window, -/// and the output's windows, one for each stream unless the streams share one pool. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] -pub struct Limits { - /// The most data bytes that one data message carries. - chunk: u32, - /// The window of the workload's standard input. - input: Window, - /// The window of each output pool. - output: Window, - /// Whether standard output and error share one pool. - shared: bool, -} - -impl Limits { - /// Returns limits with data messages of at most `chunk` bytes, the input's `input` window, - /// and one `output` window for each output stream, or `None` if the chunk or a window is - /// empty. - pub const fn new(chunk: u32, input: Window, output: Window) -> Option { - if chunk == 0 || input.size == 0 || output.size == 0 { - return None; - } - Some(Self { - chunk, - input, - output, - shared: false, - }) - } - - /// Makes standard output and error share one pool with the output window, which both - /// streams' data messages charge. - pub const fn shared_output(mut self) -> Self { - self.shared = true; - self - } - - /// Returns the most data bytes that one data message carries. - pub const fn chunk(&self) -> usize { - self.chunk as usize - } - - /// Returns the window of the workload's standard input. - pub const fn input(&self) -> Window { - self.input - } - - /// Returns the window of each output pool. - pub const fn output(&self) -> Window { - self.output - } - - /// Returns whether standard output and error share one pool. - pub const fn is_shared(&self) -> bool { - self.shared - } - - /// Returns the output pools, in the order of their positions. - fn output_pools(&self) -> &'static [Pool] { - if self.shared { - &[Pool::Output] - } else { - &[Pool::Stdout, Pool::Stderr] - } - } - - /// Returns the position of `stream`'s output pool. - fn pool_of(&self, stream: Stream) -> usize { - if self.shared { 0 } else { stream.index() } - } - - /// Returns the position of the output pool `pool`, or [`Violation::Pool`] if the limits have - /// no such output pool. - fn output_pool(&self, pool: Pool) -> Result { - match (pool, self.shared) { - (Pool::Output, true) => Ok(0), - (Pool::Stdout, false) => Ok(0), - (Pool::Stderr, false) => Ok(1), - _ => Err(Violation::Pool), - } - } - - /// Returns the checked length of a data message of `length` bytes: [`Violation::Chunk`] if - /// it is empty or longer than a chunk. - fn check_chunk(&self, length: usize) -> Result<(), Violation> { - if length == 0 || length > self.chunk() { - return Err(Violation::Chunk); - } - Ok(()) - } - - /// Returns the most data bytes that the input's window holds. - fn input_bytes(&self) -> usize { - self.input.bytes_with(self.chunk()) - } - - /// Returns the most data bytes that the output's windows hold together. - fn output_bytes(&self) -> usize { - let pools = if self.shared { 1 } else { 2 }; - self.output.bytes_with(self.chunk()).saturating_mul(pools) - } -} - -/// The local timers of one side of an execution. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] -pub struct Timers { - /// How long the execution may run before the side ends it, if it ever does. - timeout: Option, - /// How long pending output still flows once a termination begins, and how long the output - /// streams of an execution that did not settle may stay open after the settlement time. - grace: Duration, - /// How long the execution may then take to settle. - settlement: Duration, -} - -impl Timers { - /// Returns timers with no timeout, a `grace` for pending output once a termination begins, - /// and a `settlement` time after it, or after the workload's end, within which the runner - /// expects the execution to settle, and the peer expects the terminal outcome of an execution - /// that it ended. The output streams of a workload that ended on its own, but whose - /// execution did not settle, may stay open for the grace after the settlement time. - pub const fn new(grace: Duration, settlement: Duration) -> Self { - Self { - timeout: None, - grace, - settlement, - } - } - - /// Ends the execution for a [timeout](Cause::TimedOut) once `timeout` has passed since the - /// side was created: the runner, unless the workload's end reaches it before a - /// [tick](Runner::tick) fires the timeout, and the peer, if the execution has not finished. - pub const fn timeout(mut self, timeout: Duration) -> Self { - self.timeout = Some(timeout); - self - } - - /// Returns the timeout, if any. - pub const fn timeout_duration(&self) -> Option { - self.timeout - } - - /// Returns the grace of pending output. - pub const fn grace(&self) -> Duration { - self.grace - } - - /// Returns the settlement time. - pub const fn settlement(&self) -> Duration { - self.settlement - } -} - -/// Why a side ended an execution. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] -#[non_exhaustive] -pub enum Cause { - /// The peer, or its caller, cancelled the execution. - Cancelled, - /// The execution's owner dropped it: the peer's caller dropped its handle, or the runner's - /// adapter abandoned the execution. - Dropped, - /// The execution's timeout passed. - TimedOut, - /// The session between the sides was reset. - Reset, - /// The transport between the sides was lost. - Disconnected, -} - -impl fmt::Display for Cause { - fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { - formatter.write_str(match self { - Self::Cancelled => "cancelled", - Self::Dropped => "dropped", - Self::TimedOut => "timed out", - Self::Reset => "reset", - Self::Disconnected => "disconnected", - }) - } -} - -/// The status with which a workload's program ended, as its collector reaped it. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] -pub enum Status { - /// The program exited with this code. - Exited(i32), - /// A signal of this number ended the program. - Signaled(i32), -} - -/// Why a workload's program did not start: the adapter's number of the launch stage that failed, -/// and the operating system's error number of the failure. -/// -/// NVX's guest binding, `guest::exec::binding` (Linux, feature `guest`), numbers the stages as -/// `guest::spawn::LaunchStage::code` does, which `LaunchStage::from_code` reverses, and reports 0 -/// for a launch whose report it could not read. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] -pub struct LaunchFailure { - /// The adapter's number of the stage that failed. - stage: u32, - /// The error number of the failure. - error: i32, -} - -impl LaunchFailure { - /// Returns the failure of the stage that the adapter numbers `stage`, with the error number - /// `error`. - pub const fn new(stage: u32, error: i32) -> Self { - Self { stage, error } - } - - /// Returns the adapter's number of the stage that failed. - pub const fn stage(&self) -> u32 { - self.stage - } - - /// Returns the error number of the failure. - pub const fn error(&self) -> i32 { - self.error - } -} - -/// The result of settling an execution's processes, which the runner's adapter reports. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] -pub enum Settlement { - /// No process of the execution is left: the workload, its descendants, and every process - /// that held one of its pipes are gone. - Settled, - /// The adapter could not settle the execution, such as because a kill failed. - Failed, -} - -/// How an execution ended: the runner's terminal outcome, or the peer's own if none arrived. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] -pub struct Outcome { - /// The workload's status, if a collector reaped it. - status: Option, - /// Why a side ended the workload, if one did. - cause: Option, - /// Why the workload's program did not start, if it did not. - launch_failure: Option, - /// Whether each stream lost bytes, or the peer could not confirm that it ended whole. - truncated: [bool; 2], - /// Whether the execution settled, so that its resources may be reused. - reusable: bool, -} - -impl Outcome { - /// Returns the outcome of a workload that ended with `status`, if known, with no cause, no - /// launch failure, no truncation, and fenced: the form in which a peer's adapter decodes a - /// terminal record, before it applies the record's other fields. - pub const fn new(status: Option) -> Self { - Self { - status, - cause: None, - launch_failure: None, - truncated: [false, false], - reusable: false, - } - } - - /// Returns the outcome with `cause` as why a side ended the workload. - pub const fn with_cause(mut self, cause: Cause) -> Self { - self.cause = Some(cause); - self - } - - /// Returns the outcome with `failure` as why the workload's program did not start. - pub const fn with_launch_failure(mut self, failure: LaunchFailure) -> Self { - self.launch_failure = Some(failure); - self - } - - /// Returns the outcome with `stream` marked as truncated. - pub const fn with_truncated(mut self, stream: Stream) -> Self { - self.truncated[stream.index()] = true; - self - } - - /// Returns the outcome of an execution that settled, and so is reusable. - pub const fn with_reusable(mut self) -> Self { - self.reusable = true; - self - } - - /// Returns the workload's status, or `None` if no collector reaped it, as when its collector - /// lost it or a side gave up on the execution. - pub const fn status(&self) -> Option { - self.status - } - - /// Returns why a side ended the workload, or `None` if it ended on its own. A workload with a - /// cause may still have exited on its own before the side's end reached it, which its status - /// tells. - pub const fn cause(&self) -> Option { - self.cause - } - - /// Returns why the workload's program did not start, or `None` if it started, or if no - /// launch result arrived. - pub const fn launch_failure(&self) -> Option { - self.launch_failure - } - - /// Returns whether `stream` lost bytes, or the peer could not confirm that it ended whole. - /// A peer that finishes without the runner's terminal outcome conservatively marks both - /// streams as truncated, even if their ends arrived. - pub const fn is_truncated(&self, stream: Stream) -> bool { - self.truncated[stream.index()] - } - - /// Returns whether the execution settled, so that its resources may be reused, or `false` if - /// it is fenced. - pub const fn is_reusable(&self) -> bool { - self.reusable - } -} - -/// What the side did with a message that it accepted. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] -#[non_exhaustive] -pub enum Receipt { - /// The message changed the side's state. - Applied, - /// The message belongs to an earlier generation of the execution's key, and changed nothing. - Stale, - /// The side no longer needed the message, as for a credit after the execution ended, input - /// after the workload's input closed, or any message of the runner's once the peer detached - /// from it or gave up waiting for it: it changed nothing beyond the window it charged. - Late, -} - -/// A message that breaks the rules, which the side refuses without changing anything. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] -#[non_exhaustive] -pub enum Violation { - /// The message's key names another execution, or a later generation of this one. - Key, - /// A credit names a pool that the execution's limits do not have. - Pool, - /// A credit grants no units, or more units than are in flight, which would leave its sender - /// more than the window. - Credit, - /// A data message is empty, or longer than a chunk. - Chunk, - /// A data message exceeds the credit that its receiver granted. - Window, - /// Data or an end follows the stream's end, or a second terminal outcome follows the first. - Ended, - /// A message comes out of order, such as a second start, or a terminal outcome before both - /// streams' ends. - Order, -} - -impl fmt::Display for Violation { - fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { - formatter.write_str(match self { - Self::Key => "the message names another execution or a later generation", - Self::Pool => "the credit names a pool that the execution does not have", - Self::Credit => "the credit grants no units or more units than are in flight", - Self::Chunk => "the data message is empty or longer than a chunk", - Self::Window => "the data message exceeds the granted credit", - Self::Ended => "the message follows the end of its stream or of the execution", - Self::Order => "the message comes out of order", - }) - } -} - -impl std::error::Error for Violation {} - -/// Returns the instant `duration` after `now`, or `None` if no instant that late exists. -fn after(now: Instant, duration: Duration) -> Option { - now.checked_add(duration) -} - -/// Returns the earlier of two optional instants. -fn earliest(first: Option, second: Option) -> Option { - match (first, second) { - (Some(first), Some(second)) => Some(first.min(second)), - (first, second) => first.or(second), - } -} diff --git a/nvx-rs/src/exec/peer.rs b/nvx-rs/src/exec/peer.rs deleted file mode 100644 index b3391e93d..000000000 --- a/nvx-rs/src/exec/peer.rs +++ /dev/null @@ -1,679 +0,0 @@ -//! The side of an execution that requested it: see the [module](super) documentation. - -use std::collections::VecDeque; -use std::time::Instant; - -use super::credit::{self, Credit, Granted, Inbox}; -use super::{ - Cause, Key, Limits, Outcome, Pool, Receipt, Stream, Timers, Violation, after, earliest, -}; - -/// A message from the peer to the runner, which the adapter frames and sends in order, as -/// [`Peer::next_message`] returns them. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] -#[non_exhaustive] -pub enum Message { - /// End the workload for `cause`: [`Cause::Cancelled`], [`Cause::Dropped`], or - /// [`Cause::TimedOut`], as the peer's caller or timer ended the execution. - Cancel(Cause), - /// Credit of the output pool `pool`: `units` more of its window, which the runner may spend - /// on output. The first one of each pool opens its whole window. - Credit { - /// The output pool: [`Pool::Output`] if the streams share one, otherwise - /// [`Pool::Stdout`] or [`Pool::Stderr`]. - pool: Pool, - /// The units of the credit. - units: u32, - }, - /// Bytes of the workload's input, in order: the first `length` bytes of the buffer that the - /// adapter lent [`Peer::next_message`]. - Input { - /// The number of bytes, at least one and at most a chunk. - length: usize, - }, - /// The end of the workload's input, after its last byte. - InputEnd, -} - -/// What the peer tells its caller, as [`Peer::next_event`] returns them. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] -#[non_exhaustive] -pub enum Event { - /// The workload's program runs: the runner confirmed its launch. - Started, - /// `stream` ended: its sink took its last byte, or the peer discarded the rest. - End(Stream), - /// The workload's input closed before its end was sent, or the execution ended: the caller - /// sends no more input, and the peer dropped what it held. - InputClosed, - /// The execution finished, with this outcome: the peer's last event. Its cause is the - /// runner's, or, if the runner reported none or never reported, the peer's own. - Finished(Outcome), -} - -/// One output stream, as the peer receives and delivers it. -#[derive(Debug)] -struct Incoming { - /// The bytes that the runner sent and the sink has not taken. - inbox: Inbox, - /// Whether the runner ended the stream. - ended: bool, - /// Whether the stream's end event was returned. - end_event: bool, - /// Whether the peer drops the stream's bytes, as no sink takes them any more. - discarding: bool, - /// Whether the peer dropped bytes of the stream. - truncated: bool, -} - -impl Incoming { - /// Drops what the inbox holds, notes whether that lost bytes, and returns the units that - /// this frees. - fn drop_inbox(&mut self) -> u32 { - let (length, freed) = self.inbox.clear(); - self.truncated |= length > 0; - freed - } -} - -/// The workload's standard input, as the peer takes it from its caller and sends it. -#[derive(Debug)] -struct Outgoing { - /// The bytes that the caller handed over and the peer has not sent: at most one chunk. - queue: VecDeque, - /// The credit that the runner granted. - credit: Credit, - /// Whether the caller ended the input. - ended: bool, - /// Whether the end of the input was sent. - end_sent: bool, - /// Whether the input closed: the runner closed it, or the execution ended. - closed: bool, - /// Whether the event that the input closed waits to be returned. - closed_event: bool, -} - -/// The side of an execution that requested it: see the [module](super) documentation. -/// -/// The adapter creates a peer as it asks the runner's side to run a workload, and hands it the -/// runner's messages and its caller's input. In return, it takes the peer's -/// [messages](Self::next_message), which it sends the runner, and its -/// [events](Self::next_event), which it tells its caller, until the execution -/// [finishes](Event::Finished). Its sink takes each stream's [pending output](Self::pending_output) -/// as it can, and acknowledges what it took with [`consumed`](Self::consumed); its source hands -/// over input as the [write allowance](Self::write_allowance) lets it. -#[derive(Debug)] -pub struct Peer { - /// The execution's key. - key: Key, - /// The limits that both sides agreed on. - limits: Limits, - /// The peer's timers. - timers: Timers, - /// Whether the runner confirmed the workload's launch. - started: bool, - /// Whether the event that the workload started waits to be returned. - started_event: bool, - /// The workload's standard input. - input: Outgoing, - /// The output streams. - outputs: [Incoming; 2], - /// The windows of the output pools: one per stream, or the first one shared. - windows: [Granted; 2], - /// The source whose message the peer returns first next: each output pool's credit, in the - /// order of the pools, then the input. They take turns, so that none starves another. - turn: usize, - /// The runner's terminal outcome, once it arrived. - terminal: Option, - /// Why the peer ended the execution, if it did. - cause: Option, - /// Whether the cancellation waits to be sent. - cancel_message: bool, - /// When the execution's timeout passes, while it runs. - timeout_at: Option, - /// When the peer ended the execution, once it did. - terminated_at: Option, - /// When the grace of pending output ends, while the peer waits for it. - grace_until: Option, - /// When the peer gives up waiting for the terminal outcome of an execution that it ended. - give_up_at: Option, - /// Whether the peer gave up waiting for the terminal outcome. - gave_up: bool, - /// Whether the peer is detached from the runner, from which nothing more arrives. - detached: bool, - /// Whether the outcome was returned. - finished: bool, - /// The most bytes that the peer held at once. - high_water: usize, -} - -impl Peer { - /// Returns the peer of the execution `key`, with the `limits` that both sides agreed on and - /// its own `timers`, created at `now`, from which its timeout runs. - pub fn new(key: Key, limits: Limits, timers: Timers, now: Instant) -> Self { - let incoming = || Incoming { - inbox: Inbox::new(limits.output.unit()), - ended: false, - end_event: false, - discarding: false, - truncated: false, - }; - Self { - key, - limits, - timers, - started: false, - started_event: false, - input: Outgoing { - queue: VecDeque::new(), - credit: Credit::new(limits.input), - ended: false, - end_sent: false, - closed: false, - closed_event: false, - }, - outputs: [incoming(), incoming()], - windows: [Granted::new(limits.output), Granted::new(limits.output)], - turn: 0, - terminal: None, - cause: None, - cancel_message: false, - timeout_at: timers.timeout.and_then(|timeout| after(now, timeout)), - terminated_at: None, - grace_until: None, - give_up_at: None, - gave_up: false, - detached: false, - finished: false, - high_water: 0, - } - } - - /// Returns the execution's key. - pub fn key(&self) -> Key { - self.key - } - - /// Returns the limits that both sides agreed on. - pub fn limits(&self) -> Limits { - self.limits - } - - /// Hands the peer the runner's credit of `units` for the workload's input. A grant of no - /// units, or of more than the units in flight, is a [`Violation::Credit`]. - pub fn credit(&mut self, key: Key, units: u32) -> Result { - if !self.key.admits(key)? { - return Ok(Receipt::Stale); - } - if self.ignores_runner() { - return Ok(Receipt::Late); - } - self.input.credit.grant(units)?; - if self.input.closed || self.input.end_sent { - return Ok(Receipt::Late); - } - Ok(Receipt::Applied) - } - - /// Hands the peer the runner's confirmation that the workload started. A second one, or one - /// after the terminal outcome, is a [`Violation::Order`]. - pub fn started(&mut self, key: Key) -> Result { - if !self.key.admits(key)? { - return Ok(Receipt::Stale); - } - if self.ignores_runner() { - return Ok(Receipt::Late); - } - if self.started || self.terminal.is_some() { - return Err(Violation::Order); - } - self.started = true; - self.started_event = true; - Ok(Receipt::Applied) - } - - /// Hands the peer bytes of `stream` from the runner, which the stream's pool must allow, and - /// which the peer holds until its sink takes them. Output after the stream's end is a - /// [`Violation::Ended`]. Once the peer discards the stream, it drops what arrives, as - /// [`Receipt::Late`], and grants its units again at once. - pub fn output(&mut self, key: Key, stream: Stream, bytes: &[u8]) -> Result { - if !self.key.admits(key)? { - return Ok(Receipt::Stale); - } - if self.ignores_runner() { - return Ok(Receipt::Late); - } - if self.outputs[stream.index()].ended { - return Err(Violation::Ended); - } - self.limits.check_chunk(bytes.len())?; - let window = &mut self.windows[self.limits.pool_of(stream)]; - window.charge(bytes.len())?; - let incoming = &mut self.outputs[stream.index()]; - if incoming.discarding { - incoming.truncated = true; - window.free(self.limits.output.cost(bytes.len())); - return Ok(Receipt::Late); - } - incoming.inbox.push(bytes); - self.note_buffered(); - Ok(Receipt::Applied) - } - - /// Hands the peer the end of `stream` from the runner, after its last byte. A second end is - /// a [`Violation::Ended`]. The end does not report whether the runner truncated the stream; - /// only its terminal outcome does. - pub fn end_output(&mut self, key: Key, stream: Stream) -> Result { - if !self.key.admits(key)? { - return Ok(Receipt::Stale); - } - if self.ignores_runner() { - return Ok(Receipt::Late); - } - let incoming = &mut self.outputs[stream.index()]; - if incoming.ended { - return Err(Violation::Ended); - } - incoming.ended = true; - Ok(Receipt::Applied) - } - - /// Hands the peer the runner's notice that the workload's input closed early. The peer drops - /// the input that it holds, takes no more, and tells its caller. - pub fn input_closed(&mut self, key: Key) -> Result { - if !self.key.admits(key)? { - return Ok(Receipt::Stale); - } - if self.ignores_runner() || self.input.closed || self.input.end_sent { - return Ok(Receipt::Late); - } - self.close_input(); - Ok(Receipt::Applied) - } - - /// Hands the peer the runner's terminal outcome at `now`, which follows both streams' ends: - /// a terminal outcome before them is a [`Violation::Order`], and a second one a - /// [`Violation::Ended`]. The peer finishes once its sink took the rest of the output, losing - /// nothing, unless a side ended the execution: then the sink takes the rest only for the - /// peer's grace, from when the peer ended the execution, or from `now` if the runner did. - /// Once the peer detached from the runner or gave up waiting for it, the terminal outcome is - /// [late](Receipt::Late), as every message of the runner's is then, and the peer's own - /// outcome stands, whenever its caller retrieves it. - pub fn terminal( - &mut self, - now: Instant, - key: Key, - outcome: Outcome, - ) -> Result { - if !self.key.admits(key)? { - return Ok(Receipt::Stale); - } - self.tick(now); - if self.ignores_runner() { - return Ok(Receipt::Late); - } - if self.terminal.is_some() { - return Err(Violation::Ended); - } - if !self.outputs.iter().all(|incoming| incoming.ended) { - return Err(Violation::Order); - } - self.terminal = Some(outcome); - self.give_up_at = None; - self.timeout_at = None; - self.cancel_message = false; - self.close_input(); - if outcome.cause().is_some() && self.terminated_at.is_none() { - self.terminated_at = Some(now); - self.grace_until = after(now, self.timers.grace); - if self.grace_until.is_some_and(|at| now >= at) { - self.end_grace(); - } - } - Ok(Receipt::Applied) - } - - /// Returns how many more bytes of input the caller may hand over with - /// [`input`](Self::input): what the peer's queue has room for, at most one chunk ahead of the - /// runner's credit, and none once the input ended or closed, or the peer ended the - /// execution. - pub fn write_allowance(&self) -> usize { - if self.input.closed || self.input.ended || self.terminated_at.is_some() || self.finished { - return 0; - } - self.limits.chunk() - self.input.queue.len() - } - - /// Hands the peer bytes of the workload's input from its caller, in order, and returns how - /// many it took: as many as the [write allowance](Self::write_allowance) lets it. - pub fn input(&mut self, bytes: &[u8]) -> usize { - let length = bytes.len().min(self.write_allowance()); - self.input.queue.extend(&bytes[..length]); - self.note_buffered(); - length - } - - /// Ends the workload's input after the bytes that the caller handed over. - pub fn end_input(&mut self) { - if !self.input.closed { - self.input.ended = true; - } - } - - /// Returns the bytes of `stream` that the sink takes next, as many as are contiguous: a part - /// of what the peer holds, while it holds any. - pub fn pending_output(&self, stream: Stream) -> &[u8] { - self.outputs[stream.index()].inbox.front() - } - - /// Hands the peer the count of bytes of `stream`'s [pending output](Self::pending_output) - /// that the sink took, which the peer grants the runner again. - pub fn consumed(&mut self, stream: Stream, bytes: usize) { - let freed = self.outputs[stream.index()].inbox.pop(bytes); - self.windows[self.limits.pool_of(stream)].free(freed); - } - - /// Ends the execution at `now` for [`Cause::Cancelled`]: the peer sends the runner its - /// cancellation, takes no more input, delivers the pending output for its grace, and - /// waits for the terminal outcome for the settlement time after that. - pub fn cancel(&mut self, now: Instant) { - self.tick(now); - self.terminate(now, Cause::Cancelled); - } - - /// Ends the execution at `now` for [`Cause::Dropped`], because the caller dropped its - /// handle: as a [cancellation](Self::cancel), but the peer discards the output at once, as - /// no sink takes it any more. - pub fn dropped(&mut self, now: Instant) { - self.tick(now); - self.terminate(now, Cause::Dropped); - if !self.finished { - self.end_grace(); - } - } - - /// Detaches the peer from the runner at `now` for [`Cause::Reset`], because the session was - /// reset. The peer takes none of the runner's messages from then on. - pub fn reset(&mut self, now: Instant) { - self.detach(now, Cause::Reset); - } - - /// Detaches the peer from the runner at `now` for [`Cause::Disconnected`], because the - /// transport was lost. The peer takes none of the runner's messages from then on. - pub fn disconnect(&mut self, now: Instant) { - self.detach(now, Cause::Disconnected); - } - - /// Fires the timers whose instants `now` has reached: the execution's timeout, the end of - /// the grace of pending output, and the time at which the peer gives up waiting for the - /// terminal outcome of an execution that it ended. - pub fn tick(&mut self, now: Instant) { - if self.finished { - return; - } - if self.timeout_at.is_some_and(|at| now >= at) { - self.timeout_at = None; - self.terminate(now, Cause::TimedOut); - } - if self.grace_until.is_some_and(|at| now >= at) { - self.end_grace(); - } - if self.give_up_at.is_some_and(|at| now >= at) { - self.give_up_at = None; - self.gave_up = true; - self.end_grace(); - } - } - - /// Returns when the peer's next timer fires, for the adapter to [tick](Self::tick) it then, - /// or `None` while no timer runs. - pub fn deadline(&self) -> Option { - if self.finished { - return None; - } - [self.timeout_at, self.grace_until, self.give_up_at] - .into_iter() - .fold(None, earliest) - } - - /// Returns the next message to send the runner, or `None` if none waits: the cancellation - /// first, then credit of each output pool, the input, and its end once its bytes are sent. - /// Each pool's credit and the input take turns: after one of them, the others that wait go - /// first, so that none starves another, however much output the sink takes between two - /// calls. An input message carries the most bytes that the queue, the credit, a chunk, and - /// `buffer` allow, which it holds at the start of `buffer`; an empty buffer carries none. - pub fn next_message(&mut self, buffer: &mut [u8]) -> Option { - if self.finished || self.detached { - return None; - } - if std::mem::take(&mut self.cancel_message) { - return Some(Message::Cancel(self.cause.unwrap_or(Cause::Cancelled))); - } - let pools = self.limits.output_pools().len(); - let sources = pools + 1; - for offset in 0..sources { - let source = (self.turn + offset) % sources; - let message = if source < pools { - self.next_credit(source) - } else { - self.next_input(buffer) - }; - if message.is_some() { - self.turn = (source + 1) % sources; - return message; - } - } - None - } - - /// Returns credit of the output pool at `index` to send the runner, if a stream of the pool - /// is still open, no terminal outcome arrived, and the sink took bytes since the last grant. - fn next_credit(&mut self, index: usize) -> Option { - if self.terminal.is_some() { - return None; - } - let open = Stream::ALL.into_iter().any(|stream| { - self.limits.pool_of(stream) == index && !self.outputs[stream.index()].ended - }); - if !open { - return None; - } - let units = self.windows[index].take_grant()?; - let pool = self.limits.output_pools()[index]; - Some(Message::Credit { pool, units }) - } - - /// Returns the next input message to send the runner: the input, then its end once its bytes - /// are sent, unless the input closed. - fn next_input(&mut self, buffer: &mut [u8]) -> Option { - if self.input.closed { - return None; - } - let wanted = self - .input - .queue - .len() - .min(self.limits.chunk()) - .min(buffer.len()); - let length = self.input.credit.sendable(wanted); - if length > 0 { - credit::take(&mut self.input.queue, buffer, length); - self.input.credit.spend(length); - return Some(Message::Input { length }); - } - if self.input.ended && self.input.queue.is_empty() && !self.input.end_sent { - self.input.end_sent = true; - return Some(Message::InputEnd); - } - None - } - - /// Returns the next event for the caller, or `None` if none waits: that the workload - /// started, that its input closed, the end of each stream once its sink took its last byte, - /// and the outcome last, once the runner's terminal outcome arrived and the sink took the - /// rest of the output, or once the peer detached or gave up and delivered or discarded - /// what it held. - pub fn next_event(&mut self) -> Option { - if self.finished { - return None; - } - if std::mem::take(&mut self.started_event) { - return Some(Event::Started); - } - if std::mem::take(&mut self.input.closed_event) { - return Some(Event::InputClosed); - } - for stream in Stream::ALL { - let incoming = &mut self.outputs[stream.index()]; - if incoming.ended && incoming.inbox.is_empty() && !incoming.end_event { - incoming.end_event = true; - return Some(Event::End(stream)); - } - } - let outcome = self.outcome()?; - self.finished = true; - Some(Event::Finished(outcome)) - } - - /// Returns whether the peer has returned the execution's outcome. - pub fn is_finished(&self) -> bool { - self.finished - } - - /// Returns whether the runner confirmed the workload's launch. - pub fn has_started(&self) -> bool { - self.started - } - - /// Returns whether the peer is detached from the runner. - pub fn is_detached(&self) -> bool { - self.detached - } - - /// Returns the most bytes that the peer holds at once: one chunk of input, and the output's - /// windows. - pub fn memory_bound(&self) -> usize { - self.limits - .chunk() - .saturating_add(self.limits.output_bytes()) - } - - /// Returns the bytes that the peer holds now. - pub fn buffered(&self) -> usize { - self.input.queue.len() - + self - .outputs - .iter() - .map(|incoming| incoming.inbox.len()) - .sum::() - } - - /// Returns the most bytes that the peer has held at once. - pub fn high_water(&self) -> usize { - self.high_water - } - - /// Returns whether the peer takes no more of the runner's messages: it finished, detached - /// from the runner, or gave up waiting for it. So its outcome is fixed from then on, whenever - /// its caller retrieves it. - fn ignores_runner(&self) -> bool { - self.finished || self.detached || self.gave_up - } - - /// Ends the execution at `now` for `cause`, unless the peer ended it already: sends the - /// runner the cancellation, unless the runner's terminal outcome arrived, closes the input, - /// and begins the grace of pending output, after which the peer waits for the terminal - /// outcome for the settlement time. - fn terminate(&mut self, now: Instant, cause: Cause) { - if self.finished || self.terminated_at.is_some() { - return; - } - self.terminated_at = Some(now); - self.timeout_at = None; - self.cause = Some(cause); - self.close_input(); - self.grace_until = after(now, self.timers.grace); - if self.terminal.is_none() { - self.cancel_message = !self.detached; - self.give_up_at = self - .grace_until - .and_then(|grace| after(grace, self.timers.settlement)); - } - if self.grace_until.is_some_and(|at| now >= at) { - self.end_grace(); - } - } - - /// Detaches the peer from the runner at `now` for `cause`: ends the execution if the peer - /// has not, and stops waiting for the runner, from which nothing more arrives. - fn detach(&mut self, now: Instant, cause: Cause) { - self.tick(now); - if self.finished || self.detached { - return; - } - self.terminate(now, cause); - self.detached = true; - self.cancel_message = false; - self.give_up_at = None; - } - - /// Closes the input: drops what the peer holds of it, and tells the caller unless the caller - /// ended it. - fn close_input(&mut self) { - if self.input.closed { - return; - } - self.input.closed = true; - self.input.queue = VecDeque::new(); - self.input.closed_event = !self.input.end_sent; - } - - /// Ends the grace of pending output: drops what the inboxes hold, grants their units again, - /// and discards what still arrives. - fn end_grace(&mut self) { - self.grace_until = None; - for stream in Stream::ALL { - let incoming = &mut self.outputs[stream.index()]; - incoming.discarding = true; - let freed = incoming.drop_inbox(); - self.windows[self.limits.pool_of(stream)].free(freed); - } - } - - /// Returns the outcome once the peer can finish: the runner's terminal outcome, with the - /// streams that the peer truncated, once the sink took or the peer discarded the rest of the - /// output; or, once the peer detached or gave up, an outcome of its own, with no status, its - /// cause, both streams marked truncated because their ends do not report loss, and fenced. - fn outcome(&self) -> Option { - if !self - .outputs - .iter() - .all(|incoming| incoming.inbox.is_empty()) - { - return None; - } - let mut outcome = match self.terminal { - Some(terminal) => terminal, - None if self.detached || self.gave_up => Outcome::new(None) - .with_truncated(Stream::Stdout) - .with_truncated(Stream::Stderr), - None => return None, - }; - if outcome.cause().is_none() - && let Some(cause) = self.cause - { - outcome = outcome.with_cause(cause); - } - for stream in Stream::ALL { - if self.outputs[stream.index()].truncated { - outcome = outcome.with_truncated(stream); - } - } - Some(outcome) - } - - /// Notes the bytes that the peer holds, for its high-water mark. - fn note_buffered(&mut self) { - self.high_water = self.high_water.max(self.buffered()); - } -} diff --git a/nvx-rs/src/exec/runner.rs b/nvx-rs/src/exec/runner.rs deleted file mode 100644 index f43a2fd09..000000000 --- a/nvx-rs/src/exec/runner.rs +++ /dev/null @@ -1,880 +0,0 @@ -//! The side of an execution that runs its workload: see the [module](super) documentation. - -use std::collections::VecDeque; -use std::time::Instant; - -use super::credit::{self, Credit, Granted, Inbox}; -use super::{ - Cause, Key, LaunchFailure, Limits, Outcome, Pool, Receipt, Settlement, Status, Stream, Timers, - Violation, after, earliest, -}; - -/// A message from the runner to the peer, which the adapter frames and sends in order, as -/// [`Runner::next_message`] returns them. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] -#[non_exhaustive] -pub enum Message { - /// Credit of the workload's input: `units` more of the input's window, which the peer may - /// spend on input. The first one opens the whole window. - Credit { - /// The units of the credit. - units: u32, - }, - /// The workload's program runs: its launch succeeded. - Started, - /// Bytes of `stream`, in order: the first `length` bytes of the buffer that the adapter lent - /// [`Runner::next_message`]. - Output { - /// The stream that the bytes belong to. - stream: Stream, - /// The number of bytes, at least one and at most a chunk. - length: usize, - }, - /// The end of `stream`, after its last byte. - End(Stream), - /// The workload's input closed before the peer ended it, because the workload closed it, - /// ended, or was ended: the peer sends no more input, and the runner drops what arrives. - InputClosed, - /// The execution's terminal outcome, after both streams' ends: the runner's last message. - /// Once the runner is detached from the peer, by a [reset](Runner::reset), a - /// [disconnection](Runner::disconnect), or a [drop](Runner::dropped), it is the only message - /// that the runner still returns, and the adapter keeps it rather than send it. - Terminal(Outcome), -} - -/// A request to the adapter, which applies it to the workload, as [`Runner::next_request`] -/// returns them. The runner makes each one once. -#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] -#[non_exhaustive] -pub enum Request { - /// End the workload, and its processes, for `cause`, closing its input. - Terminate(Cause), - /// Read and drop the rest of `stream`, reporting the bytes with [`Runner::discarded`], until - /// it closes. - Discard(Stream), - /// Close the workload's input: the peer ended it, and the workload took every byte that the - /// runner accepted. - CloseInput, - /// Settle the execution's processes: end what is left of them, and report with - /// [`Runner::settled`] once the adapter's own check shows that none is left. - Settle, -} - -/// One output stream, as the runner reads and sends it. -#[derive(Debug, Default)] -struct Output { - /// The bytes read from the workload and not sent yet: at most one chunk. - queue: VecDeque, - /// Whether the workload's stream ended: every writer closed it, or reading it failed. - closed: bool, - /// Whether every writer closed the workload's stream, though the adapter has not read it to - /// its end: it ends once the adapter reads what it still holds. - hung_up: bool, - /// Whether the stream's end message was returned. - ended: bool, - /// Whether the runner drops the stream's bytes. - discarding: bool, - /// Whether the request to discard the stream waits to be returned. - discard_requested: bool, - /// Whether the stream lost bytes. - truncated: bool, -} - -impl Output { - /// Returns whether the stream is done: it closed, and its queue is empty. - fn is_done(&self) -> bool { - self.closed && self.queue.is_empty() - } - - /// Drops the queue, and notes whether that lost bytes. - fn drop_queue(&mut self) { - if !self.queue.is_empty() { - self.truncated = true; - } - self.queue = VecDeque::new(); - } -} - -/// The workload's standard input, as the runner receives and writes it. -#[derive(Debug)] -struct Input { - /// The bytes that the peer sent and the workload has not taken. - inbox: Inbox, - /// The input's window. - window: Granted, - /// Whether the peer ended the input. - peer_ended: bool, - /// Whether the workload's input is closed, or the runner requested to close it. - closed: bool, - /// Whether the request to close the input waits to be returned. - close_requested: bool, - /// Whether the message that the input closed early waits to be returned. - closed_message: bool, -} - -/// Where the settlement of the execution stands. -#[derive(Clone, Copy, Debug, PartialEq, Eq)] -enum Settling { - /// The runner has not requested it. - Idle, - /// The runner requested it, and expects its result by this instant, if any. - Requested(Option), - /// The execution settled. - Settled, - /// The execution is fenced: its settlement failed, or came too late. - Fenced, -} - -/// The side of an execution that runs its workload: see the [module](super) documentation. -/// -/// The adapter creates a runner when it accepts the peer's request to run a workload, and hands -/// it the peer's messages and the workload's events. In return, it takes the runner's -/// [requests](Self::next_request), which it applies to the workload, and its -/// [messages](Self::next_message), which it sends the peer, until the -/// [terminal outcome](Message::Terminal). It reads each output stream of the workload only as far -/// as the [read allowance](Self::read_allowance) lets it, and writes the -/// [pending input](Self::pending_input) to the workload as the workload takes it. -#[derive(Debug)] -pub struct Runner { - /// The execution's key. - key: Key, - /// The limits that both sides agreed on. - limits: Limits, - /// The runner's timers. - timers: Timers, - /// The workload's launch result, once it arrived. - launch: Option>, - /// Whether the message that the workload started waits to be returned. - started_message: bool, - /// The workload's status, once its end is known: `Some(None)` if its status is lost. - exit: Option>, - /// Why the runner ended the workload, if it did. - cause: Option, - /// The request to end the workload, while it waits to be returned. - terminate_request: Option, - /// Whether the request to settle the execution waits to be returned. - settle_request: bool, - /// When the workload's timeout passes, while it runs. - timeout_at: Option, - /// When the termination began, once one did. - terminated_at: Option, - /// When the grace of pending output ends, while the runner waits for it. - grace_until: Option, - /// When the runner gives up on the execution after a termination, while it waits for it, - /// unless the execution's cleanup is complete by then. - give_up_at: Option, - /// When the runner gives up on what a process may still hold after the workload's end: the - /// launch result, if it has not arrived, and, if the execution does not settle, the output - /// streams that are still open. It is the settlement time and then the grace after the - /// workload's end. - abandon_at: Option, - /// Whether the runner gave up, and fenced the execution. - gave_up: bool, - /// Whether the runner gave up on the launch result after the workload's end, and fenced the - /// execution. - launch_abandoned: bool, - /// Whether the runner is detached from the peer, to which it sends nothing more. - detached: bool, - /// The output streams. - outputs: [Output; 2], - /// The credit of the output pools: one per stream, or the first one shared. - credits: [Credit; 2], - /// The stream whose output the runner sends first next. - turn: usize, - /// Whether the last message that the runner returned was credit of the input, so that any - /// other message that waits goes next: credit and the other messages take turns. - credited: bool, - /// The workload's standard input. - input: Input, - /// Where the settlement stands. - settling: Settling, - /// Whether the terminal outcome was returned. - finished: bool, - /// The most bytes that the runner held at once. - high_water: usize, -} - -impl Runner { - /// Returns the runner of the execution `key`, with the `limits` that both sides agreed on - /// and its own `timers`, created at `now`, from which its timeout runs. - pub fn new(key: Key, limits: Limits, timers: Timers, now: Instant) -> Self { - Self { - key, - limits, - timers, - launch: None, - started_message: false, - exit: None, - cause: None, - terminate_request: None, - settle_request: false, - timeout_at: timers.timeout.and_then(|timeout| after(now, timeout)), - terminated_at: None, - grace_until: None, - give_up_at: None, - abandon_at: None, - gave_up: false, - launch_abandoned: false, - detached: false, - outputs: Default::default(), - credits: [Credit::new(limits.output), Credit::new(limits.output)], - turn: 0, - credited: false, - input: Input { - inbox: Inbox::new(limits.input.unit()), - window: Granted::new(limits.input), - peer_ended: false, - closed: false, - close_requested: false, - closed_message: false, - }, - settling: Settling::Idle, - finished: false, - high_water: 0, - } - } - - /// Returns the execution's key. - pub fn key(&self) -> Key { - self.key - } - - /// Returns the limits that both sides agreed on. - pub fn limits(&self) -> Limits { - self.limits - } - - /// Hands the runner the peer's credit of `units` for the output pool `pool`, which the - /// execution's limits must have: [`Pool::Output`] if the streams share one pool, otherwise - /// [`Pool::Stdout`] or [`Pool::Stderr`]. A grant of no units, or of more than the units in - /// flight, is a [`Violation::Credit`]. - pub fn credit(&mut self, key: Key, pool: Pool, units: u32) -> Result { - if !self.key.admits(key)? { - return Ok(Receipt::Stale); - } - let index = self.limits.output_pool(pool)?; - if self.finished { - return Ok(Receipt::Late); - } - self.credits[index].grant(units)?; - Ok(Receipt::Applied) - } - - /// Hands the runner bytes of the workload's input from the peer, which the input's window - /// must allow, and which the runner holds until the workload takes them. Input after the - /// peer ended it is a [`Violation::Ended`]. Once the workload's input closed, the runner - /// drops what arrives, as [`Receipt::Late`]. - pub fn input(&mut self, key: Key, bytes: &[u8]) -> Result { - if !self.key.admits(key)? { - return Ok(Receipt::Stale); - } - if self.finished { - return Ok(Receipt::Late); - } - if self.input.peer_ended { - return Err(Violation::Ended); - } - self.limits.check_chunk(bytes.len())?; - self.input.window.charge(bytes.len())?; - if self.input.closed { - return Ok(Receipt::Late); - } - self.input.inbox.push(bytes); - self.note_buffered(); - Ok(Receipt::Applied) - } - - /// Hands the runner the end of the workload's input from the peer. The runner requests to - /// [close the input](Request::CloseInput) once the workload took every byte before it. A - /// second end is a [`Violation::Ended`]. - pub fn end_input(&mut self, key: Key) -> Result { - if !self.key.admits(key)? { - return Ok(Receipt::Stale); - } - if self.finished { - return Ok(Receipt::Late); - } - if self.input.peer_ended { - return Err(Violation::Ended); - } - self.input.peer_ended = true; - if self.input.closed { - return Ok(Receipt::Late); - } - self.close_input_when_taken(); - Ok(Receipt::Applied) - } - - /// Hands the runner the peer's cancellation at `now`, with the peer's `cause`, such as - /// [`Cause::Cancelled`], or [`Cause::TimedOut`] for a timeout of the peer's own. It ends the - /// workload for that cause if the workload still runs, and begins the grace of its pending - /// output. A cancellation once a termination began, or once the execution ended, is - /// [`Receipt::Late`]. - pub fn cancel(&mut self, now: Instant, key: Key, cause: Cause) -> Result { - if !self.key.admits(key)? { - return Ok(Receipt::Stale); - } - self.tick(now); - if self.finished || self.terminated_at.is_some() { - return Ok(Receipt::Late); - } - self.terminate(now, cause); - Ok(Receipt::Applied) - } - - /// Hands the runner the workload's launch result: `Ok` once its program runs, after which - /// the runner tells the peer that it [started](Message::Started), or why it did not start. - /// The workload's end, with [`exited`](Self::exited), may come before or after it: an adapter - /// learns of the launch late if another process holds what reports it. The runner sends the - /// workload's start, output, and terminal outcome only once it has the result, or gave up on - /// it: after a termination, or once the settlement time and the grace have passed since the - /// workload's end. Only the first result counts, and none once the runner gave up on it, as - /// it may have sent output, which no start may follow, or the terminal outcome. - pub fn launched(&mut self, result: Result<(), LaunchFailure>) { - if !self.awaits_launch() { - return; - } - self.launch = Some(result); - if result.is_ok() { - self.started_message = !self.detached; - } else { - self.close_input(); - } - } - - /// Returns how many more bytes of `stream` the adapter may read from the workload and hand - /// over with [`output`](Self::output): what the runner's queue of the stream has room for, at - /// most one chunk ahead of the peer's credit, and none once the stream closed or the runner - /// discards it. - pub fn read_allowance(&self, stream: Stream) -> usize { - let output = &self.outputs[stream.index()]; - if output.closed || output.discarding || self.gave_up { - return 0; - } - self.limits.chunk() - output.queue.len() - } - - /// Hands the runner bytes that the adapter read from `stream`, in order, and returns how - /// many it took: all of them, unless they exceed the [read allowance](Self::read_allowance), - /// beyond which it takes none. Once the runner discards the stream, it takes and drops them, - /// which truncates the stream. - pub fn output(&mut self, stream: Stream, bytes: &[u8]) -> usize { - let chunk = self.limits.chunk(); - let output = &mut self.outputs[stream.index()]; - if output.closed || self.gave_up { - return 0; - } - if output.discarding { - output.truncated |= !bytes.is_empty(); - return bytes.len(); - } - let length = bytes.len().min(chunk - output.queue.len()); - output.queue.extend(&bytes[..length]); - self.note_buffered(); - length - } - - /// Hands the runner the count of bytes of `stream` that the adapter read and dropped, as a - /// [discard](Request::Discard) asks, which truncates the stream unless it is zero. - pub fn discarded(&mut self, stream: Stream, bytes: usize) { - self.outputs[stream.index()].truncated |= bytes > 0; - } - - /// Hands the runner the end of `stream`: every writer of the workload's stream closed it. - pub fn output_closed(&mut self, stream: Stream) { - self.outputs[stream.index()].closed = true; - } - - /// Hands the runner a failure to read `stream`, which ends the stream as its close does, and - /// truncates it. - pub fn output_failed(&mut self, stream: Stream) { - let output = &mut self.outputs[stream.index()]; - output.closed = true; - output.truncated = true; - } - - /// Hands the runner the news that every writer of `stream` closed it, though the adapter has - /// not read it to its end: the stream ends once the adapter reads what it still holds. No - /// process can hold the stream open any more, so the runner never gives up on it after the - /// workload's end, even if the execution does not settle. - pub fn output_hung_up(&mut self, stream: Stream) { - self.outputs[stream.index()].hung_up = true; - } - - /// Returns the bytes of input that the adapter writes to the workload next, as many as are - /// contiguous: a part of what the runner holds, while it holds any and the input is open. - pub fn pending_input(&self) -> &[u8] { - if self.input.closed { - return &[]; - } - self.input.inbox.front() - } - - /// Hands the runner the count of bytes of the [pending input](Self::pending_input) that the - /// workload's input took, which the runner returns to the peer as credit. - pub fn input_written(&mut self, bytes: usize) { - if self.input.closed { - return; - } - let freed = self.input.inbox.pop(bytes); - self.input.window.free(freed); - self.close_input_when_taken(); - } - - /// Hands the runner the close of the workload's input other than by its - /// [request](Request::CloseInput): the workload closed it, or the adapter did as the workload - /// ended. The runner drops the pending input, and tells the peer, unless the peer ended the - /// input. - pub fn input_closed(&mut self) { - self.close_input(); - } - - /// Hands the runner the workload's end at `now`, with its status as its collector reaped it, - /// or `None` if the status is lost. Only the first end counts. The workload's timeout fires - /// no more, even if `now` has reached it, as the workload ended before the runner learned it: - /// the end stands as the workload's own. The runner closes the input, and requests to - /// [settle](Request::Settle) the execution, whose result it expects within the settlement - /// time. If the execution does not settle, as its settlement failed or came too late, the - /// processes that hold its output streams may outlive it, so the runner gives up on the - /// streams that are still open, but for those that [hung up](Self::output_hung_up), once the - /// grace after the settlement time has passed too: it reads them no more, sends what it read, - /// and marks them truncated. A process may also hold back what reports the workload's - /// [launch](Self::launched), whether or not the execution settles, so a launch result that - /// has not arrived by then fences the execution as well: the runner gives up on the result, - /// and on those streams, and sends the workload's output and terminal outcome without it, - /// with the workload's status and no cause. - pub fn exited(&mut self, now: Instant, status: Option) { - self.timeout_at = None; - self.tick(now); - if self.exit.is_some() || self.gave_up { - return; - } - self.exit = Some(status); - self.close_input(); - self.settle_request = true; - let settlement = after(now, self.timers.settlement); - self.settling = Settling::Requested(settlement); - self.abandon_at = settlement.and_then(|at| after(at, self.timers.grace)); - } - - /// Hands the runner the result of the [settlement](Request::Settle) that it requested: the - /// execution is reusable once it settled, and fenced if it failed. A result that the runner - /// did not request, or that comes after the settlement time, changes nothing. - pub fn settled(&mut self, result: Settlement) { - if let Settling::Requested(_) = self.settling { - self.settling = match result { - Settlement::Settled => Settling::Settled, - Settlement::Failed => Settling::Fenced, - }; - } - } - - /// Ends the workload at `now` for [`Cause::Dropped`], because the adapter abandons the - /// execution, and detaches the runner from the peer. - pub fn dropped(&mut self, now: Instant) { - self.detach(now, Cause::Dropped); - } - - /// Ends the workload at `now` for [`Cause::Reset`], because the session with the peer was - /// reset, and detaches the runner from the peer. - pub fn reset(&mut self, now: Instant) { - self.detach(now, Cause::Reset); - } - - /// Ends the workload at `now` for [`Cause::Disconnected`], because the transport to the peer - /// was lost, and detaches the runner from the peer. - pub fn disconnect(&mut self, now: Instant) { - self.detach(now, Cause::Disconnected); - } - - /// Fires the timers whose instants `now` has reached: the workload's timeout, the end of the - /// grace of pending output, the settlement time, the time after a termination at which the - /// runner gives up on the execution unless its cleanup is complete, and the time after the - /// workload's end at which it gives up on a launch result that has not arrived, and on the - /// output streams of an execution that did not settle. - pub fn tick(&mut self, now: Instant) { - if self.finished { - return; - } - if self.timeout_at.is_some_and(|at| now >= at) { - self.timeout_at = None; - self.terminate(now, Cause::TimedOut); - } - if self.grace_until.is_some_and(|at| now >= at) { - self.end_grace(); - } - if let Settling::Requested(Some(by)) = self.settling - && now >= by - { - self.settling = Settling::Fenced; - } - if self.give_up_at.is_some_and(|at| now >= at) { - if self.is_cleaned_up() { - self.give_up_at = None; - } else { - self.give_up(); - } - } - if self.abandon_at.is_some_and(|at| now >= at) { - if self.awaits_launch() { - self.abandon_launch(); - } - if self.settling == Settling::Fenced { - self.abandon_open_streams(); - } - } - } - - /// Returns when the runner's next timer fires, for the adapter to [tick](Self::tick) it - /// then, or `None` while no timer runs. - pub fn deadline(&self) -> Option { - if self.finished { - return None; - } - let settlement = match self.settling { - Settling::Requested(by) => by, - _ => None, - }; - let give_up = self.give_up_at.filter(|_| !self.is_cleaned_up()); - let abandon = self.abandon_at.filter(|_| { - self.awaits_launch() - || self.settling == Settling::Fenced - && self - .outputs - .iter() - .any(|output| !output.closed && !output.hung_up) - }); - [ - self.timeout_at, - self.grace_until, - settlement, - give_up, - abandon, - ] - .into_iter() - .fold(None, earliest) - } - - /// Returns the next request to apply to the workload, or `None` if none waits: ending the - /// workload first, then discarding streams, closing the input, and settling the execution. - pub fn next_request(&mut self) -> Option { - if let Some(cause) = self.terminate_request.take() { - return Some(Request::Terminate(cause)); - } - for stream in Stream::ALL { - let output = &mut self.outputs[stream.index()]; - if std::mem::take(&mut output.discard_requested) { - return Some(Request::Discard(stream)); - } - } - if std::mem::take(&mut self.input.close_requested) { - return Some(Request::CloseInput); - } - if std::mem::take(&mut self.settle_request) { - return Some(Request::Settle); - } - None - } - - /// Returns the next message to send the peer, or `None` if none waits: credit of the input, - /// that the workload started, that its input closed early, the end of each stream once its - /// bytes are sent, the workload's output, which takes turns between the streams as both have - /// bytes and credit, and the terminal outcome last, once the workload's status, both streams' - /// ends, and the settlement are known. A stream's end goes before the other stream's output, - /// which would otherwise hold it back for as long as it flows. Credit of the input and the - /// other messages take turns: after credit, any other message that waits goes first, so that - /// neither starves the other, however much input the workload takes between two calls. An - /// output message carries the most bytes that the stream's queue, its credit, a chunk, and - /// `buffer` allow, which it holds at the start of `buffer`; an empty buffer carries none, so - /// that only other messages return. - pub fn next_message(&mut self, buffer: &mut [u8]) -> Option { - if self.finished { - return None; - } - if !self.detached { - if !self.credited - && let Some(message) = self.next_credit() - { - self.credited = true; - return Some(message); - } - if let Some(message) = self.next_other(buffer) { - self.credited = false; - return Some(message); - } - if let Some(message) = self.next_credit() { - self.credited = true; - return Some(message); - } - } - let outcome = self.outcome()?; - self.finished = true; - Some(Message::Terminal(outcome)) - } - - /// Returns whether the runner has returned its terminal outcome. - pub fn is_finished(&self) -> bool { - self.finished - } - - /// Returns whether the runner is detached from the peer. - pub fn is_detached(&self) -> bool { - self.detached - } - - /// Returns whether the runner awaits the result of the [settlement](Request::Settle) that it - /// requested: from its request until the result, the settlement time, or the runner giving - /// up, after which the adapter may stop settling the execution. - pub fn awaits_settlement(&self) -> bool { - matches!(self.settling, Settling::Requested(_)) - } - - /// Returns the most bytes that the runner holds at once: one chunk of each output stream, - /// and the input's window. - pub fn memory_bound(&self) -> usize { - self.limits - .chunk() - .saturating_mul(2) - .saturating_add(self.limits.input_bytes()) - } - - /// Returns the bytes that the runner holds now. - pub fn buffered(&self) -> usize { - self.outputs - .iter() - .map(|output| output.queue.len()) - .sum::() - + self.input.inbox.len() - } - - /// Returns the most bytes that the runner has held at once. - pub fn high_water(&self) -> usize { - self.high_water - } - - /// Returns credit of the input to send the peer, if the runner still grants it and the - /// workload took input since the last grant. - fn next_credit(&mut self) -> Option { - if !self.input_accepts() { - return None; - } - let units = self.input.window.take_grant()?; - Some(Message::Credit { units }) - } - - /// Returns the next message other than credit of the input: that the workload started, that - /// its input closed early, the end of each stream once its bytes are sent, and the workload's - /// output. - fn next_other(&mut self, buffer: &mut [u8]) -> Option { - if std::mem::take(&mut self.started_message) { - return Some(Message::Started); - } - if std::mem::take(&mut self.input.closed_message) { - return Some(Message::InputClosed); - } - if !self.awaits_launch() { - for stream in Stream::ALL { - let output = &mut self.outputs[stream.index()]; - if output.is_done() && !output.ended { - output.ended = true; - return Some(Message::End(stream)); - } - } - return self.next_output(buffer); - } - None - } - - /// Returns the next output message, taking turns between the streams. - fn next_output(&mut self, buffer: &mut [u8]) -> Option { - for _ in Stream::ALL { - let stream = Stream::ALL[self.turn]; - self.turn = 1 - self.turn; - let credit = &mut self.credits[self.limits.pool_of(stream)]; - let output = &mut self.outputs[stream.index()]; - let wanted = output - .queue - .len() - .min(self.limits.chunk()) - .min(buffer.len()); - let length = credit.sendable(wanted); - if length > 0 { - credit::take(&mut output.queue, buffer, length); - credit.spend(length); - return Some(Message::Output { stream, length }); - } - } - None - } - - /// Returns whether the runner still grants input credit: the input is open, and the peer - /// has not ended it. - fn input_accepts(&self) -> bool { - !self.input.closed && !self.input.peer_ended - } - - /// Requests to close the input once the peer ended it and the workload took every byte. - fn close_input_when_taken(&mut self) { - if self.input.peer_ended && !self.input.closed && self.input.inbox.is_empty() { - self.input.closed = true; - self.input.close_requested = true; - } - } - - /// Notes that the workload's input closed, other than by the runner's request: drops the - /// pending input, and tells the peer unless the peer ended the input. - fn close_input(&mut self) { - if self.input.closed { - return; - } - self.input.closed = true; - self.input.inbox.clear(); - self.input.closed_message = !self.input.peer_ended && !self.detached; - } - - /// Ends the workload at `now` for `cause` if it still runs, and begins the grace of pending - /// output, unless a termination began already. - fn terminate(&mut self, now: Instant, cause: Cause) { - if self.terminated_at.is_some() { - return; - } - self.terminated_at = Some(now); - self.timeout_at = None; - if self.exit.is_none() { - self.cause = Some(cause); - self.terminate_request = Some(cause); - } - self.close_input(); - self.grace_until = after(now, self.timers.grace); - self.give_up_at = self - .grace_until - .and_then(|grace| after(grace, self.timers.settlement)); - if self.grace_until.is_some_and(|at| now >= at) { - self.end_grace(); - } - } - - /// Detaches the runner from the peer at `now` for `cause`: ends the workload if it still - /// runs, and discards its output at once, as no peer is left to take it. - fn detach(&mut self, now: Instant, cause: Cause) { - self.tick(now); - if self.finished || self.detached { - return; - } - self.terminate(now, cause); - self.detached = true; - self.started_message = false; - self.input.closed_message = false; - self.end_grace(); - let give_up = after(now, self.timers.settlement); - self.give_up_at = earliest(self.give_up_at, give_up); - } - - /// Ends the grace of pending output: drops what the queues hold, and requests to discard - /// each stream that is still open. - fn end_grace(&mut self) { - self.grace_until = None; - for output in &mut self.outputs { - output.drop_queue(); - if !output.closed && !output.discarding { - output.discarding = true; - output.discard_requested = true; - } - } - } - - /// Gives up on the execution after a termination: whatever is still unknown or open stays - /// so, each stream that did not close is truncated, and the execution is fenced. - fn give_up(&mut self) { - self.give_up_at = None; - self.grace_until = None; - self.gave_up = true; - for output in &mut self.outputs { - output.drop_queue(); - if !output.closed { - output.closed = true; - output.truncated = true; - } - } - if self.exit.is_none() { - self.exit = Some(None); - } - self.settling = Settling::Fenced; - self.settle_request = false; - } - - /// Gives up on the launch result, which has not arrived once the settlement time and the - /// grace have passed since the workload's end: a process that outlived the workload holds - /// what reports it. The execution's cleanup is incomplete, so the runner fences it, and then - /// gives up on its open streams too, and it sends the workload's output and terminal outcome - /// without the result. - fn abandon_launch(&mut self) { - self.launch_abandoned = true; - self.settling = Settling::Fenced; - } - - /// Gives up on the output streams that are still open after the workload's end, once the - /// execution did not settle, but for those that hung up: the processes that hold them - /// outlived the cleanup, so they may never close. The runner reads them no more and marks - /// them truncated, but still sends what it read, and loses nothing of the streams that closed - /// or hung up. - fn abandon_open_streams(&mut self) { - self.abandon_at = None; - for output in &mut self.outputs { - if !output.closed && !output.hung_up { - output.closed = true; - output.truncated = true; - } - } - } - - /// Returns whether the runner still waits for the workload's launch result: it has not - /// arrived, and the runner has not given up on it. - fn awaits_launch(&self) -> bool { - self.launch.is_none() && !self.gave_up && !self.launch_abandoned - } - - /// Returns whether the execution's cleanup is complete: the workload's launch result and - /// status are known, both of its streams closed, and the execution settled. Nothing is left - /// to give up on then but the delivery of the last messages, which the transport paces. - fn is_cleaned_up(&self) -> bool { - self.launch.is_some() - && self.exit.is_some() - && self.outputs.iter().all(|output| output.closed) - && self.settling == Settling::Settled - } - - /// Returns the terminal outcome once everything that it reports is known: the launch - /// result, the workload's status, both streams' ends, and the settlement. - fn outcome(&self) -> Option { - let delivered = self - .outputs - .iter() - .all(|output| output.is_done() && (output.ended || self.detached)); - let settled = matches!(self.settling, Settling::Settled | Settling::Fenced); - if self.awaits_launch() || self.exit.is_none() || !delivered || !settled { - return None; - } - let mut outcome = Outcome::new(self.exit.flatten()); - if let Some(cause) = self.cause { - outcome = outcome.with_cause(cause); - } - if let Some(Err(failure)) = self.launch { - outcome = outcome.with_launch_failure(failure); - } - for stream in Stream::ALL { - if self.outputs[stream.index()].truncated { - outcome = outcome.with_truncated(stream); - } - } - if self.settling == Settling::Settled { - outcome = outcome.with_reusable(); - } - Some(outcome) - } - - /// Notes the bytes that the runner holds, for its high-water mark. - fn note_buffered(&mut self) { - self.high_water = self.high_water.max(self.buffered()); - } -} diff --git a/nvx-rs/src/exec/tests/credit.rs b/nvx-rs/src/exec/tests/credit.rs deleted file mode 100644 index 4a91b8fb8..000000000 --- a/nvx-rs/src/exec/tests/credit.rs +++ /dev/null @@ -1,452 +0,0 @@ -//! The credit arithmetic: initial grants, window edges, tiny and maximal chunks and frames, a -//! shared pool, invalid grants and over-credit, and partial consumption. - -use super::*; - -/// Returns a peer of `limits`, created now, whose first grants the runner took. -fn granted_peer(limits: Limits) -> (Peer, Vec) { - let mut peer = Peer::new(KEY, limits, timers(), Instant::now()); - let grants = std::iter::from_fn(|| peer.next_message(&mut [])).collect(); - (peer, grants) -} - -/// Returns a runner of `limits`, created now, whose workload started. -fn started_runner(limits: Limits) -> Runner { - let mut runner = Runner::new(KEY, limits, timers(), Instant::now()); - runner.launched(Ok(())); - runner -} - -/// Returns every message that `runner` returns now, with an output buffer of `capacity` bytes, -/// and each output's bytes. -fn drain(runner: &mut Runner, capacity: usize) -> Vec { - let mut buffer = vec![0; capacity]; - std::iter::from_fn(|| { - runner - .next_message(&mut buffer) - .map(|message| to_peer(message, &buffer)) - }) - .collect() -} - -#[test] -fn each_receiver_first_grants_its_whole_window_once() { - let limits = bytes(4, 10, 8); - let (_, grants) = granted_peer(limits); - assert_eq!( - grants, - [ - peer::Message::Credit { - pool: Pool::Stdout, - units: 8 - }, - peer::Message::Credit { - pool: Pool::Stderr, - units: 8 - }, - ] - ); - let (_, grants) = granted_peer(limits.shared_output()); - assert_eq!( - grants, - [peer::Message::Credit { - pool: Pool::Output, - units: 8 - }] - ); - // The runner grants the input's window before the workload's launch, so that the peer may - // send input with its request. - let mut runner = Runner::new(KEY, limits, timers(), Instant::now()); - assert_eq!(drain(&mut runner, 0), [ToPeer::Credit(10)]); - runner.launched(Ok(())); - assert_eq!(drain(&mut runner, 0), [ToPeer::Started]); -} - -#[test] -fn a_byte_window_admits_data_up_to_its_edge_and_no_further() { - let limits = bytes(8, 8, 8); - let mut peer = Peer::new(KEY, limits, timers(), Instant::now()); - // Before its first grant, the window admits nothing. - assert_eq!( - peer.output(KEY, Stream::Stdout, b"a"), - Err(Violation::Window) - ); - while peer.next_message(&mut []).is_some() {} - assert_eq!( - peer.output(KEY, Stream::Stdout, b"12345"), - Ok(Receipt::Applied) - ); - assert_eq!( - peer.output(KEY, Stream::Stdout, b"6789"), - Err(Violation::Window) - ); - assert_eq!( - peer.output(KEY, Stream::Stdout, b"678"), - Ok(Receipt::Applied) - ); - assert_eq!( - peer.output(KEY, Stream::Stdout, b"9"), - Err(Violation::Window) - ); - // The other stream has a window of its own, and a refused message changed nothing. - assert_eq!( - peer.output(KEY, Stream::Stderr, b"abcdefgh"), - Ok(Receipt::Applied) - ); - assert_eq!(peer.pending_output(Stream::Stdout), b"12345678"); - // What the sink takes opens the window again, by as much. - peer.consumed(Stream::Stdout, 1); - assert_eq!( - peer.next_message(&mut []), - Some(peer::Message::Credit { - pool: Pool::Stdout, - units: 1 - }) - ); - assert_eq!(peer.output(KEY, Stream::Stdout, b"9"), Ok(Receipt::Applied)); - assert_eq!( - peer.output(KEY, Stream::Stdout, b"0"), - Err(Violation::Window) - ); -} - -#[test] -fn data_messages_carry_at_least_a_byte_and_at_most_a_chunk() { - let limits = bytes(4, 64, 64); - let (mut peer, _) = granted_peer(limits); - assert_eq!(peer.output(KEY, Stream::Stdout, b""), Err(Violation::Chunk)); - assert_eq!( - peer.output(KEY, Stream::Stdout, b"12345"), - Err(Violation::Chunk) - ); - assert_eq!( - peer.output(KEY, Stream::Stdout, b"1234"), - Ok(Receipt::Applied) - ); - let mut runner = Runner::new(KEY, limits, timers(), Instant::now()); - assert_eq!(runner.input(KEY, b""), Err(Violation::Chunk)); - while runner.next_message(&mut []).is_some() {} - assert_eq!(runner.input(KEY, b"12345"), Err(Violation::Chunk)); - assert_eq!(runner.input(KEY, b"1234"), Ok(Receipt::Applied)); -} - -#[test] -fn the_runner_sends_the_most_that_its_credit_a_chunk_and_the_buffer_allow() { - // A chunk of one byte sends each byte alone. - let mut runner = started_runner(bytes(1, 8, 8)); - assert_eq!(runner.read_allowance(Stream::Stdout), 1); - assert_eq!(runner.output(Stream::Stdout, b"ab"), 1); - runner.credit(KEY, Pool::Stdout, 8).unwrap(); - assert_eq!( - drain(&mut runner, 8), - [ - ToPeer::Credit(8), - ToPeer::Started, - ToPeer::Output(Stream::Stdout, b"a".to_vec()) - ] - ); - assert_eq!(runner.output(Stream::Stdout, b"b"), 1); - assert_eq!( - drain(&mut runner, 8), - [ToPeer::Output(Stream::Stdout, b"b".to_vec())] - ); - - // A larger chunk sends what the credit allows, then the rest once credit returns. - let mut runner = started_runner(bytes(6, 8, 8)); - assert_eq!(runner.output(Stream::Stdout, b"abcdef"), 6); - assert_eq!(runner.read_allowance(Stream::Stdout), 0); - runner.credit(KEY, Pool::Stdout, 4).unwrap(); - let messages = drain(&mut runner, 8); - assert_eq!( - messages[2..], - [ToPeer::Output(Stream::Stdout, b"abcd".to_vec())] - ); - // The queue has room again for what it sent, one chunk ahead of the credit. - assert_eq!(runner.read_allowance(Stream::Stdout), 4); - runner.credit(KEY, Pool::Stdout, 4).unwrap(); - // A small buffer bounds a message too. - let mut small = [0u8; 1]; - assert_eq!( - runner.next_message(&mut small), - Some(runner::Message::Output { - stream: Stream::Stdout, - length: 1 - }) - ); - assert_eq!(&small, b"e"); - assert_eq!( - drain(&mut runner, 8), - [ToPeer::Output(Stream::Stdout, b"f".to_vec())] - ); - // An empty buffer carries no output, but other messages still return. - assert_eq!(runner.output(Stream::Stdout, b"g"), 1); - runner.output_closed(Stream::Stderr); - assert_eq!(drain(&mut runner, 0), [ToPeer::End(Stream::Stderr)]); -} - -#[test] -fn a_frame_window_charges_one_unit_per_message_whatever_its_length() { - let limits = Limits::new(16, Window::frames(2), Window::frames(2)).unwrap(); - let (mut peer, grants) = granted_peer(limits); - assert_eq!( - grants[0], - peer::Message::Credit { - pool: Pool::Stdout, - units: 2 - } - ); - assert_eq!(peer.output(KEY, Stream::Stdout, b"x"), Ok(Receipt::Applied)); - assert_eq!( - peer.output(KEY, Stream::Stdout, &[7; 16]), - Ok(Receipt::Applied) - ); - assert_eq!( - peer.output(KEY, Stream::Stdout, b"y"), - Err(Violation::Window) - ); - assert_eq!( - peer.output(KEY, Stream::Stdout, &[7; 17]), - Err(Violation::Chunk) - ); - // A frame's unit returns only once the sink took its last byte. - peer.consumed(Stream::Stdout, 1); - assert_eq!( - peer.next_message(&mut []), - Some(peer::Message::Credit { - pool: Pool::Stdout, - units: 1 - }) - ); - peer.consumed(Stream::Stdout, 15); - assert_eq!(peer.next_message(&mut []), None); - peer.consumed(Stream::Stdout, 1); - assert_eq!( - peer.next_message(&mut []), - Some(peer::Message::Credit { - pool: Pool::Stdout, - units: 1 - }) - ); - - // The runner fills each frame with up to a chunk of what it read, whatever the reads. - let mut runner = started_runner(limits); - runner.credit(KEY, Pool::Stdout, 1).unwrap(); - for byte in b"abc" { - assert_eq!(runner.output(Stream::Stdout, &[*byte]), 1); - } - let messages = drain(&mut runner, 16); - assert_eq!( - messages[2..], - [ToPeer::Output(Stream::Stdout, b"abc".to_vec())] - ); - // With no frame left, it reads one chunk ahead and holds it. - assert_eq!(runner.output(Stream::Stdout, &[1; 20]), 16); - assert!(drain(&mut runner, 16).is_empty()); - runner.credit(KEY, Pool::Stdout, 2).unwrap(); - assert_eq!( - drain(&mut runner, 16), - [ToPeer::Output(Stream::Stdout, vec![1; 16])] - ); -} - -#[test] -fn a_shared_pool_charges_both_streams_and_answers_to_its_own_name() { - let limits = Limits::new(8, Window::bytes(8), Window::frames(3)) - .unwrap() - .shared_output(); - let (mut peer, _) = granted_peer(limits); - assert_eq!( - peer.output(KEY, Stream::Stdout, b"o1"), - Ok(Receipt::Applied) - ); - assert_eq!( - peer.output(KEY, Stream::Stderr, b"e1"), - Ok(Receipt::Applied) - ); - assert_eq!( - peer.output(KEY, Stream::Stdout, b"o2"), - Ok(Receipt::Applied) - ); - assert_eq!( - peer.output(KEY, Stream::Stderr, b"e2"), - Err(Violation::Window) - ); - // A stalled standard output holds the pool that standard error shares. - peer.consumed(Stream::Stderr, 2); - assert_eq!( - peer.next_message(&mut []), - Some(peer::Message::Credit { - pool: Pool::Output, - units: 1 - }) - ); - assert_eq!( - peer.output(KEY, Stream::Stderr, b"e2"), - Ok(Receipt::Applied) - ); - - let mut runner = started_runner(limits); - assert_eq!(runner.credit(KEY, Pool::Stdout, 1), Err(Violation::Pool)); - assert_eq!(runner.credit(KEY, Pool::Input, 1), Err(Violation::Pool)); - runner.credit(KEY, Pool::Output, 2).unwrap(); - // Each stream reads a chunk ahead, and both draw on the pool's two frames, in turns. - assert_eq!(runner.output(Stream::Stdout, b"out"), 3); - assert_eq!(runner.output(Stream::Stderr, b"err"), 3); - let messages = drain(&mut runner, 8); - assert_eq!( - messages[2..], - [ - ToPeer::Output(Stream::Stdout, b"out".to_vec()), - ToPeer::Output(Stream::Stderr, b"err".to_vec()), - ] - ); - assert_eq!(runner.output(Stream::Stdout, b"more"), 4); - assert!(drain(&mut runner, 8).is_empty()); - // Per-stream pools answer to their own names only. - let mut runner = started_runner(bytes(8, 8, 8)); - assert_eq!(runner.credit(KEY, Pool::Output, 1), Err(Violation::Pool)); -} - -#[test] -fn invalid_grants_and_over_credit_are_refused_and_change_nothing() { - let mut runner = started_runner(bytes(4, 8, 8)); - assert_eq!(runner.credit(KEY, Pool::Stdout, 0), Err(Violation::Credit)); - assert_eq!(runner.credit(KEY, Pool::Stdout, 9), Err(Violation::Credit)); - runner.credit(KEY, Pool::Stdout, 5).unwrap(); - // More than the units in flight would leave the runner more than the window. - assert_eq!(runner.credit(KEY, Pool::Stdout, 4), Err(Violation::Credit)); - runner.credit(KEY, Pool::Stdout, 3).unwrap(); - assert_eq!(runner.credit(KEY, Pool::Stdout, 1), Err(Violation::Credit)); - // Eight bytes of credit send eight bytes, and no more, in chunks of four. - let mut sent = 0; - for _ in 0..4 { - sent += runner.output(Stream::Stdout, b"abcd"); - for message in drain(&mut runner, 4) { - if let ToPeer::Output(_, bytes) = message { - assert!(bytes.len() <= 4); - } - } - } - assert_eq!(sent, 12); - assert_eq!(runner.read_allowance(Stream::Stdout), 0); - // Once the eight bytes are in flight, the peer may return them, and no more. - assert_eq!(runner.credit(KEY, Pool::Stdout, 9), Err(Violation::Credit)); - runner.credit(KEY, Pool::Stdout, 8).unwrap(); - - let mut peer = Peer::new(KEY, bytes(4, 8, 8), timers(), Instant::now()); - assert_eq!(peer.credit(KEY, 0), Err(Violation::Credit)); - assert_eq!(peer.credit(KEY, 9), Err(Violation::Credit)); - peer.credit(KEY, 8).unwrap(); - assert_eq!(peer.credit(KEY, 1), Err(Violation::Credit)); -} - -#[test] -fn maximal_windows_and_chunks_neither_overflow_nor_admit_more_than_the_window() { - let limits = Limits::new(u32::MAX, Window::bytes(u32::MAX), Window::frames(u32::MAX)).unwrap(); - let mut runner = started_runner(limits); - runner.credit(KEY, Pool::Stdout, u32::MAX).unwrap(); - assert_eq!(runner.credit(KEY, Pool::Stdout, 1), Err(Violation::Credit)); - runner.credit(KEY, Pool::Stderr, u32::MAX - 1).unwrap(); - runner.credit(KEY, Pool::Stderr, 1).unwrap(); - // Two chunks of output and the input's window, and two windows of maximal frames. - assert_eq!(runner.memory_bound(), (u32::MAX as usize).saturating_mul(3)); - let peer = Peer::new(KEY, limits, timers(), Instant::now()); - assert_eq!(peer.memory_bound(), usize::MAX); - // Empty chunks and windows are refused. - assert!(Limits::new(0, Window::bytes(1), Window::bytes(1)).is_none()); - assert!(Limits::new(1, Window::bytes(0), Window::bytes(1)).is_none()); - assert!(Limits::new(1, Window::bytes(1), Window::frames(0)).is_none()); -} - -#[test] -fn partial_consumption_returns_only_what_the_sink_took() { - let (mut peer, _) = granted_peer(bytes(8, 8, 8)); - peer.output(KEY, Stream::Stdout, b"abcdef").unwrap(); - assert_eq!(peer.pending_output(Stream::Stdout), b"abcdef"); - peer.consumed(Stream::Stdout, 2); - assert_eq!(peer.pending_output(Stream::Stdout), b"cdef"); - assert_eq!( - peer.next_message(&mut []), - Some(peer::Message::Credit { - pool: Pool::Stdout, - units: 2 - }) - ); - // Credits coalesce until the adapter retrieves them. - peer.consumed(Stream::Stdout, 1); - peer.consumed(Stream::Stdout, 2); - assert_eq!( - peer.next_message(&mut []), - Some(peer::Message::Credit { - pool: Pool::Stdout, - units: 3 - }) - ); - // Taking more than is pending takes what is pending. - peer.consumed(Stream::Stdout, 100); - assert_eq!( - peer.next_message(&mut []), - Some(peer::Message::Credit { - pool: Pool::Stdout, - units: 1 - }) - ); - assert_eq!(peer.next_message(&mut []), None); - - // The runner returns input credit for what the workload took, as it takes it. - let mut runner = started_runner(bytes(8, 8, 8)); - drain(&mut runner, 0); - runner.input(KEY, b"12345678").unwrap(); - assert_eq!(runner.input(KEY, b"9"), Err(Violation::Window)); - assert_eq!(runner.pending_input(), b"12345678"); - runner.input_written(3); - assert_eq!(runner.pending_input(), b"45678"); - assert_eq!(drain(&mut runner, 0), [ToPeer::Credit(3)]); - runner.input(KEY, b"9ab").unwrap(); - assert_eq!(runner.input(KEY, b"c"), Err(Violation::Window)); -} - -#[test] -fn the_peer_sends_input_within_the_runner_s_credit() { - let limits = bytes(4, 6, 8); - let mut peer = Peer::new(KEY, limits, timers(), Instant::now()); - let mut buffer = [0u8; 4]; - // The source may hand over one chunk ahead of the credit. - assert_eq!(peer.write_allowance(), 4); - assert_eq!(peer.input(b"abcdefgh"), 4); - assert_eq!(peer.write_allowance(), 0); - let messages = std::iter::from_fn(|| peer.next_message(&mut buffer)).collect::>(); - assert!( - messages - .iter() - .all(|message| matches!(message, peer::Message::Credit { .. })), - "{messages:?}" - ); - peer.credit(KEY, 6).unwrap(); - assert_eq!( - peer.next_message(&mut buffer), - Some(peer::Message::Input { length: 4 }) - ); - assert_eq!(&buffer, b"abcd"); - assert_eq!(peer.input(b"efgh"), 4); - assert_eq!( - peer.next_message(&mut buffer), - Some(peer::Message::Input { length: 2 }) - ); - assert_eq!(&buffer[..2], b"ef"); - assert_eq!(peer.next_message(&mut buffer), None); - // The end follows the last byte. - peer.end_input(); - assert_eq!(peer.write_allowance(), 0); - assert_eq!(peer.next_message(&mut buffer), None); - peer.credit(KEY, 4).unwrap(); - assert_eq!( - peer.next_message(&mut buffer), - Some(peer::Message::Input { length: 2 }) - ); - assert_eq!( - peer.next_message(&mut buffer), - Some(peer::Message::InputEnd) - ); - assert_eq!(peer.next_message(&mut buffer), None); -} diff --git a/nvx-rs/src/exec/tests/keys.rs b/nvx-rs/src/exec/tests/keys.rs deleted file mode 100644 index 9748687a7..000000000 --- a/nvx-rs/src/exec/tests/keys.rs +++ /dev/null @@ -1,136 +0,0 @@ -//! Messages of an earlier generation, of another execution or a later generation, and messages -//! that arrive after the side no longer needs them. - -use super::*; - -/// The key of the same execution's earlier generation. -const EARLIER: Key = Key::new(KEY.id(), KEY.generation() - 1); -/// The key of the same execution's later generation. -const LATER: Key = Key::new(KEY.id(), KEY.generation() + 1); -/// The key of another execution. -const OTHER: Key = Key::new(KEY.id() + 1, KEY.generation()); - -#[test] -fn messages_of_an_earlier_generation_are_stale_and_change_nothing() { - let now = Instant::now(); - let mut runner = Runner::new(KEY, bytes(8, 8, 8), timers(), now); - runner.launched(Ok(())); - while runner.next_message(&mut []).is_some() {} - assert_eq!(runner.credit(EARLIER, Pool::Stdout, 8), Ok(Receipt::Stale)); - assert_eq!(runner.input(EARLIER, b"stale"), Ok(Receipt::Stale)); - assert_eq!(runner.end_input(EARLIER), Ok(Receipt::Stale)); - assert_eq!( - runner.cancel(now, EARLIER, Cause::Cancelled), - Ok(Receipt::Stale) - ); - // No credit arrived, so the output waits; no input arrived; nothing ends the workload. - runner.output(Stream::Stdout, b"out"); - assert_eq!(runner.next_message(&mut [0; 8]), None); - assert_eq!(runner.pending_input(), b""); - assert_eq!(runner.next_request(), None); - assert_eq!(runner.input(KEY, b"12345678"), Ok(Receipt::Applied)); - - let mut peer = Peer::new(KEY, bytes(8, 8, 8), timers(), now); - while peer.next_message(&mut []).is_some() {} - assert_eq!(peer.credit(EARLIER, 8), Ok(Receipt::Stale)); - assert_eq!(peer.started(EARLIER), Ok(Receipt::Stale)); - assert_eq!( - peer.output(EARLIER, Stream::Stdout, b"stale"), - Ok(Receipt::Stale) - ); - assert_eq!(peer.end_output(EARLIER, Stream::Stdout), Ok(Receipt::Stale)); - assert_eq!(peer.input_closed(EARLIER), Ok(Receipt::Stale)); - let outcome = Outcome::new(Some(Status::Exited(0))); - assert_eq!(peer.terminal(now, EARLIER, outcome), Ok(Receipt::Stale)); - assert_eq!(peer.pending_output(Stream::Stdout), b""); - assert_eq!(peer.next_event(), None); - assert!(!peer.has_started()); - assert_eq!(peer.input(b"in"), 2); - assert_eq!(peer.next_message(&mut [0; 8]), None); - // The window of the current generation is whole. - assert_eq!( - peer.output(KEY, Stream::Stdout, b"12345678"), - Ok(Receipt::Applied) - ); -} - -#[test] -fn messages_of_another_execution_or_a_later_generation_are_refused() { - let now = Instant::now(); - let mut runner = Runner::new(KEY, bytes(8, 8, 8), timers(), now); - let mut peer = Peer::new(KEY, bytes(8, 8, 8), timers(), now); - for key in [LATER, OTHER] { - assert_eq!(runner.credit(key, Pool::Stdout, 1), Err(Violation::Key)); - assert_eq!(runner.input(key, b"x"), Err(Violation::Key)); - assert_eq!(runner.end_input(key), Err(Violation::Key)); - assert_eq!( - runner.cancel(now, key, Cause::Cancelled), - Err(Violation::Key) - ); - assert_eq!(peer.credit(key, 1), Err(Violation::Key)); - assert_eq!(peer.started(key), Err(Violation::Key)); - assert_eq!(peer.output(key, Stream::Stdout, b"x"), Err(Violation::Key)); - assert_eq!(peer.end_output(key, Stream::Stdout), Err(Violation::Key)); - assert_eq!(peer.input_closed(key), Err(Violation::Key)); - assert_eq!( - peer.terminal(now, key, Outcome::new(None)), - Err(Violation::Key) - ); - } - assert_eq!(runner.next_request(), None); - assert!(!peer.has_started()); -} - -#[test] -fn messages_after_the_execution_ended_are_late() { - let mut pair = Pair::new( - bytes(64, 64, 64), - timers(), - Workload::new(100, 0), - Client::new(), - ); - pair.finish(); - let now = pair.now; - let runner = &mut pair.runner; - assert_eq!(runner.credit(KEY, Pool::Stdout, 1), Ok(Receipt::Late)); - assert_eq!(runner.input(KEY, b"late"), Ok(Receipt::Late)); - assert_eq!(runner.end_input(KEY), Ok(Receipt::Late)); - assert_eq!(runner.cancel(now, KEY, Cause::Cancelled), Ok(Receipt::Late)); - // A stale message stays stale, and a foreign one refused. - assert_eq!(runner.credit(EARLIER, Pool::Stdout, 1), Ok(Receipt::Stale)); - assert_eq!(runner.credit(OTHER, Pool::Stdout, 1), Err(Violation::Key)); - assert_eq!(runner.next_message(&mut [0; 64]), None); - assert_eq!(runner.next_request(), None); - assert_eq!(runner.deadline(), None); - - let peer = &mut pair.peer; - assert_eq!(peer.credit(KEY, 1), Ok(Receipt::Late)); - assert_eq!(peer.started(KEY), Ok(Receipt::Late)); - assert_eq!(peer.output(KEY, Stream::Stdout, b"late"), Ok(Receipt::Late)); - assert_eq!(peer.end_output(KEY, Stream::Stdout), Ok(Receipt::Late)); - assert_eq!(peer.input_closed(KEY), Ok(Receipt::Late)); - assert_eq!( - peer.terminal(now, KEY, Outcome::new(None)), - Ok(Receipt::Late) - ); - assert_eq!(peer.next_message(&mut [0; 64]), None); - assert_eq!(peer.next_event(), None); - assert_eq!(peer.deadline(), None); -} - -#[test] -fn a_cancellation_after_a_termination_began_is_late() { - let now = Instant::now(); - let mut runner = Runner::new(KEY, bytes(8, 8, 8), timers(), now); - runner.launched(Ok(())); - assert_eq!( - runner.cancel(now, KEY, Cause::TimedOut), - Ok(Receipt::Applied) - ); - assert_eq!(runner.cancel(now, KEY, Cause::Cancelled), Ok(Receipt::Late)); - assert_eq!( - runner.next_request(), - Some(runner::Request::Terminate(Cause::TimedOut)) - ); - assert_eq!(runner.next_request(), None); -} diff --git a/nvx-rs/src/exec/tests/liveness.rs b/nvx-rs/src/exec/tests/liveness.rs deleted file mode 100644 index e9365d358..000000000 --- a/nvx-rs/src/exec/tests/liveness.rs +++ /dev/null @@ -1,400 +0,0 @@ -//! Liveness: a stalled sink, an exhausted window, or a full transport holds back only the bytes -//! that wait on it. Cancellations, timeouts, resets, and drops still end the execution by their -//! deadlines, and another execution keeps streaming meanwhile. Credit that frees up before every -//! message takes turns with each side's other messages, which still go out, and a stream's end -//! goes out while the other stream's output keeps flowing. - -use super::*; - -/// The limits of these executions. -fn limits() -> Limits { - bytes(4096, 4096, 16 * 1024) -} - -/// Returns an execution whose workload writes a mebibyte to each stream to a sink that takes -/// nothing. -fn stalled() -> Pair { - let mut client = Client::new(); - client.rates = [0, 0]; - Pair::new(limits(), timers(), Workload::new(1 << 20, 1 << 20), client) -} - -/// Returns an execution whose workload streams `length` bytes of standard output to a sink that -/// takes 8 KiB in each step. -fn streaming(length: usize) -> Pair { - let mut client = Client::new(); - client.rates = [8 * 1024, 8 * 1024]; - Pair::new(limits(), timers(), Workload::new(length, 0), client) -} - -/// Steps `first` and `second` in turns, `count` times, as one adapter's loop drives both. -fn step_both(first: &mut Pair, second: &mut Pair, count: usize) { - for _ in 0..count { - first.step(); - second.step(); - } -} - -/// Moves both executions' clocks to `now`. -fn set_clock(first: &mut Pair, second: &mut Pair, now: Instant) { - first.now = now; - second.now = now; -} - -/// Asserts that `pair`'s execution is stalled: the runner reads nothing, the workload still -/// runs, the sink took nothing, and the peer holds no more than its windows. -fn assert_stalled(pair: &Pair) { - for stream in Stream::ALL { - assert_eq!(pair.runner.read_allowance(stream), 0); - } - assert!(pair.workload.status.is_none()); - assert!(pair.peer.buffered() <= 2 * 16 * 1024); - assert_eq!(pair.client.received, [0, 0]); -} - -#[test] -fn a_stalled_sink_with_an_exhausted_window_stalls_only_its_own_execution() { - let mut stalled = stalled(); - let mut other = streaming(8 << 20); - for _ in 0..STEPS { - if other.peer.is_finished() { - break; - } - step_both(&mut stalled, &mut other, 1); - } - let outcome = other - .client - .outcome() - .expect("the other execution finished"); - assert_eq!(other.client.received, [8 << 20, 0]); - assert!(!outcome.is_truncated(Stream::Stdout)); - // The stalled execution waits for its sink, with no timer that would end it. - assert_stalled(&stalled); - assert!(!stalled.runner.is_finished() && !stalled.peer.is_finished()); - assert_eq!(stalled.runner.deadline(), None); - assert_eq!(stalled.peer.deadline(), None); -} - -/// How a liveness check ends the stalled execution. -#[derive(Clone, Copy, Debug, PartialEq, Eq)] -enum Ending { - /// The peer's caller cancels it. - Cancel, - /// The runner's timeout passes. - Timeout, - /// The session is reset. - Reset, - /// The peer's caller drops its handle. - Drop, -} - -#[test] -fn endings_finish_by_their_deadlines_while_a_sink_is_stalled_and_another_execution_streams() { - const TIMEOUT: Duration = Duration::from_secs(30); - for ending in [Ending::Cancel, Ending::Timeout, Ending::Reset, Ending::Drop] { - let mut stalled = stalled(); - if ending == Ending::Timeout { - stalled.runner = Runner::new(KEY, limits(), timers().timeout(TIMEOUT), stalled.now); - } - let mut other = streaming(8 << 20); - let start = stalled.now; - other.now = start; - step_both(&mut stalled, &mut other, 50); - assert_stalled(&stalled); - let ended = match ending { - Ending::Timeout => start + TIMEOUT, - _ => start + Duration::from_secs(1), - }; - set_clock(&mut stalled, &mut other, ended); - match ending { - Ending::Cancel => stalled.peer.cancel(ended), - Ending::Timeout => {} - Ending::Reset => { - stalled.runner.reset(ended); - stalled.peer.reset(ended); - } - Ending::Drop => stalled.peer.dropped(ended), - } - let streamed = other.client.received[0]; - step_both(&mut stalled, &mut other, 50); - // The workload ended at once, whatever its streams, and the other execution streamed on. - assert_eq!( - stalled.workload.status, - Some(Status::Signaled(9)), - "{ending:?}" - ); - assert!(other.client.received[0] > streamed, "{ending:?}"); - // A cancellation and a timeout leave the pending output its grace, as the deadline shows. - // A reset leaves no peer and a drop no sink to wait for, so their output goes at once. - if matches!(ending, Ending::Cancel | Ending::Timeout) { - assert!(!stalled.runner.is_finished(), "{ending:?}"); - assert_eq!(stalled.runner.deadline(), Some(ended + GRACE), "{ending:?}"); - } - set_clock(&mut stalled, &mut other, ended + GRACE); - let streamed = other.client.received[0]; - step_both(&mut stalled, &mut other, 10); - assert!(stalled.runner.is_finished(), "{ending:?}"); - // The peer learns of the runner's timeout only from the terminal outcome, which leaves - // its own sink a grace of its own. - if ending == Ending::Timeout { - assert!(!stalled.peer.is_finished()); - assert_eq!(stalled.peer.deadline(), Some(ended + GRACE + GRACE)); - set_clock(&mut stalled, &mut other, ended + GRACE + GRACE); - step_both(&mut stalled, &mut other, 10); - } - assert!(stalled.peer.is_finished(), "{ending:?}"); - assert!(other.client.received[0] > streamed, "{ending:?}"); - let cause = match ending { - Ending::Cancel => Cause::Cancelled, - Ending::Timeout => Cause::TimedOut, - Ending::Reset => Cause::Reset, - Ending::Drop => Cause::Dropped, - }; - let terminal = stalled.terminal(); - assert_eq!(terminal.cause(), Some(cause), "{ending:?}"); - assert_eq!(terminal.status(), Some(Status::Signaled(9)), "{ending:?}"); - assert!(terminal.is_reusable(), "{ending:?}"); - // A dropping peer takes and drops everything at once, so the runner discards nothing. - assert_eq!( - terminal.is_truncated(Stream::Stdout), - ending != Ending::Drop, - "{ending:?}" - ); - let outcome = stalled.client.outcome().unwrap(); - assert_eq!(outcome.cause(), Some(cause), "{ending:?}"); - assert!(outcome.is_truncated(Stream::Stdout), "{ending:?}"); - // The peer knows the runner's settlement unless the reset kept it from hearing. - assert_eq!(outcome.is_reusable(), ending != Ending::Reset, "{ending:?}"); - // The other execution still finishes whole. - other.finish(); - assert_eq!(other.client.received, [8 << 20, 0]); - assert!(!other.client.outcome().unwrap().is_truncated(Stream::Stdout)); - } -} - -#[test] -fn a_full_transport_holds_back_only_the_messages_that_wait_for_it() { - // The transport drains within the grace after the cancellation, or once the settlement time - // after the grace has passed too: the runner's outcome tells how the execution's cleanup - // went, however long the transport held it. - for held in [GRACE, GRACE + SETTLEMENT] { - let mut blocked = Pair::new(limits(), timers(), Workload::new(1 << 20, 0), Client::new()); - let mut other = streaming(8 << 20); - // The transport from the runner to the peer stops carrying messages, and fills before - // the credit runs out. - blocked.delivers_to_peer = false; - blocked.capacity = 3; - step_both(&mut blocked, &mut other, 50); - assert_eq!(blocked.to_peer.len(), 3); - assert_eq!(blocked.runner.read_allowance(Stream::Stdout), 0); - assert!(blocked.workload.status.is_none()); - let cancelled = blocked.now; - blocked.peer.cancel(cancelled); - let streamed = other.client.received[0]; - step_both(&mut blocked, &mut other, 50); - // The cancellation went the other way, and the runner ended the workload and settled - // it, though it could send nothing. - assert_eq!(blocked.workload.status, Some(Status::Signaled(9))); - assert!(blocked.workload.requests.contains(&runner::Request::Settle)); - assert!(other.client.received[0] > streamed); - set_clock(&mut blocked, &mut other, cancelled + GRACE); - step_both(&mut blocked, &mut other, 10); - assert!( - blocked - .workload - .requests - .contains(&runner::Request::Discard(Stream::Stdout)) - ); - // The execution's cleanup is complete, so the runner has no timer left: only the - // transport holds its last messages. - assert_eq!(blocked.workload.closed, [true, true]); - assert_eq!(blocked.runner.deadline(), None); - set_clock(&mut blocked, &mut other, cancelled + held); - step_both(&mut blocked, &mut other, 10); - // Once the transport drains, the runner sends no stale output: only both ends and the - // terminal outcome follow what the transport held. - let sent = blocked.sent_by_runner.len(); - blocked.delivers_to_peer = true; - blocked.capacity = usize::MAX; - blocked.finish(); - let after = &blocked.sent_by_runner[sent..]; - assert!( - after.iter().all(|message| matches!( - message, - runner::Message::End(_) | runner::Message::Terminal(_) - )), - "{after:?}" - ); - let terminal = blocked.terminal(); - assert_eq!(terminal.cause(), Some(Cause::Cancelled), "{held:?}"); - assert!(terminal.is_truncated(Stream::Stdout), "{held:?}"); - assert!(terminal.is_reusable(), "{held:?}"); - // The peer takes the terminal outcome while it waits for it; once it gave up on it, its - // own outcome stands, fenced. - let outcome = blocked.client.outcome().unwrap(); - assert_eq!(outcome.cause(), Some(Cause::Cancelled), "{held:?}"); - assert!(outcome.is_truncated(Stream::Stdout), "{held:?}"); - assert_eq!(outcome.is_reusable(), held == GRACE, "{held:?}"); - other.finish(); - assert_eq!(other.client.received, [8 << 20, 0]); - } -} - -#[test] -fn input_credit_takes_turns_with_the_runner_s_other_messages() { - // An adapter takes one message of the runner at a time, and before each, hands it the - // workload's output as far as it reads ahead, and all the input that the peer's credit - // allows, which the workload takes at once: credit of the input frees up before every - // message. - let mut runner = Runner::new(KEY, bytes(4, 4, 16), timers(), Instant::now()); - runner.launched(Ok(())); - runner - .credit(KEY, Pool::Stdout, 16) - .expect("the runner takes the peer's grant"); - runner.output_closed(Stream::Stderr); - let output = b"abcdefgh"; - let mut read = 0; - let mut buffer = [0; 4]; - let mut credit = 0; - let mut written = 0; - let mut others = Vec::new(); - for _ in 0..16 { - if read < output.len() { - let length = runner - .read_allowance(Stream::Stdout) - .min(output.len() - read); - read += runner.output(Stream::Stdout, &output[read..read + length]); - if read == output.len() { - runner.output_closed(Stream::Stdout); - } - } - if credit > 0 { - let input = vec![b'i'; credit]; - deliver_to_runner(&mut runner, Instant::now(), ToRunner::Input(input)); - runner.input_written(credit); - written += credit; - credit = 0; - } - match runner - .next_message(&mut buffer) - .map(|message| to_peer(message, &buffer)) - { - Some(ToPeer::Credit(units)) => credit += units as usize, - Some(message) => others.push(message), - None => break, - } - } - // The workload's start, the end of standard error, which closed with no byte, the output, - // and the end of standard output went out, in turns with the credit, which kept the input - // flowing. - assert_eq!( - others, - [ - ToPeer::Started, - ToPeer::End(Stream::Stderr), - ToPeer::Output(Stream::Stdout, b"abcd".to_vec()), - ToPeer::Output(Stream::Stdout, b"efgh".to_vec()), - ToPeer::End(Stream::Stdout), - ] - ); - assert!(written >= 4 * 4, "{written}"); -} - -#[test] -fn a_stream_s_end_goes_out_while_the_other_stream_s_output_keeps_flowing() { - // An adapter takes one message of the runner at a time, and before each, hands it as much of - // one stream's output as it reads ahead, while the peer's sink takes each output message at - // once and grants its credit back: that stream always has bytes and credit. The other stream - // closed at once, with no byte. - for flowing in Stream::ALL { - let (closed, pool) = match flowing { - Stream::Stdout => (Stream::Stderr, Pool::Stdout), - Stream::Stderr => (Stream::Stdout, Pool::Stderr), - }; - let now = Instant::now(); - let mut runner = Runner::new(KEY, bytes(4, 4, 16), timers(), now); - runner.launched(Ok(())); - deliver_to_runner(&mut runner, now, ToRunner::Credit(pool, 16)); - runner.output_closed(closed); - let mut buffer = [0; 4]; - let mut others = Vec::new(); - for _ in 0..16 { - let length = runner.read_allowance(flowing); - assert_eq!(runner.output(flowing, &vec![b'x'; length]), length); - let message = runner.next_message(&mut buffer).expect("output waits"); - match to_peer(message, &buffer) { - ToPeer::Credit(_) => {} - message => { - if let ToPeer::Output(_, bytes) = &message { - let units = u32::try_from(bytes.len()).unwrap(); - deliver_to_runner(&mut runner, now, ToRunner::Credit(pool, units)); - } - others.push(message); - } - } - } - // The closed stream's end went out right after the start, and the other stream's output - // flowed on, a full chunk in each message. - assert_eq!( - others[..2], - [ToPeer::Started, ToPeer::End(closed)], - "{flowing:?}" - ); - assert!(others.len() > 10, "{flowing:?}: {others:?}"); - assert!( - others[2..] - .iter() - .all(|message| *message == ToPeer::Output(flowing, b"xxxx".to_vec())), - "{flowing:?}: {others:?}" - ); - } -} - -#[test] -fn output_credit_takes_turns_with_the_peer_s_input_and_its_other_pool() { - // An adapter takes one message of the peer at a time, and before each, its source hands over - // as much input as the peer takes, and the runner sends a byte of standard output, which the - // sink takes at once: that pool's credit frees up before every message. - let mut peer = Peer::new(KEY, bytes(4, 16, 4), timers(), Instant::now()); - let mut buffer = [0; 4]; - while peer.next_message(&mut buffer).is_some() {} - peer.credit(KEY, 16) - .expect("the peer takes the runner's grant"); - peer.output(KEY, Stream::Stderr, b"zz") - .expect("the peer takes standard error within its window"); - peer.consumed(Stream::Stderr, 2); - let input = b"0123456789abcdef"; - let mut handed = 0; - let mut sent = Vec::new(); - for _ in 0..16 { - if handed < input.len() { - handed += peer.input(&input[handed..]); - if handed == input.len() { - peer.end_input(); - } - } - peer.output(KEY, Stream::Stdout, b"x") - .expect("the peer's credit keeps the runner's window open"); - peer.consumed(Stream::Stdout, 1); - let message = peer.next_message(&mut buffer).expect("credit waits"); - sent.push(to_runner(message, &buffer)); - } - // The input, its end, and the credit of standard error went out, in turns with the credit - // of standard output. - let others = sent - .iter() - .filter(|message| !matches!(message, ToRunner::Credit(Pool::Stdout, _))) - .cloned() - .collect::>(); - assert_eq!( - others, - [ - ToRunner::Input(b"0123".to_vec()), - ToRunner::Credit(Pool::Stderr, 2), - ToRunner::Input(b"4567".to_vec()), - ToRunner::Input(b"89ab".to_vec()), - ToRunner::Input(b"cdef".to_vec()), - ToRunner::InputEnd, - ] - ); -} diff --git a/nvx-rs/src/exec/tests/memory.rs b/nvx-rs/src/exec/tests/memory.rs deleted file mode 100644 index a896fe2cf..000000000 --- a/nvx-rs/src/exec/tests/memory.rs +++ /dev/null @@ -1,89 +0,0 @@ -//! Bounded memory: streams far longer than their windows arrive whole while each side holds no -//! more than its bound, with byte windows and with a shared pool of frames. - -use super::*; - -/// Asserts that neither side of `pair` ever held more than its bound, and returns both -/// high-water marks. -fn assert_bounded(pair: &Pair) -> (usize, usize) { - let runner = (pair.runner.high_water(), pair.runner.memory_bound()); - let peer = (pair.peer.high_water(), pair.peer.memory_bound()); - assert!(runner.0 <= runner.1, "the runner held {runner:?}"); - assert!(peer.0 <= peer.1, "the peer held {peer:?}"); - (runner.0, peer.0) -} - -#[test] -fn a_stream_far_longer_than_its_window_arrives_whole_in_bounded_memory() { - const STDOUT: usize = 64 << 20; - const STDERR: usize = 1 << 20; - const WINDOW: u32 = 64 * 1024; - let limits = bytes(16 * 1024, WINDOW, WINDOW); - let mut client = Client::new(); - client.rates = [48 * 1024, 5 * 1024]; - let mut pair = Pair::new(limits, timers(), Workload::new(STDOUT, STDERR), client); - let outcome = pair.finish(); - // The client checked each byte against the workload's pattern as it took it. - assert_eq!(pair.client.received, [STDOUT, STDERR]); - assert!(!outcome.is_truncated(Stream::Stdout) && !outcome.is_truncated(Stream::Stderr)); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - let (runner, peer) = assert_bounded(&pair); - // The runner held at most a chunk of each stream, and the peer at most both windows. - assert!(runner <= 2 * limits.chunk(), "{runner}"); - assert!(peer <= 2 * WINDOW as usize, "{peer}"); - assert!(pair.in_flight_high_water <= 2 * WINDOW as usize); -} - -#[test] -fn a_shared_pool_of_frames_carries_both_streams_whole_in_bounded_memory() { - // Frames of up to 64 KiB less 256 bytes, in a pool of 64 frames that both streams share. - const CHUNK: u32 = 64 * 1024 - 256; - const LENGTH: usize = 16 << 20; - let limits = Limits::new(CHUNK, Window::frames(64), Window::frames(64)) - .unwrap() - .shared_output(); - let mut client = Client::new(); - client.rates = [100 * 1024, 30 * 1024]; - let mut pair = Pair::new(limits, timers(), Workload::new(LENGTH, LENGTH / 4), client); - let outcome = pair.finish(); - assert_eq!(pair.client.received, [LENGTH, LENGTH / 4]); - assert!(!outcome.is_truncated(Stream::Stdout) && !outcome.is_truncated(Stream::Stderr)); - let (runner, peer) = assert_bounded(&pair); - assert!(runner <= 2 * CHUNK as usize, "{runner}"); - assert!(peer <= 64 * CHUNK as usize, "{peer}"); - // Each output message filled at most one frame. - assert!(pair.sent_by_runner.iter().all(|message| match message { - runner::Message::Output { length, .. } => *length <= CHUNK as usize, - _ => true, - })); -} - -#[test] -fn input_longer_than_its_window_arrives_whole() { - for limits in [ - bytes(4096, 16 * 1024, 16 * 1024), - Limits::new(1000, Window::frames(4), Window::bytes(4096)).unwrap(), - ] { - let mut workload = Workload::new(0, 0); - workload.reads_to_end = true; - workload.input_rate = 5_000; - let mut client = Client::new(); - client.input = (0..1usize << 20) - .map(|index| (index * 7 + index / 251) as u8) - .collect(); - let mut pair = Pair::new(limits, timers(), workload, client); - let outcome = pair.finish(); - assert_eq!(pair.workload.input, pair.client.input, "{limits:?}"); - assert!(pair.workload.input_end); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert!(outcome.is_reusable()); - let (runner, peer) = assert_bounded(&pair); - // The runner held at most the input's window, and the peer at most a chunk of input. - let window = match limits.input().unit() { - Unit::Bytes => limits.input().size() as usize, - Unit::Frames => limits.input().size() as usize * limits.chunk(), - }; - assert!(runner <= window, "{runner}"); - assert!(peer <= limits.chunk(), "{peer}"); - } -} diff --git a/nvx-rs/src/exec/tests/mod.rs b/nvx-rs/src/exec/tests/mod.rs deleted file mode 100644 index 52f2cbc7b..000000000 --- a/nvx-rs/src/exec/tests/mod.rs +++ /dev/null @@ -1,577 +0,0 @@ -//! Deterministic checks of the engine, with no operating-system calls: the credit arithmetic of -//! each side, the order of each stream, the completion rules, terminal races, bounded memory, and -//! liveness, with the sides connected by bounded queues, and a simulated workload and client. - -use std::collections::VecDeque; -use std::time::{Duration, Instant}; - -use super::{ - Cause, Key, LaunchFailure, Limits, Outcome, Peer, Pool, Receipt, Runner, Settlement, Status, - Stream, Timers, Unit, Violation, Window, peer, runner, -}; - -mod credit; -mod keys; -mod liveness; -mod memory; -mod races; -mod streams; -mod truncation; - -/// The key of the executions in these tests. -const KEY: Key = Key::new(7, 3); -/// The grace of pending output in these tests. -const GRACE: Duration = Duration::from_secs(1); -/// The settlement time in these tests. -const SETTLEMENT: Duration = Duration::from_secs(5); -/// The capacity of each simulated pipe between the workload and the runner's adapter. -const PIPE: usize = 64 * 1024; -/// The most steps that a test drives a pair through before it fails. -const STEPS: usize = 1_000_000; - -/// Returns limits with chunks of `chunk` bytes, an input window of `input` bytes, and an output -/// window of `output` bytes for each stream. -fn bytes(chunk: u32, input: u32, output: u32) -> Limits { - Limits::new(chunk, Window::bytes(input), Window::bytes(output)).unwrap() -} - -/// Returns the timers of these tests: a grace of [`GRACE`], a settlement time of -/// [`SETTLEMENT`], and no timeout. -fn timers() -> Timers { - Timers::new(GRACE, SETTLEMENT) -} - -/// Returns the byte at `position` of `stream`'s simulated output: a pattern that differs between -/// the streams and between nearby positions, so that a lost, repeated, or reordered byte shows. -fn pattern(stream: Stream, position: usize) -> u8 { - let salt = match stream { - Stream::Stdout => 0x5a, - Stream::Stderr => 0xa5, - }; - (position as u8) - ^ ((position >> 8) as u8).rotate_left(3) - ^ ((position >> 16) as u8).rotate_left(5) - ^ ((position >> 24) as u8).rotate_left(7) - ^ salt -} - -/// Returns `length` bytes of `stream`'s pattern from `position`. -fn patterned(stream: Stream, position: usize, length: usize) -> Vec { - (position..position + length) - .map(|position| pattern(stream, position)) - .collect() -} - -/// A message in flight from the peer to the runner, with its data. -#[derive(Clone, Debug, PartialEq, Eq)] -enum ToRunner { - /// A cancellation. - Cancel(Cause), - /// Credit of an output pool. - Credit(Pool, u32), - /// Input. - Input(Vec), - /// The input's end. - InputEnd, -} - -/// A message in flight from the runner to the peer, with its data. -#[derive(Clone, Debug, PartialEq, Eq)] -enum ToPeer { - /// Credit of the input. - Credit(u32), - /// The workload started. - Started, - /// Output. - Output(Stream, Vec), - /// A stream's end. - End(Stream), - /// The input closed early. - InputClosed, - /// The terminal outcome. - Terminal(Outcome), -} - -/// Returns the message that `message` describes, with the data that `buffer` holds. -fn to_runner(message: peer::Message, buffer: &[u8]) -> ToRunner { - match message { - peer::Message::Cancel(cause) => ToRunner::Cancel(cause), - peer::Message::Credit { pool, units } => ToRunner::Credit(pool, units), - peer::Message::Input { length } => ToRunner::Input(buffer[..length].to_vec()), - peer::Message::InputEnd => ToRunner::InputEnd, - } -} - -/// Returns the message that `message` describes, with the data that `buffer` holds. -fn to_peer(message: runner::Message, buffer: &[u8]) -> ToPeer { - match message { - runner::Message::Credit { units } => ToPeer::Credit(units), - runner::Message::Started => ToPeer::Started, - runner::Message::Output { stream, length } => { - ToPeer::Output(stream, buffer[..length].to_vec()) - } - runner::Message::End(stream) => ToPeer::End(stream), - runner::Message::InputClosed => ToPeer::InputClosed, - runner::Message::Terminal(outcome) => ToPeer::Terminal(outcome), - } -} - -/// Hands `message` to `runner` at `now`, which must accept it. -fn deliver_to_runner(runner: &mut Runner, now: Instant, message: ToRunner) -> Receipt { - let key = runner.key(); - match message { - ToRunner::Cancel(cause) => runner.cancel(now, key, cause), - ToRunner::Credit(pool, units) => runner.credit(key, pool, units), - ToRunner::Input(bytes) => runner.input(key, &bytes), - ToRunner::InputEnd => runner.end_input(key), - } - .expect("the runner accepts the peer's message") -} - -/// Hands `message` to `peer` at `now`, which must accept it. -fn deliver_to_peer(peer: &mut Peer, now: Instant, message: ToPeer) -> Receipt { - let key = peer.key(); - match message { - ToPeer::Credit(units) => peer.credit(key, units), - ToPeer::Started => peer.started(key), - ToPeer::Output(stream, bytes) => peer.output(key, stream, &bytes), - ToPeer::End(stream) => peer.end_output(key, stream), - ToPeer::InputClosed => peer.input_closed(key), - ToPeer::Terminal(outcome) => peer.terminal(now, key, outcome), - } - .expect("the peer accepts the runner's message") -} - -/// A simulated workload, as the runner's adapter drives it: a program that writes its output -/// through pipes of [`PIPE`] bytes, takes its input at a rate, and exits once it wrote all of its -/// output, after its input's end if it reads to it. -#[derive(Debug)] -struct Workload { - /// Whether the adapter reports a successful launch at the first step. - launches: bool, - /// Whether a process holds back what reports the launch, so that the adapter reports it only - /// once this is false again. - launch_held: bool, - /// Whether the adapter reported the launch. - launched: bool, - /// The bytes of each stream that the program writes in all. - lengths: [usize; 2], - /// The bytes of each stream that the program wrote into its pipe. - written: [usize; 2], - /// The bytes of each stream that the adapter read from the pipe. - read: [usize; 2], - /// Whether the adapter reported each stream's close. - closed: [bool; 2], - /// Whether the adapter discards each stream. - discarding: [bool; 2], - /// The bytes of each stream that the adapter discarded. - discarded: [usize; 2], - /// The input that the program took. - input: Vec, - /// The most input bytes that the program takes in one step. - input_rate: usize, - /// The input bytes after which the program closes its input, if it does. - input_limit: Option, - /// Whether the program's input closed. - input_closed: bool, - /// Whether the program saw its input's end. - input_end: bool, - /// Whether the program exits only once its input closed: at its end, or because the - /// program closed it. - reads_to_end: bool, - /// Whether the program exits once it has done its work; otherwise it runs until it is - /// ended. - exits: bool, - /// The program's own exit code. - code: i32, - /// The program's status, once it ended. - status: Option, - /// Whether the adapter reported the status. - reported: bool, - /// Whether a termination fails to end the program. - unkillable: bool, - /// Whether a process that the program left behind holds each stream's pipe open once the - /// program ended, until a settlement that succeeds ends it. - held: [bool; 2], - /// Whether the adapter reported that each stream hung up: no process holds its pipe any more. - hung_up: [bool; 2], - /// The settlement that the adapter reports once the runner requests it, if any. - settlement: Option, - /// Whether the runner requested the settlement and the adapter has not reported it. - settling: bool, - /// The requests that the runner made, in order. - requests: Vec, -} - -impl Workload { - /// Returns a workload that writes `stdout` and `stderr` bytes, takes all of its input at - /// once, exits with 0 without waiting for its input's end, and settles. - fn new(stdout: usize, stderr: usize) -> Self { - Self { - launches: true, - launch_held: false, - launched: false, - lengths: [stdout, stderr], - written: [0, 0], - read: [0, 0], - closed: [false, false], - discarding: [false, false], - discarded: [0, 0], - input: Vec::new(), - input_rate: usize::MAX, - input_limit: None, - input_closed: false, - input_end: false, - reads_to_end: false, - exits: true, - code: 0, - status: None, - reported: false, - unkillable: false, - held: [false, false], - hung_up: [false, false], - settlement: Some(Settlement::Settled), - settling: false, - requests: Vec::new(), - } - } - - /// Takes one step of the workload and of its adapter at `now`: applies the runner's - /// requests, writes the program's output into its pipes, feeds it input, reads the pipes as - /// the runner allows, reports what happened, and applies the requests that this makes. - fn step(&mut self, runner: &mut Runner, now: Instant) { - runner.tick(now); - self.apply_requests(runner); - if !self.launched && !self.launch_held { - self.launched = true; - if self.launches { - runner.launched(Ok(())); - } else { - runner.launched(Err(LaunchFailure::new(5, 2))); - self.status = Some(Status::Exited(125)); - self.lengths = [0, 0]; - } - } - if self.status.is_none() { - for index in 0..2 { - let room = PIPE - (self.written[index] - self.read[index]); - self.written[index] += room.min(self.lengths[index] - self.written[index]); - } - self.take_input(runner); - if self.exits - && self.written == self.lengths - && (!self.reads_to_end || self.input_closed) - { - self.status = Some(Status::Exited(self.code)); - self.input_closed = true; - } - } - for stream in Stream::ALL { - let index = stream.index(); - let held = self.written[index] - self.read[index]; - if self.discarding[index] { - self.read[index] += held; - self.discarded[index] += held; - runner.discarded(stream, held); - } else { - let length = held.min(runner.read_allowance(stream)); - let bytes = patterned(stream, self.read[index], length); - assert_eq!(runner.output(stream, &bytes), length); - self.read[index] += length; - } - if !self.closed[index] - && !self.held[index] - && self.status.is_some() - && self.written[index] == self.read[index] - { - self.closed[index] = true; - runner.output_closed(stream); - } - } - if self.status.is_some() && !self.reported { - self.reported = true; - runner.exited(now, self.status); - } - // Once the program ended, a stream that no process holds hung up, whatever its pipe - // still holds. - for stream in Stream::ALL { - let index = stream.index(); - if self.status.is_some() && !self.held[index] && !self.hung_up[index] { - self.hung_up[index] = true; - runner.output_hung_up(stream); - } - } - self.apply_requests(runner); - if self.settling - && let Some(settlement) = self.settlement - { - self.settling = false; - if settlement == Settlement::Settled { - self.held = [false, false]; - } - runner.settled(settlement); - } - } - - /// Applies the requests that the runner makes. - fn apply_requests(&mut self, runner: &mut Runner) { - while let Some(request) = runner.next_request() { - self.requests.push(request); - match request { - runner::Request::Terminate(_) => { - if !self.unkillable && self.status.is_none() { - self.status = Some(Status::Signaled(9)); - self.input_closed = true; - } - } - runner::Request::Discard(stream) => self.discarding[stream.index()] = true, - runner::Request::CloseInput => { - self.input_end = true; - self.input_closed = true; - } - runner::Request::Settle => self.settling = true, - } - } - } - - /// Feeds the program the input that the runner holds, at the program's rate, until the - /// program closes its input. - fn take_input(&mut self, runner: &mut Runner) { - let mut budget = self.input_rate; - while budget > 0 && !self.input_closed { - if self.input_limit == Some(self.input.len()) { - self.input_closed = true; - runner.input_closed(); - break; - } - let pending = runner.pending_input(); - let mut length = pending.len().min(budget); - if let Some(limit) = self.input_limit { - length = length.min(limit - self.input.len()); - } - if length == 0 { - break; - } - self.input.extend_from_slice(&pending[..length]); - runner.input_written(length); - budget -= length; - } - } -} - -/// The peer's caller, as the peer's adapter drives it: a source of input and a sink for each -/// output stream, which takes at most a rate of bytes in each step, and checks each byte -/// against the workload's pattern. -#[derive(Debug)] -struct Client { - /// The input that the source hands over in all. - input: Vec, - /// The bytes of input that the source handed over. - sent: usize, - /// Whether the source ends the input once it handed over everything. - ends_input: bool, - /// The most bytes that each sink takes in one step: zero stalls it. - rates: [usize; 2], - /// The bytes that each sink took. - received: [usize; 2], - /// The events, in order. - events: Vec, -} - -impl Client { - /// Returns a client that sends no input, ends it, and takes each stream at once. - fn new() -> Self { - Self { - input: Vec::new(), - sent: 0, - ends_input: true, - rates: [usize::MAX, usize::MAX], - received: [0, 0], - events: Vec::new(), - } - } - - /// Takes one step of the client and of the peer's adapter at `now`: hands the peer input as - /// it allows, takes output at the sinks' rates, and collects the peer's events. - fn step(&mut self, peer: &mut Peer, now: Instant) { - peer.tick(now); - if self.sent < self.input.len() { - self.sent += peer.input(&self.input[self.sent..]); - } - if self.ends_input && self.sent == self.input.len() { - peer.end_input(); - } - for stream in Stream::ALL { - let index = stream.index(); - let mut budget = self.rates[index]; - while budget > 0 { - let pending = peer.pending_output(stream); - let length = pending.len().min(budget); - if length == 0 { - break; - } - for (offset, &byte) in pending[..length].iter().enumerate() { - let position = self.received[index] + offset; - assert_eq!( - byte, - pattern(stream, position), - "{stream:?} byte {position}" - ); - } - self.received[index] += length; - peer.consumed(stream, length); - budget -= length; - } - } - while let Some(event) = peer.next_event() { - assert!( - !matches!(self.events.last(), Some(peer::Event::Finished(_))), - "the outcome is the last event" - ); - self.events.push(event); - } - } - - /// Returns the execution's outcome, once the peer finished. - fn outcome(&self) -> Option { - self.events.iter().find_map(|event| match event { - peer::Event::Finished(outcome) => Some(*outcome), - _ => None, - }) - } -} - -/// A runner and a peer, with their workload and client, connected by a queue of messages in each -/// direction, as their adapters and a transport would connect them. -struct Pair { - /// The current instant of the simulated clock. - now: Instant, - /// The runner. - runner: Runner, - /// The peer. - peer: Peer, - /// The runner's workload. - workload: Workload, - /// The peer's caller. - client: Client, - /// The messages in flight from the peer to the runner. - to_runner: VecDeque, - /// The messages in flight from the runner to the peer. - to_peer: VecDeque, - /// The most messages that each queue holds, after which its sender's adapter retrieves no - /// more. - capacity: usize, - /// Whether the transport delivers the runner's messages to the peer. - delivers_to_peer: bool, - /// Every message that the runner returned, in order, without its data. - sent_by_runner: Vec, - /// The terminal outcomes that the runner returned. - terminals: Vec, - /// The most data bytes in flight at once from the runner to the peer. - in_flight_high_water: usize, - /// The adapters' buffer. - buffer: Vec, -} - -impl Pair { - /// Returns a pair of sides with `limits` and `timers`, created at `now`, around `workload` - /// and `client`. - fn new(limits: Limits, timers: Timers, workload: Workload, client: Client) -> Self { - let now = Instant::now(); - Self { - now, - runner: Runner::new(KEY, limits, timers, now), - peer: Peer::new(KEY, limits, timers, now), - workload, - client, - to_runner: VecDeque::new(), - to_peer: VecDeque::new(), - capacity: usize::MAX, - delivers_to_peer: true, - sent_by_runner: Vec::new(), - terminals: Vec::new(), - in_flight_high_water: 0, - buffer: vec![0; limits.chunk()], - } - } - - /// Takes one step: the workload, the client, both adapters' retrieval of messages while the - /// queues have room, and the transport's delivery. - fn step(&mut self) { - self.workload.step(&mut self.runner, self.now); - self.client.step(&mut self.peer, self.now); - while self.to_runner.len() < self.capacity { - let Some(message) = self.peer.next_message(&mut self.buffer) else { - break; - }; - self.to_runner.push_back(to_runner(message, &self.buffer)); - } - while self.to_peer.len() < self.capacity { - let Some(message) = self.runner.next_message(&mut self.buffer) else { - break; - }; - self.sent_by_runner.push(message); - if let runner::Message::Terminal(outcome) = message { - self.terminals.push(outcome); - if self.runner.is_detached() { - continue; - } - } - self.to_peer.push_back(to_peer(message, &self.buffer)); - } - let in_flight = self - .to_peer - .iter() - .map(|message| match message { - ToPeer::Output(_, bytes) => bytes.len(), - _ => 0, - }) - .sum::(); - self.in_flight_high_water = self.in_flight_high_water.max(in_flight); - while let Some(message) = self.to_runner.pop_front() { - deliver_to_runner(&mut self.runner, self.now, message); - } - while self.delivers_to_peer - && let Some(message) = self.to_peer.pop_front() - { - if !self.peer.is_detached() { - deliver_to_peer(&mut self.peer, self.now, message); - } - } - } - - /// Takes steps until `done` holds, and fails after [`STEPS`] steps. - fn run_until(&mut self, mut done: impl FnMut(&Self) -> bool) { - for _ in 0..STEPS { - if done(self) { - return; - } - self.step(); - } - panic!("the pair did not get there in time"); - } - - /// Takes steps until both sides finished, and returns the peer's outcome. - fn finish(&mut self) -> Outcome { - self.run_until(|pair| pair.runner.is_finished() && pair.peer.is_finished()); - self.client.outcome().unwrap() - } - - /// Takes `count` steps. - fn steps(&mut self, count: usize) { - for _ in 0..count { - self.step(); - } - } - - /// Moves the clock forward by `duration`. - fn advance(&mut self, duration: Duration) { - self.now += duration; - } - - /// Returns the runner's terminal outcome, which it returned exactly once. - fn terminal(&self) -> Outcome { - assert_eq!(self.terminals.len(), 1, "one terminal outcome"); - self.terminals[0] - } -} diff --git a/nvx-rs/src/exec/tests/races.rs b/nvx-rs/src/exec/tests/races.rs deleted file mode 100644 index 6dbb7171b..000000000 --- a/nvx-rs/src/exec/tests/races.rs +++ /dev/null @@ -1,485 +0,0 @@ -//! Terminal races: the workload's exit against a cancellation, a timeout, a reset, and a -//! disconnection, in either order, on both sides, each with exactly one terminal outcome, and an -//! exit that arrives once the timeout passed, but before a tick fired it. - -use super::*; - -/// How a race ends the execution, besides the workload's exit. -#[derive(Clone, Copy, Debug, PartialEq, Eq)] -enum Ending { - /// The peer's cancellation. - Cancel, - /// The runner's timeout. - Timeout, - /// A reset of the session. - Reset, - /// A lost transport. - Disconnect, -} - -impl Ending { - /// Every ending. - const ALL: [Self; 4] = [Self::Cancel, Self::Timeout, Self::Reset, Self::Disconnect]; - - /// Returns the cause that the ending gives. - fn cause(self) -> Cause { - match self { - Self::Cancel => Cause::Cancelled, - Self::Timeout => Cause::TimedOut, - Self::Reset => Cause::Reset, - Self::Disconnect => Cause::Disconnected, - } - } - - /// Applies the ending to `runner` at `now`, which its timeout has reached. - fn apply(self, runner: &mut Runner, now: Instant) { - match self { - Self::Cancel => { - runner.cancel(now, KEY, Cause::Cancelled).unwrap(); - } - Self::Timeout => runner.tick(now), - Self::Reset => runner.reset(now), - Self::Disconnect => runner.disconnect(now), - } - } -} - -/// The timeout of the runners in these races. -const TIMEOUT: Duration = Duration::from_secs(10); - -/// Drives `runner` from `now` until it finishes, settling it when it asks, granting output credit -/// if `credit`, and moving the clock forward by a grace when it waits, and returns its messages -/// and requests. -fn drive( - runner: &mut Runner, - mut now: Instant, - credit: bool, -) -> (Vec, Vec) { - let mut buffer = [0u8; 64]; - let mut messages = Vec::new(); - let mut requests = Vec::new(); - for _ in 0..16 { - runner.tick(now); - while let Some(request) = runner.next_request() { - requests.push(request); - if request == runner::Request::Settle { - runner.settled(Settlement::Settled); - } - } - if credit && !runner.is_detached() && !runner.is_finished() { - let _ = runner.credit(KEY, Pool::Stdout, 64); - } - while let Some(message) = runner.next_message(&mut buffer) { - messages.push(message); - } - if runner.is_finished() { - break; - } - now += GRACE; - } - assert!(runner.is_finished(), "the runner finished"); - (messages, requests) -} - -#[test] -fn an_exit_racing_each_ending_gives_one_terminal_outcome() { - for ending in Ending::ALL { - for exit_first in [true, false] { - let start = Instant::now(); - // Only the timeout's race gives the runner a timeout, which the exit stops when it - // comes first. - let timers = if ending == Ending::Timeout { - timers().timeout(TIMEOUT) - } else { - timers() - }; - let mut runner = Runner::new(KEY, bytes(64, 64, 64), timers, start); - runner.launched(Ok(())); - // Output that waits for credit, which the ending's grace may discard. - assert_eq!(runner.output(Stream::Stdout, b"abc"), 3); - runner.output_closed(Stream::Stdout); - runner.output_closed(Stream::Stderr); - let now = start + TIMEOUT; - let status = if exit_first { - runner.exited(now - Duration::from_millis(1), Some(Status::Exited(3))); - ending.apply(&mut runner, now); - Status::Exited(3) - } else { - ending.apply(&mut runner, now); - runner.exited(now, Some(Status::Signaled(9))); - Status::Signaled(9) - }; - // A timeout after the exit changes nothing, so the output waits for credit. - let lossless = exit_first && ending == Ending::Timeout; - let (messages, requests) = drive(&mut runner, now, lossless); - let terminals = messages - .iter() - .filter_map(|message| match message { - runner::Message::Terminal(outcome) => Some(*outcome), - _ => None, - }) - .collect::>(); - assert_eq!(terminals.len(), 1, "{ending:?} {exit_first}"); - let outcome = terminals[0]; - assert_eq!(outcome.status(), Some(status), "{ending:?} {exit_first}"); - let cause = (!exit_first).then(|| ending.cause()); - assert_eq!(outcome.cause(), cause, "{ending:?} {exit_first}"); - assert_eq!( - requests - .iter() - .filter(|request| matches!(request, runner::Request::Terminate(_))) - .count(), - usize::from(!exit_first), - "{ending:?} {exit_first}" - ); - assert_eq!(outcome.is_truncated(Stream::Stdout), !lossless); - assert!(!outcome.is_truncated(Stream::Stderr)); - assert!(outcome.is_reusable()); - // A detached runner sends nothing but keeps its terminal outcome. - let detached = matches!(ending, Ending::Reset | Ending::Disconnect); - assert_eq!(messages.len() == 1, detached, "{messages:?}"); - // Nothing follows the terminal outcome. - assert_eq!(runner.next_message(&mut [0; 64]), None); - assert_eq!(runner.next_request(), None); - assert_eq!(runner.deadline(), None); - assert_eq!(runner.cancel(now, KEY, Cause::Cancelled), Ok(Receipt::Late)); - } - } -} - -#[test] -fn an_exit_that_arrives_once_the_timeout_passed_but_before_a_tick_stops_the_timeout() { - // The adapter learns of the exit only at the timeout's instant, or a grace after it, and - // hands it over before it ticks the runner: the end stands as the workload's own, and the - // output that waits for credit loses nothing. - for late in [Duration::ZERO, GRACE] { - let start = Instant::now(); - let mut runner = Runner::new(KEY, bytes(64, 64, 64), timers().timeout(TIMEOUT), start); - runner.launched(Ok(())); - assert_eq!(runner.output(Stream::Stdout, b"abc"), 3); - runner.output_closed(Stream::Stdout); - runner.output_closed(Stream::Stderr); - let now = start + TIMEOUT + late; - runner.exited(now, Some(Status::Exited(0))); - let (messages, requests) = drive(&mut runner, now, true); - assert!( - !requests - .iter() - .any(|request| matches!(request, runner::Request::Terminate(_))), - "{late:?}: {requests:?}" - ); - let Some(runner::Message::Terminal(outcome)) = messages.last() else { - panic!("{messages:?}"); - }; - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert_eq!(outcome.cause(), None, "{late:?}"); - assert!(!outcome.is_truncated(Stream::Stdout), "{late:?}"); - assert!(outcome.is_reusable()); - } -} - -#[test] -fn the_first_cause_wins() { - let now = Instant::now(); - let mut runner = Runner::new(KEY, bytes(64, 64, 64), timers().timeout(TIMEOUT), now); - runner.launched(Ok(())); - runner.tick(now + TIMEOUT); - runner.cancel(now + TIMEOUT, KEY, Cause::Cancelled).unwrap(); - runner.disconnect(now + TIMEOUT); - assert_eq!( - runner.next_request(), - Some(runner::Request::Terminate(Cause::TimedOut)) - ); - runner.exited(now + TIMEOUT, Some(Status::Signaled(9))); - runner.output_closed(Stream::Stdout); - runner.output_closed(Stream::Stderr); - let (messages, _) = drive(&mut runner, now + TIMEOUT, false); - let Some(runner::Message::Terminal(outcome)) = messages.last() else { - panic!("{messages:?}"); - }; - assert_eq!(outcome.cause(), Some(Cause::TimedOut)); -} - -#[test] -fn the_peer_finishes_once_whether_its_cancellation_or_the_terminal_outcome_comes_first() { - let terminal = |cause: Option| { - let outcome = Outcome::new(Some(Status::Signaled(9))).with_reusable(); - match cause { - Some(cause) => outcome.with_cause(cause), - None => outcome, - } - }; - for cancel_first in [true, false] { - for runner_cause in [None, Some(Cause::Cancelled), Some(Cause::TimedOut)] { - let now = Instant::now(); - let mut peer = Peer::new(KEY, bytes(8, 8, 8), timers(), now); - while peer.next_message(&mut []).is_some() {} - peer.output(KEY, Stream::Stdout, b"held").unwrap(); - peer.end_output(KEY, Stream::Stdout).unwrap(); - peer.end_output(KEY, Stream::Stderr).unwrap(); - if cancel_first { - peer.cancel(now); - assert_eq!( - peer.next_message(&mut []), - Some(peer::Message::Cancel(Cause::Cancelled)) - ); - peer.terminal(now, KEY, terminal(runner_cause)).unwrap(); - } else { - peer.terminal(now, KEY, terminal(runner_cause)).unwrap(); - peer.cancel(now); - // The runner finished, so the peer sends no cancellation. - assert_eq!(peer.next_message(&mut []), None); - } - // The stalled sink holds the output for the grace, then the peer discards it. - assert!( - !std::iter::from_fn(|| peer.next_event()) - .any(|event| matches!(event, peer::Event::Finished(_))) - ); - peer.tick(now + GRACE); - let finished = std::iter::from_fn(|| peer.next_event()) - .filter_map(|event| match event { - peer::Event::Finished(outcome) => Some(outcome), - _ => None, - }) - .collect::>(); - assert_eq!(finished.len(), 1); - let outcome = finished[0]; - // The runner's cause stands; the peer's own fills in for none. - assert_eq!(outcome.cause(), runner_cause.or(Some(Cause::Cancelled))); - assert!(outcome.is_truncated(Stream::Stdout)); - assert!(!outcome.is_truncated(Stream::Stderr)); - assert!(outcome.is_reusable()); - assert_eq!(peer.next_event(), None); - } - } -} - -#[test] -fn a_peer_detached_before_the_terminal_outcome_finishes_on_its_own() { - for ending in [Ending::Reset, Ending::Disconnect] { - let now = Instant::now(); - let mut peer = Peer::new(KEY, bytes(8, 8, 8), timers(), now); - while peer.next_message(&mut []).is_some() {} - peer.output(KEY, Stream::Stdout, b"held").unwrap(); - peer.end_output(KEY, Stream::Stderr).unwrap(); - match ending { - Ending::Reset => peer.reset(now), - _ => peer.disconnect(now), - } - // The sink still takes what arrived, for the grace. - assert_eq!(peer.pending_output(Stream::Stdout), b"held"); - peer.consumed(Stream::Stdout, 4); - let events = std::iter::from_fn(|| peer.next_event()).collect::>(); - let Some(peer::Event::Finished(outcome)) = events.last() else { - panic!("{events:?}"); - }; - // Neither stream's end can confirm that it ended whole without the terminal outcome. - assert_eq!(outcome.status(), None); - assert_eq!(outcome.cause(), Some(ending.cause())); - assert!(outcome.is_truncated(Stream::Stdout)); - assert!(outcome.is_truncated(Stream::Stderr)); - assert!(!outcome.is_reusable()); - assert_eq!(peer.next_message(&mut [0; 8]), None); - // What arrives after the detachment comes too late. - assert_eq!( - peer.terminal(now, KEY, Outcome::new(None)), - Ok(Receipt::Late) - ); - } -} - -/// Returns a peer that received four of twelve bytes of `stream` and both streams' ends, the -/// runner's withheld terminal outcome, and the instant at which the cancellation's grace ended. -fn peer_after_truncated_output(stream: Stream) -> (Peer, Outcome, Instant) { - let start = Instant::now(); - let limits = bytes(4, 64, 64); - let mut runner = Runner::new(KEY, limits, timers(), start); - let mut peer = Peer::new(KEY, limits, timers(), start); - let mut buffer = [0; 4]; - while let Some(message) = peer.next_message(&mut buffer) { - deliver_to_runner(&mut runner, start, to_runner(message, &buffer)); - } - runner.launched(Ok(())); - let written = b"0123456789ab"; - assert_eq!(runner.output(stream, &written[..4]), 4); - while let Some(message) = runner.next_message(&mut buffer) { - deliver_to_peer(&mut peer, start, to_peer(message, &buffer)); - } - assert_eq!(peer.pending_output(stream), &written[..4]); - peer.consumed(stream, 4); - peer.cancel(start); - while let Some(message) = peer.next_message(&mut buffer) { - deliver_to_runner(&mut runner, start, to_runner(message, &buffer)); - } - assert_eq!( - runner.next_request(), - Some(runner::Request::Terminate(Cause::Cancelled)) - ); - let now = start + GRACE; - runner.tick(now); - for output in Stream::ALL { - assert_eq!( - runner.next_request(), - Some(runner::Request::Discard(output)) - ); - } - runner.discarded(stream, written.len() - 4); - for output in Stream::ALL { - runner.output_closed(output); - } - runner.exited(now, Some(Status::Signaled(9))); - assert_eq!(runner.next_request(), Some(runner::Request::Settle)); - while let Some(message) = runner.next_message(&mut buffer) { - deliver_to_peer(&mut peer, now, to_peer(message, &buffer)); - } - assert_eq!( - std::iter::from_fn(|| peer.next_event()).collect::>(), - vec![ - peer::Event::Started, - peer::Event::InputClosed, - peer::Event::End(Stream::Stdout), - peer::Event::End(Stream::Stderr), - ] - ); - runner.settled(Settlement::Settled); - let Some(runner::Message::Terminal(terminal)) = runner.next_message(&mut buffer) else { - panic!("the terminal outcome follows the settlement"); - }; - assert_eq!( - terminal, - Outcome::new(Some(Status::Signaled(9))) - .with_cause(Cause::Cancelled) - .with_truncated(stream) - .with_reusable() - ); - (peer, terminal, now) -} - -#[test] -fn ended_streams_stay_truncated_after_a_peer_detaches_without_the_terminal_outcome() { - for stream in Stream::ALL { - for ending in [Ending::Reset, Ending::Disconnect] { - let (mut peer, terminal, now) = peer_after_truncated_output(stream); - match ending { - Ending::Reset => peer.reset(now), - _ => peer.disconnect(now), - } - assert_eq!(peer.terminal(now, KEY, terminal), Ok(Receipt::Late)); - let expected = Outcome::new(None) - .with_cause(Cause::Cancelled) - .with_truncated(Stream::Stdout) - .with_truncated(Stream::Stderr); - assert_eq!( - peer.next_event(), - Some(peer::Event::Finished(expected)), - "{stream:?} {ending:?}" - ); - assert_eq!(peer.next_event(), None); - } - } -} - -#[test] -fn ended_streams_stay_truncated_after_a_peer_gives_up_without_the_terminal_outcome() { - for stream in Stream::ALL { - let (mut peer, terminal, now) = peer_after_truncated_output(stream); - let deadline = now + SETTLEMENT; - peer.tick(deadline - Duration::from_nanos(1)); - assert_eq!(peer.next_event(), None); - peer.tick(deadline); - assert_eq!(peer.terminal(deadline, KEY, terminal), Ok(Receipt::Late)); - let expected = Outcome::new(None) - .with_cause(Cause::Cancelled) - .with_truncated(Stream::Stdout) - .with_truncated(Stream::Stderr); - assert_eq!( - peer.next_event(), - Some(peer::Event::Finished(expected)), - "{stream:?}" - ); - assert_eq!(peer.next_event(), None); - } -} - -#[test] -fn a_terminal_outcome_keeps_precise_truncation_even_if_the_peer_then_detaches() { - for stream in Stream::ALL { - for ending in [Ending::Reset, Ending::Disconnect] { - let (mut peer, terminal, now) = peer_after_truncated_output(stream); - assert_eq!(peer.terminal(now, KEY, terminal), Ok(Receipt::Applied)); - match ending { - Ending::Reset => peer.reset(now), - _ => peer.disconnect(now), - } - assert_eq!( - peer.next_event(), - Some(peer::Event::Finished(terminal)), - "{stream:?} {ending:?}" - ); - assert_eq!(peer.next_event(), None); - } - } -} - -#[test] -fn the_runner_s_messages_after_the_peer_detached_are_late_while_its_sink_drains() { - for ending in [Ending::Reset, Ending::Disconnect] { - let now = Instant::now(); - let mut peer = Peer::new(KEY, bytes(8, 8, 8), timers(), now); - while peer.next_message(&mut []).is_some() {} - peer.output(KEY, Stream::Stdout, b"held").unwrap(); - match ending { - Ending::Reset => peer.reset(now), - _ => peer.disconnect(now), - } - // The runner's last messages arrive while the sink still holds what came before, and - // before the caller retrieved the peer's outcome: the peer's own outcome stands. - assert_eq!(peer.output(KEY, Stream::Stdout, b"more"), Ok(Receipt::Late)); - for stream in Stream::ALL { - assert_eq!(peer.end_output(KEY, stream), Ok(Receipt::Late)); - } - let terminal = Outcome::new(Some(Status::Exited(0))).with_reusable(); - assert_eq!(peer.terminal(now, KEY, terminal), Ok(Receipt::Late)); - assert_eq!(peer.pending_output(Stream::Stdout), b"held"); - peer.consumed(Stream::Stdout, 4); - let events = std::iter::from_fn(|| peer.next_event()).collect::>(); - let Some(peer::Event::Finished(outcome)) = events.last() else { - panic!("{events:?}"); - }; - assert_eq!(outcome.status(), None, "{ending:?}"); - assert_eq!(outcome.cause(), Some(ending.cause())); - assert!(outcome.is_truncated(Stream::Stdout) && outcome.is_truncated(Stream::Stderr)); - assert!(!outcome.is_reusable()); - } -} - -#[test] -fn the_runner_s_messages_after_the_peer_gave_up_on_it_are_late() { - // The runner answers the cancellation only once the peer gave up waiting for it, but before - // the peer's caller retrieved its outcome: the peer's own outcome stands. - let now = Instant::now(); - let mut peer = Peer::new(KEY, bytes(8, 8, 8), timers(), now); - while peer.next_message(&mut []).is_some() {} - peer.started(KEY).unwrap(); - peer.cancel(now); - assert_eq!( - peer.next_message(&mut []), - Some(peer::Message::Cancel(Cause::Cancelled)) - ); - let late = now + GRACE + SETTLEMENT; - peer.tick(late); - assert_eq!(peer.output(KEY, Stream::Stdout, b"late"), Ok(Receipt::Late)); - for stream in Stream::ALL { - assert_eq!(peer.end_output(KEY, stream), Ok(Receipt::Late)); - } - let terminal = Outcome::new(Some(Status::Exited(0))).with_reusable(); - assert_eq!(peer.terminal(late, KEY, terminal), Ok(Receipt::Late)); - let events = std::iter::from_fn(|| peer.next_event()).collect::>(); - let Some(peer::Event::Finished(outcome)) = events.last() else { - panic!("{events:?}"); - }; - assert_eq!(outcome.status(), None); - assert_eq!(outcome.cause(), Some(Cause::Cancelled)); - assert!(outcome.is_truncated(Stream::Stdout) && outcome.is_truncated(Stream::Stderr)); - assert!(!outcome.is_reusable()); -} diff --git a/nvx-rs/src/exec/tests/streams.rs b/nvx-rs/src/exec/tests/streams.rs deleted file mode 100644 index f1bec5c70..000000000 --- a/nvx-rs/src/exec/tests/streams.rs +++ /dev/null @@ -1,222 +0,0 @@ -//! The order of each stream, the end of the input after every accepted byte, a workload that -//! closes its input early, and a launch that fails. - -use super::*; - -/// Returns the input bytes `0, 1, 2, ...` of `length` bytes. -fn counting(length: usize) -> Vec { - (0..length).map(|index| index as u8).collect() -} - -#[test] -fn each_stream_keeps_its_order_and_its_end_follows_its_last_byte() { - let mut client = Client::new(); - client.rates = [700, 300]; - let mut pair = Pair::new( - bytes(1000, 4096, 3000), - timers(), - Workload::new(10_000, 7_000), - client, - ); - let outcome = pair.finish(); - // The client checked every byte against the workload's pattern as it took it. - assert_eq!(pair.client.received, [10_000, 7_000]); - assert_eq!(outcome, pair.terminal()); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert_eq!(outcome.cause(), None); - assert!(outcome.is_reusable()); - assert!(!outcome.is_truncated(Stream::Stdout) && !outcome.is_truncated(Stream::Stderr)); - - let sent = &pair.sent_by_runner; - let position = |wanted: runner::Message| sent.iter().position(|message| *message == wanted); - let started = position(runner::Message::Started).unwrap(); - for stream in Stream::ALL { - let end = position(runner::Message::End(stream)).unwrap(); - let outputs = sent - .iter() - .enumerate() - .filter(|(_, message)| { - matches!(message, runner::Message::Output { stream: of, .. } if *of == stream) - }) - .map(|(index, _)| index) - .collect::>(); - assert!(outputs.iter().all(|&index| started < index && index < end)); - } - assert!(matches!(sent.last(), Some(runner::Message::Terminal(_)))); - assert_eq!( - sent.iter() - .filter(|message| matches!(message, runner::Message::Terminal(_))) - .count(), - 1 - ); - // The peer tells its caller in the same order. - let events = &pair.client.events; - assert_eq!(events.first(), Some(&peer::Event::Started)); - assert!(matches!(events.last(), Some(peer::Event::Finished(_)))); - for stream in Stream::ALL { - assert_eq!( - events - .iter() - .filter(|event| **event == peer::Event::End(stream)) - .count(), - 1 - ); - } -} - -#[test] -fn the_input_ends_only_after_every_accepted_byte() { - let mut workload = Workload::new(0, 0); - workload.reads_to_end = true; - workload.input_rate = 3; - let mut client = Client::new(); - client.input = counting(50); - let mut pair = Pair::new(bytes(8, 16, 64), timers(), workload, client); - let outcome = pair.finish(); - // The workload took every byte before its input closed, which ended the program. - assert_eq!(pair.workload.input, counting(50)); - assert_eq!( - pair.workload - .requests - .iter() - .filter(|request| **request == runner::Request::CloseInput) - .count(), - 1 - ); - assert!(pair.workload.input_end); - // The peer ended the input, so the runner never said that it closed early. - assert!(!pair.sent_by_runner.contains(&runner::Message::InputClosed)); - assert!(!pair.client.events.contains(&peer::Event::InputClosed)); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert!(outcome.is_reusable()); -} - -#[test] -fn a_workload_that_closes_its_input_early_never_leaves_the_source_waiting() { - let mut workload = Workload::new(2_000, 0); - workload.reads_to_end = true; - workload.input_limit = Some(5); - let mut client = Client::new(); - client.input = counting(100_000); - let mut pair = Pair::new(bytes(512, 1024, 1024), timers(), workload, client); - let outcome = pair.finish(); - assert_eq!(pair.workload.input, counting(5)); - assert!(pair.sent_by_runner.contains(&runner::Message::InputClosed)); - assert!(pair.client.events.contains(&peer::Event::InputClosed)); - // The source stopped once the input closed, short of its 100,000 bytes, and the output - // still arrived whole. - assert!(pair.client.sent < 100_000); - assert_eq!(pair.peer.write_allowance(), 0); - assert_eq!(pair.client.received, [2_000, 0]); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert!(!outcome.is_truncated(Stream::Stdout)); - assert!(outcome.is_reusable()); -} - -#[test] -fn input_after_the_workload_s_input_closed_is_dropped_within_the_window() { - let mut runner = Runner::new(KEY, bytes(4, 8, 8), timers(), Instant::now()); - runner.launched(Ok(())); - while runner.next_message(&mut []).is_some() {} - assert_eq!(runner.input(KEY, b"abcd"), Ok(Receipt::Applied)); - runner.input_closed(); - assert_eq!(runner.pending_input(), b""); - assert_eq!( - runner.next_message(&mut []), - Some(runner::Message::InputClosed) - ); - // The peer may still send what its credit allowed before it learned, and no more. - assert_eq!(runner.input(KEY, b"efgh"), Ok(Receipt::Late)); - assert_eq!(runner.input(KEY, b"i"), Err(Violation::Window)); - assert_eq!(runner.end_input(KEY), Ok(Receipt::Late)); - assert_eq!(runner.end_input(KEY), Err(Violation::Ended)); - assert_eq!(runner.next_request(), None); -} - -#[test] -fn closing_the_input_keeps_the_output_flowing() { - let mut workload = Workload::new(50_000, 3_000); - workload.reads_to_end = true; - let mut client = Client::new(); - client.input = counting(10); - client.rates = [1_000, 1_000]; - let mut pair = Pair::new(bytes(2048, 4096, 4096), timers(), workload, client); - let outcome = pair.finish(); - assert_eq!(pair.workload.input, counting(10)); - assert_eq!(pair.client.received, [50_000, 3_000]); - assert!(!outcome.is_truncated(Stream::Stdout)); -} - -#[test] -fn a_failed_launch_sends_no_start_and_reports_why() { - let mut workload = Workload::new(0, 0); - workload.launches = false; - let mut pair = Pair::new(bytes(64, 64, 64), timers(), workload, Client::new()); - let outcome = pair.finish(); - assert!(!pair.sent_by_runner.contains(&runner::Message::Started)); - assert!(!pair.client.events.contains(&peer::Event::Started)); - assert!(!pair.peer.has_started()); - assert_eq!(outcome.launch_failure(), Some(LaunchFailure::new(5, 2))); - assert_eq!(outcome.status(), Some(Status::Exited(125))); - assert!(outcome.is_reusable()); -} - -#[test] -fn a_second_start_end_or_terminal_is_out_of_order() { - let mut peer = Peer::new(KEY, bytes(8, 8, 8), timers(), Instant::now()); - let now = Instant::now(); - assert_eq!(peer.started(KEY), Ok(Receipt::Applied)); - assert_eq!(peer.started(KEY), Err(Violation::Order)); - let terminal = Outcome::new(Some(Status::Exited(0))).with_reusable(); - // The terminal outcome follows both streams' ends. - assert_eq!(peer.terminal(now, KEY, terminal), Err(Violation::Order)); - assert_eq!(peer.end_output(KEY, Stream::Stdout), Ok(Receipt::Applied)); - assert_eq!(peer.end_output(KEY, Stream::Stdout), Err(Violation::Ended)); - while peer.next_message(&mut []).is_some() {} - assert_eq!( - peer.output(KEY, Stream::Stdout, b"x"), - Err(Violation::Ended) - ); - assert_eq!(peer.terminal(now, KEY, terminal), Err(Violation::Order)); - assert_eq!(peer.end_output(KEY, Stream::Stderr), Ok(Receipt::Applied)); - assert_eq!(peer.terminal(now, KEY, terminal), Ok(Receipt::Applied)); - assert_eq!(peer.terminal(now, KEY, terminal), Err(Violation::Ended)); - let events = std::iter::from_fn(|| peer.next_event()).collect::>(); - assert_eq!( - events, - [ - peer::Event::Started, - peer::Event::InputClosed, - peer::Event::End(Stream::Stdout), - peer::Event::End(Stream::Stderr), - peer::Event::Finished(terminal), - ] - ); -} - -#[test] -fn a_start_after_the_terminal_outcome_of_a_failed_launch_is_out_of_order() { - // The runner reported that the launch failed, so no start follows, even before the peer's - // caller retrieved the outcome. - let now = Instant::now(); - let mut peer = Peer::new(KEY, bytes(8, 8, 8), timers(), now); - for stream in Stream::ALL { - assert_eq!(peer.end_output(KEY, stream), Ok(Receipt::Applied)); - } - let terminal = Outcome::new(Some(Status::Exited(125))) - .with_launch_failure(LaunchFailure::new(5, 2)) - .with_reusable(); - assert_eq!(peer.terminal(now, KEY, terminal), Ok(Receipt::Applied)); - assert_eq!(peer.started(KEY), Err(Violation::Order)); - assert!(!peer.has_started()); - let events = std::iter::from_fn(|| peer.next_event()).collect::>(); - assert_eq!( - events, - [ - peer::Event::InputClosed, - peer::Event::End(Stream::Stdout), - peer::Event::End(Stream::Stderr), - peer::Event::Finished(terminal), - ] - ); -} diff --git a/nvx-rs/src/exec/tests/truncation.rs b/nvx-rs/src/exec/tests/truncation.rs deleted file mode 100644 index 0e740c0f8..000000000 --- a/nvx-rs/src/exec/tests/truncation.rs +++ /dev/null @@ -1,525 +0,0 @@ -//! The abnormal paths and the normal one: a termination's grace and the truncation that follows -//! it, lossless normal completion, failed reads, failed and late settlements, the streams that an -//! execution that does not settle leaves open, and those that hung up, launch results that a -//! process holds back after the workload's end, and sides that give up and fence the execution. - -use super::*; - -/// Returns whether `outcome` marks either stream as truncated. -fn truncated(outcome: Outcome) -> bool { - Stream::ALL - .into_iter() - .any(|stream| outcome.is_truncated(stream)) -} - -#[test] -fn a_workload_that_ends_on_its_own_loses_no_byte_however_slow_the_sink() { - let mut client = Client::new(); - client.rates = [100, 37]; - let mut pair = Pair::new( - bytes(1024, 1024, 4096), - timers(), - Workload::new(100_000, 50_000), - client, - ); - // Each step takes longer than the grace and the settlement time together. - for _ in 0..STEPS { - if pair.peer.is_finished() { - break; - } - pair.advance(GRACE + SETTLEMENT); - pair.step(); - // The runner's terminal outcome waits for the output that the sink has yet to take, - // beyond what its windows hold. - if pair.runner.is_finished() { - assert!(pair.client.received[0] + 4096 >= 100_000); - assert!(pair.client.received[1] + 4096 >= 50_000); - } - } - let outcome = pair.client.outcome().unwrap(); - assert_eq!(pair.client.received, [100_000, 50_000]); - assert_eq!(outcome, pair.terminal()); - assert_eq!(outcome.cause(), None); - assert!(!truncated(outcome)); - assert!(outcome.is_reusable()); -} - -#[test] -fn a_termination_drains_pending_output_for_its_grace_then_marks_the_loss() { - let mut client = Client::new(); - client.rates = [0, 0]; - let mut pair = Pair::new( - bytes(4096, 4096, 16 * 1024), - timers(), - Workload::new(1 << 20, 100), - client, - ); - // The stalled sink exhausts both windows, and the workload blocks on its full pipe. - pair.steps(20); - assert_eq!(pair.runner.read_allowance(Stream::Stdout), 0); - assert!(pair.workload.status.is_none()); - let cancelled = pair.now; - pair.peer.cancel(cancelled); - pair.steps(20); - assert!( - pair.workload.status.is_some(), - "the cancellation ended the workload" - ); - assert!(!pair.runner.is_finished() && !pair.peer.is_finished()); - pair.advance(GRACE); - pair.steps(5); - assert!(pair.runner.is_finished() && pair.peer.is_finished()); - let terminal = pair.terminal(); - assert_eq!(terminal.cause(), Some(Cause::Cancelled)); - assert_eq!(terminal.status(), Some(Status::Signaled(9))); - assert!(terminal.is_truncated(Stream::Stdout)); - // Standard error's 100 bytes were in flight within its window, and the peer discarded them. - assert!(!terminal.is_truncated(Stream::Stderr)); - let outcome = pair.client.outcome().unwrap(); - assert!(outcome.is_truncated(Stream::Stdout) && outcome.is_truncated(Stream::Stderr)); - assert_eq!(outcome.cause(), Some(Cause::Cancelled)); - assert!(outcome.is_reusable()); - assert!( - pair.workload - .requests - .contains(&runner::Request::Discard(Stream::Stdout)) - ); - assert!(pair.workload.discarded[0] > 0); -} - -#[test] -fn a_termination_whose_pending_output_drains_within_the_grace_loses_nothing() { - let mut client = Client::new(); - client.rates = [0, 0]; - let mut pair = Pair::new( - bytes(4096, 4096, 16 * 1024), - timers(), - Workload::new(1 << 20, 0), - client, - ); - pair.steps(20); - pair.peer.cancel(pair.now); - // The sink takes everything again within the grace: what the workload wrote before its end - // all arrives, though the cancellation cut the workload short. - pair.client.rates = [usize::MAX, usize::MAX]; - pair.finish(); - let outcome = pair.client.outcome().unwrap(); - assert_eq!(outcome.cause(), Some(Cause::Cancelled)); - assert!(!truncated(outcome), "{outcome:?}"); - assert_eq!(pair.client.received[0], pair.workload.written[0]); - assert!(pair.workload.written[0] < 1 << 20); -} - -#[test] -fn a_failed_read_truncates_its_stream_without_a_cause() { - let now = Instant::now(); - let mut runner = Runner::new(KEY, bytes(8, 8, 8), timers(), now); - runner.launched(Ok(())); - runner.output_failed(Stream::Stdout); - runner.output_closed(Stream::Stderr); - runner.exited(now, Some(Status::Exited(0))); - runner.settled(Settlement::Settled); - let messages = std::iter::from_fn(|| runner.next_message(&mut [0; 8])).collect::>(); - let Some(runner::Message::Terminal(outcome)) = messages.last() else { - panic!("{messages:?}"); - }; - assert_eq!(outcome.cause(), None); - assert!(outcome.is_truncated(Stream::Stdout)); - assert!(!outcome.is_truncated(Stream::Stderr)); - assert_eq!(runner.read_allowance(Stream::Stdout), 0); -} - -#[test] -fn a_failed_or_late_settlement_fences_the_execution() { - for settlement in [Some(Settlement::Failed), None] { - let mut workload = Workload::new(10, 0); - workload.settlement = settlement; - let mut pair = Pair::new(bytes(64, 64, 64), timers(), workload, Client::new()); - pair.run_until(|pair| pair.workload.requests.contains(&runner::Request::Settle)); - pair.steps(3); - if settlement.is_none() { - // The runner waits for the settlement for its settlement time. - assert!(!pair.runner.is_finished()); - assert!(pair.runner.awaits_settlement()); - assert!(pair.runner.deadline().is_some()); - pair.advance(SETTLEMENT); - } - let outcome = pair.finish(); - assert!(!pair.runner.awaits_settlement(), "{settlement:?}"); - assert!(!outcome.is_reusable(), "{settlement:?}"); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert!(!truncated(outcome)); - // A result after the settlement time changes nothing. - pair.runner.settled(Settlement::Settled); - assert!(!pair.terminal().is_reusable()); - } -} - -#[test] -fn an_execution_that_does_not_settle_gives_up_on_the_streams_still_open_after_its_end() { - // A process that the workload left behind holds standard output open, and the settlement - // fails, or never comes. The runner leaves the stream the settlement time and then the - // grace after the workload's end to close, then gives up on it: it sends what it read, and - // marks only that stream truncated. - for settlement in [Some(Settlement::Failed), None] { - let mut workload = Workload::new(10_000, 100); - workload.held = [true, false]; - workload.settlement = settlement; - let mut pair = Pair::new(bytes(1024, 1024, 4096), timers(), workload, Client::new()); - pair.run_until(|pair| pair.workload.reported); - let exited = pair.now; - pair.run_until(|pair| { - pair.client.received == [10_000, 100] - && pair - .client - .events - .contains(&peer::Event::End(Stream::Stderr)) - }); - pair.steps(5); - assert!(!pair.runner.is_finished(), "{settlement:?}"); - assert!( - !pair - .client - .events - .contains(&peer::Event::End(Stream::Stdout)) - ); - if settlement.is_none() { - assert!(pair.runner.awaits_settlement()); - assert_eq!(pair.runner.deadline(), Some(exited + SETTLEMENT)); - pair.advance(SETTLEMENT); - pair.steps(5); - assert!(!pair.runner.awaits_settlement()); - assert!(!pair.runner.is_finished()); - } - assert_eq!( - pair.runner.deadline(), - Some(exited + SETTLEMENT + GRACE), - "{settlement:?}" - ); - pair.now = exited + SETTLEMENT + GRACE - Duration::from_millis(1); - pair.steps(5); - assert!(!pair.runner.is_finished(), "{settlement:?}"); - pair.now = exited + SETTLEMENT + GRACE; - let outcome = pair.finish(); - assert_eq!(outcome, pair.terminal()); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert_eq!(outcome.cause(), None); - assert!(outcome.is_truncated(Stream::Stdout)); - assert!(!outcome.is_truncated(Stream::Stderr)); - assert!(!outcome.is_reusable()); - assert_eq!(pair.client.received, [10_000, 100]); - assert_eq!(pair.runner.read_allowance(Stream::Stdout), 0); - assert!( - pair.sent_by_runner - .contains(&runner::Message::End(Stream::Stdout)) - ); - } -} - -#[test] -fn an_execution_that_does_not_settle_loses_nothing_of_the_streams_that_hung_up() { - // No process holds the streams once the workload ended, but the settlement fails, or never - // comes, and the sink takes nothing for long after the settlement time and the grace. The - // streams hung up, so the runner never gives up on them: it waits for the sink, and loses - // nothing, though the execution is fenced. - for settlement in [Some(Settlement::Failed), None] { - let mut workload = Workload::new(10_000, 5_000); - workload.settlement = settlement; - let mut client = Client::new(); - client.rates = [0, 0]; - let mut pair = Pair::new(bytes(1024, 1024, 4096), timers(), workload, client); - pair.run_until(|pair| pair.workload.reported); - pair.steps(5); - pair.advance((SETTLEMENT + GRACE) * 10); - pair.steps(5); - assert!(!pair.runner.is_finished(), "{settlement:?}"); - assert_eq!(pair.runner.deadline(), None, "{settlement:?}"); - pair.client.rates = [usize::MAX, usize::MAX]; - let outcome = pair.finish(); - assert_eq!(pair.client.received, [10_000, 5_000], "{settlement:?}"); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert_eq!(outcome.cause(), None); - assert!(!truncated(outcome), "{settlement:?}"); - assert!(!outcome.is_reusable(), "{settlement:?}"); - } -} - -#[test] -fn a_settled_execution_waits_for_its_sink_however_long_its_streams_stay_open() { - // The settlement ends the process that held standard output, and the stalled sink takes the - // rest long after the settlement time and the grace: nothing is lost. - let mut workload = Workload::new(10_000, 0); - workload.held = [true, false]; - let mut client = Client::new(); - client.rates = [0, 0]; - let mut pair = Pair::new(bytes(1024, 1024, 4096), timers(), workload, client); - pair.run_until(|pair| pair.workload.reported); - pair.steps(5); - assert!(!pair.runner.awaits_settlement()); - assert_eq!(pair.runner.deadline(), None); - pair.advance((SETTLEMENT + GRACE) * 10); - pair.steps(5); - assert!(!pair.runner.is_finished()); - pair.client.rates = [usize::MAX, usize::MAX]; - let outcome = pair.finish(); - assert_eq!(pair.client.received, [10_000, 0]); - assert!(!truncated(outcome)); - assert!(outcome.is_reusable()); -} - -#[test] -fn the_runner_gives_up_on_a_launch_result_that_a_process_holds_back_after_the_workload_s_end() { - // A process holds back what reports the workload's launch, whether the execution settles, - // its settlement fails, or never comes, and only the runner has a timeout, which the - // workload's end disarmed before it passed. The runner leaves the launch result the - // settlement time and then the grace after the workload's end to arrive, holding the start - // and the output meanwhile, then gives up on it: it sends the output, which the streams that - // hung up keep whole, and the outcome, without a start, with the workload's status and no - // cause, and fences the execution. The result arrives once the runner sent output and an - // end, but before the terminal outcome, and counts for nothing, so no start follows them. - const TIMEOUT: Duration = Duration::from_secs(2); - let limits = bytes(64, 64, 64); - for settlement in [Some(Settlement::Settled), Some(Settlement::Failed), None] { - let mut workload = Workload::new(100, 10); - workload.launch_held = true; - workload.settlement = settlement; - let mut pair = Pair::new(limits, timers(), workload, Client::new()); - pair.runner = Runner::new(KEY, limits, timers().timeout(TIMEOUT), pair.now); - pair.run_until(|pair| pair.workload.reported); - let exited = pair.now; - pair.steps(5); - let held = |pair: &Pair| { - !pair.runner.is_finished() - && !pair.sent_by_runner.iter().any(|message| { - matches!( - message, - runner::Message::Started - | runner::Message::Output { .. } - | runner::Message::End(_) - ) - }) - }; - assert!(held(&pair), "{settlement:?}"); - if settlement.is_none() { - assert_eq!(pair.runner.deadline(), Some(exited + SETTLEMENT)); - pair.advance(SETTLEMENT); - pair.steps(5); - assert!(!pair.runner.awaits_settlement()); - assert!(held(&pair)); - } - assert_eq!( - pair.runner.deadline(), - Some(exited + SETTLEMENT + GRACE), - "{settlement:?}" - ); - pair.now = exited + SETTLEMENT + GRACE - Duration::from_millis(1); - pair.steps(5); - assert!(held(&pair), "{settlement:?}"); - pair.now = exited + SETTLEMENT + GRACE; - pair.run_until(|pair| !held(pair)); - assert!(!pair.runner.is_finished(), "{settlement:?}"); - pair.workload.launch_held = false; - let outcome = pair.finish(); - assert!(pair.workload.launched); - assert_eq!(outcome, pair.terminal()); - assert_eq!(outcome.status(), Some(Status::Exited(0)), "{settlement:?}"); - assert_eq!(outcome.cause(), None, "{settlement:?}"); - assert_eq!(outcome.launch_failure(), None); - assert!(!truncated(outcome), "{settlement:?}"); - assert!(!outcome.is_reusable(), "{settlement:?}"); - assert_eq!(pair.client.received, [100, 10]); - assert!(!pair.sent_by_runner.contains(&runner::Message::Started)); - assert!(!pair.peer.has_started()); - // A launch result after the terminal outcome changes nothing. - pair.runner.launched(Ok(())); - assert_eq!(pair.runner.next_message(&mut []), None); - } -} - -#[test] -fn a_launch_result_after_a_termination_s_give_up_counts_for_nothing() { - // A cancellation ends a workload whose launch result never arrives, and which never ends, so - // the runner gives up on it once the grace and the settlement time have passed. The result - // arrives once the runner sent a stream's end, but before the terminal outcome, and counts - // for nothing: no start follows the end. - let now = Instant::now(); - let mut runner = Runner::new(KEY, bytes(8, 8, 8), timers(), now); - assert_eq!( - runner.cancel(now, KEY, Cause::Cancelled), - Ok(Receipt::Applied) - ); - runner.tick(now + GRACE + SETTLEMENT); - let mut sent = Vec::new(); - while let Some(message) = runner.next_message(&mut [0; 8]) { - sent.push(message); - if matches!(message, runner::Message::End(_)) { - break; - } - } - assert!( - matches!(sent.last(), Some(runner::Message::End(_))), - "{sent:?}" - ); - runner.launched(Ok(())); - sent.extend(std::iter::from_fn(|| runner.next_message(&mut [0; 8]))); - assert!(!sent.contains(&runner::Message::Started), "{sent:?}"); - let Some(runner::Message::Terminal(outcome)) = sent.last() else { - panic!("{sent:?}"); - }; - assert_eq!(outcome.cause(), Some(Cause::Cancelled)); - assert_eq!(outcome.launch_failure(), None); - assert!(!outcome.is_reusable()); -} - -#[test] -fn a_launch_result_that_arrives_after_the_workload_s_end_but_in_time_still_counts() { - // A process holds back what reports the workload's launch until just before the runner would - // give up on it: the runner sends the start first, the outcome is lossless, and the execution - // is reusable. - let mut workload = Workload::new(100, 10); - workload.launch_held = true; - let mut pair = Pair::new(bytes(64, 64, 64), timers(), workload, Client::new()); - pair.run_until(|pair| pair.workload.reported); - pair.now += SETTLEMENT + GRACE - Duration::from_millis(1); - pair.steps(5); - assert!(!pair.runner.is_finished()); - assert!(!pair.sent_by_runner.contains(&runner::Message::Started)); - pair.workload.launch_held = false; - let outcome = pair.finish(); - assert_eq!(outcome, pair.terminal()); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert_eq!(outcome.cause(), None); - assert!(!truncated(outcome)); - assert!(outcome.is_reusable()); - assert_eq!(pair.client.received, [100, 10]); - let started = pair - .sent_by_runner - .iter() - .position(|message| *message == runner::Message::Started) - .expect("the runner sent the start"); - assert!( - pair.sent_by_runner[..started] - .iter() - .all(|message| matches!( - message, - runner::Message::Credit { .. } | runner::Message::InputClosed - )), - "{:?}", - pair.sent_by_runner - ); -} - -#[test] -fn a_settlement_that_the_runner_did_not_request_counts_for_nothing() { - let now = Instant::now(); - let mut runner = Runner::new(KEY, bytes(8, 8, 8), timers(), now); - runner.launched(Ok(())); - runner.output_closed(Stream::Stdout); - runner.output_closed(Stream::Stderr); - // The workload still runs, so nothing settles it yet. - runner.settled(Settlement::Settled); - assert!(!runner.awaits_settlement()); - runner.exited(now, Some(Status::Exited(0))); - assert!(runner.awaits_settlement()); - assert_eq!(runner.next_request(), Some(runner::Request::Settle)); - while let Some(message) = runner.next_message(&mut []) { - assert!( - !matches!(message, runner::Message::Terminal(_)), - "{message:?}" - ); - } - runner.settled(Settlement::Failed); - assert!(!runner.awaits_settlement()); - let Some(runner::Message::Terminal(outcome)) = runner.next_message(&mut []) else { - panic!("the terminal outcome follows the settlement"); - }; - assert!(!outcome.is_reusable()); -} - -#[test] -fn the_runner_gives_up_on_a_workload_that_will_not_end_and_fences_it() { - let mut workload = Workload::new(1 << 20, 0); - workload.unkillable = true; - let mut client = Client::new(); - client.rates = [0, 0]; - let mut pair = Pair::new(bytes(4096, 4096, 4096), timers(), workload, client); - pair.steps(10); - let cancelled = pair.now; - pair.peer.cancel(cancelled); - pair.steps(10); - pair.advance(GRACE); - pair.steps(10); - // The workload still runs and holds its stream open, so the runner waits. - assert!(!pair.runner.is_finished()); - assert_eq!(pair.runner.deadline(), Some(cancelled + GRACE + SETTLEMENT)); - pair.advance(SETTLEMENT); - pair.steps(5); - assert!(pair.runner.is_finished() && pair.peer.is_finished()); - let terminal = pair.terminal(); - assert_eq!(terminal.status(), None); - assert_eq!(terminal.cause(), Some(Cause::Cancelled)); - assert!(terminal.is_truncated(Stream::Stdout)); - assert!(!terminal.is_reusable()); - // The runner still ended both streams before its terminal outcome. - for stream in Stream::ALL { - assert!(pair.sent_by_runner.contains(&runner::Message::End(stream))); - } - assert_eq!(pair.client.outcome(), Some(terminal)); - assert_eq!(pair.runner.read_allowance(Stream::Stdout), 0); -} - -#[test] -fn a_peer_that_ended_the_execution_gives_up_on_a_silent_runner() { - let mut pair = Pair::new( - bytes(64, 64, 64), - timers(), - Workload::new(1 << 20, 0), - Client::new(), - ); - pair.steps(5); - // The transport stops delivering the runner's messages. - pair.delivers_to_peer = false; - let cancelled = pair.now; - pair.peer.cancel(cancelled); - pair.steps(5); - pair.advance(GRACE); - pair.steps(5); - assert!(!pair.peer.is_finished()); - assert_eq!(pair.peer.deadline(), Some(cancelled + GRACE + SETTLEMENT)); - pair.advance(SETTLEMENT); - pair.steps(1); - let outcome = pair.client.outcome().expect("the peer gave up"); - assert_eq!(outcome.status(), None); - assert_eq!(outcome.cause(), Some(Cause::Cancelled)); - assert!(outcome.is_truncated(Stream::Stdout) && outcome.is_truncated(Stream::Stderr)); - assert!(!outcome.is_reusable()); -} - -#[test] -fn the_peer_s_own_timeout_cancels_and_names_itself() { - let mut client = Client::new(); - client.rates = [0, 0]; - let mut pair = Pair::new( - bytes(64, 64, 64), - timers().timeout(Duration::from_secs(30)), - Workload::new(1 << 20, 0), - client, - ); - // Only the peer's timer runs out: the runner's workload got the same timeout, but the - // runner reports the peer's cause, which arrives first. - pair.runner = Runner::new(KEY, bytes(64, 64, 64), timers(), pair.now); - pair.steps(5); - pair.advance(Duration::from_secs(30)); - pair.steps(5); - let terminations = pair - .workload - .requests - .iter() - .filter(|request| matches!(request, runner::Request::Terminate(_))) - .collect::>(); - assert_eq!(terminations, [&runner::Request::Terminate(Cause::TimedOut)]); - pair.advance(GRACE); - let outcome = pair.finish(); - assert_eq!(outcome.cause(), Some(Cause::TimedOut)); - assert_eq!(pair.terminal().cause(), Some(Cause::TimedOut)); -} diff --git a/nvx-rs/src/guest/exec.rs b/nvx-rs/src/guest/exec.rs index bc560c224..fcf4933df 100644 --- a/nvx-rs/src/guest/exec.rs +++ b/nvx-rs/src/guest/exec.rs @@ -11,10 +11,7 @@ //! It needs neither NVX's control protocol nor an async runtime, and it keeps no policy: the //! caller decides which programs run and how many at once, where their output goes, how much of //! it to take, and what each outcome means. NVX's managed agent is one caller, which runs one -//! execution at a time and forwards its output and outcome as NVXC frames. A caller whose peer -//! runs the [execution engine](crate::exec)'s other side drives the execution through a -//! [`Binding`](binding::Binding) instead, which turns the engine's credit, input, endings, and -//! settlement into the execution's calls, and its events into the engine's inputs. +//! execution at a time and forwards its output and outcome as NVXC frames. //! //! # Driving //! @@ -125,8 +122,6 @@ use super::process::Child; use super::spawn::{LaunchFailure, LaunchReport, Spawned}; use super::sys::{self, Errno, FileDescriptor, ProcessId}; -pub mod binding; - /// One of a program's output streams. #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] pub enum Stream { @@ -703,15 +698,6 @@ impl Execution { self.outputs.iter().any(Output::is_open) } - /// Returns whether every writer of `stream` closed it, without reading it: the stream ends - /// once the execution reads what its pipe still holds. A stream that closed hung up too. - pub(crate) fn output_hung_up(&self, stream: Stream) -> bool { - match &self.outputs[stream.index()].reader { - Some(reader) => sys::hung_up(reader.borrowed()).unwrap_or(false), - None => true, - } - } - /// Lets the execution read `bytes` more of `stream`: its flow-control seam. It reads a /// stream only while the stream has an allowance, and each output event uses up as much of /// it. An allowance that reaches `usize::MAX` never runs out. @@ -1045,13 +1031,6 @@ impl Execution { } } - /// Gives the options' cgroup back to the caller, whose check found it empty after the - /// program ended: the execution kills its processes no more, not even when it drops, so - /// that the caller may reuse the cgroup at once. - pub(crate) fn release_cgroup(&mut self) { - self.cgroup = None; - } - /// Ends the program for `cause`, unless the execution has ended, and returns whether `cause` /// is why it ended the program: the first cause while the program is unreaped. fn end(&mut self, cause: Cause) -> bool { diff --git a/nvx-rs/src/guest/exec/binding.rs b/nvx-rs/src/guest/exec/binding.rs deleted file mode 100644 index 8bd45f4d8..000000000 --- a/nvx-rs/src/guest/exec/binding.rs +++ /dev/null @@ -1,602 +0,0 @@ -//! The runner of the [execution engine](crate::exec), bound to an [`Execution`] of the guest's -//! driver: the glue that every guest adapter would otherwise write. -//! -//! A [`Binding`] applies the [`Runner`]'s requests to the execution: its read allowances become -//! [grants](Execution::grant), its discards [discards](Execution::discard), the input that it -//! accepted [writes](Execution::write_input) and, once the peer ended it, -//! [a close](Execution::close_input), its terminations a [cancellation](Execution::cancel), a -//! [disconnection](Execution::disconnect), or a [termination](Execution::terminate), and its -//! settlement a kill and an occupancy check of the execution's [cgroup](super::Options::cgroup). It -//! feeds the execution's events back to the runner: the launch report, the output and its ends, -//! the bytes that the execution discarded, the program's end, once the execution collected its -//! status or the caller's own collector handed it over with [`observe`](Binding::observe), as -//! [`Options::caller_reaps`](super::Options::caller_reaps) has it, and, after the program's end, -//! the streams that [hung up](crate::exec::Runner::output_hung_up), which no process holds any -//! more. -//! -//! A [signal](Binding::signal) that the adapter sends the program itself, such as the first step -//! of a graceful stop, passes through to the execution, which it does not end: the runner sees -//! nothing of it until the program exits. -//! -//! A launch that failed reaches the runner as a [`LaunchFailure`] whose -//! [stage](LaunchFailure::stage) is the [number](crate::guest::spawn::LaunchStage::code) of the -//! [`LaunchStage`](crate::guest::spawn::LaunchStage) that failed, which -//! [`LaunchStage::from_code`](crate::guest::spawn::LaunchStage::from_code) turns back into the -//! stage, or 0, which numbers no stage, if the binding could not read the launch report. The -//! numbers are part of the API: a stage keeps its number, and a new stage takes a number of its -//! own. -//! -//! The adapter hands the binding the peer's messages, frames the runner's -//! [messages](crate::exec::runner::Message) that [`next`](Binding::next) returns, and drives the -//! binding without blocking, as it would the execution alone: it calls `next` until that returns -//! `None`, then waits until one of the binding's [descriptors](Binding::descriptors) polls -//! readable, its [input writer](Binding::input_writer) polls writable, its -//! [deadline](Binding::deadline) passes, or a message arrives. A stalled peer, sink, or transport -//! only withholds credit, so the binding stops reading the stream that waits on it, and nothing -//! else. Each call takes a bounded number of steps, its writes of the input among them, so that -//! no execution holds the adapter's loop: a call that stops with steps left sets the deadline to -//! its own instant, which the adapter's next wait finds passed already. Once the binding returned -//! a message, it steps the execution, if it can, before it returns another, so that input that -//! the adapter hands it between calls never keeps it from reading the program's output. -//! -//! Settlement goes through the cgroup that the execution's options name, if any, in the scope -//! that they give: once the program's status is known, the binding kills the cgroup's processes, -//! and reports the execution settled once the cgroup is empty, which it checks at most every 10 -//! ms. From then on, the cgroup is the caller's again: neither the execution nor its drop kills -//! the cgroup's processes, so the caller may reuse the cgroup once the outcome is reusable, -//! whether or not it dropped the binding. A zombie still occupies the cgroup until its parent -//! collects it, so a caller whose executions' orphans it adopts, such as a PID 1, collects their -//! statuses between its calls. -//! Without a cgroup, nothing proves that the program's descendants are gone, so the execution -//! never settles, and is fenced: the runner then gives up on the output streams that a process -//! may still hold the settlement time and the grace after the program's end, while it waits as -//! long as the peer takes for the rest of those that hung up. The runner's timers time the -//! execution: an execution with a [timeout](super::Options::timeout) of its own ends its program -//! without telling the runner why. - -use std::collections::VecDeque; -use std::io; -use std::os::fd::BorrowedFd; -use std::os::unix::process::ExitStatusExt; -use std::process::ExitStatus; -use std::time::{Duration, Instant}; - -use super::{Event as DriverEvent, Execution, KillTarget, Source, Stream as DriverStream}; -use crate::exec::{ - Cause, Key, LaunchFailure, Pool, Receipt, Runner, Settlement, Status, Stream, Violation, runner, -}; - -/// The interval between two checks of a settling cgroup. -const SETTLE_INTERVAL: Duration = Duration::from_millis(10); - -/// The most steps that one call of [`Binding::next`] takes: steps of the execution, each of which -/// reads at most the caller's buffer, and writes of the input, each of at most a chunk. So neither -/// output that the execution discards as fast as a process writes it, nor input that a process -/// reads as fast as the binding writes it, holds the caller's loop. -const STEPS: usize = 16; - -/// What one step of a [`Binding`] reports. -#[derive(Debug)] -#[non_exhaustive] -pub enum Event { - /// A message for the peer, which the adapter frames and sends in order. An output message's - /// bytes are at the start of the buffer that the adapter lent [`Binding::next`]. - Message(runner::Message), - /// A kill that ending the program needed failed, as the execution reports it. The binding - /// goes on: the runner's deadline bounds a program that the kill did not end. - KillFailed(KillTarget, io::Error), - /// Killing or checking the execution's cgroup failed, which fenced the execution. - SettlementFailed(io::Error), - /// A write of the program's input failed, other than because the program closed its end. The - /// binding closed the input, so that the program reads its end, and the runner tells the peer - /// that the input closed. - InputFailed(io::Error), -} - -/// An [`Execution`] that the execution engine's [`Runner`] drives: see the [module](self) -/// documentation. -/// -/// Dropping a binding drops its execution, which ends a program that has not ended, and the -/// processes of its cgroup unless the execution settled. -#[derive(Debug)] -pub struct Binding { - /// The execution. - execution: Execution, - /// The runner. - runner: Runner, - /// How many bytes of each stream the binding has granted the execution, and the execution - /// has not read. - granted: [usize; 2], - /// Whether the binding fed the execution's launch report to the runner. - launched: bool, - /// Whether the binding reported the program's end to the runner. - exited: bool, - /// Whether the binding told the runner that each stream hung up. - hung_up: [bool; 2], - /// Whether the execution ended: the driver has no step left. - ended: bool, - /// When the binding checks the settling cgroup next, while it settles it. - settling: Option, - /// The instant of the last call of [`next`](Self::next) if it took [`STEPS`] steps without an - /// event to return, and may have more to take: the binding's deadline, which has passed - /// already. - paused: Option, - /// Whether the execution's turn is due: once the binding returned a message, it takes a step - /// of the execution, if it has one, before it returns another, so that input credit, which - /// the writes of every call may free, never keeps it from reading the program's output. - execution_turn: bool, - /// The events that wait to be returned: failures, which the binding reports before messages. - events: VecDeque, -} - -impl Binding { - /// Binds `runner` to `execution`, whose program its adapter spawned, or adopted, for the - /// runner's execution. An execution without an input pipe gives its program no input that the - /// peer could send, so the runner tells the peer that the input closed. - pub fn new(execution: Execution, runner: Runner) -> Self { - let mut binding = Self { - execution, - runner, - granted: [0, 0], - launched: false, - exited: false, - hung_up: [false, false], - ended: false, - settling: None, - paused: None, - execution_turn: true, - events: VecDeque::new(), - }; - if binding.execution.input().is_none() { - binding.runner.input_closed(); - } - binding - } - - /// Returns the runner, whose state the adapter may inspect. - pub fn runner(&self) -> &Runner { - &self.runner - } - - /// Returns the execution, whose state the adapter may inspect. - pub fn execution(&self) -> &Execution { - &self.execution - } - - /// Returns the program's process ID. - pub fn id(&self) -> u32 { - self.execution.id() - } - - /// Hands the runner the peer's credit: see [`Runner::credit`]. - pub fn credit(&mut self, key: Key, pool: Pool, units: u32) -> Result { - self.runner.credit(key, pool, units) - } - - /// Hands the runner the peer's input: see [`Runner::input`]. - pub fn input(&mut self, key: Key, bytes: &[u8]) -> Result { - self.runner.input(key, bytes) - } - - /// Hands the runner the end of the peer's input: see [`Runner::end_input`]. - pub fn end_input(&mut self, key: Key) -> Result { - self.runner.end_input(key) - } - - /// Hands the runner the peer's cancellation: see [`Runner::cancel`]. - pub fn cancel(&mut self, now: Instant, key: Key, cause: Cause) -> Result { - self.runner.cancel(now, key, cause) - } - - /// Ends the execution because the adapter abandons it: see [`Runner::dropped`]. - pub fn dropped(&mut self, now: Instant) { - self.runner.dropped(now); - } - - /// Ends the execution because the session was reset: see [`Runner::reset`]. - pub fn reset(&mut self, now: Instant) { - self.runner.reset(now); - } - - /// Ends the execution because the transport was lost: see [`Runner::disconnect`]. - pub fn disconnect(&mut self, now: Instant) { - self.runner.disconnect(now); - } - - /// Sends the program the signal numbered `signal`, without ending the execution, and returns - /// whether the program was unreaped, as [`Execution::signal`] does. The runner learns nothing - /// of the signal: it sees the program's end, with the program's own status, once the program - /// exits. - pub fn signal(&mut self, signal: i32) -> io::Result { - self.execution.signal(signal) - } - - /// Hands the execution the status that the caller's own collector collected for the process - /// `id`, and returns whether it was the program's, as [`Execution::observe`] does. The next - /// call of [`next`](Self::next) reports the program's end to the runner before anything else. - pub fn observe(&mut self, id: u32, status: ExitStatus) -> bool { - self.execution.observe(id, status) - } - - /// Takes the next step without blocking, at `now`, and returns what it reports, or `None` if - /// nothing can happen until a descriptor polls ready, the deadline passes, or a message - /// arrives. Each read of an output stream goes into `buffer`, which bounds it, and each - /// output message's bytes are at the start of `buffer`. - /// - /// Each call first collects the program's status without blocking, unless the caller's own - /// collector takes it, and takes the program's launch report once it has arrived. Within a - /// step, the binding then reports the program's end to the runner once the execution has its - /// status, whether or not the launch report has arrived, which a process that another thread - /// forked meanwhile may hold back, tells the runner which streams hung up once a - /// timer is due after the program's end, fires the runner's timers, checks a settling - /// cgroup if a check is due, writes the input that the program's pipe takes, at most a chunk - /// at a time, applies the runner's requests, those that the last write raised included, such - /// as closing the input after its last byte, grants the reads that the runner allows, and - /// returns the runner's next message; failing that, it takes the execution's next step, feeds - /// its event to the runner, and starts over. So the program's end reaches the runner before - /// its timers fire or a queued message returns, even in a first call that comes after the - /// runner's timeout. Once the binding returned a message, the execution's turn comes before - /// the next: it takes the execution's next step, if it has one, and starts over, so that - /// input that the caller hands it between calls, which frees credit for each of them, never - /// keeps it from reading the program's output. The binding also starts over while anything - /// else changed, so that every request that a step raised is applied before it returns - /// `None`, and a due check runs however much output flows. It bounds the work of one call: - /// the steps of the execution and the writes of the input count against one bound, of which - /// the writes leave a step for the execution's turn while it is due, and a call that reaches - /// it without an event to return sets the [deadline](Self::deadline) to `now`, so that the - /// caller calls it again at once, after its other work. After the terminal outcome, it - /// returns `None`. - pub fn next(&mut self, now: Instant, buffer: &mut [u8]) -> Option { - self.paused = None; - self.collect_status(now); - self.take_launch_report(); - let mut steps = 0; - loop { - let mut progressed = self.report_status(now); - self.report_hang_ups(now); - self.runner.tick(now); - progressed |= self.settle(now); - progressed |= self.write_input(&mut steps); - progressed |= self.apply_requests(now); - self.grant(); - if let Some(event) = self.events.pop_front() { - return Some(event); - } - if std::mem::take(&mut self.execution_turn) && self.step_execution(now, buffer) { - steps += 1; - continue; - } - if let Some(message) = self.runner.next_message(buffer) { - self.execution_turn = true; - return Some(Event::Message(message)); - } - if self.runner.is_finished() { - return None; - } - if steps >= STEPS { - self.paused = Some(now); - return None; - } - if self.step_execution(now, buffer) { - steps += 1; - continue; - } - if !progressed { - return None; - } - } - } - - /// Returns the descriptors that move the execution on, each with what it waits on, which - /// the caller polls for reading, as [`Execution::descriptors`] describes. An output stream is - /// among them only while the runner lets the binding read it. - pub fn descriptors(&self) -> impl Iterator)> { - let active = !self.runner.is_finished(); - self.execution.descriptors().filter(move |_| active) - } - - /// Lends the writer of the program's input while the runner holds input for it, which the - /// caller polls for writing. - pub fn input_writer(&self) -> Option> { - if self.runner.pending_input().is_empty() { - return None; - } - self.execution.input() - } - - /// Returns when the binding must take a step even if no descriptor polls ready: the earliest - /// of the runner's deadline, the execution's, the next check of a settling cgroup, and the - /// instant of the last call of [`next`](Self::next) if it stopped with steps left. - pub fn deadline(&self) -> Option { - if self.runner.is_finished() { - return None; - } - [ - self.runner.deadline(), - self.execution.deadline(), - self.settling.filter(|_| self.runner.awaits_settlement()), - self.paused, - ] - .into_iter() - .flatten() - .min() - } - - /// Returns whether the runner returned its terminal outcome. - pub fn is_finished(&self) -> bool { - self.runner.is_finished() - } - - /// Applies the runner's requests to the execution, and returns whether there were any. - fn apply_requests(&mut self, now: Instant) -> bool { - let mut applied = false; - while let Some(request) = self.runner.next_request() { - applied = true; - match request { - runner::Request::Terminate(Cause::Cancelled) => { - self.execution.cancel(); - } - runner::Request::Terminate(Cause::Disconnected) => self.execution.disconnect(), - runner::Request::Terminate(_) => { - self.execution.terminate(); - } - runner::Request::Discard(stream) => self.execution.discard(driver_stream(stream)), - runner::Request::CloseInput => self.execution.close_input(), - runner::Request::Settle => self.begin_settlement(now), - } - } - applied - } - - /// Writes the input that the runner holds while the program's pipe takes it, at most a chunk - /// in each write, each of which counts as one of the call's `steps`, until they reach - /// [`STEPS`], or one less while the execution's turn is due, so that a step is left for it; - /// and tells the runner once the input closed: because the program closed its end, because - /// the execution closed it as the program ended, or because a write failed, after which the - /// binding closes it too, so that the program reads its end, and reports the failure. Returns - /// whether it wrote any byte. - fn write_input(&mut self, steps: &mut usize) -> bool { - let chunk = self.runner.limits().chunk(); - let limit = STEPS - usize::from(self.execution_turn); - let mut wrote = false; - while *steps < limit { - let pending = self.runner.pending_input(); - if pending.is_empty() { - break; - } - let pending = &pending[..pending.len().min(chunk)]; - match self.execution.write_input(pending) { - Ok(0) => break, - Ok(length) => { - wrote = true; - *steps += 1; - self.runner.input_written(length); - } - Err(error) if error.kind() == io::ErrorKind::WouldBlock => break, - Err(error) => { - // The execution closes the input itself only once the program closed its - // end, which is no failure: the binding closes it after any other failure, - // and reports that. - self.execution.close_input(); - self.runner.input_closed(); - if error.kind() != io::ErrorKind::BrokenPipe { - self.events.push_back(Event::InputFailed(error)); - } - break; - } - } - } - if self.execution.input().is_none() { - self.runner.input_closed(); - } - wrote - } - - /// Grants the execution the reads that the runner allows beyond what it granted already. - fn grant(&mut self) { - for stream in Stream::ALL { - let allowance = self.runner.read_allowance(stream); - let granted = &mut self.granted[index(stream)]; - if allowance > *granted { - self.execution - .grant(driver_stream(stream), allowance - *granted); - *granted = allowance; - } - } - } - - /// Takes the execution's next step at `now`, if it has one, feeds its event to the runner, - /// and reports the program's end once its status is known. Returns whether it took a step or - /// reported the end. - fn step_execution(&mut self, now: Instant, buffer: &mut [u8]) -> bool { - if self.ended { - return false; - } - let stepped = match self.execution.next_event(buffer) { - Ok(Some(event)) => { - self.feed(event); - true - } - Ok(None) => false, - // Another collector took the program's status, or collecting it failed: the - // program has ended, and its status is lost. - Err(_) => self.report_exit(now, None), - }; - let reported = self.report_status(now); - stepped || reported - } - - /// Feeds one event of the execution to the runner. - fn feed(&mut self, event: DriverEvent<'_>) { - match event { - DriverEvent::Launch(report) => { - self.launched = true; - self.runner.launched(match report { - Ok(None) => Ok(()), - Ok(Some(failure)) => Err(LaunchFailure::new( - failure.stage().code(), - failure.raw_os_error(), - )), - Err(error) => Err(LaunchFailure::new(0, error.raw_os_error().unwrap_or(0))), - }); - } - DriverEvent::Output(stream, bytes) => { - let stream = engine_stream(stream); - let granted = &mut self.granted[index(stream)]; - *granted = granted.saturating_sub(bytes.len()); - self.runner.output(stream, bytes); - } - DriverEvent::Discarded(stream, length) => { - self.runner.discarded(engine_stream(stream), length); - } - DriverEvent::Closed(stream) => self.runner.output_closed(engine_stream(stream)), - DriverEvent::ReadFailed(stream, _) => self.runner.output_failed(engine_stream(stream)), - DriverEvent::KillFailed(target, error) => { - self.events.push_back(Event::KillFailed(target, error)); - } - DriverEvent::Ended(_) => self.ended = true, - } - } - - /// Collects the program's status at `now` without blocking, if the execution collects it - /// itself. A failed collection reports the program's end with its status lost, as a failed - /// step does. - fn collect_status(&mut self, now: Instant) { - if self.execution.collect().is_err() { - self.report_exit(now, None); - } - } - - /// Feeds the program's launch report to the runner once it has arrived, unless the binding - /// did already. - fn take_launch_report(&mut self) { - if !self.launched - && let Some(report) = self.execution.launch() - { - self.feed(DriverEvent::Launch(report)); - } - } - - /// Reports the program's end at `now` to the runner once the execution has its status, unless - /// the binding did already, and returns whether it reported the end now. The end does not - /// wait for the launch report: a process that another thread forked while the program was - /// spawning holds the report's writer until it replaces itself with its program or ends, and - /// the runner's timeout would fire meanwhile. The runner holds the start, the output, and the - /// terminal outcome until the report arrives, or until the settlement time and the grace have - /// passed since the program's end, when it gives up on the report and fences the execution. - fn report_status(&mut self, now: Instant) -> bool { - match self.execution.status() { - Some(status) => self.report_exit(now, Some(status)), - None => false, - } - } - - /// Reports the program's end at `now` to the runner, with its status if known, unless it did - /// already, and returns whether it reported it now. - fn report_exit(&mut self, now: Instant, status: Option) -> bool { - if self.exited { - return false; - } - self.exited = true; - let status = status.and_then(|status| { - status - .code() - .map(Status::Exited) - .or_else(|| status.signal().map(Status::Signaled)) - }); - self.runner.exited(now, status); - true - } - - /// Tells the runner which output streams hung up, once it knows of the program's end and one - /// of its timers is due at `now`, before the timer fires: no process holds such a stream any - /// more, so the runner never gives up on it, even if the execution does not settle. - fn report_hang_ups(&mut self, now: Instant) { - if !self.exited || self.runner.deadline().is_none_or(|at| at > now) { - return; - } - for stream in Stream::ALL { - let hung_up = &mut self.hung_up[index(stream)]; - if !*hung_up && self.execution.output_hung_up(driver_stream(stream)) { - *hung_up = true; - self.runner.output_hung_up(stream); - } - } - } - - /// Begins to settle the execution at `now`: kills the processes of its cgroup, and checks - /// the cgroup from then on. Without a cgroup, or if the kill fails, the settlement fails. - fn begin_settlement(&mut self, now: Instant) { - let Some((cgroup, scope)) = &self.execution.cgroup else { - self.runner.settled(Settlement::Failed); - return; - }; - match cgroup.kill(*scope) { - Ok(()) => self.settling = Some(now), - Err(error) => { - self.runner.settled(Settlement::Failed); - self.events.push_back(Event::SettlementFailed(error)); - } - } - } - - /// Checks the settling cgroup at `now`, if a check is due: the execution settled once no - /// process occupies the cgroup, and its settlement failed if the check does. Returns whether - /// the settlement ended. - fn settle(&mut self, now: Instant) -> bool { - if !self.runner.awaits_settlement() { - self.settling = None; - } - if self.settling.is_none_or(|due| now < due) { - return false; - } - let Some((cgroup, scope)) = &self.execution.cgroup else { - return false; - }; - match cgroup.is_occupied(*scope) { - Ok(false) => { - self.settling = None; - self.execution.release_cgroup(); - self.runner.settled(Settlement::Settled); - true - } - Ok(true) => { - self.settling = now.checked_add(SETTLE_INTERVAL); - false - } - Err(error) => { - self.settling = None; - self.runner.settled(Settlement::Failed); - self.events.push_back(Event::SettlementFailed(error)); - true - } - } - } -} - -/// Returns `stream`'s position in [`Stream::ALL`]. -fn index(stream: Stream) -> usize { - match stream { - Stream::Stdout => 0, - Stream::Stderr => 1, - } -} - -/// Returns the driver's stream of the engine's `stream`. -fn driver_stream(stream: Stream) -> DriverStream { - match stream { - Stream::Stdout => DriverStream::Stdout, - Stream::Stderr => DriverStream::Stderr, - } -} - -/// Returns the engine's stream of the driver's `stream`. -fn engine_stream(stream: DriverStream) -> Stream { - match stream { - DriverStream::Stdout => Stream::Stdout, - DriverStream::Stderr => Stream::Stderr, - } -} - -#[cfg(test)] -mod tests; diff --git a/nvx-rs/src/guest/exec/binding/tests.rs b/nvx-rs/src/guest/exec/binding/tests.rs deleted file mode 100644 index c375e6ac8..000000000 --- a/nvx-rs/src/guest/exec/binding/tests.rs +++ /dev/null @@ -1,1828 +0,0 @@ -//! Checks of bound executions of real programs, each with a peer in the same process: streams -//! beyond their windows, input beyond its window, an input that the program closes early, one -//! that closes with its last byte while output waits, and one whose write fails, a program that -//! the caller started itself, a launch stage that fails, which the peer learns by its number, -//! settlement before reuse and the fences of failed cleanups, the bounded wait for output that a -//! descendant holds once an execution does not settle, due settlement checks while output drains -//! slowly, statuses that the execution or the caller collects, which reach the runner ahead of -//! queued output and of its timeout, even in a first call after it, and while the launch report -//! is late, a signal that the program handles, which reaches the runner only as the program's -//! end, the bounded wait for a launch report that a process holds for good, the bounds on the -//! steps and on the writes of one call, output that flows while the caller hands over input -//! before every call, and two executions in one loop, one of them with a stalled sink or a pipe -//! writer that escaped its kill. The loop waits only for what the bindings and peers say that -//! they wait on, so a step that nothing leads to stalls a check rather than passing it. - -use std::ffi::CString; -use std::os::fd::OwnedFd; -use std::os::unix::process::CommandExt; -use std::path::Path; -use std::time::{Duration, Instant}; - -use nix::poll::{PollFd, PollFlags, PollTimeout}; -use nix::sys::socket::{self, AddressFamily, SockFlag, SockType, sockopt}; - -use super::*; -use crate::exec::{Limits, Outcome, Peer, Timers, Window, peer}; -use crate::guest::cgroup::{ExecutionCgroup, Layout, Scope}; -use crate::guest::exec::{Options, Pipes}; -use crate::guest::process::Child; -use crate::guest::spawn::{Command, LaunchStage}; -use crate::guest::sys::{self, ProcessId}; - -/// How long a test waits for its executions before it fails. -const PATIENCE: Duration = Duration::from_secs(30); -/// The grace of pending output in these tests. -const GRACE: Duration = Duration::from_millis(300); -/// The settlement time in these tests. -const SETTLEMENT: Duration = Duration::from_secs(10); -/// The key of each execution in these tests. -const KEY: Key = Key::new(1, 1); -/// The longest that a deadline may be overrun while the test's loop serves other executions. -const SLACK: Duration = Duration::from_secs(2); - -/// Returns limits with chunks of 16 KiB and byte windows of `window` bytes. -fn limits(window: u32) -> Limits { - Limits::new(16 * 1024, Window::bytes(window), Window::bytes(window)).unwrap() -} - -/// Returns the timers of these tests. -fn timers() -> Timers { - Timers::new(GRACE, SETTLEMENT) -} - -/// Returns a command that runs `/bin/sh -c script`, with `$0` set to `sh`. -fn shell(script: &str) -> Command { - Command::new( - c"/bin/sh".to_owned(), - vec![ - c"sh".to_owned(), - c"-c".to_owned(), - CString::new(script).unwrap(), - ], - ) -} - -/// Returns `seq 1 count`'s output. -fn sequence(count: usize) -> Vec { - (1..=count) - .map(|number| format!("{number}\n")) - .collect::() - .into_bytes() -} - -/// Returns a fake cgroup in `root`: a workload cgroup whose `cgroup.kill` records the kills, -/// and whose `pids.current` says `pids`. A test plays the kernel's part. -fn fake_cgroup(root: &Path, pids: &str) -> ExecutionCgroup { - let layout = Layout::new(root.join("agent"), root.join("workload")).unwrap(); - std::fs::create_dir_all(layout.workload()).unwrap(); - std::fs::write(layout.workload().join("cgroup.kill"), b"").unwrap(); - std::fs::write(layout.workload().join("pids.current"), pids).unwrap(); - ExecutionCgroup::open(&layout, None).unwrap() -} - -/// One bound execution and its peer, connected directly, with the peer's sink and source. -struct Side { - /// The runner's side: the binding of the execution. - binding: Binding, - /// The peer. - peer: Peer, - /// The bytes of each stream that the sink took. - output: [Vec; 2], - /// Whether the sink takes nothing. - stalled: bool, - /// The input that the source hands over. - input: Vec, - /// The bytes of input that the source handed over. - sent: usize, - /// The peer's events. - events: Vec, - /// The runner's terminal outcome, once it returned it. - terminal: Option, - /// The failures that the binding reported. - failures: Vec, -} - -impl Side { - /// Spawns `command` with pipes, its standard input piped if `input` and the null device - /// otherwise, in a process group of its own, and binds a runner with `limits` and `timers` - /// to its execution with `options`, with a peer of its own. - fn start( - command: Command, - input: bool, - options: Options, - limits: Limits, - timers: Timers, - ) -> Self { - let (pipes, ends) = Pipes::new(input).unwrap(); - let stdin = match ends.stdin { - Some(stdin) => stdin, - None => OwnedFd::from(std::fs::File::open("/dev/null").unwrap()), - }; - let spawned = command - .stdin(stdin) - .stdout(ends.stdout) - .stderr(ends.stderr) - .new_process_group() - .spawn() - .unwrap(); - let execution = Execution::new(spawned, pipes, options.process_group()).unwrap(); - Self::bound(execution, limits, timers) - } - - /// Binds a runner with `limits` and `timers` to `execution`, with a peer of its own. - fn bound(execution: Execution, limits: Limits, timers: Timers) -> Self { - let now = Instant::now(); - Self { - binding: Binding::new(execution, Runner::new(KEY, limits, timers, now)), - peer: Peer::new(KEY, limits, timers, now), - output: [Vec::new(), Vec::new()], - stalled: false, - input: Vec::new(), - sent: 0, - events: Vec::new(), - terminal: None, - failures: Vec::new(), - } - } - - /// Takes every step that the binding and the peer can take at `now` without waiting, and - /// returns whether anything happened. - fn pump(&mut self, now: Instant, buffer: &mut [u8]) -> bool { - let mut progressed = false; - while let Some(event) = self.binding.next(now, buffer) { - progressed = true; - self.handle(now, event, buffer); - } - self.serve(now, buffer) || progressed - } - - /// Takes what one step of the binding at `now` reported: hands the peer a message, whose - /// output is in `buffer`, and records a failure. - fn handle(&mut self, now: Instant, event: Event, buffer: &[u8]) { - match event { - Event::Message(message) => self.deliver(now, message, buffer), - Event::KillFailed(target, error) => { - self.failures.push(format!("kill {target:?}: {error}")); - } - Event::SettlementFailed(error) => { - self.failures.push(format!("settlement: {error}")); - } - Event::InputFailed(error) => { - self.failures.push(format!("input: {error}")); - } - } - } - - /// Takes every step that the peer, its source, and its sink can take at `now` without - /// waiting, hands the peer's messages to the binding at once, and returns whether anything - /// happened. - fn serve(&mut self, now: Instant, buffer: &mut [u8]) -> bool { - let mut progressed = false; - if self.sent < self.input.len() { - let length = self.peer.input(&self.input[self.sent..]); - self.sent += length; - progressed |= length > 0; - } - if self.sent == self.input.len() { - self.peer.end_input(); - } - if !self.stalled { - for stream in Stream::ALL { - loop { - let pending = self.peer.pending_output(stream); - if pending.is_empty() { - break; - } - let length = pending.len(); - self.output[index(stream)].extend_from_slice(pending); - self.peer.consumed(stream, length); - progressed = true; - } - } - } - self.peer.tick(now); - while let Some(event) = self.peer.next_event() { - progressed = true; - self.events.push(event); - } - while let Some(message) = self.peer.next_message(buffer) { - progressed = true; - match message { - peer::Message::Cancel(cause) => self.binding.cancel(now, KEY, cause), - peer::Message::Credit { pool, units } => self.binding.credit(KEY, pool, units), - peer::Message::Input { length } => self.binding.input(KEY, &buffer[..length]), - peer::Message::InputEnd => self.binding.end_input(KEY), - } - .expect("the runner accepts the peer's message"); - } - progressed - } - - /// Hands the peer the runner's `message` at `now`, whose output is in `buffer`. - fn deliver(&mut self, now: Instant, message: runner::Message, buffer: &[u8]) { - let receipt = match message { - runner::Message::Credit { units } => self.peer.credit(KEY, units), - runner::Message::Started => self.peer.started(KEY), - runner::Message::Output { stream, length } => { - self.peer.output(KEY, stream, &buffer[..length]) - } - runner::Message::End(stream) => self.peer.end_output(KEY, stream), - runner::Message::InputClosed => self.peer.input_closed(KEY), - runner::Message::Terminal(outcome) => { - assert!(self.terminal.is_none(), "one terminal outcome"); - self.terminal = Some(outcome); - if self.binding.runner().is_detached() { - return; - } - self.peer.terminal(now, KEY, outcome) - } - }; - receipt.expect("the peer accepts the runner's message"); - } - - /// Returns whether both sides finished. - fn is_finished(&self) -> bool { - self.binding.is_finished() && self.peer.is_finished() - } - - /// Returns the peer's outcome, once it finished. - fn outcome(&self) -> Option { - self.events.iter().find_map(|event| match event { - peer::Event::Finished(outcome) => Some(*outcome), - _ => None, - }) - } -} - -/// Waits until a descriptor of `sides` polls ready, or the earliest of their deadlines and -/// `until` passes, as an adapter's loop does. -fn wait(sides: &[&mut Side], until: Instant) { - let mut descriptors = Vec::new(); - for side in sides { - for (_, descriptor) in side.binding.descriptors() { - descriptors.push(PollFd::new(descriptor, PollFlags::POLLIN)); - } - if let Some(descriptor) = side.binding.input_writer() { - descriptors.push(PollFd::new(descriptor, PollFlags::POLLOUT)); - } - } - let deadline = sides - .iter() - .flat_map(|side| [side.binding.deadline(), side.peer.deadline()]) - .flatten() - .fold(until, Instant::min); - let timeout = deadline.saturating_duration_since(Instant::now()); - // Poll counts whole milliseconds, so a shorter wait would return before its deadline. - let timeout = if timeout.is_zero() { - timeout - } else { - timeout.max(Duration::from_millis(1)) - }; - match nix::poll::poll(&mut descriptors, PollTimeout::try_from(timeout).unwrap()) { - Ok(_) | Err(nix::errno::Errno::EINTR) => {} - Err(error) => panic!("poll failed: {error}"), - } -} - -/// Drives `sides` in one loop, as an adapter does, until `done` holds: it takes every step that -/// they can take, then waits for what their bindings and peers wait on, and for `wake` if any. -/// It fails unless `done` holds within the test's patience, and fails as soon as a wait runs -/// into it, so a step that nothing they wait on leads to stalls the test rather than passing it. -fn run(sides: &mut [&mut Side], wake: Option, mut done: impl FnMut(&[&mut Side]) -> bool) { - let patience = Instant::now() + PATIENCE; - let until = wake.map_or(patience, |wake| wake.min(patience)); - let mut buffer = vec![0u8; 64 * 1024]; - loop { - let mut progressed = false; - for side in sides.iter_mut() { - progressed |= side.pump(Instant::now(), &mut buffer); - } - if done(sides) { - return; - } - if !progressed { - wait(sides, until); - } - assert!(Instant::now() < patience, "the executions moved on in time"); - } -} - -/// Drives `sides` in one loop until `done` holds: see [`run`]. -fn drive(sides: &mut [&mut Side], done: impl FnMut(&[&mut Side]) -> bool) { - run(sides, None, done); -} - -/// Drives `sides` in one loop for `duration`: see [`run`]. -fn drive_for(sides: &mut [&mut Side], duration: Duration) { - let until = Instant::now() + duration; - run(sides, Some(until), |_| Instant::now() >= until); -} - -/// Drives `side` alone until both of its sides finished, and returns the peer's outcome. -fn finish(side: &mut Side) -> Outcome { - drive(&mut [side], |sides| sides[0].is_finished()); - side.outcome().unwrap() -} - -#[test] -fn output_beyond_its_windows_arrives_whole_and_the_outcome_is_lossless() { - let temporary = tempfile::tempdir().unwrap(); - let cgroup = fake_cgroup(temporary.path(), "0\n"); - let mut side = Side::start( - shell("seq 1 400000; seq 1 20000 >&2"), - false, - Options::new().cgroup(&cgroup, Scope::Execution), - limits(64 * 1024), - timers(), - ); - let outcome = finish(&mut side); - assert_eq!(side.output[0], sequence(400_000)); - assert_eq!(side.output[1], sequence(20_000)); - assert_eq!(Some(outcome), side.terminal); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert_eq!(outcome.cause(), None); - assert!(!outcome.is_truncated(Stream::Stdout) && !outcome.is_truncated(Stream::Stderr)); - assert!(outcome.is_reusable()); - assert!(side.failures.is_empty(), "{:?}", side.failures); - let runner = side.binding.runner(); - assert!(runner.high_water() <= runner.memory_bound()); - assert!(side.peer.high_water() <= side.peer.memory_bound()); - // Settlement killed the execution's cgroup, whose emptiness the binding checked. - assert_eq!( - std::fs::read(cgroup.path().join("cgroup.kill")).unwrap(), - b"1" - ); - assert_eq!( - side.binding.next(Instant::now(), &mut [0; 16]).map(drop), - None - ); - assert!(side.binding.descriptors().next().is_none()); - assert_eq!(side.binding.deadline(), None); -} - -#[test] -fn input_longer_than_its_window_reaches_the_program_intact() { - // With byte windows, and with frames of up to 64 KiB less 256 bytes in a shared pool. - let frames = Limits::new(64 * 1024 - 256, Window::frames(4), Window::frames(64)) - .unwrap() - .shared_output(); - for limits in [limits(16 * 1024), frames] { - let temporary = tempfile::tempdir().unwrap(); - let cgroup = fake_cgroup(temporary.path(), "0\n"); - let mut side = Side::start( - shell("exec cat"), - true, - Options::new().cgroup(&cgroup, Scope::Execution), - limits, - timers(), - ); - side.input = (0..3usize << 20) - .map(|index| (index * 7 + index / 251) as u8) - .collect(); - let outcome = finish(&mut side); - assert_eq!(side.output[0], side.input, "{limits:?}"); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert!(!outcome.is_truncated(Stream::Stdout)); - assert!(outcome.is_reusable()); - let runner = side.binding.runner(); - assert!(runner.high_water() <= runner.memory_bound()); - assert!(side.peer.high_water() <= side.peer.memory_bound()); - } -} - -#[test] -fn a_program_that_closes_its_input_early_stops_the_peer_s_source() { - let temporary = tempfile::tempdir().unwrap(); - let cgroup = fake_cgroup(temporary.path(), "0\n"); - let mut side = Side::start( - shell("head -c 5; printf done >&2"), - true, - Options::new().cgroup(&cgroup, Scope::Execution), - limits(16 * 1024), - timers(), - ); - side.input = vec![b'x'; 16 << 20]; - let outcome = finish(&mut side); - assert_eq!(side.output, [b"xxxxx".to_vec(), b"done".to_vec()]); - assert!(side.events.contains(&peer::Event::InputClosed)); - // The source stopped long before its 16 MiB. - assert!(side.sent < 1 << 20, "{}", side.sent); - // A program that closes its input is no failure. - assert!(side.failures.is_empty(), "{:?}", side.failures); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert!(!outcome.is_truncated(Stream::Stdout)); - assert!(outcome.is_reusable()); -} - -#[test] -fn an_adopted_program_runs_bound_as_a_spawned_one_does() { - let temporary = tempfile::tempdir().unwrap(); - let cgroup = fake_cgroup(temporary.path(), "0\n"); - // The test starts the program by a launcher other than `spawn`, the standard library's. - let (stdin, input) = std::io::pipe().unwrap(); - let (stdout, stdout_writer) = std::io::pipe().unwrap(); - let (stderr, stderr_writer) = std::io::pipe().unwrap(); - let mut command = std::process::Command::new("/bin/sh"); - command - .args(["-c", "cat; printf done >&2; exit 3"]) - .stdin(stdin) - .stdout(stdout_writer) - .stderr(stderr_writer) - .process_group(0); - // The execution collects the program's status, rather than the standard library's handle. - #[allow(clippy::zombie_processes)] - let program = command.spawn().unwrap(); - // The command holds the program's ends until it drops. - drop(command); - let pipes = Pipes::from_ends(Some(input.into()), stdout.into(), stderr.into()).unwrap(); - let execution = Execution::adopt( - Child::with_pidfd(program.id()).unwrap(), - pipes, - Options::new() - .process_group() - .cgroup(&cgroup, Scope::Execution), - ) - .unwrap(); - let mut side = Side::bound(execution, limits(16 * 1024), timers()); - side.input = b"input".to_vec(); - let outcome = finish(&mut side); - assert_eq!(side.events.first(), Some(&peer::Event::Started)); - assert_eq!(side.output, [b"input".to_vec(), b"done".to_vec()]); - assert_eq!(Some(outcome), side.terminal); - assert_eq!(outcome.status(), Some(Status::Exited(3))); - assert_eq!(outcome.cause(), None); - assert!(outcome.is_reusable()); - assert!(side.failures.is_empty(), "{:?}", side.failures); -} - -#[test] -fn a_failed_launch_stage_reaches_the_peer_by_its_number() { - let temporary = tempfile::tempdir().unwrap(); - let cgroup = fake_cgroup(temporary.path(), "0\n"); - let mut side = Side::start( - shell("exit 0").working_directory(c"/nonexistent/directory".to_owned()), - false, - Options::new().cgroup(&cgroup, Scope::Execution), - limits(16 * 1024), - timers(), - ); - let outcome = finish(&mut side); - assert_eq!(Some(outcome), side.terminal); - let failure = outcome.launch_failure().expect("the launch failed"); - // The caller tells the stage through NVX's numbering, rather than a number of its own. - assert_eq!( - LaunchStage::from_code(failure.stage()), - Some(LaunchStage::WorkingDirectory) - ); - assert_eq!(failure.error(), nix::errno::Errno::ENOENT as i32); - assert_eq!(outcome.status(), Some(Status::Exited(125))); - assert!(side.failures.is_empty(), "{:?}", side.failures); -} - -#[test] -fn the_input_closes_with_its_last_byte_while_output_waits_to_be_returned() { - // The program writes a line, then copies its input to its output. The runner holds the line - // until the peer's credit reaches the binding, together with the input and its end: the - // call that writes the input's last byte returns the line, and closes the program's input - // before it does. - let temporary = tempfile::tempdir().unwrap(); - let cgroup = fake_cgroup(temporary.path(), "0\n"); - let mut side = Side::start( - shell("echo ready; exec cat"), - true, - Options::new().cgroup(&cgroup, Scope::Execution), - limits(4096), - timers(), - ); - // The runner's messages reach the peer, and the peer's wait, until the runner holds the - // line, and the peer knows that the program started: a call of the binding may read the line - // after it looked for the launch report. - let mut buffer = vec![0u8; 64 * 1024]; - let patience = Instant::now() + PATIENCE; - loop { - let now = Instant::now(); - let mut progressed = false; - while let Some(event) = side.binding.next(now, &mut buffer) { - progressed = true; - side.handle(now, event, &buffer); - } - if side.binding.runner().buffered() == b"ready\n".len() && side.peer.has_started() { - break; - } - if !progressed { - wait(&[&mut side], patience); - } - assert!(Instant::now() < patience, "the program wrote in time"); - } - side.input = b"data".to_vec(); - let now = Instant::now(); - side.serve(now, &mut buffer); - let event = side.binding.next(now, &mut buffer); - assert!( - matches!( - event, - Some(Event::Message(runner::Message::Output { - stream: Stream::Stdout, - length: 6, - })) - ), - "{event:?}" - ); - assert!(side.binding.execution().input().is_none()); - side.handle(now, event.unwrap(), &buffer); - let outcome = finish(&mut side); - assert_eq!(side.output[0], b"ready\ndata"); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert!(!outcome.is_truncated(Stream::Stdout)); - assert!(outcome.is_reusable()); -} - -#[test] -fn a_write_that_fails_closes_the_program_s_input() { - // No pipe refuses a write but for `EPIPE`, so the program reads a socket of records instead, - // whose writer refuses a record longer than its buffer with `EMSGSIZE`, as it refuses the - // binding's first chunk. The binding closes the writer then, and reports the failure, as the - // runner drops the rest of the input, so the program reads the input's end, and ends on its - // own. - let temporary = tempfile::tempdir().unwrap(); - let cgroup = fake_cgroup(temporary.path(), "0\n"); - let (reader, writer) = socket::socketpair( - AddressFamily::Unix, - SockType::SeqPacket, - None, - SockFlag::SOCK_CLOEXEC, - ) - .unwrap(); - // Linux doubles the size, and refuses records longer than the result, less 32 bytes. - socket::setsockopt(&writer, sockopt::SndBuf, &4096).unwrap(); - let writer = sys::FileDescriptor::from_owned(writer); - writer.set_nonblocking().unwrap(); - let (mut pipes, ends) = Pipes::new(true).unwrap(); - pipes.input = Some(writer); - let spawned = shell("exec cat") - .stdin(reader) - .stdout(ends.stdout) - .stderr(ends.stderr) - .new_process_group() - .spawn() - .unwrap(); - let options = Options::new() - .cgroup(&cgroup, Scope::Execution) - .process_group(); - let execution = Execution::new(spawned, pipes, options).unwrap(); - let mut side = Side::bound(execution, limits(64 * 1024), timers()); - side.input = vec![b'i'; 16 * 1024]; - let outcome = finish(&mut side); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert_eq!(outcome.cause(), None); - assert!(side.output.iter().all(Vec::is_empty)); - assert!(outcome.is_reusable()); - let refused = io::Error::from(nix::errno::Errno::EMSGSIZE); - assert_eq!(side.failures, [format!("input: {refused}")]); -} - -/// Returns the process that `output` names by its ID on one line, opened by a pidfd that polls -/// readable once it ends. -fn descendant(output: &[u8]) -> (ProcessId, crate::guest::sys::FileDescriptor) { - let id = std::str::from_utf8(output).unwrap().trim().parse().unwrap(); - let process = ProcessId::from_positive(id).unwrap(); - (process, sys::open_pidfd(process).unwrap()) -} - -#[test] -fn descendants_and_pipe_writers_settle_before_reuse_in_a_cgroup_that_the_test_plays() { - // The fake cgroup counts the program and two descendants that outlive it: one in a session - // of its own that holds the program's standard output, and one that holds nothing. The test - // plays the kernel: it ends the descendants once the cgroup is killed, and only then counts - // no process. - let temporary = tempfile::tempdir().unwrap(); - let cgroup = fake_cgroup(temporary.path(), "3\n"); - let kill = cgroup.path().join("cgroup.kill"); - let pids = cgroup.path().join("pids.current"); - let mut side = Side::start( - shell( - "setsid sleep 60 2>/dev/null & echo $!; \ - sleep 60 /dev/null 2>&1 & echo $! >&2", - ), - false, - Options::new().cgroup(&cgroup, Scope::Execution), - limits(16 * 1024), - timers(), - ); - // The program ends at once, and its standard error with it, as no descendant holds it. Its - // standard output stays open, and the cgroup is killed. - drive(&mut [&mut side], |sides| { - sides[0].output.iter().all(|output| output.ends_with(b"\n")) - && sides[0].events.contains(&peer::Event::End(Stream::Stderr)) - && sides[0].binding.execution().status().is_some() - }); - assert_eq!(std::fs::read(&kill).unwrap(), b"1"); - assert!(!side.events.contains(&peer::Event::End(Stream::Stdout))); - let holder = descendant(&side.output[0]); - let detached = descendant(&side.output[1]); - // The kernel's kill: the descendants end, and the pipe's last writer with them. - for (process, _) in [&holder, &detached] { - sys::kill_process(*process).unwrap(); - } - for (_, pidfd) in [&holder, &detached] { - let patience = u16::try_from(PATIENCE.as_millis()).unwrap(); - assert_eq!( - sys::readable_or_hung_up(&[pidfd.borrowed()], patience).unwrap(), - [true] - ); - } - // The program's end, the streams' ends, and the descendants' end are not settlement: the - // cgroup still counts processes, so the execution stays unsettled. - drive(&mut [&mut side], |sides| { - sides[0].events.contains(&peer::Event::End(Stream::Stdout)) - }); - drive_for(&mut [&mut side], Duration::from_millis(300)); - assert!(!side.binding.is_finished()); - assert!(side.terminal.is_none()); - assert!(side.binding.runner().awaits_settlement()); - // Once the cgroup is empty, the execution settles, and is reusable. - std::fs::write(&pids, b"0\n").unwrap(); - let outcome = finish(&mut side); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert!(outcome.is_reusable()); - assert!(!outcome.is_truncated(Stream::Stdout)); - assert!(side.failures.is_empty(), "{:?}", side.failures); -} - -#[test] -fn the_settlement_kills_what_the_program_leaves_behind_in_a_cgroup_that_the_test_plays() { - // The program's descendant holds neither output stream, and both streams end while the - // program runs, so that once it ends, the settlement's kill is the cgroup's only one. - let temporary = tempfile::tempdir().unwrap(); - let cgroup = fake_cgroup(temporary.path(), "2\n"); - let kill = cgroup.path().join("cgroup.kill"); - let go = temporary.path().join("go"); - let mut side = Side::start( - shell(&format!( - "sleep 60 /dev/null 2>&1 & echo $!; exec >&- 2>&-; \ - until [ -e '{}' ]; do sleep 0.01; done", - go.display() - )), - false, - Options::new().cgroup(&cgroup, Scope::Execution), - limits(16 * 1024), - timers(), - ); - drive(&mut [&mut side], |sides| { - Stream::ALL - .into_iter() - .all(|stream| sides[0].events.contains(&peer::Event::End(stream))) - }); - let (process, pidfd) = descendant(&side.output[0]); - assert!(side.binding.execution().status().is_none()); - assert_eq!(std::fs::read(&kill).unwrap(), b""); - // The program ends; the settlement kills the cgroup, and checks it from then on. - std::fs::write(&go, b"").unwrap(); - drive(&mut [&mut side], |sides| { - sides[0].binding.execution().status().is_some() - }); - assert_eq!(std::fs::read(&kill).unwrap(), b"1"); - assert!(side.binding.runner().awaits_settlement()); - assert!(!side.binding.is_finished()); - // The kernel's kill ends the descendant, after which the cgroup counts no process. - sys::kill_process(process).unwrap(); - let patience = u16::try_from(PATIENCE.as_millis()).unwrap(); - assert_eq!( - sys::readable_or_hung_up(&[pidfd.borrowed()], patience).unwrap(), - [true] - ); - std::fs::write(cgroup.path().join("pids.current"), b"0\n").unwrap(); - let outcome = finish(&mut side); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert!(outcome.is_reusable()); - assert!(side.failures.is_empty(), "{:?}", side.failures); -} - -#[test] -fn the_settlement_begins_as_the_program_ends_while_a_descendant_holds_its_output() { - // A descendant in a session of its own holds both of the program's output streams, and the - // fake cgroup counts no process. The program ends once the test lets it, when nothing is - // left to read, so that nothing but its end leads to the settlement, which is over before - // the output ends. Once the outcome is reusable, the caller reuses the cgroup before it drops - // the binding, which then kills nothing in it. - let temporary = tempfile::tempdir().unwrap(); - let cgroup = fake_cgroup(temporary.path(), "0\n"); - let go = temporary.path().join("go"); - let mut side = Side::start( - shell(&format!( - "setsid sleep 60 & echo $!; until [ -e '{}' ]; do sleep 0.01; done", - go.display() - )), - false, - Options::new().cgroup(&cgroup, Scope::Execution), - limits(16 * 1024), - timers(), - ); - drive(&mut [&mut side], |sides| { - sides[0].output[0].ends_with(b"\n") - }); - let holder = Stray(descendant(&side.output[0])); - std::fs::write(&go, b"").unwrap(); - drive(&mut [&mut side], |sides| { - sides[0].binding.execution().status().is_some() - && !sides[0].binding.runner().awaits_settlement() - }); - assert!( - side.events - .iter() - .all(|event| !matches!(event, peer::Event::End(_))) - ); - assert!(!side.binding.is_finished()); - // The output's end is the last that the terminal outcome waits for. - drop(holder); - let outcome = finish(&mut side); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert!(outcome.is_reusable()); - assert!(!outcome.is_truncated(Stream::Stdout)); - // The settlement killed the cgroup, which is the caller's from then on. - let kill = cgroup.path().join("cgroup.kill"); - assert_eq!(std::fs::read(&kill).unwrap(), b"1"); - std::fs::write(&kill, b"").unwrap(); - drop(side); - assert_eq!(std::fs::read(&kill).unwrap(), b""); -} - -/// A workload cgroup in the real hierarchy, which the test removes, with everything in it, when -/// it ends. -struct RealWorkload(Layout); - -impl RealWorkload { - /// Creates a new workload cgroup below the root of the hierarchy at `/sys/fs/cgroup`, with - /// the test's own cgroup as the agent's. It returns `None` unless the test runs as root in a - /// cgroup v2 hierarchy with the `pids` controller, in which it can create a cgroup. - fn new(name: &str) -> Option { - let root = Path::new("/sys/fs/cgroup"); - let controllers = std::fs::read_to_string(root.join("cgroup.controllers")).ok()?; - if !nix::unistd::geteuid().is_root() - || !controllers - .split_whitespace() - .any(|controller| controller == "pids") - { - return None; - } - let own = std::fs::read_to_string("/proc/self/cgroup").ok()?; - let own = own.lines().find_map(|line| line.strip_prefix("0::"))?; - let agent = root.join(own.trim_start_matches('/')); - let workload = root.join(format!("nvx-binding-{}-{name}", std::process::id())); - let layout = Layout::new(agent, workload).ok()?; - std::fs::create_dir(layout.workload()).ok()?; - Some(Self(layout)) - } -} - -impl Drop for RealWorkload { - /// Kills whatever is left in the workload cgroup, then removes its cgroups. - fn drop(&mut self) { - let workload = self.0.workload(); - let _ = std::fs::write(workload.join("cgroup.kill"), b"1"); - for _ in 0..1000 { - if std::fs::read(workload.join("pids.current")).is_ok_and(|count| count == b"0\n") { - break; - } - std::thread::sleep(Duration::from_millis(10)); - } - if let Ok(entries) = std::fs::read_dir(workload) { - for entry in entries.flatten() { - if entry.file_type().is_ok_and(|kind| kind.is_dir()) { - let _ = std::fs::remove_dir(entry.path()); - } - } - } - let _ = std::fs::remove_dir(workload); - } -} - -#[test] -fn descendants_setsid_and_pipe_writers_settle_before_reuse_in_a_real_cgroup() { - // The test changes the real hierarchy, which needs root; elsewhere it does nothing. - let Some(workload) = RealWorkload::new("settle") else { - return; - }; - let cgroup = ExecutionCgroup::prepare(&workload.0, Some("exec"), &[]).unwrap(); - // The program ends only once the test has opened its descendants, which the settlement - // kills as soon as the binding collects the program's status. - let temporary = tempfile::tempdir().unwrap(); - let opened = temporary.path().join("opened"); - let mut side = Side::start( - shell(&format!( - "setsid sleep 60 & echo $!; \ - sleep 60 /dev/null 2>&1 & echo $! >&2; \ - while [ ! -e '{}' ]; do sleep 0.01; done", - opened.display() - )) - .cgroup(cgroup.placement().unwrap()), - false, - Options::new().cgroup(&cgroup, Scope::Execution), - limits(16 * 1024), - timers(), - ); - drive(&mut [&mut side], |sides| { - sides[0].output.iter().all(|output| output.ends_with(b"\n")) - }); - let holder = descendant(&side.output[0]); - let detached = descendant(&side.output[1]); - std::fs::write(&opened, b"").unwrap(); - let outcome = finish(&mut side); - // The terminal outcome came once the cgroup was empty: both descendants, the one in a - // session of its own that held the output included, are gone. - assert!(!cgroup.is_occupied(Scope::Execution).unwrap()); - for (_, pidfd) in [&holder, &detached] { - assert_eq!( - sys::readable_or_hung_up(&[pidfd.borrowed()], 0).unwrap(), - [true] - ); - } - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert!(outcome.is_reusable(), "{:?}", side.failures); - assert!(!outcome.is_truncated(Stream::Stdout)); - // The caller reuses the cgroup at once, for a new program, before it drops the binding: the - // new program outlives the drop. - let spawned = shell("exec sleep 60") - .cgroup(cgroup.placement().unwrap()) - .spawn() - .unwrap(); - let id = spawned.id(); - let pidfd = sys::open_pidfd(ProcessId::from_positive(i64::from(id)).unwrap()).unwrap(); - assert_eq!(spawned.into_report().wait().unwrap(), None); - drop(side); - assert!(cgroup.is_occupied(Scope::Execution).unwrap()); - assert_eq!( - sys::readable_or_hung_up(&[pidfd.borrowed()], 0).unwrap(), - [false] - ); - cgroup.kill(Scope::Execution).unwrap(); - let status = Child::new(id).unwrap().wait().unwrap(); - assert_eq!(status.signal(), Some(9)); - cgroup.remove().unwrap(); -} - -#[test] -fn a_cleanup_failure_fences_the_execution() { - // A cgroup whose kill fails, one that never empties, and none at all. - let temporary = tempfile::tempdir().unwrap(); - let unkillable = fake_cgroup(&temporary.path().join("unkillable"), "0\n"); - std::fs::remove_file(unkillable.path().join("cgroup.kill")).unwrap(); - let occupied = fake_cgroup(&temporary.path().join("occupied"), "1\n"); - let short = Timers::new(GRACE, Duration::from_millis(300)); - for (options, timers, failed) in [ - ( - Options::new().cgroup(&unkillable, Scope::Execution), - timers(), - true, - ), - ( - Options::new().cgroup(&occupied, Scope::Execution), - short, - false, - ), - (Options::new(), timers(), false), - ] { - let mut side = Side::start(shell("printf ran"), false, options, limits(4096), timers); - let outcome = finish(&mut side); - assert_eq!(side.output[0], b"ran"); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert!(!outcome.is_reusable()); - assert!(!outcome.is_truncated(Stream::Stdout)); - assert_eq!( - side.failures - .iter() - .any(|failure| failure.starts_with("settlement")), - failed, - "{:?}", - side.failures - ); - } -} - -#[test] -fn an_unrelated_child_s_status_stays_with_its_owner() { - let mut unrelated = std::process::Command::new("/bin/sh") - .args(["-c", "exit 7"]) - .spawn() - .unwrap(); - while !sys::tests::has_exited(ProcessId::from_child(&unrelated)).unwrap() { - std::thread::yield_now(); - } - let temporary = tempfile::tempdir().unwrap(); - let cgroup = fake_cgroup(temporary.path(), "0\n"); - let mut side = Side::start( - shell("printf ran"), - false, - Options::new().cgroup(&cgroup, Scope::Execution), - limits(4096), - timers(), - ); - assert!(finish(&mut side).is_reusable()); - // Nothing collected the unrelated child's status, which its own wait still finds. - assert_eq!(unrelated.wait().unwrap().code(), Some(7)); -} - -#[test] -fn a_caller_that_reaps_hands_the_program_s_status_over() { - let temporary = tempfile::tempdir().unwrap(); - let cgroup = fake_cgroup(temporary.path(), "0\n"); - let mut side = Side::start( - shell("printf ran; exit 6"), - false, - Options::new() - .cgroup(&cgroup, Scope::Execution) - .caller_reaps(), - limits(4096), - timers(), - ); - let id = side.binding.id(); - // The binding never waits for the program, nor lists its pidfd. - assert!( - side.binding - .descriptors() - .all(|(source, _)| source != Source::Exit) - ); - // The program ends, which nothing in the binding waits for. - drive(&mut [&mut side], |sides| { - sides[0].output[0] == b"ran" - && Stream::ALL - .into_iter() - .all(|stream| sides[0].events.contains(&peer::Event::End(stream))) - }); - let process = ProcessId::from_positive(i64::from(id)).unwrap(); - let started = Instant::now(); - while !sys::tests::has_exited(process).unwrap() { - assert!(started.elapsed() < PATIENCE, "the program ended in time"); - std::thread::sleep(Duration::from_millis(1)); - } - drive_for(&mut [&mut side], Duration::from_millis(100)); - assert!(!side.binding.is_finished()); - // The caller's own collector takes the status, and hands it over. - let status = Child::new(id).unwrap().wait().unwrap(); - assert!(side.binding.observe(id, status)); - let outcome = finish(&mut side); - assert_eq!(outcome.status(), Some(Status::Exited(6))); - assert!(outcome.is_reusable()); -} - -#[test] -fn a_signal_that_the_program_handles_reaches_the_runner_only_as_the_program_s_end() { - let temporary = tempfile::tempdir().unwrap(); - let cgroup = fake_cgroup(temporary.path(), "0\n"); - // The program handles `SIGTERM` by writing more than the windows hold, and exiting 0, and - // waits for a descendant in its process group, which the signal ends too, without dash's - // report of the signal that ended it. It exits 3 unless its handler runs. - let mut side = Side::start( - shell( - "trap 'seq 1 20000; exit 0' TERM; sleep 30 /dev/null 2>&1 & \ - printf ready; wait $! 2>/dev/null; exit 3", - ), - false, - Options::new().cgroup(&cgroup, Scope::Execution), - limits(4096), - timers(), - ); - drive(&mut [&mut side], |sides| sides[0].output[0] == b"ready"); - let refused = side.binding.signal(nix::libc::SIGKILL).unwrap_err(); - assert_eq!(refused.raw_os_error(), Some(nix::libc::EINVAL)); - assert!(side.binding.signal(nix::libc::SIGTERM).unwrap()); - let outcome = finish(&mut side); - let mut output = b"ready".to_vec(); - output.extend(sequence(20_000)); - assert_eq!(side.output, [output, Vec::new()]); - assert_eq!(Some(outcome), side.terminal); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert_eq!(outcome.cause(), None); - assert!(!outcome.is_truncated(Stream::Stdout)); - assert!(outcome.is_reusable()); - assert!(side.failures.is_empty(), "{:?}", side.failures); - assert!(!side.binding.signal(nix::libc::SIGTERM).unwrap()); -} - -#[test] -fn the_program_s_end_reaches_the_runner_before_queued_output_whoever_collects_its_status() { - // The program writes 24 KiB to a sink that takes nothing: the peer holds a window, 4 KiB, - // the runner a chunk, 16 KiB, and the pipe the rest, 4 KiB, which even a pipe of one page - // holds. Then it exits 3, once the test lets it, before the runner's timeout, while nothing - // collects its status: neither the execution, which the test does not drive meanwhile, nor - // the caller's own collector, which then hands it over. Every call from then on comes after - // the timeout, and the sink takes the output 64 bytes at a time, so that every call has - // queued output to return: the program's end reached the runner first, so the timeout never - // fires. - const TIMEOUT: Duration = Duration::from_secs(20); - const PAYLOAD: usize = 24 * 1024; - let limits = limits(4096); - for caller_reaps in [false, true] { - let temporary = tempfile::tempdir().unwrap(); - let cgroup = fake_cgroup(temporary.path(), "0\n"); - let gate = temporary.path().join("gate"); - let mut options = Options::new().cgroup(&cgroup, Scope::Execution); - if caller_reaps { - options = options.caller_reaps(); - } - let mut side = Side::start( - shell(&format!( - "head -c {PAYLOAD} /dev/zero; while [ ! -e '{}' ]; do sleep 0.01; done; exit 3", - gate.display() - )), - false, - options, - limits, - Timers::new(GRACE, TIMEOUT * 3).timeout(TIMEOUT), - ); - // Only the runner has the timeout. The calls after it come within its settlement time. - side.peer = Peer::new(KEY, limits, timers(), Instant::now()); - let late = Instant::now() + TIMEOUT; - side.stalled = true; - drive(&mut [&mut side], |sides| { - sides[0].peer.buffered() == 4096 - && sides[0].binding.runner().read_allowance(Stream::Stdout) == 0 - }); - std::fs::write(&gate, b"").unwrap(); - let id = side.binding.id(); - let process = ProcessId::from_positive(i64::from(id)).unwrap(); - let started = Instant::now(); - while !sys::tests::has_exited(process).unwrap() { - assert!(started.elapsed() < PATIENCE, "the program ended in time"); - std::thread::sleep(Duration::from_millis(1)); - } - if caller_reaps { - let status = Child::new(id).unwrap().wait().unwrap(); - assert!(side.binding.observe(id, status)); - } - side.stalled = false; - let mut buffer = [0u8; 64]; - side.serve(late, &mut buffer); - let event = side.binding.next(late, &mut buffer); - assert!( - matches!( - event, - Some(Event::Message(runner::Message::Output { - stream: Stream::Stdout, - .. - })) - ), - "{event:?}" - ); - side.handle(late, event.unwrap(), &buffer); - let patience = Instant::now() + PATIENCE; - while !side.is_finished() { - assert!(Instant::now() < patience, "the execution finished in time"); - if !side.pump(late, &mut buffer) { - wait(&[&mut side], Instant::now() + Duration::from_millis(10)); - } - } - let terminal = side.terminal.unwrap(); - assert_eq!(terminal.status(), Some(Status::Exited(3)), "{caller_reaps}"); - assert_eq!(terminal.cause(), None, "{caller_reaps}"); - assert!(!terminal.is_truncated(Stream::Stdout), "{caller_reaps}"); - assert!(terminal.is_reusable(), "{caller_reaps}"); - assert_eq!(side.output[0], vec![0; PAYLOAD], "{caller_reaps}"); - assert!(side.failures.is_empty(), "{:?}", side.failures); - } -} - -#[test] -fn the_program_s_end_reaches_the_runner_before_its_timeout_in_a_first_call_after_it() { - // The program writes a word and exits 4 at once, and its launch report arrives, but the first - // call comes only after the runner's timeout, whoever collects the program's status: the - // launch report and the program's end reach the runner before the timeout fires, so the - // output arrives whole, and the outcome names no cause. - const TIMEOUT: Duration = Duration::from_secs(20); - let limits = limits(4096); - for caller_reaps in [false, true] { - let temporary = tempfile::tempdir().unwrap(); - let cgroup = fake_cgroup(temporary.path(), "0\n"); - let mut options = Options::new().cgroup(&cgroup, Scope::Execution); - if caller_reaps { - options = options.caller_reaps(); - } - let mut side = Side::start( - shell("printf hello; exit 4"), - false, - options, - limits, - Timers::new(GRACE, TIMEOUT * 3).timeout(TIMEOUT), - ); - // Only the runner has the timeout. - side.peer = Peer::new(KEY, limits, timers(), Instant::now()); - let late = Instant::now() + TIMEOUT; - let id = side.binding.id(); - let process = ProcessId::from_positive(i64::from(id)).unwrap(); - let started = Instant::now(); - while !sys::tests::has_exited(process).unwrap() { - assert!(started.elapsed() < PATIENCE, "the program ended in time"); - std::thread::sleep(Duration::from_millis(1)); - } - if caller_reaps { - let status = Child::new(id).unwrap().wait().unwrap(); - assert!(side.binding.observe(id, status)); - } - let report = side - .binding - .descriptors() - .find_map(|(source, descriptor)| matches!(source, Source::Launch).then_some(descriptor)) - .expect("the launch report is pending"); - let patience = u16::try_from(PATIENCE.as_millis()).unwrap(); - assert_eq!( - sys::readable_or_hung_up(&[report], patience).unwrap(), - [true] - ); - let mut buffer = [0u8; 64]; - let patience = Instant::now() + PATIENCE; - while !side.is_finished() { - assert!(Instant::now() < patience, "the execution finished in time"); - if !side.pump(late, &mut buffer) { - wait(&[&mut side], Instant::now() + Duration::from_millis(10)); - } - } - let terminal = side.terminal.unwrap(); - assert_eq!(terminal.status(), Some(Status::Exited(4)), "{caller_reaps}"); - assert_eq!(terminal.cause(), None, "{caller_reaps}"); - assert!(!terminal.is_truncated(Stream::Stdout), "{caller_reaps}"); - assert!(terminal.is_reusable(), "{caller_reaps}"); - assert_eq!(side.output[0], b"hello", "{caller_reaps}"); - assert!(side.failures.is_empty(), "{:?}", side.failures); - } -} - -#[test] -fn the_program_s_end_reaches_the_runner_before_its_timeout_while_its_launch_report_is_late() { - // The program writes a word and exits 4 at once, but the test holds a copy of its launch - // report's writer, as a process that another thread forked while the program was spawning - // would, so the report arrives only once the program has ended and the first calls, which - // come after the runner's timeout, have taken every step that they could. The program's end - // reaches the runner before the timeout fires all the same, whoever collects its status, and - // the runner holds the program's start and output until the report arrives: the output - // arrives whole, and the outcome names no cause. - const TIMEOUT: Duration = Duration::from_secs(20); - let limits = limits(4096); - for caller_reaps in [false, true] { - let temporary = tempfile::tempdir().unwrap(); - let cgroup = fake_cgroup(temporary.path(), "0\n"); - let mut options = Options::new().cgroup(&cgroup, Scope::Execution); - if caller_reaps { - options = options.caller_reaps(); - } - let (pipes, ends) = Pipes::new(false).unwrap(); - let (spawned, writer) = shell("printf hello; exit 4") - .stdin(OwnedFd::from(std::fs::File::open("/dev/null").unwrap())) - .stdout(ends.stdout) - .stderr(ends.stderr) - .new_process_group() - .spawn_holding_report() - .unwrap(); - let execution = Execution::new(spawned, pipes, options.process_group()).unwrap(); - let mut side = Side::bound( - execution, - limits, - Timers::new(GRACE, TIMEOUT * 3).timeout(TIMEOUT), - ); - // Only the runner has the timeout. - side.peer = Peer::new(KEY, limits, timers(), Instant::now()); - let late = Instant::now() + TIMEOUT; - let id = side.binding.id(); - let process = ProcessId::from_positive(i64::from(id)).unwrap(); - let started = Instant::now(); - while !sys::tests::has_exited(process).unwrap() { - assert!(started.elapsed() < PATIENCE, "the program ended in time"); - std::thread::sleep(Duration::from_millis(1)); - } - if caller_reaps { - let status = Child::new(id).unwrap().wait().unwrap(); - assert!(side.binding.observe(id, status)); - } - let mut buffer = [0u8; 64]; - while side.pump(late, &mut buffer) {} - assert!(!side.peer.has_started(), "{caller_reaps}"); - assert!(side.output[0].is_empty(), "{caller_reaps}"); - assert!(side.terminal.is_none(), "{caller_reaps}"); - drop(writer); - let patience = Instant::now() + PATIENCE; - while !side.is_finished() { - assert!(Instant::now() < patience, "the execution finished in time"); - if !side.pump(late, &mut buffer) { - wait(&[&mut side], Instant::now() + Duration::from_millis(10)); - } - } - let terminal = side.terminal.unwrap(); - assert_eq!(terminal.status(), Some(Status::Exited(4)), "{caller_reaps}"); - assert_eq!(terminal.cause(), None, "{caller_reaps}"); - assert!(!terminal.is_truncated(Stream::Stdout), "{caller_reaps}"); - assert!(terminal.is_reusable(), "{caller_reaps}"); - assert_eq!(side.output[0], b"hello", "{caller_reaps}"); - assert!(side.failures.is_empty(), "{:?}", side.failures); - } -} - -#[test] -fn the_runner_gives_up_in_time_on_a_launch_report_that_a_process_holds_after_the_program_s_end() { - // The program writes a word and exits 4 at once, but the test holds a copy of its launch - // report's writer throughout, as a process that another thread forked while the program was - // spawning would if it neither replaced itself with its program nor ended, whether the - // execution settles or, without a cgroup, does not. Only the runner has a timeout, and the - // first calls come after it. The runner holds the program's start and output until the - // settlement time and the grace have passed since the program's end, then gives up on the - // report: the output arrives whole, without a start, and the outcome keeps the program's - // status, names no cause, and is fenced. - const TIMEOUT: Duration = Duration::from_secs(20); - let limits = limits(4096); - for settles in [true, false] { - let temporary = tempfile::tempdir().unwrap(); - let cgroup = fake_cgroup(temporary.path(), "0\n"); - let options = if settles { - Options::new().cgroup(&cgroup, Scope::Execution) - } else { - Options::new() - }; - let (pipes, ends) = Pipes::new(false).unwrap(); - let (spawned, writer) = shell("printf hello; exit 4") - .stdin(OwnedFd::from(std::fs::File::open("/dev/null").unwrap())) - .stdout(ends.stdout) - .stderr(ends.stderr) - .new_process_group() - .spawn_holding_report() - .unwrap(); - let execution = Execution::new(spawned, pipes, options.process_group()).unwrap(); - let mut side = Side::bound( - execution, - limits, - Timers::new(GRACE, SETTLEMENT).timeout(TIMEOUT), - ); - // Only the runner has the timeout. - side.peer = Peer::new(KEY, limits, timers(), Instant::now()); - let late = Instant::now() + TIMEOUT; - let id = side.binding.id(); - let process = ProcessId::from_positive(i64::from(id)).unwrap(); - let started = Instant::now(); - while !sys::tests::has_exited(process).unwrap() { - assert!(started.elapsed() < PATIENCE, "the program ended in time"); - std::thread::sleep(Duration::from_millis(1)); - } - let mut buffer = [0u8; 64]; - // A process that another test's thread forked while the program was spawning may hold - // the program's output pipes for a moment too, and the runner gives up on the streams - // still open at the instant at which it gives up on the report, so the test waits for - // both streams to close first. - let patience = Instant::now() + PATIENCE; - while Stream::ALL - .into_iter() - .any(|stream| side.binding.runner().read_allowance(stream) > 0) - { - assert!( - Instant::now() < patience, - "the program's streams closed in time" - ); - if !side.pump(late, &mut buffer) { - wait(&[&mut side], Instant::now() + Duration::from_millis(10)); - } - } - while side.pump(late, &mut buffer) {} - let given_up = late + SETTLEMENT + GRACE; - assert_eq!(side.binding.deadline(), Some(given_up), "{settles}"); - while side.pump(given_up - Duration::from_millis(1), &mut buffer) {} - assert!(!side.peer.has_started(), "{settles}"); - assert!(side.output[0].is_empty(), "{settles}"); - assert!(side.terminal.is_none(), "{settles}"); - let patience = Instant::now() + PATIENCE; - while !side.is_finished() { - assert!(Instant::now() < patience, "the execution finished in time"); - if !side.pump(given_up, &mut buffer) { - wait(&[&mut side], Instant::now() + Duration::from_millis(10)); - } - } - let terminal = side.terminal.unwrap(); - assert_eq!(terminal.status(), Some(Status::Exited(4)), "{settles}"); - assert_eq!(terminal.cause(), None, "{settles}"); - assert_eq!(terminal.launch_failure(), None, "{settles}"); - assert!(!terminal.is_truncated(Stream::Stdout), "{settles}"); - assert!(!terminal.is_reusable(), "{settles}"); - assert_eq!(side.output[0], b"hello", "{settles}"); - assert!(!side.peer.has_started(), "{settles}"); - assert!(side.failures.is_empty(), "{:?}", side.failures); - drop(writer); - } -} - -/// How the liveness harness ends the stalled execution. -#[derive(Clone, Copy, Debug, PartialEq, Eq)] -enum Ending { - /// The peer's caller cancels it. - Cancel, - /// The runner's timeout passes. - Timeout, - /// The session is reset. - Reset, -} - -#[test] -fn a_stalled_sink_stalls_only_its_own_execution_and_endings_still_finish_in_time() { - const TIMEOUT: Duration = Duration::from_secs(2); - for ending in [Ending::Cancel, Ending::Timeout, Ending::Reset] { - let temporary = tempfile::tempdir().unwrap(); - let stalled_cgroup = fake_cgroup(&temporary.path().join("stalled"), "0\n"); - let other_cgroup = fake_cgroup(&temporary.path().join("other"), "0\n"); - let stalled_timers = match ending { - Ending::Timeout => timers().timeout(TIMEOUT), - _ => timers(), - }; - let mut stalled = Side::start( - shell("exec head -c 16777216 /dev/zero"), - false, - Options::new().cgroup(&stalled_cgroup, Scope::Execution), - limits(64 * 1024), - stalled_timers, - ); - let created = Instant::now(); - stalled.stalled = true; - // The stalled sink exhausts its window, and its program blocks on its full pipe. - drive(&mut [&mut stalled], |sides| { - sides[0].binding.runner().read_allowance(Stream::Stdout) == 0 - && sides[0].peer.buffered() == 64 * 1024 - }); - // Another execution streams meanwhile, and keeps streaming after a MiB. - let mut other = Side::start( - shell("seq 1 600000"), - false, - Options::new().cgroup(&other_cgroup, Scope::Execution), - limits(64 * 1024), - timers(), - ); - drive(&mut [&mut stalled, &mut other], |sides| { - sides[1].output[0].len() > 1 << 20 - }); - assert!(stalled.binding.execution().status().is_none(), "{ending:?}"); - assert!(stalled.output[0].is_empty()); - assert_eq!(stalled.peer.buffered(), 64 * 1024, "{ending:?}"); - let streamed = other.output[0].len(); - let ended = match ending { - Ending::Cancel => { - let now = Instant::now(); - stalled.peer.cancel(now); - // The cancellation reaches the runner at once, so that both sides' graces end - // together: a runner whose grace outlasted the peer's would take the window that - // the peer's ended grace frees, and could send the rest without a loss of its own. - stalled.serve(now, &mut [0; 64]); - now - } - Ending::Timeout => created + TIMEOUT, - Ending::Reset => { - let now = Instant::now(); - stalled.binding.reset(now); - stalled.peer.reset(now); - now - } - }; - // The stalled execution ends, while the other one streams on. - let mut finished = None; - drive(&mut [&mut stalled, &mut other], |sides| { - if sides[0].is_finished() { - finished.get_or_insert_with(Instant::now); - } - finished.is_some() && sides[1].output[0].len() > streamed - }); - // Each side finished within its grace, and the peer, which learns of the timeout only - // from the terminal outcome, within a second one. - let finished = finished.unwrap(); - let grace = match ending { - Ending::Timeout => GRACE + GRACE, - _ => GRACE, - }; - assert!( - finished <= ended + grace + SLACK, - "{ending:?} took {:?}", - finished - ended - ); - let cause = match ending { - Ending::Cancel => Cause::Cancelled, - Ending::Timeout => Cause::TimedOut, - Ending::Reset => Cause::Reset, - }; - let terminal = stalled.terminal.unwrap(); - assert_eq!(terminal.cause(), Some(cause), "{ending:?}"); - assert_eq!(terminal.status(), Some(Status::Signaled(9)), "{ending:?}"); - assert!(terminal.is_truncated(Stream::Stdout), "{ending:?}"); - assert!(terminal.is_reusable(), "{ending:?}"); - let outcome = stalled.outcome().unwrap(); - assert_eq!(outcome.cause(), Some(cause), "{ending:?}"); - assert!(outcome.is_truncated(Stream::Stdout), "{ending:?}"); - // The other execution still finishes whole. - let outcome = finish(&mut other); - assert_eq!(other.output[0], sequence(600_000), "{ending:?}"); - assert!(!outcome.is_truncated(Stream::Stdout)); - } -} - -#[test] -fn a_call_takes_a_bounded_number_of_steps_and_asks_to_be_called_again() { - // The binding reads at most a window and a chunk, 20 KiB, of the program's 24 KiB, so the - // rest, 4 KiB, which any pipe holds, is left in the pipe once the program wrote everything. - // The session is reset then, and the binding discards the rest with a buffer of 16 bytes: - // in more steps than one call takes. - let temporary = tempfile::tempdir().unwrap(); - let cgroup = fake_cgroup(temporary.path(), "0\n"); - let written = temporary.path().join("written"); - let mut side = Side::start( - shell(&format!( - "head -c 24576 /dev/zero; : >'{}'; exec sleep 60", - written.display() - )), - false, - Options::new().cgroup(&cgroup, Scope::Execution), - limits(4096), - timers(), - ); - side.stalled = true; - drive(&mut [&mut side], |sides| { - sides[0].peer.buffered() == 4096 - && sides[0].binding.runner().read_allowance(Stream::Stdout) == 0 - }); - // The rest fits in the pipe, so the program writes it without another read. - let started = Instant::now(); - while !written.exists() { - assert!(started.elapsed() < PATIENCE, "the program wrote in time"); - std::thread::sleep(Duration::from_millis(1)); - } - let now = Instant::now(); - side.binding.reset(now); - side.peer.reset(now); - assert!(side.binding.next(now, &mut [0; 16]).is_none()); - assert_eq!(side.binding.deadline(), Some(now)); - let outcome = finish(&mut side); - assert_eq!(outcome.cause(), Some(Cause::Reset)); - let terminal = side.terminal.unwrap(); - assert_eq!(terminal.cause(), Some(Cause::Reset)); - assert!(terminal.is_truncated(Stream::Stdout)); - assert!(terminal.is_reusable()); -} - -#[test] -fn a_call_writes_a_bounded_number_of_chunks_of_input_and_asks_to_be_called_again() { - // The runner holds a whole window of input, 8 KiB in chunks of 128 bytes, before the binding - // writes any of it, and the program reads all of it. Each call writes at most a chunk at a - // time, and at most `STEPS` times, so 2 KiB, where a pipe, which takes a page at least, - // would take 4 KiB or more at once. A call that stops at that bound with input left sets the - // deadline to its own instant. - const CHUNK: usize = 128; - const WINDOW: usize = 8192; - let temporary = tempfile::tempdir().unwrap(); - let cgroup = fake_cgroup(temporary.path(), "0\n"); - let window = Window::bytes(u32::try_from(WINDOW).unwrap()); - let chunk = u32::try_from(CHUNK).unwrap(); - let mut side = Side::start( - shell("exec cat >/dev/null"), - true, - Options::new().cgroup(&cgroup, Scope::Execution), - Limits::new(chunk, window, Window::bytes(4096)).unwrap(), - timers(), - ); - side.input = vec![b'i'; WINDOW]; - let mut buffer = [0u8; 4096]; - let now = Instant::now(); - while let Some(event) = side.binding.next(now, &mut buffer) { - side.handle(now, event, &buffer); - } - while side.sent < WINDOW { - assert!( - side.serve(now, &mut buffer), - "the peer hands its input over" - ); - } - assert_eq!(side.binding.runner().buffered(), WINDOW); - let mut paused = false; - let patience = Instant::now() + PATIENCE; - while side.binding.runner().buffered() > 0 { - assert!( - Instant::now() < patience, - "the program took the input in time" - ); - let before = side.binding.runner().buffered(); - let now = Instant::now(); - let event = side.binding.next(now, &mut buffer); - let written = before - side.binding.runner().buffered(); - assert!(written <= STEPS * CHUNK, "one call wrote {written} bytes"); - match event { - Some(event) => side.handle(now, event, &buffer), - // A pipe takes a write of a chunk, which is atomic, whole or not at all, so a call - // that wrote this much wrote `STEPS` times. - None if written == STEPS * CHUNK && side.binding.runner().buffered() > 0 => { - assert_eq!(side.binding.deadline(), Some(now)); - paused = true; - } - None => wait(&[&mut side], patience), - } - } - assert!(paused, "a call stopped at the bound with input left"); - let outcome = finish(&mut side); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert_eq!(outcome.cause(), None); - assert!(outcome.is_reusable()); -} - -#[test] -fn output_flows_while_the_caller_hands_over_input_before_every_call() { - // The program writes its output once the test lets it, which waits in its pipe, then reads - // its input to its end. Once the peer knows that the program started, which its output - // waits for, the caller takes one event of the binding at a time, and before each, hands the - // binding the input that the peer's credit allows, which the program's pipe takes at once, - // so that every call frees credit. The source holds a page of input, which any pipe takes - // whole, so no write waits for the program to read: the output must arrive before the - // source runs out. - const INPUT: usize = 4096; - let temporary = tempfile::tempdir().unwrap(); - let cgroup = fake_cgroup(temporary.path(), "0\n"); - let go = temporary.path().join("go"); - let written = temporary.path().join("written"); - let mut side = Side::start( - shell(&format!( - "while [ ! -e '{}' ]; do sleep 0.01; done; printf ready; : >'{}'; \ - exec cat >/dev/null", - go.display(), - written.display() - )), - true, - Options::new().cgroup(&cgroup, Scope::Execution), - Limits::new(16, Window::bytes(64), Window::bytes(4096)).unwrap(), - timers(), - ); - let mut buffer = [0u8; 4096]; - let patience = Instant::now() + PATIENCE; - while !side.peer.has_started() { - assert!(Instant::now() < patience, "the program started in time"); - let now = Instant::now(); - match side.binding.next(now, &mut buffer) { - Some(event) => side.handle(now, event, &buffer), - None => wait(&[&mut side], patience), - } - } - std::fs::write(&go, b"").unwrap(); - while !written.exists() { - assert!(Instant::now() < patience, "the program wrote in time"); - std::thread::sleep(Duration::from_millis(1)); - } - side.input = vec![b'i'; INPUT]; - while side.output[0].is_empty() { - assert!( - side.sent < INPUT, - "the output arrived while the input flowed" - ); - let now = Instant::now(); - side.serve(now, &mut buffer); - if let Some(event) = side.binding.next(now, &mut buffer) { - side.handle(now, event, &buffer); - } - } - assert!(side.sent > 0); - // The source ends its input there, and the program ends with it. - side.input.truncate(side.sent); - let outcome = finish(&mut side); - assert_eq!(side.output[0], b"ready"); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert_eq!(outcome.cause(), None); - assert!(!outcome.is_truncated(Stream::Stdout)); - assert!(outcome.is_reusable()); -} - -/// A process that escaped its execution, which the test kills when it ends, however it ends, -/// unless the process ended already. -struct Stray((ProcessId, sys::FileDescriptor)); - -impl Drop for Stray { - /// Kills the process unless its pidfd says that it ended. - fn drop(&mut self) { - let (process, pidfd) = &self.0; - if sys::readable_or_hung_up(&[pidfd.borrowed()], 0).is_ok_and(|ended| ended == [false]) { - let _ = sys::kill_process(*process); - } - } -} - -/// Waits until the program names its descendant in `file`, by its ID on one line, and returns -/// the descendant. -fn stray(file: &Path) -> Stray { - let started = Instant::now(); - loop { - if let Ok(line) = std::fs::read(file) - && line.ends_with(b"\n") - { - return Stray(descendant(&line)); - } - assert!( - started.elapsed() < PATIENCE, - "the program named its descendant in time" - ); - std::thread::sleep(Duration::from_millis(1)); - } -} - -#[test] -fn a_pipe_writer_that_escapes_the_kill_fences_its_execution_in_time_and_holds_no_other() { - // The fake cgroup kills nothing: the program's descendant, in a session of its own, writes - // to the program's standard output as fast as it can, and outlives the program's - // cancellation, as a process that escaped the cgroup would. Its sink takes nothing, so its - // execution discards what it writes once the grace ends, and gives up once the settlement - // time after the grace ends too. - let temporary = tempfile::tempdir().unwrap(); - let escaped_cgroup = fake_cgroup(&temporary.path().join("escaped"), "0\n"); - let other_cgroup = fake_cgroup(&temporary.path().join("other"), "0\n"); - let writer = temporary.path().join("writer"); - let settlement = Duration::from_millis(500); - let mut escaped = Side::start( - shell(&format!( - "setsid cat /dev/zero 2>/dev/null & echo $! >'{}'; exec sleep 60", - writer.display() - )), - false, - Options::new().cgroup(&escaped_cgroup, Scope::Execution), - limits(64 * 1024), - Timers::new(GRACE, settlement), - ); - escaped.stalled = true; - let _stray = stray(&writer); - drive(&mut [&mut escaped], |sides| { - sides[0].peer.buffered() == 64 * 1024 - }); - // Another execution streams meanwhile, and keeps streaming after a MiB. - let mut other = Side::start( - shell("seq 1 600000"), - false, - Options::new().cgroup(&other_cgroup, Scope::Execution), - limits(64 * 1024), - timers(), - ); - drive(&mut [&mut escaped, &mut other], |sides| { - sides[1].output[0].len() > 1 << 20 - }); - let streamed = other.output[0].len(); - let cancelled = Instant::now(); - escaped.peer.cancel(cancelled); - let mut finished = None; - drive(&mut [&mut escaped, &mut other], |sides| { - if sides[0].is_finished() { - finished.get_or_insert_with(Instant::now); - } - finished.is_some() && sides[1].output[0].len() > streamed - }); - let finished = finished.unwrap(); - assert!( - finished <= cancelled + GRACE + settlement + SLACK, - "took {:?}", - finished - cancelled - ); - // The program ended and its cgroup settled, but its output never did, so the execution is - // fenced, and its standard output truncated. - let terminal = escaped.terminal.unwrap(); - assert_eq!(terminal.cause(), Some(Cause::Cancelled)); - assert_eq!(terminal.status(), Some(Status::Signaled(9))); - assert!(terminal.is_truncated(Stream::Stdout)); - assert!(!terminal.is_reusable()); - assert!(!escaped.outcome().unwrap().is_reusable()); - // The other execution still finishes whole. - let outcome = finish(&mut other); - assert_eq!(other.output[0], sequence(600_000)); - assert!(!outcome.is_truncated(Stream::Stdout)); - assert!(outcome.is_reusable()); -} - -#[test] -fn an_execution_that_does_not_settle_gives_up_in_time_on_the_output_that_a_descendant_holds() { - // The program ends at once, while a descendant in a session of its own holds its standard - // output, which nothing ends: neither the fake cgroup, which kills nothing and always counts - // a process, nor the execution without a cgroup, whose settlement fails at once. The - // execution does not settle, so the runner gives up on the stream the settlement time and - // the grace after the program's end, and still sends what it read. - let settlement = Duration::from_millis(300); - let temporary = tempfile::tempdir().unwrap(); - let occupied = fake_cgroup(temporary.path(), "1\n"); - for cgroup in [Some(&occupied), None] { - let options = match cgroup { - Some(cgroup) => Options::new().cgroup(cgroup, Scope::Execution), - None => Options::new(), - }; - let mut side = Side::start( - shell("setsid sleep 60 2>/dev/null & echo $!"), - false, - options, - limits(16 * 1024), - Timers::new(GRACE, settlement), - ); - drive(&mut [&mut side], |sides| { - sides[0].output[0].ends_with(b"\n") && sides[0].binding.execution().status().is_some() - }); - let exited = Instant::now(); - let _holder = Stray(descendant(&side.output[0])); - let outcome = finish(&mut side); - let finished = Instant::now(); - assert!( - finished <= exited + settlement + GRACE + SLACK, - "took {:?}", - finished - exited - ); - let terminal = side.terminal.unwrap(); - assert_eq!(outcome, terminal); - assert_eq!(terminal.status(), Some(Status::Exited(0))); - assert_eq!(terminal.cause(), None); - assert!( - terminal.is_truncated(Stream::Stdout), - "with a cgroup: {}", - cgroup.is_some() - ); - assert!(!terminal.is_truncated(Stream::Stderr)); - assert!(!terminal.is_reusable()); - assert!(side.failures.is_empty(), "{:?}", side.failures); - } -} - -#[test] -fn an_execution_that_does_not_settle_loses_nothing_of_the_output_that_no_process_holds() { - // The program leaves nothing behind, and ends once its output fits in the window of its - // stalled sink, the runner's queue, and its pipe, which holds the rest, about 1 KiB, as any - // pipe does, but the execution does not settle: the fake cgroup always counts a process, and - // the execution without a cgroup fails its settlement at once. Its streams hung up, so the - // runner waits for the sink long after the settlement time and the grace, and loses nothing. - let settlement = Duration::from_millis(100); - let temporary = tempfile::tempdir().unwrap(); - let occupied = fake_cgroup(temporary.path(), "1\n"); - for cgroup in [Some(&occupied), None] { - let options = match cgroup { - Some(cgroup) => Options::new().cgroup(cgroup, Scope::Execution), - None => Options::new(), - }; - let mut side = Side::start( - shell("seq 1 7000; seq 1 100 >&2"), - false, - options, - limits(16 * 1024), - Timers::new(GRACE, settlement), - ); - side.stalled = true; - drive(&mut [&mut side], |sides| { - sides[0].binding.execution().status().is_some() - }); - // Past the time at which the runner gives up on the streams that a process may hold. - drive_for(&mut [&mut side], (settlement + GRACE) * 2); - let with = cgroup.is_some(); - assert!(!side.is_finished(), "with a cgroup: {with}"); - side.stalled = false; - let outcome = finish(&mut side); - assert_eq!(side.output[0], sequence(7000), "with a cgroup: {with}"); - assert_eq!(side.output[1], sequence(100), "with a cgroup: {with}"); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert_eq!(outcome.cause(), None); - assert!( - !outcome.is_truncated(Stream::Stdout) && !outcome.is_truncated(Stream::Stderr), - "with a cgroup: {with}" - ); - assert!(!outcome.is_reusable()); - assert!(side.failures.is_empty(), "{:?}", side.failures); - } -} - -#[test] -fn a_due_settlement_check_runs_while_output_drains_slowly() { - // The program writes 32 KiB, which its pipe holds, and ends. The adapter takes 64 bytes at a - // time, and 2 ms to send each message, so that every call has output to return for longer - // than the settlement time after the program's end: the settlement's due checks still run, - // and the execution settles. - let temporary = tempfile::tempdir().unwrap(); - let cgroup = fake_cgroup(temporary.path(), "0\n"); - let settlement = Duration::from_millis(100); - let mut side = Side::start( - shell("head -c 32768 /dev/zero"), - false, - Options::new().cgroup(&cgroup, Scope::Execution), - limits(16 * 1024), - Timers::new(GRACE, settlement), - ); - let mut buffer = [0u8; 64]; - let started = Instant::now(); - let mut exited = None; - while !side.is_finished() { - assert!( - started.elapsed() < PATIENCE, - "the execution finished in time" - ); - let now = Instant::now(); - match side.binding.next(now, &mut buffer) { - Some(event) => { - side.handle(now, event, &buffer); - std::thread::sleep(Duration::from_millis(2)); - } - None => wait(&[&mut side], started + PATIENCE), - } - side.serve(now, &mut buffer); - if exited.is_none() && side.binding.execution().status().is_some() { - exited = Some(Instant::now()); - } - } - // The output drained for longer than the settlement time after the program's end. - let drained = Instant::now() - exited.unwrap(); - assert!(drained > settlement, "drained in {drained:?}"); - assert_eq!(side.output[0], vec![0; 32768]); - let outcome = side.outcome().unwrap(); - assert_eq!(outcome.status(), Some(Status::Exited(0))); - assert!(outcome.is_reusable()); - assert!(!outcome.is_truncated(Stream::Stdout)); - assert!(side.failures.is_empty(), "{:?}", side.failures); -} diff --git a/nvx-rs/src/guest/mod.rs b/nvx-rs/src/guest/mod.rs index 7e67ddf1d..f0fc50cb7 100644 --- a/nvx-rs/src/guest/mod.rs +++ b/nvx-rs/src/guest/mod.rs @@ -6,8 +6,7 @@ //! each child that it names, and of no other; [`cgroup`], through which it places, accounts, //! kills, and settles the processes of its executions; [`mappings`], the host mappings that it //! mounts, and its account of a host's mapping table; [`exec`], which drives each program as one -//! execution, with its launch, its bounded output and input, its ending, and its outcome, alone -//! or, through its [`binding`](exec::binding), under the [execution engine](crate::exec); and, +//! execution, with its launch, its bounded output and input, its ending, and its outcome; and, //! on x86-64, [`snapshot`], which composes a strictly checked Fresh capture by one caller deadline, //! without execution control; and `activation`, through which a template's guest prepares for and requests a snapshot's //! capture before its root's assembly, behind the caller's fence of workload admission, with diff --git a/nvx-rs/src/guest/spawn.rs b/nvx-rs/src/guest/spawn.rs index d562aaf4f..7a1601303 100644 --- a/nvx-rs/src/guest/spawn.rs +++ b/nvx-rs/src/guest/spawn.rs @@ -80,8 +80,7 @@ use super::sys::{ /// step, which [`Command::reset_signals`] asks for otherwise. /// /// Each stage has a number, its [`code`](Self::code), with which the child's launch report names -/// the stage that failed, and which a [`Binding`](super::exec::binding::Binding) reports to its -/// runner as the stage of a [`LaunchFailure`](crate::exec::LaunchFailure). +/// the stage that failed. /// [`from_code`](Self::from_code) turns a number back into its stage. The numbers are part of the /// API: a stage keeps its number, and a new stage takes a number of its own, wherever it comes in /// the order. @@ -194,28 +193,7 @@ impl LaunchStage { } } - /// Returns the stage that `code` numbers, or `None` if no stage has that number, such as 0, - /// which a [`Binding`](super::exec::binding::Binding) reports for a launch whose report it - /// could not read. - /// - /// ``` - /// use nvx::exec::{LaunchFailure, Outcome}; - /// use nvx::guest::spawn::LaunchStage; - /// - /// /// Returns whether the program of `outcome` did not start because its working directory - /// /// was missing or out of its reach. - /// fn lacks_working_directory(outcome: &Outcome) -> bool { - /// outcome.launch_failure().is_some_and(|failure| { - /// LaunchStage::from_code(failure.stage()) == Some(LaunchStage::WorkingDirectory) - /// }) - /// } - /// - /// let failure = LaunchFailure::new(LaunchStage::WorkingDirectory.code(), 2); - /// assert!(lacks_working_directory( - /// &Outcome::new(None).with_launch_failure(failure) - /// )); - /// assert_eq!(LaunchStage::from_code(0), None); - /// ``` + /// Returns the stage that `code` numbers, or `None` if no stage has that number, such as 0. pub fn from_code(code: u32) -> Option { Self::ALL.into_iter().find(|stage| stage.code() == code) } diff --git a/nvx-rs/src/guest/spawn/tests.rs b/nvx-rs/src/guest/spawn/tests.rs index cad0f0652..a5cd650d6 100644 --- a/nvx-rs/src/guest/spawn/tests.rs +++ b/nvx-rs/src/guest/spawn/tests.rs @@ -25,24 +25,6 @@ impl Command { PreparedChild::new(self.spec).expect("the command prepares") } - /// Starts the child as [`spawn`](Self::spawn) does, and returns with it a copy of its launch - /// report's writer: the report arrives only once the copy closes too, as it does when a - /// process that another thread forked while the child was spawning holds the writer, for - /// focused tests. - pub(crate) fn spawn_holding_report(self) -> io::Result<(Spawned, OwnedFd)> { - let child = PreparedChild::new(self.spec)?; - let writer = child.report_writer_copy()?; - let started = child.spawn()?; - let spawned = Spawned { - process: started.process, - report: LaunchReport { - reader: started.report, - }, - barrier: started.barrier.map(|barrier| Barrier { writer: barrier }), - }; - Ok((spawned, writer.into())) - } - /// Replaces the system calls of the child's credential stage, which then succeed, or fail /// with `failure`, without privileges. pub(crate) fn simulate_credentials(mut self, failure: Option) -> Self { diff --git a/nvx-rs/src/guest/sys/spawn/tests.rs b/nvx-rs/src/guest/sys/spawn/tests.rs index b22de6243..3dcaa5415 100644 --- a/nvx-rs/src/guest/sys/spawn/tests.rs +++ b/nvx-rs/src/guest/sys/spawn/tests.rs @@ -90,18 +90,6 @@ impl PreparedChild { self.cgroup.is_some() } - /// Returns a close-on-exec copy of the launch report's writer, which holds the report back - /// while it is open, as a process that another thread forks before the child does, for - /// focused tests. - pub(crate) fn report_writer_copy(&self) -> Result { - let copy = nix::fcntl::fcntl( - self.report_writer.borrowed(), - FcntlArg::F_DUPFD_CLOEXEC(FIRST_NONSTANDARD), - )?; - // SAFETY: `F_DUPFD_CLOEXEC` returned a new descriptor, which nothing else owns. - Ok(unsafe { FileDescriptor::from_raw(copy) }) - } - /// Returns whether the child waits at a barrier, enters an execution, and leads a new /// session, for focused launch tests. pub(crate) fn barrier_entry_and_session(&self) -> (bool, bool, bool) { diff --git a/nvx-rs/src/lib.rs b/nvx-rs/src/lib.rs index c4d2ce295..92de26cc4 100644 --- a/nvx-rs/src/lib.rs +++ b/nvx-rs/src/lib.rs @@ -17,11 +17,6 @@ //! - [`contract`]: the control-contract versions that this crate speaks and the compatibility of //! NVX releases with them, and the versions of the contracts through which OpenVMM restores a //! snapshot. -//! - [`exec`]: the credit and stdio engine of an execution, which the side that runs a workload -//! and the side that requested it share with any adapter, whatever its framing: the state of -//! each stream, the credit that bounds the bytes in flight, the order of each stream's data and -//! end, and the execution's outcome and settlement. It needs only the standard library, so -//! every configuration has it. //! - [`net`]: IP networks in CIDR notation and their subtraction, and the forms of a VM's //! hostname and guest network address. //! - [`gpt`]: GUID Partition Table disks whose first partition configures an adapter and whose @@ -44,9 +39,9 @@ //! that it names (`guest::process`), places, kills, and settles the processes of its //! executions in cgroups (`guest::cgroup`), describes host mappings and keeps the account of a //! host's mapping table (`guest::mappings`), and drives each program as one execution, with its -//! launch, bounded output, input, ending, and outcome (`guest::exec`), alone or under the -//! execution engine (`guest::exec::binding`), the typed calls through which it runs the time -//! ABI's steps (`guest::time`) and reseeds after a restore (`guest::reseed`) in its own process, +//! launch, bounded output, input, ending, and outcome (`guest::exec`), the typed calls through +//! which it runs the time ABI's steps (`guest::time`) and reseeds after a restore +//! (`guest::reseed`) in its own process, //! and the preparation of a template's guest for a capture before its root's assembly, and the //! repair of its clones (`guest::clone`). The crate's `nvx-*` guest binaries run the programs. //! NVX's own guest space, which assembles a layered root, reads GPT disks, mounts host mappings, @@ -144,7 +139,6 @@ pub mod activation; pub mod contract; mod error; -pub mod exec; pub mod gpt; mod hypervisor; pub mod net;