Move to Hyperlight 0.18's virtqueue transport

CHANGELOG.md

@@ -9,22 +9,25 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 - The `java` runtime (tier 2): Java 25 on a jlinked runtime, with Java source run by JShell in the guest's JVM, so statements, expressions and declarations mix and stay for the next call, and a call that only declares a `main` (a compact source file, or a class) runs it. It serves guest function calls (a static method, its input converted to a record, collection or simple value, a future awaited) and makes host function calls through `hyperlight.Host.call`; `System.getenv()` sees the host's variables, and `System.exit()` ends the call with its status; the next call starts a fresh JVM while the guest has the memory for one (two restarts at 512 MiB: the kernel keeps an exited process's memory), and fails without crashing when it doesn't. The image is 84 MiB and runs in 512 MiB; a CDS archive of the classes the dispatcher loads while it warms up comes with it. `hluk init --template java` starts a project.
 - A `dev` channel: each push to `main` whose CI passes replaces the `dev` pre-release (the `hluk` binaries and a `SHA256SUMS`) and publishes the images of that commit (`:dev-<sha7>`, `:initrd-dev-<sha7>`), so `main` can be installed and run between releases: `HLUK_VERSION=dev` with `install.sh` or `install.ps1`. A dev `hluk` pulls the images of its own commit and prints its build with `--version` (`0.17.0+dev.44ee170`); only the last two builds' images are kept. Releases and their tags are unchanged (`dev.yml`, docs/RELEASE.md).
 - A project site, <https://hyperlight-dev.github.io/hyperlight-unikraft>: startup times from CI, a density measurement (1,000 Python sandboxes in one process, about 2.8 MiB of host memory each), every template, and terminal recordings of `hluk init`, `hluk build` and a guest function call. Its source is `site/`, and `site.yml` publishes it to the `gh-pages` branch on pushes to `main`, beside the CI benchmark history. `just site-serve` previews it, `just site-record` re-records the demos and `site/tools/density.sh` repeats the density measurement.
-- `AppSandbox::drain_stderr`, the guest's standard error apart from its standard output, and the host function behind it, `HostPrintErr` (see Changed).
+- `AppSandbox::drain_stderr`, the guest's standard error apart from its standard output, and the host function behind it, `HostWriteErr` (see Changed).
 - `SandboxBuilder::output_limit(bytes)`: a bound, enforced on the host, on the guest output a sandbox keeps between drains, for its output (`drain_output`, both streams) and its standard error (`drain_stderr`) each. The host keeps the beginning and counts the rest, which `AppSandbox::take_output` and `take_stderr` report with the text (`CapturedOutput`); the guest carries on. Unlimited by default. `SandboxBuilder::echo_output(false)` stops the copy to the host process's stdout and stderr, so the capture is all there is.
 
 ### Changed
 
-- A guest's standard error is a stream of its own, through a new host function, `HostPrintErr`: `hluk run` writes it to its own stderr instead of its stdout, and `AppSandbox::drain_stderr` returns it apart from standard output. `drain_output` still returns both, in the order written, and drops what `drain_stderr` has not taken, so take standard error first. The kernel's own messages go to standard error too, out of the program's standard output, and so does the report of `hluk run --profile`, which shares stderr with them. A host of your own that does not register `HostPrintErr` gets standard error through `HostPrint`, as before.
+- A guest's standard error is a stream of its own, through a new host function, `HostWriteErr`: `hluk run` writes it to its own stderr instead of its stdout, and `AppSandbox::drain_stderr` returns it apart from standard output. `drain_output` still returns both, in the order written, and drops what `drain_stderr` has not taken, so take standard error first. The kernel's own messages go to standard error too, out of the program's standard output, and so does the report of `hluk run --profile`, which shares stderr with them. A host of your own that does not register `HostWriteErr` gets standard error through `HostWrite`.
 - Guest output keeps its newlines: `drain_output`, `drain_stderr` and `hluk run` no longer have a `\r` before each `\n`.
-- Snapshots saved by an earlier release are refused (the kernel and the host functions changed); save them again.
 - The Linux release binaries are static, built with musl (`hluk-v<version>-x86_64-unknown-linux-musl.tar.gz` and `aarch64-unknown-linux-musl`), so they run on any distribution. The glibc builds of earlier releases need the glibc of the runner that built them, 2.39, which Debian 12 and Ubuntu 22.04 do not have (#141). `install.sh` installs the static one and, for a release before this, the glibc one.
+- Hyperlight's virtqueue transport (hyperlight v0.18): the kernel and the host talk over two VIRTIO packed rings in scratch memory instead of the PEB's I/O stacks. Host function calls are a request and its reply in one G2H chain; guest function calls arrive in buffers the kernel posts on H2G, and their results go back on G2H. Byte vectors travel after the FlatBuffer rather than inside it. The pools are sized for 64 KiB host-call payloads, rounded up to whole pages: one host call now carries 69,620 bytes, and a guest function call 73,716. hostfs reads and writes move 64 KiB per host call instead of 32 KiB: against v0.17, reading a 64 MiB file from Python is about 55% faster and writing one about 50% (KVM). `hyperlight-host` and `hyperlight-common` move to 0.18.
+- Snapshots saved by an earlier release or build are refused (the kernel, the host functions and the host transport changed); save them again. A native Unikraft kernel (`--kernel`, `SandboxBuilder::from_kernel`) built against an earlier `plat/hyperlight` no longer boots: rebuild it against this kernel tree.
 
 ### Fixed
 
 - `hluk run`, `hluk snapshot` and `hluk bench` kept everything a guest printed in memory until they exited, though they never read it: a long-running guest that kept printing grew the host process without end. They keep none of it now; the output still reaches the terminal.
+- A path below a host mount's root longer than 1,023 bytes was cut short, and the syscall made on what was left: `mkdir` reported success and created a directory with a truncated name. It now fails with `ENAMETOOLONG`.
+- A path on a host mount that was not UTF-8 (`os.stat(b'/mnt/data/\xff')`) failed the guest function call that used it, since the host takes paths as strings: the kernel now refuses it, and the syscall fails with `EILSEQ`.
+- Console output written in one piece longer than 4 KiB (a long line, say) reached the host cut to its first 4 KiB, with nothing to say so; and output that was not UTF-8, or a write that ended inside a character (`cat` of a large UTF-8 file can do it), failed the guest function call that printed it. The kernel now sends console output as bytes, in as many host calls as it takes, through new `HostWrite` and `HostWriteErr` host functions, and the host decodes them, a character split between writes included, with invalid bytes as U+FFFD, for stdout and `drain_output()` alike. Stdin, the other way, had the same flaw: a character cut at a 4 KiB read, or bytes that were not UTF-8, reached the guest as U+FFFD. `ReadStdin` now hands over bytes too (the guest's terminal layer still treats them as a terminal does: it echoes them, and reads Ctrl-D as end of input).
 - `hluk init NAME --template go` wrote a `go.mod` that Go refused when the name was `go`, `toolchain` or a standard-library package such as `fmt`: the module path is now `example.com/NAME`.
 - A condition variable broadcast, or several signals, woke one waiter and left the others asleep in programs linked with musl, which most images use (node, dotnet-jit, java, quickjs, bash, and programs built on Alpine): the kernel refused `FUTEX_REQUEUE`, which musl's condition variables use to hand each woken waiter's successor to the mutex. It showed as rare, timing-dependent hangs, such as Java's `HttpClient` stalling on its first request on Windows hosts. `FUTEX_CMP_REQUEUE` also woke a waiter when asked to wake none, looped when requeuing onto the same futex, and crashed the kernel when a requeued waiter timed out. All four are generic Unikraft fixes, each reproduced and verified on QEMU/KVM.
-- A program's write of more than 4 KiB at once reached the host cut to its first 4 KiB, while the program was told all of it was written: `print("x" * 10000)` printed 4,096 bytes. The kernel sends such a write in host function calls of at most 4 KiB each, cut between UTF-8 characters, never inside one.
 - Standard error reached the host as standard output: the kernel opened fd 2 as a copy of fd 1. It is a stream of its own now (see Changed).
 - Every newline a guest printed reached the host as CR-LF (`\r\n`), from a workaround for serial consoles that need a carriage return. The host gets the bytes the program wrote (see Changed).
 - A command that failed to execute (a mistyped command in the bash image, `subprocess.run(['/missing'])` in python-shell) crashed the guest the next time the parent waited: the vfork child freed its parent's stack on its way out, as if it were its own. A subprocess started from a thread other than the main one failed with `ENOMEM`, for want of an allocator for the new program's stack. Both are generic Unikraft fixes, verified on QEMU/KVM.

Cargo.toml

@@ -34,7 +34,7 @@ cli = [
 mshv = ["hyperlight-host/mshv3"]
 
 [dependencies]
-hyperlight-common = "0.17"
+hyperlight-common = "0.18"
 serde_json = "1"
 thiserror = "2"
 tracing = "0.1"
@@ -76,16 +76,19 @@ sha2 = { version = "0.11", optional = true }
 #   Windows — WHP, compiled in unconditionally for the target (no feature).
 #   macOS   — Hypervisor.framework via the default `hvf` feature.
 [target.'cfg(target_os = "linux")'.dependencies]
-hyperlight-host = { version = "0.17", default-features = false, features = ["kvm", "build-metadata"] }
+hyperlight-host = { version = "0.18", default-features = false, features = ["kvm", "build-metadata"] }
 
 # Non-Linux keeps hyperlight's defaults: HVF (macOS) is on, WHP (Windows) is
 # unconditional, and the Linux-only kvm/mshv3 features are inert here.
 [target.'cfg(not(target_os = "linux"))'.dependencies]
-hyperlight-host = "0.17"
+hyperlight-host = "0.18"
 
 [target.'cfg(windows)'.dependencies]
 # Private working set for the bench RSS line (already in the tree via clap).
 windows-sys = { version = "0.61", features = ["Win32_System_ProcessStatus", "Win32_System_Threading"] }
 
 [dev-dependencies]
 tempfile = "3"
+# The FlatBuffer decoder's external-value source, mirrored in the tests.
+anyhow = "1"
+bytes = "1"

build.rs

@@ -11,11 +11,11 @@ use std::path::{Path, PathBuf};
 use sha2::{Digest, Sha256};
 
 /// The host side of the guest contract: the host functions and their
-/// meaning, the I/O stack sizes, the heap and scratch layout, the guest
-/// MSR list, and the hyperlight-host release that reads a snapshot.  Bump
-/// it when any of those changes in a way a snapshot would notice; a
+/// meaning, the virtqueue pool sizes, the heap and scratch layout, the
+/// guest MSR list, and the hyperlight-host release that reads a snapshot.
+/// Bump it when any of those changes in a way a snapshot would notice; a
 /// kernel change rolls the key by itself.
-const SNAPSHOT_CONTRACT: u32 = 4;
+const SNAPSHOT_CONTRACT: u32 = 5;
 
 /// The embedded kernel, whose code and host-call protocol a snapshot
 /// carries in its memory: one per architecture, named as Unikraft names

defconfig-elfloader

@@ -87,7 +87,7 @@ CONFIG_LIBHOSTSOCK=y
 # can read host-provided input via sys.stdin, process.stdin, etc.
 CONFIG_LIBPOSIX_TTY_STDIN_SERIAL=y
 CONFIG_LIBPOSIX_TTY_SERIAL_EOF_FINAL=y
-# stderr is a stream of its own (HostPrintErr), and newlines reach the
+# stderr is a stream of its own (HostWriteErr), and newlines reach the
 # host as the program wrote them, not as CR-LF.  Pinned here on purpose,
 # though the kernel defaults them so for Hyperlight.
 CONFIG_LIBPOSIX_TTY_SERIAL_STDERR=y