Repository navigation
Move to Hyperlight 0.18's virtqueue transport - #142
Merged
Merged
Conversation
danbugs
added
the
ci/arm64
danbugs
marked this pull request as ready for review
October 7, 2026 20:51
danbugs
changed the title
There was a problem hiding this comment.
🔵 Needs a closer look
The transport and snapshot migration needs cross-platform human validation, and output finalization remains unresolved.
1 open finding
What changed in this PR
Migrates hluk’s host–guest communication to Hyperlight’s virtqueue transport, preparing for v0.18.
Changes:
- Configures transport pools and increases hostfs transfers to 64 KiB.
- Adds byte-based console I/O and regression tests.
- Updates snapshot compatibility, dependency pins, and protocol documentation.
| File | Description |
|---|---|
| tests/step.rs | Updates transport terminology in comments. |
| tests/quickjs.rs | Adjusts oversized-result test boundaries. |
| tests/python.rs | Tests console bytes, stdin, and path limits. |
| src/lib.rs | Migrates transport, console handling, and encoding tests. |
| src/hostfs.rs | Increases transfer chunks to 64 KiB. |
| examples/python/fs_large.py | Exercises larger transfer chunks. |
| drivers/hl_env.h | Updates buffer-sizing comments. |
| drivers/hl_driver.h | Documents transport-derived limits. |
| docs/RELEASE.md | Updates snapshot compatibility guidance. |
| docs/net.md | Documents revised network transfer limits. |
| docs/fs.md | Documents chunk sizes and path errors. |
| docs/driver.md | Updates driver protocol limits. |
| docs/calls.md | Explains virtqueues and performance measurements. |
| demos/pptx-gen/Cargo.lock | Refreshes demo dependencies. |
| CHANGELOG.md | Records migration and compatibility changes. |
| Cargo.toml | Pins temporary Hyperlight fork dependencies. |
| Cargo.lock | Locks updated dependencies. |
| build.rs | Bumps the snapshot contract. |
| .gitmodules | Selects the temporary virtqueue kernel fork. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Hyperlight 0.18 replaces the PEB's I/O stacks with two VIRTIO packed rings in scratch memory. hyperlight-host and hyperlight-common move to 0.18, and the kernel moves with it, to unikraft's plat-hyperlight-v2 with the virtqueue transport; both kernels and the native test fixture are rebuilt from it. Size the transport pools for 64 KiB transfers instead of the I/O stacks. Each half of the guest-to-host pool, a host call's request and its reply, holds 64 KiB plus framing, rounded up to 18 pages, so HOST_CALL_MAX is now what the guest computes from them, 69,620 bytes; the host-to-guest pool holds a guest function call of 73,716. The quickjs result-size test follows the new limits. Console output comes in through HostWrite(Vec<u8>) and HostWriteErr(Vec<u8>) instead of the strings HostPrint and HostPrintErr, which failed on output that was not UTF-8. Each stream is decoded on its own as it arrives, a character split between writes held until its end comes and bytes that are not UTF-8 read as U+FFFD; standard output goes to stdout and drain_output(), standard error to stderr, drain_output() and drain_stderr(), as before, under output_limit and echo_output. A character still cut short reads as U+FFFD when the process exits or a call poisons the sandbox. A host without HostWriteErr still gets standard error through HostWrite (SandboxBuilder::without_host_write_err, for testing), and HostPrint, which this kernel no longer calls, is taken as standard output. ReadStdin hands over bytes, not a lossy string. MultiUseSandbox is now Sandbox. SNAPSHOT_CONTRACT goes to 5: the host transport changed, and snapshots from earlier builds are refused. Signed-off-by: danbugs <danilochiarlone@gmail.com>
The guest reads and writes host files in chunks of the size GetHostFsChunkSize returns, 32 KiB. Over the virtqueues a host call carries 69,620 bytes, and a 64 KiB chunk reads and writes about 40% faster (Python, 64 MiB file, KVM); 256 KiB gains about 15% more for 0.8 MiB more pool per sandbox, and 1 MiB nothing. Return 64 KiB, checked at compile time against what one host call carries. fs_large.py keeps writing three full chunks. Signed-off-by: danbugs <danilochiarlone@gmail.com>
Each of these failed before the kernel's last changes: - output that was not UTF-8 failed the guest function call; - stdin cut inside a character, or not UTF-8, arrived as U+FFFD; - a host-mount path that was not UTF-8 failed the guest function call, and one over 1,023 bytes was cut short and acted on. A line longer than one host call is checked to arrive whole, as it does since the kernel sends output in as many calls as it takes. The long-path test runs on Linux only: the host directories it makes are past the path limits of macOS and Windows. Signed-off-by: danbugs <danilochiarlone@gmail.com>
calls.md gets a section on the transport and the knobs measured for it; the limits in calls.md, driver.md and net.md and the chunk size in fs.md follow the new figures, and fs.md says which paths are refused. RELEASE.md names the pool sizes where it named the I/O stacks. The CHANGELOG records the move and what it fixes. Signed-off-by: danbugs <danilochiarlone@gmail.com>
danbugs
changed the title
danbugs
force-pushed
the
hyperlight-v0.18-virtio
branch
from
October 15, 2026 16:13
d4e1c96 to
1ed4085
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moves hluk and its kernel from the PEB I/O stacks to Hyperlight 0.18's virtqueue transport.
hyperlight-hostandhyperlight-commonare 0.18 from crates.io, and the kernel submodule isunikraft/unikraft'splat-hyperlight-v2at3c363e9d, where the guest side of the transport has landed. Rebased onmainafter #145 and #146, so standard error stays a stream of its own and output stays withinoutput_limit.HostWrite/HostWriteErrreplace the unreleasedHostPrint/HostPrintErr, so output that isn't UTF-8 no longer fails the call. Each stream is decoded on its own, with a character split between writes joined, underoutput_limitandecho_output. A host withoutHostWriteErrstill gets standard error throughHostWrite.EILSEQ/ENAMETOOLONG.SNAPSHOT_CONTRACT5); native kernels must be rebuilt.HostPrintErrandwithout_host_print_errfrom #145 are renamed toHostWriteErrandwithout_host_write_err; neither has been released. #145's "writes cut into 4 KiB host calls" CHANGELOG entry is dropped, since the byte transport replaces that behaviour before it ever shipped.