Repository navigation
Add bounded host-backed storage for broker files - #1770
Open
Weiteng Chen (CvvT) wants to merge 12 commits into
Open
Weiteng Chen (CvvT) wants to merge 12 commits into
Weiteng Chen (CvvT) wants to merge 12 commits into
Conversation
Weiteng Chen (CvvT)
marked this pull request as draft
November 1, 2026 09:07
Weiteng Chen (CvvT)
force-pushed
the
weiteng/broker-file-storage
branch
from
November 2, 2026 18:05
79df568 to
1a42125
Compare
Prepare lazy shared-file storage and Linux/Windows host providers without exposing a file-mapping protocol endpoint or changing guest mmap behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Grow shared file memory on writes like pwrite, reporting short writes, and report exhausted storage as NoSpace rather than an I/O error. Document that each storage call is atomic so backends access it without holding their own locks, centralize storage error conversions, and return the descriptor's writability with the shared file. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Only sections derived from the handle reach runners, and their protection and the rights of the delivered section handle already enforce read-only access, so the read-only and writable duplicates cost two handles per file for nothing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
On Linux, make a reader's read-only reopen of a memfd for each export and close it once delivered, so each file in host shared memory holds one descriptor. HostHandle may now own the handle it exports. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add BrokerCoreLimits::max_shared_files, which the userland broker applies to its in-memory file system, so the limit is set with the broker's other limits. Zero keeps every file in broker memory. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Windows refuses to shrink a file while any section of it exists, and runners hold sections for as long as they map the file, so truncating a mapped file failed with EBUSY. Track the file's size in the broker instead: a shrink the host refuses zeroes the bytes past the new end and lowers the size only, and a later resize shrinks the host file once nothing maps it. Bytes past the end that runners write through their mappings are zeroed before the file grows over them. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The code says what it does: map the layer that holds the file, as reads do. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A handle that shared file memory exports is now a boxed trait object that knows its own raw value, whether the memory lends one of its own handles or makes one for the export, as BorrowedFd and OwnedFd do. This drops the separate raw value, the optional owner, and the two unsafe constructors. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move the host-file storage and the Linux and Windows modules of shared_objects.rs into shared_objects/, unchanged, as the runner module is laid out. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
HostFileMemory keeps the shared read and write logic in host_file.rs and calls methods that linux.rs and windows.rs each implement on it: the size, resizing, bookkeeping around writes, and exporting handles. Windows's logical size and the error codes of each host move to its module. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove tests that restate error-mapping match tables, drop mapped-file tests covered elsewhere, and fold the append/truncate-on-open checks into the coherence test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Match once on in-memory file contents instead of checking for shared memory and then reaching inline bytes that cannot be shared. - Loop over direct reads and writes, staging a chunk only when the rest of a buffer cannot take a direct transfer; direct read_at and write_at take a buffer offset for that. - Stop zeroing bytes past the end of mapped Windows files on growth; a refused shrink still zeroes the tail it cuts off. - Drop the memfd mode change and the F_SEAL_SEAL seal: runners' seccomp filter allows neither open, fchmod, nor F_ADD_SEALS. - Restore the empty-write check for unmapped files, and drop the static backing data test, which the contents enum already guarantees. - Trim shared file storage docs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Weiteng Chen (CvvT)
force-pushed
the
weiteng/broker-file-storage
branch
from
November 3, 2026 22:17
43bb28b to
8222a0d
Compare
Weiteng Chen (CvvT)
marked this pull request as ready for review
November 3, 2026 22:54
|
🤖 SemverChecks 🤖 Click for details |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds storage that lets a file's contents live in host shared memory, so that later PRs can map broker files into runners with
MAP_SHARED. In-mem fs supports this by moving a file's contents into aSharedFileMemorythe first time it's called and returns the host object as aSharedFile. From then on, reads, writes and truncations all use that storage, and writes behave likepwrite.The userland broker provides the storage on each host. Linux uses a memfd while Windows uses a delete-on-close temporary file. Windows can't shrink a file while it's mapped, so the broker tracks the file's size itself, zeroes the bytes a refused shrink cuts off, and shrinks the host file once nothing maps it.
One limitation is that we need one host descriptor or handle per shared file and the total number of host fd is limited. Additionally, because we don't know when the shared file is no longer needed, we never close it.