Repository navigation
nfs: spill AppleDouble companion bodies to a per-mount scratch area (lp73) - #714
Merged
21 commits merged intoNov 17, 2026
Merged
21 commits merged into
21 commits merged into
Conversation
….rs (lp73) Relocation only, no behavior change: `Kernel::release`, `Kernel::held`, `retained_count` and the `Released` struct move out of `kernel.rs` into `kernel/released.rs` verbatim, and `pub use released::Released` keeps every existing path working. This is the first commit of the AppleDouble split (LP636): `kernel.rs` would otherwise cross the 999-line ceiling once the companion work lands. No test or documentation change; the moved functions are the same text in a new module. Production LOC: 163505 -> 163511 (delta +6) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 8cfb6e2 (163505) and this tree (163511). The +6 is the new module's own two `use` lines, its `impl Kernel` block and the `mod released;` / `pub use released::Released;` declaration in kernel.rs. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
The client lays a `._<name>` companion out as an AppleDouble v2 image: a header, a FinderInfo entry that runs into an `ATTR` table of extended attributes, and a resource-fork entry at the end. `apple_double_codec` reads those fields (`Image::parse`) and, when the fields encode back to the very bytes the client wrote (`Image::compact`), keeps them as its own record, so no attribute, value or byte the layout carries is dropped, and one attribute's alignment padding survives a shorter name replacing a longer one. `Image::new`/`Image::encode` lay an image out as the client does, which is what makes the round trip exact and what the macOS evidence compares against. Malformed, truncated or differently laid-out images, and a resource fork with data, parse to `None` and are left as the client wrote them. This is the first layer of the AppleDouble split (LP636): the codec alone, with its own unit tests; the companion that holds its decoded form comes next. Tests: new `tests/apple_double_codec.rs`, 6 cases (byte-exact round trip for every value size, many attributes, alignment padding, whole units, a resource fork, and malformed images that must not decode), all pass. Production LOC: 163511 -> 163792 (delta +281) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 1810b4e (163511) and this tree (163792). Engine +281 (`apple_double_codec.rs` +280, `lib.rs` +1). Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
A macOS client writes each companion once, in full, and reads it back unchanged. A body is therefore classified whenever a write or a resize leaves it: one that parses back to the very bytes the companion holds (`Image::compact`) and fits `DECODED_ENTRY_BYTES` is kept as its FinderInfo and attributes and counted against `DECODED_BYTES` instead of its image, and every READ synthesizes the same bytes again. Anything else - a resource fork, another layout, a half-written image - is stored as the client wrote it. This replaces `TOTAL_BYTES` with two budgets that say what they bound: `RESIDENT_BYTES` (32 MiB) of written bodies kept in memory, and `DECODED_BYTES` (512 MiB) of decoded ones counted as `Image::cost`. A write or resize that neither decodes nor fits the resident budget answers `NFS3ERR_NOSPC` and leaves the companion exactly as it was; the next layer spills those bodies to disk instead of refusing them. `Released` now reports `companions_decoded` and `companion_decoded_bytes`, and `companions_release`/`companions_held` answer the `Holding` record the counters live in. Second layer of the AppleDouble split (LP636), on top of the codec. Tests: new `tests/apple_double_decode.rs` with the client's own write sequence (a half-written header, the rest, an edit that breaks the image and one that completes it again, truncate below and resize back up), read back byte for byte. Full `layerfs-nfs` suite: 41 passed, 0 failed. Production LOC: 163792 -> 163886 (delta +94) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 210d5f8 (163792) and this tree (163886). Engine +94. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
The NFS route guide now names `apple_double_codec.rs` and states what each bound actually holds: `RESIDENT_BYTES` for written bodies and `DECODED_BYTES` / `DECODED_ENTRY_BYTES` for the decoded form, replacing the `TOTAL_BYTES` row the code no longer has. The paragraph after the table says which bodies decode (complete images that encode back to the same bytes) and which stay as written. Documentation only; it describes the codec layer as it stands. Production LOC: 163886 -> 163886 (delta 0) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent fa5dd34 (163886) and this tree (163886). Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
…get (lp73) A client's companion body no longer has to fit in memory and no longer takes memory from every other export. Bodies beyond `RESIDENT_BYTES` (32 MiB) are spilled, least recently touched first, to a scratch directory of the export's own (`<mounts>/.appledouble/<namespace>/<serial>`), and a write that still does not fit the budget after bounded eviction rounds is written out itself and edited in its file, which needs no budget at all. - `apple_double_spill.rs` owns the scratch files: one per companion serial, directory `0700` and files `0600` whatever the umask, written through a `<serial>.<n>.tmp` that only the adopting call renames to `<serial>`, no fsync and no space reserved ahead. A spilled body is opened by path per `pread`/`pwrite`/`set_len` and never held open, so companions far outnumbering the process's file limit all work. A failed file operation is answered with the NFS status its errno means (`ENOSPC`/`EDQUOT`, `EFBIG`, `EACCES`, `EPERM`, `EROFS`, else `NFS3ERR_IO`). - `apple_double/edit.rs` holds READ, WRITE and SETATTR and the eviction: one edit of a companion at a time under that companion's own gate, so a truncate and a write can never leave a length its file does not have, and the file I/O never holds the companion lock. - `ExportConfig` gains `companion_spill_dir` and `companion_resident_bytes`. The daemon gives each export `<mounts>/.appledouble/<namespace>`, removes it at startup before any Attach, at release and when a failed Attach unwinds, and touches nothing else under the mount home. - `Released` reports `companion_bytes`. Third layer of the AppleDouble split (LP636), on top of the codec; `HELD` and the name index are the layer after this one. Production LOC: 163886 -> 164334 (delta +448) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 584cc3e (163886) and this tree (164334). Engine +448, the daemon's `export.rs` share included. Tests for the layer follow in the next commit. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
The rig can now set the resident budget and an explicit spill directory, and the cases exercise the layer end to end: - a body smaller than the budget stays resident and a larger one spills; the spilled file carries the bytes and the length the client wrote; - an edit that leaves a decoded image not whole spills it instead of growing memory, while a truncate of a decoded body needs no room at all; - a spill only replaces a companion no other call changed meanwhile (`spill_applies`); - a truncate and a write of one spilled companion start together, round after round, and the length a reply reports is always the length its file holds; - companions four times the process's own `RLIMIT_NOFILE` all read and write, because a spilled body is opened by path per operation: that case lowers a process-wide limit and counts the process's descriptors, so it re-runs itself in its own process (`run_alone`); - Attach cleans a leftover `.appledouble` of a daemon that died, release and unmount remove it, and startup reclamation removes it while keeping every other entry under the mount home. `libc` is a new dev-dependency of `layerfs-nfs`, only for the file-limit case. Tests: `cargo test -p layerfs-nfs` 46 passed, 0 failed (the raw sum reads 47 because the file-limit case runs itself once in a child process). Production LOC: 164334 -> 164334 (delta 0) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 7b38dad (164334) and this tree (164334). Tests only. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
The guide names `edit.rs` and `apple_double_spill.rs`, describes what the `RESIDENT_BYTES` row does now (spill to `companion_spill_dir`, and which status each host errno maps to), and records the scratch file's own rules: one file per companion serial under `<mounts>/.appledouble/<namespace>`, `0700`/`0600`, written as a temporary and adopted by rename only when no call changed the body, opened by path per operation and never held open, deleted by startup reclamation and by release. `ExportConfig` lists the two new fields, and the `wcc_data` row now says a companion mutation answers from `directory_stats`. Documentation only; it describes the spill layer as it stands. Production LOC: 164334 -> 164334 (delta 0) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 91434f9 (164334) and this tree (164334). Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
…s for (lp73) A decoded companion was re-encoded in full under the companion lock on every READ: up to `FILE_BYTES` (1 MiB) allocated and copied to serve a reply of at most `TRANSFER_WINDOW` (256 KiB) bytes, and often far less. `Image::encode_range` lays the image out and fills only `[start, end)`, so a READ pays for the bytes it answers with; `Image::encode` is that same layout over the whole image, so there is one implementation of the layout rather than two that can drift. `edit.rs`'s spill and `edit_resident` still need the whole image and keep calling `encode()`. Whether a READ is served from a window or from the whole image is not observable: the reply bytes are the same either way. Review: finding 7 (full re-encode on every READ). Tests: new `any_window_of_the_image_encodes_to_that_window` compares every window of an image (a padding byte the client left behind, values of several sizes, the empty resource fork) with the same slice of `encode()`, including empty windows, a window past the image and one that starts after it ends. `cargo test -p layerfs-nfs --test apple_double_codec --test apple_double_decode`: 7 + 1 passed, 0 failed. Anti-proof: with `encode_range` clamped to start at 0 again the window test fails (`left` starts at image offset 0 while the expected window starts mid-image). Production LOC: 163886 -> 163908 (delta +22) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 584cc3e (163886) and of the staged tree 20b10a3 (163908). Engine +22. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
…t (lp73) `DECODED_BYTES` (512 MiB per export) was memory nothing could evict: many Workspaces can be mounted at once, so a per-export budget that is never paid back does not fit. Decoded images now take the same `RESIDENT_BYTES` (32 MiB) as bodies kept as written, a body at its length and a decoded one as `Image::cost`, so the budget sees every byte the export holds. `Companion::resident_cost` is that one number; `attach`, `detach` and `edit_body` count it, `edit_body` decides with it (a body either fits or the whole edit answers `NFS3ERR_NOSPC` and leaves the companion as it was), and `classify` no longer consults a second budget. `DECODED_ENTRY_BYTES` stays, now as the most `Image::cost` one decoded companion may be counted for: a whole image costing more keeps the client's bytes, so one companion cannot take the whole pool. This layer has no disk to spill to, so a body the budget does not take is still refused; the next layer spills it. `Released` reports the merged pool as `companion_bytes` and keeps `companions_decoded` as a count, instead of `companion_decoded_bytes` beside a separate resident total. Review: owner decision 3 (`DECODED_BYTES` rejected), first half (the budget and the counts); the spill half is the next layer. Tests: `the_clients_write_sequence_is_served_byte_for_byte` now asserts the merged pool: the decoded image's `Image::cost()` after the image arrives, the written body's length while it does not decode, the cost again once it does, and zero after the companion is removed. `cargo test -p layerfs-nfs`: 42 passed, 0 failed. Anti-proof: with a decoded body counted as zero again the test fails at the first assertion (`left: 0, right: 4372`). Production LOC: 163908 -> 163896 (delta -12) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent b01e805 (163908) and of the staged tree bfc489b (163896). Engine -12. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
The layer below changed the read path, the budget and the counters; this layer had moved the read and the edit into `apple_double/edit.rs` and given the budget a spill directory. Conflicts resolved by keeping this layer's structure with the layer below's semantics: - `Image::encode_range` serves a READ of a decoded body (the read moved to `edit.rs`, so the window it now asks for is applied there); - `DECODED_BYTES` stays deleted: `Holding` and `Released` report one `resident_bytes`/`companion_bytes` pool plus the decoded count, and `keep_decoded` asks the resident budget, not a second one; - `classify` keeps this layer's "a body on disk stays as the spill left it" and drops the decoded budget check (the per-entry cap stays); - `adopt_spilled`, `oldest_resident` and this layer's edit path are unchanged here. `a_decoded_companion_is_edited_under_a_full_resident_budget` sets a 5,120-byte budget: it takes the image's attributes (4,369) but not its 8 KiB of bytes, which is what the merged budget now means. Tests: `cargo test -p layerfs-nfs`: 47 passed, 0 failed (raw `test result` sum 48: `spilled_companions_outnumber_the_open_file_limit` runs itself again with `--exact` in a child process, whose line is counted too). Production LOC: 164334 -> 164340 (delta +6) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 8183f89 (164334) and of the staged tree (first-parent merge comparison, 164340). Engine +6. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
With decoded images counted against `RESIDENT_BYTES`, a decoded body is memory like any other and has to leave memory like any other. `oldest_resident` now takes the least recently used companion whose `resident_cost` is not zero, which is a body kept as written or a decoded one, so eviction reaches decoded bodies. Writing a decoded body out needs nothing new: `spill_one` and `spill_gated` already encode it and write those bytes, which are exactly the image the client wrote. A whole image that does not fit the budget is therefore no longer refused (`NFS3ERR_NOSPC`) nor kept decoded beyond it: it goes to its spill file and is edited there, as any body does. Review: owner decision 3 (`DECODED_BYTES` rejected), second half (the spill). Tests: new `eviction_takes_the_decoded_body_before_the_written_ones` under a 4,096-byte budget: a decoded image of 213 bytes, then two 2,000-byte bodies. The decoded body is the least recently used memory, so it is the one spilled (`companions_decoded` 1 -> 0), the two written bodies stay resident (`4000` bytes), and the image reads back byte for byte from its file. Anti-proof: with `oldest_resident` filtered to bodies kept as written again, the test fails (`companions_decoded` stays 1 and the written body is evicted instead). Production LOC: 164340 -> 164337 (delta -3) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 7a9f20c (164340) and of the staged tree (164337). Engine -3. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
Eviction walked every companion to find the least recently used one, and `spill_for` repeated that walk up to `SPILL_ROUNDS` times, so one edit could scan the index under the companion lock several times. `Companions::by_use` is now a `BTreeSet<(used tick, serial)>` of the companions holding memory: `attach` inserts an entry when a body takes budget, `detach` removes it, and `set_used` (behind `use_serial`, the guest's last touch of the body) moves it. `oldest_resident` is the first entry, skipping the caller's own serial, so a round costs one step into the order (O(log n)) and no walk. `spill_for` no longer bounds its rounds by the number of companions: it takes the oldest body, and if that body changed under it the other call has already moved its entry to the newest end, so the next round looks at a different one; the rounds end when nothing holding memory is left. `SPILL_ROUNDS` goes away. The companion lock now does O(1) or O(log n) work per step with no I/O: `attach`/`detach`/`set_used` one set operation, `oldest_resident` one lookup, and the file written before the lock is taken and renamed after it is dropped. The kernel table lock is untouched by any of this. Review: blocking finding 2 (eviction is an O(n) scan under the lock, repeated). Tests: new `eviction_follows_the_order_of_use_not_the_shape_of_the_index`: 64 companions of 16 bytes under a 1,024-byte budget, three of them read again, then eight more; the spill directory holds exactly the serials of the eight least recently used ones, in that order, and the three read again stay resident. Anti-proof: with `oldest_resident` reversed the test fails, spilling `SERIALS+2, +64..+70` (the newest) instead of `SERIALS+3..+10`. Production LOC: 164337 -> 164342 (delta +5) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent e323c0e (164337) and of the staged tree (164342). Engine +5. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
…e lock (lp73) `Companions::adopt_spilled` ran `fs::rename` while holding the companion lock, so the module documentation's "file I/O never holds the companion lock" was not true of the step that puts the file in place. The adoption is now three steps with the lock dropped between them: `Companions::spill_target` (lock) answers the companion's own file and whether the body is still the one the file was written from, `fs::rename` runs outside the lock, and `Companions::commit_spilled` (lock) switches the body to `Body::Spilled` with its length. A reader keeps reading the body in memory until the rename is done, so no reader ever sees a half written or missing file. The exclusion the rename needs is the companion's own edit gate, which already means "one edit of this companion at a time": a gated edit holds it across its own adoption, and eviction now takes it with `try_lock`, skipping a companion another call is editing (it never waits for another call's I/O, so no two spills of one companion can meet at the file). `spill_for` therefore passes over a victim it could not spill instead of picking it again, and a companion that is gone is nothing to spill rather than an error. Review: finding 5 (disk I/O under the companion lock); the module docs of `edit.rs` and `apple_double_spill.rs` and `nfs.md` now state what the code does. Tests: `cargo test -p layerfs-nfs`: 48 passed, 0 failed (raw `test result` sum 49 with the child-process rerun of `spilled_companions_outnumber_the_open_file_limit`), including the spill, race and file-limit cases that adopt bodies. Production LOC: 164342 -> 164372 (delta +30) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 73f62ef (164342) and of the staged tree (164372). Engine +30. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
`companion_read` took the companion's length under the companion lock and then read the spilled file outside it with `read_exact_at`. A truncation landing between the two made the file shorter than the window, and `read_exact_at` turned that into `NFS3ERR_IO` for a read of bytes the client had already been told were there. One `read_at` of that window is the whole reply: the bytes the file still holds, however many, are a short read at the truncation point, and `eof` still follows the length the lock saw. No retry, no second read. Review: finding 6 (spurious `NFS3ERR_IO` from READ). Tests: new `a_read_racing_a_truncation_is_a_short_read_not_an_io_error`, 512 rounds of one client reading a 64 KiB spilled body while another truncates it to zero: every reply is the body or a prefix of it, and the test client panics on any error status. Anti-proof: with `read_exact_at` back the test fails at round 0 (`panicked at tests/support/client.rs:248: self.read(...).unwrap()`). Production LOC: 164372 -> 164374 (delta +2) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 0cbd34c (164372) and of the staged tree (164374). Engine +2. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
…lp73) Owner decision 2 approved `<mounts>/.appledouble/` on condition that the architecture and the cleanup docs name it a new disposable area. nfs.md and the code already described it; the two remaining places did not. `docs/architecture/engine/daemon/README.md`: the "Starts NFS serving" row of the startup table now says the same reclaim that force-unmounts leftovers deletes `<mounts>/.appledouble` (directories `0700`, files `0600`, no fsync, no space reserved ahead) and touches nothing else under the mount home. `docs/general/cleanup.md`: the classification table gains the scratch area as a disposable per-mount resource, with what it is and who deletes it. The test that says reclaim removes only `.appledouble` is still here: `startup_reclaim_clears_leftover_companion_spill_directories` puts a `kept/` file beside a leftover `.appledouble/7/stale`, and asserts the first survives and the second does not. Review: owner decision 2, condition "document it in the kernel routes, NFS and cleanup docs, and keep the reclaim test". Tests: `cargo test -p layerfs-nfs` at this commit: 49 passed, 0 failed (raw `test result` sum 50 with the child-process rerun of `spilled_companions_outnumber_the_open_file_limit`), including that daemon test. Production LOC: 164374 -> 164374 (delta 0) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 485dd2f (164374) and of the staged tree (164374, engine 75,739 in both). Docs and tests only. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
…p73) `cargo fmt --check` reported three files of this layer: the `emit` call rustfmt wraps in `Image::encode_range`, and two assertions of the new codec and decode tests. This is that pass and nothing else. The one remaining fmt report, `api/crates/layerfs-cli/tests/concurrent_reads.rs`, is not touched by this branch (`git log 8cfb6e2..HEAD -- <file>` is empty) and is left alone. Checks: `cargo +1.85.1 fmt --all -- --check` reports only that file; `cargo +1.85.1 clippy --locked -p layerfs-nfs --all-targets -- -D warnings` is clean; `cargo test -p layerfs-nfs` passes every binary. Production LOC: 163896 -> 163901 (delta +5) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 7ef50a0 (163896) and of the staged tree (163901). Engine +5: rustfmt splits the `emit` call, which the counter counts. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
The layer below made its own files fmt- and clippy-clean. Only `tests/apple_double_decode.rs` conflicted: this layer inserts a test where the layer below removes a blank line. The test stays and the file was formatted again, so the result is the same pass applied to this layer's file. Checks: `cargo +1.85.1 fmt --all -- --check` reports only `api/crates/layerfs-cli/tests/concurrent_reads.rs` (not this branch); `cargo +1.85.1 clippy --locked -p layerfs-nfs --all-targets -- -D warnings` is clean; `cargo test -p layerfs-nfs` passes every binary. Production LOC: 164374 -> 164379 (delta +5) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent e3e46ae (164374) and of the staged tree (164379). Engine +5 from the layer below. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
… (lp73) `spill_for` passed a victim's error straight out with `?`. A victim another call had just removed answered `NFS3ERR_STALE`, and one whose spill file could not be written answered `NFS3ERR_NOSPC`/`NFS3ERR_IO`, so an unrelated WRITE failed on a companion it never named while its own body was still in memory. A victim that leaves nothing to give back, that another call changed under its file, that its own edit holds, or that cannot be written out is now passed over and the round looks at the body after it. The writer only answers its own error, from spilling its own body, which needs no budget; a writer that finds no room and cannot write out its own body still reports that. Review: finding P2-2 (an eviction victim's error reaching an unrelated writer). The report claimed this was already skipped; the code and the report now agree. Tests: new `an_eviction_victim_that_cannot_be_written_out_does_not_fail_the_writer`. Two resident bodies of 1,000 bytes in a 2,000-byte budget, the oldest one's spill path made a non-empty directory so its spill cannot be adopted, then a third 1,000-byte write: it lands, the oldest body stays in memory byte-identical, the second is on disk and reads back, and only that one file is in the spill directory. Anti-proof: with `?` back the third write answers `NFS3ERR_IO` (`Err value: 5`) and the test fails on it. Production LOC: 164379 -> 164379 (delta 0) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent bdb87af (164379) and of the staged tree (164379). Engine 0: the round's condition keeps one line and the rest is comment. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
Three edges of the spill path, none of them a behaviour the client can rely on today: - A body written out whole is spilled at the length the edit wanted, and `commit_spilled` records that longer length before the edit touches the file. A `write_all_at`/`set_len` that then fails (`NFS3ERR_NOSPC`, `NFS3ERR_IO`) answered the error but left the companion longer, zero-filled: a refused edit that had changed the body's size. `apply_edit` now keeps the length the companion had and puts it back when the file operation fails, so a failed WRITE/SETATTR leaves the length as it was, as a refused slot does. - `spill_one` took the victim's edit gate with `try_lock` and read every `Err` as "busy", `Poisoned` included: a companion whose edit panicked could never be evicted, so it held its body for the export's life. A poisoned gate is now taken anyway, as every other holder of it does. - A spilled READ's one `pread` answered `NFS3ERR_IO` when a signal interrupted it, although no byte of it had moved. It is asked again in that case only; the read is still one read of one window, and a short read is still the reply. Review: findings P3-2, P3-3 and P3-5. Tests: none new. The length restore only shows on a filesystem error after a successful spill (the file is already at the wanted length, so `RLIMIT_FSIZE` and a directory at the spill path fail before it), the poisoned gate needs a panicking edit, and `EINTR` needs a signal landing inside one `pread`; all three are timing- or filesystem-dependent, and the change is a guard each. `cargo test -p layerfs-nfs`: 51 passed, 0 failed (raw sum 52 with the child-process rerun). Production LOC: 164379 -> 164405 (delta +26) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent ec17db8 (164379) and of the staged tree (164405). Engine +26: the length snapshot and restore (8), the poisoned-gate match (5), the interrupted-read loop and its helper (11), and the two rewritten error paths. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
No conflicts. Upstream since `8cfb6e273` brought the standalone fuser Git fork (Ephemeral-AI-Lab#686, which removes `engine/vendor` and `tools/check_fuser_integrity.py`) with its pin guard (Ephemeral-AI-Lab#687), and the GC spec items (Ephemeral-AI-Lab#688). None of those files is this layer's, so the merged tree is upstream's plus this branch's commits. Production LOC: 163901 -> 163901 (delta 0) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded (the removed vendored fuser is third-party code and never counted). Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 428ffc3 (163901) and of the merged tree (163901). Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
… (lp73) The codec layer now carries `upstream/layerfs_v2_0_0` at `cb8ced9d3`; that merge had no conflicts, and neither does this one. Upstream's changes since `8cfb6e273` (the standalone fuser Git fork, which removes `engine/vendor` and `tools/check_fuser_integrity.py`, its pin guard, and the GC spec items) touch no file of this layer, so the result is the upstream tree plus this layer's commits. Production LOC: 164405 -> 164405 (delta 0) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded (the removed vendored fuser is third-party code and never counted). Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent aec83af (164405) and of the merged tree (164405). Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
This was referenced Nov 16, 2026
Closed
5742338
jiangkoumo
pushed a commit
to jiangkoumo/layerfs
that referenced
this pull request
Nov 17, 2026
…er-mount scratch area (lp73) Merges the PR head e3c80ed unchanged; review fixes follow in their own commits. Production LOC: 170247 -> 170751 (delta +504) core 39,795 -> 39,795 (0) engine 75,670 -> 76,174 (+504) layerfs-daemon +15, layerfs-nfs +489 control 48,436 -> 48,436 (0) api 6,346 -> 6,346 (0) reference absent (retired R9c; last counted 65,417 at tree 498dd19) Method: tools/production_loc.py --detail on first parent 7c0f937 and the merged tree (merge: first-parent comparison) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jiangkoumo
pushed a commit
to jiangkoumo/layerfs
that referenced
this pull request
Nov 17, 2026
- A WRITE that grows a spilled body no longer shows its new length before its bytes. `apply_edit` wrote the body out pre-extended to the edit's length, and `commit_spilled` recorded that length before `edit.disk` wrote anything. A GETATTR or READ in between (neither takes the edit gate) saw the larger size and read zeros, and a failed edit shrank it back, so a client could see a length no WRITE made. The body is now written out at the length it has, the edit grows its file itself, and `record_edit` moves the length after the bytes are there. `restore_len` and the length it saved are gone, and with them the early `Stale` return that skipped `unreserve` after a reservation. - `spill_target` never replaces a body that is already on disk, whoever asks. Before, a spill with no `used` snapshot renamed its file onto the live one first and found out afterwards. That was unreachable while the edit gate holds, but it would have deleted the live body. - With no spill directory, `spill_for` stops at once instead of copying or encoding every resident victim under the companion lock only to fail on each. One victim whose file cannot be written is still passed over for the next (`an_eviction_victim_that_cannot_be_written_out_does_not_fail_the_writer` keeps that). - `spill_applies` was public only for a test that fed it integers: both are removed, and `spill_target` compares the tick itself. Deleted test: `write.rs::a_spill_replaces_a_companion_body_only_when_no_call_changed_it`. The behaviour it named holds: every body change runs under the edit gate, and a spill takes the gate with `try_lock` and checks the `used` tick. - `spilled_companions_outnumber_the_open_file_limit` waits for its child with a 90 s deadline and kills a child that hangs, instead of `status()`, which waits indefinitely. Docs: nfs.md, the paragraph on one edit per companion. Tests: no deterministic client-side test reproduces the GETATTR-during-spill window. The covering suites pass on macOS (apple_double_decode 31, write 15). Production LOC: 171,523 -> 171,508 (delta -15) core 39,795 -> 39,795 (0) engine 76,946 -> 76,931 (-15) layerfs-nfs -15 control 48,436 -> 48,436 (0) api 6,346 -> 6,346 (0) reference absent (retired R9c; last counted 65,417 at tree 498dd19) Method: tools/production_loc.py --detail on parent 519f8db and the staged tree Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jiangkoumo
added a commit
to jiangkoumo/layerfs
that referenced
this pull request
Nov 17, 2026
Bring the Ephemeral-AI-Lab#136 native unmount helper reaping and detach-stage custody work onto upstream/layerfs_v2_0_0 (5742338), including Ephemeral-AI-Lab#712/Ephemeral-AI-Lab#715/Ephemeral-AI-Lab#716/Ephemeral-AI-Lab#718 and the lp73 AppleDouble companion/spill series (Ephemeral-AI-Lab#713/Ephemeral-AI-Lab#714/Ephemeral-AI-Lab#722 via Ephemeral-AI-Lab#727). Conflicts resolved keeping both sides' behavior intact: - control/crates/layerfs-sandbox-backend/src/local/os/macos/darwin.rs: Keep upstream's Ephemeral-AI-Lab#712 `mount_line` and deadline-bounded `mount_table` together with Ephemeral-AI-Lab#644's shared `layerfs_bridge::process::Retained` queue (`RETAINED`), dropping the duplicate `Mutex<Vec<Child>>` / `reap_retained` / `retain` helpers so both `mount_table` and `force_unmount` reap through `RETAINED` (`reap`, `record` on synchronous reap, `retain` for background reaping on timeout). In `force::close` (Ephemeral-AI-Lab#716) and `machine_force::stop` (Ephemeral-AI-Lab#718), `exec::cancel::settle` cancels running workspace commands first, before `Sandbox::stop` -> `os::mounts::detach` -> `darwin::force_unmount`, avoiding overlap with `RETAINED`. - engine/crates/layerfs-daemon/src/control/export.rs: Keep both `ExportServing::is_mounted` (Ephemeral-AI-Lab#644) and `companion_spill_dir` / `spill_root` (Ephemeral-AI-Lab#714), combine `.appledouble` leftover cleanup with the `mount_table`-aware `mounted_under` call in `ExportServing::reclaim`, and preserve `companion_spill_dir` in `ExportConfig` and `spill_root` rmdir after `ready.directory` removal in `teardown_export` (while `server.shutdown()` -> `Kernel::release()` removes the per-namespace spill dir first). Checks: `tools/check_product_boundary.py` PASS; `tools/test_*.py` 180/180; `lf-ci.sh fast` 2258/2259 (1 pre-existing upstream Ephemeral-AI-Lab#724 failure in `layerfs-api::machine_files`, reproduced identically at 5742338); `layerfs-nfs::helper_deadline` 8/8, `daemon` 2/2, `export` 10/10, `apple_double_codec` 7/7, `apple_double_decode` 32/32; `layerfs-sandbox` `record_codec` 5/5 & `unmount_record` 9/9; `layerfs-daemon` non-mount 31/31 & selected mount lane 12/12 binaries (36 passed, 1 ignored helper); Clippy clean on changed crates; `rustfmt --check` clean on all changed files (`fmt --all --check` fails only on 3 pre-existing upstream files at 5742338). `aarch64-apple-darwin` check blocked on Linux host C toolchain (`blake3` `-arch arm64`): macOS build and live tests NOT_RUN. Durable: NOT_RUN — disabled by owner until explicit reauthorization. Production LOC: 163827 -> 171849 (delta +8022); first parent 89116b1. core 39787 -> 39795 (+8), engine 75121 -> 77198 (+2077), control 42842 -> 48510 (+5668), api 6077 -> 6346 (+269). Reference absent (retired R9c; last counted 65,417 at tree 498dd19). Method: python3 -B tools/production_loc.py --detail on git archive of first parent 89116b1 and the merged working tree.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tier: P1 (engine/crates/layerfs-nfs, engine/crates/layerfs-daemon) ; risks: risk:kernel, risk:deps, risk:security, risk:delete ; gate: yes: adds a dev-dependency, deletes a per-mount scratch area ; de-escalated: none
Docs: docs/architecture/engine/kernel-routes/nfs.md, docs/architecture/engine/daemon/README.md, docs/general/cleanup.md
Durable: NOT_RUN — disabled by owner until explicit reauthorization
Second of three PRs for #636 (AppleDouble companions beyond 4,096 per export).
It is stacked on the first PR, #713 (
lp636-codec, headff28f6118): the branchmerges that layer, so this diff contains the codec, and it has to be merged
after #713 for the codec. The third PR (the name index) is not part of this one.
Summary
The codec layer keeps an AppleDouble body in memory and refuses an edit that
does not fit the resident budget. This layer gives a body somewhere to go.
images included, they share the same
RESIDENT_BYTES(32 MiB) pool — iswritten to
<mounts>/.appledouble/<namespace>/<serial>. A body that cannotfit at all is written out whole and then edited on its file, so growing a
companion answers success instead of
NFS3ERR_NOSPC.Companions::by_use(BTreeSet<(used tick, serial)>): onelookup under the lock, not a scan of every companion, and no fixed round
count. Two commits here are prefixed
perf(nfs)because of that shape — anorder instead of a scan, and moving the adoption rename out of the lock;
nothing was timed, and they are not claimed as measured speedups.
create_newtemporary file and thefs::renamerun outside it, one write per companion at a time (the edit gate;eviction uses
try_lockand skips a busy companion). A reader keeps readingthe previous body in memory until the swap is committed, so it never sees a
half-written file. If a victim cannot be written out (removed concurrently, or
its file fails), only that victim is skipped — the error does not reach the
writer that merely needed room. The writer gets an error only when nothing
could be freed and its own body cannot be written out either.
pread: the length is read under the lock and the bytes outsideit, so a concurrent truncation gives a short read instead of a spurious
NFS3ERR_IO; an interruptedpreadis asked once more (nothing had moved).A disk edit that fails after the body was written out restores the length the
companion had, so a refused write does not leave a longer body behind.
<mounts>/.appledoubleis the per-mount disposable area the owner approved:0700directories,0600files, no fsync, no space reserved ahead. Releaseand the daemon's startup reclaim delete it, and reclaim touches nothing else
in the mount home (a test keeps a neighbouring file and directory alive
through a reclaim). Both the architecture pages and the cleanup page say so.
libcenters as a dev-dependency only (theRLIMIT_NOFILEtest), which isthe
risk:deps;tests/write.rscallsgetrlimit/setrlimit(
risk:security); release and startup callremove_dir_all(<mounts>/.appledouble)(risk:delete).layerfs_v2_0_0atcb8ced9d3through thefirst layer's merge (
ff28f6118) and then merged here ase3c80ed1c; neithermerge had a conflict, and neither changed a file of this feature.
Validation
At
e3c80ed1c, on Linux, withlf-ci.sh(upstream fast lane, then the nightlyslow and mount lanes) and
lf-ci-pg.sh. Cargo commands were queued through theworktree's build lock; each test invocation had a 120 s wall timeout and none
reached it.
rc=0: 1994 tests run, 1994 passed, 26 skipped.rc=0: 150 tests run, 150 passed, 11 skipped.rc=0; 39 tests passed;NOT_RUN=0.cargo +1.85.1 test --locked -p layerfs-nfs: 51 passed, 0 failed (write11,apple_double_decode6,apple_double_codec7,daemon2,export5,protocol15,collect1,helper_deadline3,refresh1; the rawtest resultlines sum to 52 becausespilled_companions_outnumber_the_open_file_limitre-runs itself with--exactin a child process and that line is counted once more).cargo +1.85.1 clippy --locked -p layerfs-nfs -p layerfs-daemon --all-targets -- -D warnings:rc=0, no warnings.rustfmt --checkon every changed file:clean.
python3 -B tools/check_product_boundary.py: PASS (1379 files).an_eviction_victim_that_cannot_be_written_out_does_not_fail_the_writer,a_read_racing_a_truncation_is_a_short_read_not_an_io_error,eviction_takes_the_decoded_body_before_the_written_ones,eviction_follows_the_order_of_use_not_the_shape_of_the_index,a_spill_replaces_a_companion_body_only_when_no_call_changed_it,a_spilled_companion_keeps_its_length_and_bytes_together,a_decoded_companion_is_edited_under_a_full_resident_budget,spilled_companions_outnumber_the_open_file_limit,startup_reclaim_clears_leftover_companion_spill_directories.macOS proof
Re-run at this head (
e3c80ed1c, release build) after the upstream merge:macOS 27.2 (arm64, M2) kernel NFSv3, same mount options as the first PR.
On the same Mac,
cargo +1.85.1 clippy -p layerfs-nfs -p layerfs-daemon --all-targets -- -D warningswas clean andcargo +1.85.1 test -p layerfs-nfspassed 54/54.
For the record, an earlier head answered 507 of those 600 files without their
attributes once, while the export still held all 652 companions; later reruns
and controls were correct, including the three above, so it was never
reproduced. The
slotted=/unreachable=diagnostics added while chasing itbelong to the third PR and are not in this head's release line.
Risks
risk:kernel(the export's sidecar path),risk:delete(release and startupdelete
<mounts>/.appledouble),risk:security(setrlimitin the file-limittest),
risk:deps(libc, dev-dependency only). The scratch area isexport-local and disposable:
0700/0600, no fsync, no space reserved ahead,deleted at release and at startup, and reclaim removes only that directory.
Bodies that do not fit and cannot be written out still answer
NFS3ERR_NOSPCand keep their bytes. Not covered here: the name index and the 65,536-companion
limit (the third PR), and concurrent-editing pressure beyond what the tests
drive.
Production LOC
Base
ff28f6118(the first PR's head) = 163901 → heade3c80ed1c= 164405,+504.
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
git archive <rev> | tar -x -C <dir>, thenpython3 -I tools/production_loc.py --root <dir>(tool SHA-256d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6).Related to #636; it does not close it.