Repository navigation
nfs: land #637, #713, #714, #722 with review fixes (lp73) - #727
Merged
Merged
Conversation
A READDIRPLUS entry carries its attributes and its handle. On the macOS NFSv3 client an entry without them builds no vnode, so a cold traversal of a committed tree falls back to one synchronous LOOKUP per listed name (diagnostic numbers only; see the pull request's measurement section). What the reply pays is bounded per window instead of per entry wherever the base has a batch port. The reading visit answers from memory what it holds: the window's local inode rows, whose kinds and facts come from the one statement that listed the names, and any inherited inode whose value, metadata, remembered length and remembered child-directory count are resident. Only a window memory left undecided takes the one Store reader, and it then reads the window's inherited serials in one `lookup_many` and its ordinary file lengths in one `provide_lengths`, so 8 and 200 names cost the same inode and length batches. What stays per entry, as before this change: a distinct non-inline metadata root is read once, a listed subdirectory's child-directory count lists that subdirectory, and a symlink entry reads its target. A window of ordinary files with inline metadata therefore costs exactly the two batches; a window naming those entries costs more, growing with them and not with the number of names. LOOKUP is one owner job (B4): the job that resolves a name has already read the directory's row, so it reports both. Deliberate change: a failed LOOKUP, `.`, `..` and the export's silly names answer the directory facts this export last observed for it, or no attributes when it has never seen the directory, instead of a fresh GETATTR of the parent. RFC 1813 keeps those attributes optional, so both forms are legal; the answer is now at most one observation old. The listed directory's own `dir_attributes` are its local row or the facts the export last observed in OPENDIR, never a read of its own, and the publishing visit takes one kernel lookup reference per handed-out handle in the job that makes its offset valid. Tests: layerfs-workspace 184 passed, layerfs-nfs 28 passed, layerfs-overlay 79 passed (1 skipped), layerfs-mount-ports 0 tests; clippy -D warnings clean over the changed crates. Deleted tests: none. The workspace test asserts one batch length demand, no single-key length demand, the same object batch count at 8 and 200 names, facts for every entry including a subdirectory, and no Store reader for a window of local rows; the export test asserts attributes and handles on every listed name, one owner visit for the whole enumeration, at most one Store reader per reply and none for a local directory; the daemon's Linux mount observer forwards the new `attributes` argument, which the previous version of this change missed on every target that is not macOS. Production LOC: 159977 -> 160158 (delta +181) Scope and method: `python3 -B tools/production_loc.py` over first-party product Rust/SQL/JavaScript only, at upstream/layerfs_v2_0_0 (af2ad92) and at this tree; tests, docs and tools are excluded, and the v0.1.6 reference scope is absent (retired R9c).
… batches The batched-window tests read an in-memory fixture store, so they say nothing about the Store a mounted Workspace actually reads. This adds one test-only file: it mounts a Workspace on a real PostgreSQL Store and on a real SQLite Store over the same tree, lists the root over READDIRPLUS through the product's mount path (OPENDIR, the reading visit, the window's facts outside the owner), and counts the port demands the window itself makes. Both Stores spend one batch inode demand and one batch length demand for the whole window, and the number of names does not change either count: `object_batches`/`length_batches` are exactly 1/1 at 8, 40 and 200 inherited names, and the one length demand names each file once (8, 200 and 40 ids). The assertions are equalities, not bounds, and the test says why they hold: this fixture keeps every metadata root inline and builds its subdirectory's child-directory count with the Commit, so no entry here pays a read of its own. An entry whose metadata root is an object, a subdirectory counted by a scan and a symlink's target are still read one at a time. Every entry's facts are compared field by field with a per-name LOOKUP of the same serial and with the bytes the fixture wrote, and the reply's offsets are published, reused unchanged and resumed. Needs LAYERFS_TEST_POSTGRES_URL; without it the two PostgreSQL cases report NOT_RUN and assert nothing, while the SQLite cases still run. The whole file is behind `#![cfg(feature = "postgresql")]`, so a build without that feature runs none of it. Production LOC: 160158 -> 160158 (delta 0) Scope and method: `python3 -B tools/production_loc.py` over first-party product Rust/SQL/JavaScript only, at d0e21d0 and at this tree; this change adds one test file, which is not counted.
The batching this route adds is real but the count was still overstated: `lookup_many` walks the inode table level by level and reads each level's pages in one batch demand, so a window pays one batch inode demand for each level the cache does not already answer, not one for the window. The number of names adds no demand and a deeper tree adds one. The windows the tests count are also not a fresh mount's first window: the visits before them read the inode table's upper level into the same cache, so only the levels they left cost a demand. Both places now say so, and `nfs.md` no longer calls the cost "two demands whatever its size"; it states the per-level rule and what the earlier visits make resident. No behaviour changes: comments and documentation only, and the two PostgreSQL assertions and the workspace assertions are untouched. Production LOC: 160158 -> 160158 (delta 0) Scope and method: `python3 -B tools/production_loc.py` over first-party product Rust/SQL/JavaScript only, at f2973a6 and at this tree; this change touches comments and documentation only.
The length batch pays only while the cache can remember what it answers: `remember_length` stores nothing when the allowance is smaller than one entry, and `file_length` then asks the provider for each file in turn. The row that promises one length demand now says so. Documentation only. Production LOC: 160158 -> 160158 (delta 0) Scope and method: `python3 -B tools/production_loc.py` over first-party product Rust/SQL/JavaScript only, at 64ad98b and at this tree; this change touches one documentation row only.
The comment above the batch assertion named a measured count. The test asserts the two sizes agree and stay below three, and what a run sees depends on the inode tree levels the visits before the window left unread, so the comment now says that instead of fixing a number. Comments only; assertions unchanged. Production LOC: 160158 -> 160158 (delta 0) Scope and method: `python3 -B tools/production_loc.py` over first-party product Rust/SQL/JavaScript only, at 71606c1 and at this tree; test comments are outside that scope.
Update the follow-on row in nfs.md so it no longer says a window of ordinary files with inline metadata costs "exactly the two batches" (inode demands scale per unread tree level, and length batching requires cache capacity), and clarify in kernel_directory::window_reads that OPENDIR and the reading visit fill the upper inode tree level before the window's reads are counted. Production LOC: 160158 -> 160158 (delta 0) Scope and method: python3 -B tools/production_loc.py over first-party product Rust/SQL/JavaScript only (reference absent, retired R9c), at f87d032 and at this tree; this change touches docs and test comments only.
Resolved doc conflicts: - docs/architecture/engine/kernel-routes/nfs.md: kept the new upstream structure while merging in the READDIRPLUS per-level batching/length-cache/dircount/dir_attributes rules and the LOOKUP and GETATTR section. - docs/architecture/engine/kernel-routes/ports.md: accepted upstream deletion (its substantive Decided/LOOKUP change is ported to README.md in the following commit). Production LOC: 160158 -> 160158 (delta 0) Scope and method: first-parent comparison at 871fb04 vs this merge tree via python3 -B tools/production_loc.py over first-party product Rust/SQL/JavaScript only (reference absent, retired R9c).
Port the KernelRead::prepare Value(Decided) and LOOKUP parent-directory note from the removed docs/architecture/engine/kernel-routes/ports.md into docs/architecture/engine/kernel-routes/README.md. Production LOC: 160158 -> 160158 (delta 0) Scope and method: python3 -B tools/production_loc.py over first-party product Rust/SQL/JavaScript only (reference absent, retired R9c), at 4fa8381 and at this tree; documentation only.
Clean merge: no code or documentation conflict, so no resolution was needed. The upstream side brings #678, #679/#680, #663, #642, #681 and #647. Production LOC: 160158 -> 160802 (delta +644) Scope and method: first-parent comparison at 67559fd vs this merge tree via python3 -B tools/production_loc.py over first-party product Rust/SQL/JavaScript only (reference absent, retired R9c).
…ale one
A refused LOOKUP answered `.or_else(|| kernel.directory_wcc(parent).0)`:
`directory_stats`, the last stat this export observed for the directory.
Nothing ages that entry — it is replaced only when the export observes the
directory again and removed only on unlink — so the comment claiming "at
most one observation old" was not a time bound. An install that changes a
directory and is not observed, or any other writer to the same Workspace,
left every negative LOOKUP in it answering the directory's old mtime/ctime;
a macOS client loads those lookup post-op attributes, restarts `actimeo` and
keeps the stale listing and name cache indefinitely.
The deciding job already reads the parent's row, so its refusal now carries
it: `KernelReadDecision::Refused { refusal, directory }` sends the fresh
Inode from `decide` (a name that binds nothing) through `KernelRead`, and
`nfs.rs` answers the failure's `dir_attributes` from it — one owner visit,
fresh facts, no second job. `.`, `..`, silly names and refusals decided
before any visit read no directory row and answer `None`, so the client
falls back to its own `actimeo` instead of restarting the timer on facts
nothing refreshes. `directory_wcc` now serves AppleDouble companions only,
which is what its own comment describes, and the false time-bound comments
are gone.
`Kernel::lookup` now returns `Refused { status, directory }`; `Kernel`,
`mount-ports` and the workspace decision type carry the new payload. The
refusal types, the answer type and the port-failure mapping move from
`kernel.rs` to the new `read.rs` so no file exceeds 999 lines (kernel.rs
989 -> 987, read.rs 79).
Tests: `export.rs::a_failed_lookup_answers_fresh_directory_facts_never_a_stale_observation`
adds a name to a directory from a second engine mount of the same Workspace,
which the first export does not observe, and requires the failed LOOKUP's
`dir_attributes` to be the directory's current facts (nlink 3, current
mtime) and never the pre-change ones (nlink 2), plus `None` for `.`, `..`
and a silly name; it fails against the previous behaviour (nlink 2 and the
old mtime).
`kernel_visit.rs::a_refused_lookup_answers_the_directory_its_job_read` pins
the decision payload to the directory's current row.
Production LOC: 160802 -> 160869 (delta +67)
Scope and method: python3 -B tools/production_loc.py over first-party
product Rust/SQL/JavaScript only (reference absent, retired R9c), at the
merge tree 877a1aa and at the committed tree of this commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`KernelDirectoryWindow::finish_with` called `listing_with_facts` for the whole window — up to PAGE_ROWS = 1024 names — and only then truncated it to the names `offer.existing()` publishes. A re-sent reply, which is exactly the truncated case, read the facts of every name in the window and sent a few: one batch length demand and the per-level inode batches for all of them. The base route read facts after truncation. Names and kinds now come first: `listing` merges the window and reads the inherited kinds in one batch inode demand per inode-tree level (no length demand, which belongs to facts), so both the reading visit and the request keep their bounded per-level count. The offer truncates that window, and `facts` then reads only the entries the reply sends. `listing_with_facts` is gone; `inherited_values` keeps its one length demand and `inode_values` is the kinds-only form it now splits into. Test: `kernel_directory.rs::a_re_sent_reply_reads_no_fact_of_a_name_it_does_not_send` publishes a two-name reply over a nine-name window and requires the re-sent READDIRPLUS to remember exactly two lengths (`file_lengths`); reading facts before truncation read eight. Production LOC: 160869 -> 160847 (delta -22) Scope and method: python3 -B tools/production_loc.py over first-party product Rust/SQL/JavaScript only (reference absent, retired R9c), at 917e554e6 and at the staged tree of this commit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`memory_stat` repeated the `base.inode(serial)` lookup `listing` had just
done for every name of a READDIRPLUS window, inside the owner visit. It is
bounded and reads no Store (`resident` answers memory only), but it is the
same page walk twice per name.
`listing` now returns those base values with the listing it built
(`Kinds { listing, values }`), and `memory_stat` takes the value by
reference, so the visit reads each inherited inode once. The cold path keeps
`Kinds.values` unused: `facts` reads the names the reply sends, after the
offer truncates the window.
No new test: the duplicate was an in-memory lookup with no observable I/O,
and `kernel_directory.rs`'s window tests cover both paths.
Production LOC: 160847 -> 160864 (delta +17)
Scope and method: python3 -B tools/production_loc.py over first-party
product Rust/SQL/JavaScript only (reference absent, retired R9c), at
272fa823c and at the staged tree of this commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The NFS guide still described the retired behaviour: a failed LOOKUP, `.`, `..` and a silly name answered the directory facts the export last published or observed, "trading up to one observation period of staleness". Nothing ages that observation, so there is no such period; the export answered it until it happened to observe the directory again. Now the guide says what the source does: - a LOOKUP that fails answers the directory's own fresh row whenever its job read that row, which every looked-up name does; - `.`, `..`, a silly name and a refusal decided before any visit send no `dir_attributes`, so the client falls back to its own `actimeo`; - no LOOKUP answer repeats the export's last observation; - the remembered directory facts now serve only the AppleDouble companion `wcc_data` and a listed inherited directory's `dir_attributes`; - READDIRPLUS reads names and kinds first and only the sent names' facts. `read.rs` is added to the source list and its LOOKUP row, and the ports guide says a refused LOOKUP carries the directory row its job read. Production LOC: 160864 -> 160864 (delta 0) Scope and method: python3 -B tools/production_loc.py over first-party product Rust/SQL/JavaScript only (reference absent, retired R9c); documentation only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
READDIR and READDIRPLUS answered an inherited directory's `post_op_attr` from `.or_else(|| kernel.directory_wcc(serial).0)`: `directory_stats`, the facts this export last observed for that directory, which its OPENDIR visit read whenever the descriptor was opened. Nothing ages that entry — it is replaced only when the export observes the directory again — so a directory whose facts moved on after the open was answered as it was then, for as long as the descriptor lived: the client loads those reply attributes, restarts `actimeo` and keeps its listing and name cache. The upstream base has no such fallback; this PR introduced it while changing LOOKUP. A reply's `dir_attributes` are now exactly the row its listing window read: fresh facts from that visit, costing no further read. An inherited directory has no local row, so its window reads none and the reply carries no attributes at all — RFC 1813 makes the field optional — which leaves the client's own `actimeo` in charge instead of restarting it on an observation nothing refreshes. `directory_wcc` now serves the AppleDouble companion `wcc_data` alone, which is all its own comment describes. Tests: `export.rs::an_inherited_directorys_listing_never_answers_a_stale_observation` adds a name to an inherited directory from a second export of the same Workspace after this export's OPENDIR observed it, and requires the first reply's directory attributes to be absent and the continued reply's to be the current facts (nlink 3) and never the pre-change ones (nlink 2); it fails against the previous behaviour, which answered the observed nlink 2. `export.rs::readdirplus_lists_a_directory_with_attributes_and_handles` now expects no `dir_attributes` for its inherited `src`. The NFS guide's directories table says the same, and the remembered-directory-facts row no longer lists a listing as one of their readers. Production LOC: 160864 -> 160863 (delta -1) Scope and method: python3 -B tools/production_loc.py over first-party product Rust/SQL/JavaScript only (reference absent, retired R9c), at the parent tree 2fc186e and at this commit's staged tree. The four comment lines and the guide rows are not production code; the net -1 is the removed fallback line.
….rs (lp73) Relocation only, no behavior change: `Kernel::release`, `Kernel::held`, `retained_count` and the `Released` struct move out of `kernel.rs` into `kernel/released.rs` verbatim, and `pub use released::Released` keeps every existing path working. This is the first commit of the AppleDouble split (LP636): `kernel.rs` would otherwise cross the 999-line ceiling once the companion work lands. No test or documentation change; the moved functions are the same text in a new module. Production LOC: 163505 -> 163511 (delta +6) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 8cfb6e2 (163505) and this tree (163511). The +6 is the new module's own two `use` lines, its `impl Kernel` block and the `mod released;` / `pub use released::Released;` declaration in kernel.rs. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
The client lays a `._<name>` companion out as an AppleDouble v2 image: a header, a FinderInfo entry that runs into an `ATTR` table of extended attributes, and a resource-fork entry at the end. `apple_double_codec` reads those fields (`Image::parse`) and, when the fields encode back to the very bytes the client wrote (`Image::compact`), keeps them as its own record, so no attribute, value or byte the layout carries is dropped, and one attribute's alignment padding survives a shorter name replacing a longer one. `Image::new`/`Image::encode` lay an image out as the client does, which is what makes the round trip exact and what the macOS evidence compares against. Malformed, truncated or differently laid-out images, and a resource fork with data, parse to `None` and are left as the client wrote them. This is the first layer of the AppleDouble split (LP636): the codec alone, with its own unit tests; the companion that holds its decoded form comes next. Tests: new `tests/apple_double_codec.rs`, 6 cases (byte-exact round trip for every value size, many attributes, alignment padding, whole units, a resource fork, and malformed images that must not decode), all pass. Production LOC: 163511 -> 163792 (delta +281) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 1810b4e (163511) and this tree (163792). Engine +281 (`apple_double_codec.rs` +280, `lib.rs` +1). Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
A macOS client writes each companion once, in full, and reads it back unchanged. A body is therefore classified whenever a write or a resize leaves it: one that parses back to the very bytes the companion holds (`Image::compact`) and fits `DECODED_ENTRY_BYTES` is kept as its FinderInfo and attributes and counted against `DECODED_BYTES` instead of its image, and every READ synthesizes the same bytes again. Anything else - a resource fork, another layout, a half-written image - is stored as the client wrote it. This replaces `TOTAL_BYTES` with two budgets that say what they bound: `RESIDENT_BYTES` (32 MiB) of written bodies kept in memory, and `DECODED_BYTES` (512 MiB) of decoded ones counted as `Image::cost`. A write or resize that neither decodes nor fits the resident budget answers `NFS3ERR_NOSPC` and leaves the companion exactly as it was; the next layer spills those bodies to disk instead of refusing them. `Released` now reports `companions_decoded` and `companion_decoded_bytes`, and `companions_release`/`companions_held` answer the `Holding` record the counters live in. Second layer of the AppleDouble split (LP636), on top of the codec. Tests: new `tests/apple_double_decode.rs` with the client's own write sequence (a half-written header, the rest, an edit that breaks the image and one that completes it again, truncate below and resize back up), read back byte for byte. Full `layerfs-nfs` suite: 41 passed, 0 failed. Production LOC: 163792 -> 163886 (delta +94) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 210d5f8 (163792) and this tree (163886). Engine +94. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
The NFS route guide now names `apple_double_codec.rs` and states what each bound actually holds: `RESIDENT_BYTES` for written bodies and `DECODED_BYTES` / `DECODED_ENTRY_BYTES` for the decoded form, replacing the `TOTAL_BYTES` row the code no longer has. The paragraph after the table says which bodies decode (complete images that encode back to the same bytes) and which stay as written. Documentation only; it describes the codec layer as it stands. Production LOC: 163886 -> 163886 (delta 0) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent fa5dd34 (163886) and this tree (163886). Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
…get (lp73) A client's companion body no longer has to fit in memory and no longer takes memory from every other export. Bodies beyond `RESIDENT_BYTES` (32 MiB) are spilled, least recently touched first, to a scratch directory of the export's own (`<mounts>/.appledouble/<namespace>/<serial>`), and a write that still does not fit the budget after bounded eviction rounds is written out itself and edited in its file, which needs no budget at all. - `apple_double_spill.rs` owns the scratch files: one per companion serial, directory `0700` and files `0600` whatever the umask, written through a `<serial>.<n>.tmp` that only the adopting call renames to `<serial>`, no fsync and no space reserved ahead. A spilled body is opened by path per `pread`/`pwrite`/`set_len` and never held open, so companions far outnumbering the process's file limit all work. A failed file operation is answered with the NFS status its errno means (`ENOSPC`/`EDQUOT`, `EFBIG`, `EACCES`, `EPERM`, `EROFS`, else `NFS3ERR_IO`). - `apple_double/edit.rs` holds READ, WRITE and SETATTR and the eviction: one edit of a companion at a time under that companion's own gate, so a truncate and a write can never leave a length its file does not have, and the file I/O never holds the companion lock. - `ExportConfig` gains `companion_spill_dir` and `companion_resident_bytes`. The daemon gives each export `<mounts>/.appledouble/<namespace>`, removes it at startup before any Attach, at release and when a failed Attach unwinds, and touches nothing else under the mount home. - `Released` reports `companion_bytes`. Third layer of the AppleDouble split (LP636), on top of the codec; `HELD` and the name index are the layer after this one. Production LOC: 163886 -> 164334 (delta +448) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 584cc3e (163886) and this tree (164334). Engine +448, the daemon's `export.rs` share included. Tests for the layer follow in the next commit. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
The rig can now set the resident budget and an explicit spill directory, and the cases exercise the layer end to end: - a body smaller than the budget stays resident and a larger one spills; the spilled file carries the bytes and the length the client wrote; - an edit that leaves a decoded image not whole spills it instead of growing memory, while a truncate of a decoded body needs no room at all; - a spill only replaces a companion no other call changed meanwhile (`spill_applies`); - a truncate and a write of one spilled companion start together, round after round, and the length a reply reports is always the length its file holds; - companions four times the process's own `RLIMIT_NOFILE` all read and write, because a spilled body is opened by path per operation: that case lowers a process-wide limit and counts the process's descriptors, so it re-runs itself in its own process (`run_alone`); - Attach cleans a leftover `.appledouble` of a daemon that died, release and unmount remove it, and startup reclamation removes it while keeping every other entry under the mount home. `libc` is a new dev-dependency of `layerfs-nfs`, only for the file-limit case. Tests: `cargo test -p layerfs-nfs` 46 passed, 0 failed (the raw sum reads 47 because the file-limit case runs itself once in a child process). Production LOC: 164334 -> 164334 (delta 0) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 7b38dad (164334) and this tree (164334). Tests only. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
The guide names `edit.rs` and `apple_double_spill.rs`, describes what the `RESIDENT_BYTES` row does now (spill to `companion_spill_dir`, and which status each host errno maps to), and records the scratch file's own rules: one file per companion serial under `<mounts>/.appledouble/<namespace>`, `0700`/`0600`, written as a temporary and adopted by rename only when no call changed the body, opened by path per operation and never held open, deleted by startup reclamation and by release. `ExportConfig` lists the two new fields, and the `wcc_data` row now says a companion mutation answers from `directory_stats`. Documentation only; it describes the spill layer as it stands. Production LOC: 164334 -> 164334 (delta 0) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 91434f9 (164334) and this tree (164334). Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
A companion created while the export knows its main file (`<name>` in the same directory) now belongs to that inode: every name of the inode reaches it, so hard links share one, it follows the inode through RENAME and it goes with the inode's last name. Such a companion holding a complete AppleDouble image is `whole`: it takes no slot of the index bound, and `rename`/`silly rename`/ `remove` of the main file drop it through `Kernel::companion_drop_for` and `companion_drop_inode`. `HELD` rises from 4,096 companions to 65,536 slotted entries: at the bound, orphan slotted companions are reclaimed first, and only if 65,536 live ones remain does CREATE answer `NFS3ERR_NOSPC`, count `names_refused` and log at most once per 60 s per export. The name index is the other half of it. `Table::aliases` remembers a hard-linked inode's names beyond `Table::names`, so `entry_taken` (when the engine reports `nlink <= 1`), `renamed`, `finish_unlink` and retirement drop only their own serial's keys from `by_name` instead of scanning it; a name that was moving to another inode is dropped through `Table::unbind_any`. CREATE at the bound releases the companion lock to collect orphans, then re-reads the companion name and main file under the lock it now holds, so two GUARDED creates of one name yield one serial and one `NFS3ERR_EXIST`. `Companions::slots` walks only slotted entries - whole images add nothing - and `classify` decides again under RENAME's destination, so a companion moved onto an unknown main takes its slot back while a spilled body keeps the class the write that spilled it left (a decode there would read its file under the lock). `Kernel::companions_release` reports `Holding`, and `Released` carries `companions_slotted` and `companions` counted over whole images too. This is the last layer of the AppleDouble split (LP636) and reproduces `2e09f0295`'s companion code; the review's open findings are handled in the next task. Tests: `cargo test -p layerfs-nfs` 60 passed, 0 failed (raw sum 61: the file-limit case re-runs itself in a child process): 16 write, 11 decode, 6 codec, 2 daemon cases. Production LOC: 164334 -> 164574 (delta +240) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 8183f89 (164334) and this tree (164574). Engine +240. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
…lp73) The remaining cases of the layer: a whole image takes no slot and is read back from every name of its inode; a resource fork, an unknown main and a companion moved off its inode keep the stored-as-written path and their slot; `HELD` is 65,536 and a small bound reclaims orphans instead of refusing; two GUARDED creates of one name, and two of them over the last free slot, leave one entry; a reclaimed companion never loses a live main file's attributes, including one that gained a hard link; a renamed hard-link alias keeps the inode's recalled name bound; and companions survive a main-file rename, silly rename and replacing rename exactly as the client expects. `apple_double_companions_spill_preserve_all_bytes_reclaim_orphans_and_clean_up` is the long one: 4,160 companions past the old 4,096 count and a 64 KiB resident budget, every write read back byte for byte, the spill directory's modes, the unlinks a main-file removal or replacing RENAME performs, the `FILE_BYTES` refusal, and the `HELD` bound with orphan reclamation before `NFS3ERR_NOSPC`. The rig can set `companions_held` too. Production LOC: 164574 -> 164574 (delta 0) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent dc2c54c (164574) and this tree (164574). Tests only. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
The AppleDouble section now states what `HELD` (65,536 slotted entries) does at the bound - reclaim orphans, then `NFS3ERR_NOSPC` with one rate-limited log per export - and that a companion whose main inode the export knows belongs to that inode: hard links share it, RENAME keeps it under the new name and its aliases, and its last name or silly rename drops it. It records that a whole image takes no `HELD` slot, that `companion_rename` re-classifies under the destination, and how `Table::names`/`Table::aliases` keep unlink, `entry_taken`, `renamed` and retirement from scanning `by_name`, and how CREATE re-reads the name and main file after collecting orphans. Documentation only; it describes the layer as it stands. Production LOC: 164574 -> 164574 (delta 0) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 2cc52c813 (164574) and this tree (164574). Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
…s for (lp73) A decoded companion was re-encoded in full under the companion lock on every READ: up to `FILE_BYTES` (1 MiB) allocated and copied to serve a reply of at most `TRANSFER_WINDOW` (256 KiB) bytes, and often far less. `Image::encode_range` lays the image out and fills only `[start, end)`, so a READ pays for the bytes it answers with; `Image::encode` is that same layout over the whole image, so there is one implementation of the layout rather than two that can drift. `edit.rs`'s spill and `edit_resident` still need the whole image and keep calling `encode()`. Whether a READ is served from a window or from the whole image is not observable: the reply bytes are the same either way. Review: finding 7 (full re-encode on every READ). Tests: new `any_window_of_the_image_encodes_to_that_window` compares every window of an image (a padding byte the client left behind, values of several sizes, the empty resource fork) with the same slice of `encode()`, including empty windows, a window past the image and one that starts after it ends. `cargo test -p layerfs-nfs --test apple_double_codec --test apple_double_decode`: 7 + 1 passed, 0 failed. Anti-proof: with `encode_range` clamped to start at 0 again the window test fails (`left` starts at image offset 0 while the expected window starts mid-image). Production LOC: 163886 -> 163908 (delta +22) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 584cc3e (163886) and of the staged tree 20b10a3 (163908). Engine +22. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
…t (lp73) `DECODED_BYTES` (512 MiB per export) was memory nothing could evict: many Workspaces can be mounted at once, so a per-export budget that is never paid back does not fit. Decoded images now take the same `RESIDENT_BYTES` (32 MiB) as bodies kept as written, a body at its length and a decoded one as `Image::cost`, so the budget sees every byte the export holds. `Companion::resident_cost` is that one number; `attach`, `detach` and `edit_body` count it, `edit_body` decides with it (a body either fits or the whole edit answers `NFS3ERR_NOSPC` and leaves the companion as it was), and `classify` no longer consults a second budget. `DECODED_ENTRY_BYTES` stays, now as the most `Image::cost` one decoded companion may be counted for: a whole image costing more keeps the client's bytes, so one companion cannot take the whole pool. This layer has no disk to spill to, so a body the budget does not take is still refused; the next layer spills it. `Released` reports the merged pool as `companion_bytes` and keeps `companions_decoded` as a count, instead of `companion_decoded_bytes` beside a separate resident total. Review: owner decision 3 (`DECODED_BYTES` rejected), first half (the budget and the counts); the spill half is the next layer. Tests: `the_clients_write_sequence_is_served_byte_for_byte` now asserts the merged pool: the decoded image's `Image::cost()` after the image arrives, the written body's length while it does not decode, the cost again once it does, and zero after the companion is removed. `cargo test -p layerfs-nfs`: 42 passed, 0 failed. Anti-proof: with a decoded body counted as zero again the test fails at the first assertion (`left: 0, right: 4372`). Production LOC: 163908 -> 163896 (delta -12) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent b01e805 (163908) and of the staged tree bfc489b (163896). Engine -12. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
The layer below changed the read path, the budget and the counters; this layer had moved the read and the edit into `apple_double/edit.rs` and given the budget a spill directory. Conflicts resolved by keeping this layer's structure with the layer below's semantics: - `Image::encode_range` serves a READ of a decoded body (the read moved to `edit.rs`, so the window it now asks for is applied there); - `DECODED_BYTES` stays deleted: `Holding` and `Released` report one `resident_bytes`/`companion_bytes` pool plus the decoded count, and `keep_decoded` asks the resident budget, not a second one; - `classify` keeps this layer's "a body on disk stays as the spill left it" and drops the decoded budget check (the per-entry cap stays); - `adopt_spilled`, `oldest_resident` and this layer's edit path are unchanged here. `a_decoded_companion_is_edited_under_a_full_resident_budget` sets a 5,120-byte budget: it takes the image's attributes (4,369) but not its 8 KiB of bytes, which is what the merged budget now means. Tests: `cargo test -p layerfs-nfs`: 47 passed, 0 failed (raw `test result` sum 48: `spilled_companions_outnumber_the_open_file_limit` runs itself again with `--exact` in a child process, whose line is counted too). Production LOC: 164334 -> 164340 (delta +6) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 8183f89 (164334) and of the staged tree (first-parent merge comparison, 164340). Engine +6. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
With decoded images counted against `RESIDENT_BYTES`, a decoded body is memory like any other and has to leave memory like any other. `oldest_resident` now takes the least recently used companion whose `resident_cost` is not zero, which is a body kept as written or a decoded one, so eviction reaches decoded bodies. Writing a decoded body out needs nothing new: `spill_one` and `spill_gated` already encode it and write those bytes, which are exactly the image the client wrote. A whole image that does not fit the budget is therefore no longer refused (`NFS3ERR_NOSPC`) nor kept decoded beyond it: it goes to its spill file and is edited there, as any body does. Review: owner decision 3 (`DECODED_BYTES` rejected), second half (the spill). Tests: new `eviction_takes_the_decoded_body_before_the_written_ones` under a 4,096-byte budget: a decoded image of 213 bytes, then two 2,000-byte bodies. The decoded body is the least recently used memory, so it is the one spilled (`companions_decoded` 1 -> 0), the two written bodies stay resident (`4000` bytes), and the image reads back byte for byte from its file. Anti-proof: with `oldest_resident` filtered to bodies kept as written again, the test fails (`companions_decoded` stays 1 and the written body is evicted instead). Production LOC: 164340 -> 164337 (delta -3) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 7a9f20c (164340) and of the staged tree (164337). Engine -3. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
Eviction walked every companion to find the least recently used one, and `spill_for` repeated that walk up to `SPILL_ROUNDS` times, so one edit could scan the index under the companion lock several times. `Companions::by_use` is now a `BTreeSet<(used tick, serial)>` of the companions holding memory: `attach` inserts an entry when a body takes budget, `detach` removes it, and `set_used` (behind `use_serial`, the guest's last touch of the body) moves it. `oldest_resident` is the first entry, skipping the caller's own serial, so a round costs one step into the order (O(log n)) and no walk. `spill_for` no longer bounds its rounds by the number of companions: it takes the oldest body, and if that body changed under it the other call has already moved its entry to the newest end, so the next round looks at a different one; the rounds end when nothing holding memory is left. `SPILL_ROUNDS` goes away. The companion lock now does O(1) or O(log n) work per step with no I/O: `attach`/`detach`/`set_used` one set operation, `oldest_resident` one lookup, and the file written before the lock is taken and renamed after it is dropped. The kernel table lock is untouched by any of this. Review: blocking finding 2 (eviction is an O(n) scan under the lock, repeated). Tests: new `eviction_follows_the_order_of_use_not_the_shape_of_the_index`: 64 companions of 16 bytes under a 1,024-byte budget, three of them read again, then eight more; the spill directory holds exactly the serials of the eight least recently used ones, in that order, and the three read again stay resident. Anti-proof: with `oldest_resident` reversed the test fails, spilling `SERIALS+2, +64..+70` (the newest) instead of `SERIALS+3..+10`. Production LOC: 164337 -> 164342 (delta +5) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent e323c0e (164337) and of the staged tree (164342). Engine +5. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
…e lock (lp73) `Companions::adopt_spilled` ran `fs::rename` while holding the companion lock, so the module documentation's "file I/O never holds the companion lock" was not true of the step that puts the file in place. The adoption is now three steps with the lock dropped between them: `Companions::spill_target` (lock) answers the companion's own file and whether the body is still the one the file was written from, `fs::rename` runs outside the lock, and `Companions::commit_spilled` (lock) switches the body to `Body::Spilled` with its length. A reader keeps reading the body in memory until the rename is done, so no reader ever sees a half written or missing file. The exclusion the rename needs is the companion's own edit gate, which already means "one edit of this companion at a time": a gated edit holds it across its own adoption, and eviction now takes it with `try_lock`, skipping a companion another call is editing (it never waits for another call's I/O, so no two spills of one companion can meet at the file). `spill_for` therefore passes over a victim it could not spill instead of picking it again, and a companion that is gone is nothing to spill rather than an error. Review: finding 5 (disk I/O under the companion lock); the module docs of `edit.rs` and `apple_double_spill.rs` and `nfs.md` now state what the code does. Tests: `cargo test -p layerfs-nfs`: 48 passed, 0 failed (raw `test result` sum 49 with the child-process rerun of `spilled_companions_outnumber_the_open_file_limit`), including the spill, race and file-limit cases that adopt bodies. Production LOC: 164342 -> 164372 (delta +30) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 73f62ef (164342) and of the staged tree (164372). Engine +30. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
… names (lp73) macOS 27.2, build 525fcfb, reproducible 20/20 and 5/5: a file carrying attributes, `ln a b`, then `rm a`, leaves `b` with no attributes at all and `._b` answering `NFS3ERR_NOENT`, while the companion is still in the export's index. The companion was the inode's, so nothing about the inode changed; what the client sent is what removed it. Reproduced in the rig with the client's sequence: `/bin/ln a b` links the main file, then the client drops the `._` name of the name it no longer uses with a REMOVE of `._a`, and REMOVE took the whole companion: `held().companions` 1 to 0, `._b` `NFS3ERR_NOENT`, the attributes gone. A client that carries the attributes to the new name also sends a LINK of the companion name (`._a` to `._b`), which this export refused with `NFS3ERR_ACCES`. - A companion now holds every `._` name of the inode it was linked under (`Companion::also`, indexed and unindexed with its own name, one name promoted when the recorded one goes). - `companion_link` serves LINK of a companion serial: another `._` name of the same companion, `NFS3ERR_EXIST` for a name already held or another companion's, `NFS3ERR_ACCES` for a name of the Workspace. - `companion_remove` drops one name while the companion holds another, or while its inode has another name the export knows (`Table::names` plus `Table::aliases`); only the last name of a one-name inode is the client clearing the attributes, which still takes the companion, as does any name of a companion no inode owns. Review: macOS re-test, finding 1 (hard link then remove the original name). This is the same rule the hard-link contract already stated, which the name scoped REMOVE broke. Tests: two new. `a_hard_link_keeps_the_companion_under_its_other_names`: main file and attributes, LINK, REMOVE `._a`, then REMOVE `a`; `._b` reads the same companion and bytes at every step and the companion goes with `b`. `a_linked_companion_name_is_another_name_of_the_same_companion`: LINK of the companion name succeeds and reads the same bytes, a held name answers `NFS3ERR_EXIST`, a plain name `NFS3ERR_ACCES`, REMOVE of `._a` keeps `._b`, and REMOVE of a one-name inode's only `._` name still removes the companion. Anti-proofs: with the keep rule forced off both tests fail (`left: 0, right: 1` companions, the attributes gone), and with LINK of a companion back to `NFS3ERR_ACCES` the second fails on that LINK. `cargo test -p layerfs-nfs`: 71 passed, 0 failed (raw sum 72 with the child-process rerun). Production LOC: 164797 -> 164884 (delta +87) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent f6cb678 (164797) and of the staged tree (164884). Engine +87: the companion's extra names and the three index operations on them, `companion_link`, the name-count rule in `companion_remove`, and the routed LINK. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
…(lp73) macOS 27.2, build 525fcfb: in one run of 600 files carrying ~100 KiB of attributes each, the first 507 in write order read back with no attributes at all while the export still held all 652 companions, 326 bodies stayed on disk, and two re-runs of the same recipe plus two controls were 100% correct. Nothing in the export's own counts had moved, so the loss was not an eviction: either those companions were still held but no name reached them, or the client never got them back for a reason of its own. The static review found no path that removes a companion's name keys and keeps the companion - every one of them takes the companion with it, or leaves it reachable through its inode - so the run could not be reproduced from the code, and a loss with the index intact has to be told apart from one where the index lost the entry. `Companions::unreachable` counts the companions neither their recorded name nor another name of their inode reaches, and the export's release line now carries it: `held= slotted= decoded= resident_bytes= unreachable=`. It runs once, over the companions held at release, so it costs nothing on the serving path. A non-zero count beside a client that lost attributes says the name, not the body, was lost; a zero says the body itself was gone. Documented with the release line in the nfs route's AppleDouble section. Review: macOS re-test, finding 2 (507 of 600 attributes lost, not reproduced). Tests: none new. The counter can only be non-zero through the very defect it reports, so an ordinary test would assert zero; the release path is covered by every test that stops an export (the line is printed with `held=`). Production LOC: 164884 -> 164900 (delta +16) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 45d0903 (164884) and of the staged tree (164900). Engine +16: the counting method and the release line's field. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
The `HELD` bound was decided in one place, but on the path that needs I/O between the decision and the commit - a WRITE, SETATTR or RENAME whose body spills and is then edited on its file - `admits` passed before `spill_gated` and the file write, and `record_edit`/`classify` took the slot afterwards without asking again. Two concurrent calls could both be told the last slot was free and both take one, so `slots.len()` could exceed `held_limit` by as much as the number of edits in flight, which contradicts "the bound holds". `Companions::reserve` decides and holds in one step under the companion lock (`Slot::Taken`/`Held`/`Full`); reservations count against the bound in `admits`, and a create or reclaim cannot steal one. The disk-edit path and `rename_companion` reserve before their I/O and give the reservation back if the edit does not land (`unreserve`, `take_serial`, the failure paths of `apply_edit`/`record_edit`/`rename_companion`), so a reservation can neither be held for good nor outlive its companion. CREATE and the in-memory edit decide and take the slot under one lock and keep using `admits`. Review: finding P2-1 (the bound could be exceeded under concurrency), and the route document's claim that the bound holds is now true as written. Tests: new `the_slot_bound_holds_when_writes_claim_the_last_slot_together`. Eight companions holding whole images (no slot), one slot, a body budget that cannot hold the broken body, and one breaking WRITE from each over its own connection after a barrier: exactly one writer takes the slot, the other seven answer `NFS3ERR_NOSPC`, `slotted` is 1, and every companion still reads back as its image or that edit. Anti-proof: with `admits` back in place of `reserve` the test failed 23 of 40 runs with two or three writers succeeding (`[Ok(()), Err(28), Ok(()), ...]`); with the reservation it passed 40 of 40. Production LOC: 164900 -> 164930 (delta +30) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent c96af5b (164900) and of the staged tree (164930). Engine +30: the reservation set, `Slot`, `reserve`/`unreserve`, the two decision sites and the three release paths. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
Reclamation asks the engine about a candidate's main file before dropping it, and treated only "no such file" as dead. A main file whose directory the client had removed cannot answer "no such file": the engine has no such parent, so the lookup answers `NFS3ERR_STALE` (`NFS3ERR_NOTDIR` for a parent that is a file), and the candidate was kept for the export's life. Its slot stayed taken, every later CREATE at the bound repeated the same failing lookup, and the bound could stay full until release. `NFS3ERR_STALE` and `NFS3ERR_NOTDIR` now count as "the main file is not there either" and the candidate is reclaimed, which is what a removed parent means for its entries. Any other answer still leaves the companion alone, and the per-name engine lookup and the slot walk are unchanged. Review: finding P2-3 (a candidate whose main file's directory is gone is never reclaimed). Tests: new `a_companion_of_a_removed_directory_is_reclaimed`: one slot, a companion inside a directory whose main file `x` never existed (so `inode: None` and the slot), the client removes the directory (`rmdir` succeeds - nothing in it is visible to the engine), then a CREATE at the bound: its reclamation's lookup answers `NFS3ERR_STALE`, the companion is dropped and the CREATE lands with one companion and one slot. Anti-proof: with only `NFS3ERR_NOENT` counted the same CREATE answers `NFS3ERR_NOSPC` (`Err value: 28`) and the test fails there, so the lookup really answers a stale parent rather than a missing entry. Production LOC: 164930 -> 164932 (delta +2) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 386c5b0 (164930) and of the staged tree (164932). Engine +2: the extra answers in the reclaim's match. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
…vershoot (lp73) Two statements about the resident budget were wrong in the direction of claiming more than the code does. A decoded image's `Image::cost` against `RESIDENT_BYTES` is a cap on what one decoded body is charged, not a limit on what a companion may store: an image costing more than `DECODED_ENTRY_BYTES` is kept as the client wrote it and charged at its length. And an edit that leaves a body whole decodes it as it records it rather than asking for room first, so the pool may stand one decoded body over the budget until the next edit needs room - "never memory the budget cannot see" was too strong. The module documentation, the `DECODED_ENTRY_BYTES` item and the `RESIDENT_BYTES` row of the nfs route now say that. The `DECODED_ENTRY_BYTES` value stays 128 KiB: with the images a macOS client lays out it is what keeps a many-attribute image from being charged a whole 4 KiB unit's worth of body for a few dozen bytes of attributes. Review: finding P3-1 (the wording of the un-budgeted decode) and the D3 note on `DECODED_ENTRY_BYTES` (owner confirmation is asked for in the pull request). Tests: none (documentation only). `cargo test -p layerfs-nfs`: 73 passed, 0 failed (raw sum 74 with the child-process rerun). Production LOC: 164932 -> 164932 (delta 0) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 0b87716 (164932) and of the staged tree (164932). Engine 0: documentation comments only. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
A companion can hold several `._` names of its inode (a LINK of a companion name adds one), but reclamation's candidate test read only the name a companion records. A companion the client never paired with a main inode (`inode: None`) that was then linked under a second name was therefore proposed as an orphan on the strength of the first name alone: the export looked up that name's main file, answered "no such file", and dropped a companion that its other name - whose main file the export does know - still reached. One linked name lost the attributes of the other. `orphans()` now walks every name a companion holds: one whose main file a name of the index holds keeps the companion, and only a companion none of whose names the index knows becomes a candidate, for the lookup of the first such name. The slot walk, the engine lookup and the decision that follows are unchanged. Tests: new `a_companion_reached_by_one_of_its_names_is_not_an_orphan`: one slot, a companion created in a directory for a main file that does not exist (`inode: None`), LINKed under a second `._` name, then the main file of that second name created; a CREATE at the bound reclaims, finds the companion reachable through the linked name and leaves it, so the CREATE is refused and the companion still reads back. Anti-proof: with the recorded name alone the same CREATE lands (`left: Ok(()), right: Err(28)`) because the live companion was reclaimed. `cargo test -p layerfs-nfs`: 74 passed, 0 failed (raw sum 75 with the child-process rerun). Production LOC: 164932 -> 164941 (delta +9) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded. Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent d114c13 (164932) and of the staged tree (164941). Engine +9: the loop over a companion's names in `orphans`. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
A macOS client that cannot read a companion's attributes looks the same from
the export whatever happened: a name it never asked for, a name the export's
index could not resolve, and an answer its own caches then dropped all leave no
trace behind. A run of the real client is the only way to tell them apart, so
this adds an opt-in trace of exactly those requests.
`LAYERFS_NFS_TRACE=1` (read once per process, off unless it is exactly `1`)
prints one line per request that names an AppleDouble companion - the `._`
names themselves and the main file names of the inodes that hold one - to the
daemon's standard error:
layerfs-nfs trace: op=LOOKUP dir=6 name="._b" main=Some(9)
route=inode companion=Some(4611686018427387904) status=Ok
`main` is the main inode the export's name index gave the name (or `None`), and
`route` says which part of the companion index answered - the companion's own
name key, the main inode, or neither - so a name no companion answers is
distinguishable from one whose main file the index forgot. A CREATE that finds
the companion already held reports whether it would truncate it, a SETATTR of a
`._` file is traced by handle with its size, `BIND` reports a main name the
kernel table re-attributed to an inode with a companion (`alias` while the
inode keeps several names, `unbound` when the one it held is all it has),
`UNPAIR` reports a main name whose paired `._` name the export dropped, and
`RENAME`/`LINK`/`REMOVE`/`CREATE` report the companion they decided. Nothing
else changes: no state is touched, and every hook costs one `OnceLock` load
while the trace is off.
Tests: no new test - the trace changes no behaviour and the hooks are on paths
`cargo test -p layerfs-nfs` (74 passed, 0 failed, raw sum 75) already drives;
checked by hand against a LINK'd companion's `._` names, where the lines above
are the actual output. `lf-ci.sh` fast: 2017 passed, 0 failed.
clippy -p layerfs-nfs --all-targets: no warning.
Production LOC: 164941 -> 165152 (delta +211)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on the first parent 54b9c1b (164941) and on the staged tree (165152). Engine
+211: `trace.rs` and its hooks in `apple_double.rs`, `edit.rs`, `kernel.rs` and
`change.rs`. Reference absent (retired R9c; last counted 65,417 at tree
498dd19).
`Kernel::companion_drop_for` dropped the `._<name>` key when a main file name was removed, so `rm a` of a file `b` still named answered `._a` with NFS3ERR_NOENT although the inode and its companion were alive (a macOS 27.2 trace showed the shape three times in one matrix). A companion that belongs to an inode now keeps that `._` name until the inode's last name, which already takes the companion with every `._` name it holds (`companion_drop_inode` -> `take_serial` -> `unindex`); a companion no main inode owns goes as before. A new file under the freed name reaches the old companion nowhere: `Companions::resolve` takes a name key only for the inode its companion belongs to. `Holding` and `Released::companions_unreachable` carry the release count, so a status read reports the same number as the release line. Tests (tests/apple_double_decode.rs): the matrix's g2/g3/g5 orders (`ln`, `rm` before and after one read of the linked name), a new file under the removed name, and the inode's last name reclaiming the companion and every `._` name key, with `._a`/`._b` reading one whole image. The three order tests failed on the parent with `._a` = NFS3ERR_NOENT. Docs: nfs.md states the hard-link rule, `UNPAIR name-kept`, the native route's `env_clear` trace wrapper and the live `unreachable` count. Production LOC: 165152 -> 165155 (delta +3); scope: first-party product source under core|engine|control|api, tools/production_loc.py on the parent tree and the staged tree; reference absent (retired R9c; last counted 65,417 at tree 498dd19).
No conflicts. Upstream since `8cfb6e273` brought the standalone fuser Git fork (#686, which removes `engine/vendor` and `tools/check_fuser_integrity.py`) with its pin guard (#687), and the GC spec items (#688). None of those files is this layer's, so the merged tree is upstream's plus this branch's commits. Production LOC: 163901 -> 163901 (delta 0) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded (the removed vendored fuser is third-party code and never counted). Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent 428ffc3 (163901) and of the merged tree (163901). Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
… (lp73) The codec layer now carries `upstream/layerfs_v2_0_0` at `cb8ced9d3`; that merge had no conflicts, and neither does this one. Upstream's changes since `8cfb6e273` (the standalone fuser Git fork, which removes `engine/vendor` and `tools/check_fuser_integrity.py`, its pin guard, and the GC spec items) touch no file of this layer, so the result is the upstream tree plus this layer's commits. Production LOC: 164405 -> 164405 (delta 0) Scope: first-party core/engine/control/api production source and shipped runtime SQL; tests, docs, tools and manifests excluded (the removed vendored fuser is third-party code and never counted). Method: python3 tools/production_loc.py (SHA-256 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6) on git archives of the first parent aec83af (164405) and of the merged tree (164405). Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
The macOS NFS client answers a `._` name that is absent from a directory's cached READDIRPLUS listing with "not found" without ever asking the server (the skipdu path in nfs_vnop_lookup), and this export serves AppleDouble companions without listing them, so a hard-linked file's extended attributes could read back empty. Turning READDIRPLUS off makes every `._` name arrive as a LOOKUP. Nothing else about the mount or the listing changes. Production LOC: 165155 -> 165155 (delta 0); core/engine/control/api production source only (api/crates/layerfs-*/src, api/python/src/layerfs, api JS packages), counted with tools/production_loc.py at the parent and the committed tree; comments, tests and docs excluded.
…Lab/layerfs-project#76) Merges the PR head e12c5cf unchanged; review fixes follow in their own commits. Production LOC: 169609 -> 169851 (delta +242) core 39,795 -> 39,795 (0) engine 75,032 -> 75,274 (+242) layerfs-daemon +4, layerfs-mount-ports +17, layerfs-nfs +64, layerfs-workspace +157 control 48,436 -> 48,436 (0) api 6,346 -> 6,346 (0) reference absent (retired R9c; last counted 65,417 at tree 498dd19) Method: tools/production_loc.py --detail on first parent dd4a2cc and the merged tree (merge: first-parent comparison) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merges the PR head ff28f61 unchanged; review fixes follow in their own commits. Production LOC: 169851 -> 170247 (delta +396) core 39,795 -> 39,795 (0) engine 75,274 -> 75,670 (+396) layerfs-nfs +396 control 48,436 -> 48,436 (0) api 6,346 -> 6,346 (0) reference absent (retired R9c; last counted 65,417 at tree 498dd19) Method: tools/production_loc.py --detail on first parent e7349ff and the merged tree (merge: first-parent comparison) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… area (lp73) Merges the PR head e3c80ed unchanged; review fixes follow in their own commits. Production LOC: 170247 -> 170751 (delta +504) core 39,795 -> 39,795 (0) engine 75,670 -> 76,174 (+504) layerfs-daemon +15, layerfs-nfs +489 control 48,436 -> 48,436 (0) api 6,346 -> 6,346 (0) reference absent (retired R9c; last counted 65,417 at tree 498dd19) Method: tools/production_loc.py --detail on first parent 7c0f937 and the merged tree (merge: first-parent comparison) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…p73) Merges the PR head 1cdb96a. One conflict, the tail of Kernel::lookup in engine/crates/layerfs-nfs/src/kernel.rs: #637 returns the job's directory row with the entry (`Answer`), #722 traces the main file; the resolution keeps both. Kernel::reclaim_orphans adapts to #637's `observe`, which now answers `Answer` / `Refused` instead of a tuple / `Status`, with no change in what it decides. Review fixes follow in their own commits. Production LOC: 170751 -> 171501 (delta +750) core 39,795 -> 39,795 (0) engine 76,174 -> 76,924 (+750) layerfs-nfs +750 control 48,436 -> 48,436 (0) api 6,346 -> 6,346 (0) reference absent (retired R9c; last counted 65,417 at tree 498dd19) Method: tools/production_loc.py --detail on first parent efa138d and the merged tree (merge: first-parent comparison) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…637 review) #637 stopped LOOKUP answering `directory_wcc` (the export's last observation of a directory, which nothing ages) except on one path: a LOOKUP of an AppleDouble companion still answered it. Finder asks for `._` names all the time, so an install or another writer's change to the directory kept reaching the client as the old mtime/nlink. That LOOKUP is answered by the export alone and no job reads the directory, so it now sends no `dir_attributes`, as `.`, `..` and silly names already do. A failed LOOKUP's directory row (#637's `Refused::directory`) is now kept as the export's observation, as a successful LOOKUP's row is, so a companion CREATE/REMOVE/RENAME `wcc_data` after it answers those facts rather than older ones. Docs: nfs.md's LOOKUP row names the companion and the kept observation. Test: new `export.rs::a_companion_lookup_never_answers_a_stale_directory_observation`. It fails on the parent (b9b2a31) with the stale `nlink 2` directory and passes here. Production LOC: 171,501 -> 171,498 (delta -3) core 39,795 -> 39,795 (0) engine 76,924 -> 76,921 (-3) layerfs-nfs -3 control 48,436 -> 48,436 (0) api 6,346 -> 6,346 (0) reference absent (retired R9c; last counted 65,417 at tree 498dd19) Method: tools/production_loc.py --detail on parent b9b2a31 and the staged tree Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…review) #722 left engine/crates/layerfs-nfs/src/apple_double.rs at 1,272 physical lines, and tools/check_product_boundary.py failed on it. This is relocation only, with no behaviour change: - `impl Companions` (names, inodes, slots, resident budget, eviction order) moves to `apple_double/index.rs`; - `Kernel::orphans` and `Kernel::reclaim_orphans` move to `apple_double/reclaim.rs`; - moved methods become `pub(super)`, and each file imports what it uses. The moved lines are otherwise byte-identical (checked by comparing the sorted non-import lines before and after). apple_double.rs is now 809 lines, index.rs 345, reclaim.rs 141. The boundary check passes (1,417 files). Docs: nfs.md's AppleDouble section lists the two new files. Tests: none changed. Production LOC: 171,498 -> 171,516 (delta +18; relocation: the new files' own imports) core 39,795 -> 39,795 (0) engine 76,921 -> 76,939 (+18) layerfs-nfs +18 control 48,436 -> 48,436 (0) api 6,346 -> 6,346 (0) reference absent (retired R9c; last counted 65,417 at tree 498dd19) Method: tools/production_loc.py --detail on parent 9db8f4d and the staged tree Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ion (#722 review) Kernel::reclaim_orphans decides outside both locks: each candidate whose main file is unknown costs one engine lookup, and the commit then took the lock again and re-checked only the companion's inode and slot. A call that made the companion live in that window (a RENAME of `._x` onto a name whose main file exists, a CREATE of the main file followed by a WRITE to `._x`, a READ) did not stop the drop. The binding half could also overwrite another companion's entry for the same main inode. `Orphan` now carries the companion's `used` tick from the decision, and `Companions::unchanged` requires it, the same inode and the slot still taken, both before dropping a companion and before binding one back. A main inode that already has its companion is not rebound; the candidate stays and is looked up again at the next reclaim. Every call that reaches a companion moves its tick. Review finding rejected after testing: the review said a lookup in a directory whose reference the export had collected would answer `Stale` and so drop a live companion. It does not: the engine answers the lookup in a collected directory. The new test collects the directory (a later GETATTR of it resolves it again) and passes both before and after this change, so it is coverage, not a regression test. Test: new `apple_double_decode.rs::a_companion_in_a_collected_directory_is_kept_while_its_main_file_is_there`. The race itself has no deterministic client-side reproduction (the window is between the out-of-lock engine lookup and the commit), so no test reproduces it. Production LOC: 171,516 -> 171,525 (delta +9) core 39,795 -> 39,795 (0) engine 76,939 -> 76,948 (+9) layerfs-nfs +9 control 48,436 -> 48,436 (0) api 6,346 -> 6,346 (0) reference absent (retired R9c; last counted 65,417 at tree 498dd19) Method: tools/production_loc.py --detail on parent fb28450 and the staged tree Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- SETATTR of a companion runs as one edit. It checked its guard under one lock, resized under a second and set mode/mtime under a third, so a guarded SETATTR could pass its ctime check and resize a body a WRITE had changed in between. It now holds the companion's edit gate from the guard to the mode and time, and a WRITE reads its pre-op attributes under the gate (`edit_gate` + `edit_gated` replace `edit_companion`). A REMOVE takes no gate, so a SETATTR racing one can still answer `Stale` after resizing a body that is gone with its companion; nothing is left behind. - A whole image is decoded only when its decoded form costs less than its bytes (`worth_decoding`). Each attribute is charged its name, value and 56 bytes, so an image of many small attributes cost several times its length decoded, could be refused room its bytes had, and was charged more than it held. - An edit of a decoded body asks for the room its bytes add beyond the decoded charge it gives back, not for its whole length, so overwriting or shrinking one no longer spills more than it needs. - `Image::parse` bounds its first allocation by what the image can hold rather than by the client's entry count (up to 65,535 entries). The resident budget stays at 32 MiB (owner decision 2026-11-17); #713's description called it upstream's, but the base branch had 64 MiB. The architecture page already says 32 MiB. Not changed: every WRITE to a decoded body still re-encodes and re-parses the whole body under the companion lock; that needs an in-place decoded edit. Docs: nfs.md (the DECODED_ENTRY_BYTES row, and the SETATTR and WRITE gate) and the module docs of apple_double.rs. Test: new `apple_double_decode.rs::an_image_whose_attributes_cost_more_than_its_bytes_is_kept_as_written` (600 two-byte names). It fails on the parent (77e7368), where the image is decoded (decoded count 1), and passes here. The SETATTR race has no deterministic client-side reproduction, so it has no test. Production LOC: 171,525 -> 171,523 (delta -2) core 39,795 -> 39,795 (0) engine 76,948 -> 76,946 (-2) layerfs-nfs -2 control 48,436 -> 48,436 (0) api 6,346 -> 6,346 (0) reference absent (retired R9c; last counted 65,417 at tree 498dd19) Method: tools/production_loc.py --detail on parent 77e7368 and the staged tree Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A WRITE that grows a spilled body no longer shows its new length before its bytes. `apply_edit` wrote the body out pre-extended to the edit's length, and `commit_spilled` recorded that length before `edit.disk` wrote anything. A GETATTR or READ in between (neither takes the edit gate) saw the larger size and read zeros, and a failed edit shrank it back, so a client could see a length no WRITE made. The body is now written out at the length it has, the edit grows its file itself, and `record_edit` moves the length after the bytes are there. `restore_len` and the length it saved are gone, and with them the early `Stale` return that skipped `unreserve` after a reservation. - `spill_target` never replaces a body that is already on disk, whoever asks. Before, a spill with no `used` snapshot renamed its file onto the live one first and found out afterwards. That was unreachable while the edit gate holds, but it would have deleted the live body. - With no spill directory, `spill_for` stops at once instead of copying or encoding every resident victim under the companion lock only to fail on each. One victim whose file cannot be written is still passed over for the next (`an_eviction_victim_that_cannot_be_written_out_does_not_fail_the_writer` keeps that). - `spill_applies` was public only for a test that fed it integers: both are removed, and `spill_target` compares the tick itself. Deleted test: `write.rs::a_spill_replaces_a_companion_body_only_when_no_call_changed_it`. The behaviour it named holds: every body change runs under the edit gate, and a spill takes the gate with `try_lock` and checks the `used` tick. - `spilled_companions_outnumber_the_open_file_limit` waits for its child with a 90 s deadline and kills a child that hangs, instead of `status()`, which waits indefinitely. Docs: nfs.md, the paragraph on one edit per companion. Tests: no deterministic client-side test reproduces the GETATTR-during-spill window. The covering suites pass on macOS (apple_double_decode 31, write 15). Production LOC: 171,523 -> 171,508 (delta -15) core 39,795 -> 39,795 (0) engine 76,946 -> 76,931 (-15) layerfs-nfs -15 control 48,436 -> 48,436 (0) api 6,346 -> 6,346 (0) reference absent (retired R9c; last counted 65,417 at tree 498dd19) Method: tools/production_loc.py --detail on parent 519f8db and the staged tree Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…722 review) - RENAME of one of a companion's names moves only that name. `rename_companion` never got the source name: it overwrote the companion's primary name and kept every LINK-made name, the renamed one included. With `._a` linked as `._b`, `mv b c` and `._b -> ._c` left `._b` answering and dropped `._a`'s own key. A companion reached only through its inode keeps the names it holds and gains the target. - A slot reservation is never left for a companion that is gone. `Companions::reserve` takes nothing for a serial another call removed (a REMOVE takes no edit gate), and `record_edit` gives the reservation back before it can find the companion gone. Either path used to leave the bound one slot smaller for the life of the export. - macOS mounts no longer add `nordirplus` (owner decision 2026-11-17). This reverts 46550ff: the loss it targeted reproduced 0/5 with and without the option, and every READDIR then costs a LOOKUP per entry. The hard-link loss with READDIRPLUS on stays an open, unconfirmed risk. Review finding not taken: making REMOVE of `._a` drop the whole companion while `a` still exists (to serve `xattr -c` on a hard-linked file). The client's own sequence for `ln a b` removes `._a` while `a` exists and expects the attributes under `b` to stay. That is the macOS loss #722 fixed, and `a_hard_link_keeps_the_companion_under_its_other_names` pins it. Docs: nfs.md, the companion names paragraph and the macOS client options. Test: new `apple_double_decode.rs::renaming_one_linked_companion_name_moves_only_that_name`. It fails on the parent (3466c3d), where `._b` still answers, and passes here. The reservation race has no deterministic client-side reproduction. Production LOC: 171,508 -> 171,521 (delta +13) core 39,795 -> 39,795 (0) engine 76,931 -> 76,944 (+13) layerfs-nfs +13 control 48,436 -> 48,436 (0) api 6,346 -> 6,346 (0) reference absent (retired R9c; last counted 65,417 at tree 498dd19) Method: tools/production_loc.py --detail on parent 3466c3d and the staged tree Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…itionally - rustfmt on apple_double/index.rs and the two NFS test files changed by the review fixes. - engine/crates/layerfs-store-service/tests/postgresql_readdirplus_window.rs (from #637) was behind `#![cfg(feature = "postgresql")]`, but #726 on the base branch removed that feature and always compiles the PostgreSQL engine. Clippy refused the unknown cfg value, and the file would never have run. The gate is removed, as in the other postgresql_* tests, which report NOT_RUN without LAYERFS_TEST_POSTGRES_URL. Production LOC: 171,521 -> 171,524 (delta +3; rustfmt line breaks) core 39,795 -> 39,795 (0) engine 76,944 -> 76,947 (+3) layerfs-nfs +3 control 48,436 -> 48,436 (0) api 6,346 -> 6,346 (0) reference absent (retired R9c; last counted 65,417 at tree 498dd19) Method: tools/production_loc.py --detail on parent fc40696 and the staged tree Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jiangkoumo
added a commit
to jiangkoumo/layerfs
that referenced
this pull request
Nov 17, 2026
Bring the Ephemeral-AI-Lab#136 native unmount helper reaping and detach-stage custody work onto upstream/layerfs_v2_0_0 (5742338), including Ephemeral-AI-Lab#712/Ephemeral-AI-Lab#715/Ephemeral-AI-Lab#716/Ephemeral-AI-Lab#718 and the lp73 AppleDouble companion/spill series (Ephemeral-AI-Lab#713/Ephemeral-AI-Lab#714/Ephemeral-AI-Lab#722 via Ephemeral-AI-Lab#727). Conflicts resolved keeping both sides' behavior intact: - control/crates/layerfs-sandbox-backend/src/local/os/macos/darwin.rs: Keep upstream's Ephemeral-AI-Lab#712 `mount_line` and deadline-bounded `mount_table` together with Ephemeral-AI-Lab#644's shared `layerfs_bridge::process::Retained` queue (`RETAINED`), dropping the duplicate `Mutex<Vec<Child>>` / `reap_retained` / `retain` helpers so both `mount_table` and `force_unmount` reap through `RETAINED` (`reap`, `record` on synchronous reap, `retain` for background reaping on timeout). In `force::close` (Ephemeral-AI-Lab#716) and `machine_force::stop` (Ephemeral-AI-Lab#718), `exec::cancel::settle` cancels running workspace commands first, before `Sandbox::stop` -> `os::mounts::detach` -> `darwin::force_unmount`, avoiding overlap with `RETAINED`. - engine/crates/layerfs-daemon/src/control/export.rs: Keep both `ExportServing::is_mounted` (Ephemeral-AI-Lab#644) and `companion_spill_dir` / `spill_root` (Ephemeral-AI-Lab#714), combine `.appledouble` leftover cleanup with the `mount_table`-aware `mounted_under` call in `ExportServing::reclaim`, and preserve `companion_spill_dir` in `ExportConfig` and `spill_root` rmdir after `ready.directory` removal in `teardown_export` (while `server.shutdown()` -> `Kernel::release()` removes the per-namespace spill dir first). Checks: `tools/check_product_boundary.py` PASS; `tools/test_*.py` 180/180; `lf-ci.sh fast` 2258/2259 (1 pre-existing upstream Ephemeral-AI-Lab#724 failure in `layerfs-api::machine_files`, reproduced identically at 5742338); `layerfs-nfs::helper_deadline` 8/8, `daemon` 2/2, `export` 10/10, `apple_double_codec` 7/7, `apple_double_decode` 32/32; `layerfs-sandbox` `record_codec` 5/5 & `unmount_record` 9/9; `layerfs-daemon` non-mount 31/31 & selected mount lane 12/12 binaries (36 passed, 1 ignored helper); Clippy clean on changed crates; `rustfmt --check` clean on all changed files (`fmt --all --check` fails only on 3 pre-existing upstream files at 5742338). `aarch64-apple-darwin` check blocked on Linux host C toolchain (`blake3` `-arch arm64`): macOS build and live tests NOT_RUN. Durable: NOT_RUN — disabled by owner until explicit reauthorization. Production LOC: 163827 -> 171849 (delta +8022); first parent 89116b1. core 39787 -> 39795 (+8), engine 75121 -> 77198 (+2077), control 42842 -> 48510 (+5668), api 6077 -> 6346 (+269). Reference absent (retired R9c; last counted 65,417 at tree 498dd19). Method: python3 -B tools/production_loc.py --detail on git archive of first parent 89116b1 and the merged working tree.
mengmengmengqiang
pushed a commit
that referenced
this pull request
Nov 17, 2026
This was referenced Nov 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tier: P1 (engine/crates/layerfs-nfs, engine/crates/layerfs-daemon, engine/crates/layerfs-workspace, engine/crates/layerfs-mount-ports) ; risks: risk:kernel, risk:delete ; gate: yes: NFS cache coherence, reclaims orphan companions, deletes a per-mount scratch area
Docs: docs/architecture/engine/kernel-routes/nfs.md (plus the pages #637, #713, #714 and #722 changed)
Durable: NOT_RUN — disabled by owner until explicit reauthorization
Lands #637, #713, #714 and #722 together, with the review fixes on top. Each PR head is merged unchanged, so GitHub marks all four merged when this one lands. The fixes are separate commits, one per reviewed layer.
Commits
e7349ff6cb7c0f93717aefa138dcceb9b2a3188dKernel::lookup, keeping #637'sAnswerand #722's trace.reclaim_orphansadapts to #637'sobservesignature9db8f4d35cfb28450f5bapple_double.rs(1,272 lines, failing the 999-line check) is split intoapple_double/index.rsandapple_double/reclaim.rs; the moved lines are byte-identical77e7368e52519f8db475Image::parsebounds its first allocation3466c3db31spill_appliestestfc40696bbd._name moves only that name; no leaked slot reservation;nordirplusdropped (owner decision)d4c6c8297fcfg(feature = "postgresql")gate, a feature #726 removedOwner decisions applied: the resident budget stays at 32 MiB (#713 called it upstream's; the base had 64 MiB). macOS mounts do not add
nordirplus.Review findings not taken (with evidence)
lookups_held = 8the directory's reference is collected (a later GETATTR resolves it again), and the engine still answers the lookup of its main file. The new testa_companion_in_a_collected_directory_is_kept_while_its_main_file_is_therepasses before and after the fixes, so it is coverage only.._awhileaexists should drop the whole companion. Not taken. The macOS client's ownln a bsequence removes._awhileaexists and expectsb's attributes to stay; that is the loss nfs: one AppleDouble companion per inode, 65,536 per export, nordirplus on macOS (lp73) #722 fixed, anda_hard_link_keeps_the_companion_under_its_other_namespins it.an_eviction_victim_that_cannot_be_written_out_does_not_fail_the_writer). Only the no-spill-directory case stops early.Still open (not fixed here)
nordirplustargeted) is unconfirmed: 0/5 in the authors' last runs.._file: the codec is tested against its ownImage::newlayout.Validation
At
d4c6c8297f, on macOS (arm64) throughtools/worktree.py cargowith the pinned 1.85.1:layerfs-nfs,layerfs-workspace,layerfs-mount-ports,layerfs-store-serviceandlayerfs-daemonran, each under a 120 s limit: 174 binaries, 555 passed, 0 failed, 18 ignored. Linux-only daemon tests compile out on macOS.-D warningson those five packages: clean.cargo fmt --all -- --check: only files this branch does not touch (api/crates/layerfs-cli/tests/concurrent_reads.rsand twolayerfs-sandbox-backendfiles, as on the base).tools/check_product_boundary.py: PASS (1,417 files).python3 -B -m unittest discover -s tools: OK.export.rs::a_companion_lookup_never_answers_a_stale_directory_observation: the parent answers the stalenlink 2directory.apple_double_decode.rs::an_image_whose_attributes_cost_more_than_its_bytes_is_kept_as_written: the parent decodes it.apple_double_decode.rs::renaming_one_linked_companion_name_moves_only_that_name: on the parent,._bstill answers.LAYERFS_TEST_POSTGRES_URLunset); a live macOS mount; any measurement. The four PR authors' own Linux and macOS runs were on their heads, not on this branch.Production LOC
Range, first parent
dd4a2cc40ftod4c6c8297f: 169,609 -> 171,524 (delta +1,915)Per commit: #637 merge +242, #713 merge +396, #714 merge +504, #722 merge +750; then the fixes −3, +18 (relocation imports), +9, −2, −15, +13 and +3 (rustfmt). Each commit message carries its own block.
🤖 Generated with Claude Code