Skip to content

nfs: land #637, #713, #714, #722 with review fixes (lp73) - #727

Merged
yifanxuaaa merged 67 commits into
layerfs_v2_0_0from
claude/lp73-integration
Nov 17, 2026
Merged

yifanxuaaa merged 67 commits into
layerfs_v2_0_0from
claude/lp73-integration

Conversation

@yifanxuaaa

Copy link
Copy Markdown
Contributor

Tier: P1 (engine/crates/layerfs-nfs, engine/crates/layerfs-daemon, engine/crates/layerfs-workspace, engine/crates/layerfs-mount-ports) ; risks: risk:kernel, risk:delete ; gate: yes: NFS cache coherence, reclaims orphan companions, deletes a per-mount scratch area
Docs: docs/architecture/engine/kernel-routes/nfs.md (plus the pages #637, #713, #714 and #722 changed)
Durable: NOT_RUN — disabled by owner until explicit reauthorization

Lands #637, #713, #714 and #722 together, with the review fixes on top. Each PR head is merged unchanged, so GitHub marks all four merged when this one lands. The fixes are separate commits, one per reviewed layer.

Commits

Commit What
e7349ff6cb 7c0f93717a efa138dcce Merge #637, #713, #714, unchanged, with no conflicts
b9b2a3188d Merge #722. One conflict, the tail of Kernel::lookup, keeping #637's Answer and #722's trace. reclaim_orphans adapts to #637's observe signature
9db8f4d35c #637: a companion LOOKUP no longer answers the export's last directory observation, which nothing ages; a failed LOOKUP's directory row is kept as the observation
fb28450f5b #722: relocation only. apple_double.rs (1,272 lines, failing the 999-line check) is split into apple_double/index.rs and apple_double/reclaim.rs; the moved lines are byte-identical
77e7368e52 #722: reclamation drops or rebinds a companion only if no call used it since the decision; never overwrites another companion's inode entry
519f8db475 #713: SETATTR runs under the edit gate (guard to mode and time); decode only when cheaper than the bytes; a decoded edit asks only for the room it adds; Image::parse bounds its first allocation
3466c3db31 #714: a growing spilled WRITE records its length only after its bytes are in the file; never replace an already-spilled body; no eviction rounds without a spill directory; bounded child wait; delete the integer-only spill_applies test
fc40696bbd #722: RENAME of a linked ._ name moves only that name; no leaked slot reservation; nordirplus dropped (owner decision)
d4c6c8297f rustfmt; #637's PostgreSQL test loses its cfg(feature = "postgresql") gate, a feature #726 removed

Owner decisions applied: the resident budget stays at 32 MiB (#713 called it upstream's; the base had 64 MiB). macOS mounts do not add nordirplus.

Review findings not taken (with evidence)

  • Reclamation in a collected directory (raised as a likely blocker). Not a defect. With lookups_held = 8 the directory's reference is collected (a later GETATTR resolves it again), and the engine still answers the lookup of its main file. The new test a_companion_in_a_collected_directory_is_kept_while_its_main_file_is_there passes before and after the fixes, so it is coverage only.
  • REMOVE of ._a while a exists should drop the whole companion. Not taken. The macOS client's own ln a b sequence removes ._a while a exists and expects b's attributes to stay; that is the loss nfs: one AppleDouble companion per inode, 65,536 per export, nordirplus on macOS (lp73) #722 fixed, and a_hard_link_keeps_the_companion_under_its_other_names pins it.
  • Stop all eviction rounds at the first I/O failure. Not taken. One victim whose file cannot be written is passed over by design (an_eviction_victim_that_cannot_be_written_out_does_not_fail_the_writer). Only the no-spill-directory case stops early.

Still open (not fixed here)

  • Every WRITE to a decoded body re-encodes and re-parses the whole body under the companion lock.
  • Whole-image companions are never reclaimed when their main file goes through another route (Exec, FUSE); their index entries stay until release. Their bodies are budgeted and spill.
  • At a full bound, every CREATE walks up to 65,536 slots under the table lock.
  • The hard-link loss with READDIRPLUS on (what nordirplus targeted) is unconfirmed: 0/5 in the authors' last runs.
  • No fixture captured from a real macOS ._ file: the codec is tested against its own Image::new layout.

Validation

At d4c6c8297f, on macOS (arm64) through tools/worktree.py cargo with the pinned 1.85.1:

  • Every test binary of layerfs-nfs, layerfs-workspace, layerfs-mount-ports, layerfs-store-service and layerfs-daemon ran, each under a 120 s limit: 174 binaries, 555 passed, 0 failed, 18 ignored. Linux-only daemon tests compile out on macOS.
  • Clippy -D warnings on those five packages: clean.
  • cargo fmt --all -- --check: only files this branch does not touch (api/crates/layerfs-cli/tests/concurrent_reads.rs and two layerfs-sandbox-backend files, as on the base).
  • tools/check_product_boundary.py: PASS (1,417 files). python3 -B -m unittest discover -s tools: OK.
  • New tests that fail on their parent and pass at the fix:
    • export.rs::a_companion_lookup_never_answers_a_stale_directory_observation: the parent answers the stale nlink 2 directory.
    • apple_double_decode.rs::an_image_whose_attributes_cost_more_than_its_bytes_is_kept_as_written: the parent decodes it.
    • apple_double_decode.rs::renaming_one_linked_companion_name_moves_only_that_name: on the parent, ._b still answers.
  • The SETATTR, spill-length, reservation and reclaim races have no deterministic client-side reproduction; no test claims them.
  • NOT_RUN: the Linux fast, slow and mount lanes; the PostgreSQL lane (LAYERFS_TEST_POSTGRES_URL unset); a live macOS mount; any measurement. The four PR authors' own Linux and macOS runs were on their heads, not on this branch.

Production LOC

Range, first parent dd4a2cc40f to d4c6c8297f: 169,609 -> 171,524 (delta +1,915)

  core     39,795 -> 39,795 (0)
  engine   75,032 -> 76,947 (+1,915)  layerfs-daemon +19, layerfs-mount-ports +17, layerfs-nfs +1,722, layerfs-workspace +157
  control  48,436 -> 48,436 (0)
  api       6,346 ->  6,346 (0)
  reference absent (retired R9c; last counted 65,417 at tree 498dd1917812)
  Method: tools/production_loc.py --detail on each commit's first parent and its tree

Per commit: #637 merge +242, #713 merge +396, #714 merge +504, #722 merge +750; then the fixes −3, +18 (relocation imports), +9, −2, −15, +13 and +3 (rustfmt). Each commit message carries its own block.

🤖 Generated with Claude Code

jiangkoumo and others added 30 commits November 13, 2026 03:51
A READDIRPLUS entry carries its attributes and its handle. On the macOS
NFSv3 client an entry without them builds no vnode, so a cold traversal
of a committed tree falls back to one synchronous LOOKUP per listed name
(diagnostic numbers only; see the pull request's measurement section).

What the reply pays is bounded per window instead of per entry wherever
the base has a batch port. The reading visit answers from memory what it
holds: the window's local inode rows, whose kinds and facts come from the
one statement that listed the names, and any inherited inode whose value,
metadata, remembered length and remembered child-directory count are
resident. Only a window memory left undecided takes the one Store reader,
and it then reads the window's inherited serials in one `lookup_many` and
its ordinary file lengths in one `provide_lengths`, so 8 and 200 names
cost the same inode and length batches.

What stays per entry, as before this change: a distinct non-inline
metadata root is read once, a listed subdirectory's child-directory count
lists that subdirectory, and a symlink entry reads its target. A window
of ordinary files with inline metadata therefore costs exactly the two
batches; a window naming those entries costs more, growing with them and
not with the number of names.

LOOKUP is one owner job (B4): the job that resolves a name has already
read the directory's row, so it reports both. Deliberate change: a
failed LOOKUP, `.`, `..` and the export's silly names answer the
directory facts this export last observed for it, or no attributes when
it has never seen the directory, instead of a fresh GETATTR of the
parent. RFC 1813 keeps those attributes optional, so both forms are
legal; the answer is now at most one observation old.

The listed directory's own `dir_attributes` are its local row or the
facts the export last observed in OPENDIR, never a read of its own, and
the publishing visit takes one kernel lookup reference per handed-out
handle in the job that makes its offset valid.

Tests: layerfs-workspace 184 passed, layerfs-nfs 28 passed,
layerfs-overlay 79 passed (1 skipped), layerfs-mount-ports 0 tests;
clippy -D warnings clean over the changed crates. Deleted tests: none.
The workspace test asserts one batch length demand, no single-key length
demand, the same object batch count at 8 and 200 names, facts for every
entry including a subdirectory, and no Store reader for a window of local
rows; the export test asserts attributes and handles on every listed
name, one owner visit for the whole enumeration, at most one Store reader
per reply and none for a local directory; the daemon's Linux mount
observer forwards the new `attributes` argument, which the previous
version of this change missed on every target that is not macOS.

Production LOC: 159977 -> 160158 (delta +181)
Scope and method: `python3 -B tools/production_loc.py` over first-party
product Rust/SQL/JavaScript only, at upstream/layerfs_v2_0_0 (af2ad92)
and at this tree; tests, docs and tools are excluded, and the v0.1.6
reference scope is absent (retired R9c).
… batches

The batched-window tests read an in-memory fixture store, so they say
nothing about the Store a mounted Workspace actually reads. This adds one
test-only file: it mounts a Workspace on a real PostgreSQL Store and on a
real SQLite Store over the same tree, lists the root over READDIRPLUS
through the product's mount path (OPENDIR, the reading visit, the
window's facts outside the owner), and counts the port demands the window
itself makes.

Both Stores spend one batch inode demand and one batch length demand for
the whole window, and the number of names does not change either count:
`object_batches`/`length_batches` are exactly 1/1 at 8, 40 and 200
inherited names, and the one length demand names each file once (8, 200
and 40 ids). The assertions are equalities, not bounds, and the test says
why they hold: this fixture keeps every metadata root inline and builds
its subdirectory's child-directory count with the Commit, so no entry
here pays a read of its own. An entry whose metadata root is an object, a
subdirectory counted by a scan and a symlink's target are still read one
at a time.

Every entry's facts are compared field by field with a per-name LOOKUP of
the same serial and with the bytes the fixture wrote, and the reply's
offsets are published, reused unchanged and resumed.

Needs LAYERFS_TEST_POSTGRES_URL; without it the two PostgreSQL cases
report NOT_RUN and assert nothing, while the SQLite cases still run. The
whole file is behind `#![cfg(feature = "postgresql")]`, so a build
without that feature runs none of it.

Production LOC: 160158 -> 160158 (delta 0)
Scope and method: `python3 -B tools/production_loc.py` over first-party
product Rust/SQL/JavaScript only, at d0e21d0 and at this tree; this
change adds one test file, which is not counted.
The batching this route adds is real but the count was still overstated:
`lookup_many` walks the inode table level by level and reads each level's
pages in one batch demand, so a window pays one batch inode demand for
each level the cache does not already answer, not one for the window.
The number of names adds no demand and a deeper tree adds one.

The windows the tests count are also not a fresh mount's first window:
the visits before them read the inode table's upper level into the same
cache, so only the levels they left cost a demand. Both places now say
so, and `nfs.md` no longer calls the cost "two demands whatever its size";
it states the per-level rule and what the earlier visits make resident.

No behaviour changes: comments and documentation only, and the two
PostgreSQL assertions and the workspace assertions are untouched.

Production LOC: 160158 -> 160158 (delta 0)
Scope and method: `python3 -B tools/production_loc.py` over first-party
product Rust/SQL/JavaScript only, at f2973a6 and at this tree; this
change touches comments and documentation only.
The length batch pays only while the cache can remember what it answers:
`remember_length` stores nothing when the allowance is smaller than one
entry, and `file_length` then asks the provider for each file in turn.
The row that promises one length demand now says so. Documentation only.

Production LOC: 160158 -> 160158 (delta 0)
Scope and method: `python3 -B tools/production_loc.py` over first-party
product Rust/SQL/JavaScript only, at 64ad98b and at this tree; this
change touches one documentation row only.
The comment above the batch assertion named a measured count. The test
asserts the two sizes agree and stay below three, and what a run sees
depends on the inode tree levels the visits before the window left
unread, so the comment now says that instead of fixing a number.
Comments only; assertions unchanged.

Production LOC: 160158 -> 160158 (delta 0)
Scope and method: `python3 -B tools/production_loc.py` over first-party
product Rust/SQL/JavaScript only, at 71606c1 and at this tree; test
comments are outside that scope.
Update the follow-on row in nfs.md so it no longer says a window of ordinary files with inline metadata costs "exactly the two batches" (inode demands scale per unread tree level, and length batching requires cache capacity), and clarify in kernel_directory::window_reads that OPENDIR and the reading visit fill the upper inode tree level before the window's reads are counted.

Production LOC: 160158 -> 160158 (delta 0)
Scope and method: python3 -B tools/production_loc.py over first-party product Rust/SQL/JavaScript only (reference absent, retired R9c), at f87d032 and at this tree; this change touches docs and test comments only.
Resolved doc conflicts:
- docs/architecture/engine/kernel-routes/nfs.md: kept the new upstream structure while merging in the READDIRPLUS per-level batching/length-cache/dircount/dir_attributes rules and the LOOKUP and GETATTR section.
- docs/architecture/engine/kernel-routes/ports.md: accepted upstream deletion (its substantive Decided/LOOKUP change is ported to README.md in the following commit).

Production LOC: 160158 -> 160158 (delta 0)
Scope and method: first-parent comparison at 871fb04 vs this merge tree via python3 -B tools/production_loc.py over first-party product Rust/SQL/JavaScript only (reference absent, retired R9c).
Port the KernelRead::prepare Value(Decided) and LOOKUP parent-directory note from the removed docs/architecture/engine/kernel-routes/ports.md into docs/architecture/engine/kernel-routes/README.md.

Production LOC: 160158 -> 160158 (delta 0)
Scope and method: python3 -B tools/production_loc.py over first-party product Rust/SQL/JavaScript only (reference absent, retired R9c), at 4fa8381 and at this tree; documentation only.
Clean merge: no code or documentation conflict, so no resolution was
needed. The upstream side brings #678, #679/#680, #663, #642, #681 and
#647.

Production LOC: 160158 -> 160802 (delta +644)
Scope and method: first-parent comparison at 67559fd vs this merge tree
via python3 -B tools/production_loc.py over first-party product
Rust/SQL/JavaScript only (reference absent, retired R9c).
…ale one

A refused LOOKUP answered `.or_else(|| kernel.directory_wcc(parent).0)`:
`directory_stats`, the last stat this export observed for the directory.
Nothing ages that entry — it is replaced only when the export observes the
directory again and removed only on unlink — so the comment claiming "at
most one observation old" was not a time bound. An install that changes a
directory and is not observed, or any other writer to the same Workspace,
left every negative LOOKUP in it answering the directory's old mtime/ctime;
a macOS client loads those lookup post-op attributes, restarts `actimeo` and
keeps the stale listing and name cache indefinitely.

The deciding job already reads the parent's row, so its refusal now carries
it: `KernelReadDecision::Refused { refusal, directory }` sends the fresh
Inode from `decide` (a name that binds nothing) through `KernelRead`, and
`nfs.rs` answers the failure's `dir_attributes` from it — one owner visit,
fresh facts, no second job. `.`, `..`, silly names and refusals decided
before any visit read no directory row and answer `None`, so the client
falls back to its own `actimeo` instead of restarting the timer on facts
nothing refreshes. `directory_wcc` now serves AppleDouble companions only,
which is what its own comment describes, and the false time-bound comments
are gone.

`Kernel::lookup` now returns `Refused { status, directory }`; `Kernel`,
`mount-ports` and the workspace decision type carry the new payload. The
refusal types, the answer type and the port-failure mapping move from
`kernel.rs` to the new `read.rs` so no file exceeds 999 lines (kernel.rs
989 -> 987, read.rs 79).

Tests: `export.rs::a_failed_lookup_answers_fresh_directory_facts_never_a_stale_observation`
adds a name to a directory from a second engine mount of the same Workspace,
which the first export does not observe, and requires the failed LOOKUP's
`dir_attributes` to be the directory's current facts (nlink 3, current
mtime) and never the pre-change ones (nlink 2), plus `None` for `.`, `..`
and a silly name; it fails against the previous behaviour (nlink 2 and the
old mtime).
`kernel_visit.rs::a_refused_lookup_answers_the_directory_its_job_read` pins
the decision payload to the directory's current row.

Production LOC: 160802 -> 160869 (delta +67)
Scope and method: python3 -B tools/production_loc.py over first-party
product Rust/SQL/JavaScript only (reference absent, retired R9c), at the
merge tree 877a1aa and at the committed tree of this commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`KernelDirectoryWindow::finish_with` called `listing_with_facts` for the
whole window — up to PAGE_ROWS = 1024 names — and only then truncated it to
the names `offer.existing()` publishes. A re-sent reply, which is exactly
the truncated case, read the facts of every name in the window and sent a
few: one batch length demand and the per-level inode batches for all of
them. The base route read facts after truncation.

Names and kinds now come first: `listing` merges the window and reads the
inherited kinds in one batch inode demand per inode-tree level (no length
demand, which belongs to facts), so both the reading visit and the request
keep their bounded per-level count. The offer truncates that window, and
`facts` then reads only the entries the reply sends. `listing_with_facts`
is gone; `inherited_values` keeps its one length demand and `inode_values`
is the kinds-only form it now splits into.

Test: `kernel_directory.rs::a_re_sent_reply_reads_no_fact_of_a_name_it_does_not_send`
publishes a two-name reply over a nine-name window and requires the re-sent
READDIRPLUS to remember exactly two lengths (`file_lengths`); reading facts
before truncation read eight.

Production LOC: 160869 -> 160847 (delta -22)
Scope and method: python3 -B tools/production_loc.py over first-party
product Rust/SQL/JavaScript only (reference absent, retired R9c), at
917e554e6 and at the staged tree of this commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`memory_stat` repeated the `base.inode(serial)` lookup `listing` had just
done for every name of a READDIRPLUS window, inside the owner visit. It is
bounded and reads no Store (`resident` answers memory only), but it is the
same page walk twice per name.

`listing` now returns those base values with the listing it built
(`Kinds { listing, values }`), and `memory_stat` takes the value by
reference, so the visit reads each inherited inode once. The cold path keeps
`Kinds.values` unused: `facts` reads the names the reply sends, after the
offer truncates the window.

No new test: the duplicate was an in-memory lookup with no observable I/O,
and `kernel_directory.rs`'s window tests cover both paths.

Production LOC: 160847 -> 160864 (delta +17)
Scope and method: python3 -B tools/production_loc.py over first-party
product Rust/SQL/JavaScript only (reference absent, retired R9c), at
272fa823c and at the staged tree of this commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The NFS guide still described the retired behaviour: a failed LOOKUP, `.`,
`..` and a silly name answered the directory facts the export last published
or observed, "trading up to one observation period of staleness". Nothing
ages that observation, so there is no such period; the export answered it
until it happened to observe the directory again.

Now the guide says what the source does:
- a LOOKUP that fails answers the directory's own fresh row whenever its job
  read that row, which every looked-up name does;
- `.`, `..`, a silly name and a refusal decided before any visit send no
  `dir_attributes`, so the client falls back to its own `actimeo`;
- no LOOKUP answer repeats the export's last observation;
- the remembered directory facts now serve only the AppleDouble companion
  `wcc_data` and a listed inherited directory's `dir_attributes`;
- READDIRPLUS reads names and kinds first and only the sent names' facts.

`read.rs` is added to the source list and its LOOKUP row, and the ports
guide says a refused LOOKUP carries the directory row its job read.

Production LOC: 160864 -> 160864 (delta 0)
Scope and method: python3 -B tools/production_loc.py over first-party
product Rust/SQL/JavaScript only (reference absent, retired R9c);
documentation only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
READDIR and READDIRPLUS answered an inherited directory's `post_op_attr` from
`.or_else(|| kernel.directory_wcc(serial).0)`: `directory_stats`, the facts
this export last observed for that directory, which its OPENDIR visit read
whenever the descriptor was opened. Nothing ages that entry — it is replaced
only when the export observes the directory again — so a directory whose
facts moved on after the open was answered as it was then, for as long as the
descriptor lived: the client loads those reply attributes, restarts
`actimeo` and keeps its listing and name cache. The upstream base has no such
fallback; this PR introduced it while changing LOOKUP.

A reply's `dir_attributes` are now exactly the row its listing window read:
fresh facts from that visit, costing no further read. An inherited directory
has no local row, so its window reads none and the reply carries no
attributes at all — RFC 1813 makes the field optional — which leaves the
client's own `actimeo` in charge instead of restarting it on an observation
nothing refreshes. `directory_wcc` now serves the AppleDouble companion
`wcc_data` alone, which is all its own comment describes.

Tests: `export.rs::an_inherited_directorys_listing_never_answers_a_stale_observation`
adds a name to an inherited directory from a second export of the same
Workspace after this export's OPENDIR observed it, and requires the first
reply's directory attributes to be absent and the continued reply's to be the
current facts (nlink 3) and never the pre-change ones (nlink 2); it fails
against the previous behaviour, which answered the observed nlink 2.
`export.rs::readdirplus_lists_a_directory_with_attributes_and_handles` now
expects no `dir_attributes` for its inherited `src`. The NFS guide's
directories table says the same, and the remembered-directory-facts row no
longer lists a listing as one of their readers.

Production LOC: 160864 -> 160863 (delta -1)
Scope and method: python3 -B tools/production_loc.py over first-party
product Rust/SQL/JavaScript only (reference absent, retired R9c), at the
parent tree 2fc186e and at this commit's staged tree. The four comment
lines and the guide rows are not production code; the net -1 is the removed
fallback line.
….rs (lp73)

Relocation only, no behavior change: `Kernel::release`, `Kernel::held`,
`retained_count` and the `Released` struct move out of `kernel.rs` into
`kernel/released.rs` verbatim, and `pub use released::Released` keeps every
existing path working. This is the first commit of the AppleDouble split
(LP636): `kernel.rs` would otherwise cross the 999-line ceiling once the
companion work lands.

No test or documentation change; the moved functions are the same text in a
new module.

Production LOC: 163505 -> 163511 (delta +6)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent 8cfb6e2 (163505) and this tree (163511).
The +6 is the new module's own two `use` lines, its `impl Kernel` block and the
`mod released;` / `pub use released::Released;` declaration in kernel.rs.
Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
The client lays a `._<name>` companion out as an AppleDouble v2 image: a
header, a FinderInfo entry that runs into an `ATTR` table of extended
attributes, and a resource-fork entry at the end. `apple_double_codec` reads
those fields (`Image::parse`) and, when the fields encode back to the very
bytes the client wrote (`Image::compact`), keeps them as its own record, so no
attribute, value or byte the layout carries is dropped, and one attribute's
alignment padding survives a shorter name replacing a longer one.

`Image::new`/`Image::encode` lay an image out as the client does, which is what
makes the round trip exact and what the macOS evidence compares against.
Malformed, truncated or differently laid-out images, and a resource fork with
data, parse to `None` and are left as the client wrote them.

This is the first layer of the AppleDouble split (LP636): the codec alone, with
its own unit tests; the companion that holds its decoded form comes next.

Tests: new `tests/apple_double_codec.rs`, 6 cases (byte-exact round trip for
every value size, many attributes, alignment padding, whole units, a resource
fork, and malformed images that must not decode), all pass.

Production LOC: 163511 -> 163792 (delta +281)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent 1810b4e (163511) and this tree (163792).
Engine +281 (`apple_double_codec.rs` +280, `lib.rs` +1).
Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
A macOS client writes each companion once, in full, and reads it back
unchanged. A body is therefore classified whenever a write or a resize leaves
it: one that parses back to the very bytes the companion holds
(`Image::compact`) and fits `DECODED_ENTRY_BYTES` is kept as its FinderInfo and
attributes and counted against `DECODED_BYTES` instead of its image, and every
READ synthesizes the same bytes again. Anything else - a resource fork, another
layout, a half-written image - is stored as the client wrote it.

This replaces `TOTAL_BYTES` with two budgets that say what they bound:
`RESIDENT_BYTES` (32 MiB) of written bodies kept in memory, and `DECODED_BYTES`
(512 MiB) of decoded ones counted as `Image::cost`. A write or resize that
neither decodes nor fits the resident budget answers `NFS3ERR_NOSPC` and leaves
the companion exactly as it was; the next layer spills those bodies to disk
instead of refusing them.

`Released` now reports `companions_decoded` and `companion_decoded_bytes`, and
`companions_release`/`companions_held` answer the `Holding` record the counters
live in.

Second layer of the AppleDouble split (LP636), on top of the codec.

Tests: new `tests/apple_double_decode.rs` with the client's own write sequence
(a half-written header, the rest, an edit that breaks the image and one that
completes it again, truncate below and resize back up), read back byte for
byte. Full `layerfs-nfs` suite: 41 passed, 0 failed.

Production LOC: 163792 -> 163886 (delta +94)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent 210d5f8 (163792) and this tree (163886).
Engine +94. Reference absent (retired R9c; last counted 65,417 at tree
498dd19).
The NFS route guide now names `apple_double_codec.rs` and states what each
bound actually holds: `RESIDENT_BYTES` for written bodies and `DECODED_BYTES` /
`DECODED_ENTRY_BYTES` for the decoded form, replacing the `TOTAL_BYTES` row the
code no longer has. The paragraph after the table says which bodies decode
(complete images that encode back to the same bytes) and which stay as written.

Documentation only; it describes the codec layer as it stands.

Production LOC: 163886 -> 163886 (delta 0)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent fa5dd34 (163886) and this tree (163886).
Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
…get (lp73)

A client's companion body no longer has to fit in memory and no longer takes
memory from every other export. Bodies beyond `RESIDENT_BYTES` (32 MiB) are
spilled, least recently touched first, to a scratch directory of the export's
own (`<mounts>/.appledouble/<namespace>/<serial>`), and a write that still does
not fit the budget after bounded eviction rounds is written out itself and
edited in its file, which needs no budget at all.

- `apple_double_spill.rs` owns the scratch files: one per companion serial,
  directory `0700` and files `0600` whatever the umask, written through a
  `<serial>.<n>.tmp` that only the adopting call renames to `<serial>`, no
  fsync and no space reserved ahead. A spilled body is opened by path per
  `pread`/`pwrite`/`set_len` and never held open, so companions far outnumbering
  the process's file limit all work. A failed file operation is answered with
  the NFS status its errno means (`ENOSPC`/`EDQUOT`, `EFBIG`, `EACCES`,
  `EPERM`, `EROFS`, else `NFS3ERR_IO`).
- `apple_double/edit.rs` holds READ, WRITE and SETATTR and the eviction: one
  edit of a companion at a time under that companion's own gate, so a truncate
  and a write can never leave a length its file does not have, and the file
  I/O never holds the companion lock.
- `ExportConfig` gains `companion_spill_dir` and `companion_resident_bytes`.
  The daemon gives each export `<mounts>/.appledouble/<namespace>`, removes it
  at startup before any Attach, at release and when a failed Attach unwinds, and
  touches nothing else under the mount home.
- `Released` reports `companion_bytes`.

Third layer of the AppleDouble split (LP636), on top of the codec; `HELD` and
the name index are the layer after this one.

Production LOC: 163886 -> 164334 (delta +448)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent 584cc3e (163886) and this tree (164334).
Engine +448, the daemon's `export.rs` share included. Tests for the layer
follow in the next commit. Reference absent (retired R9c; last counted 65,417
at tree 498dd19).
The rig can now set the resident budget and an explicit spill directory, and
the cases exercise the layer end to end:

- a body smaller than the budget stays resident and a larger one spills; the
  spilled file carries the bytes and the length the client wrote;
- an edit that leaves a decoded image not whole spills it instead of growing
  memory, while a truncate of a decoded body needs no room at all;
- a spill only replaces a companion no other call changed meanwhile
  (`spill_applies`);
- a truncate and a write of one spilled companion start together, round after
  round, and the length a reply reports is always the length its file holds;
- companions four times the process's own `RLIMIT_NOFILE` all read and write,
  because a spilled body is opened by path per operation: that case lowers a
  process-wide limit and counts the process's descriptors, so it re-runs itself
  in its own process (`run_alone`);
- Attach cleans a leftover `.appledouble` of a daemon that died, release and
  unmount remove it, and startup reclamation removes it while keeping every
  other entry under the mount home.

`libc` is a new dev-dependency of `layerfs-nfs`, only for the file-limit case.

Tests: `cargo test -p layerfs-nfs` 46 passed, 0 failed (the raw sum reads 47
because the file-limit case runs itself once in a child process).

Production LOC: 164334 -> 164334 (delta 0)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent 7b38dad (164334) and this tree (164334).
Tests only. Reference absent (retired R9c; last counted 65,417 at tree
498dd19).
The guide names `edit.rs` and `apple_double_spill.rs`, describes what the
`RESIDENT_BYTES` row does now (spill to `companion_spill_dir`, and which status
each host errno maps to), and records the scratch file's own rules: one file
per companion serial under `<mounts>/.appledouble/<namespace>`, `0700`/`0600`,
written as a temporary and adopted by rename only when no call changed the
body, opened by path per operation and never held open, deleted by startup
reclamation and by release. `ExportConfig` lists the two new fields, and the
`wcc_data` row now says a companion mutation answers from `directory_stats`.

Documentation only; it describes the spill layer as it stands.

Production LOC: 164334 -> 164334 (delta 0)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent 91434f9 (164334) and this tree (164334).
Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
A companion created while the export knows its main file (`<name>` in the same
directory) now belongs to that inode: every name of the inode reaches it, so
hard links share one, it follows the inode through RENAME and it goes with the
inode's last name. Such a companion holding a complete AppleDouble image is
`whole`: it takes no slot of the index bound, and `rename`/`silly rename`/
`remove` of the main file drop it through `Kernel::companion_drop_for` and
`companion_drop_inode`. `HELD` rises from 4,096 companions to 65,536 slotted
entries: at the bound, orphan slotted companions are reclaimed first, and only
if 65,536 live ones remain does CREATE answer `NFS3ERR_NOSPC`, count
`names_refused` and log at most once per 60 s per export.

The name index is the other half of it. `Table::aliases` remembers a
hard-linked inode's names beyond `Table::names`, so `entry_taken` (when the
engine reports `nlink <= 1`), `renamed`, `finish_unlink` and retirement drop
only their own serial's keys from `by_name` instead of scanning it; a name that
was moving to another inode is dropped through `Table::unbind_any`. CREATE at
the bound releases the companion lock to collect orphans, then re-reads the
companion name and main file under the lock it now holds, so two GUARDED
creates of one name yield one serial and one `NFS3ERR_EXIST`.

`Companions::slots` walks only slotted entries - whole images add nothing - and
`classify` decides again under RENAME's destination, so a companion moved onto
an unknown main takes its slot back while a spilled body keeps the class the
write that spilled it left (a decode there would read its file under the
lock). `Kernel::companions_release` reports `Holding`, and `Released` carries
`companions_slotted` and `companions` counted over whole images too.

This is the last layer of the AppleDouble split (LP636) and reproduces
`2e09f0295`'s companion code; the review's open findings are handled in the
next task.

Tests: `cargo test -p layerfs-nfs` 60 passed, 0 failed (raw sum 61: the
file-limit case re-runs itself in a child process): 16 write, 11 decode, 6
codec, 2 daemon cases.

Production LOC: 164334 -> 164574 (delta +240)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent 8183f89 (164334) and this tree (164574).
Engine +240. Reference absent (retired R9c; last counted 65,417 at tree
498dd19).
…lp73)

The remaining cases of the layer: a whole image takes no slot and is read back
from every name of its inode; a resource fork, an unknown main and a companion
moved off its inode keep the stored-as-written path and their slot; `HELD` is
65,536 and a small bound reclaims orphans instead of refusing; two GUARDED
creates of one name, and two of them over the last free slot, leave one entry;
a reclaimed companion never loses a live main file's attributes, including one
that gained a hard link; a renamed hard-link alias keeps the inode's recalled
name bound; and companions survive a main-file rename, silly rename and
replacing rename exactly as the client expects.

`apple_double_companions_spill_preserve_all_bytes_reclaim_orphans_and_clean_up`
is the long one: 4,160 companions past the old 4,096 count and a 64 KiB
resident budget, every write read back byte for byte, the spill directory's
modes, the unlinks a main-file removal or replacing RENAME performs, the
`FILE_BYTES` refusal, and the `HELD` bound with orphan reclamation before
`NFS3ERR_NOSPC`.

The rig can set `companions_held` too.

Production LOC: 164574 -> 164574 (delta 0)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent dc2c54c (164574) and this tree (164574).
Tests only. Reference absent (retired R9c; last counted 65,417 at tree
498dd19).
The AppleDouble section now states what `HELD` (65,536 slotted entries) does at
the bound - reclaim orphans, then `NFS3ERR_NOSPC` with one rate-limited log per
export - and that a companion whose main inode the export knows belongs to that
inode: hard links share it, RENAME keeps it under the new name and its aliases,
and its last name or silly rename drops it. It records that a whole image takes
no `HELD` slot, that `companion_rename` re-classifies under the destination, and
how `Table::names`/`Table::aliases` keep unlink, `entry_taken`, `renamed` and
retirement from scanning `by_name`, and how CREATE re-reads the name and main
file after collecting orphans.

Documentation only; it describes the layer as it stands.

Production LOC: 164574 -> 164574 (delta 0)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent 2cc52c813 (164574) and this tree (164574).
Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
…s for (lp73)

A decoded companion was re-encoded in full under the companion lock on every
READ: up to `FILE_BYTES` (1 MiB) allocated and copied to serve a reply of at
most `TRANSFER_WINDOW` (256 KiB) bytes, and often far less. `Image::encode_range`
lays the image out and fills only `[start, end)`, so a READ pays for the bytes
it answers with; `Image::encode` is that same layout over the whole image, so
there is one implementation of the layout rather than two that can drift.

`edit.rs`'s spill and `edit_resident` still need the whole image and keep
calling `encode()`. Whether a READ is served from a window or from the whole
image is not observable: the reply bytes are the same either way.

Review: finding 7 (full re-encode on every READ).

Tests: new `any_window_of_the_image_encodes_to_that_window` compares every
window of an image (a padding byte the client left behind, values of several
sizes, the empty resource fork) with the same slice of `encode()`, including
empty windows, a window past the image and one that starts after it ends.
`cargo test -p layerfs-nfs --test apple_double_codec --test apple_double_decode`:
7 + 1 passed, 0 failed. Anti-proof: with `encode_range` clamped to start at 0
again the window test fails (`left` starts at image offset 0 while the expected
window starts mid-image).

Production LOC: 163886 -> 163908 (delta +22)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent 584cc3e (163886) and of the staged tree
20b10a3 (163908). Engine +22. Reference absent (retired R9c; last counted
65,417 at tree 498dd19).
…t (lp73)

`DECODED_BYTES` (512 MiB per export) was memory nothing could evict: many
Workspaces can be mounted at once, so a per-export budget that is never paid
back does not fit. Decoded images now take the same `RESIDENT_BYTES` (32 MiB)
as bodies kept as written, a body at its length and a decoded one as
`Image::cost`, so the budget sees every byte the export holds.

`Companion::resident_cost` is that one number; `attach`, `detach` and
`edit_body` count it, `edit_body` decides with it (a body either fits or the
whole edit answers `NFS3ERR_NOSPC` and leaves the companion as it was), and
`classify` no longer consults a second budget. `DECODED_ENTRY_BYTES` stays, now
as the most `Image::cost` one decoded companion may be counted for: a whole
image costing more keeps the client's bytes, so one companion cannot take the
whole pool. This layer has no disk to spill to, so a body the budget does not
take is still refused; the next layer spills it.

`Released` reports the merged pool as `companion_bytes` and keeps
`companions_decoded` as a count, instead of `companion_decoded_bytes` beside a
separate resident total.

Review: owner decision 3 (`DECODED_BYTES` rejected), first half (the budget and
the counts); the spill half is the next layer.

Tests: `the_clients_write_sequence_is_served_byte_for_byte` now asserts the
merged pool: the decoded image's `Image::cost()` after the image arrives, the
written body's length while it does not decode, the cost again once it does, and
zero after the companion is removed. `cargo test -p layerfs-nfs`: 42 passed,
0 failed. Anti-proof: with a decoded body counted as zero again the test fails
at the first assertion (`left: 0, right: 4372`).

Production LOC: 163908 -> 163896 (delta -12)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent b01e805 (163908) and of the staged tree
bfc489b (163896). Engine -12. Reference absent (retired R9c; last counted
65,417 at tree 498dd19).
The layer below changed the read path, the budget and the counters; this layer
had moved the read and the edit into `apple_double/edit.rs` and given the budget
a spill directory. Conflicts resolved by keeping this layer's structure with the
layer below's semantics:

- `Image::encode_range` serves a READ of a decoded body (the read moved to
  `edit.rs`, so the window it now asks for is applied there);
- `DECODED_BYTES` stays deleted: `Holding` and `Released` report one
  `resident_bytes`/`companion_bytes` pool plus the decoded count, and
  `keep_decoded` asks the resident budget, not a second one;
- `classify` keeps this layer's "a body on disk stays as the spill left it" and
  drops the decoded budget check (the per-entry cap stays);
- `adopt_spilled`, `oldest_resident` and this layer's edit path are unchanged
  here.

`a_decoded_companion_is_edited_under_a_full_resident_budget` sets a 5,120-byte
budget: it takes the image's attributes (4,369) but not its 8 KiB of bytes, which
is what the merged budget now means.

Tests: `cargo test -p layerfs-nfs`: 47 passed, 0 failed (raw `test result` sum
48: `spilled_companions_outnumber_the_open_file_limit` runs itself again with
`--exact` in a child process, whose line is counted too).

Production LOC: 164334 -> 164340 (delta +6)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent 8183f89 (164334) and of the staged tree
(first-parent merge comparison, 164340). Engine +6. Reference absent (retired
R9c; last counted 65,417 at tree 498dd19).
With decoded images counted against `RESIDENT_BYTES`, a decoded body is memory
like any other and has to leave memory like any other. `oldest_resident` now
takes the least recently used companion whose `resident_cost` is not zero,
which is a body kept as written or a decoded one, so eviction reaches decoded
bodies. Writing a decoded body out needs nothing new: `spill_one` and
`spill_gated` already encode it and write those bytes, which are exactly the
image the client wrote.

A whole image that does not fit the budget is therefore no longer refused
(`NFS3ERR_NOSPC`) nor kept decoded beyond it: it goes to its spill file and is
edited there, as any body does.

Review: owner decision 3 (`DECODED_BYTES` rejected), second half (the spill).

Tests: new `eviction_takes_the_decoded_body_before_the_written_ones` under a
4,096-byte budget: a decoded image of 213 bytes, then two 2,000-byte bodies.
The decoded body is the least recently used memory, so it is the one spilled
(`companions_decoded` 1 -> 0), the two written bodies stay resident (`4000`
bytes), and the image reads back byte for byte from its file. Anti-proof: with
`oldest_resident` filtered to bodies kept as written again, the test fails
(`companions_decoded` stays 1 and the written body is evicted instead).

Production LOC: 164340 -> 164337 (delta -3)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent 7a9f20c (164340) and of the staged tree
(164337). Engine -3. Reference absent (retired R9c; last counted 65,417 at tree
498dd19).
Eviction walked every companion to find the least recently used one, and
`spill_for` repeated that walk up to `SPILL_ROUNDS` times, so one edit could
scan the index under the companion lock several times.

`Companions::by_use` is now a `BTreeSet<(used tick, serial)>` of the companions
holding memory: `attach` inserts an entry when a body takes budget, `detach`
removes it, and `set_used` (behind `use_serial`, the guest's last touch of the
body) moves it. `oldest_resident` is the first entry, skipping the caller's own
serial, so a round costs one step into the order (O(log n)) and no walk.
`spill_for` no longer bounds its rounds by the number of companions: it takes
the oldest body, and if that body changed under it the other call has already
moved its entry to the newest end, so the next round looks at a different one;
the rounds end when nothing holding memory is left. `SPILL_ROUNDS` goes away.

The companion lock now does O(1) or O(log n) work per step with no I/O:
`attach`/`detach`/`set_used` one set operation, `oldest_resident` one lookup,
and the file written before the lock is taken and renamed after it is dropped.
The kernel table lock is untouched by any of this.

Review: blocking finding 2 (eviction is an O(n) scan under the lock, repeated).

Tests: new `eviction_follows_the_order_of_use_not_the_shape_of_the_index`: 64
companions of 16 bytes under a 1,024-byte budget, three of them read again, then
eight more; the spill directory holds exactly the serials of the eight least
recently used ones, in that order, and the three read again stay resident.
Anti-proof: with `oldest_resident` reversed the test fails, spilling
`SERIALS+2, +64..+70` (the newest) instead of `SERIALS+3..+10`.

Production LOC: 164337 -> 164342 (delta +5)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent e323c0e (164337) and of the staged tree
(164342). Engine +5. Reference absent (retired R9c; last counted 65,417 at tree
498dd19).
…e lock (lp73)

`Companions::adopt_spilled` ran `fs::rename` while holding the companion lock,
so the module documentation's "file I/O never holds the companion lock" was not
true of the step that puts the file in place.

The adoption is now three steps with the lock dropped between them:
`Companions::spill_target` (lock) answers the companion's own file and whether
the body is still the one the file was written from, `fs::rename` runs outside
the lock, and `Companions::commit_spilled` (lock) switches the body to
`Body::Spilled` with its length. A reader keeps reading the body in memory until
the rename is done, so no reader ever sees a half written or missing file.

The exclusion the rename needs is the companion's own edit gate, which already
means "one edit of this companion at a time": a gated edit holds it across its
own adoption, and eviction now takes it with `try_lock`, skipping a companion
another call is editing (it never waits for another call's I/O, so no two spills
of one companion can meet at the file). `spill_for` therefore passes over a
victim it could not spill instead of picking it again, and a companion that is
gone is nothing to spill rather than an error.

Review: finding 5 (disk I/O under the companion lock); the module docs of
`edit.rs` and `apple_double_spill.rs` and `nfs.md` now state what the code does.

Tests: `cargo test -p layerfs-nfs`: 48 passed, 0 failed (raw `test result` sum
49 with the child-process rerun of
`spilled_companions_outnumber_the_open_file_limit`), including the spill, race
and file-limit cases that adopt bodies.

Production LOC: 164342 -> 164372 (delta +30)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent 73f62ef (164342) and of the staged tree
(164372). Engine +30. Reference absent (retired R9c; last counted 65,417 at tree
498dd19).
jiangkoumo and others added 23 commits November 14, 2026 18:02
… names (lp73)

macOS 27.2, build 525fcfb, reproducible 20/20 and 5/5: a file carrying
attributes, `ln a b`, then `rm a`, leaves `b` with no attributes at all and
`._b` answering `NFS3ERR_NOENT`, while the companion is still in the export's
index. The companion was the inode's, so nothing about the inode changed; what
the client sent is what removed it.

Reproduced in the rig with the client's sequence: `/bin/ln a b` links the main
file, then the client drops the `._` name of the name it no longer uses with a
REMOVE of `._a`, and REMOVE took the whole companion: `held().companions` 1 to
0, `._b` `NFS3ERR_NOENT`, the attributes gone. A client that carries the
attributes to the new name also sends a LINK of the companion name (`._a` to
`._b`), which this export refused with `NFS3ERR_ACCES`.

- A companion now holds every `._` name of the inode it was linked under
  (`Companion::also`, indexed and unindexed with its own name, one name promoted
  when the recorded one goes).
- `companion_link` serves LINK of a companion serial: another `._` name of the
  same companion, `NFS3ERR_EXIST` for a name already held or another
  companion's, `NFS3ERR_ACCES` for a name of the Workspace.
- `companion_remove` drops one name while the companion holds another, or while
  its inode has another name the export knows (`Table::names` plus
  `Table::aliases`); only the last name of a one-name inode is the client
  clearing the attributes, which still takes the companion, as does any name of
  a companion no inode owns.

Review: macOS re-test, finding 1 (hard link then remove the original name).
This is the same rule the hard-link contract already stated, which the name
scoped REMOVE broke.

Tests: two new. `a_hard_link_keeps_the_companion_under_its_other_names`: main
file and attributes, LINK, REMOVE `._a`, then REMOVE `a`; `._b` reads the same
companion and bytes at every step and the companion goes with `b`.
`a_linked_companion_name_is_another_name_of_the_same_companion`: LINK of the
companion name succeeds and reads the same bytes, a held name answers
`NFS3ERR_EXIST`, a plain name `NFS3ERR_ACCES`, REMOVE of `._a` keeps `._b`, and
REMOVE of a one-name inode's only `._` name still removes the companion.
Anti-proofs: with the keep rule forced off both tests fail (`left: 0, right: 1`
companions, the attributes gone), and with LINK of a companion back to
`NFS3ERR_ACCES` the second fails on that LINK. `cargo test -p layerfs-nfs`: 71
passed, 0 failed (raw sum 72 with the child-process rerun).

Production LOC: 164797 -> 164884 (delta +87)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent f6cb678 (164797) and of the staged tree
(164884). Engine +87: the companion's extra names and the three index
operations on them, `companion_link`, the name-count rule in
`companion_remove`, and the routed LINK. Reference absent (retired R9c; last
counted 65,417 at tree 498dd19).
…(lp73)

macOS 27.2, build 525fcfb: in one run of 600 files carrying ~100 KiB of
attributes each, the first 507 in write order read back with no attributes at
all while the export still held all 652 companions, 326 bodies stayed on disk,
and two re-runs of the same recipe plus two controls were 100% correct. Nothing
in the export's own counts had moved, so the loss was not an eviction: either
those companions were still held but no name reached them, or the client never
got them back for a reason of its own. The static review found no path that
removes a companion's name keys and keeps the companion - every one of them
takes the companion with it, or leaves it reachable through its inode - so the
run could not be reproduced from the code, and a loss with the index intact has
to be told apart from one where the index lost the entry.

`Companions::unreachable` counts the companions neither their recorded name nor
another name of their inode reaches, and the export's release line now carries
it: `held= slotted= decoded= resident_bytes= unreachable=`. It runs once, over
the companions held at release, so it costs nothing on the serving path. A
non-zero count beside a client that lost attributes says the name, not the body,
was lost; a zero says the body itself was gone. Documented with the release
line in the nfs route's AppleDouble section.

Review: macOS re-test, finding 2 (507 of 600 attributes lost, not reproduced).

Tests: none new. The counter can only be non-zero through the very defect it
reports, so an ordinary test would assert zero; the release path is covered by
every test that stops an export (the line is printed with `held=`).

Production LOC: 164884 -> 164900 (delta +16)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent 45d0903 (164884) and of the staged tree
(164900). Engine +16: the counting method and the release line's field.
Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
The `HELD` bound was decided in one place, but on the path that needs I/O
between the decision and the commit - a WRITE, SETATTR or RENAME whose body
spills and is then edited on its file - `admits` passed before `spill_gated`
and the file write, and `record_edit`/`classify` took the slot afterwards
without asking again. Two concurrent calls could both be told the last slot was
free and both take one, so `slots.len()` could exceed `held_limit` by as much as
the number of edits in flight, which contradicts "the bound holds".

`Companions::reserve` decides and holds in one step under the companion lock
(`Slot::Taken`/`Held`/`Full`); reservations count against the bound in
`admits`, and a create or reclaim cannot steal one. The disk-edit path and
`rename_companion` reserve before their I/O and give the reservation back if
the edit does not land (`unreserve`, `take_serial`, the failure paths of
`apply_edit`/`record_edit`/`rename_companion`), so a reservation can neither be
held for good nor outlive its companion. CREATE and the in-memory edit decide
and take the slot under one lock and keep using `admits`.

Review: finding P2-1 (the bound could be exceeded under concurrency), and the
route document's claim that the bound holds is now true as written.

Tests: new `the_slot_bound_holds_when_writes_claim_the_last_slot_together`.
Eight companions holding whole images (no slot), one slot, a body budget that
cannot hold the broken body, and one breaking WRITE from each over its own
connection after a barrier: exactly one writer takes the slot, the other seven
answer `NFS3ERR_NOSPC`, `slotted` is 1, and every companion still reads back as
its image or that edit. Anti-proof: with `admits` back in place of `reserve` the
test failed 23 of 40 runs with two or three writers succeeding (`[Ok(()),
Err(28), Ok(()), ...]`); with the reservation it passed 40 of 40.

Production LOC: 164900 -> 164930 (delta +30)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent c96af5b (164900) and of the staged tree
(164930). Engine +30: the reservation set, `Slot`, `reserve`/`unreserve`, the
two decision sites and the three release paths. Reference absent (retired R9c;
last counted 65,417 at tree 498dd19).
Reclamation asks the engine about a candidate's main file before dropping it,
and treated only "no such file" as dead. A main file whose directory the client
had removed cannot answer "no such file": the engine has no such parent, so the
lookup answers `NFS3ERR_STALE` (`NFS3ERR_NOTDIR` for a parent that is a file),
and the candidate was kept for the export's life. Its slot stayed taken, every
later CREATE at the bound repeated the same failing lookup, and the bound could
stay full until release.

`NFS3ERR_STALE` and `NFS3ERR_NOTDIR` now count as "the main file is not there
either" and the candidate is reclaimed, which is what a removed parent means
for its entries. Any other answer still leaves the companion alone, and the
per-name engine lookup and the slot walk are unchanged.

Review: finding P2-3 (a candidate whose main file's directory is gone is never
reclaimed).

Tests: new `a_companion_of_a_removed_directory_is_reclaimed`: one slot, a
companion inside a directory whose main file `x` never existed (so `inode: None`
and the slot), the client removes the directory (`rmdir` succeeds - nothing in
it is visible to the engine), then a CREATE at the bound: its reclamation's
lookup answers `NFS3ERR_STALE`, the companion is dropped and the CREATE lands
with one companion and one slot. Anti-proof: with only `NFS3ERR_NOENT` counted
the same CREATE answers `NFS3ERR_NOSPC` (`Err value: 28`) and the test fails
there, so the lookup really answers a stale parent rather than a missing entry.

Production LOC: 164930 -> 164932 (delta +2)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent 386c5b0 (164930) and of the staged tree
(164932). Engine +2: the extra answers in the reclaim's match. Reference absent
(retired R9c; last counted 65,417 at tree 498dd19).
…vershoot (lp73)

Two statements about the resident budget were wrong in the direction of
claiming more than the code does. A decoded image's `Image::cost` against
`RESIDENT_BYTES` is a cap on what one decoded body is charged, not a limit on
what a companion may store: an image costing more than `DECODED_ENTRY_BYTES` is
kept as the client wrote it and charged at its length. And an edit that leaves
a body whole decodes it as it records it rather than asking for room first, so
the pool may stand one decoded body over the budget until the next edit needs
room - "never memory the budget cannot see" was too strong.

The module documentation, the `DECODED_ENTRY_BYTES` item and the
`RESIDENT_BYTES` row of the nfs route now say that. The `DECODED_ENTRY_BYTES`
value stays 128 KiB: with the images a macOS client lays out it is what keeps a
many-attribute image from being charged a whole 4 KiB unit's worth of body for a
few dozen bytes of attributes.

Review: finding P3-1 (the wording of the un-budgeted decode) and the D3 note on
`DECODED_ENTRY_BYTES` (owner confirmation is asked for in the pull request).

Tests: none (documentation only). `cargo test -p layerfs-nfs`: 73 passed, 0
failed (raw sum 74 with the child-process rerun).

Production LOC: 164932 -> 164932 (delta 0)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent 0b87716 (164932) and of the staged tree
(164932). Engine 0: documentation comments only. Reference absent (retired R9c;
last counted 65,417 at tree 498dd19).
A companion can hold several `._` names of its inode (a LINK of a companion
name adds one), but reclamation's candidate test read only the name a companion
records. A companion the client never paired with a main inode (`inode: None`)
that was then linked under a second name was therefore proposed as an orphan on
the strength of the first name alone: the export looked up that name's main
file, answered "no such file", and dropped a companion that its other name -
whose main file the export does know - still reached. One linked name lost the
attributes of the other.

`orphans()` now walks every name a companion holds: one whose main file a name
of the index holds keeps the companion, and only a companion none of whose
names the index knows becomes a candidate, for the lookup of the first such
name. The slot walk, the engine lookup and the decision that follows are
unchanged.

Tests: new `a_companion_reached_by_one_of_its_names_is_not_an_orphan`: one slot,
a companion created in a directory for a main file that does not exist
(`inode: None`), LINKed under a second `._` name, then the main file of that
second name created; a CREATE at the bound reclaims, finds the companion
reachable through the linked name and leaves it, so the CREATE is refused and
the companion still reads back. Anti-proof: with the recorded name alone the
same CREATE lands (`left: Ok(()), right: Err(28)`) because the live companion
was reclaimed. `cargo test -p layerfs-nfs`: 74 passed, 0 failed (raw sum 75 with
the child-process rerun).

Production LOC: 164932 -> 164941 (delta +9)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent d114c13 (164932) and of the staged tree
(164941). Engine +9: the loop over a companion's names in `orphans`. Reference
absent (retired R9c; last counted 65,417 at tree 498dd19).
A macOS client that cannot read a companion's attributes looks the same from
the export whatever happened: a name it never asked for, a name the export's
index could not resolve, and an answer its own caches then dropped all leave no
trace behind. A run of the real client is the only way to tell them apart, so
this adds an opt-in trace of exactly those requests.

`LAYERFS_NFS_TRACE=1` (read once per process, off unless it is exactly `1`)
prints one line per request that names an AppleDouble companion - the `._`
names themselves and the main file names of the inodes that hold one - to the
daemon's standard error:

  layerfs-nfs trace: op=LOOKUP dir=6 name="._b" main=Some(9)
    route=inode companion=Some(4611686018427387904) status=Ok

`main` is the main inode the export's name index gave the name (or `None`), and
`route` says which part of the companion index answered - the companion's own
name key, the main inode, or neither - so a name no companion answers is
distinguishable from one whose main file the index forgot. A CREATE that finds
the companion already held reports whether it would truncate it, a SETATTR of a
`._` file is traced by handle with its size, `BIND` reports a main name the
kernel table re-attributed to an inode with a companion (`alias` while the
inode keeps several names, `unbound` when the one it held is all it has),
`UNPAIR` reports a main name whose paired `._` name the export dropped, and
`RENAME`/`LINK`/`REMOVE`/`CREATE` report the companion they decided. Nothing
else changes: no state is touched, and every hook costs one `OnceLock` load
while the trace is off.

Tests: no new test - the trace changes no behaviour and the hooks are on paths
`cargo test -p layerfs-nfs` (74 passed, 0 failed, raw sum 75) already drives;
checked by hand against a LINK'd companion's `._` names, where the lines above
are the actual output. `lf-ci.sh` fast: 2017 passed, 0 failed.
clippy -p layerfs-nfs --all-targets: no warning.

Production LOC: 164941 -> 165152 (delta +211)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded. Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on the first parent 54b9c1b (164941) and on the staged tree (165152). Engine
+211: `trace.rs` and its hooks in `apple_double.rs`, `edit.rs`, `kernel.rs` and
`change.rs`. Reference absent (retired R9c; last counted 65,417 at tree
498dd19).
`Kernel::companion_drop_for` dropped the `._<name>` key when a main file
name was removed, so `rm a` of a file `b` still named answered `._a` with
NFS3ERR_NOENT although the inode and its companion were alive (a macOS
27.2 trace showed the shape three times in one matrix).

A companion that belongs to an inode now keeps that `._` name until the
inode's last name, which already takes the companion with every `._` name
it holds (`companion_drop_inode` -> `take_serial` -> `unindex`); a
companion no main inode owns goes as before. A new file under the freed
name reaches the old companion nowhere: `Companions::resolve` takes a name
key only for the inode its companion belongs to.

`Holding` and `Released::companions_unreachable` carry the release count,
so a status read reports the same number as the release line.

Tests (tests/apple_double_decode.rs): the matrix's g2/g3/g5 orders (`ln`,
`rm` before and after one read of the linked name), a new file under the
removed name, and the inode's last name reclaiming the companion and every
`._` name key, with `._a`/`._b` reading one whole image. The three order
tests failed on the parent with `._a` = NFS3ERR_NOENT.

Docs: nfs.md states the hard-link rule, `UNPAIR name-kept`, the native
route's `env_clear` trace wrapper and the live `unreachable` count.

Production LOC: 165152 -> 165155 (delta +3); scope: first-party product
source under core|engine|control|api, tools/production_loc.py on the parent
tree and the staged tree; reference absent (retired R9c; last counted
65,417 at tree 498dd19).
No conflicts. Upstream since `8cfb6e273` brought the standalone fuser Git fork
(#686, which removes `engine/vendor` and `tools/check_fuser_integrity.py`) with
its pin guard (#687), and the GC spec items (#688). None of those files is this
layer's, so the merged tree is upstream's plus this branch's commits.

Production LOC: 163901 -> 163901 (delta 0)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded (the removed vendored
fuser is third-party code and never counted). Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent 428ffc3 (163901) and of the merged tree
(163901). Reference absent (retired R9c; last counted 65,417 at tree
498dd19).
… (lp73)

The codec layer now carries `upstream/layerfs_v2_0_0` at `cb8ced9d3`; that
merge had no conflicts, and neither does this one. Upstream's changes since
`8cfb6e273` (the standalone fuser Git fork, which removes `engine/vendor` and
`tools/check_fuser_integrity.py`, its pin guard, and the GC spec items) touch
no file of this layer, so the result is the upstream tree plus this layer's
commits.

Production LOC: 164405 -> 164405 (delta 0)
Scope: first-party core/engine/control/api production source and shipped
runtime SQL; tests, docs, tools and manifests excluded (the removed vendored
fuser is third-party code and never counted). Method:
python3 tools/production_loc.py (SHA-256
 d3148db88f91b09eb74eb659b2f700209776a24d022f09774dc7bb38ea1eecf6)
on git archives of the first parent aec83af (164405) and of the merged tree
(164405). Reference absent (retired R9c; last counted 65,417 at tree
498dd19).
The macOS NFS client answers a `._` name that is absent from a directory's cached READDIRPLUS listing with "not found" without ever asking the server (the skipdu path in nfs_vnop_lookup), and this export serves AppleDouble companions without listing them, so a hard-linked file's extended attributes could read back empty.

Turning READDIRPLUS off makes every `._` name arrive as a LOOKUP. Nothing else about the mount or the listing changes.

Production LOC: 165155 -> 165155 (delta 0); core/engine/control/api production source only (api/crates/layerfs-*/src, api/python/src/layerfs, api JS packages), counted with tools/production_loc.py at the parent and the committed tree; comments, tests and docs excluded.
…Lab/layerfs-project#76)

Merges the PR head e12c5cf unchanged; review fixes follow in their own commits.

Production LOC: 169609 -> 169851 (delta +242)
  core     39,795 -> 39,795 (0)
  engine   75,032 -> 75,274 (+242)  layerfs-daemon +4, layerfs-mount-ports +17, layerfs-nfs +64, layerfs-workspace +157
  control  48,436 -> 48,436 (0)
  api       6,346 ->  6,346 (0)
  reference absent (retired R9c; last counted 65,417 at tree 498dd19)
  Method: tools/production_loc.py --detail on first parent dd4a2cc and the merged tree (merge: first-parent comparison)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merges the PR head ff28f61 unchanged; review fixes follow in their own commits.

Production LOC: 169851 -> 170247 (delta +396)
  core     39,795 -> 39,795 (0)
  engine   75,274 -> 75,670 (+396)  layerfs-nfs +396
  control  48,436 -> 48,436 (0)
  api       6,346 ->  6,346 (0)
  reference absent (retired R9c; last counted 65,417 at tree 498dd19)
  Method: tools/production_loc.py --detail on first parent e7349ff and the merged tree (merge: first-parent comparison)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… area (lp73)

Merges the PR head e3c80ed unchanged; review fixes follow in their own commits.

Production LOC: 170247 -> 170751 (delta +504)
  core     39,795 -> 39,795 (0)
  engine   75,670 -> 76,174 (+504)  layerfs-daemon +15, layerfs-nfs +489
  control  48,436 -> 48,436 (0)
  api       6,346 ->  6,346 (0)
  reference absent (retired R9c; last counted 65,417 at tree 498dd19)
  Method: tools/production_loc.py --detail on first parent 7c0f937 and the merged tree (merge: first-parent comparison)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…p73)

Merges the PR head 1cdb96a. One conflict, the tail of Kernel::lookup in
engine/crates/layerfs-nfs/src/kernel.rs: #637 returns the job's directory row
with the entry (`Answer`), #722 traces the main file; the resolution keeps
both. Kernel::reclaim_orphans adapts to #637's `observe`, which now answers
`Answer` / `Refused` instead of a tuple / `Status`, with no change in what it
decides. Review fixes follow in their own commits.

Production LOC: 170751 -> 171501 (delta +750)
  core     39,795 -> 39,795 (0)
  engine   76,174 -> 76,924 (+750)  layerfs-nfs +750
  control  48,436 -> 48,436 (0)
  api       6,346 ->  6,346 (0)
  reference absent (retired R9c; last counted 65,417 at tree 498dd19)
  Method: tools/production_loc.py --detail on first parent efa138d and the merged tree (merge: first-parent comparison)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…637 review)

#637 stopped LOOKUP answering `directory_wcc` (the export's last observation
of a directory, which nothing ages) except on one path: a LOOKUP of an
AppleDouble companion still answered it. Finder asks for `._` names all the
time, so an install or another writer's change to the directory kept reaching
the client as the old mtime/nlink. That LOOKUP is answered by the export alone
and no job reads the directory, so it now sends no `dir_attributes`, as `.`,
`..` and silly names already do.

A failed LOOKUP's directory row (#637's `Refused::directory`) is now kept as
the export's observation, as a successful LOOKUP's row is, so a companion
CREATE/REMOVE/RENAME `wcc_data` after it answers those facts rather than
older ones.

Docs: nfs.md's LOOKUP row names the companion and the kept observation.
Test: new `export.rs::a_companion_lookup_never_answers_a_stale_directory_observation`.
It fails on the parent (b9b2a31) with the stale `nlink 2` directory and passes here.

Production LOC: 171,501 -> 171,498 (delta -3)
  core     39,795 -> 39,795 (0)
  engine   76,924 -> 76,921 (-3)  layerfs-nfs -3
  control  48,436 -> 48,436 (0)
  api       6,346 ->  6,346 (0)
  reference absent (retired R9c; last counted 65,417 at tree 498dd19)
  Method: tools/production_loc.py --detail on parent b9b2a31 and the staged tree

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…review)

#722 left engine/crates/layerfs-nfs/src/apple_double.rs at 1,272 physical
lines, and tools/check_product_boundary.py failed on it. This is relocation
only, with no behaviour change:

- `impl Companions` (names, inodes, slots, resident budget, eviction order)
  moves to `apple_double/index.rs`;
- `Kernel::orphans` and `Kernel::reclaim_orphans` move to
  `apple_double/reclaim.rs`;
- moved methods become `pub(super)`, and each file imports what it uses.

The moved lines are otherwise byte-identical (checked by comparing the sorted
non-import lines before and after). apple_double.rs is now 809 lines,
index.rs 345, reclaim.rs 141. The boundary check passes (1,417 files).

Docs: nfs.md's AppleDouble section lists the two new files.
Tests: none changed.

Production LOC: 171,498 -> 171,516 (delta +18; relocation: the new files'
own imports)
  core     39,795 -> 39,795 (0)
  engine   76,921 -> 76,939 (+18)  layerfs-nfs +18
  control  48,436 -> 48,436 (0)
  api       6,346 ->  6,346 (0)
  reference absent (retired R9c; last counted 65,417 at tree 498dd19)
  Method: tools/production_loc.py --detail on parent 9db8f4d and the staged tree

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ion (#722 review)

Kernel::reclaim_orphans decides outside both locks: each candidate whose main
file is unknown costs one engine lookup, and the commit then took the lock
again and re-checked only the companion's inode and slot. A call that made the
companion live in that window (a RENAME of `._x` onto a name whose main file
exists, a CREATE of the main file followed by a WRITE to `._x`, a READ) did
not stop the drop. The binding half could also overwrite another companion's
entry for the same main inode.

`Orphan` now carries the companion's `used` tick from the decision, and
`Companions::unchanged` requires it, the same inode and the slot still taken,
both before dropping a companion and before binding one back. A main inode that
already has its companion is not rebound; the candidate stays and is looked up
again at the next reclaim. Every call that reaches a companion moves its tick.

Review finding rejected after testing: the review said a lookup in a directory
whose reference the export had collected would answer `Stale` and so drop a
live companion. It does not: the engine answers the lookup in a collected
directory. The new test collects the directory (a later GETATTR of it
resolves it again) and passes both before and after this change, so it is
coverage, not a regression test.

Test: new `apple_double_decode.rs::a_companion_in_a_collected_directory_is_kept_while_its_main_file_is_there`.
The race itself has no deterministic client-side reproduction (the window is
between the out-of-lock engine lookup and the commit), so no test reproduces it.

Production LOC: 171,516 -> 171,525 (delta +9)
  core     39,795 -> 39,795 (0)
  engine   76,939 -> 76,948 (+9)  layerfs-nfs +9
  control  48,436 -> 48,436 (0)
  api       6,346 ->  6,346 (0)
  reference absent (retired R9c; last counted 65,417 at tree 498dd19)
  Method: tools/production_loc.py --detail on parent fb28450 and the staged tree

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- SETATTR of a companion runs as one edit. It checked its guard under one
  lock, resized under a second and set mode/mtime under a third, so a
  guarded SETATTR could pass its ctime check and resize a body a WRITE had
  changed in between. It now holds the companion's edit gate from the guard
  to the mode and time, and a WRITE reads its pre-op attributes under the
  gate (`edit_gate` + `edit_gated` replace `edit_companion`). A REMOVE
  takes no gate, so a SETATTR racing one can still answer `Stale` after
  resizing a body that is gone with its companion; nothing is left behind.
- A whole image is decoded only when its decoded form costs less than its
  bytes (`worth_decoding`). Each attribute is charged its name, value and
  56 bytes, so an image of many small attributes cost several times its
  length decoded, could be refused room its bytes had, and was charged more
  than it held.
- An edit of a decoded body asks for the room its bytes add beyond the
  decoded charge it gives back, not for its whole length, so overwriting or
  shrinking one no longer spills more than it needs.
- `Image::parse` bounds its first allocation by what the image can hold
  rather than by the client's entry count (up to 65,535 entries).

The resident budget stays at 32 MiB (owner decision 2026-11-17); #713's
description called it upstream's, but the base branch had 64 MiB. The
architecture page already says 32 MiB.

Not changed: every WRITE to a decoded body still re-encodes and re-parses
the whole body under the companion lock; that needs an in-place decoded edit.

Docs: nfs.md (the DECODED_ENTRY_BYTES row, and the SETATTR and WRITE gate)
and the module docs of apple_double.rs.
Test: new `apple_double_decode.rs::an_image_whose_attributes_cost_more_than_its_bytes_is_kept_as_written`
(600 two-byte names). It fails on the parent (77e7368), where the image is decoded
(decoded count 1), and passes here. The SETATTR race has no deterministic
client-side reproduction, so it has no test.

Production LOC: 171,525 -> 171,523 (delta -2)
  core     39,795 -> 39,795 (0)
  engine   76,948 -> 76,946 (-2)  layerfs-nfs -2
  control  48,436 -> 48,436 (0)
  api       6,346 ->  6,346 (0)
  reference absent (retired R9c; last counted 65,417 at tree 498dd19)
  Method: tools/production_loc.py --detail on parent 77e7368 and the staged tree

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A WRITE that grows a spilled body no longer shows its new length before its
  bytes. `apply_edit` wrote the body out pre-extended to the edit's length,
  and `commit_spilled` recorded that length before `edit.disk` wrote anything.
  A GETATTR or READ in between (neither takes the edit gate) saw the larger
  size and read zeros, and a failed edit shrank it back, so a client could see
  a length no WRITE made. The body is now written out at the length it has,
  the edit grows its file itself, and `record_edit` moves the length after the
  bytes are there. `restore_len` and the length it saved are gone, and with
  them the early `Stale` return that skipped `unreserve` after a reservation.
- `spill_target` never replaces a body that is already on disk, whoever asks.
  Before, a spill with no `used` snapshot renamed its file onto the live one
  first and found out afterwards. That was unreachable while the edit gate
  holds, but it would have deleted the live body.
- With no spill directory, `spill_for` stops at once instead of copying or
  encoding every resident victim under the companion lock only to fail on each.
  One victim whose file cannot be written is still passed over for the next
  (`an_eviction_victim_that_cannot_be_written_out_does_not_fail_the_writer`
  keeps that).
- `spill_applies` was public only for a test that fed it integers: both are
  removed, and `spill_target` compares the tick itself. Deleted test:
  `write.rs::a_spill_replaces_a_companion_body_only_when_no_call_changed_it`.
  The behaviour it named holds: every body change runs under the edit gate,
  and a spill takes the gate with `try_lock` and checks the `used` tick.
- `spilled_companions_outnumber_the_open_file_limit` waits for its child with
  a 90 s deadline and kills a child that hangs, instead of `status()`, which
  waits indefinitely.

Docs: nfs.md, the paragraph on one edit per companion.
Tests: no deterministic client-side test reproduces the GETATTR-during-spill
window. The covering suites pass on macOS (apple_double_decode 31, write 15).

Production LOC: 171,523 -> 171,508 (delta -15)
  core     39,795 -> 39,795 (0)
  engine   76,946 -> 76,931 (-15)  layerfs-nfs -15
  control  48,436 -> 48,436 (0)
  api       6,346 ->  6,346 (0)
  reference absent (retired R9c; last counted 65,417 at tree 498dd19)
  Method: tools/production_loc.py --detail on parent 519f8db and the staged tree

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…722 review)

- RENAME of one of a companion's names moves only that name.
  `rename_companion` never got the source name: it overwrote the companion's
  primary name and kept every LINK-made name, the renamed one included. With
  `._a` linked as `._b`, `mv b c` and `._b -> ._c` left `._b` answering
  and dropped `._a`'s own key. A companion reached only through its inode
  keeps the names it holds and gains the target.
- A slot reservation is never left for a companion that is gone.
  `Companions::reserve` takes nothing for a serial another call removed (a
  REMOVE takes no edit gate), and `record_edit` gives the reservation back
  before it can find the companion gone. Either path used to leave the bound
  one slot smaller for the life of the export.
- macOS mounts no longer add `nordirplus` (owner decision 2026-11-17). This
  reverts 46550ff: the loss it targeted reproduced 0/5 with and without the
  option, and every READDIR then costs a LOOKUP per entry. The hard-link loss
  with READDIRPLUS on stays an open, unconfirmed risk.

Review finding not taken: making REMOVE of `._a` drop the whole companion
while `a` still exists (to serve `xattr -c` on a hard-linked file). The
client's own sequence for `ln a b` removes `._a` while `a` exists and
expects the attributes under `b` to stay. That is the macOS loss #722 fixed,
and `a_hard_link_keeps_the_companion_under_its_other_names` pins it.

Docs: nfs.md, the companion names paragraph and the macOS client options.
Test: new `apple_double_decode.rs::renaming_one_linked_companion_name_moves_only_that_name`.
It fails on the parent (3466c3d), where `._b` still answers, and passes here. The
reservation race has no deterministic client-side reproduction.

Production LOC: 171,508 -> 171,521 (delta +13)
  core     39,795 -> 39,795 (0)
  engine   76,931 -> 76,944 (+13)  layerfs-nfs +13
  control  48,436 -> 48,436 (0)
  api       6,346 ->  6,346 (0)
  reference absent (retired R9c; last counted 65,417 at tree 498dd19)
  Method: tools/production_loc.py --detail on parent 3466c3d and the staged tree

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…itionally

- rustfmt on apple_double/index.rs and the two NFS test files changed by
  the review fixes.
- engine/crates/layerfs-store-service/tests/postgresql_readdirplus_window.rs
  (from #637) was behind `#![cfg(feature = "postgresql")]`, but #726 on the
  base branch removed that feature and always compiles the PostgreSQL engine.
  Clippy refused the unknown cfg value, and the file would never have run.
  The gate is removed, as in the other postgresql_* tests, which report
  NOT_RUN without LAYERFS_TEST_POSTGRES_URL.

Production LOC: 171,521 -> 171,524 (delta +3; rustfmt line breaks)
  core     39,795 -> 39,795 (0)
  engine   76,944 -> 76,947 (+3)  layerfs-nfs +3
  control  48,436 -> 48,436 (0)
  api       6,346 ->  6,346 (0)
  reference absent (retired R9c; last counted 65,417 at tree 498dd19)
  Method: tools/production_loc.py --detail on parent fc40696 and the staged tree

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@yifanxuaaa
yifanxuaaa merged commit 5742338 into layerfs_v2_0_0 Nov 17, 2026
@yifanxuaaa
yifanxuaaa deleted the claude/lp73-integration branch November 17, 2026 04:28
jiangkoumo added a commit to jiangkoumo/layerfs that referenced this pull request Nov 17, 2026
Bring the Ephemeral-AI-Lab#136 native unmount helper reaping and detach-stage custody work
onto upstream/layerfs_v2_0_0 (5742338), including Ephemeral-AI-Lab#712/Ephemeral-AI-Lab#715/Ephemeral-AI-Lab#716/Ephemeral-AI-Lab#718
and the lp73 AppleDouble companion/spill series (Ephemeral-AI-Lab#713/Ephemeral-AI-Lab#714/Ephemeral-AI-Lab#722 via Ephemeral-AI-Lab#727).

Conflicts resolved keeping both sides' behavior intact:
- control/crates/layerfs-sandbox-backend/src/local/os/macos/darwin.rs:
  Keep upstream's Ephemeral-AI-Lab#712 `mount_line` and deadline-bounded `mount_table`
  together with Ephemeral-AI-Lab#644's shared `layerfs_bridge::process::Retained` queue
  (`RETAINED`), dropping the duplicate `Mutex<Vec<Child>>` / `reap_retained`
  / `retain` helpers so both `mount_table` and `force_unmount` reap
  through `RETAINED` (`reap`, `record` on synchronous reap, `retain` for
  background reaping on timeout). In `force::close` (Ephemeral-AI-Lab#716) and
  `machine_force::stop` (Ephemeral-AI-Lab#718), `exec::cancel::settle` cancels running
  workspace commands first, before `Sandbox::stop` -> `os::mounts::detach`
  -> `darwin::force_unmount`, avoiding overlap with `RETAINED`.
- engine/crates/layerfs-daemon/src/control/export.rs:
  Keep both `ExportServing::is_mounted` (Ephemeral-AI-Lab#644) and `companion_spill_dir` /
  `spill_root` (Ephemeral-AI-Lab#714), combine `.appledouble` leftover cleanup with the
  `mount_table`-aware `mounted_under` call in `ExportServing::reclaim`,
  and preserve `companion_spill_dir` in `ExportConfig` and `spill_root`
  rmdir after `ready.directory` removal in `teardown_export` (while
  `server.shutdown()` -> `Kernel::release()` removes the per-namespace
  spill dir first).

Checks: `tools/check_product_boundary.py` PASS; `tools/test_*.py` 180/180;
`lf-ci.sh fast` 2258/2259 (1 pre-existing upstream Ephemeral-AI-Lab#724 failure in
`layerfs-api::machine_files`, reproduced identically at 5742338);
`layerfs-nfs::helper_deadline` 8/8, `daemon` 2/2, `export` 10/10,
`apple_double_codec` 7/7, `apple_double_decode` 32/32; `layerfs-sandbox`
`record_codec` 5/5 & `unmount_record` 9/9; `layerfs-daemon` non-mount
31/31 & selected mount lane 12/12 binaries (36 passed, 1 ignored helper);
Clippy clean on changed crates; `rustfmt --check` clean on all changed
files (`fmt --all --check` fails only on 3 pre-existing upstream files at
5742338). `aarch64-apple-darwin` check blocked on Linux host C toolchain
(`blake3` `-arch arm64`): macOS build and live tests NOT_RUN.

Durable: NOT_RUN — disabled by owner until explicit reauthorization.
Production LOC: 163827 -> 171849 (delta +8022); first parent 89116b1.
  core 39787 -> 39795 (+8), engine 75121 -> 77198 (+2077),
  control 42842 -> 48510 (+5668), api 6077 -> 6346 (+269).
  Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
  Method: python3 -B tools/production_loc.py --detail on git archive of
  first parent 89116b1 and the merged working tree.
mengmengmengqiang pushed a commit that referenced this pull request Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants