Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/architecture/engine/kernel-routes/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ stops: a stopped mount starts nothing new and still gives back what it holds.

```text
LOOKUP/GETATTR/OPEN/OPENDIR KernelRead::prepare
loop: observe_visit -> Value | Refused | Needs(n): base(); supply; repeat
loop: observe_visit -> Value(Decided) | Refused | Needs(n): base(); supply; repeat
READ/READLINK KernelData::read
read_visit -> local window; base() only for inherited bytes
mutation KernelMutation::perform
Expand All @@ -85,7 +85,7 @@ stops: a stopped mount starts nothing new and still gives back what it holds.

| Type or module | Kernel requests | Source |
| --- | --- | --- |
| `KernelRead` | LOOKUP, GETATTR, OPEN, OPENDIR; holds nothing after its visits | [`lookup.rs`](../../../../engine/crates/layerfs-mount-ports/src/operations/lookup.rs) |
| `KernelRead` | LOOKUP, GETATTR, OPEN, OPENDIR; holds nothing after its visits (a LOOKUP also carries the directory it named, and a refused LOOKUP the directory row its own job read) | [`lookup.rs`](../../../../engine/crates/layerfs-mount-ports/src/operations/lookup.rs) |
| `KernelData` | READ, READLINK | [`read.rs`](../../../../engine/crates/layerfs-mount-ports/src/operations/read.rs) |
| `KernelMutation` | Every changing request; holds one `Publication` ticket until `replied` | [`mutation.rs`](../../../../engine/crates/layerfs-mount-ports/src/operations/mutation.rs) |
| `DirectoryStream` | READDIR; only accepted names become valid offsets | [`directory.rs`](../../../../engine/crates/layerfs-mount-ports/src/operations/directory.rs) |
Expand Down
21 changes: 17 additions & 4 deletions docs/architecture/engine/kernel-routes/nfs.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ and MOUNT are written over `std::net` with no third-party dependency and no
platform `cfg` ([`lib.rs`](../../../../engine/crates/layerfs-nfs/src/lib.rs)).
The daemon starts it and has the host's own NFS client mount it.

## Server (`server.rs`, `kernel.rs`, `block.rs`)
## Server (`server.rs`, `kernel.rs`, `read.rs`, `block.rs`)

```text
host NFS client (mount_nfs on macOS)
Expand Down Expand Up @@ -108,18 +108,22 @@ drawn once per export; it makes guessing costly and does not authenticate.
## Directories (`directory.rs`)

```text
READDIR / READDIRPLUS (handle, cookie, cookieverf, budget)
READDIR / READDIRPLUS (handle, cookie, cookieverf, maxcount)
cookie 0 -> open_directory: fresh descriptor; verifier = its id
cookie >0 -> descriptor (serial, verifier) still open? no: BAD_COOKIE
cookies: 1 = ".", 2 = "..", then engine-published offsets
one reply = at most one engine window; entries that fit the budget
one reply = at most one engine window; entries that fit maxcount
-> publish_cookies(those names); eof only when the window ended and fit
```

| Rule | Source |
| --- | --- |
| The cookie is the engine's kernel offset on one implicit directory descriptor named by `cookieverf`; a retired offset or a listing continued after an install is `NFS3ERR_BAD_COOKIE`, and the client lists again | [`directory.rs`](../../../../engine/crates/layerfs-nfs/src/directory.rs), [`kernel.rs`](../../../../engine/crates/layerfs-nfs/src/kernel.rs) |
| A READDIRPLUS entry carries the facts its listing already read and its handle; each handle takes its one lookup reference in the publishing visit | [`kernel_directory.rs`](../../../../engine/crates/layerfs-workspace/src/operations/kernel_directory.rs), [`kernel_cookie.rs`](../../../../engine/crates/layerfs-overlay/src/lifetime/kernel_cookie.rs) |
| A READDIRPLUS entry carries the facts its window read — a local inode's current row, or an inherited inode's immutable base stat — and the handle of its serial, so the client needs no LOOKUP per name; both are optional in RFC 1813 and this export sends both for every named entry whose offsets the reply publishes, each handle taking its one lookup reference in the publishing visit (reused offset ranges take none) | [`directory.rs`](../../../../engine/crates/layerfs-nfs/src/directory.rs), [`kernel_directory.rs`](../../../../engine/crates/layerfs-workspace/src/operations/kernel_directory.rs), [`kernel_cookie.rs`](../../../../engine/crates/layerfs-overlay/src/lifetime/kernel_cookie.rs) |
| A window's inherited inode values cost one batch inode demand for each level of the inode tree the cache does not already answer (`lookup_many` reads a level's pages in one `read_canonical_batch`, resident levels cost none) and its ordinary file lengths one batch length demand (`provide_lengths`) while the cache can remember them (too small for one entry remembers nothing, so each file asks for its own length); entries memory already answers cost no Store reader. Names and kinds come first, the offer then truncates the window to the names the reply publishes, and only those names' facts are read, so a re-sent reply reads nothing for a name it does not send. Test counts where earlier visits warmed upper levels are not a fresh mount's first window | [`kernel_directory.rs`](../../../../engine/crates/layerfs-workspace/src/operations/kernel_directory.rs), [`client.rs`](../../../../engine/crates/layerfs-workspace/src/base/client.rs) |
| Those batches are not everything an entry costs, and this route changes none of the rest: an inherited inode whose metadata is not inline reports one portable-metadata demand per distinct metadata root, a listed subdirectory one listing of that subdirectory for its child-directory count, and a symlink entry one read of its target; only ordinary files with inline metadata pay only those batches (when the cache remembers lengths) | [`kernel_directory.rs`](../../../../engine/crates/layerfs-workspace/src/operations/kernel_directory.rs), [`view.rs`](../../../../engine/crates/layerfs-workspace/src/base/view.rs), [`links.rs`](../../../../engine/crates/layerfs-workspace/src/base/links.rs) |
| READDIR has one count, which bounds the whole reply; READDIRPLUS sends `dircount` before it, and only `maxcount` bounds this reply | [`nfs.rs`](../../../../engine/crates/layerfs-nfs/src/nfs.rs), [`directory.rs`](../../../../engine/crates/layerfs-nfs/src/directory.rs) |
| A reply's `dir_attributes` are the facts of the listed directory its window read — a local row, fresh from that visit, costing no extra read. An inherited directory has no local row, so its window reads none and the reply carries no attributes; the client then falls back to its own `actimeo` rather than restarting it on facts nothing refreshes | [`directory.rs`](../../../../engine/crates/layerfs-nfs/src/directory.rs), [`kernel_directory.rs`](../../../../engine/crates/layerfs-workspace/src/operations/kernel_directory.rs) |
| `.` and `..` carry no attributes; `..` comes from the parent recorded at lookup, create, move or listing | [`kernel.rs`](../../../../engine/crates/layerfs-nfs/src/kernel.rs) |
| Silly names and AppleDouble companions are never listed | [`names.rs`](../../../../engine/crates/layerfs-nfs/src/names.rs), [`apple_double.rs`](../../../../engine/crates/layerfs-nfs/src/apple_double.rs) |

Expand All @@ -145,6 +149,15 @@ drawn once per export; it makes guessing costly and does not authenticate.
| SETATTR's guard compares a fresh GETATTR ctime, not atomic with the change; access time is dropped | [`change.rs`](../../../../engine/crates/layerfs-nfs/src/change.rs) |
| ACCESS grants owner bits to the inode's uid or uid 0, group bits to its group, else other bits; a read-only export grants no change | [`attributes.rs`](../../../../engine/crates/layerfs-nfs/src/attributes.rs) |

## LOOKUP and GETATTR (`nfs.rs`, `kernel.rs`)

| Rule | Source |
| --- | --- |
| One owner visit answers a LOOKUP: the job that resolves a name has already read the directory's own row, so the reply's `dir_attributes` cost no second visit | [`nfs.rs`](../../../../engine/crates/layerfs-nfs/src/nfs.rs), [`kernel.rs`](../../../../engine/crates/layerfs-nfs/src/kernel.rs), [`kernel_read.rs`](../../../../engine/crates/layerfs-workspace/src/operations/kernel_read.rs) |
| A LOOKUP that fails answers the directory's own fresh facts whenever the job that failed read its row, which is every name this export looks up: a name that binds nothing still read its parent's row in the same visit. `.`, `..`, a silly name and a refusal decided before any visit read no directory row and send no `dir_attributes`, so the client falls back to its own `actimeo`; RFC 1813 makes both fields optional. No LOOKUP answer repeats the export's last observation of the directory, which nothing ages | [`nfs.rs`](../../../../engine/crates/layerfs-nfs/src/nfs.rs), [`kernel.rs`](../../../../engine/crates/layerfs-nfs/src/kernel.rs), [`kernel_read.rs`](../../../../engine/crates/layerfs-workspace/src/operations/kernel_read.rs) |
| `read.rs` holds what a read answers: the entry, its directory and its descriptors, and the refusal with its status (`NFS3ERR_STALE` for an engine `Stale`, else `NFS3ERR_IO`) and the directory row the job read | [`read.rs`](../../../../engine/crates/layerfs-nfs/src/read.rs) |
| An export remembers the facts it last published or observed for a directory (LOOKUP, GETATTR, OPENDIR, a publishing change) and answers from them only where no job reads that directory: an AppleDouble companion's `wcc_data`, whose change never touches it. A directory listing is not such a place: its reply says only what its own window read | [`kernel.rs`](../../../../engine/crates/layerfs-nfs/src/kernel.rs), [`apple_double.rs`](../../../../engine/crates/layerfs-nfs/src/apple_double.rs) |

## Refresh after install (`refresh.rs`, daemon `export.rs`, `invalidate.rs`)

```text
Expand Down
1 change: 1 addition & 0 deletions docs/architecture/engine/workspace/operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,7 @@ job.
| --- | --- | --- |
| Offset from before the handle was last listed from 0 | `rewound()`: lists nothing | [`kernel_directory.rs`](../../../../engine/crates/layerfs-workspace/src/operations/kernel_directory.rs) |
| Window of whiteouts only | Empty with a continuation; the request reads the next window | same |
| READDIRPLUS reply | Names and kinds come first (batched per inode-tree level), the reply's offer truncates the window to the names it publishes, and only those names' facts are read outside the owner: a re-sent reply reads nothing for a name it does not send | [`kernel_directory.rs`](../../../../engine/crates/layerfs-workspace/src/operations/kernel_directory.rs) |
| Names the kernel buffer accepted | Become resume cookies; nothing is held between the two visits or after the reply | same |

## Kernel mutations
Expand Down
10 changes: 7 additions & 3 deletions engine/crates/layerfs-daemon/src/service/filesystem_port.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ use layerfs_mount_ports::ports::{
};
use layerfs_overlay::{KernelCookieOffer, KernelMount, Publication};
use layerfs_workspace::{
BaseView, CanonicalClient, JobOutcome, KernelDirectoryWindow, KernelInput,
BaseView, CanonicalClient, JobOutcome, KernelDirectoryRead, KernelDirectoryWindow, KernelInput,
KernelMutationOutcome, KernelReadOperation, KernelReadOutcome, KernelVisitRequest,
KernelWindow, Recognition, RecognizeWork, VisitFacts, WorkspaceError, WriteData,
RECOGNIZE_BYTES,
Expand Down Expand Up @@ -106,6 +106,7 @@ impl RequestServices for FilesystemPort {
handle: u64,
offset: u64,
after: Option<Vec<u8>>,
attributes: bool,
) -> ServiceFuture<'_, ServiceReply<Arc<KernelDirectoryWindow>>> {
// A stopped mount refuses before anything about the request is read.
if let Err(fenced) = self.fenced(true) {
Expand All @@ -116,8 +117,11 @@ impl RequestServices for FilesystemPort {
mount,
serial,
handle,
offset,
after,
KernelDirectoryRead {
offset,
after,
attributes,
},
);
match visit {
Ok(visit) => self.call(
Expand Down
2 changes: 1 addition & 1 deletion engine/crates/layerfs-daemon/tests/mounted_sqlite_full.rs
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ macro_rules! forward {
}
impl RequestServices for ObserveRequest {
forward! {
directory_visit(mount:KernelMount,serial:u64,handle:u64,offset:u64,after:Option<Vec<u8>>) -> ServiceReply<Arc<KernelDirectoryWindow>>;
directory_visit(mount:KernelMount,serial:u64,handle:u64,offset:u64,after:Option<Vec<u8>>,attributes:bool) -> ServiceReply<Arc<KernelDirectoryWindow>>;
publish_cookies(offer:KernelCookieOffer,names:Vec<Vec<u8>>) -> ServiceReply<()>;
close_directory(mount:KernelMount,serial:u64,handle:u64) -> ServiceReply<()>;
read_visit(mount:KernelMount,serial:u64,handle:Option<u64>,offset:u64,length:u32) -> ServiceReply<Arc<KernelWindow>>;
Expand Down
26 changes: 16 additions & 10 deletions engine/crates/layerfs-mount-ports/src/operations/directory.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,7 @@ pub struct DirectoryStream {
offset: u64,
parent: Option<u64>,
window: Option<KernelDirectoryWindow>,
/// The reply shows every entry's facts, as READDIRPLUS does: the listing
/// then reads them from the authoritative base, outside the owner.
/// The reply shows each entry's facts (READDIRPLUS).
attributes: bool,
ended: bool,
}
Expand Down Expand Up @@ -183,7 +182,14 @@ impl DirectoryStream {
let started = std::time::Instant::now();
let reply = self
.services
.directory_visit(self.mount, self.serial, self.handle, self.offset, after)
.directory_visit(
self.mount,
self.serial,
self.handle,
self.offset,
after,
self.attributes,
)
.await;
crate::phase::charge(crate::phase::Phase::DirectoryVisit, started);
let reply = reply?;
Expand Down Expand Up @@ -223,11 +229,12 @@ impl DirectoryStream {
let Some(window) = self.window.take().filter(|_| !self.ended) else {
return Ok(None);
};
// Only a window whose inherited names or kinds memory did not
// hold leaves the receive loop and takes one Store reader; a
// reply that shows facts needs that reader even when it does.
// Only a window whose inherited names, kinds or facts memory did
// not hold leaves the receive loop and takes one Store reader: a
// local directory, and a window whose inherited entries are all
// resident, are answered without it.
let started = std::time::Instant::now();
let base = if self.attributes || window.listing.is_none() {
let base = if window.needs_base(self.attributes) {
// Provider reads never run on the loop that received the request.
Some(self.services.base().await?)
} else {
Expand Down Expand Up @@ -277,9 +284,8 @@ impl DirectoryBatch {
}
/// Records the entries whose handles this reply hands out, so the
/// publishing visit takes their kernel references with the offsets.
/// `accepted` is the prefix the reply buffer took; only published
/// offsets can make a handle's fence answer, so a reused reply adopts
/// nothing.
/// `accepted` is the prefix the reply buffer took; only published offsets
/// can make a handle's fence answer, so a reused reply adopts nothing.
pub fn adopt(&mut self, accepted: usize) {
let entries: Vec<(u64, bool)> = self.batch.entries[..accepted]
.iter()
Expand Down
23 changes: 19 additions & 4 deletions engine/crates/layerfs-mount-ports/src/operations/lookup.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ use crate::ports::{BaseDemandFailed, Fenced, RequestServices, ServiceError, Serv
use layerfs_overlay::KernelMount;
use layerfs_workspace::{
KernelReadDecision, KernelReadFailure, KernelReadOperation, KernelReadOutcome, KernelReadValue,
Refusal, VisitFacts,
Refusal, ViewStat, VisitFacts,
};
use std::{fmt, sync::Arc};

Expand All @@ -17,6 +17,9 @@ pub(super) struct Custody {
pub data_input: Option<super::ReadDataInput>,
// Retain the engine receipt through every cloned outcome consumer.
pub receipt: Option<ServiceReply<Arc<KernelReadOutcome>>>,
/// The directory row a refused LOOKUP read in the same job, fresh from
/// that job (see [`KernelRead::refused_directory`]).
pub refused_directory: Option<ViewStat>,
}
/// A deciding answer and the receipt of the visit that decided it.
/// The request holds nothing else in the engine; the kernel's lookup, open
Expand Down Expand Up @@ -126,6 +129,13 @@ impl KernelRead {
pub fn value(&self) -> Result<&KernelReadValue, Refusal> {
self.value.as_ref().map_err(|refusal| *refusal)
}
/// The directory row a refused LOOKUP read in its own job, when it read
/// one: the fresh facts its reply's directory attributes are made of.
/// `None` for every other refusal and for every other operation, none of
/// which read a directory row.
pub fn refused_directory(&self) -> Option<&ViewStat> {
self.custody.refused_directory.as_ref()
}
/// Preserve a successfully decided request when kernel reply conversion or
/// another downstream invariant fails before consumer disposal.
pub fn retain(self, reason: ServiceError) -> ReadFailure {
Expand Down Expand Up @@ -160,6 +170,7 @@ impl Custody {
operation,
data_input: None,
receipt: None,
refused_directory: None,
}
}
pub(super) const fn target(&self) -> (KernelMount, u64, Option<u64>) {
Expand Down Expand Up @@ -189,17 +200,21 @@ impl Custody {
let receipt = receipt?;
let original = receipt.get().clone();
let needs = match (&original.result, &original.decision) {
(Ok(None), Some(KernelReadDecision::Value(inode))) => {
let stat = inode.clone().into();
(Ok(None), Some(KernelReadDecision::Value(decided))) => {
let stat = decided.inode.clone().into();
self.receipt = Some(receipt);
return Ok(Ok(KernelReadValue {
stat,
parent: decided.parent.clone().map(Into::into),
file: original.open_candidate,
directory: original.directory_candidate,
original,
}));
}
(Ok(None), Some(KernelReadDecision::Refused(refusal))) => return Ok(Err(*refusal)),
(Ok(None), Some(KernelReadDecision::Refused { refusal, directory })) => {
self.refused_directory = directory.clone().map(Into::into);
return Ok(Err(*refusal));
}
(Ok(None), Some(KernelReadDecision::Needs(needs))) if !needs.is_empty() => needs,
_ => {
self.receipt = Some(receipt);
Expand Down
7 changes: 5 additions & 2 deletions engine/crates/layerfs-mount-ports/src/ports.rs
Original file line number Diff line number Diff line change
Expand Up @@ -161,15 +161,18 @@ pub trait RequestServices: Send + Sync {
/// READDIR's reading visit as one read-only owner job on the open
/// directory `handle`: the position of the kernel `offset`, one window
/// of current names (after `after` when it continues an earlier window
/// of the same request) and the offsets the reply may hand out. It
/// records nothing, so its request holds nothing afterwards.
/// of the same request) and the offsets the reply may hand out. When
/// `attributes` is set (READDIRPLUS), the visit also answers what memory
/// already holds of each entry's facts. It records nothing, so its
/// request holds nothing afterwards.
fn directory_visit(
&self,
mount: KernelMount,
serial: u64,
handle: u64,
offset: u64,
after: Option<Vec<u8>>,
attributes: bool,
) -> ServiceFuture<'_, ServiceReply<Arc<KernelDirectoryWindow>>>;
/// READDIR's publishing visit, before the reply is sent: exactly the
/// names the reply buffer accepted become valid offsets, in one
Expand Down
Loading