diff --git a/docs/architecture/engine/kernel-routes/README.md b/docs/architecture/engine/kernel-routes/README.md index c345942caf..744bfe1358 100644 --- a/docs/architecture/engine/kernel-routes/README.md +++ b/docs/architecture/engine/kernel-routes/README.md @@ -73,7 +73,7 @@ stops: a stopped mount starts nothing new and still gives back what it holds. ```text LOOKUP/GETATTR/OPEN/OPENDIR KernelRead::prepare - loop: observe_visit -> Value | Refused | Needs(n): base(); supply; repeat + loop: observe_visit -> Value(Decided) | Refused | Needs(n): base(); supply; repeat READ/READLINK KernelData::read read_visit -> local window; base() only for inherited bytes mutation KernelMutation::perform @@ -85,7 +85,7 @@ stops: a stopped mount starts nothing new and still gives back what it holds. | Type or module | Kernel requests | Source | | --- | --- | --- | -| `KernelRead` | LOOKUP, GETATTR, OPEN, OPENDIR; holds nothing after its visits | [`lookup.rs`](../../../../engine/crates/layerfs-mount-ports/src/operations/lookup.rs) | +| `KernelRead` | LOOKUP, GETATTR, OPEN, OPENDIR; holds nothing after its visits (a LOOKUP also carries the directory it named, and a refused LOOKUP the directory row its own job read) | [`lookup.rs`](../../../../engine/crates/layerfs-mount-ports/src/operations/lookup.rs) | | `KernelData` | READ, READLINK | [`read.rs`](../../../../engine/crates/layerfs-mount-ports/src/operations/read.rs) | | `KernelMutation` | Every changing request; holds one `Publication` ticket until `replied` | [`mutation.rs`](../../../../engine/crates/layerfs-mount-ports/src/operations/mutation.rs) | | `DirectoryStream` | READDIR; only accepted names become valid offsets | [`directory.rs`](../../../../engine/crates/layerfs-mount-ports/src/operations/directory.rs) | diff --git a/docs/architecture/engine/kernel-routes/nfs.md b/docs/architecture/engine/kernel-routes/nfs.md index 4c8f4e0325..b97660ab33 100644 --- a/docs/architecture/engine/kernel-routes/nfs.md +++ b/docs/architecture/engine/kernel-routes/nfs.md @@ -14,7 +14,7 @@ and MOUNT are written over `std::net` with no third-party dependency and no platform `cfg` ([`lib.rs`](../../../../engine/crates/layerfs-nfs/src/lib.rs)). The daemon starts it and has the host's own NFS client mount it. -## Server (`server.rs`, `kernel.rs`, `block.rs`) +## Server (`server.rs`, `kernel.rs`, `read.rs`, `block.rs`) ```text host NFS client (mount_nfs on macOS) @@ -108,18 +108,22 @@ drawn once per export; it makes guessing costly and does not authenticate. ## Directories (`directory.rs`) ```text - READDIR / READDIRPLUS (handle, cookie, cookieverf, budget) + READDIR / READDIRPLUS (handle, cookie, cookieverf, maxcount) cookie 0 -> open_directory: fresh descriptor; verifier = its id cookie >0 -> descriptor (serial, verifier) still open? no: BAD_COOKIE cookies: 1 = ".", 2 = "..", then engine-published offsets - one reply = at most one engine window; entries that fit the budget + one reply = at most one engine window; entries that fit maxcount -> publish_cookies(those names); eof only when the window ended and fit ``` | Rule | Source | | --- | --- | | The cookie is the engine's kernel offset on one implicit directory descriptor named by `cookieverf`; a retired offset or a listing continued after an install is `NFS3ERR_BAD_COOKIE`, and the client lists again | [`directory.rs`](../../../../engine/crates/layerfs-nfs/src/directory.rs), [`kernel.rs`](../../../../engine/crates/layerfs-nfs/src/kernel.rs) | -| A READDIRPLUS entry carries the facts its listing already read and its handle; each handle takes its one lookup reference in the publishing visit | [`kernel_directory.rs`](../../../../engine/crates/layerfs-workspace/src/operations/kernel_directory.rs), [`kernel_cookie.rs`](../../../../engine/crates/layerfs-overlay/src/lifetime/kernel_cookie.rs) | +| A READDIRPLUS entry carries the facts its window read — a local inode's current row, or an inherited inode's immutable base stat — and the handle of its serial, so the client needs no LOOKUP per name; both are optional in RFC 1813 and this export sends both for every named entry whose offsets the reply publishes, each handle taking its one lookup reference in the publishing visit (reused offset ranges take none) | [`directory.rs`](../../../../engine/crates/layerfs-nfs/src/directory.rs), [`kernel_directory.rs`](../../../../engine/crates/layerfs-workspace/src/operations/kernel_directory.rs), [`kernel_cookie.rs`](../../../../engine/crates/layerfs-overlay/src/lifetime/kernel_cookie.rs) | +| A window's inherited inode values cost one batch inode demand for each level of the inode tree the cache does not already answer (`lookup_many` reads a level's pages in one `read_canonical_batch`, resident levels cost none) and its ordinary file lengths one batch length demand (`provide_lengths`) while the cache can remember them (too small for one entry remembers nothing, so each file asks for its own length); entries memory already answers cost no Store reader. Names and kinds come first, the offer then truncates the window to the names the reply publishes, and only those names' facts are read, so a re-sent reply reads nothing for a name it does not send. Test counts where earlier visits warmed upper levels are not a fresh mount's first window | [`kernel_directory.rs`](../../../../engine/crates/layerfs-workspace/src/operations/kernel_directory.rs), [`client.rs`](../../../../engine/crates/layerfs-workspace/src/base/client.rs) | +| Those batches are not everything an entry costs, and this route changes none of the rest: an inherited inode whose metadata is not inline reports one portable-metadata demand per distinct metadata root, a listed subdirectory one listing of that subdirectory for its child-directory count, and a symlink entry one read of its target; only ordinary files with inline metadata pay only those batches (when the cache remembers lengths) | [`kernel_directory.rs`](../../../../engine/crates/layerfs-workspace/src/operations/kernel_directory.rs), [`view.rs`](../../../../engine/crates/layerfs-workspace/src/base/view.rs), [`links.rs`](../../../../engine/crates/layerfs-workspace/src/base/links.rs) | +| READDIR has one count, which bounds the whole reply; READDIRPLUS sends `dircount` before it, and only `maxcount` bounds this reply | [`nfs.rs`](../../../../engine/crates/layerfs-nfs/src/nfs.rs), [`directory.rs`](../../../../engine/crates/layerfs-nfs/src/directory.rs) | +| A reply's `dir_attributes` are the facts of the listed directory its window read — a local row, fresh from that visit, costing no extra read. An inherited directory has no local row, so its window reads none and the reply carries no attributes; the client then falls back to its own `actimeo` rather than restarting it on facts nothing refreshes | [`directory.rs`](../../../../engine/crates/layerfs-nfs/src/directory.rs), [`kernel_directory.rs`](../../../../engine/crates/layerfs-workspace/src/operations/kernel_directory.rs) | | `.` and `..` carry no attributes; `..` comes from the parent recorded at lookup, create, move or listing | [`kernel.rs`](../../../../engine/crates/layerfs-nfs/src/kernel.rs) | | Silly names and AppleDouble companions are never listed | [`names.rs`](../../../../engine/crates/layerfs-nfs/src/names.rs), [`apple_double.rs`](../../../../engine/crates/layerfs-nfs/src/apple_double.rs) | @@ -145,6 +149,15 @@ drawn once per export; it makes guessing costly and does not authenticate. | SETATTR's guard compares a fresh GETATTR ctime, not atomic with the change; access time is dropped | [`change.rs`](../../../../engine/crates/layerfs-nfs/src/change.rs) | | ACCESS grants owner bits to the inode's uid or uid 0, group bits to its group, else other bits; a read-only export grants no change | [`attributes.rs`](../../../../engine/crates/layerfs-nfs/src/attributes.rs) | +## LOOKUP and GETATTR (`nfs.rs`, `kernel.rs`) + +| Rule | Source | +| --- | --- | +| One owner visit answers a LOOKUP: the job that resolves a name has already read the directory's own row, so the reply's `dir_attributes` cost no second visit | [`nfs.rs`](../../../../engine/crates/layerfs-nfs/src/nfs.rs), [`kernel.rs`](../../../../engine/crates/layerfs-nfs/src/kernel.rs), [`kernel_read.rs`](../../../../engine/crates/layerfs-workspace/src/operations/kernel_read.rs) | +| A LOOKUP that fails answers the directory's own fresh facts whenever the job that failed read its row, which is every name this export looks up: a name that binds nothing still read its parent's row in the same visit. `.`, `..`, a silly name and a refusal decided before any visit read no directory row and send no `dir_attributes`, so the client falls back to its own `actimeo`; RFC 1813 makes both fields optional. No LOOKUP answer repeats the export's last observation of the directory, which nothing ages | [`nfs.rs`](../../../../engine/crates/layerfs-nfs/src/nfs.rs), [`kernel.rs`](../../../../engine/crates/layerfs-nfs/src/kernel.rs), [`kernel_read.rs`](../../../../engine/crates/layerfs-workspace/src/operations/kernel_read.rs) | +| `read.rs` holds what a read answers: the entry, its directory and its descriptors, and the refusal with its status (`NFS3ERR_STALE` for an engine `Stale`, else `NFS3ERR_IO`) and the directory row the job read | [`read.rs`](../../../../engine/crates/layerfs-nfs/src/read.rs) | +| An export remembers the facts it last published or observed for a directory (LOOKUP, GETATTR, OPENDIR, a publishing change) and answers from them only where no job reads that directory: an AppleDouble companion's `wcc_data`, whose change never touches it. A directory listing is not such a place: its reply says only what its own window read | [`kernel.rs`](../../../../engine/crates/layerfs-nfs/src/kernel.rs), [`apple_double.rs`](../../../../engine/crates/layerfs-nfs/src/apple_double.rs) | + ## Refresh after install (`refresh.rs`, daemon `export.rs`, `invalidate.rs`) ```text diff --git a/docs/architecture/engine/workspace/operations.md b/docs/architecture/engine/workspace/operations.md index 480d5e7a02..5c7e1d8fdc 100644 --- a/docs/architecture/engine/workspace/operations.md +++ b/docs/architecture/engine/workspace/operations.md @@ -167,6 +167,7 @@ job. | --- | --- | --- | | Offset from before the handle was last listed from 0 | `rewound()`: lists nothing | [`kernel_directory.rs`](../../../../engine/crates/layerfs-workspace/src/operations/kernel_directory.rs) | | Window of whiteouts only | Empty with a continuation; the request reads the next window | same | +| READDIRPLUS reply | Names and kinds come first (batched per inode-tree level), the reply's offer truncates the window to the names it publishes, and only those names' facts are read outside the owner: a re-sent reply reads nothing for a name it does not send | [`kernel_directory.rs`](../../../../engine/crates/layerfs-workspace/src/operations/kernel_directory.rs) | | Names the kernel buffer accepted | Become resume cookies; nothing is held between the two visits or after the reply | same | ## Kernel mutations diff --git a/engine/crates/layerfs-daemon/src/service/filesystem_port.rs b/engine/crates/layerfs-daemon/src/service/filesystem_port.rs index 173c851221..c2ec044c5a 100644 --- a/engine/crates/layerfs-daemon/src/service/filesystem_port.rs +++ b/engine/crates/layerfs-daemon/src/service/filesystem_port.rs @@ -10,7 +10,7 @@ use layerfs_mount_ports::ports::{ }; use layerfs_overlay::{KernelCookieOffer, KernelMount, Publication}; use layerfs_workspace::{ - BaseView, CanonicalClient, JobOutcome, KernelDirectoryWindow, KernelInput, + BaseView, CanonicalClient, JobOutcome, KernelDirectoryRead, KernelDirectoryWindow, KernelInput, KernelMutationOutcome, KernelReadOperation, KernelReadOutcome, KernelVisitRequest, KernelWindow, Recognition, RecognizeWork, VisitFacts, WorkspaceError, WriteData, RECOGNIZE_BYTES, @@ -106,6 +106,7 @@ impl RequestServices for FilesystemPort { handle: u64, offset: u64, after: Option>, + attributes: bool, ) -> ServiceFuture<'_, ServiceReply>> { // A stopped mount refuses before anything about the request is read. if let Err(fenced) = self.fenced(true) { @@ -116,8 +117,11 @@ impl RequestServices for FilesystemPort { mount, serial, handle, - offset, - after, + KernelDirectoryRead { + offset, + after, + attributes, + }, ); match visit { Ok(visit) => self.call( diff --git a/engine/crates/layerfs-daemon/tests/mounted_sqlite_full.rs b/engine/crates/layerfs-daemon/tests/mounted_sqlite_full.rs index d60ac12e34..2b718a38f2 100644 --- a/engine/crates/layerfs-daemon/tests/mounted_sqlite_full.rs +++ b/engine/crates/layerfs-daemon/tests/mounted_sqlite_full.rs @@ -68,7 +68,7 @@ macro_rules! forward { } impl RequestServices for ObserveRequest { forward! { - directory_visit(mount:KernelMount,serial:u64,handle:u64,offset:u64,after:Option>) -> ServiceReply>; + directory_visit(mount:KernelMount,serial:u64,handle:u64,offset:u64,after:Option>,attributes:bool) -> ServiceReply>; publish_cookies(offer:KernelCookieOffer,names:Vec>) -> ServiceReply<()>; close_directory(mount:KernelMount,serial:u64,handle:u64) -> ServiceReply<()>; read_visit(mount:KernelMount,serial:u64,handle:Option,offset:u64,length:u32) -> ServiceReply>; diff --git a/engine/crates/layerfs-mount-ports/src/operations/directory.rs b/engine/crates/layerfs-mount-ports/src/operations/directory.rs index 874c7bb002..61c8732d82 100644 --- a/engine/crates/layerfs-mount-ports/src/operations/directory.rs +++ b/engine/crates/layerfs-mount-ports/src/operations/directory.rs @@ -20,8 +20,7 @@ pub struct DirectoryStream { offset: u64, parent: Option, window: Option, - /// The reply shows every entry's facts, as READDIRPLUS does: the listing - /// then reads them from the authoritative base, outside the owner. + /// The reply shows each entry's facts (READDIRPLUS). attributes: bool, ended: bool, } @@ -183,7 +182,14 @@ impl DirectoryStream { let started = std::time::Instant::now(); let reply = self .services - .directory_visit(self.mount, self.serial, self.handle, self.offset, after) + .directory_visit( + self.mount, + self.serial, + self.handle, + self.offset, + after, + self.attributes, + ) .await; crate::phase::charge(crate::phase::Phase::DirectoryVisit, started); let reply = reply?; @@ -223,11 +229,12 @@ impl DirectoryStream { let Some(window) = self.window.take().filter(|_| !self.ended) else { return Ok(None); }; - // Only a window whose inherited names or kinds memory did not - // hold leaves the receive loop and takes one Store reader; a - // reply that shows facts needs that reader even when it does. + // Only a window whose inherited names, kinds or facts memory did + // not hold leaves the receive loop and takes one Store reader: a + // local directory, and a window whose inherited entries are all + // resident, are answered without it. let started = std::time::Instant::now(); - let base = if self.attributes || window.listing.is_none() { + let base = if window.needs_base(self.attributes) { // Provider reads never run on the loop that received the request. Some(self.services.base().await?) } else { @@ -277,9 +284,8 @@ impl DirectoryBatch { } /// Records the entries whose handles this reply hands out, so the /// publishing visit takes their kernel references with the offsets. - /// `accepted` is the prefix the reply buffer took; only published - /// offsets can make a handle's fence answer, so a reused reply adopts - /// nothing. + /// `accepted` is the prefix the reply buffer took; only published offsets + /// can make a handle's fence answer, so a reused reply adopts nothing. pub fn adopt(&mut self, accepted: usize) { let entries: Vec<(u64, bool)> = self.batch.entries[..accepted] .iter() diff --git a/engine/crates/layerfs-mount-ports/src/operations/lookup.rs b/engine/crates/layerfs-mount-ports/src/operations/lookup.rs index 5cebfc3bb0..0a77df6119 100644 --- a/engine/crates/layerfs-mount-ports/src/operations/lookup.rs +++ b/engine/crates/layerfs-mount-ports/src/operations/lookup.rs @@ -3,7 +3,7 @@ use crate::ports::{BaseDemandFailed, Fenced, RequestServices, ServiceError, Serv use layerfs_overlay::KernelMount; use layerfs_workspace::{ KernelReadDecision, KernelReadFailure, KernelReadOperation, KernelReadOutcome, KernelReadValue, - Refusal, VisitFacts, + Refusal, ViewStat, VisitFacts, }; use std::{fmt, sync::Arc}; @@ -17,6 +17,9 @@ pub(super) struct Custody { pub data_input: Option, // Retain the engine receipt through every cloned outcome consumer. pub receipt: Option>>, + /// The directory row a refused LOOKUP read in the same job, fresh from + /// that job (see [`KernelRead::refused_directory`]). + pub refused_directory: Option, } /// A deciding answer and the receipt of the visit that decided it. /// The request holds nothing else in the engine; the kernel's lookup, open @@ -126,6 +129,13 @@ impl KernelRead { pub fn value(&self) -> Result<&KernelReadValue, Refusal> { self.value.as_ref().map_err(|refusal| *refusal) } + /// The directory row a refused LOOKUP read in its own job, when it read + /// one: the fresh facts its reply's directory attributes are made of. + /// `None` for every other refusal and for every other operation, none of + /// which read a directory row. + pub fn refused_directory(&self) -> Option<&ViewStat> { + self.custody.refused_directory.as_ref() + } /// Preserve a successfully decided request when kernel reply conversion or /// another downstream invariant fails before consumer disposal. pub fn retain(self, reason: ServiceError) -> ReadFailure { @@ -160,6 +170,7 @@ impl Custody { operation, data_input: None, receipt: None, + refused_directory: None, } } pub(super) const fn target(&self) -> (KernelMount, u64, Option) { @@ -189,17 +200,21 @@ impl Custody { let receipt = receipt?; let original = receipt.get().clone(); let needs = match (&original.result, &original.decision) { - (Ok(None), Some(KernelReadDecision::Value(inode))) => { - let stat = inode.clone().into(); + (Ok(None), Some(KernelReadDecision::Value(decided))) => { + let stat = decided.inode.clone().into(); self.receipt = Some(receipt); return Ok(Ok(KernelReadValue { stat, + parent: decided.parent.clone().map(Into::into), file: original.open_candidate, directory: original.directory_candidate, original, })); } - (Ok(None), Some(KernelReadDecision::Refused(refusal))) => return Ok(Err(*refusal)), + (Ok(None), Some(KernelReadDecision::Refused { refusal, directory })) => { + self.refused_directory = directory.clone().map(Into::into); + return Ok(Err(*refusal)); + } (Ok(None), Some(KernelReadDecision::Needs(needs))) if !needs.is_empty() => needs, _ => { self.receipt = Some(receipt); diff --git a/engine/crates/layerfs-mount-ports/src/ports.rs b/engine/crates/layerfs-mount-ports/src/ports.rs index f7c6f1c789..1bfd63ba81 100644 --- a/engine/crates/layerfs-mount-ports/src/ports.rs +++ b/engine/crates/layerfs-mount-ports/src/ports.rs @@ -161,8 +161,10 @@ pub trait RequestServices: Send + Sync { /// READDIR's reading visit as one read-only owner job on the open /// directory `handle`: the position of the kernel `offset`, one window /// of current names (after `after` when it continues an earlier window - /// of the same request) and the offsets the reply may hand out. It - /// records nothing, so its request holds nothing afterwards. + /// of the same request) and the offsets the reply may hand out. When + /// `attributes` is set (READDIRPLUS), the visit also answers what memory + /// already holds of each entry's facts. It records nothing, so its + /// request holds nothing afterwards. fn directory_visit( &self, mount: KernelMount, @@ -170,6 +172,7 @@ pub trait RequestServices: Send + Sync { handle: u64, offset: u64, after: Option>, + attributes: bool, ) -> ServiceFuture<'_, ServiceReply>>; /// READDIR's publishing visit, before the reply is sent: exactly the /// names the reply buffer accepted become valid offsets, in one diff --git a/engine/crates/layerfs-nfs/src/change.rs b/engine/crates/layerfs-nfs/src/change.rs index 7932d219b5..0f567d8f94 100644 --- a/engine/crates/layerfs-nfs/src/change.rs +++ b/engine/crates/layerfs-nfs/src/change.rs @@ -183,7 +183,7 @@ impl Kernel { let mutation = match wait(KernelMutation::perform(services, request)) { Ok(mutation) => mutation, Err(failure) => { - let status = if crate::kernel::failure_is_stale(&failure.reason) { + let status = if crate::read::failure_is_stale(&failure.reason) { Status::Stale } else { Status::Io @@ -261,9 +261,11 @@ impl Kernel { /// The entry a name binds now, or none. fn existing(&self, parent: u64, name: &[u8]) -> Result, Status> { match self.lookup(parent, name) { - Ok(stat) => Ok(Some(stat)), - Err(Status::NoEnt) => Ok(None), - Err(status) => Err(status), + Ok((stat, _)) => Ok(Some(stat)), + Err(refused) => match refused.status { + Status::NoEnt => Ok(None), + status => Err(status), + }, } } /// CREATE of a regular file. diff --git a/engine/crates/layerfs-nfs/src/directory.rs b/engine/crates/layerfs-nfs/src/directory.rs index 5842d6c6dd..7f7e00fb19 100644 --- a/engine/crates/layerfs-nfs/src/directory.rs +++ b/engine/crates/layerfs-nfs/src/directory.rs @@ -12,10 +12,12 @@ //! directory's end, so the same reply says so; any other reply's next call //! resumes after the last accepted cookie. //! -//! A READDIRPLUS entry carries the attributes the listing already read — the -//! local inode row of the name, or the inherited inode's own immutable facts -//! — and the handle of the entry's serial. Both are optional in RFC 1813; -//! here they are present for every named entry whose offsets this request +//! A READDIRPLUS entry carries the attributes the listing read — the local +//! inode row of the name, or the inherited inode's immutable facts, read for +//! the window in one batch demand for file lengths and one inode batch per +//! level of the inode tree the cache does not already answer — +//! and the handle of the entry's serial. Both are optional in RFC 1813; here +//! they are present for every named entry whose offsets this request //! publishes, and the publishing visit takes one kernel lookup reference per //! handed-out handle in that same job, so nothing is looked up twice and no //! entry is left without the reference its handle presents. @@ -23,7 +25,8 @@ use crate::{ attributes::{post_op_maybe, Identity, Status}, block::wait, handle::HANDLE_BYTES, - kernel::{failure, Kernel}, + kernel::Kernel, + read::failure, xdr::{opaque_size, Writer}, }; use layerfs_content::object::inode_leaf::InodeKind; @@ -210,6 +213,15 @@ pub fn list(kernel: &Kernel, request: Listing) -> Result, Status> { (at_end, directory) } }; + // A window that read the listed directory's own row answers it: the + // reply's `post_op_attr` is those fresh facts and no further read. An + // inherited directory has no local row, so its window reads none. + // Answering the facts this export last observed for it instead — what its + // OPENDIR visit read, whenever the descriptor was opened — restarts the + // client's `actimeo` on a snapshot that nothing ages, so a directory + // whose facts moved on since is answered as it was then, for as long as + // the descriptor lives. RFC 1813 makes the field optional, so such a + // reply says nothing and the client falls back to its own timer. finish(entries, directory.as_ref(), handle, eof) } diff --git a/engine/crates/layerfs-nfs/src/kernel.rs b/engine/crates/layerfs-nfs/src/kernel.rs index dd03a6ab8f..86c0834636 100644 --- a/engine/crates/layerfs-nfs/src/kernel.rs +++ b/engine/crates/layerfs-nfs/src/kernel.rs @@ -40,13 +40,14 @@ use crate::{ block::wait, change::Wcc, handle::{HandleError, SqliteHandles}, + read::{failure, read_failure, Answer, Refused}, }; use layerfs_content::{filesystem::PathName, object::inode_leaf::InodeKind}; use layerfs_mount_ports::{ - operations::{KernelData, KernelRead, MutationFailure, ReadDataInput, ReadFailure}, - ports::{Fence, MountServices, RequestServices, ServiceError}, + operations::{KernelData, KernelRead, MutationFailure, ReadDataInput}, + ports::{Fence, MountServices, RequestServices}, }; -use layerfs_overlay::{KernelMount, OverlayError, READ_WINDOW}; +use layerfs_overlay::{KernelMount, READ_WINDOW}; use layerfs_workspace::{KernelReadOperation, ViewStat}; use std::{ collections::{HashMap, HashSet}, @@ -135,7 +136,6 @@ pub(crate) struct OpenFile { pub handle: u64, pub size: u64, } - /// One kernel mount served over NFS: its ports and its synthetic kernel state. pub struct Kernel { services: Arc, @@ -285,11 +285,12 @@ impl Kernel { None, KernelReadOperation::Lookup { parent, name: path }, ) + .map_err(|refused| refused.status) }); match found { - Ok((stat, _, _)) => { - self.entry_taken(&stat, parent, &name); - if stat.serial == serial { + Ok(answer) => { + self.entry_taken(&answer.stat, parent, &name); + if answer.stat.serial == serial { self.resolved.fetch_add(1, Ordering::Relaxed); return Ok(()); } @@ -321,21 +322,37 @@ impl Kernel { protected: u64, handle: Option, operation: KernelReadOperation, - ) -> Result<(ViewStat, Option, Option), Status> { - let (services, request) = self.request()?; + ) -> Result { + let (services, request) = self.request().map_err(|status| Refused { + status, + directory: None, + })?; let found = wait(KernelRead::prepare( services, self.mount, request, protected, handle, operation, )) - .map_err(read_failure)?; - let answer = found.value().map(|value| { - ( - value.stat.clone(), - value.file.map(|file| file.owner_id()), - value.directory.map(|directory| directory.owner_id()), - ) - }); - wait(found.dispose()).map_err(read_failure)?; - answer.map_err(refusal) + .map_err(|failure| Refused { + status: read_failure(failure), + directory: None, + })?; + let answer = match found.value() { + Ok(value) => Ok(Answer { + stat: value.stat.clone(), + parent: value.parent.clone(), + file: value.file.map(|file| file.owner_id()), + directory: value.directory.map(|directory| directory.owner_id()), + }), + // A refusal still carries the directory row its job read, when it + // read one: fresh facts, not this export's last observation. + Err(status) => Err(Refused { + status: refusal(status), + directory: found.refused_directory().cloned(), + }), + }; + wait(found.dispose()).map_err(|failure| Refused { + status: read_failure(failure), + directory: None, + })?; + answer } /// GETATTR. An inode that lost its last name while a silly name keeps it /// is observed through its descriptor; any other inode no name reaches @@ -354,58 +371,86 @@ impl Kernel { }; let stat = self .observe(serial, handle, KernelReadOperation::Getattr { serial }) - .map(|(stat, _, _)| stat) - .map_err(|status| { - if status == Status::NoEnt { - Status::Stale - } else { - status - } + .map(|answer| answer.stat) + .map_err(|refused| match refused.status { + Status::NoEnt => Status::Stale, + status => status, })?; self.observed(&stat); Ok(stat) } /// LOOKUP of `name` in directory `parent`, including `.`, `..` and the - /// export's own silly names. - pub fn lookup(&self, parent: u64, name: &[u8]) -> Result { + /// export's own silly names. One owner job answers both the entry and, + /// when the name resolved, the directory itself: the same job had to read + /// the directory's row to resolve it, so the reply's directory attributes + /// cost nothing more. + /// + /// A failure answers the directory's own fresh facts only when the job + /// that failed read its row, which is every name this export looked up: + /// a name that binds nothing still read the parent's row. `.`, `..`, a + /// silly name and a refusal decided before any visit read no directory + /// row and answer `None`, so the client falls back to its own `actimeo` + /// instead of restarting its timer on facts nothing refreshes. + pub fn lookup( + &self, + parent: u64, + name: &[u8], + ) -> Result<(ViewStat, Option), Refused> { match name { - b"" => return Err(Status::NoEnt), - b"." => return self.getattr(parent), + b"" => return Err(Refused::plain(Status::NoEnt)), + b"." => { + return self + .getattr(parent) + .map(|stat| (stat, None)) + .map_err(Refused::plain) + } b".." => { let up = if parent == self.root() { parent } else { let known = self.table().parents.get(&parent).copied(); - known.ok_or(Status::NoEnt)? + known.ok_or(Refused::plain(Status::NoEnt))? }; - self.held_reference(up)?; - return self.getattr(up); + self.held_reference(up).map_err(Refused::plain)?; + return self + .getattr(up) + .map(|stat| (stat, None)) + .map_err(Refused::plain); } - _ if name.len() > 255 => return Err(Status::NameTooLong), + _ if name.len() > 255 => return Err(Refused::plain(Status::NameTooLong)), _ => {} } + // An AppleDouble companion lives in the export alone, so its reply's + // directory half is the facts this export last answered for the + // directory: a companion change never touches it. if let Some(stat) = self.companion_named(parent, name) { - return Ok(stat); + return Ok((stat, self.directory_wcc(parent).0)); } if self.refuses_apple_double() && crate::names::apple_double(name) { if parent != self.root() && !self.table().parents.contains_key(&parent) { - return Err(Status::NotDir); + return Err(Refused::plain(Status::NotDir)); } - return Err(Status::NoEnt); + return Err(Refused::plain(Status::NoEnt)); } let silly = self.table().silly.get(&(parent, name.to_vec())).copied(); if let Some(serial) = silly { - return self.getattr(serial); + return self + .getattr(serial) + .map(|stat| (stat, None)) + .map_err(Refused::plain); } - let path = PathName::from_bytes(name).map_err(|_| Status::Inval)?; - let (stat, _, _) = self.observe( + let path = PathName::from_bytes(name).map_err(|_| Refused::plain(Status::Inval))?; + let answer = self.observe( parent, None, KernelReadOperation::Lookup { parent, name: path }, )?; + if let Some(directory) = &answer.parent { + self.observed(directory); + } // The deciding visit took one engine reference; keep exactly one. - self.entry_taken(&stat, parent, name); - Ok(stat) + self.entry_taken(&answer.stat, parent, name); + Ok((answer.stat, answer.parent)) } /// A visit or mutation that decided `stat` as the entry `name` of /// `parent` took one engine lookup reference on it: the export keeps @@ -611,9 +656,11 @@ impl Kernel { } } let operation = KernelReadOperation::Open { serial, writable }; - let (stat, file, _) = self.observe(serial, None, operation)?; - let handle = file.ok_or(Status::ServerFault)?; - Ok(self.file_taken(serial, handle, &stat, writable)) + let answer = self + .observe(serial, None, operation) + .map_err(|refused| refused.status)?; + let handle = answer.file.ok_or(Status::ServerFault)?; + Ok(self.file_taken(serial, handle, &answer.stat, writable)) } /// Records an open file descriptor (from `OPEN` or `CREATE`), evicting /// the oldest unpinned descriptor when the bound is reached. @@ -730,9 +777,13 @@ impl Kernel { if crate::apple_double::companion(serial) { return Err(Status::NotDir); } - let (_, _, directory) = - self.observe(serial, None, KernelReadOperation::Opendir { serial })?; - let handle = directory.ok_or(Status::ServerFault)?; + let answer = self + .observe(serial, None, KernelReadOperation::Opendir { serial }) + .map_err(|refused| refused.status)?; + let handle = answer.directory.ok_or(Status::ServerFault)?; + // The visit read the directory's own row: remember it, so a LOOKUP in + // this directory answers its `dir_attributes` from memory. + self.observed(&answer.stat); let evicted = { let mut table = self.table(); let tick = table.tick(); @@ -934,36 +985,3 @@ pub struct Released { fn oldest(entries: impl Iterator) -> Option { entries.min_by_key(|(_, tick)| *tick).map(|(key, _)| key) } -fn stale(error: &(dyn std::error::Error + 'static)) -> bool { - let mut current = Some(error); - while let Some(error) = current { - if matches!( - error.downcast_ref::(), - Some(OverlayError::Stale) - ) { - return true; - } - current = error.source(); - } - false -} -/// Whether a failed step's cause chain holds an engine `Stale`. -pub(crate) fn failure_is_stale(error: &ServiceError) -> bool { - stale(error.as_ref()) -} -/// A failed port step: an engine `Stale` is the client's stale handle; any -/// other failure is an I/O error of this request alone. -pub fn failure(error: ServiceError) -> Status { - if stale(error.as_ref()) { - Status::Stale - } else { - Status::Io - } -} -fn read_failure(failure: ReadFailure) -> Status { - if stale(failure.reason.as_ref()) { - Status::Stale - } else { - Status::Io - } -} diff --git a/engine/crates/layerfs-nfs/src/lib.rs b/engine/crates/layerfs-nfs/src/lib.rs index f5cb633717..665eedf8a3 100644 --- a/engine/crates/layerfs-nfs/src/lib.rs +++ b/engine/crates/layerfs-nfs/src/lib.rs @@ -27,6 +27,7 @@ pub mod kernel; mod mount; pub mod names; mod nfs; +pub mod read; pub mod refresh; pub mod rpc; mod server; diff --git a/engine/crates/layerfs-nfs/src/nfs.rs b/engine/crates/layerfs-nfs/src/nfs.rs index ecffcda5f0..1b05fc15de 100644 --- a/engine/crates/layerfs-nfs/src/nfs.rs +++ b/engine/crates/layerfs-nfs/src/nfs.rs @@ -107,23 +107,37 @@ pub fn call( 3 => { let parent = handle(kernel, arguments).map_err(garbage)?; let name = arguments.opaque(NAME_LIMIT).map_err(garbage)?; - // Both answers carry the directory's attributes (RFC 1813 - // LOOKUP3resok and LOOKUP3resfail), so the client keeps its - // cached view of the directory without a GETATTR of its own. - let directory = parent - .as_ref() - .ok() - .and_then(|parent| kernel.getattr(*parent).ok()); - match parent.and_then(|parent| kernel.lookup(parent, name)) { - Ok(stat) => { + // One owner job answers the entry and the directory it was named + // in (RFC 1813 LOOKUP3resok's `obj_attributes` and + // `dir_attributes`, and the failure's). The job had to read the + // directory's own row to resolve the name, so its attributes cost + // no second job, and a name that binds nothing still read that row: + // the failed answer carries the same fresh facts. + // + // Every answer that read no directory row says so instead of + // repeating this export's last observation: `.`, `..`, a silly + // name and a refusal decided before the job. Stale attributes are + // worse than none here — the client would load them into its + // attribute cache, restart `actimeo` and keep a listing nothing + // refreshes — and RFC 1813 keeps both fields optional. + let found = match parent { + Ok(parent) => kernel.lookup(parent, name), + Err(status) => { + out.u32(status.code()); + post_op_maybe(kernel.identity(), out, None); + return Ok(()); + } + }; + match found { + Ok((stat, directory)) => { out.u32(Status::Ok.code()); out.opaque(&kernel.encode(stat.serial)); kernel.identity().post_op(out, &stat); post_op_maybe(kernel.identity(), out, directory.as_ref()); } - Err(status) => { - out.u32(status.code()); - post_op_maybe(kernel.identity(), out, directory.as_ref()); + Err(refused) => { + out.u32(refused.status.code()); + post_op_maybe(kernel.identity(), out, refused.directory.as_ref()); } } Ok(()) @@ -188,8 +202,10 @@ pub fn call( let verifier = arguments.fixed(8).map_err(garbage)?; let mut budget = arguments.u32().map_err(garbage)?; if plus { - // dircount bounds the names alone; maxcount the whole reply. - let _dircount = budget; + // A READDIRPLUS request carries the entries' own count + // (`dircount`) before the reply's (`maxcount`). Only the + // reply's bounds what this export sends: the client sized its + // buffer by it, and names alone are never what fills a reply. budget = arguments.u32().map_err(garbage)?; } let mut word = [0; 8]; diff --git a/engine/crates/layerfs-nfs/src/read.rs b/engine/crates/layerfs-nfs/src/read.rs new file mode 100644 index 0000000000..4b0e31e027 --- /dev/null +++ b/engine/crates/layerfs-nfs/src/read.rs @@ -0,0 +1,79 @@ +//! What one kernel read answers this export, and what a failed one means. +//! +//! A read is served by one owner job (see [`crate::kernel`]). Its answer is +//! the entry it names and, for a LOOKUP, the directory it was named in, which +//! that job had to read anyway. A refusal keeps the directory row its own job +//! read, when it read one, so a reply's `dir_attributes` are current facts and +//! never this export's last observation: nothing ages such an observation, so +//! a client that loaded it would restart its own `actimeo` on facts that do +//! not change until the export happens to observe the directory again. +use crate::attributes::Status; +use layerfs_mount_ports::{operations::ReadFailure, ports::ServiceError}; +use layerfs_overlay::OverlayError; +use layerfs_workspace::ViewStat; + +/// One kernel read's answer, as this export uses it. +pub(crate) struct Answer { + pub stat: ViewStat, + /// The directory a LOOKUP named the inode in: the same visit read it, so + /// the reply's directory attributes cost no second owner job. + pub parent: Option, + pub file: Option, + pub directory: Option, +} + +/// A read this export refused: the status its reply answers, and the fresh +/// directory facts the refusing job read, when it read one. A LOOKUP that +/// binds no name read its parent's row in the same job, so the failure's +/// `dir_attributes` are those facts and never a last observation that no +/// time bound ages (see [`crate::kernel::Kernel::lookup`]). +pub struct Refused { + pub status: Status, + pub directory: Option, +} +impl Refused { + /// A refusal no visit decided: no job read a directory row for it. + pub(crate) const fn plain(status: Status) -> Self { + Self { + status, + directory: None, + } + } +} + +fn stale(error: &(dyn std::error::Error + 'static)) -> bool { + let mut current = Some(error); + while let Some(error) = current { + if matches!( + error.downcast_ref::(), + Some(OverlayError::Stale) + ) { + return true; + } + current = error.source(); + } + false +} + +/// Whether a failed step's cause chain holds an engine `Stale`. +pub(crate) fn failure_is_stale(error: &ServiceError) -> bool { + stale(error.as_ref()) +} + +/// A failed port step: an engine `Stale` is the client's stale handle; any +/// other failure is an I/O error of this request alone. +pub fn failure(error: ServiceError) -> Status { + if stale(error.as_ref()) { + Status::Stale + } else { + Status::Io + } +} + +pub(crate) fn read_failure(failure: ReadFailure) -> Status { + if stale(failure.reason.as_ref()) { + Status::Stale + } else { + Status::Io + } +} diff --git a/engine/crates/layerfs-nfs/tests/export.rs b/engine/crates/layerfs-nfs/tests/export.rs index e14ea03528..2f7fb6181e 100644 --- a/engine/crates/layerfs-nfs/tests/export.rs +++ b/engine/crates/layerfs-nfs/tests/export.rs @@ -9,6 +9,9 @@ mod client; #[path = "support/rig.rs"] mod rig; #[allow(dead_code)] +#[path = "support/spy.rs"] +mod spy; +#[allow(dead_code)] #[path = "support/tree.rs"] mod tree; use client::{Attr, Client, BADHANDLE, BAD_COOKIE, NOENT, ROFS, STALE}; @@ -19,6 +22,7 @@ use std::{ net::TcpListener, os::unix::fs::{MetadataExt, PermissionsExt}, path::{Path, PathBuf}, + sync::{atomic::Ordering, Arc}, time::Duration, }; use tree::{oracle, walk, Node}; @@ -114,7 +118,7 @@ fn readdirplus_lists_a_directory_with_attributes_and_handles() { let server = rig.export(TcpListener::bind("127.0.0.1:0").unwrap(), UID, GID, true); let mut client = Client::connect(server.local_addr(), UID, GID); let root = client.mnt("/"); - let (src, src_attr) = client.lookup(&root, b"src").unwrap(); + let (src, _) = client.lookup(&root, b"src").unwrap(); let held = server.held().lookups; // One reply lists the whole directory: the window that lists its last @@ -125,10 +129,12 @@ fn readdirplus_lists_a_directory_with_attributes_and_handles() { .unwrap(); assert_eq!(client.calls - calls, 1, "one READDIRPLUS for the directory"); assert!(eof, "the reply that lists the last name is the end"); + // `src` is inherited from the committed base: the window reads no local + // row of it, so the reply says nothing about the directory rather than + // repeating the facts this export last observed for it. assert_eq!( - directory, - Some(src_attr), - "dir_attributes of the listed directory" + directory, None, + "no dir_attributes for an inherited directory" ); let names: BTreeSet> = entries.iter().map(|entry| entry.name.clone()).collect(); assert_eq!( @@ -247,6 +253,71 @@ fn readdirplus_reports_a_local_directorys_own_rows() { fs::remove_dir_all(base).unwrap(); } +#[test] +fn an_inherited_directorys_listing_never_answers_a_stale_observation() { + let base = temp("inherited"); + let tree = base.join("tree"); + source(&tree); + let rig = rig::Rig::new(&tree, &base.join("rig"), Duration::from_secs(30)); + let first = rig.export(TcpListener::bind("127.0.0.1:0").unwrap(), UID, GID, false); + let mut client = Client::connect(first.local_addr(), UID, GID); + let root = client.mnt("/"); + // `many` comes from the committed base: the overlay holds no local row of + // it, so no listing visit reads the directory's own row. + let (many, _) = client.lookup(&root, b"many").unwrap(); + let before = client.getattr(&many).unwrap(); + assert_eq!(before.nlink, 2, "the directory has no child directory"); + + // The first reply lists the two dots only, so the descriptor stays open + // across the change below and the next reply continues this enumeration. + // Its window read no row of `many`, so it carries no attributes of it + // rather than the facts the OPENDIR visit observed. + let (dots, verifier, eof, directory) = client + .readdir_with_directory(&many, 0, 0, 300, true) + .unwrap(); + assert!(!eof, "the enumeration continues after the dots"); + assert_eq!( + directory, None, + "an inherited directory's window reads no row to answer" + ); + let cookie = dots.last().expect("the dots").cookie; + + // A second export of the same Workspace adds a child directory to + // `many`, which this export does not observe: its last observation of the + // directory is now older than the directory's own row, and nothing ages + // it. + let second = rig.export(TcpListener::bind("127.0.0.1:0").unwrap(), UID, GID, false); + let mut other = Client::connect(second.local_addr(), UID, GID); + let other_root = other.mnt("/"); + let (target, _) = other.lookup(&other_root, b"many").unwrap(); + other.mkdir(&target, b"child", 0o755).unwrap(); + + // The continued reply says the directory's current facts or nothing at + // all; it never repeats the pre-change observation. + let (_, _, _, directory) = client + .readdir_with_directory(&many, cookie, verifier, 4096, true) + .unwrap(); + let fresh = client.getattr(&many).unwrap(); + assert_eq!(fresh.nlink, 3, "the directory gained a child directory"); + assert_ne!( + directory, + Some(before), + "never the last observation of the directory" + ); + assert!( + directory.is_none() || directory == Some(fresh), + "the current facts or no attributes at all: {directory:?}" + ); + + drop(client); + drop(other); + let released = first.stop(); + second.stop(); + let (_, outstanding) = rig.stop(released); + assert_eq!(outstanding, 0); + fs::remove_dir_all(base).unwrap(); +} + #[test] fn a_large_directory_keeps_its_names_and_its_references() { let base = temp("large"); @@ -374,3 +445,242 @@ fn cookies_refusals_and_stale_handles() { assert_eq!(outstanding, 0); fs::remove_dir_all(base).unwrap(); } + +#[test] +fn a_lookup_spends_one_owner_visit() { + let base = temp("lookup"); + let tree = base.join("tree"); + source(&tree); + let rig = rig::Rig::new(&tree, &base.join("rig"), Duration::from_secs(30)); + let watch = Arc::new(spy::Watch::default()); + let services = Arc::new(spy::Spy { + inner: Arc::new(layerfs_daemon::Mounted(rig.bound.clone())), + watch: watch.clone(), + }); + let server = rig.export_services( + TcpListener::bind("127.0.0.1:0").unwrap(), + services, + UID, + GID, + false, + ); + let mut client = Client::connect(server.local_addr(), UID, GID); + let root = client.mnt("/"); + + // RFC 1813 answers the entry's attributes and its directory's: the one + // visit that resolved the name had to read the directory's row too. The + // names are looked up once first, so the visit below is the steady state + // and not the mount's first read of a base name. + client.lookup(&root, b"src").unwrap(); + assert_eq!(client.lookup(&root, b"absent").map(|_| ()), Err(NOENT)); + let root_attr: Attr = client.getattr(&root).unwrap(); + + let visits = watch.visits.load(Ordering::Acquire); + let (status, found, directory) = client.lookup_with_directory(&root, b"src"); + assert_eq!(status, 0); + let (_, src_attr) = found.clone().expect("the directory's own name"); + assert_eq!( + watch.visits.load(Ordering::Acquire) - visits, + 1, + "one owner visit for a LOOKUP and its directory" + ); + assert_eq!( + directory, + Some(root_attr.clone()), + "the directory it was named in" + ); + + // A missing name is one visit as well, and answers the directory facts + // the export already published for it. + let visits = watch.visits.load(Ordering::Acquire); + let (status, found, directory) = client.lookup_with_directory(&root, b"absent"); + assert_eq!((status, found), (NOENT, None)); + assert_eq!( + watch.visits.load(Ordering::Acquire) - visits, + 1, + "one owner visit for a failing LOOKUP" + ); + assert_eq!(directory, Some(root_attr)); + + // The name resolved to the same inode a later LOOKUP reports, and the + // directory half of the reply is the directory's own facts. + let (_, again) = client.lookup(&root, b"src").unwrap(); + assert_eq!(again.fileid, src_attr.fileid, "the same inode"); + drop(client); + let released = server.stop(); + let (_, outstanding) = rig.stop(released); + assert_eq!(outstanding, 0); + fs::remove_dir_all(base).unwrap(); +} + +#[test] +fn a_listing_spends_one_owner_visit_and_reads_its_window_in_batches() { + let base = temp("listing"); + let tree = base.join("tree"); + source(&tree); + let rig = rig::Rig::new(&tree, &base.join("rig"), Duration::from_secs(30)); + let watch = Arc::new(spy::Watch::default()); + let services = Arc::new(spy::Spy { + inner: Arc::new(layerfs_daemon::Mounted(rig.bound.clone())), + watch: watch.clone(), + }); + let server = rig.export_services( + TcpListener::bind("127.0.0.1:0").unwrap(), + services, + UID, + GID, + false, + ); + let mut client = Client::connect(server.local_addr(), UID, GID); + let root = client.mnt("/"); + let (many, _) = client.lookup(&root, b"many").unwrap(); + + // A directory of 700 inherited names over several replies: one OPENDIR + // visit for the whole enumeration, at most one Store reader per reply, + // and never one per listed name — the window's names, kinds and facts + // are read in batches outside the owner. + let (visits, bases) = ( + watch.visits.load(Ordering::Acquire), + watch.bases.load(Ordering::Acquire), + ); + let (entries, calls) = client.list(&many, 64 * 1024, true); + assert!(calls > 1, "a budget spreads one directory over replies"); + assert_eq!(entries.len(), MANY + 2, "every name, once"); + for entry in entries + .iter() + .filter(|entry| entry.name != b"." && entry.name != b"..") + { + // Every inherited name arrives with both optional fields, so the + // client needs no LOOKUP per name on a cold tree. + assert!( + entry.attr.is_some(), + "READDIRPLUS attributes: {:?}", + entry.name + ); + assert!( + entry.handle.is_some(), + "READDIRPLUS handle: {:?}", + entry.name + ); + } + assert_eq!( + watch.visits.load(Ordering::Acquire) - visits, + 1, + "one owner visit for the whole enumeration" + ); + assert!( + watch.bases.load(Ordering::Acquire) - bases <= calls, + "at most one Store reader per reply, never one per name" + ); + // Each handed-out handle holds one reference, and an identical reply + // reuses the published offsets instead of taking more. + let held = server.held().lookups; + assert_eq!(held, 1 + MANY, "the directory and one reference per entry"); + let (again, _) = client.list(&many, 64 * 1024, true); + assert_eq!(again.len(), entries.len(), "the same names again"); + assert_eq!( + server.held().lookups, + held, + "no second reference for a listed name" + ); + + // A directory this export created is answered from the rows its visit + // already read: its listing takes no Store reader at all. + client.mkdir(&root, b"local", 0o755).unwrap(); + let (local, _) = client.lookup(&root, b"local").unwrap(); + client.touch(&local, b"a", 0o644).unwrap(); + let bases = watch.bases.load(Ordering::Acquire); + let (entries, _, _) = client + .readdir(&local, 0, 0, 4096, true) + .map(|(entries, _, eof)| (entries, eof, ())) + .unwrap(); + let created = entries + .iter() + .find(|entry| entry.name == b"a") + .expect("the created name"); + assert!( + created.attr.is_some() && created.handle.is_some(), + "a local row carries its facts and its handle" + ); + assert_eq!( + watch.bases.load(Ordering::Acquire), + bases, + "a local listing takes no Store reader" + ); + + drop(client); + let released = server.stop(); + let (_, outstanding) = rig.stop(released); + assert_eq!(outstanding, 0); + fs::remove_dir_all(base).unwrap(); +} + +#[test] +fn a_failed_lookup_answers_fresh_directory_facts_never_a_stale_observation() { + let base = temp("lookup-directory"); + let tree = base.join("tree"); + source(&tree); + let rig = rig::Rig::new(&tree, &base.join("rig"), Duration::from_secs(30)); + let first = rig.export(TcpListener::bind("127.0.0.1:0").unwrap(), UID, GID, false); + let mut client = Client::connect(first.local_addr(), UID, GID); + let root = client.mnt("/"); + let made = client.mkdir(&root, b"watched", 0o755).unwrap(); + let before = client.getattr(&made.handle).unwrap(); + assert_eq!(before.nlink, 2, "an empty directory"); + + // A second engine mount of the same Workspace adds a name to the + // directory this export watches. This export observes no publication for + // it, which is the install and other-writer case: its last observation of + // the directory is now older than the directory's own row. + let second = rig.export(TcpListener::bind("127.0.0.1:0").unwrap(), UID, GID, false); + let mut other = Client::connect(second.local_addr(), UID, GID); + let other_root = other.mnt("/"); + let (target, _) = other.lookup(&other_root, b"watched").unwrap(); + other.mkdir(&target, b"child", 0o755).unwrap(); + + // The failed LOOKUP answers the directory's current facts: the job that + // found no name read the directory's row in the same visit. Nothing may + // answer the last observation instead, which no time bound ages. + let (status, found, directory) = client.lookup_with_directory(&made.handle, b"absent"); + assert_eq!((status, found), (NOENT, None)); + let fresh = client.getattr(&made.handle).unwrap(); + assert_eq!(fresh.nlink, 3, "the directory gained a child directory"); + assert_eq!(directory, Some(fresh.clone()), "the current facts"); + assert_ne!(directory, Some(before), "never the last observation"); + + // A name this export decides without an owner job reads no directory row + // and answers no directory attributes, so a client falls back to its own + // `actimeo` rather than restarting it on facts nothing refreshes. + let (status, found, directory) = client.lookup_with_directory(&made.handle, b"."); + assert_eq!(status, 0); + assert!(found.is_some()); + assert_eq!(directory, None, "no attributes for `.`"); + let (status, found, directory) = client.lookup_with_directory(&made.handle, b".."); + assert_eq!(status, 0); + assert!(found.is_some()); + assert_eq!(directory, None, "no attributes for `..`"); + + // A silly name is kept by this export alone; its reply names none either. + let open = client.touch(&made.handle, b"open", 0o644).unwrap(); + client + .rename( + &made.handle, + b"open", + &made.handle, + b".nfs00000000000012ab00000001", + ) + .unwrap(); + let (status, found, directory) = + client.lookup_with_directory(&made.handle, b".nfs00000000000012ab00000001"); + assert_eq!(status, 0); + assert_eq!(found.map(|(handle, _)| handle), Some(open.handle)); + assert_eq!(directory, None, "no attributes for a silly name"); + + drop(client); + drop(other); + let released = first.stop(); + second.stop(); + let (_, outstanding) = rig.stop(released); + assert_eq!(outstanding, 0); + fs::remove_dir_all(base).unwrap(); +} diff --git a/engine/crates/layerfs-nfs/tests/support/spy.rs b/engine/crates/layerfs-nfs/tests/support/spy.rs index ea761166d9..c96973dafb 100644 --- a/engine/crates/layerfs-nfs/tests/support/spy.rs +++ b/engine/crates/layerfs-nfs/tests/support/spy.rs @@ -42,6 +42,12 @@ pub struct Watch { /// Each engine read window the export asked for: offset and length. One /// call may fill its whole transfer from several of them. pub reads: Mutex>, + /// Owner read visits (`observe_visit`): the jobs a LOOKUP, GETATTR or + /// OPEN spends. One LOOKUP spends exactly one. + pub visits: AtomicU64, + /// Store readers a request took for a base demand: at most one per reply, + /// never one per listed name. + pub bases: AtomicU64, } pub struct Spy { pub inner: Arc, @@ -70,9 +76,10 @@ impl RequestServices for Request { handle: u64, offset: u64, after: Option>, + attributes: bool, ) -> ServiceFuture<'_, ServiceReply>> { self.inner - .directory_visit(mount, serial, handle, offset, after) + .directory_visit(mount, serial, handle, offset, after, attributes) } fn publish_cookies( &self, @@ -123,10 +130,12 @@ impl RequestServices for Request { operation: KernelReadOperation, facts: Arc, ) -> ServiceFuture<'_, ServiceReply>> { + self.watch.visits.fetch_add(1, Ordering::AcqRel); self.inner .observe_visit(mount, request, serial, handle, operation, facts) } fn base(&self) -> ServiceFuture<'_, BaseView> { + self.watch.bases.fetch_add(1, Ordering::AcqRel); self.inner.base() } fn mutate_visit( diff --git a/engine/crates/layerfs-nfs/tests/write.rs b/engine/crates/layerfs-nfs/tests/write.rs index a6c94d0f74..8d93ae59a2 100644 --- a/engine/crates/layerfs-nfs/tests/write.rs +++ b/engine/crates/layerfs-nfs/tests/write.rs @@ -831,12 +831,13 @@ impl layerfs_mount_ports::ports::RequestServices for CountingRequest { handle: u64, offset: u64, after: Option>, + attributes: bool, ) -> layerfs_mount_ports::ports::ServiceFuture< '_, layerfs_mount_ports::ports::ServiceReply>, > { self.inner - .directory_visit(mount, serial, handle, offset, after) + .directory_visit(mount, serial, handle, offset, after, attributes) } fn publish_cookies( &self, @@ -1014,8 +1015,9 @@ fn create_write_remove_and_apple_double_avoid_redundant_owner_jobs_and_orphans() // 4. AppleDouble companion changing lifecycle (CREATE, WRITE, SETATTR, // REMOVE) issues 0 SQLite owner jobs (0 observe_visit, 0 mutate_visit), - // and LOOKUP of an absent companion issues 0 lookup observe_visit - // (only the 1 parent getattr in nfs.rs proc 3). + // and LOOKUP of an absent companion issues none either: the export + // refuses the name before any visit, and the reply's directory + // attributes come from the facts it already published. let before = counts.snapshot(); let ad = c.touch(&root, b"._single.txt", 0o644).unwrap(); // The directory's wcc_data is not a hole: a companion changes nothing in @@ -1055,8 +1057,8 @@ fn create_write_remove_and_apple_double_avoid_redundant_owner_jobs_and_orphans() assert_eq!(c.lookup(&root, b"._single.txt").map(|_| ()), Err(NOENT)); assert_eq!( counts.snapshot(), - (before.0 + 1, before.1, before.2, before.3), - "AppleDouble LOOKUP avoids the target lookup visit" + before, + "an absent AppleDouble LOOKUP issues no owner job at all" ); // 5. REMOVE drops the last link and retires the synthetic kernel's file diff --git a/engine/crates/layerfs-store-service/tests/postgresql_readdirplus_window.rs b/engine/crates/layerfs-store-service/tests/postgresql_readdirplus_window.rs new file mode 100644 index 0000000000..1fdd58a3eb --- /dev/null +++ b/engine/crates/layerfs-store-service/tests/postgresql_readdirplus_window.rs @@ -0,0 +1,617 @@ +//! LP73 Ask 1-B on a PostgreSQL Store: a READDIRPLUS window reads its +//! inherited facts in batches — one batch length demand for the window and +//! one batch inode demand for each level of the inode tree the cache does not +//! already answer — as a SQLite Store does, and answers the facts a per-name +//! LOOKUP answers. What it counts is a partly warm window: the visits before +//! it read the inode table's upper level into the same cache, so its batches +//! are not a fresh mount's first window's. +//! +//! The window is driven through the product's mount path, not a fixture store: +//! a Workspace bound on the Store, OPENDIR, the reading visit, and the window's +//! facts read outside the owner over the Workspace's own base. Those are the +//! calls `layerfs-mount-ports`' `DirectoryStream` makes for a READDIRPLUS +//! reply. `Store::work()` counts the port demands themselves, so a PostgreSQL +//! and a SQLite Store are compared on the same counters. +//! +//! Needs a PostgreSQL server: set `LAYERFS_TEST_POSTGRES_URL` (for example +//! `host=127.0.0.1 port=5432 user=layerfs password=… dbname=layerfs_test +//! sslmode=disable`). Without it the two PostgreSQL cases report NOT_RUN and +//! assert nothing, while the SQLite cases still run. The whole file is behind +//! `#![cfg(feature = "postgresql")]`, so a build without that feature runs +//! none of them. +#![cfg(feature = "postgresql")] +#[path = "support/installed_store.rs"] +#[allow(dead_code)] +mod installed; + +use layerfs_content::filesystem::attributes::PortableMetadata; +use layerfs_content::object::inode_leaf::InodeKind; +use layerfs_history::{ + BranchId, ForkRequest, ForkSource, HistoryCatalog, HistoryCatalogConfig, HistoryName, + LayerStackId, WorkspaceId, +}; +use layerfs_persistence::{PostgresHandles, PostgresSelection, StoreSelection}; +use layerfs_project::{init, InitRequest}; +use layerfs_storage::{ReservationBlocks, Storage, StoragePolicy}; +use layerfs_store_service::bootstrap::open_store_observed; +use layerfs_store_service::store::{BindRequest, StoreWork}; +use layerfs_store_service::{Owner, OwnerConfig, StoreService}; +use layerfs_telemetry::timer::Timing; +use layerfs_workspace::{ + CanonicalCache, CanonicalClient, KernelDirectoryRead, KernelDirectoryWindow, + KernelReadDecision, VisitFacts, +}; +use std::collections::BTreeMap; +use std::path::{Path, PathBuf}; +use std::sync::Arc; +use std::time::{Duration, Instant}; + +const BINDING: &[u8] = b"readdirplus-window-batches"; +const CURSOR: [u8; 32] = [73; 32]; +const PROJECT: &str = "window"; +const BRANCH_NAME: &str = "work"; +/// The daemon's Store object cache. +const CACHE_BYTES: usize = 64 * 1024 * 1024; +/// Most OPENDIR rounds one undecided visit gets before the test fails. +const ROUNDS: usize = 4; +/// The branch the SQLite fixture's installed Store already holds: stack +/// `[23; 16]`, branch `[24; 16]`, named `main` under the project `installed`. +const INSTALLED_BRANCH: [u8; 16] = [24; 16]; + +/// The names below the root: `files` regular files of known bytes, and one +/// subdirectory holding one file of its own. +fn tree(files: usize) -> Vec<(String, Vec)> { + (0..files) + .map(|n| { + let name = format!("f{n:04}"); + let bytes = vec![(n % 251) as u8; 16 + n % 97]; + (name, bytes) + }) + .collect() +} + +fn build(source: &Path, files: usize) { + for (name, bytes) in tree(files) { + std::fs::write(source.join(name), bytes).unwrap(); + } + let sub = source.join("sub"); + std::fs::create_dir(&sub).unwrap(); + std::fs::write(sub.join("inner"), b"the subdirectory's only file\n").unwrap(); +} + +/// The port demands one window's facts made. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +struct Counts { + object_batches: u64, + object_ids: u64, + length_batches: u64, + length_ids: u64, +} +impl Counts { + fn between(before: StoreWork, after: StoreWork) -> Self { + Self { + object_batches: after.object_batches - before.object_batches, + object_ids: after.object_ids - before.object_ids, + length_batches: after.length_batches - before.length_batches, + length_ids: after.length_ids - before.length_ids, + } + } +} +/// One entry's facts as the reply showed them. +#[derive(Clone, Debug, Eq, PartialEq)] +struct Fact { + name: String, + kind: InodeKind, + metadata: PortableMetadata, + logical_len: u64, + subdirs: u64, +} +impl Fact { + /// What two Stores must answer identically: name, kind, permission bits, + /// length and child-directory count. The mtime is the source tree's, and + /// each test builds its own. + fn shape(&self) -> (&str, InodeKind, u32, u64, u64) { + ( + &self.name, + self.kind, + self.metadata.mode, + self.logical_len, + self.subdirs, + ) + } +} + +/// What one READDIRPLUS window answered and what it cost. +struct Measured { + counts: Counts, + facts: Vec, + exhausted: bool, +} + +/// One READDIRPLUS window of `branch`'s root, driven as a mount drives +/// it: OPENDIR, the reading visit, one Store reader for the window's facts, +/// and the reply's offsets published and then reused. +fn measure( + service: &StoreService, + owner: &Owner, + files: usize, + tag: &str, + branch: BranchId, +) -> Measured { + let bound = service + .store + .bind( + owner.client(), + BindRequest { + workspace: WorkspaceId::from_authority([91; 32]).unwrap(), + branch, + }, + ) + .unwrap_or_else(|refusal| panic!("{tag}: bind refused: {refusal:?}")) + .workspace; + let operation = bound.operation().unwrap(); + let route = bound.route(); + let jobs = operation.overlay(); + let workspace = operation.workspace(); + let base = workspace.base().unwrap(); + let root = base.root().root_inode().serial(); + let mount = jobs.kernel_mount(route, root).unwrap(); + // The cache starts empty, so every fact this window shows is the Store's + // to read. The OPENDIR below and the facts it supplies fill it with the + // inode table's upper level, so the window that follows is not a fresh + // mount's first one. + let resident = Arc::new(CanonicalClient::resident(Arc::new(CanonicalCache::new( + 1 << 20, + )))); + + // OPENDIR: owner jobs that ask the Store nothing, and the facts they name + // read outside them. + let mut facts = VisitFacts::default(); + let mut directory = None; + for _ in 0..ROUNDS { + let visit = workspace + .kernel_opendir_visit(resident.clone(), mount, 1, root, Arc::new(facts.clone())) + .unwrap(); + let before = service.store.work(); + let outcome = jobs.kernel_observe_visit(route, Box::new(visit)).unwrap(); + assert_eq!( + service.store.work(), + before, + "{tag}: an owner job made a Store demand" + ); + match &outcome.decision { + Some(KernelReadDecision::Needs(needs)) => facts + .supply(&workspace.base().unwrap(), needs, None) + .unwrap(), + Some(KernelReadDecision::Value(_)) => { + directory = outcome.directory_candidate; + break; + } + other => panic!("{tag}: undecided OPENDIR: {other:?}"), + } + } + let directory = directory.unwrap_or_else(|| panic!("{tag}: an open directory")); + + // The reading visit: one read-only owner job that makes no Store demand. + let visit = workspace + .kernel_directory_visit( + resident.clone(), + mount, + root, + directory.owner_id(), + KernelDirectoryRead { + offset: 2, + after: None, + attributes: true, + }, + ) + .unwrap(); + let before_visit = service.store.work(); + let reply = jobs.kernel_directory_visit(route, visit).unwrap(); + assert_eq!( + service.store.work(), + before_visit, + "{tag}: the reading visit made a Store demand" + ); + let window = KernelDirectoryWindow::clone(&reply); + drop(reply); + assert!( + window.needs_base(true), + "{tag}: the window leaves its facts to the Store" + ); + + // The window's facts, outside the owner, over the Workspace's reader: one + // batch length demand for the window and one batch inode demand per tree + // level the OPENDIR left unread. + let before = service.store.work(); + let batch = window.finish_with(Some(&base), true).unwrap(); + let after = service.store.work(); + let counts = Counts::between(before, after); + eprintln!("{tag}: port demands of one window: {counts:?}"); + + let names: Vec> = batch + .entries + .iter() + .map(|entry| entry.name.clone()) + .collect(); + let mut shown = Vec::with_capacity(batch.entries.len()); + for entry in &batch.entries { + let name = String::from_utf8(entry.name.clone()).unwrap(); + let stat = entry + .stat + .as_ref() + .unwrap_or_else(|| panic!("{tag}: no facts for {name}")); + assert_eq!(stat.kind, entry.kind, "{tag}: kind of {name}"); + // A per-name LOOKUP answers the same facts, field for field: the + // window's batch read and the single demand agree. + let lookup = base.stat(entry.serial).unwrap(); + assert_eq!(stat.kind, lookup.value.kind, "{tag}: LOOKUP kind of {name}"); + assert_eq!( + stat.metadata, lookup.metadata, + "{tag}: LOOKUP metadata of {name}" + ); + assert_eq!( + stat.logical_len, lookup.logical_len, + "{tag}: LOOKUP length of {name}" + ); + shown.push(Fact { + name, + kind: stat.kind, + metadata: stat.metadata, + logical_len: stat.logical_len, + subdirs: stat.subdirs, + }); + } + // Every inherited name and the subdirectory, exactly once, with the bytes + // the fixture wrote. + let expected: BTreeMap = tree(files) + .into_iter() + .map(|(name, bytes)| (name, bytes.len())) + .collect(); + assert_eq!(shown.len(), files + 1, "{tag}: every name of the window"); + for fact in &shown { + if let Some(len) = expected.get(&fact.name) { + assert_eq!( + fact.kind, + InodeKind::RegularFile, + "{tag}: kind {}", + fact.name + ); + assert_eq!(fact.logical_len, *len as u64, "{tag}: length {}", fact.name); + } else { + assert_eq!(fact.name, "sub", "{tag}: the subdirectory"); + assert_eq!(fact.kind, InodeKind::Directory, "{tag}: kind sub"); + assert_eq!(fact.subdirs, 0, "{tag}: sub holds no directory"); + assert_eq!(fact.logical_len, 0, "{tag}: a directory has no length"); + } + } + + // The reply's offsets are published once, and a request inside that reply + // is answered at them instead of recording them again: the handles a + // READDIRPLUS reply hands out follow these offsets. + let first = batch.first; + let exhausted = batch.exhausted; + assert!(exhausted, "{tag}: one window lists the root"); + let offer = batch + .publish + .clone() + .unwrap_or_else(|| panic!("{tag}: a fresh reply publishes its names")); + jobs.kernel_directory_publish(route, offer, names.clone()) + .unwrap(); + let reuse = workspace + .kernel_directory_visit( + resident.clone(), + mount, + root, + directory.owner_id(), + KernelDirectoryRead { + offset: 2, + after: None, + attributes: true, + }, + ) + .unwrap(); + let reply = jobs.kernel_directory_visit(route, reuse).unwrap(); + let window = KernelDirectoryWindow::clone(&reply); + drop(reply); + let reused = window.finish_with(Some(&base), true).unwrap(); + assert_eq!( + reused + .entries + .iter() + .map(|entry| entry.name.clone()) + .collect::>(), + names, + "{tag}: the published reply is answered at its offsets" + ); + assert_eq!(reused.first, first, "{tag}: at the published offsets"); + assert!( + reused.publish.is_none(), + "{tag}: a reused reply records nothing again" + ); + // A seek inside the published reply resumes strictly after its name, and + // hands out a fresh range for what follows it. + let seek = workspace + .kernel_directory_visit( + resident, + mount, + root, + directory.owner_id(), + KernelDirectoryRead { + offset: first + names.len() as u64 - 2, + after: None, + attributes: true, + }, + ) + .unwrap(); + let reply = jobs.kernel_directory_visit(route, seek).unwrap(); + let window = KernelDirectoryWindow::clone(&reply); + drop(reply); + let next = window.finish_with(Some(&base), true).unwrap(); + assert_eq!( + next.entries + .iter() + .map(|entry| entry.name.clone()) + .collect::>(), + names[names.len() - 1..], + "{tag}: the offset resumes strictly after its name" + ); + assert!( + next.entries.iter().all(|entry| entry.stat.is_some()), + "{tag}: the resumed window's facts too" + ); + + jobs.kernel_directory_close(route, mount, root, directory.owner_id()) + .unwrap(); + jobs.kernel_revoke(route, mount).unwrap(); + bound.close().unwrap(); + owner.client().retire_namespace(route.namespace()).unwrap(); + Measured { + counts, + facts: shown, + exhausted, + } +} + +fn catalog() -> HistoryCatalogConfig { + HistoryCatalogConfig { + binding_key: BINDING.to_vec(), + cursor_key: CURSOR, + incarnation: 1, + } +} + +fn url() -> Option { + let url = std::env::var("LAYERFS_TEST_POSTGRES_URL").ok(); + if url.is_none() { + eprintln!("NOT_RUN: LAYERFS_TEST_POSTGRES_URL is unset"); + } + url +} + +/// The test's own PostgreSQL schema and source tree, dropped when it ends, +/// pass or fail. +struct Postgres { + url: String, + schema: String, + directory: PathBuf, + stack: LayerStackId, +} +impl Postgres { + fn new(tag: &str, url: &str, files: usize, stack: [u8; 16]) -> Self { + let directory = + std::env::temp_dir().join(format!("lfs73-window-{tag}-{}", std::process::id())); + std::fs::create_dir_all(&directory).unwrap(); + let schema = format!("lfs73_window_{tag}_{}", std::process::id()); + let source = directory.join("source"); + std::fs::create_dir(&source).unwrap(); + build(&source, files); + Self { + url: url.to_owned(), + schema, + directory, + stack: LayerStackId::from_authority(stack), + } + } + /// One PostgreSQL Store holding the fixture's tree on one Branch. + fn open(&self) -> (Owner, StoreService) { + let selection = PostgresSelection::new(&self.url, &self.schema); + let created = + PostgresHandles::create(&selection, StoragePolicy::frozen_default(), &catalog()) + .unwrap(); + let storage = Storage::new(created.storage.clone()).unwrap(); + let initialized = Timing::disabled("init", |timing| { + init( + &storage, + &created.history, + InitRequest { + source: &self.directory.join("source"), + acquisition: &created.acquisition, + stack: self.stack, + name: HistoryName::new(PROJECT).unwrap(), + scope_seed: [75; 32], + deadline: Instant::now() + Duration::from_secs(120), + }, + timing, + ) + }) + .0 + .unwrap(); + created + .history + .fork(&ForkRequest { + stack: initialized.stack.id, + branch: pg_branch(self.stack), + name: HistoryName::new(BRANCH_NAME).unwrap(), + source: ForkSource::Layer(initialized.stack.head_layer), + }) + .unwrap(); + drop(storage); + drop(created); + let config = OwnerConfig::default(); + let opened = open_store_observed( + StoreSelection::Postgres(selection), + BINDING, + CURSOR, + config.namespaces, + CACHE_BYTES, + ReservationBlocks::default(), + ) + .unwrap(); + let owner = Owner::start( + &self.directory.join("overlay.sqlite"), + layerfs_overlay::ProfileConfig::default(), + config, + ) + .unwrap(); + let service = StoreService::new(opened, &owner); + (owner, service) + } +} +impl Drop for Postgres { + fn drop(&mut self) { + if let Ok(mut client) = postgres::Client::connect(&self.url, postgres::NoTls) { + let _ = client.batch_execute(&format!("DROP SCHEMA IF EXISTS {} CASCADE", self.schema)); + } + let _ = std::fs::remove_dir_all(&self.directory); + } +} + +/// The Branch this test's PostgreSQL fixtures fork. +fn pg_branch(stack: LayerStackId) -> BranchId { + BranchId::named(stack, &HistoryName::new(BRANCH_NAME).unwrap()) +} + +/// One READDIRPLUS window over an Init'd PostgreSQL Store's root. +fn postgres_window(url: &str, files: usize, tag: &str, stack: [u8; 16]) -> Measured { + let fixture = Postgres::new(tag, url, files, stack); + let (owner, service) = fixture.open(); + let measured = measure(&service, &owner, files, tag, pg_branch(fixture.stack)); + drop(service); + owner.stop().unwrap(); + measured +} + +/// The same window over a SQLite Store built from the same tree, through the +/// installed-Store fixture the Store's own tests use. +fn sqlite_window(files: usize, tag: &str) -> Measured { + let host = installed::Fixture::built(tag, |source| { + build(source, files); + files + 2 + }); + let config = OwnerConfig::default(); + let opened = open_store_observed( + host.config.clone(), + installed::BINDING, + installed::CURSOR, + config.namespaces, + CACHE_BYTES, + ReservationBlocks::default(), + ) + .unwrap(); + let owner = Owner::start( + &host.directory.join("overlay.sqlite"), + layerfs_overlay::ProfileConfig::default(), + config, + ) + .unwrap(); + let service = StoreService::new(opened, &owner); + let measured = measure( + &service, + &owner, + files, + tag, + BranchId::from_authority(INSTALLED_BRANCH), + ); + drop(service); + owner.stop().unwrap(); + host.cleanup(); + measured +} + +#[test] +fn a_postgresql_window_reads_its_facts_in_batches() { + let Some(url) = url() else { + return; + }; + let small = postgres_window(&url, 8, "small", [81; 16]); + let large = postgres_window(&url, 200, "large", [82; 16]); + // One length demand carries the whole window, and it names each ordinary + // file once: no entry asks the provider for its own length. + for (measured, files) in [(&small, 8u64), (&large, 200)] { + assert_eq!(measured.counts.length_batches, 1, "one length demand"); + assert_eq!( + measured.counts.length_ids, files, + "the one length demand names each file once" + ); + } + // The window's object demands and length demands do not grow with the + // names it lists: one batch length demand carries the window's ordinary + // files, and one batch inode demand carries its inherited serials for each + // tree level the cache does not already hold, so the names add no batch + // while a deeper tree adds one. Here the visits before the window read the + // tree's upper level into the same cache, leaving one level: these are not + // a fully cold window's counts. This + // tree's metadata roots are inline and its subdirectory's child count was + // built with the Commit, so no entry here costs a read of its own; an + // entry whose metadata root is an object, a subdirectory counted by a scan + // and a symlink's target are still read one at a time. + assert_eq!( + large.counts.object_batches, small.counts.object_batches, + "8 names: {:?}, 200 names: {:?}", + small.counts, large.counts + ); + for (measured, files) in [(&small, 8u64), (&large, 200)] { + assert_eq!( + measured.counts.object_batches, 1, + "{files} names: one batch inode demand" + ); + } + assert_eq!(small.facts.len(), 9); + assert_eq!(large.facts.len(), 201); + assert!(small.exhausted && large.exhausted); +} + +#[test] +fn a_sqlite_window_makes_the_same_batches_and_answers_the_same_facts() { + let sqlite = sqlite_window(40, "sqlite"); + assert_eq!( + (sqlite.counts.object_batches, sqlite.counts.length_batches), + (1, 1), + "one batch inode demand and one length demand: {:?}", + sqlite.counts + ); + assert_eq!(sqlite.counts.length_ids, 40); + assert_eq!(sqlite.facts.len(), 41); + let Some(url) = url() else { + return; + }; + // The same tree on a PostgreSQL Store, mounted and listed the same way. + let postgres = postgres_window(&url, 40, "compare", [83; 16]); + assert_eq!( + ( + postgres.counts.object_batches, + postgres.counts.length_batches, + postgres.counts.length_ids + ), + ( + sqlite.counts.object_batches, + sqlite.counts.length_batches, + sqlite.counts.length_ids + ), + "the same demands on both Stores: PostgreSQL {:?}, SQLite {:?}", + postgres.counts, + sqlite.counts + ); + // What both Stores must answer identically. The mtime of each fixture's + // own source tree differs between the two runs; it is compared there with + // that tree's own LOOKUP, not across Stores. + assert_eq!( + postgres.facts.iter().map(Fact::shape).collect::>(), + sqlite.facts.iter().map(Fact::shape).collect::>(), + "the same facts on both Stores" + ); + // The offset a reply starts at is the handle's own counter, not a cross + // Store contract; that it lists the whole directory in one window is. + assert!(postgres.exhausted && sqlite.exhausted); +} diff --git a/engine/crates/layerfs-store-service/tests/refresh_lookups.rs b/engine/crates/layerfs-store-service/tests/refresh_lookups.rs index d0c21d221e..0874986666 100644 --- a/engine/crates/layerfs-store-service/tests/refresh_lookups.rs +++ b/engine/crates/layerfs-store-service/tests/refresh_lookups.rs @@ -208,7 +208,7 @@ fn refresh_lookups() { visits += 1; match &outcome.decision { Some(KernelReadDecision::Value(_)) => break, - Some(KernelReadDecision::Refused(_)) => { + Some(KernelReadDecision::Refused { .. }) => { missing += 1; break; } diff --git a/engine/crates/layerfs-workspace/src/lib.rs b/engine/crates/layerfs-workspace/src/lib.rs index 323588aa04..dd93212f7e 100644 --- a/engine/crates/layerfs-workspace/src/lib.rs +++ b/engine/crates/layerfs-workspace/src/lib.rs @@ -55,12 +55,13 @@ pub use operations::file::recognize::{ }; pub use operations::kernel_data::{KernelDataVisit, KernelWindow}; pub use operations::kernel_directory::{ - KernelDirectoryBatch, KernelDirectoryEntry, KernelDirectoryListing, KernelDirectoryVisit, - KernelDirectoryWindow, + KernelDirectoryBatch, KernelDirectoryEntry, KernelDirectoryListing, KernelDirectoryRead, + KernelDirectoryVisit, KernelDirectoryWindow, }; pub use operations::kernel_mutation::KernelMutationOutcome; pub use operations::kernel_read::{ - KernelReadDecision, KernelReadFailure, KernelReadOperation, KernelReadOutcome, KernelReadValue, + Decided, KernelReadDecision, KernelReadFailure, KernelReadOperation, KernelReadOutcome, + KernelReadValue, }; pub use operations::kernel_visit::{ KernelInput, KernelMutationVisit, KernelReadVisit, KernelVisitRequest, diff --git a/engine/crates/layerfs-workspace/src/operations/kernel_directory.rs b/engine/crates/layerfs-workspace/src/operations/kernel_directory.rs index c6a9d54b63..10764718f2 100644 --- a/engine/crates/layerfs-workspace/src/operations/kernel_directory.rs +++ b/engine/crates/layerfs-workspace/src/operations/kernel_directory.rs @@ -3,16 +3,29 @@ //! kernel offset, and one window of current local names with their kinds. //! Inherited names and kinds are merged inside the job when the canonical //! cache holds them; otherwise the request merges them outside the owner -//! over one Store reader, at the base root the visit named. The request then -//! fills its reply and a publishing visit records exactly the accepted -//! names. Nothing is held between the two visits or after the reply. +//! over one Store reader, at the base root the visit named. A reply that +//! shows each entry's facts (READDIRPLUS) takes them from the same memory +//! first; what is left is read outside the owner once the reply's published +//! names are known: one batch length demand and one batch inode demand for +//! each level of the inode tree the cache does not already answer, for the +//! names that reply sends and not for the rest of the window, so those counts +//! follow the tree's depth and the reply's size and not the number of listed +//! names. What those batches +//! do not cover is per entry and unchanged here: a non-inline metadata root, +//! a listed subdirectory's child-directory count, and a symlink's target are +//! still read once for each entry that needs them. The request then fills its +//! reply and a publishing visit records exactly the accepted names. Nothing +//! is held between the two visits or after the reply. use crate::{BaseView, SourceView, ViewStat, Workspace, WorkspaceResult}; -use layerfs_content::{object::inode_leaf::InodeKind, ContentError}; +use layerfs_content::filesystem::inode::{lookup_many, InodeReadWork}; +use layerfs_content::filesystem::Resolved; +use layerfs_content::object::inode_leaf::{InodeKind, InodeValue}; +use layerfs_content::{ContentError, ObjectId}; use layerfs_overlay::{ KernelCookieOffer, KernelDirectoryCursor, KernelDirectoryPage, KernelMount, Overlay, OverlayError, OverlayResult, PAGE_ROWS, }; -use std::{fmt, sync::Arc}; +use std::{collections::BTreeMap, collections::BTreeSet, fmt, sync::Arc}; #[derive(Clone, Debug, Eq, PartialEq)] pub struct KernelDirectoryEntry { @@ -30,6 +43,16 @@ pub struct KernelDirectoryListing { pub continuation: Option>, pub visited: usize, } +/// Where one enumeration request reads and what its reply shows. +#[derive(Clone, Debug, Default, Eq, PartialEq)] +pub struct KernelDirectoryRead { + /// The kernel offset this request resumes at. + pub offset: u64, + /// The name it continues after, when it continues an earlier window. + pub after: Option>, + /// The reply shows each entry's facts (READDIRPLUS). + pub attributes: bool, +} /// One READDIR reading visit as one read-only owner job. #[derive(Clone)] pub struct KernelDirectoryVisit { @@ -39,20 +62,23 @@ pub struct KernelDirectoryVisit { offset: u64, after: Option>, resident: BaseView, + /// The reply shows each entry's facts (READDIRPLUS): this visit answers + /// from memory what it holds, and the request reads only what it left. + attributes: bool, } /// What the reading visit found, as plain values its request owns. #[derive(Clone, Debug)] pub struct KernelDirectoryWindow { pub page: KernelDirectoryPage, /// The merged names of the window, when memory answered every inherited - /// name and kind inside the job. + /// name and kind inside the job; a reply that shows facts also takes + /// from here every fact memory held. pub listing: Option, /// The offsets the reply may hand out. Absent when the job decided that /// the window lists no name. pub offer: Option, /// The facts of the directory being listed, when the visit read its own - /// local row and the reply shows them: a READDIRPLUS reply's - /// `dir_attributes` then costs no further read. + /// local row: a reply's `dir_attributes` then costs no further read. pub directory: Option, } /// The entries one reply may offer, in order, with their offsets. @@ -87,17 +113,22 @@ impl fmt::Debug for KernelDirectoryVisit { } } impl Workspace { - /// `resident` answers from memory only. No I/O here. `after` continues - /// an earlier window of the same request past names that listed nothing. + /// `resident` answers from memory only. No I/O here. `read` names the + /// position (continuing past names that listed nothing) and whether the + /// reply shows each entry's facts. pub fn kernel_directory_visit( &self, resident: Arc, mount: KernelMount, serial: u64, handle: u64, - offset: u64, - after: Option>, + read: KernelDirectoryRead, ) -> WorkspaceResult { + let KernelDirectoryRead { + offset, + after, + attributes, + } = read; let base = self.base()?; if mount.engine() != self.route().engine() || mount.root_serial() != base.root().root_inode().serial() @@ -111,6 +142,7 @@ impl Workspace { offset, after, resident: base.with_client(resident), + attributes, }) } } @@ -145,7 +177,25 @@ impl KernelDirectoryVisit { } let listing = (self.resident.identity().0.to_bytes() == page.source().root()) .then(|| listing(&self.resident, &page).ok()) - .flatten(); + .flatten() + .map(|Kinds { listing, values }| { + let mut listing = listing; + // A reply that shows facts takes the local rows this window + // already read and whatever else memory answers, from the base + // values the kinds came from: no inode is read twice here, so + // only the names memory left undecided take the one Store read. + if self.attributes { + for entry in &mut listing.entries { + entry.stat = memory_stat( + &self.resident, + &page, + entry.serial, + values.get(&entry.serial), + ); + } + } + listing + }); let offer = match &listing { Some(listing) if listing.entries.is_empty() => None, _ => Some(db.offer_kernel_cookies(&page)?), @@ -169,6 +219,16 @@ impl KernelDirectoryWindow { pub fn rewound(&self) -> bool { *self.page.cursor() == KernelDirectoryCursor::Rewound } + /// Whether this reply needs the Store reader at all: the visit left the + /// merge undone, or the reply shows facts of a name memory did not hold. + /// A local directory, and a window whose inherited entries are all + /// resident, take no reader and issue no Store read. + pub fn needs_base(&self, attributes: bool) -> bool { + match &self.listing { + None => true, + Some(listing) => attributes && listing.entries.iter().any(|entry| entry.stat.is_none()), + } + } /// The entries this window may offer and their offsets. `base` is the /// Workspace's base over an admitted Store reader and is required when /// the visit left the merge undone; it is read at the root the visit @@ -180,14 +240,23 @@ impl KernelDirectoryWindow { /// the offer's fresh range. /// /// A plain READDIR reply shows no entry's facts, so this reads no - /// portable metadata and no file length. + /// portable metadata and no file length for one: only the names, their + /// kinds and their offsets. pub fn finish(self, base: Option<&BaseView>) -> WorkspaceResult { self.finish_with(base, false) } /// The same for a reply that also shows what each accepted entry is, as - /// READDIRPLUS does: every entry's facts and the directory's own are read - /// here, outside the owner, from the authoritative base — a local entry's - /// facts are its current row, an inherited one's its immutable inode. + /// READDIRPLUS does. Names and kinds come first — from memory in the + /// reading visit, else here in one batch inode demand for each level of the + /// inode tree the cache does not already answer — and the offer then + /// truncates that window to the names this reply publishes, so only those + /// names' facts are read: one batch length demand for them and the same + /// per-level batches. A re-sent reply therefore reads nothing for a name it + /// will not send. Those batches are not the whole + /// cost of an entry: + /// [`base_facts`] still reads an inherited inode's non-inline metadata + /// root, a listed subdirectory's child-directory count and a symlink's + /// target once per entry that needs them, as the base route always did. pub fn finish_with( self, base: Option<&BaseView>, @@ -199,15 +268,22 @@ impl KernelDirectoryWindow { offer, directory, } = self; - // A reply that shows an entry's facts reads the authoritative base, - // not the resident one the reading visit merged with. + // The authoritative base is read at the root the visit named, and + // only when memory did not answer the window's names or its facts. + let undecided = attributes + && merged + .as_ref() + .is_none_or(|listing| listing.entries.iter().any(|entry| entry.stat.is_none())); let base = match base { - Some(base) if attributes || merged.is_none() => Some(base.at(page.source().root())?), + Some(base) if merged.is_none() || undecided => Some(base.at(page.source().root())?), _ => None, }; let listing = match (merged, base.as_deref()) { - (Some(listing), _) if !attributes => listing, - (_, Some(base)) => listing(base, &page)?, + (Some(listing), _) => listing, + // Names and kinds only: a re-sent reply that publishes fewer + // names than this window lists reads no fact of one it will not + // send. `facts` reads what the truncated reply does send. + (_, Some(base)) => listing(base, &page)?.listing, (_, None) => { return Err( ContentError::InvalidRecord("kernel directory window without its base").into(), @@ -238,15 +314,11 @@ impl KernelDirectoryWindow { _ => (offer.first(), Some(offer)), }, }; - // Only the names this reply actually offers are read, and only a - // reply that shows facts reads anything at all. - let (directory, entries) = match base.as_deref() { - Some(base) if attributes => ( - directory.or_else(|| base_stat(base, page.directory().serial()).ok()), - facts(base, &page, entries)?, - ), - _ => (None, entries), - }; + if attributes { + if let Some(base) = base.as_deref() { + facts(base, &page, &mut entries)?; + } + } let exhausted = !truncated && listing.continuation.is_none(); Ok(KernelDirectoryBatch { entries, @@ -258,33 +330,129 @@ impl KernelDirectoryWindow { }) } } -/// One listing's entries with the facts a reply shows for each: a local -/// serial's current row, or an inherited one's immutable base inode. +/// Fills the facts a reply shows for the entries that still lack them: the +/// window's inherited serials in one batch inode demand for each level of the +/// inode tree the cache does not already answer, and the lengths of its +/// regular files in one batch length demand, so listing more names costs no +/// more of either while a deeper tree costs one more inode batch per level. A +/// local serial's facts are its current row. What +/// [`base_facts`] still reads once per entry is not batched here. fn facts( base: &BaseView, page: &KernelDirectoryPage, - entries: Vec, -) -> WorkspaceResult> { - entries + entries: &mut [KernelDirectoryEntry], +) -> WorkspaceResult<()> { + let inherited: Vec = entries + .iter() + .filter(|entry| entry.stat.is_none() && local_row(page, entry.serial).is_none()) + .map(|entry| entry.serial) + .collect::>() .into_iter() - .map(|mut entry| { - entry.stat = Some( - match page - .local_inodes - .binary_search_by_key(&entry.serial, |row| row.0) - { - Ok(index) => ViewStat::from(page.local_inodes[index].1.clone()), - Err(_) => base_stat(base, entry.serial)?, - }, - ); - Ok(entry) - }) - .collect() + .collect(); + let found = inherited_values(base, &inherited)?; + for entry in entries.iter_mut().filter(|entry| entry.stat.is_none()) { + entry.stat = Some(match local_row(page, entry.serial) { + Some(row) => ViewStat::from(row.clone()), + None => { + let value = *found.get(&entry.serial).ok_or(ContentError::PathNotFound)?; + base_facts(base, entry.serial, value)? + } + }); + } + Ok(()) +} +/// The base values of the window's inherited serials, in one batch inode +/// demand for each level of the inode tree the cache does not already answer. +fn inode_values(base: &BaseView, serials: &[u64]) -> WorkspaceResult> { + if serials.is_empty() { + return Ok(BTreeMap::new()); + } + let values = lookup_many( + base.client().as_ref(), + base.table(), + serials, + &mut InodeReadWork::default(), + )?; + Ok(serials + .iter() + .copied() + .zip(values) + .filter_map(|(serial, value)| Some((serial, value?))) + .collect()) +} +/// The base values of the window's inherited serials, in one batch inode +/// demand for each level of the inode tree the cache does not already answer, +/// after one length demand remembered the window's ordinary file lengths. +fn inherited_values( + base: &BaseView, + serials: &[u64], +) -> WorkspaceResult> { + let values = inode_values(base, serials)?; + // Every ordinary file's length, from the one demand this window makes. + let roots: Vec = values + .values() + .filter(|value| value.kind == InodeKind::RegularFile) + .map(|value| value.content_root) + .collect(); + if !roots.is_empty() { + base.client().provide_lengths(&roots)?; + } + Ok(values) +} +/// The facts of one inherited name: its base inode, the metadata every +/// attribute carries, its content length and, for a directory, the count its +/// POSIX link count is derived from. The metadata root, a directory's child +/// directory count and a symlink's target are read here one entry at a time: +/// the base has no batch port for them, and this route adds none. +fn base_facts(base: &BaseView, serial: u64, value: InodeValue) -> WorkspaceResult { + let stat = + base.stat_resolved_with_lengths(Resolved { serial, value }, base.client().as_ref())?; + let subdirs = match stat.value.kind { + InodeKind::Directory => base.subdirs(stat.value)?, + _ => 0, + }; + Ok(ViewStat::of_base(stat, subdirs)) +} +/// The local row of one listed serial at this window's view, when it has one. +fn local_row(page: &KernelDirectoryPage, serial: u64) -> Option<&layerfs_overlay::Inode> { + page.local_inodes + .binary_search_by_key(&serial, |row| row.0) + .ok() + .map(|index| &page.local_inodes[index].1) +} +/// The content kind of a local row's kind. +const fn kind_of(kind: layerfs_overlay::InodeKind) -> InodeKind { + match kind { + layerfs_overlay::InodeKind::File => InodeKind::RegularFile, + layerfs_overlay::InodeKind::Directory => InodeKind::Directory, + layerfs_overlay::InodeKind::Symlink => InodeKind::Symlink, + } +} +/// The facts of one entry as far as memory alone answers them: a local row, +/// or an inherited inode whose value this window's listing already read, with +/// the metadata, length and child-directory count this daemon's caches hold. +/// Anything else is absent here, and the request's one Store read supplies it. +fn memory_stat( + base: &BaseView, + page: &KernelDirectoryPage, + serial: u64, + value: Option<&InodeValue>, +) -> Option { + if let Some(row) = local_row(page, serial) { + return Some(ViewStat::from(row.clone())); + } + base_facts(base, serial, *value?).ok() +} +/// One window's listed names over `base`, before their kinds and facts. +struct MergedNames { + names: Vec<(Vec, u64)>, + continuation: Option>, + visited: usize, } /// One window's merged names over `base`, which must be the page's own base /// root. This issues no SQL and acquires no kernel lookup reference. An /// empty result with a continuation needs another window of the same request. -fn listing(base: &BaseView, page: &KernelDirectoryPage) -> WorkspaceResult { +fn merge(base: &BaseView, page: &KernelDirectoryPage) -> WorkspaceResult { let view = SourceView { base: base.clone(), source: page.source(), @@ -306,8 +474,8 @@ fn listing(base: &BaseView, page: &KernelDirectoryPage) -> WorkspaceResult WorkspaceResult match page.local_inodes[index].1.kind { - layerfs_overlay::InodeKind::File => InodeKind::RegularFile, - layerfs_overlay::InodeKind::Directory => InodeKind::Directory, - layerfs_overlay::InodeKind::Symlink => InodeKind::Symlink, - }, - Err(_) => view.base().inode(serial)?.value.kind, + Ok(MergedNames { + names: listing.entries, + continuation: listing.continuation, + visited: listing.visited, + }) +} +/// One window's merged names over `base` with their kinds, and the base +/// values those inherited kinds came from: a reply that shows facts takes them +/// from here, so no inode this listing read is read again. +struct Kinds { + listing: KernelDirectoryListing, + /// Base values of the inherited names, by serial. + values: BTreeMap, +} +/// One window's merged names over `base` with their kinds: a local row answers +/// its own kind, and the window's inherited serials come in one batch inode +/// demand for each level of the inode tree the cache does not already answer. +/// Kinds alone cost no length demand, which belongs to a reply's facts. +fn listing(base: &BaseView, page: &KernelDirectoryPage) -> WorkspaceResult { + let merged = merge(base, page)?; + let inherited: Vec = merged + .names + .iter() + .map(|(_, serial)| *serial) + .filter(|serial| local_row(page, *serial).is_none()) + .collect::>() + .into_iter() + .collect(); + let values = inode_values(base, &inherited)?; + let mut entries = Vec::with_capacity(merged.names.len()); + for (name, serial) in merged.names { + let kind = match local_row(page, serial) { + Some(row) => kind_of(row.kind), + None => { + let value = *values.get(&serial).ok_or(ContentError::PathNotFound)?; + value.kind + } }; entries.push(KernelDirectoryEntry { name, @@ -335,20 +528,12 @@ fn listing(base: &BaseView, page: &KernelDirectoryPage) -> WorkspaceResult WorkspaceResult { - let stat = base.stat(serial)?; - let subdirs = match stat.value.kind { - InodeKind::Directory => base.subdirs(stat.value)?, - _ => 0, - }; - Ok(ViewStat::of_base(stat, subdirs)) -} diff --git a/engine/crates/layerfs-workspace/src/operations/kernel_read.rs b/engine/crates/layerfs-workspace/src/operations/kernel_read.rs index a0bfea566e..ad457daa71 100644 --- a/engine/crates/layerfs-workspace/src/operations/kernel_read.rs +++ b/engine/crates/layerfs-workspace/src/operations/kernel_read.rs @@ -32,15 +32,34 @@ pub enum KernelReadOperation { #[derive(Debug)] pub enum KernelReadDecision { Needs(Vec), - Value(Inode), - Refused(Refusal), + Value(Decided), + /// A refusal and, when the refusing job read it, the directory that job + /// read: a LOOKUP that binds no name still read its parent's row, so the + /// failure's directory attributes are those fresh facts and cost no + /// second visit. `None` when no job read a directory (a refusal decided + /// before the visit, or one about the name itself). + Refused { + refusal: Refusal, + directory: Option, + }, Failed(WorkspaceError), } +/// What a read decided: the inode it names and, for a LOOKUP, the directory +/// it named it in. That job read the directory's own row to resolve the name, +/// so a reply that shows the directory's attributes needs no second visit. +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct Decided { + pub inode: Inode, + pub parent: Option, +} pub type KernelReadOutcome = KernelObservation; #[derive(Debug)] pub struct KernelReadValue { pub stat: ViewStat, + /// The directory a LOOKUP named the inode in, from the same job: a reply + /// that carries the directory's attributes reads nothing more for them. + pub parent: Option, pub file: Option, pub directory: Option, pub original: Arc, @@ -90,20 +109,25 @@ pub(crate) fn decide_read( kernel: None, needs: Vec::new(), }; - let decision = match decide(operation, &mut eval) { - Ok(Some(inode)) => KernelReadDecision::Value(inode), + // The directory a refusing decision read, when it read one. + let mut refused_directory = None; + let decision = match decide(operation, &mut eval, &mut refused_directory) { + Ok(Some(decided)) => KernelReadDecision::Value(decided), Ok(None) if !eval.needs.is_empty() => KernelReadDecision::Needs(eval.needs), Ok(None) => KernelReadDecision::Failed( ContentError::InvalidRecord("empty kernel read needs").into(), ), Err(WorkspaceError::Overlay(error)) => return Err(error), - Err(WorkspaceError::Refused(refusal)) => KernelReadDecision::Refused(refusal), + Err(WorkspaceError::Refused(refusal)) => KernelReadDecision::Refused { + refusal, + directory: refused_directory, + }, Err(error) => KernelReadDecision::Failed(error), }; Ok(match decision { KernelReadDecision::Needs(_) => KernelDecision::Needs(decision), - KernelReadDecision::Value(ref inode) => KernelDecision::Finished { - inode: Some(inode.clone()), + KernelReadDecision::Value(ref decided) => KernelDecision::Finished { + inode: Some(decided.inode.clone()), value: decision, }, _ => KernelDecision::Finished { @@ -112,20 +136,37 @@ pub(crate) fn decide_read( }, }) } -fn decide(operation: &KernelReadOperation, eval: &mut Eval<'_>) -> WorkspaceResult> { +/// One read's decision. `refused` receives the directory row a refusal read, +/// when the refused operation read one (LOOKUP of a name that binds nothing). +fn decide( + operation: &KernelReadOperation, + eval: &mut Eval<'_>, + refused: &mut Option, +) -> WorkspaceResult> { match operation { // The source's independent read reference permits removed metadata. KernelReadOperation::Getattr { serial } | KernelReadOperation::Data { serial } => { - eval.target(*serial) + Ok(eval.target(*serial)?.map(|inode| Decided { + inode, + parent: None, + })) + } + KernelReadOperation::Opendir { serial } => { + Ok(eval.directory(*serial)?.map(|inode| Decided { + inode, + parent: None, + })) } - KernelReadOperation::Opendir { serial } => eval.directory(*serial), // The kernel names an inode it still references. A file whose // last name is gone stays openable under that reference, so a // path open racing an unlink or replacement gets the old file. KernelReadOperation::Open { serial, .. } => { let inode = eval.target(*serial)?; match inode { - Some(inode) if inode.kind == InodeKind::File => Ok(Some(inode)), + Some(inode) if inode.kind == InodeKind::File => Ok(Some(Decided { + inode, + parent: None, + })), Some(inode) if inode.kind == InodeKind::Directory => { Err(WorkspaceError::Refused(Refusal::IsDirectory)) } @@ -137,11 +178,22 @@ fn decide(operation: &KernelReadOperation, eval: &mut Eval<'_>) -> WorkspaceResu let directory = eval.directory(*parent)?; let layers = eval.layers(*parent, name)?; let bound = eval.bound(*parent, directory.as_ref(), name, layers); - let (Some(_), Some(bound)) = (directory, bound) else { + let (Some(directory), Some(bound)) = (directory, bound) else { return Ok(None); }; - let serial = bound.ok_or(WorkspaceError::Refused(Refusal::Missing))?; - eval.target(serial) + let Some(serial) = bound else { + // The name binds nothing. This job still read the directory's + // own row, so its fresh facts answer the failure's directory + // attributes without a second visit. + *refused = Some(directory); + return Err(WorkspaceError::Refused(Refusal::Missing)); + }; + // The name resolved, so the reply also says what the directory + // is: the same job read its row, and no second visit is needed. + Ok(eval.target(serial)?.map(|inode| Decided { + inode, + parent: Some(directory), + })) } } } diff --git a/engine/crates/layerfs-workspace/tests/common/mod.rs b/engine/crates/layerfs-workspace/tests/common/mod.rs index 83fbcbb5d5..702bfffdb6 100644 --- a/engine/crates/layerfs-workspace/tests/common/mod.rs +++ b/engine/crates/layerfs-workspace/tests/common/mod.rs @@ -28,10 +28,18 @@ pub struct Store { pub lengths: Arc>>, /// Canonical bytes served upstream, to tell metadata demand from payload. pub served: Arc, + /// Object batch demands: one per `read_canonical_batch` call, whatever + /// the number of objects it names. A window's SQL batches, not its rows. + pub batches: Arc, + /// Length demands that asked for several ids in one call. + pub length_batches: Arc, + /// Length demands that asked for one id at a time. + pub length_demands: Arc, } impl AuthenticatedObjects for Store { fn read_canonical_batch(&self, ids: &[ObjectId]) -> ContentResult>> { self.demand.fetch_add(ids.len() as u64, Ordering::Relaxed); + self.batches.fetch_add(1, Ordering::Relaxed); let objects = self.objects.lock().unwrap(); ids.iter() .map(|id| { @@ -47,6 +55,7 @@ impl AuthenticatedObjects for Store { } impl layerfs_workspace::FileLengths for Store { fn file_length(&self, id: ObjectId) -> layerfs_workspace::WorkspaceResult { + self.length_demands.fetch_add(1, Ordering::Relaxed); self.lengths .lock() .unwrap() @@ -54,6 +63,18 @@ impl layerfs_workspace::FileLengths for Store { .copied() .ok_or(ContentError::MissingObject.into()) } + fn file_lengths(&self, ids: &[ObjectId]) -> layerfs_workspace::WorkspaceResult> { + self.length_batches.fetch_add(1, Ordering::Relaxed); + let lengths = self.lengths.lock().unwrap(); + ids.iter() + .map(|id| { + lengths + .get(id) + .copied() + .ok_or(ContentError::MissingObject.into()) + }) + .collect() + } } impl FinalizedConsumer for Store { fn accept(&mut self, object: FinalizedObject) -> ContentResult<()> { diff --git a/engine/crates/layerfs-workspace/tests/directory_links.rs b/engine/crates/layerfs-workspace/tests/directory_links.rs index 2940c31d30..59c278ad5d 100644 --- a/engine/crates/layerfs-workspace/tests/directory_links.rs +++ b/engine/crates/layerfs-workspace/tests/directory_links.rs @@ -575,7 +575,12 @@ fn a_visit_that_misses_the_count_is_undecided_writes_nothing_and_decides_once_su panic!("the supplied facts did not decide the lookup") }; assert_eq!( - (found.serial, found.entries, found.subdirs, found.nlink), + ( + found.inode.serial, + found.inode.entries, + found.inode.subdirs, + found.inode.nlink + ), (MIXED, 5, 2, 1) ); // Another request, carrying nothing, decides in its one visit: the @@ -688,7 +693,13 @@ fn remembered_counts_are_bounded_and_an_evicted_one_is_derived_again() { let Some(KernelReadDecision::Value(found)) = observe(&b, mount, "d00000", &facts) else { panic!("the carried facts did not decide the lookup") }; - assert_eq!((found.serial, found.entries, found.subdirs), (100, 3, 2)); + assert_eq!( + (found.inode.serial, found.inode.entries, found.inode.subdirs), + (100, 3, 2) + ); + // The same job answered the directory the name was looked up in, so a + // reply that shows the directory's attributes pays no second visit. + assert_eq!(found.parent.expect("the lookup's directory").serial, 1); assert_eq!(work().directory_count_scans, capacity + 3); // Both were evicted: each is derived again, correctly, and remembered diff --git a/engine/crates/layerfs-workspace/tests/kernel_directory.rs b/engine/crates/layerfs-workspace/tests/kernel_directory.rs index 2f1e4f4438..48c9b2153b 100644 --- a/engine/crates/layerfs-workspace/tests/kernel_directory.rs +++ b/engine/crates/layerfs-workspace/tests/kernel_directory.rs @@ -4,14 +4,16 @@ mod harness; use harness::{create, mkdir, path, rename, rmdir, unlink, Bench, T1, T2}; use layerfs_content::{ filesystem::{ - update_filesystem, DirectoryUpdate, FilesystemInput, FilesystemObjects, FilesystemResources, + attributes::PortableMetadata, build_filesystem, scope_for_seed, update_filesystem, + DirectoryUpdate, FilesystemInput, FilesystemObjects, FilesystemResources, InodeUpdate, }, - object::inode_leaf::InodeKind, + object::inode_leaf::{InodeKind, InodeMetadata, InodeValue}, + ObjectId, }; use layerfs_overlay::{KernelDirectory, KernelMount, OverlayError, PAGE_ROWS}; use layerfs_workspace::{ - CanonicalCache, CanonicalClient, KernelDirectoryBatch, KernelDirectoryWindow, - KernelReadDecision, KernelReadOperation, VisitFacts, + CanonicalCache, CanonicalClient, KernelDirectoryBatch, KernelDirectoryRead, + KernelDirectoryWindow, KernelReadDecision, KernelReadOperation, VisitFacts, }; use std::sync::Arc; /// A memory-only client over a cache that holds nothing. @@ -69,6 +71,16 @@ fn visit( directory: KernelDirectory, offset: u64, after: Option>, +) -> Result { + visit_with(b, directory, offset, after, false) +} +/// The same, for a reply that shows each entry's facts (READDIRPLUS). +fn visit_with( + b: &Bench, + directory: KernelDirectory, + offset: u64, + after: Option>, + attributes: bool, ) -> Result { let visit = b .workspace @@ -77,8 +89,11 @@ fn visit( directory.mount(), directory.serial(), directory.owner_id(), - offset, - after, + KernelDirectoryRead { + offset, + after, + attributes, + }, ) .unwrap(); let before = b.demand(); @@ -316,3 +331,253 @@ fn next_directory_read_observes_parent_moved_by_the_atomic_namespace_job() { .unwrap(); b.overlay.revoke_kernel_mount(mount).unwrap(); } +#[test] +fn a_re_sent_reply_reads_no_fact_of_a_name_it_does_not_send() { + let b = Bench::over("kernel-directory-resend", window_fixture(8), 4 << 20); + let mount = b.overlay.create_kernel_mount(1).unwrap(); + let directory = open(&b, mount, 1, 1); + let first = batch(&b, visit(&b, directory, 2, None).unwrap()); + assert_eq!(first.entries.len(), 9, "eight names and the subdirectory"); + // Publish only the first two names, as a reply with a smaller budget does: + // a later reply at the same offset reuses that range and sends no name + // beyond it. + let names: Vec<_> = first + .entries + .iter() + .map(|entry| entry.name.clone()) + .collect(); + let offer = first.publish.as_ref().expect("a fresh reply to publish"); + b.overlay + .publish_kernel_cookies(offer, &names[..2]) + .unwrap(); + assert_eq!( + b.cache.diagnostics().unwrap().file_lengths, + 0, + "a plain listing reads no length" + ); + // Only the two names this reply sends have their facts read: seven names + // of the same window are read nothing, so only two lengths are demanded + // and remembered. + let window = visit_with(&b, directory, 2, None, true).unwrap(); + let listed = window + .finish_with(Some(&b.workspace.base().unwrap()), true) + .unwrap(); + assert_eq!(listed.entries.len(), 2); + assert!(listed.entries.iter().all(|entry| entry.stat.is_some())); + assert_eq!( + b.cache.diagnostics().unwrap().file_lengths, + 2, + "two names sent, two lengths read" + ); + b.overlay + .close_kernel_directory(directory.mount(), directory.serial(), directory.owner_id()) + .unwrap(); + b.overlay.revoke_kernel_mount(mount).unwrap(); +} +/// A root holding `files` inherited regular files, each with its own content, +/// and one inherited subdirectory with a file of its own: a window whose +/// names, kinds, lengths and child-directory count all come from the base. +fn window_fixture(files: u64) -> common::Fixture { + let store = common::Store::default(); + let scope = scope_for_seed([21; 32]); + let directory = InodeMetadata::Inline(PortableMetadata { + mode: 0o755, + mtime_seconds: 7, + mtime_nanoseconds: 3, + }); + let regular = InodeMetadata::Inline(PortableMetadata { + mode: 0o644, + mtime_seconds: 7, + mtime_nanoseconds: 3, + }); + let mut inodes = Vec::new(); + for serial in 1..=2 { + inodes.push(InodeUpdate { + serial, + value: InodeValue { + kind: InodeKind::Directory, + content_root: ObjectId::for_bytes(&[serial as u8; 8]), + metadata_root: directory, + namespace_ref_count: 0, + }, + }); + } + let mut changes = Vec::new(); + for n in 0..files { + let root = common::file(&store, format!("file {n} of the window").as_bytes()); + inodes.push(InodeUpdate { + serial: 3 + n, + value: InodeValue { + kind: InodeKind::RegularFile, + content_root: root, + metadata_root: regular, + namespace_ref_count: 0, + }, + }); + changes.push((common::name(&format!("f{n:04}")), Some(3 + n))); + } + // Names bind in byte order: the subdirectory follows the files. + changes.push((common::name("sub"), Some(2))); + let inner = common::file(&store, b"the subdirectory's only name"); + inodes.push(InodeUpdate { + serial: 3 + files, + value: InodeValue { + kind: InodeKind::RegularFile, + content_root: inner, + metadata_root: regular, + namespace_ref_count: 0, + }, + }); + let directories = vec![ + DirectoryUpdate { parent: 1, changes }, + DirectoryUpdate { + parent: 2, + changes: vec![(common::name("only"), Some(3 + files))], + }, + ]; + let new: Vec = (1..=3 + files).collect(); + let mut sink = store.clone(); + let mut objects = FilesystemObjects::new(&store, &mut sink); + let input = FilesystemInput { + base: None, + scope, + root_serial: 1, + directories: &directories, + inodes: &inodes, + new_inodes: &new, + resources: FilesystemResources::default(), + }; + let root = build_filesystem(&mut objects, &input, None).unwrap().root; + common::Fixture { + store, + root, + scope, + bytes: Vec::new(), + } +} +/// One READDIRPLUS window over `files` inherited names, as a reply that shows each +/// entry's facts reads it: (object batches, length batches, single length +/// demands, entries listed). +fn window_reads(files: u64) -> (u64, u64, u64, usize) { + use std::sync::atomic::Ordering; + let b = Bench::over( + &format!("kernel-directory-facts-{files}"), + window_fixture(files), + 4 << 20, + ); + let mount = b.overlay.create_kernel_mount(1).unwrap(); + let directory = open(&b, mount, 1, 1); + // The cache starts empty, so the window's names and facts come from the + // Store; the OPENDIR and reading visit below fill the inode table's upper + // level into the same cache before the window's own reads are counted. + let window = visit_with(&b, directory, 2, None, true).unwrap(); + assert!(window.listing.is_none() && window.needs_base(true)); + let store = &b.fixture.store; + let before = ( + store.batches.load(Ordering::Relaxed), + store.length_batches.load(Ordering::Relaxed), + store.length_demands.load(Ordering::Relaxed), + ); + let base = b.workspace.base().unwrap(); + let listed = window.finish_with(Some(&base), true).unwrap(); + let after = ( + store.batches.load(Ordering::Relaxed), + store.length_batches.load(Ordering::Relaxed), + store.length_demands.load(Ordering::Relaxed), + ); + // Every inherited name, the subdirectory included, carries its facts. + for entry in &listed.entries { + assert!(entry.stat.is_some(), "facts of {:?}", entry.name); + } + let sub = listed + .entries + .iter() + .find(|entry| entry.name == b"sub") + .expect("the subdirectory"); + let sub = sub.stat.as_ref().unwrap(); + assert_eq!((sub.kind, sub.subdirs), (InodeKind::Directory, 0)); + assert_eq!(listed.entries.len() as u64, files + 1); + ( + after.0 - before.0, + after.1 - before.1, + after.2 - before.2, + listed.entries.len(), + ) +} +#[test] +fn a_window_that_shows_facts_reads_it_in_batches() { + let small = window_reads(8); + let large = window_reads(200); + assert_eq!((small.3, large.3), (9, 201), "every inherited name"); + // One length demand for the whole window, and it is a batch: no entry + // asks the provider for its own length. + assert_eq!((small.1, large.1), (1, 1), "one batch length demand"); + assert_eq!((small.2, large.2), (0, 0), "no single length demand"); + // The window's object batches do not grow with the names it lists: an + // inode tree level whose pages are resident costs no demand, and the rest + // come one batch per level, not one read per name. (What the count is + // here depends on the levels the visits before the window left unread: in + // this fixture they read the tree's upper level into the same cache, so + // this is not a fully cold window's count.) This fixture + // keeps every metadata root inline and builds its subdirectory's child + // count during construction, so neither costs a demand here; a tree whose + // metadata roots are objects, or a subdirectory counted by a scan, pays + // one read per entry that needs it, as before. + assert_eq!( + small.0, large.0, + "the same object batches for 8 and 200 names" + ); + assert!(large.0 < 3, "{} object batches", large.0); +} +#[test] +fn a_local_window_shows_its_rows_without_a_store_reader() { + use std::sync::atomic::Ordering; + let b = Bench::new("kernel-directory-local-facts"); + let made = b.applied(mkdir(1, "all-local"), T1).unwrap().serial; + let mount = b.overlay.create_kernel_mount(1).unwrap(); + // The kernel's own first LOOKUP of the name records the directory's + // parent, which its OPENDIR reads. + let lookup = b + .workspace + .kernel_read_visit( + empty(), + mount, + 1, + None, + KernelReadOperation::Lookup { + parent: 1, + name: common::name("all-local"), + }, + Arc::new(VisitFacts::default()), + ) + .unwrap(); + assert!(lookup.perform(&b.overlay).result.is_ok()); + let directory = open(&b, mount, 2, made); + b.applied(create(made, "one"), T1); + b.applied(create(made, "two"), T2); + let store = &b.fixture.store; + let before = ( + store.batches.load(Ordering::Relaxed), + store.length_demands.load(Ordering::Relaxed), + ); + // Every name of this window is a local row the visit already read: the + // reply shows all their facts, and the request takes no Store reader. + let window = visit_with(&b, directory, 2, None, true).unwrap(); + assert!(window.listing.is_some()); + assert!(!window.needs_base(true), "memory answered every fact"); + let listed = window.finish_with(None, true).unwrap(); + assert_eq!(listed.entries.len(), 2); + assert!(listed.entries.iter().all(|entry| entry.stat.is_some())); + assert_eq!( + ( + store.batches.load(Ordering::Relaxed), + store.length_demands.load(Ordering::Relaxed) + ), + before, + "a local window reads nothing" + ); + b.overlay + .close_kernel_directory(directory.mount(), directory.serial(), directory.owner_id()) + .unwrap(); + b.overlay.revoke_kernel_mount(mount).unwrap(); +} diff --git a/engine/crates/layerfs-workspace/tests/kernel_visit.rs b/engine/crates/layerfs-workspace/tests/kernel_visit.rs index 6aea36ab7e..42b80d892e 100644 --- a/engine/crates/layerfs-workspace/tests/kernel_visit.rs +++ b/engine/crates/layerfs-workspace/tests/kernel_visit.rs @@ -93,8 +93,8 @@ fn observe( assert!(outcome.directory_candidate.is_none()); match outcome.decision { Some(KernelReadDecision::Needs(needs)) => Seen::Needs(needs), - Some(KernelReadDecision::Value(inode)) => Seen::Value(inode), - Some(KernelReadDecision::Refused(refusal)) => Seen::Refused(refusal), + Some(KernelReadDecision::Value(decided)) => Seen::Value(decided.inode), + Some(KernelReadDecision::Refused { refusal, .. }) => Seen::Refused(refusal), other => panic!("undecided read visit: {other:?}"), } } @@ -127,8 +127,8 @@ fn open( assert!(outcome.directory_candidate.is_none()); let seen = match outcome.decision { Some(KernelReadDecision::Needs(needs)) => Seen::Needs(needs), - Some(KernelReadDecision::Value(inode)) => Seen::Value(inode), - Some(KernelReadDecision::Refused(refusal)) => Seen::Refused(refusal), + Some(KernelReadDecision::Value(decided)) => Seen::Value(decided.inode), + Some(KernelReadDecision::Refused { refusal, .. }) => Seen::Refused(refusal), other => panic!("undecided open visit: {other:?}"), }; assert_eq!( @@ -962,7 +962,7 @@ fn a_visit_through_a_read_only_descriptor_is_refused_and_changes_nothing() { .perform(&b.overlay); assert!(matches!( stat.decision, - Some(KernelReadDecision::Value(ref inode)) if inode.serial == serial + Some(KernelReadDecision::Value(ref decided)) if decided.inode.serial == serial )); b.overlay .kernel_close_file(mount, serial, file.owner_id()) @@ -1079,3 +1079,55 @@ fn a_resident_client_answers_only_from_memory_and_only_small_objects() { ); assert_eq!(memory.read_canonical_batch(&[absent]).unwrap().len(), 1); } + +#[test] +fn a_refused_lookup_answers_the_directory_its_job_read() { + let b = Bench::new("kernel-read-refused-directory"); + let mount = b.overlay.create_kernel_mount(1).unwrap(); + let mut facts = VisitFacts::default(); + // The name binds nothing: the job still read the root's own row, and its + // refusal carries that row, so a reply's directory attributes need no + // second visit and are never a stale observation. + for _ in 0..4 { + let visit = b + .workspace + .kernel_read_visit( + empty(), + mount, + 1, + None, + lookup(1, "absent"), + Arc::new(facts.clone()), + ) + .unwrap(); + let demand = b.demand(); + let outcome = visit.perform(&b.overlay); + assert_eq!(b.demand(), demand, "an owner visit asked the provider"); + match outcome.decision { + Some(KernelReadDecision::Needs(needs)) => facts + .supply(&b.workspace.base().unwrap(), &needs, None) + .unwrap(), + Some(KernelReadDecision::Refused { refusal, directory }) => { + assert_eq!(refusal, Refusal::Missing); + let directory = directory.expect("the directory the job read"); + let fresh = b.stat(1).unwrap(); + assert_eq!( + directory.serial, fresh.serial, + "the directory the name was in" + ); + assert_eq!( + (directory.mtime_seconds, directory.mtime_nanoseconds), + ( + fresh.metadata.mtime_seconds, + fresh.metadata.mtime_nanoseconds + ), + "the directory's own current row" + ); + assert_eq!(directory.nlink, fresh.namespace_refs, "its reference count"); + return; + } + other => panic!("undecided read visit: {other:?}"), + } + } + panic!("the missing lookup was not decided in four visits"); +}