update Redfs rhel10 to all features from ubuntu hwe - #204
Open
hbirth wants to merge 24 commits into
Open
Conversation
hbirth
requested review from
achhenderson,
bsbernd,
cding-ddn,
hazhou-ddn and
yongzech
August 27, 2026 08:23
A FUSE_DLM_WB_LOCK reply and a NOTIFY revoke run on different threads, so a revoke can arrive before the grant is recorded, find nothing, and leave a grant that is never taken back. revoke_gen caught this with a per-inode counter that cannot say which range was hit, so any revoke re-requested every grant in flight. Add enum fuse_dlm_range_state: REQUESTED or REVOKED on the new cache->pending list while in flight, GRANTED in cache->ranges. A revoke marks the pending requests it overlaps, fuse_dlm_request_commit() drops a marked grant, and unlinking the request and recording its grant is one step under the cache lock. Pending requests stay off the interval tree, so no tree walker needs a state filter. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
struct fuse_dlm_range carried a lifecycle state and a mode, the mode stored as FUSE_PCACHE_LK_READ/_WRITE (1 and 2) in a field typed enum fuse_page_lock_mode, whose enumerators are 0 and 1. The two are never independent, and fuse_dlm_range_is_locked() relied on the value ordering to let a write grant cover a read. Replace GRANTED with READ and WRITE and drop the mode field and the FUSE_PCACHE_LK_* values. fuse_dlm_state_satisfies() replaces the ordinal comparison. fuse_dlm_request_begin() no longer takes a mode: the one that reaches the tree is the one passed to the commit. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
fuse_dlm_unlock_range() removed the ranges it revoked, so a grant taken away looked exactly like a range that was never held. Nothing then stops writeback sending the page cache under it, and a writer that passed fuse_dlm_lock_is_held() before the revoke and dirtied the folio after it has its bytes sent for a range another node now holds. Mark such a range FUSE_DLM_RANGE_REVOKED instead, and keep it only for as long as there is page cache under it to describe; filemap_range_has_page() answers that, and one with nothing cached is still removed. A revoked range covers nothing, so the IO paths ask for the grant again, and writeback calls fuse_dlm_regrant_range() to take the range back before sending what it found revoked. Splitting at the revoke bounds replaces the trim and punch hole arithmetic, so fuse_dlm_punch_hole() goes and fuse_dlm_split_at() arrives in its place. fuse_dlm_ranges_dropped() frees the revoked ranges over page cache the caller has established is gone. The kernel-doc of the record said page offset throughout while every one of those arguments is a byte offset that happens to be page aligned, and the same file computes real page indices a few lines from some of them. Say byte offset. [hbi: adapted from ubuntu-hwe 1ef5278, with the wording fix 2480565 and the record side of d75a174 and 9bb6233 folded in. That branch grew a per range content bound and removed it again; this one never had it, so fuse_dlm_dirty_run() and the run classification it fed are not ported and writeback regrants whole runs.] Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
fuse_dlm_request_commit() reports -EAGAIN when a revoke overlapped a request while it was on the wire, and __fuse_get_dlm_lock() went round again with nothing stopping it. A remote node revoking as fast as the grants are handed out keeps that going for as long as it likes, and writeback asks for a grant with a folio locked, so the loop is not merely slow, it holds a folio hostage and the task is unkillable while it does. Give it a count and a signal check. Both are generous: every pass is a whole round trip, so reaching either means the range is genuinely being fought over and the caller is better told than left spinning. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
fuse_dlm_lock_range_locked() put every range overlapping the grant on the upgrade list whole, without splitting at the grant bounds the way fuse_dlm_unlock_range() does. A range extending past the grant then had its uncovered part upgraded with it: holding a read grant on [0, 8191] and being granted write on [4096, 8191] recorded the whole node as held for write, so fuse_dlm_lock_is_held() reported [0, 4095] covered, a cached write there never sent FUSE_DLM_WB_LOCK, and the server went on thinking this client held only a read there -- cluster exclusion broken without any error to see. The revoked case is worse. Writeback re-grants only the run it is about to send, but the commit flipped the whole revoked range back to held, so revoked-dirty bytes outside the re-granted run reclassified as FUSE_DLM_RUN_DIRTY and were sent without taking the range again -- exactly the lost update FUSE_DLM_RANGE_REVOKED exists to prevent. Split at both grant bounds before walking, so upgrades apply only inside the grant. Both halves of a split keep state and content, and fuse_dlm_try_merge() at the end recoalesces whatever stayed equal. Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
Readahead fills the page cache past the range fuse_cache_read_iter() locked, so those folios get no revoke when a remote node writes them. Request a read grant over the whole window in fuse_readahead() before any folio is consumed, and skip the window when the request fails. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
A data invalidation always dropped the range with invalidate_inode_pages2_range(), which launders dirty folios and waits for a FUSE_WRITE reply, even for a range holding nothing. Ask instead, under the gate so nothing populates or dirties in between. filemap_range_has_page() says whether anything is cached at all, and filemap_range_needs_writeback() whether laundering is needed; when it is not, invalidate_mapping_pages() drops the same folios without ever waiting for the server. [hbi: adapted from ubuntu-hwe ef3f058 as it ends up after 9bb6233. That branch asked a per range content bound (fuse_dlm_range_may_be_dirty()) and later replaced it with the page cache queries used here, so the bound is skipped and its fuse_dlm_range_touched() hunk in fuse_cache_write_iter() with it.] Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
A revoked range is kept now instead of being forgotten, but nothing acted on it: writeback queued whatever the page cache held, so bytes dirtied under a grant the server has since taken away still went out under no grant at all. Take the range back first. Every dirty folio here was written whole by this client, because the unaligned edges of a cached write go to the server directly and the interior covers whole pages, so there is nothing to classify; only the grant to make sure of. fuse_dlm_regrant_range() re-requests a range that has gone and walks the record once under the lock held for read when it has not. A server without DLM answers -ENOSYS, which is not a failure. On a real failure the folio goes back on the dirty list so the next writeback tries again. Neither caller does that for us: write_cache_pages() clears the folio before calling in, and fuse_launder_folio() clears it itself, so returning the error without redirtying would drop bytes the server has never seen. Both writeback entry points take it: fuse_writepages_fill() for ordinary writeback and fuse_writepage_locked(), which is what ->launder_folio sends through. [hbi: the record side of ubuntu-hwe 1ef5278 as d75a174 leaves it. That branch has a single iomap writeback range op where this one has two entry points, and it ported through a run classification this branch never grew.] Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
A revoked range is kept only to say that the page cache under it was dirtied under a grant that has gone. The invalidate that follows the revoke is what takes that cache away, so nothing was left to free the ranges and they accumulated for the life of the inode, each one costing writeback a grant request for a folio that is no longer there. Free them once the drop has happened, and only when it really happened: invalidate_mapping_pages() skips a busy folio and invalidate_inode_pages2_range() can fail on one, and a folio that survived still needs its record. Ask the page cache again rather than assume. The record is told about whole pages, since that is how a grant is recorded. [hbi: the inode.c side of ubuntu-hwe 1ef5278.] Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
wb_inval_rwsem fenced cached IO out for the whole of a NOTIFY invalidate, so a writer could not dirty the page cache under a grant being revoked. The record does that now: a revoke marks the range instead of forgetting it, and writeback takes the range again before sending anything marked that way. A read never needed the fence. Gone with it: the re-validation and retry loops that existed only because the gate had to be dropped around a FUSE_DLM_WB_LOCK round trip, fuse_cache_wr_dlm_lock()'s unrecorded flag, FUSE_DLM_READ_RETRIES, and the percpu_rw_semaphore itself with its per-inode allocation and the eviction-time free. setattr and the atomic O_TRUNC path lose it too, holding i_rwsem exclusive. Both arms of fuse_cache_write_iter() had entered the gate, the writeback one and the writethrough one the killpriv fallback reaches with the writeback cache still on; both keep the forced-direct-IO re-check that used to sit inside it. [hbi: port of ubuntu-hwe 02a9e2a. Its rationale also names the per range content bound, which that branch has since removed and this one never had; the revoked range and the writeback regrant are what carry the argument. Comments left pointing at the gate after the removal are corrected here rather than kept.] Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
The forced-direct-IO latch is checked once, so a buffered write that passed the check just before a notify set the latch dirties the page cache after the notify dropped the mapping. The coherency gate used to fence that. What is left behind is invisible to the direct path: a direct read misses the dirty folio, and a direct write lands underneath it, after which the invalidate launders rather than drops and puts the stale folio on the server on top. Write the range back before re-routing. [port of ubuntu-hwe 53140d0; this branch has two late re-route sites, the writeback arm and the writethrough one the killpriv fallback reaches, and both take the flush. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
fuse_do_setattr() revokes the grants past the new size and discards the page cache there, but never tells the record. A revoked range is kept because it describes page cache dirtied before the grant went, so every truncate leaves ranges describing folios that no longer exist. Nothing frees them before the inode, and until then the inode reports unwritten data and every later invalidate over that region launders. Call fuse_dlm_ranges_dropped() from the first whole page above the new size; the page holding the new end of the file survives, so its record has to. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
fuse_do_setattr() frees the record above the new size, but the invalidate_inode_pages2() it runs empties the mapping below it too, and the record there was left alone. A dirty folio is laundered by that invalidate, which puts its bytes on the server, but laundering lowers the record only for a folio the page cache does not consider valid (fuse_dlm_range_sent()), so an ordinary uptodate dirty folio leaves a DIRTY range describing page cache that no longer exists. The next unaligned write to such a page allocates a fresh folio, leaves it unfilled, and records only its own bytes. The stale DIRTY run over the rest of the page then makes fuse_read_folio_merge() keep folio bytes nobody wrote, and writeback classifies and sends them -- bad data both to a local reader and to the server. fsx reaches this with any truncate-then-partial-write sequence. Drop the record below the new size as well, and only when the drop really emptied it: a busy folio that survives the invalidate still needs its record, so ask filemap_range_has_page() first, the same rule the NOTIFY invalidate path applies before its drop. Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
The atomic O_TRUNC branch of fuse_do_setattr() releases every DLM range before dropping the cache, but the far more common path -- fuse_open() with fc->atomic_o_trunc -- called truncate_pagecache() and left the whole record standing. Ranges recorded DIRTY then described folios that no longer existed, and the next unaligned write to such a page left a fresh folio unfilled while the stale run made fuse_read_folio_merge() keep its unwritten bytes and writeback send them: bad data from something as plain as 'echo x > file' followed by a partial write. Release the record before the truncate, exactly as fuse_do_setattr() does; i_rwsem is held exclusive here (is_wb_truncate), so no cached write is between recording and dirtying. The FOPEN_KEEP_CACHE-less open one line below drops the mapping with invalidate_inode_pages2() and has the same problem. Drop the record there too, but only when the invalidate really emptied the mapping, under the same filemap_range_has_page() rule the NOTIFY path applies: a busy folio that survived keeps its record. Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
FALLOC_FL_PUNCH_HOLE and FALLOC_FL_ZERO_RANGE flush the span, punch it on the server, and drop it from the page cache with truncate_pagecache_range() -- and told the record nothing. The flush lowers the record only for folios the page cache does not consider valid, so an uptodate dirty folio inside the hole left a DIRTY range describing page cache that no longer exists. The next unaligned write into the hole allocates a fresh folio, leaves it unfilled, and records its own bytes; the stale DIRTY run over the rest of the page makes fuse_read_folio_merge() keep folio bytes nobody wrote and writeback send them. fsx exercises punch hole against partial writes constantly, which is where the bad-data failures on this branch come from. Free the record over the whole pages inside the hole, under the same filemap_range_has_page() rule the NOTIFY path applies before its drop. The partial pages at the edges survive the truncate with their punched part zeroed -- those zeroes are real bytes the folio really holds -- so their record has to survive with them. Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
__fuse_copy_file_range() flushes the destination span, lets the server copy, and then drops the copied pages with truncate_inode_pages_range() as stale -- without telling the record. As on the truncate and punch hole paths, an uptodate dirty folio flushed and then dropped leaves a DIRTY range describing page cache that no longer exists, and the next partial write there keeps and flushes folio bytes nobody wrote. Free the record over the dropped span, gated on filemap_range_has_page() like the other drops: the copy runs under i_rwsem but faults do not, and a folio a fault put back after the truncate keeps its record. Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
fuse_attr_cache_mask() trusted the local i_size over the server's only while a recorded write grant covered [attr->size, i_size). Two paths leave locally-extended data with no such record: a fault dirties pages under a page-mkwrite lock that is never recorded, and a local truncate revokes its own tail grants, after which cached writes extend the file again. On the next attribute refresh the check failed, the server's smaller size was applied, and truncate_pagecache() discarded dirty pages past it -- cached data destroyed by a GETATTR. The invalidate that follows then dropped the rest of the mapping with the record left standing, arming the stale-DIRTY bad-data path on every page of the file. generic/075 hits this within a few hundred fsx operations; the failure point moves with the attribute timeout, which is what made the runs look nondeterministic. Keep STATX_SIZE cached also while anything in [attr->size, i_size) is dirty or under writeback: those bytes exist only here, and the server cannot have a newer opinion about a size it has never seen. A remote truncate still lands, exactly as the design intends: its revoke launders and drops the tail first, so nothing is dirty there by the time the smaller size arrives. The no-grant early return learns the same rule, since a mapping dirtied only through page-mkwrite has no recorded grant at all. Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
When an attribute reply is applied with the size not served from the cache, fuse_change_attributes_i() truncates the page cache to the server's size and, when the data may be stale, drops the whole mapping with invalidate_inode_pages2() -- and told the record nothing. This is the same hole just closed on the truncate, O_TRUNC open, punch hole and copy_file_range paths, reached from every GETATTR, and it is the drop behind the generic/075 corruption: the server-side trace shows a folio whose bytes were flushed correctly once, dropped here with its record left DIRTY, repopulated by a later partial write that leaves the folio unfilled, and then written back with the stale record naming the unfilled head -- zeroes sent over data the server already had. Free the record over what really went: unconditionally above the new size, where truncate_pagecache() leaves nothing behind, and over the rest only when the invalidate emptied the mapping, under the same filemap_range_has_page() rule as the other drops, so a folio that survived or was faulted back keeps its record. Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
The handler revoked the range and left whatever was dirty under it to the drop that follows. Those bytes then have to go out under a grant that is already gone: writeback takes the range again first, a DLM round trip from inside the handler the server is waiting on. Send them before the revoke, while the grant still covers them, and wait for them there. do_writepages() runs in this context, so the grant is asked for while it is still held and the request never leaves the client; the drop that follows then finds nothing under writeback to block on, where laundering it would have waited for the same replies. A server that revokes from a thread it also needs to answer FUSE_WRITE on still deadlocks, the same contract fuse_notify_invalidate_range() states for a frozen inode. The error is left to the mapping, where fsync collects it. [hbi: ubuntu-hwe c9fab6c together with the writeback-before-revoke block it amends, which arrived there inside a259b45.] Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
With the writeback cache the kernel owns append positioning: a cached O_APPEND write is placed at the local i_size, and writeback later sends FUSE_WRITE requests with explicit offsets. Those offsets are only honoured if the server does not re-apply O_APPEND itself -- on Linux, pwrite(2) on a descriptor opened O_APPEND appends regardless of the offset argument. libfuse's passthrough examples know this and strip the flag, but only on FUSE_OPEN; a file created with FUSE_CREATE kept an O_APPEND backing descriptor, and every writeback run landed at the server's EOF instead of its offset. That is not just a theoretical hazard. Writeback legitimately covers the same bytes twice -- a folio whose record still names sent bytes is re-sent from its page start -- and with offsets honoured that re-send is idempotent. Appended instead, each flush's overlap with the previous one is duplicated at EOF and the file grows: generic/069 fails with exactly this shape (an appended file 230356 bytes too long, every byte of the excess a repeated flush overlap; server-side IO traces show each pwrite landing at EOF, sum of pwrites equal to the final file size). Strip O_APPEND from the flags sent in FUSE_OPEN, FUSE_CREATE and the compound open, the same way O_TRUNC is suppressed without atomic_o_trunc. The server cannot use the flag correctly under writeback caching anyway: appending server-side would order writeback runs by arrival, not by offset. Non-writeback mounts are unchanged, since there the server really does own append positioning. Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
An open without FOPEN_KEEP_CACHE drops the mapping with invalidate_inode_pages2(), unserialised, so concurrent openers of one file all walk it and take each folio's lock in turn. They convoy folio by folio and each still pays for the whole walk: seven tasks reopening a cached file took 4.96s apiece, in lockstep, with the server idle and 86k lock sleeps each. Hold the invalidate lock across the walk. The first opener empties the mapping; the rest return from the mapping_empty() test in invalidate_inode_pages2_range() without touching a folio. Exclusive also fences faults, as truncate does over the same walk, and keeps the emptiness test the DLM record drop depends on from reading a mapping another opener is halfway through. IO can repopulate the mapping after the unlock, but such a folio is added after the drop and keeps its page, so a later reader finds both the page and its record. fuse_create_open() drops the cache the same way, so share a helper. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
mm sizes the readahead window from per-fd offset arithmetic, so threads walking one file at a stride each see their own hits as isolated, never ramp, and settle just above a single read. Every read then waits on a request of its own, pipeline one deep: 73-88KB per request against a server whose threads sat 86% idle. Round the window up to whole requests before any folio is fetched. Filling out a partial request costs no extra request, and readahead_expand() folds the growth into ra->size, so the next window starts from the wider shape. Only the trailing edge moves and the expansion stops at the first cached folio, so nothing is refetched and a window bounded by neighbouring data stays bounded. ra->ra_pages caps the unit, keeping read_ahead_kb authoritative. The DLM read grant moves below the expansion, so the covered window is the populated one. That request size is also what the connection carries in one go, which is what s_bdi->io_pages describes: left at its 128K default it clamps a large explicit read or a FADV_WILLNEED to a window meant only to bound guessing. Set it to fc->max_pages. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
fuse_readahead() expanded the window to a whole request before sending anything. readahead_expand() locks one folio per page, so that walk ran with nothing in flight and other readers of the range blocked on its first locked folio until the request finally went out. Send what mm already built, then expand once those folios are in flight. Same window width, different order. readahead_expand() appends at _index + _nr_pages and __readahead_batch() retires a batch by moving both, so expanding with a batch outstanding lands where it should. The grant covers the intended window rather than the realised one, which is the safe direction. Wait on the page the batch handed over rather than looking its index back up in the mapping. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
mm keeps readahead state per open file. Several threads reading one shared file through their own descriptors each carry a separate idea of the stream, so only the first to reach a hole allocates anything and the rest find the folios present and issue nothing. The PG_readahead trigger is cleared by whichever thread reaches it first, and that thread's state usually describes a window another thread built, so page_cache_async_ra() takes its interleaved path, finds no hole within read_ahead_kb and issues nothing. Every window then costs a synchronous miss with all the readers waiting on it. The inode is what those readers share. Once the window is in flight, claim the range past it under fi->lock and send one request for it, so one reader primes it and the others skip it. Mark its first folio so a reader crossing into the range still gives mm a trigger, which now finds the hole this request stops at. Bounded by i_size, skipped when the connection is congested, and covered by the same DLM read grant the window takes. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.