Redfs ubuntu resolute 7.0.0 14.14 - #147
Merged
hbirth merged 77 commits intoAug 17, 2026
Merged
Conversation
Rename trace_fuse_request_send to trace_fuse_request_enqueue Add trace_fuse_request_send Add trace_fuse_request_bg_enqueue Add trace_fuse_request_enqueue This helps to track entire request time and time in different queues. Signed-off-by: Bernd Schubert <bschubert@ddn.com> (imported from commit 4a7f142)
If pinned pages are used the application can write into these pages and io_uring_cmd_complete_in_task() is not needed. Signed-off-by: Bernd Schubert <bschubert@ddn.com> (imported from commit 5f0264c)
readhead is currently limited to bdi->ra_pages. One can change that after the mount with something like minor=$(stat -c "%d" /path/to/fuse) echo 1024 > /sys/class/bdi/0:$(minor)/read_ahead_kb Issue is that fuse-server cannot do that from its ->init method, as it has to know about device minor, which blocks before init is complete. Fuse already sets the bdi value, but upper limit is the current bdi value. For CAP_SYS_ADMIN we can allow higher values. Signed-off-by: Bernd Schubert <bschubert@ddn.com> (imported from commit 763c96d)
Due to user buffer misalignent we actually need one page more, i.e. 1025 instead of 1024, will be handled differently. For now we just bump up the max. (imported from commit 3f71501)
When having writeback cache enabled it is beneficial for data consistency to communicate to the FUSE server when the kernel prepares a page for caching. This lets the FUSE server react and lock the page. Additionally the kernel lets the FUSE server decide how much data it locks by the same call and keeps the given information in the dlm lock management. If the feature is not supported it will be disabled after first unsuccessful use. - Add DLM_LOCK fuse opcode - Add cache page lock caching for writeback cache functionality. This means sending out a FUSE call whenever the kernel prepares a page for writeback cache. The kernel will manage the cache so that it will keep track of already acquired locks. (except for the case that is documented in the code) - Use rb-trees for the management of the already 'locked' page ranges - Use rw_semaphore for synchronization in fuse_dlm_cache (imported from commit 287c884)
Renumber the operation code to a high value to avoid conflicts with upstream. (imported from commit 27a0e9e)
Add support to invalidate inode aliases when doing inode invalidation. This is useful for distributed file systems, which use DLM for cache coherency. So, when a client losts its inode lock, it should invalidate its inode cache and dentry cache since the other client may delete this file after getting inode lock. Signed-off-by: Yong Ze Chen <yochen@ddn.com> (imported from commit 49720b5)
Send a DLM_WB_LOCK request in the page_mkwrite handler to enable FUSE filesystems to acquire a distributed lock manager (DLM) lock for protecting upcoming dirty pages when a previously read-only mapped page is about to be written. Signed-off-by: Cheng Ding <cding@ddn.com> (imported from commit ec36c45)
Allow read_folio to return EAGAIN error and translate it to AOP_TRUNCATE_PAGE to retry page fault and read operations. This is used to prevent deadlock of folio lock/DLM lock order reversal: - Fault or read operations acquire folio lock first, then DLM lock. - FUSE daemon blocks new DLM lock acquisition while it invalidating page cache. invalidate_inode_pages2_range() acquires folio lock To prevent deadlock, the FUSE daemon will fail its DLM lock acquisition with EAGAIN if it detects an in-flight page cache invalidating operation. Signed-off-by: Cheng Ding <cding@ddn.com> (imported from commit 8ecf118)
generic/488 fails with fuse2fs in the following fashion: generic/488 _check_generic_filesystem: filesystem on /dev/sdf is inconsistent (see /var/tmp/fstests/generic/488.full for details) This test opens a large number of files, unlinks them (which really just renames them to fuse hidden files), closes the program, unmounts the filesystem, and runs fsck to check that there aren't any inconsistencies in the filesystem. Unfortunately, the 488.full file shows that there are a lot of hidden files left over in the filesystem, with incorrect link counts. Tracing fuse_request_* shows that there are a large number of FUSE_RELEASE commands that are queued up on behalf of the unlinked files at the time that fuse_conn_destroy calls fuse_abort_conn. Had the connection not aborted, the fuse server would have responded to the RELEASE commands by removing the hidden files; instead they stick around. Create a function to push all the background requests to the queue and then wait for the number of pending events to hit zero, and call this before fuse_abort_conn. That way, all the pending events are processed by the fuse server and we don't end up with a corrupt filesystem. Signed-off-by: Darrick J. Wong <djwong@kernel.org> (imported from commit d4262f9)
This is a preparation to allow fuse-io-uring bg queue flush from flush_bg_queue() This does two function renames: fuse_uring_flush_bg -> fuse_uring_flush_queue_bg fuse_uring_abort_end_requests -> fuse_uring_flush_bg And fuse_uring_abort_end_queue_requests() is moved to fuse_uring_stop_queues(). Signed-off-by: Bernd Schubert <bschubert@ddn.com> (imported from commit e70ef24)
This is useful to have a unique API to flush background requests. For example when the bg queue gets flushed before the remaining of fuse_conn_destroy(). Signed-off-by: Bernd Schubert <bschubert@ddn.com> (imported from commit fc4120c)
When calling the fuse server with a dlm request and the fuse server responds with some other error than ENOSYS most likely the lock size will be set to zero. In that case the kernel will abort the fuse connection. This is completely unnecessary. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com> (imported from commit 0bc2f9c)
Check whether dlm is still enabled when interpreting the returned error from fuse server. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com> (imported from commit f6fbf7c)
- Increase the possible lock size to 64 bit. - change semantics of DLM locks to request start and end - change semantics of DLM request return to mark start and end of the locked area - better prepare dlm lock range cache rb-tree for unaligned byte range locks which could return any value as long as it is larger than the range requested - add the case where start and end are zero to destroy the cache Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com> (imported from commit 87968c7)
Fix reference count leak of payload pages during fuse argument copies. Signed-off-by: Cheng Ding <cding@ddn.com> (imported from commit 8b75cf0)
This is another preparation and will be used for decision which queue to add a request to. Signed-off-by: Bernd Schubert <bschubert@ddn.com> Reviewed-by: Joanne Koong <joannelkoong@gmail.com> (imported from commit e4698fa)
This is preparation for follow up commits that allow to run with a reduced number of queues. Signed-off-by: Bernd Schubert <bschubert@ddn.com> (imported from commit 2e27c33)
Add per-CPU and per-NUMA node bitmasks to track which io-uring queues are registered. Signed-off-by: Bernd Schubert <bschubert@ddn.com> (imported from commit be6edce)
Queues selection (fuse_uring_get_queue) can handle reduced number queues - using io-uring is possible now even with a single queue and entry. The FUSE_URING_REDUCED_Q flag is being introduce tell fuse server that reduced queues are possible, i.e. if the flag is set, fuse server is free to reduce number queues. Signed-off-by: Bernd Schubert <bschubert@ddn.com> (imported from commit f620f3d)
Running background IO on a different core makes quite a difference. fio --directory=/tmp/dest --name=iops.\$jobnum --rw=randread \ --bs=4k --size=1G --numjobs=1 --iodepth=4 --time_based\ --runtime=30s --group_reporting --ioengine=io_uring\ --direct=1 unpatched READ: bw=272MiB/s (285MB/s) ... patched READ: bw=650MiB/s (682MB/s) Reason is easily visible, the fio process is migrating between CPUs when requests are submitted on the queue for the same core. With --iodepth=8 unpatched READ: bw=466MiB/s (489MB/s) patched READ: bw=641MiB/s (672MB/s) Without io-uring (--iodepth=8) READ: bw=729MiB/s (764MB/s) Without fuse (--iodepth=8) READ: bw=2199MiB/s (2306MB/s) (Test were done with <libfuse>/example/passthrough_hp -o allow_other --nopassthrough \ [-o io_uring] /tmp/source /tmp/dest ) Additional notes: With FURING_NEXT_QUEUE_RETRIES=0 (--iodepth=8) READ: bw=903MiB/s (946MB/s) With just a random qid (--iodepth=8) READ: bw=429MiB/s (450MB/s) With --iodepth=1 unpatched READ: bw=195MiB/s (204MB/s) patched READ: bw=232MiB/s (243MB/s) With --iodepth=1 --numjobs=2 unpatched READ: bw=366MiB/s (384MB/s) patched READ: bw=472MiB/s (495MB/s) With --iodepth=1 --numjobs=8 unpatched READ: bw=1437MiB/s (1507MB/s) patched READ: bw=1529MiB/s (1603MB/s) fuse without io-uring READ: bw=1314MiB/s (1378MB/s), 1314MiB/s-1314MiB/s ... no-fuse READ: bw=2566MiB/s (2690MB/s), 2566MiB/s-2566MiB/s ... In summary, for async requests the core doing application IO is busy sending requests and processing IOs should be done on a different core. Spreading the load on random cores is also not desirable, as the core might be frequency scaled down and/or in C1 sleep states. Not shown here, but differnces are much smaller when the system uses performance govenor instead of schedutil (ubuntu default). Obviously at the cost of higher system power consumption for performance govenor - not desirable either. Results without io-uring (which uses fixed libfuse threads per queue) heavily depend on the current number of active threads. Libfuse uses default of max 10 threads, but actual nr max threads is a parameter. Also, no-fuse-io-uring results heavily depend on, if there was already running another workload before, as libfuse starts these threads dynamically - i.e. the more threads are active, the worse the performance. Signed-off-by: Bernd Schubert <bschubert@ddn.com> (imported from commit c6399ea)
This is to further improve performance. fio --directory=/tmp/dest --name=iops.\$jobnum --rw=randread \ --bs=4k --size=1G --numjobs=1 --iodepth=4 --time_based\ --runtime=30s --group_reporting --ioengine=io_uring\ --direct=1 unpatched READ: bw=650MiB/s (682MB/s) patched: READ: bw=995MiB/s (1043MB/s) with --iodepth=8 unpatched READ: bw=641MiB/s (672MB/s) patched READ: bw=966MiB/s (1012MB/s) Reason is that with --iodepth=x (x > 1) fio submits multiple async requests and a single queue might become CPU limited. I.e. spreading the load helps. (imported from commit 2e73b0b)
With the reduced queue feature io-uring is marked as ready after
receiving the 1st ring entry. At this time other queues just
might be in the process of registration and then a race happens
fuse_uring_queue_fuse_req -> no queue entry registered yet
list_add_tail -> fuse request gets queued
So far fetching requests from the list only happened from
FUSE_IO_URING_CMD_COMMIT_AND_FETCH, but without new requests
on the same queue, it would actually never send requests
from that queue - the request was stuck.
(imported from commit 3bfb6cd)
fuse.h: add new opcode FUSE_COMPOUND fuse_compound.c: add new functionality to pack multiple fuse operations into one compound command file.c: add an implementation of open+getattr Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com> (imported from commit d9e7351) (imported from commit 1607a03) (imported from commit 9df5e4c) (imported from commit 9921bcd) (imported from commit 09d6f59) (imported from commit 41b40bd)
There was a race between fuse_uring_cancel() and
fuse_uring_register()/fuse_uring_next_fuse_req(),
which comes from the queue reduction feature.
Race was
core-A core-B
fuse_uring_register
spin_lock(&queue->lock);
fuse_uring_ent_avail()
spin_unlock(&queue->lock);
fuse_uring_cancel()
spin_lock(&queue->lock);
ent->state = FRRS_USERSPACE;
list_move()
fuse_uring_next_fuse_req()
spin_lock(&queue->lock);
fuse_uring_ent_avail(ent, queue);
fuse_uring_send_next_to_ring()
spin_unlock(&queue->lock);
fuse_uring_send_next_to_ring
I.e. fuse_uring_ent_avail() was called two times and the 2nd time
when the entry was actually already handled by fuse_uring_cancel().
Solution is to not call fuse_uring_ent_avail() from
fuse_uring_register. With that the entry is not in state
FRRS_AVAILABLE and fuse_uring_cancel() will not touch it.
fuse_uring_send_next_to_ring() will mark it as FRRS_AVAILABLE,
and then either assign a request to it and change state again
or will not touch it at all anymore - race fixed.
This will be folded into the upstream queue reduction patches
and therefore has the RED-34640 commit message.
Also entirely removed is fuse_uring_do_register() as remaining
work can be done by the caller.
Signed-off-by: Bernd Schubert <bschubert@ddn.com>
(imported from commit 932feba)
This is just to avoid code dup with an upcoming commit. Signed-off-by: Bernd Schubert <bschubert@ddn.com> (imported from commit ec3217f)
This issue could be observed sometimes during libfuse xfstests, from dmseg prints some like "kernel: WARNING: CPU: 4 PID: 0 at fs/fuse/dev_uring.c:204 fuse_uring_destruct+0x1f5/0x200 [fuse]". The cause is, if when fuse daemon just submitted FUSE_IO_URING_CMD_REGISTER SQEs, then umount or fuse daemon quits at this very early stage. After all uring queues stopped, might have one or more unprocessed FUSE_IO_URING_CMD_REGISTER SQEs get processed then some new ring entities are created and added to ent_avail_queue, and immediately fuse_uring_cancel moved them to ent_in_userspace after SQEs get canceled. These ring entities were not moved to ent_released, and stayed in ent_in_userspace when fuse_uring_destruct was called. One way to solve it would be to also free 'ent_in_userspace' in fuse_uring_destruct(), but from code point of view it is hard to see why it is needed. As suggested by Joanne, another solution is to avoid moving entries in fuse_uring_cancel() to the 'ent_in_userspace' list and just releasing them directly. Fixes: b6236c8 ("fuse: {io-uring} Prevent mount point hang on fuse-server termination") Cc: Joanne Koong <joannelkoong@gmail.com> Cc: <stable@vger.kernel.org> # v6.14 Signed-off-by: Jian Huang Li <ali@ddn.com> Signed-off-by: Bernd Schubert <bschubert@ddn.com> (imported from commit 30d0473)
This fixes a memory leak. (imported from commit f75b62f)
Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com> (cherry picked from commit 0ea57a5) Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
A FUSE_NOTIFY_INVAL_INODE data invalidation means another (remote) entity
is modifying the file.
Rather than react to a single notify, keep a per-inode moving average of
how fast data invalidations arrive for the whole file: an EWMA of the
inter-arrival interval, updated under fi->lock on every notify
(fuse_notify_inval_hot()). The inode is latched only once the average
spacing drops below an internal threshold (FUSE_NOTIFY_DIO_INTERVAL) while
a local writer is open; a lone or occasional notify keeps the average high
and does not trip the switch. The heuristic has no external knob -- its
parameters (EWMA weight, threshold, seed) are source-level constants.
Introduce the forced-direct-IO latch (FUSE_I_FORCE_DIO):
- fuse_reverse_inval_inode() folds each data invalidation into the moving
average and sets the latch when it trips with a local writer present;
- fuse_file_{read,write}_iter() and fuse_cache_write_iter() route to the
direct path while latched; fuse_dio_{wr_exclusive_lock,lock,unlock}()
use the shared parallel-dio path and bypass the cached/uncached
accounting;
- fuse_file_io_open() opens new files uncached so they do not re-enter
caching mode;
- fuse_prepare_release() clears the latch once the last writer is gone
and fuse_file_release() drops any clean folios a racing read
repopulated; fuse_file_mmap() reverts to caching mode (a mapping needs
the page cache).
Latching to direct IO is only coherent if no buffered write can deposit
dirty folios into the page cache after it has been dropped. Add a
per-inode rw_semaphore, wb_inval_rwsem, to serialise the buffered-write
page-cache dirtying against the latch transition. The writeback path
holds it for read around the dirtying and re-checks the latch under it;
fuse_reverse_inval_inode() holds it for write around its invalidate +
latch set. The notification may be delivered by the same server thread
that still owes a reply to an in-flight write holding the inode lock, so
it takes the rwsem with a trylock and never blocks: if the writer has gone
it skips the latch and only invalidates the notified range. The writer's
read-side section stays free of server round-trips because under fc->dlm
the partial-write RMW read is skipped.
Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
(cherry picked from commit eeec69b)
Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
fuse_readahead() batches whole folios into a single request, capped at min(fc->max_pages, fc->max_read/PAGE_SIZE) pages, but fuse_init_file_inode() let the page cache build folios up to MAX_PAGECACHE_ORDER. A large sequential read could thus produce a folio bigger than one request can carry: the first loop iteration took the folio_pages > cur_pages path, fired WARN_ON(!pages), and broke with ap->num_folios == 0. fuse_send_readpages() was still called and dereferenced a NULL ap->folios[0] via folio_pos(), oopsing at CR2=0x20 (folio->index). Cap the folio order to the per-request page limit so the page cache can never build an unserviceable folio. There is a patch for upstream that does exactly the same. In later versions this is not needed but for 6.17 we need this. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com> (cherry picked from commit 85f949c) Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
Extending FOPEN_PARALLEL_DIRECT_WRITES writes were forced onto the
exclusive inode lock, re-serializing the parallel phase. The exclusive
lock only bundled "write + advance i_size + undo-on-failure" into one
unit. But i_size is committed by fuse_write_update_attr() under
fi->lock, only on a successful growing write and independent of the
inode rwsem -- so shared-lock writers commit size correctly and have
nothing to undo. Drop the past-EOF exclusive triggers and gate the
whole-file fuse_do_truncate() rollback on holding the exclusive lock.
Lock mode is passed to __fuse_direct_IO(); i_size is committed at the
same point in every path, only the failure rollback differs:
non-exclusive (relaxed, parallel):
fuse_direct_write_iter
fuse_dio_lock -> inode_lock_shared (exclusive=false)
__fuse_direct_IO(.., false)
fuse_direct_io() write to server
fuse_write_update_attr() commit i_size (on success)
no rollback
exclusive (append / caching / !parallel):
fuse_direct_write_iter
fuse_dio_lock -> inode_lock (exclusive=true)
__fuse_direct_IO(.., true)
fuse_direct_io() write to server
fuse_write_update_attr() commit i_size (on success)
ret<0 & extend -> fuse_do_truncate() rollback
exclusive (caching-mode O_DIRECT):
fuse_cache_write_iter -> inode_lock (exclusive)
generic_file_direct_write -> fuse_direct_IO
__fuse_direct_IO(.., true)
fuse_direct_io() write to server
fuse_write_update_attr() commit i_size (on success)
ret<0 & extend -> fuse_do_truncate() rollback
Signed-off-by: Bernd Schubert <bschubert@ddn.com>
(cherry picked from commit c99a893)
Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
fuse_dio_lock() takes an uncached_io reference (via fuse_inode_uncached_io_start()) only when FUSE_I_FORCE_DIO is clear, while fuse_dio_unlock() decided whether to drop it by re-reading FUSE_I_FORCE_DIO. On this tree the latch is toggled asynchronously by the inode-invalidation notify-storm path, so the bit can differ between the lock and the unlock of a single direct write: - clear at lock (reference taken), set before unlock: the reference is never dropped, leaving fi->iocachectr permanently negative and hanging the next caching-mode open; - set at lock (no reference), cleared before unlock: fuse_inode_uncached_io_end() is called without a matching start, tripping WARN_ON(fi->iocachectr >= 0) and corrupting the counter. Record in fuse_dio_lock() whether a reference was actually taken and have fuse_dio_unlock() drop it based on that captured decision instead of re-testing the racy bit, so the accounting stays balanced regardless of any FORCE_DIO transition mid-write. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com> (cherry picked from commit 6bf7ec0) Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com> (cherry picked from commit 9d6feaa) Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
Acquire the dlm lock from fuse server for the normal buffer read path to ensure the distributed page cache across different nodes can be co-existing and consistency. More importantly, this change will correct the DLM lock and folio locks ordering for the buffer read path, thus can avoid the potential deadlock between the buffer read and page cache invalidation processes. Signed-off-by Hai Zhong Zhou <hazhou@ddn.com> (cherry picked from commit cf349bc) Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
A FUSE_NOTIFY_INVAL_INODE is a coherency event: once the server signals a remote modify, no local read may return a page it has superseded. The invalidate ran unserialized against cache-serving reads, so a buffered read could hand back a stale folio it still held a reference to. Convert the per-inode wb_inval_rwsem to a percpu_rw_semaphore and take its read side around the cache-serving read as well as the existing buffered write. The read side is per-CPU, so it scales on a shared file; the NOTIFY takes the write side blocking, giving the invalidate priority -- it parks new readers, drains in-flight ones, then drops the cache. Every gated invalidate now runs under the write side, not just the storm-latching one. The gate is allocated only for writeback+dlm regular files and is NULL elsewhere (best-effort invalidate, as before). The blocking write side may run on the notify-delivering server thread, so it is safe only under a server that services request replies on other threads; redfs' dlm server provides that contract. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com> (cherry picked from commit d6180a2) Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
The buffered read path now acquires a DLM read lock via fuse_get_dlm_lock(..., FUSE_PAGE_LOCK_READ). Before sending the request to the server, fuse_get_dlm_lock() calls fuse_dlm_range_is_locked() to skip regions we already hold. That coverage check compared the held lock mode for exact equality (range->mode != lock_mode), so a range we already hold with an exclusive WRITE lock was reported as not-locked for a READ request. Because fuse_dlm_lock_range() intentionally does not downgrade a WRITE lock on a read, the region stays WRITE-locked and every subsequent read re-requests a DLM read lock from the server. This made read-after-write and re-read workloads flood the server with redundant FUSE_DLM_WB_LOCK requests, never converging. A held WRITE lock (exclusive) subsumes a READ lock. Treat a range as uncovered only when the held mode is strictly weaker than the requested mode (range->mode < lock_mode). READ requests are now satisfied by either a READ or a WRITE lock, while WRITE requests still require an existing WRITE lock (upgrade otherwise), matching the compatibility rules already documented in fuse_dlm_lock_range(). Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com> (cherry picked from commit 4e3bef0) Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
This fixes the problem where iomap was sending READ requests for file sections after EOF. For an expanding file we can assume zeros for the passage after EOF. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com> (cherry picked from commit ca1664e) Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
In fuse_cache_write_iter, writeback_cache mode was always refreshing STATX_SIZE along with STATX_MODE before a buffered write. The size refresh is only needed for the O_APPEND path, where the kernel must know the current EOF before extending the file. For ordinary writes, fetching size is unnecessary work and can race with concurrent writes and then impact writeback performance. Keep refreshing STATX_MODE in all cases so SUID clearing still sees an up-to-date mode. Request STATX_SIZE only when the file is opened with O_APPEND. Signed-off-by Hai Zhong Zhou <hazhou@ddn.com> (cherry picked from commit a41215d) Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
fuse_get_cache_mask() returned STATX_MTIME|CTIME|SIZE whenever writeback_cache was enabled, causing the kernel to trust its locally cached mtime/ctime/size over whatever the server returned in a GETATTR reply. This is unsafe under DLM: another node can hold a PW lock on the inode and modify its size/mtime independently, and the local writeback_cache values have no way of reflecting that. Under the DLM protocol, however, this override is unnecessary in the first place: a GETATTR always acquires a PR sattr lock, which forces every node holding a conflicting PW lock -- including the local node, for its own buffered writes -- to flush dirty pages before the server renders the reply. So under DLM the server's answer is always at least as fresh as anything cached locally, for all three attributes, not just size. Make fuse_get_cache_mask() return 0 whenever fc->dlm is set, when writeback_cache is enabled, so the kernel always trusts the server's attr/size reply in that case. The STATX_MTIME|CTIME|SIZE local-cache override remains only as a fallback for servers without DLM support, where no such flush-before-grant guarantee exists. Signed-off-by Hai Zhong Zhou <hazhou@ddn.com> (cherry picked from commit dc73360) Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
fuse_dlm_try_merge() locates the first merge candidate by walking from rb_first_cached() until it reaches the region just granted. The walk runs under the write-held cache rwsem on every fuse_dlm_lock_range() call, and the tree it walks holds every cached grant of the inode. Strided writers (IOR hard-write) accumulate grants that cannot merge with each other, so the tree keeps growing and every new grant pays a scan of all grants below it -- quadratic over the run, with fuse_dlm_range_is_locked() readers blocked behind each scan. Seed the merge with fuse_page_it_iter_first() on the region widened by one unit to each side instead; finding the lowest overlapping range is what the interval tree is there for. This also repairs two edge cases of the linear scan: a region starting at offset 0 made 'start - 1' wrap so the scan degenerated and merging was silently skipped, and a region ending at U64_MAX overflowed 'end + 1' in the loop bound, ending the merge after the first range. Both bounds now saturate. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com> (cherry picked from commit b2daa2e)
Both cached IO paths request their DLM lock first and then go to sleep on things a NOTIFY invalidate can be holding: the read path blocks on the coherency gate (writer priority), the write path additionally sleeps on a contended i_rwsem. A NOTIFY invalidate running in that window revokes exactly the lock just granted (fuse_dlm_unlock_range()), so the task wakes up and populates or dirties the page cache with no DLM coverage. Close the window without ever sending a FUSE_DLM_WB_LOCK request while holding the gate (a grant that had to wait on an invalidate delivered to this same client would deadlock against our own gate hold): - Drop the lock record under the gate write side in fuse_reverse_inval_inode(), so revocation and page drop are one atomic step with respect to the gate. - After entering the gate read side, re-check the grant against the live lock tree; if it was revoked while we waited, drop the gate, re-request, re-enter and check again. With the revoke now gated, passing the check means the lock cannot go away for the whole gate hold: a revoke arriving mid-operation parks until the IO is done. - Keep the write path's lock request ahead of the inode lock: the round trip must not capture the writer-priority i_rwsem for unbounded cluster-grant latency, and the in-gate re-validation already closes the grant-to-use window. Only O_APPEND moves below the lock, because its range is the current EOF -- stable only under the exclusive inode lock. This also fixes the append range itself: generic_write_checks() rewrites ki_pos to i_size for IOCB_APPEND, so the old 'i_size + ki_pos' double-counted (ki_pos is absolute, not relative) and locked a range disjoint from where the data lands. fuse_get_dlm_lock() now reports whether the grant is recorded, and the re-validation never re-requests a grant that failed, so it cannot spin (the read path seeds this from its pre-gate request instead of discarding that result). A grant the server issued but that could not be recorded (small-allocation -ENOMEM) reports FUSE_DLM_GRANT_UNRECORDED: coverage exists cluster-wide, so failing the IO would be wrong -- it proceeds, it just cannot re-validate. Empty ranges are trivially held, so a zero-length IO neither sends a doomed request nor spins in the retry loops. The write path returns a real failure to the caller instead of dirtying the cache without DLM coverage; only -ENOSYS still degrades to a plain cached write, since it means the server has no DLM at all and clears fc->dlm. The read path keeps falling through unlocked and additionally bounds its retry: a reader-only inode has no force-DIO latch to end a revoke storm, so after a few re-requests the read is served unlocked rather than looping in the kernel for the duration of the storm. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com> (cherry picked from commit 98ccde5)
The NOTIFY_INVAL_INODE revoke computed
fuse_dlm_unlock_range(fi, offset, pg_end == -1 ? 0 : offset + len - 1)
which is wrong at both degenerate ends: a to-EOF invalidate (len <= 0,
e.g. a remote truncate) with offset > 0 becomes the inverted range
[offset, 0] and removes nothing, so the revoked grant stays visible to
the re-validating IO paths forever -- cached writes with no
server-side lock, zero-filled RMW reads; and an invalidate of byte 0
(offset 0, len 1) becomes [0, 0], the "destroy everything" sentinel,
wiping every grant of the inode.
Map the range in one helper shared by the gated and the ungated
branch: to-EOF revokes through U64_MAX, and the bounds widen to page
boundaries to match how grants are recorded -- revoking too much only
costs a re-request, too little leaves a stale grant.
Drop the in-band (0, 0) sentinel: whole-file invalidates walk the
normal removal path, release-all is fuse_dlm_cache_release_locks(),
and an inverted range is rejected with -EINVAL instead of silently
ignored.
Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
(cherry picked from commit eb3464f)
…y gate Two revocation paths bypassed the revoke-under-gate invariant the IO paths re-validate against: - fuse_reverse_inval_inode() skipped the gate once mapping_mapped() turned true, revoking concurrently with gate holders -- but fuse_cache_read_iter()/fuse_cache_write_iter() enter the gate unconditionally, so a single mmap() reopened the race. Keep the gate for mmapped inodes; only the force-DIO latch stays disabled for them (a mapping needs the page cache, and fuse_file_mmap() reverts any latch it races with). - The local truncates in fuse_do_setattr() -- the atomic-O_TRUNC open shortcut and the after-setattr trim -- revoked and dropped the cache with no gate at all, so an already re-validated reader could repopulate the truncated range. Take the gate write side around revoke + drop. This cannot deadlock: both run under exclusive i_rwsem, which no gate holder waits on (the write path takes i_rwsem before the gate, the read path never takes it). Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com> (cherry picked from commit 3523bf6) Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
A FUSE_DLM_WB_LOCK reply and a NOTIFY invalidate are serviced on different threads, so a revoke aimed at the grant a reply carries can be processed before fuse_get_dlm_lock() records it: the revoke finds nothing to remove, and the requester then records an already-dead grant that no later NOTIFY will target -- a permanent false positive for the re-validating IO paths. Add a revocation generation to the lock cache, bumped under the cache lock by every revoke path -- unconditionally, because the racing revoke sees an empty overlap precisely when the grant is in flight. fuse_get_dlm_lock() samples it before sending and records through fuse_dlm_lock_range_gen(), which refuses with -EAGAIN once the generation has moved; the grant is then re-requested instead of recorded, bounded so a revoke storm cannot pin the IO here (past the bound the failure reports like any request failure). Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com> (cherry picked from commit 5c94fbb) Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
fuse_get_dlm_lock() re-requests the DLM lock when fuse_dlm_lock_range_gen() returns -EAGAIN, i.e. a revoke was processed while the grant request was in flight and the grant it returned may already be dead. That restart was bounded by FUSE_DLM_RECORD_TRIES, and past the bound the function returned -EAGAIN. Reporting that as a request failure is wrong: no one else holds the range at that point, the caller simply lost a race with a revoke, and the write path turns the error into a failed write. Retry unconditionally instead. Every pass issues a fresh FUSE_DLM_WB_LOCK round trip to the server, so a revoke storm throttles the loop rather than spinning it, and the loop ends as soon as one grant survives long enough to be recorded. Drop the now-unused bound and its counter. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com> (cherry picked from commit 2b6ed88) Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
fuse_reverse_inval_inode() latches an inode into direct IO when a remote writer keeps invalidating a file that is also open for writing here. That trades the writeback cache away for as long as the latch holds, which only pays off on workloads that actually see such invalidation storms. Make it opt-in through a new 'enable_notify_dio' module parameter, default off. FUSE_I_FORCE_DIO is set in exactly one place, so gating that single site is enough: every other reference only tests or clears the bit, and with the bit never set those paths behave as they did before the latch existed. The moving average is still folded on every invalidation while the parameter is off, so enabling it at runtime takes effect on the next storm instead of after a warm-up. Clearing it stops new latches but leaves already-latched inodes to run out on the usual exits (last writer closes, or mmap). Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com> (cherry picked from commit ac617b0) Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
fuse_cache_write_iter() picks between the exclusive and the relaxed shared inode lock with fuse_cache_wr_exclusive_lock(), which returns "shared" only when fc->dlm is set. Since "fuse: re-validate the DLM grant after waiting on the coherency gate" that decision is made before the DLM write lock is requested, and the request itself can clear fc->dlm: a server that does not implement FUSE_DLM_WB_LOCK answers -ENOSYS, which fuse_get_dlm_lock() handles by clearing fc->dlm and reporting success. The write then proceeds in a state that was unreachable before: the shared lock was chosen believing DLM was active, but DLM is now known to be absent. That combination is not benign. The shared path claims the i_size extension up front so iomap never updates i_size itself, which also stops iomap_block_needs_zeroing() from ever firing for the write's own range; the zero-fill that compensates for it in fuse_iomap_read_folio_range() is gated on fc->dlm and so no longer runs. An expanding write therefore falls through to fuse_do_readfolio() and sends a READ for a range past EOF that cannot hold data. Against a file the client opened write-only the server fails that read -- passthrough_hp returns EBADF -- and the write fails with it. Re-evaluate the lock mode after the request, while no lock is held yet, so a server without DLM support gets the exclusive path and iomap's own beyond-EOF zeroing back. This showed up as generic/105, 123, 215, 246, 378, 423, 519 and 597 all failing with EBADF on the first write to a newly created file, and bisected to the commit named above. Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
achhenderson
force-pushed
the
redfs-ubuntu-resolute-7.0.0-14.14
branch
from
August 12, 2026 23:40
437b1bd to
4a9ddcf
Compare
fuse_dlm_punch_hole() splits a grant by shortening the original range and adding a second one for the tail. It assigns range->end before calling fuse_page_it_remove(). The ranges live in an rbtree of intervals. Each node caches a maximum range->end which describes the max of its whole subtree. This enables lookups to skip a subtree whose intervals all end before the query starts. Removing a node also causes the tree's intervals to be re-balanced, and the cached maxima of the subtree nodes must be updated accordingly. But editing range->end before removing a node means that rebalancing is not computed correctly. Since the sub-nodes were not indexed by this value, the cached maxima no longer describe the remaining nodes. So a later fuse_page_it_iter_first() then either prunes a subtree that does hold an overlap, or descends into one that does not. Similar bugs were fixed in five other sites in this file in commit "fuse: seed DLM grant merging with an interval-tree lookup". Unfixed, the incorrect maxima can cause an oops under fsstress. This path is harder to expose since it needs a hole that is strictly interior to the grant, and was found by review, but it can also cause the same oops if left uncorrected. Apply the same ordering here: unlink first, then edit, then relink. The new_range is copied before the edit, so it still carries the original end and only its start needs adjusting. Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
fuse_write_inode() -> fuse_flush_times() pushes out the mtime/ctime that the kernel owns locally while the writeback cache is on. On the wire that request is indistinguishable from a userspace "touch -m": both arrive as SETATTR with FATTR_MTIME | FATTR_CTIME | FATTR_FH, because trust_local_cmtime makes iattr_to_fattr() send CTIME whenever the writeback cache is enabled. A server that wants to handle a cache flush differently from an explicit attribute change - skipping a cluster-wide lock, merging rather than overwriting - has no way to tell them apart. Add FATTR_WRITEBACK, a control bit in fuse_setattr_in.valid alongside the existing non-attribute bits FATTR_FH, FATTR_LOCKOWNER and FATTR_KILL_SUIDGID. It selects no attribute, it only states that the request originates from writeback. Bit 30 is used rather than the next free one. libfuse mirrors the wire bits into its own FUSE_SET_ATTR_* namespace, where bits 12 to 17 are already taken by library-internal flags, and it masks incoming requests against that namespace; a bit picked from the low end would collide there and need translating on the way in. Bit 30 is free on both sides and clear of the sign bit of the int that the libfuse setattr operation takes, so one value works end to end. The bit is negotiated at INIT time with FUSE_SETATTR_WRITEBACK and is only set on a connection whose server asked for it, so servers that do not know the bit never receive it. Only ->write_inode() is marked. The other kernel-initiated SETATTR, fuse_do_truncate() rolling back a failed extending direct-IO write, is deliberately left unmarked: it is a size correction rather than an attribute writeback, and conflating the two would make the flag ambiguous. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com> (cherry picked from commit c5196db) Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
fuse_reverse_inval_inode() invalidates with invalidate_inode_pages2_range(),
which waits out folios under writeback and launders dirty ones. Both need a
FUSE_WRITE reply, and while fi->writectr < 0 none can arrive:
fuse_flush_writepages() parks the request on fi->queued_writes until
fuse_release_nowrite().
A truncate holds that freeze across its whole SETATTR, and the server revokes
the truncated range from inside the SETATTR handler, so the notify blocks the
very thread that owes the reply lifting the freeze. generic/014 deadlocks
within seconds, in folio_wait_writeback() under fuse_launder_folio() under
fuse_reverse_inval_inode().
fuse_do_setattr() already states the rule ("Only call
invalidate_inode_pages2() after removing FUSE_NOWRITE, otherwise
fuse_launder_folio() would deadlock"). Give the notify path the same: while
frozen, use invalidate_mapping_pages(), which skips dirty and under-writeback
folios instead of waiting on them. The stale clean folios still go, the DLM
grant is revoked either way, and the freezes that span a request drop the
cache themselves when they finish.
Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
(cherry picked from commit b77b85c)
Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
fuse_cache_write_iter() holds wb_inval_rwsem for read across the write, and the kiocb_modified() -> file_remove_privs() that precedes it runs under that same gate. Without handle_killpriv[_v2] the privilege kill asks the server (GETATTR, then SETATTR), and a server that invalidates the inode from inside such a handler blocks in percpu_down_write() draining the gate reader that is waiting for its reply. generic/193 hangs there. The gate only has to fence the page-cache dirtying, so run the write checks, the privilege kill and the timestamp update before entering it. task_io_account_write() stays behind the gate, so a write that the forced-DIO re-check reroutes is not counted twice. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com> (cherry picked from commit 736375f) Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
fuse_get_cache_mask() returns 0 once the connection has DLM, so every GETATTR reply overwrites i_size, mtime and ctime, and truncate_pagecache() then drops the tail the client still holds dirty. The only way a server can make that answer true is to revoke the client from inside the handler, which deadlocks against the coherency gate. A write grant already guarantees that no other node can touch the range, so keep the cached values while one is held: the size when the server reports less than i_size and [srv_size, i_size) is fully granted, mtime and ctime while the cache under the grant is still dirty. A remote truncate has to revoke first, so the smaller size that follows is applied as usual. The attribute-driven invalidation now keys off STATX_SIZE instead of the whole mask, so a reply that does shrink i_size still truncates the page cache when only the timestamps were served from the cache. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com> (cherry picked from commit a6ca3d1) Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
Without handle_killpriv[_v2], fuse_setattr() kills the bits by asking the server, and fuse_do_setattr() freezes writepages around that SETATTR. fuse_set_nowrite() asserts BUG_ON(fi->writectr < 0) under fi->lock, which assumes the caller holds i_rwsem exclusive: with the writeback cache and DLM, buffered writes hold it only shared. Two writers to a suid file can then both pass dentry_needs_remove_privs() before either has cleared the bits, and the second one hits the assert. It oopses inside spin_lock(&fi->lock), so fi->lock stays held and the i_rwsem read count leaks: the inode wedges and the box follows. The race window is a full GETATTR plus SETATTR, so it is not narrow, and an unprivileged user can set the bit on a file it owns. Keep those writes off the writeback path, which is the one that relaxes i_rwsem to shared, the way handle_killpriv_v2 writes already are. Scoped to DLM connections, since every other configuration already holds i_rwsem exclusive for a buffered write, and only writes that still find the bits set pay for it. Note that the non-writeback path takes no DLM lock, so those writes leave clean folios in the page cache without a grant covering them. That gap already exists for handle_killpriv_v2 and is not addressed here. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com> (cherry picked from commit d109f0f) Signed-off-by: Allison Henderson <allison.henderson@ddn.com>
achhenderson
force-pushed
the
redfs-ubuntu-resolute-7.0.0-14.14
branch
from
August 15, 2026 17:54
388e2e9 to
f40355d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.