mm: page_alloc: do not give all non-blocking requests reserve access - #2656
Open
vfsci-bot[bot] wants to merge 1 commit into
Open
vfsci-bot[bot] wants to merge 1 commit into
vfsci-bot[bot] wants to merge 1 commit into
Conversation
…ations Commit 5d8edfb ("iomap: Copy larger chunks from userspace") introduced high-order folio allocations in the iomap buffered write path. When memory is fragmented, each failed costly-order allocation enters __alloc_pages_slowpath() which runs direct compaction and drain_all_pages(), causing a 0.38x throughput drop on PostgreSQL pgbench (simple-update) with 1024 clients on a 96-vCPU arm64 system. The root issue is that direct compaction is too expensive for hot allocation paths that have fallbacks to smaller allocations. __filemap_get_folio_mpol() already marks higher-order allocations with __GFP_NORETRY | __GFP_NOWARN, signalling that the caller can handle failure. However, the page allocator still attempts full direct compaction for costly orders with __GFP_NORETRY, which is unnecessarily aggressive when the caller will simply retry at a lower order. For costly-order allocations with __GFP_NORETRY, clear __GFP_DIRECT_RECLAIM at the very start of the slowpath, before can_direct_reclaim, can_compact and the nofail checks are evaluated. This makes the entire slowpath treat the request as non-blocking: no direct reclaim, no direct compaction and no drain_all_pages() IPI across every CPU. kswapd (and in turn kcompactd) is still woken further down for background defragmentation, so compaction keeps working for long-term system health while being removed from the latency-critical direct allocation path. Allocations that also request __GFP_THISNODE are exempted. That flag pairing identifies the local-node-first THP attempt issued by alloc_pages_mpol() (mempolicy.c), which relies on direct compaction to form transparent huge pages. Test environment: Hardware: AWS EC2 m8g.24xlarge (96 vCPU, arm64) 12x 1TB IO2 32000 IOPS RAID0 XFS OS: AL2023 Kernel: v7.3-rc1 Database: PostgreSQL 18.4 Workload: pgbench simple-update, 1024 clients, 96 threads, 1200s Results (average of 3 runs, TPS): Config Avg TPS % vs Baseline baseline (no patch) 59,408 - With this patch 155,409 +161.6% Link: https://lore.kernel.org/all/20260403193535.9970-1-dipiets@amazon.it/T/#t [v1] Link: https://lore.kernel.org/linux-mm/20260420161404.642-1-dipiets@amazon.it/T/#u [v2] Link: https://lore.kernel.org/all/20260710143437.12379-1-dipiets@amazon.it/T/#u [v3] Fixes: 5d8edfb ("iomap: Copy larger chunks from userspace") Cc: stable@vger.kernel.org Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: David Hildenbrand <david@redhat.com> Cc: Michal Hocko <mhocko@suse.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Matthew Wilcox <willy@infradead.org> Cc: Christoph Hellwig <hch@lst.de> Cc: Dave Chinner <dgc@kernel.org> Cc: Ritesh Harjani <ritesh.list@gmail.com> Cc: linux-mm@kvack.org Cc: linux-fsdevel@vger.kernel.org Cc: linux-xfs@vger.kernel.org Signed-off-by: Salvatore Dipietro <dipiets@amazon.it> Link: https://lore.kernel.org/20260904115629.3993331-1-dipiets@amazon.it Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: Johannes Weiner <hannes@cmpxchg.org> Acked-by: Zi Yan <ziy@nvidia.com> Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Series: https://patchwork.kernel.org/project/linux-fsdevel/list/?series=1170459
Submitter: Johannes Weiner
Version: 1
Patches: 2/2
Message-ID:
<arFBWvzXPYrKuIYi@cmpxchg.org>Base: vfs.base.ci
Lore: https://lore.kernel.org/linux-fsdevel/arFBWvzXPYrKuIYi@cmpxchg.org
Automated by ml2pr