[erikwb]

The problem

On G17, reclaim may release the backing of the VM-private objects of an idle
VM. The VM's next job then waits until the memory is back. A client that
would rather not pay that wait for some of its objects (a server that holds
model weights between requests, an application that gives its memory Vulkan
priority 1.0) had no way to say which: when reclaim took idle memory of its
VM, any object could be the one to go.

The change

drm/asahi: GEM_RESIDENCY, keep an object's memory resident adds
DRM_IOCTL_ASAHI_GEM_RESIDENCY, which sets or clears a per-object keep flag
on G17 GPUs. The flag is a priority among the objects that reclaim may
release, which is what its Vulkan use (a memory priority, a hint) needs, and
no more than that:

• Which reclaim releases GPU memory does not change. The driver's VM
  shrinkers are not memcg-aware, so only reclaim on behalf of the whole
  system calls them: kswapd, direct reclaim, a request to the root memory
  cgroup, drop_caches. Reclaim inside a memory cgroup below the root (a
  cgroup at its limit, a memory.reclaim request to it) releases no GPU
  memory, kept or not.
• Which VMs reclaim takes from does not change: an idle one, with no job,
  binding change or other use in progress or in the last two seconds, and
  past the backoff that follows a restoration (16 times what the restoration
  took, between 1 and 30 seconds). The memory of a VM in use stays, kept or
  not.
• Within such a VM, kept objects are released after the others, and only
  when those left the scan target unmet.
• Contents advised DRM_ASAHI_GEM_MADV_DONTNEED are purged as before, kept or
  not.
• The request is therefore a hint. Userspace cannot rely on a kept object
  being resident, and a client that stays active, or whose memory cgroup is
  the one running out of memory, keeps its GPU memory whether it asks or not.
• The flag is changed under the object's reservation, which idle reclaim
  holds for the VM it walks, so a request applies to every reclaim pass that
  starts after the ioctl returns. Backing that was already released comes
  back with the VM's next job, as before.
• Shared, imported and user-memory objects are never released and accept the
  flag without effect on them. Kernel objects are refused.
• Kept objects count against the memory limits as before. The device total
  of their sizes is reported as user_kept in the GPU's gpu_memory sysfs
  file.

How the order is kept: a user VM gets a second shrinker for its kept objects.
Its batch is larger than any scan target, so the shrinker core calls its scan
only when the target reaches everything the shrinker counted. With the
driver's seeks that is the last two of the thirteen reclaim priorities, or
the last three once scans deferred by earlier passes have built up. That scan
releases what the VM's first shrinker would and only then, if the target is
not met, the kept objects; and the second shrinker is registered after the
first, so a reclaim pass that reaches it has asked the first one before. The
first shrinker (purge, and release of objects that are not kept) is
unchanged. Reclaim comes to those priorities when the earlier ones did not
free what it was asked for: under memory pressure that nothing else relieves,
for a request that names more than the system has to spare (a
memory.reclaim request to the root cgroup when there is little page cache
to take), and at once with drop_caches. A second pass inside the existing
scan was not enough: a scan is not told the reclaim priority, so kept objects
counted there would go at any priority whenever a VM had nothing else to
give, and left uncounted, a VM with only kept objects would never be scanned.

selftests: drm/asahi: GEM_RESIDENCY adds a kselftest program for the render
node (tools/testing/selftests/drivers/gpu/asahi-residency, listed in the
selftests' TARGETS). Besides the accounting it exercises reclaim. Those
cases act on the whole system, need root and run only when asked for in the
environment:

• ASAHI_RESIDENCY_RECLAIM=1 runs light_reclaim_spares_kept_objects,
  kept_objects_stay_purgeable and requests_serialize_with_reclaim, which
  write to memory.reclaim of the root memory cgroup.
• ASAHI_RESIDENCY_DROP_CACHES=1 runs
  kept_objects_go_under_urgent_pressure, which writes 2 to
  /proc/sys/vm/drop_caches: every shrinker of the system at the last
  priority.

A memory.reclaim request names an amount, not a priority: the kernel works
down its reclaim priorities until it has reclaimed that much, and where
nothing else can be reclaimed it ends at the last ones, at which kept objects
go like everything else. The three cases therefore bring the memory to
reclaim themselves, a sparse temporary file of 2 GiB in the working
directory that is read once into the page cache, and ask for 4 MiB at a
time, which the kernel usually finds there within the first priorities. The
race case has a control object that is not kept and must be released while
the requests race, which shows that reclaim got to the VM. The cases skip
without four times their objects' size of available memory (objects of
8 GiB), in a working directory on tmpfs, and in a memory cgroup with
memory.min or memory.low protection.

When a kept object was released, a case has to tell whether reclaim had
leave to release it. It does that in one of two ways, and its output says
which:

• From a record of the shrinker passes, where it can have one: the events
  vmscan:mm_shrink_slab_start and vmscan:mm_shrink_slab_end in a trace
  instance (tracefs at /sys/kernel/tracing, kernel symbols). The shrinker
  core calls a VM's shrinker of kept objects only in a pass whose target
  reaches everything that shrinker counted, and the start event carries both
  numbers.
  • Whose shrinker: the events name a shrinker by its pointer, which says
    nothing of the VM. A case knows its own VM's shrinker by what it counts,
    the pages of the VM's idle kept objects: it adds a random 0 to 1023 pages
    to the size of its kept object and takes for its VM's shrinker the one
    that never counted anything else. Passes over other VMs' shrinkers of
    kept objects do not enter into the verdict.
  • When: leave to release the object is a pass that called that shrinker
    before the case first saw the object gone and that had not ended when
    the case last saw it resident. A pass has ended with the
    mm_shrink_slab_end event of the same shrinker from the same task. The
    trace instance runs on the monotonic clock, which the case reads before
    and after each look at what its client holds.
  • The verdicts: with such a pass the case is skipped with "as it may" and
    the number and priority of those passes. Without one it fails. If more
    than one shrinker in the record counts what the test's VM counts and a
    pass called one of them, the record cannot tell whose pass it was: the
    case is skipped with "inconclusive", which claims nothing about the
    release.
  • The record is used only when no event was lost and it shows the driver's
    other shrinker at work; the driver's shrinkers are told by the names of
    their scan functions.
• Otherwise from signs of the environment, which are heuristics: whether the
  kernel met every request, whether 512 MiB of the file were still cached
  after each, and whether kswapd or direct reclaim scanned pages meanwhile.
  A kept object released with one of these signs skips the case, with the
  sign and the remark that it is a heuristic; without one the case fails.
  The signs make it likely, and do not show, that reclaim stayed at its
  first priorities: reclaim that goes to the last ones while every request
  is met and the cache stays fails a case that the record would skip, and
  unrelated reclaim in the system turns a failure into a skip.

kept_objects_go_under_urgent_pressure also requires, where it has the
record, a pass that called its VM's shrinker of kept objects before the
object went. Every series of reclaim requests is checked for an error (a
failed request, a client total that could not be read) before its result is
taken for a released object.

The trace instance belongs to the program, not to a case: main() makes it
before the cases fork, as asahi-residency-<pid> and only when a reclaim
case was asked for, and removes it when the program exits or is ended by
SIGINT, SIGTERM, SIGHUP or SIGQUIT. Those signals wait while the instance is
set up or removed, and the handler makes only async-signal-safe calls, on
paths made beforehand. A case only empties the buffer and switches the two
events on and off, so a case that fails, times out or is killed leaves
nothing behind. An instance of that name that exists already is
somebody else's: it is left as it is and the cases go by the heuristic.

UAPI additions: ioctl DRM_ASAHI_GEM_RESIDENCY (id 13),
struct drm_asahi_gem_residency, the flag DRM_ASAHI_GEM_RESIDENCY_KEEP and
the feature DRM_ASAHI_FEATURE_GEM_RESIDENCY, bit 35.

Testing

Hardware: M5 Max MacBook Pro (J716c).

• Build: integration/aurora ea10c4d plus dma-mapping: add DMA_ATTR_NO_BOUNCE #1608
  and drm/asahi: GEM_USERPTR, buffer objects over the caller's memory #1609 plus this branch, with the J716c
  configuration, at each commit. No new warning: the build has the same 31
  Rust warnings at each commit, none in code the branch adds.
• KUnit, arm64 with 16K pages in QEMU:
  ./tools/testing/kunit/kunit.py run --arch=arm64 --kunitconfig=drivers/soc/apple/sep.kunitconfig:
  Ran 546 tests: passed: 544, skipped: 2, with the new asahi_memacct test
  kept_bytes_follow_the_object. KUnit does not reach the shrinkers.
• Host tests, as CI runs them: python3 -I tools/asahi/run-host-tests.py:
  219/224 passed. The five that fail (apple-dcp, g15-pb-growth-tests
  and g15-stats-drain-tests, each of the last two also with --mutations)
  fail identically on integration/aurora ea10c4d on the build host,
  which lacks the Rust 1.93.1 toolchain the g15 tests ask for.
• scripts/checkpatch.pl: 0 errors and 0 warnings on the driver commit; the
  selftest commit gets only the MAINTAINERS note for its new files.
• On the J716c, kernel 7.3.0-ewb111+, built from integration/aurora
  ea10c4d plus the two open J716c device tree pull requests and the
  kernel branches of this set, with the kernel code of this branch and of
  the branches below it as it is here: the selftest of this branch built
  there with gcc 16.1.1 against this branch's UAPI header:

  • unprivileged, without the two variables:
    # Totals: pass:3 fail:0 xfail:0 xpass:0 skip:4 error:0
    (kept_size_is_counted_once, bad_arguments,
    shared_objects_take_the_flag; each reclaim case says which variable is
    missing);

  • as root with both variables:
    # Totals: pass:7 fail:0 xfail:0 xpass:0 skip:0 error:0. With two bound
    objects of 8 GiB, one of them kept, the first reclaim request released
    the one that was not kept and 16 more left the kept one resident
    (light_reclaim_spares_kept_objects); once the request was withdrawn,
    the next reclaim request released it. A kept object advised DONTNEED was
    purged by the first request (kept_objects_stay_purgeable). While a
    thread set and cleared the request on one object 11.3 million times,
    reclaim released the control object with its 11th request and the
    toggled object, a kept object beside them stayed resident, also through
    16 further requests, and user_kept came back to its starting value
    (requests_serialize_with_reclaim). The cold cache had 1995 to 2023 MiB
    of its 2048 left after each case. These cases had the record of the
    shrinker passes, with one shrinker that counted what the test's VM
    counts: "reclaim passes over the shrinker of this VM's kept objects: 51,
    of which 0 called it (traced, 1 such shrinker)", and 70 with 0 in the
    race. kept_objects_go_under_urgent_pressure released the kept object
    as well (the client held nothing afterwards), its contents were
    retained, user_kept still counted it, and the record showed the pass:
    "reclaim passes that called the shrinker of this VM's kept objects
    before it went: 1, down to priority 0 (traced)";

  • as root with ASAHI_RESIDENCY_RECLAIM=1, right after that run and once
    more later: # Totals: pass:6 fail:0 xfail:0 xpass:0 skip:1 error:0
    both times, with 51 and 77, then 51 and 74, passes over the VM's
    shrinker of kept objects and again none that called it;

  • the same with the shrinker calls also traced from outside the program
    (vmscan:mm_shrink_slab_start in a second trace instance): of 6972
    calls, 5114 were at priority 11 and 1858 at priority 12; the driver's
    first shrinker was called 179 times and its shrinkers of kept objects
    considered 125 times, never with a target that reached the count. The
    program's own record in that run had 51 and 74 passes over its VM's
    shrinker of kept objects, the same 125, and none that called it;

  • as root with ASAHI_RESIDENCY_RECLAIM=1 and tracefs hidden from the
    program (a private mount over /sys/kernel/tracing):
    # Totals: pass:6 fail:0 xfail:0 xpass:0 skip:1 error:0, each reclaim
    case noting "the shrinker passes were not traced";

  • five changed copies of the program, to see light_reclaim_spares_kept_objects
    fail, skip and call a run inconclusive, each with the record and with
    tracefs hidden:

    • one that never asks to keep the object fails (Expected (long long)kept_size (8609349632) == resident (0); the size differs from
      run to run by the random pages), with the record ("no
      reclaim pass called the shrinker of this VM's kept objects before it
      went (traced: 0 shrinkers of kept objects count what this VM's
      counts, ...)") and without it;
    • one that writes 2 to drop_caches between the two phases of the case,
      so that reclaim does come to the kept object, is skipped with "the kept
      object went after 1 requests, as it may: reclaim passes that called
      the shrinker of this VM's kept objects before it went: 1, down to
      priority 0 (traced)"; without the record it fails, since the
      heuristic has no sign for such a pass;
    • one that does not keep its object while another client has a small
      kept object and every shrinker is run at the last priority: a pass
      calls the other client's shrinker of kept objects and the test's own
      object goes with ordinary reclaim. It fails ("0 shrinkers of kept
      objects count what this VM's counts"): another VM's pass is no leave;
    • one in which a pass with leave calls a shrinker with the test's count
      and ends before the case first looks, after which the object is not
      kept. It fails ("1 shrinkers of kept objects count what this VM's
      counts, with 1 passes and 1 calls at other times"): a pass that ended
      before the object was last seen resident is no leave;
    • one with a second client whose kept object has the same size, and
      drop_caches between the phases, is skipped with "inconclusive: 2
      shrinkers of kept objects count what this VM's counts, and 2 passes
      called one of them before it went (traced)".

    Without the record all five fail.

  • the trace instance: with a case killed together with its process group
    while it recorded (what the harness does at a timeout), the program
    reported the case as failed and removed its instance; ended by SIGTERM
    while a case recorded, it removed its instance (exit status 143); started
    with an instance of its own name made beforehand, with an event enabled
    in it, it left that instance as it was and the case passed with "the
    shrinker passes were not traced". Sent SIGTERM at a random moment of its
    first 0.3 s, 30 times (each run in a PID namespace of its own, so that
    nothing of it outlived the run), it ended with status 143 and left no
    instance each time. After every run above
    /sys/kernel/tracing/instances had no asahi-residency-* left.

• On 7.3.0-ewb111+, Vulkan CTS (commit 7ba03cd) with Mesa aurora-wip plus
  the honeykrisp change that maps memory priority 1.0 to this ioctl
  (VK_EXT_memory_priority, VK_EXT_pageable_device_local_memory):
  dEQP-VK.memory.binding.priority*, dEQP-VK.memory.binding.maintenance6.priority*,
  dEQP-VK.memory.pageable_allocation.* and the two extensions' feature
  cases under dEQP-VK.api: 430 passed, 0 failed, 0 not supported.
  user_kept reads 0 again after the run.

Depends on

#1609 (which is stacked on #1608):
the driver's ioctl table has no gaps, so the two new ioctls need an order,
and GEM_USERPTR takes id 12. The code of this branch does not otherwise use
it. The Vulkan side is a separate pull request against aurora-silicon/mesa.

Not covered

• The flag gives no protection against reclaim inside a memory cgroup below
  the root or for a VM that is in use: GPU memory is not released there with
  or without it. Making the shrinkers memcg-aware is not part of this.
• Memory pressure that takes the kernel to its last reclaim priorities of its
  own accord (kswapd or direct reclaim close to failing allocations) was not
  produced: the last priority was reached with drop_caches only.
• The reclaim cases were run while the machine had other reclaimable page
  cache, 8.4 GiB at the least. A machine with none, where the requests can
  only be met from the cases' own file, was not tried.
• The selftest tells its VM's shrinker of kept objects by its count, made
  particular by a random 0 to 1023 pages. A shrinker of another VM that
  counts exactly the same, and only that, while the test's own never shows
  in the record, would be taken for the test's; with both in the record the
  case says "inconclusive". The record tells the driver's shrinkers by the
  symbol names of their scan functions; where those are not in the trace (no
  kernel symbols), the cases fall back to the heuristic.
• The program's trace instance is left behind only if the program itself is
  killed with SIGKILL. A later run does not take it over; it has to be
  removed by hand (rmdir /sys/kernel/tracing/instances/asahi-residency-<pid>).
• The order holds per VM: one VM's kept objects may be released while another
  VM still has idle objects that are not kept, if that VM is in use at the
  moment or its shrinker comes later in the same pass.
• GPUs other than G17: they do not report the feature and the ioctl fails
  with EOPNOTSUPP.

🤖 Generated with Claude Code

[DjDeveloperr]

Reviewed at cb8258c, and run on an M5 Pro (J714s, T6050, G17S) with the same test kernel as #1609: integration/aurora 69a9fbb + d76f650 + 86708ce + #1608..#1611.

Verdict: approve.

No change without the ioctl. I checked this by reading, because the shrinker change reaches every G17 GPU, the T8140 included. With no object kept:

• idle_reclaim_count(false) counts exactly what it counted before: advised objects, plus idle objects whose keep_resident() is false.
• reclaim_idle_pages(target, false) runs only its first pass, with the old body.
• The new kept shrinker's count_kept returns 0, so do_shrink_slab() returns before any scan.

Other checks:

• The KEPT_BATCH = LONG_MAX reasoning matches do_shrink_slab(): the loop runs only when total_scan >= freeable, so priorities 1 and 0 (and 2 once nr_deferred reaches its cap of 2x the count).
• Registration order (kept second) is right, and Drop frees kept before raw.
• The flag is set under the object's resv, which is the VM's resv for every reclaimable (VM-private) object, so a reclaim walk cannot see a half-applied change.
• The sysfs layout change (user_kept, size 56) is asserted on both the Rust and the C side.

Non-blocking suggestions:

1. Per-VM cost. Every user VM now gets a second registered shrinker. Every system-wide reclaim pass also runs a second count walk over all of the VM's private BOs (two try-locks and a resv read per BO), even for VMs that never asked to keep anything. That is most of them. On an 8 GB T8140, kswapd runs often. A per-VM count of kept objects, kept up to date in set_keep_resident(), would let count_kept return 0 without the walk. Registering the second shrinker lazily, on the first keep request, would avoid the extra one entirely.
2. What user_kept counts. It includes user-memory (#1609) and shared objects that are flagged but can never be released. The sysfs comment says "whether their memory is held at the moment or not", which covers it, but a sentence saying that it counts requests, not reclaimable bytes, would help readers.

On the M5 Pro (J714s):

• Selftest asahi-residency, unprivileged: 3 pass, 4 skip (each reclaim case names its variable).
• As root with ASAHI_RESIDENCY_RECLAIM=1 ASAHI_RESIDENCY_DROP_CACHES=1: 5 pass, 2 skip.
  • kept_objects_stay_purgeable passes: purged after 1 request.
  • kept_objects_go_under_urgent_pressure passes.
  • light_reclaim_spares_kept_objects and requests_serialize_with_reclaim skip with "not enough available memory for the test objects". Their two and three 8 GiB objects need about 68 and 100 GiB available (4x the objects plus 4 GiB), and this 64 GB machine had 61.6 GiB, so the case that shows kept objects surviving light reclaim couldn't run here. Smaller objects (or sizes taken from MemAvailable) would let both run on 64 GB machines.
  • No trace instance was left, and user_kept read 0 before and after.
• Vulkan side, mesa #243 (on aurora-wip be147e36250b, built on the device): 426/426 pass, no GPU fault:
  • dEQP-VK.memory.binding.priority* 112/112;
  • dEQP-VK.memory.binding.maintenance6.priority* 112/112;
  • dEQP-VK.memory.pageable_allocation.* 202/202.
• Ordinary GPU work on the same boot, with the second shrinker registered per VM: multi-queue timeline 214/214, synchronization2 600 pass / 421 not supported, gpubench 41/41, 0 recoveries.

Build shortcut, for the record: the M5 test kernel was built incrementally with include/linux/dma-mapping.h and include/drm/drm_gem_shmem_helper.h at their old mtimes, so files that only include them weren't recompiled. The hunks are a file-scope #define plus comment, and a same-type, same-position field rename; this head has no green CI compile to stand in for a clean build (below).

CI: no check has run on this head. Its base j716c/gpu-gem-userptr isn't a host-tests target, and sep-source's paths don't include drivers/gpu/drm/asahi, so there is no CI compile of this PR yet. A run against integration/aurora once #1609 lands (or a manual dispatch) would give one.

[DjDeveloperr]

Current readiness/CI inspection on @DjDeveloperr’s behalf.

#1609 is merged and this PR already targets integration/aurora. This exact head still has no workflow runs. Its10 changed files are Asahi driver, UAPI and selftest paths; the current SEP source-check filter does not include those paths, and its rust-build job compiles apple-sep.o. A lifecycle re-trigger or a green SEP-only job alone would not establish a current Asahi-driver compile.

The author’s per-commit J716c build, QEMU and full7/7 hardware reclaim results in the description are separate reported evidence; the earlier J714s review used an incremental kernel and skipped the two oversized reclaim cases. Those scopes are retained rather than relabelled as clean current-merge CI. Actual validation of the final composed Asahi source/host tests remains needed before treating this as ready to merge.

No blind re-run, close/reopen, source churn or reclaim/drop_caches test was performed on the working host. Source approval is not being replaced by a fabricated check result; the missing coverage needs an appropriate build/CI route or exact clean final-composition proof.

[DjDeveloperr]

Merging this at cb8258cba2e2.

Evidence

• CI: host-tests passed 302/302 on a fresh run, "Merge cb8258c into 4277bdd" (run 43443602758). The base has since gained 8 commits (drm/apple DCP, DTs, regview, the apple-dcp selftest), none of them in this PR's paths. The merge tree with the current base is clean.
• Compile: CI only: erikwb GPU PRs #1610 + #1611 + #1752 composed on integration/aurora (do not merge) #1834 is a CI-only draft: integration/aurora 4277bddd plus drm/asahi: GEM_RESIDENCY, keep an object's memory resident #1610, drm/asahi: g17: M5: report the performance state, its frequency and busy time #1611 and drm/asahi: g17: per-queue watchdog allowance for compute commands #1752. Its kunit job built the whole kernel with DRM_ASAHI=y and ran 305/305 KUnit tests. The asahi warnings in that log are base code, not this PR's lines.
• Hardware: the head is unchanged since the M5 Pro (J714s) run in my approval.

Independent review: no blocker. Non-blocking follow-ups for @erikwb, none needed for this merge:

1. mmu/residency.rs, count_kept → idle_reclaim_count(true): on every shrink_slab call it try-locks each VM's resv and walks all of its private BOs. It does this even when nothing is kept anywhere, and on busy VMs too, because idle (line 457) never returns early. Returning 0 when kept && LEDGER.user_kept == 0, and when kept && !idle, removes the per-VM cost from the earlier review.
2. gem.rs, set_keep_resident: it takes the resv with the uninterruptible dma_resv_lock. Two ways to avoid an unkillable wait behind a restore or bind (or an exporter, for an imported object):
   • use the interruptible lock;
   • skip the lock for objects that aren't idle_reclaim_candidate, where the flag has no effect.
3. mmu/residency.rs reclaim: the first pass drops an error from invalidate_private_bo_mappings (let _), and the kept pass still runs. Kept objects can then go while later non-kept ones stay. Breaking on Err keeps the order.
4. KEPT_BATCH doc: "the last two priorities" means in practice priority 0, plus priority 1 once work has been deferred.

For the merge order: #1752 adds DRM_ASAHI_FEATURE_COMPUTE_TIMEOUT (bit 36) at the same spot in asahi_drm.h, so it will conflict with this PR. The resolution keeps both, in bit order, as in #1834.