[erikwb]

The problem

agx_stats on the M5 reported utilisation only: pstate, the frequencies
and busy_ns read "unsupported", and a client's fdinfo had its engines' busy
time but not the clock. There was no way to tell whether the GPU ran at full
clock during a workload, or how long it had been busy.

The change

drm/asahi: g17: M5: report the performance state, its frequency and busy time
decodes two more kinds of record of the M5 firmware's statistics ring:

• The performance state reports (tag 0x1b, time stamped by the interval
  record 0x19 before it) give pstate, and the board's performance state
  table gives achieved_frequency_hz for it. A report whose two state words
  differ, or that names a state outside the table, is not used. While the
  GPU is gated the state is stale, as before.
• The utilisation records (tag 0x13, every 8 ms) carry their own timestamps,
  and busy_ns integrates the device utilisation over the window each record
  closes. A record after a gap of more than two periods only sets a new
  baseline, so an idle gap is never credited.
• requested_frequency_hz, power_mw and temperature_raw stay
  unsupported. Other firmwares are unchanged.
• Documentation/ABI/testing/sysfs-driver-asahi-agx-stats and
  Documentation/gpu/asahi-usage-stats.rst describe the M5 values.

rust: drm: file: print the engines' frequency in fdinfo lets a Rust driver
print drm-maxfreq-<engine> (Documentation/gpu/drm-usage-stats.rst) and
drm-curfreq-<engine>, the key drivers print for the frequency an engine
runs at, beside drm-engine-<engine>. Every engine gets the same pair,
which suits a device whose engines share one clock. The default prints
nothing.

drm/asahi: g17: M5: report the GPU frequency in fdinfo prints them on the
M5 for the vertex, fragment and compute engines, which one clock drives:
the current frequency while the firmware reports a state, and the highest
frequency in the board's table. An fdinfo read on the M5 now takes the device
mutex once to drain the statistics rings, as a read of agx_stats does.

Documentation/gpu/asahi-usage-stats.rst had drm-maxfreq-* left out on
purpose, because engine times must not be scaled by a frequency. This series
keeps drm-cycles-* absent and documents the two keys as the device's clock
at the time of the read, not as a weight for the engine times.

Testing

Hardware: M5 Max MacBook Pro (J716c).

• Build: integration/aurora c868213 plus this branch, with the J716c
  configuration, at each commit. No warning in the files the branch touches.

• KUnit, arm64 with 16K pages in QEMU:
  ./tools/testing/kunit/kunit.py run --arch=arm64 --kunitconfig=drivers/soc/apple/sep.kunitconfig:
  Ran 601 tests: passed: 599, skipped: 2, with three new tests in
  asahi_g17_telemetry (9 of 9).

• Host tests, as CI runs them: python3 -I tools/asahi/run-host-tests.py:
  297/302 passed. The five that fail (apple-dcp, g15-pb-growth-tests
  and g15-stats-drain-tests, each of the last two also with --mutations)
  fail identically on integration/aurora c868213 on the build host,
  which lacks the Rust 1.93.1 toolchain the g15 tests ask for.
  Among them g17-telemetry-integration-tests with each of its eleven
  mutations, g17-telemetry-tests (9 unit tests, three new) and
  legacy-frequency-tests (2, one new).

• scripts/checkpatch.pl: 0 errors, 0 warnings on the three commits.

• On the J716c, with a Vulkan test program whose compute jobs are a
  single-thread loop of dependent integer operations (54.0 M iterations/s at
  1620 MHz), reading agx_stats while it runs:

  load	result
  one 1.4 s job that fills the GPU, agx_stats read every 100 ms	12 of 12 reads: pstate 13, achieved_frequency_hz 1620000000, util_device_percent 100
  60 jobs of 20000 iterations, 30 ms apart, agx_stats read every 1 ms	48 jobs ran in about 1792 us (335 MHz), 12 in under 600 us. Of the reads that found a current state, 48 gave pstate 1, achieved_frequency_hz 338000000 and 11 gave pstate 13, achieved_frequency_hz 1620000000. The other reads found the GPU gated (pstate_status stale)
  three 300 ms jobs at full clock, 300 ms apart (two runs)	busy_ns advanced by 871 ms and 863 ms for 905 ms and 919 ms of work, over 1.83 s and 1.84 s of wall time

• On the J716c, the fdinfo of the test program during a 1.4 s job that fills
  the GPU has, for each of vertex, fragment and compute,
  drm-curfreq-<engine>: 1620000000 Hz and
  drm-maxfreq-<engine>: 1620000000 Hz. An idle client's fdinfo, with the
  GPU gated, has the drm-maxfreq lines only.

• The J716c runs above were made with this series on the
  integration/aurora of 2026-11-09 (e7f996f plus the then-open J716c
  pull requests), kernel 7.3.0, Mesa aurora-wip with the open J716c pull
  requests.

• On integration/aurora ea10c4d with this series (kernel
  7.3.0-ewb107+ on the J716c), the same program gave: 12 of 12 reads at
  full clock with pstate 13, achieved_frequency_hz 1620000000 and
  util_device_percent 100; 59 of the 60 short jobs at about 1792 us and
  one under 600 us, the reads that found a current state giving pstate 1,
  achieved_frequency_hz 338000000 (57) or pstate 13,
  achieved_frequency_hz 1620000000 (2); busy_ns advancing by 848 ms for
  918 ms of work over 1.85 s of wall time; and the fdinfo lines above.

• On integration/aurora c868213 with this series, drm/asahi: GEM_RESIDENCY, keep an object's memory resident #1610 and drm/asahi: g17: per-queue watchdog allowance for compute commands #1752
  (kernel 7.3.0-ewb122+ on the J716c): 12 of 12 reads at full clock with
  pstate 13 and achieved_frequency_hz 1620000000 in each of three runs
  (util_device_percent 100 in the two runs after the desktop had
  started, 0 in the first run after boot); all 60 short jobs at
  about 1792 us, the reads that found a current state giving pstate 1,
  achieved_frequency_hz 338000000 (49) or pstate 13,
  achieved_frequency_hz 1620000000 (1); busy_ns advancing by 896 ms for
  932 ms of work over 1.85 s of wall time; the fdinfo lines above, and
  drm-maxfreq-<engine>: 1620000000 Hz only for the idle compositor. The
  kernel log had no Oops, BUG, WARNING or GPU recovery line.

Depends on

Nothing.

Known limitation reported on J815

The J815 owner reports carrying this series on a J815 with J813 board data and V26A428_M5. Under a frame-paced load, pstate was stale in 35 of 37 samples and busy_ns stayed zero despite incoming statistics records. Frequent core-power boundaries clear the per-epoch accounting baseline and discard the queued prefix. This is a limitation of the conservative policy described above; the J716c sustained-load measurements do not establish useful accounting on every M5 workload. The report does not provide a full composed-kernel commit/module identity.

The owner proposes two follow-up commits, 8c58f1d (status labels) and 2c515ec (power-gate accounting/publication). Their reported J815 improvement is 36/36 valid samples and growing busy_ns, with 9/9 telemetry tests. Those commits are not part of this PR. Review found that the proposed policy change needs accounting high-water-mark/overlap protection, ordered current-state publication distinct from historical records, a decision about non-M5 output, and matching ABI/fdinfo documentation. A separate reviewed follow-up is preferred; a retained last-known frequency is not automatically a qualified current frequency while gated.

The existing pre-merge check that agx_stats and fdinfo remain unchanged on a non-M5 G17 is still pending on the published series. No new J815 hardware run is requested by this review.

Not covered

• M5 machines other than the J716c: the decode is the same for every board
  that runs the M5 command set, but only the J716c was measured.
• Performance states other than 1 and 13: the test loads only produced the
  lowest and the highest.
• requested_frequency_hz, power_mw and temperature_raw are not
  implemented.
• No fdinfo consumer (gputop, nvtop) was run against the new keys; the
  machine has neither installed.

🤖 Generated with Claude Code

[DjDeveloperr]

Reviewed at f7604d7, and run on an M5 Pro (J714s, T6050, G17S, firmware 26A428). Test kernel: integration/aurora 69a9fbb + d76f650 + 86708ce + #1608..#1611. This series' files are byte-identical to the PR head.

Verdict: approve.

Other GPUs (checked by reading):

• Telemetry::m5() is chosen only for variant.m5_commands, and both new record arms are if self.m5. On the T8140 (G17P), the default telemetry ignores tags 0x19/0x1b, busy_status stays unsupported, and reports_pstate() still returns false.
• The fdinfo keys need the board table (cfg.m5). Without it, frequency() returns None before it reads anything, so a non-M5 fdinfo read takes no device mutex.
• The larger Readout (80 bytes) is filled only by read_g17; the legacy GPUs format through read_frequency. The C side's asserts match.
• The J714 board table has the same 14 entries as J716c's (state 1 = 338 MHz, state 13 = 1620 MHz), so the decode applies unchanged.

On the M5 Pro (J714s):
agx_stats sampled every 100 ms, with fdinfo read during each phase. Baseline (integration/aurora 447b752 without the series, idle plus the same compute.heavy load): pstate, achieved_frequency_hz and busy_ns read unsupported throughout.

phase (gpubench, ~6.4 s)	samples	pstate / frequency	util_device	busy_ns advance
idle	32	stale	0	0
compute.heavy alu, depth 8	64	13 / 1620 MHz in 63, 1 stale at the start	mean 98%	+6345 ms in 6.38 s (99%)
idle	33	stale	0	0
compute.light, depth 1 (~160 us per submit)	64	13 / 1620 MHz in all 64	utilisation stale	0
idle	29	stale	0	0
render.heavy fill 1080p, depth 8	64	13 / 1620 MHz in 63, 1 stale at the start	mean 98%	+6360 ms in 6.38 s (100%)
idle	34	stale	0	0

• fdinfo: under load every client has drm-curfreq-{vertex,fragment,compute}: 1620000000 Hz beside drm-maxfreq-*: 1620000000 Hz. With the GPU gated it has drm-maxfreq-* only. The drm-engine-* keys are still there beside them.
• Other states: no state other than 13 appeared in these loads, so the board-table lookup for lower states was not exercised here.
• No regression on the same boot: multi-queue timeline 214/214, synchronization2 600/421 NS, gpubench 41/41, 0 recoveries, 0 Oops/WARN.

One observation, not a regression: during compute.light at depth 1, the firmware delivered no tag-0x13 records for the whole 6.3 s. utilisation_status was stale and utilisation_samples 0, while the client's drm-engine-compute rose by ~45 ms. So busy_ns did not advance during that sparse load, although pstate 13 was reported. This is the existing utilisation path, and busy_ns inherits it. Worth one sentence in the ABI doc: busy_ns counts only windows the firmware reports, and sparse sub-millisecond work may go uncounted.

Non-blocking:

1. Hard-coded top state. M5_MAX_PSTATE = 13 restates the table length. Deriving it from perf_states.len() - 1 would keep the two from drifting.
2. Device mutex per fdinfo read. On the M5, each fdinfo read now takes the device mutex, which is the submission path's hot lock. That's fine for a top-like tool at 1-2 Hz. A monitor that reads every client's fdinfo in a tight loop would add contention, and the docs already say so.

Build shortcut, for the record: the M5 test kernel was built incrementally with include/linux/dma-mapping.h and include/drm/drm_gem_shmem_helper.h at their old mtimes, so files that only include them weren't recompiled. The hunks are a file-scope #define plus comment, and a same-type, same-position field rename; this head has no green CI compile to stand in for a clean build (below).

CI: kunit, the only full kernel build, is red on this head (https://github.com/aurora-silicon/linux/actions/runs/43125705476/job/130877105315). It failed compiling drivers/pmdomain/apple/pmp-report.c (get_cpu_idle_time undeclared), a file this series doesn't touch: integration/aurora's state when the check ran, fixed since by #1594 (8333109). So there is no green CI compile of this PR yet. A re-run reuses the original merge commit, so it needs a fresh run (a new push, or a re-trigger after the base update). host-tests, source and rust-build are green (rust-build compiles only apple-sep.o).

[RyanTheTide]

@erikwb kunit fails on pmp-report.c:1704 (get_cpu_idle_time), which comes from the base this branch sits on, not from your change. Please rebase onto current integration/aurora so CI is green. One small thing: frequency() takes the last row of perf_states as the maximum; taking the max over the table, or a comment that the table is sorted, would make that safe for other boards.

[erikwb]

Thanks. Rebased onto current integration/aurora (c868213), which has the idle-accounting change behind the kunit failure; head is now 17eca40.

frequency() now takes the highest frequency over the board's table instead of the last row, and the doc comment, asahi-usage-stats.rst and the commit message say "highest frequency in the board's table". The other two commits are unchanged.

Rechecked on the new base: each commit builds with the J716c config, KUnit 599 passed / 2 skipped, and a J716c boot with this series (plus #1610 and #1752) gives the same agx_stats and fdinfo results as before; the testing section has the numbers. Separately, g17/mod.rs still reads perf_states[13] for GET_PARAMS max_frequency_khz; that is outside this series, but I can send the same change there if you want it.

[RyanTheTide]

Rebase and the max-over-table change look good, and CI is green. Before merge it needs a check that agx_stats and fdinfo are unchanged on a non-M5 G17 before merging. A separate patch for the perf_states[13] read in GET_PARAMS would be welcome.

[DjDeveloperr]

Current-head readiness review on @DjDeveloperr’s behalf.

The rebase/max-over-table change is present. frequency() derives the maximum from the board table and returns before read_firmware()/the device lock when no M5 table exists. The default DriverFile hook emits no frequency keys, and M5 decoding remains gated by the variant flag. I found no new source blocker in those changes.

Verified actual green Host43360666695/job131673136521 and SEP43360666812 build/test steps. The hosted QEMU job131673138336 reports294 tests; it is not the author’s separately reported599-pass/2-skip run with the G17 telemetry suite. Keep those scopes distinct. The existing M5 Pro and J716c hardware reports retain their exact composed-kernel limits.

This is still held by Ryan’s5886924588 pre-merge requirement: agx_stats and fdinfo unchanged on a non-M5 G17, with current source/composition identity. Green checks or the M5 rows do not supply that result. The separate GET_PARAMS perf_states[13] cleanup remains nonblocking follow-up. No merge, new hardware run or global GPU/reclaim action performed.

[m0d16l14n1]

@erikwb thanks for this. I carried #1611 on a J815 (MacBook Air 15" M5, J813 board data on V26A428_M5). Idle it reads as expected, but under load pstate was stale in 35 of 37 samples and busy_ns stayed 0, although 0x19/0x1b/0x13 records were flowing (role 1 0x19 ×56290, 0x1b ×29791; last 0x1b = pstate 12 = 1578 MHz, matching fdinfo drm-curfreq).

Cause: under a frame-paced load the J815 GPU power-gates about 95 times a second (power_epoch 10861 after a 20 s run). Each boundary (observe_statistics_power) invalidated the pstate, reset the busy baseline (so integrate_busy never had two records ≤ 16 ms apart in one epoch) and set a cutoff that discarded the records already queued.

I have two small commits on top of your current head (17eca40), in case you want to fold them in or have them as a follow-up after this lands:

• drm/asahi: agx_stats: Say why pstate and frequency aren't valid: prints the status word (stale/unavailable/unsupported) instead of "unsupported" whenever the status isn't valid.
• drm/asahi: g17: Keep M5 GPU statistics across power gates: a power boundary keeps M5's last pstate current (only suspend/recovery/quarantine invalidate it); each 0x13 record credits min(gap, one 8 ms period); records kept out of the host-time window by a boundary are still decoded by their own timestamps; gpu_power_gated in agx_stats, and a diagnostic per-pstate residency from 0x19 in debugfs.

Branch: https://github.com/m0d16l14n1/linux/commits/j815/g17-m5-stats-gates

On J815 with es2gears + vkcube for 20 s, sampled every 0.5 s: pstate valid in 36/36 samples (state 12 = 1578 MHz = fdinfo), gpu_power_gated 1, busy_ns growing with the load. The telemetry module's rustc unit tests pass (9/9).

This does intentionally change your "stale while gated" rule for M5, so it's your call. If you'd rather I open it as a separate PR once #1611 is merged, I'll do that.

Assisted-by: Claude Opus 5.5

[DjDeveloperr]

Codex (GPT-6.1 Sol), reviewing on @DjDeveloperr’s behalf.

@erikwb @m0d16l14n1 Thanks for the J815 measurements. I checked the two public follow-up commits and had an independent source review. This is a concrete limitation of #1611's conservative per-epoch accounting: on the reported frame-paced J815 workload, incoming records do not prevent 35/37 stale state samples and busy_ns=0. It is not evidence that the deliberately documented stale-while-gated rule should simply become valid-while-gated. I have added the report and its scope to the description.

I favor a separate reviewed follow-up rather than folding 8c58f1d + 2c515ec into this PR unchanged:

• In 2c515ec, power_boundary() calls invalidate(), clearing busy_ts; the first later record then receives a full 8 ms credit. Together with admitting cutoff records through record_timed(), this loses the accounting high-water mark across gates. Source permits an older queued timestamp to earn another period after a newer timestamp was already counted. Keep accounting ordering/overlap protection separate from the utilisation epoch, and qualify the first-record duration.
• record_timed() routes cutoff 0x19/0x1b records into the current-state publisher without a firmware-timestamp ordering check. Across role drains, an older queued state can replace a newer one and get a fresh host receive time. Historical accounting and publication of the current pstate need separate admission rules.
• gpu_power_gated is added unconditionally in the common qualified sysfs formatter, so the proposed series also changes G16S/G17P output. That cannot inherit this PR's unchanged-non-M5 claim.
• ABI/usage docs still promise stale pstate and absent drm-curfreq while gated, the old busy-window rule, and queued-prefix discard. Retaining last-known state as valid changes those contracts; the extra sysfs flag does not qualify a retained fdinfo current-frequency value. Document and validate the chosen semantics rather than counting more valid samples as correctness proof.

The status-label-only 8c58f1d is separable and reasonable with the corresponding M5 placeholder documentation. Neither candidate is adopted here. Suspend/recovery/quarantine guards remain in the candidate; these findings concern the power-boundary path, not an asserted bypass of those guards.

The existing non-M5 G17 agx_stats/fdinfo regression gate on published 17eca40 remains pending. The J815 results and reported 9/9 tests cover different questions and do not close it. No additional hardware run requested from the J815 owner in this review.

[DjDeveloperr]

Independent review of 17eca40c50cd: no blocker.

What changed since f7604d79: I compared the three commits patch by patch. The only change besides line numbers is in the third commit: frequency() now takes max() over the board table instead of last(), with matching doc, rst and commit-message changes.

• On J714s and J716c the top state is also the last row, so the M5 Pro results in my approval still hold.
• On T8142/J813, states 12 and 13 share the top value (1578 MHz), so the two methods agree there too.

Non-M5 output, by source:

• T8140/T6040/T6041 print the same bytes in agx_stats (five "unsupported" lines, as before) and in fdinfo (no drm-curfreq/drm-maxfreq, no device mutex).
• The legacy G13–G16 path is untouched.
• This doesn't replace @RyanTheTide's hardware check on a non-M5 G17 (the J700 row), which this still waits for.

CI: green on this head. The CI-only #1834 (integration/aurora + #1610 + this PR + #1752) also passed: whole-kernel kunit, 305/305.

• drm/asahi: GEM_RESIDENCY, keep an object's memory resident #1610 has merged since. It touches file.rs and sysfs.c two lines from this PR's hunks, but merges cleanly with it, as in CI only: erikwb GPU PRs #1610 + #1611 + #1752 composed on integration/aurora (do not merge) #1834.

Non-blocking notes for @erikwb (line numbers at the head):

1. The 0x10 arm is open on M5 (g17/telemetry.rs:186-199 with g17/stats.rs:94-98).
   • The legacy 0x10 arm isn't limited to non-M5, and reports_pstate(graph, true) now lets it publish. The M5 ring admits tag 0x10 (bit 16 of STATISTICS_TYPES).
   • If M5 firmware ever sends one, its selector would read as a pstate (selector 6 → 988 MHz in agx_stats and drm-curfreq), and a backwards timestamp would clear busy_ts.
   • None of our runs showed one. Suggest 0x10 if !self.m5 =>.
2. busy_status on M5 stays valid while the statistics ring is quarantined, crashed, recovering or suspended (telemetry.rs:258-263). In those states busy_ns freezes but still reads valid, while the ABI doc says a quarantined ring disables the exports. Either report stale there, or document the freeze.
3. One busy_ts baseline for both firmware roles (telemetry.rs:227-237). If both M5 roles send 0x13 and records queue between drains, time could be counted twice. Our measured ≤100% suggests only one role sends it. Which roles do?
4. Two doc lines no longer hold on M5:
   • The ABI doc's "Tick deltas are never accumulated as GPU busy time": M5 busy_ns is exactly that.
   • asahi-usage-stats.rst's "does not … access hardware" for fdinfo: an M5 read reads the core-power register twice.
5. The layout asserts don't catch swapped fields (sysfs.c:236-238, telemetry.rs:43-47): two fields swapped on one side still pass. Offset asserts at 64 and 72 on both sides would catch it.

[RyanTheTide]

Non-M5 G17 hardware gate (pre-merge): PASS

Ran an A/B on an A18 Pro J700 (apple,agx-t8140, G17P, non-M5) to confirm this PR is a no-op on hardware without the M5 performance-state table.

• Base c86821352a57b (without this PR) vs test = this PR head 17eca40c50cd.
• Both built with the same config (DRM_ASAHI=m, CONFIG_MODVERSIONS off), identical kernel.release, so aurora.ko is swappable on one installed kernel. The two builds export an identical symbol set (Module.symvers: 32325 = 32325) and every undefined symbol in the PR's module resolves against the base vmlinux, so the on-disk module swap is ABI-safe. The J815 policy commits (8c58f1dc, 2c515ecc) are not included.
• Module SHA-256: base 49849c01…, PR 25a47007….

Captured agx_stats (idle + an ordinary vkcube GPU load in the desktop session) and a GPU client's fdinfo on each side:

• agx_stats key set identical (14 keys), idle and load.
• pstate_status, requested_frequency_hz, achieved_frequency_hz, busy_ns, power_mw, temperature_raw: unsupported on both base and PR, idle and load — unchanged. On t8140 reports_pstate() is false and pstate_status != VALID, so the new achieved-frequency / busy-time branches are never taken (matching the ABI note in the PR).
• fdinfo drm-* key set identical; no new drm-curfreq-* / drm-maxfreq-* keys on the PR side (engine_frequency() returns None without the board's performance-state table). drm-engine-* counters advanced under load on both builds.
• No new GPU error/fault/warn lines on either side.

The only difference seen was in the firmware-utilisation telemetry (tag 0x13): util_{tiler,renderer,device}_percent / utilisation_status read valid/numeric in one load window and stale/unsupported in another. This PR does not touch those fields, and they flip within a single unchanged build (e.g. idle valid vs load stale on the base), i.e. live-sampler freshness variance — not a change from this PR.

Conclusion: no agx_stats/fdinfo ABI change, and the unsupported states and error returns are unchanged, on non-M5 G17 (t8140).

[DjDeveloperr]

Approving at 17eca40c50cd. Merging now.

• Non-M5 G17 gate: passed. @RyanTheTide's J700 (T8140, G17P) A/B above shows identical agx_stats and fdinfo key sets, and the new fields stay unsupported. There are no drm-curfreq/drm-maxfreq keys and no new GPU error lines.
• CI: all 7 checks are green on a fresh run of "Merge 17eca40 into 954a6a7", which is integration/aurora with #1610. kunit builds the whole kernel with DRM_ASAHI=y.
  • The base has since gained 4 commits, all J714s DT files, none in this PR's paths.
  • The merge tree with the current base is clean.
• M5 Pro (J714s): the hardware run in my earlier approval was at f7604d79. The only functional change since is frequency() taking the max over the table instead of the last row, and on J714s both are state 13 = 1620 MHz.
  • The composed test kernel with #1610, #1611 and #1752 on integration/aurora (#1834) goes through an M5 slot next. I'll post its rows here.
• Review: the independent review found no blocker. Its non-blocking notes are in my comment above.