Repository navigation
drm/asahi: g17: M5 Pro: wait for other queues' compute in firmware when it ends visible - #1411
Merged
DjDeveloperr merged 1 commit intoNov 10, 2026
Conversation
Contributor
Author
|
Full kernel build of this branch at |
…en it ends visible On M5 Pro a compute command waiting for another queue's compute command waits on the host: the producer's completion interrupt, the event worker and a new publication sit between the two kicks, and a cross-queue timeline semaphore hop takes 155 us. macOS on the same firmware keeps the wait in firmware and hops in 17.4 us. Traces of macOS's hop show how its kicks are encoded: the consumer names the producer's kick and its own parent in the completion class (1), each descriptor counts its queue's completion ordinal and carries it as the submission sequence, the kick waits on its own context's pool event, and the dispatch base is the qualified 0x10c08860 with the auxiliary control set only for a kick whose parent is pipelined (class 2). The previous M5 Pro form (ordinal 0, auxiliary control 1) hangs as soon as a parent takes class 1. M5 Pro compute kicks now take macOS's form. A firmware wait is satisfied when the producer completes, which a stream ending in Honeykrisp's weak barrier reports before its writes are visible: with the waits in firmware, dEQP-VK.synchronization2.op.multi_queue. timeline_semaphore read stale data in 31 of 214 cases, every one with a compute or transfer producer. A stream that ends with the strong memory-scoped flush is visible at completion. Add DRM_ASAHI_COMPUTE_VISIBLE_COMPLETION, by which userspace declares such a stream, and report DRM_ASAHI_FEATURE_COMPUTE_VISIBLE_COMPLETION on M5 Pro; there only a submission whose compute command sets it is waited for in firmware, and the others keep the host wait. Measured on J714s (pinned to CPU 12, two boots of each kernel): the hop is 155.3 us before, 155.9 us with this change and current Mesa, and 21.5 us with Mesa that sets the flag. Light compute and render submits are unchanged within run-to-run noise. dEQP s0, compute, glsl, draw, texture, synchronization, synchronization2, the multi-queue samples and occlusion queries have no failures with either Mesa. Signed-off-by: Diljit Singh <hey@djdev.me>
DjDeveloperr
force-pushed
the
dj/g17-m5-visible-completion
branch
from
November 10, 2026 09:10
93f5fb7 to
8aee10d
Compare
DjDeveloperr
changed the base branch from
funkzy/t6050-gpu-on-integration
to
j716c/m5-max-host-producer-waits
November 10, 2026 09:10
DjDeveloperr
changed the base branch from
j716c/m5-max-host-producer-waits
to
funkzy/t6050-gpu-on-integration
November 10, 2026 10:20
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On M5 Pro (J714s) a compute command that waits for another queue's compute command waits on the host. The producer's completion interrupt, the event worker and a new publication sit between the two kicks, so a cross-queue timeline-semaphore hop takes 155 µs. macOS on the same firmware (26A428) keeps the wait in firmware and hops in 17.4 µs.
Depends on #1070. This branch is on #1070's head (
26d82876dbcf). Until #1070 merges, its commit shows here too. The change itself is the last commit,93f5fb7b50da.What macOS does
The kick and descriptor fields come from macOS-HV AGX traces of gpubench-mtl's hop (runs hv9 and hv10). The CDM tails come from WindowServer's compute streams in hv11.
completion_ordinal, and carries the same count assubmit_sequence.0x10c08860. The auxiliary control (0x0a731) is 1 only on a kick whose parent is class 2.Linux's previous M5 Pro form sent ordinal 0, sequence 0, a class-2 parent, fixed event 131, dispatch base
0x14c08a60and auxiliary control 1. Moving the parent to class 1 without the matching auxiliary control hangs the queue: Hyprland times out at boot, and dEQP loses the device.Why the wait couldn't simply move to firmware
A firmware wait is satisfied when the producer completes. Honeykrisp ends API dispatches and vk_meta copies, fills and updates with a weak barrier, which reports completion before the writes land.
With macOS's encoding and the waits in firmware,
dEQP-VK.synchronization2.op.multi_queue.timeline_semaphoreread stale data in 31 of 214 cases ("Memory contents don't match"). Every one had a compute or transfer producer. Ending those streams with macOS's own launch tail (barrier flags 1) still failed 30 of 214. Only the strong, memory-scoped flush (flags 5) is visible at completion.The change
DRM_ASAHI_COMPUTE_VISIBLE_COMPLETION(compute flag bit 20) lets userspace declare a stream that ends with the strong flush.DRM_ASAHI_FEATURE_COMPUTE_VISIBLE_COMPLETION(bit 34) is reported on M5 Pro only.The Mesa half is https://github.com/aurora-silicon/mesa-aurora/pull/33.
Measured (J714s, pinned to CPU 12; two boots per kernel, measured before any other work)
Light submits are unchanged within run-to-run noise (±5–10% per run).
dEQP on this change, with each Mesa, 0 Fail in every list:
gpubench core-a and feat: 41/41 with each Mesa. No GPU recoveries or timeouts.
How it was tested
The tests used aurora.ko built from this change on the argb integration base and loaded into 7.3.0-m5-integration. That out-of-tree module build sees the integration tree's prebuilt UAPI bindings, so the tested build spelled the two new values out. Separately, bindgen run on this branch's header generates both
DRM_ASAHI_FEATURE_COMPUTE_VISIBLE_COMPLETIONanddrm_asahi_compute_flags_DRM_ASAHI_COMPUTE_VISIBLE_COMPLETION. Apart from those two constants and doc-comment wording, the diff is the tested one, and it applies to #1070's head with no conflicts.Update (2026-11-10): P-core A/B and the Mesa half
Every pinned run above used CPU 12, which is a P core in argb's CPU order (cpu0-11 M, cpu12-17 P). Re-measured pinned to all P cores (cpu12-17), on fresh boots, measured before anything else, two rounds each:
The stack here is argb + #1401 + this change + #1399 + #1405. The last two columns differ by 15% on light compute across separate boots, but that's boot-to-boot variation, not the flag. With the current Mesa, the tail-only Mesa and the flag-setting Mesa interleaved on one boot of that stack (three rounds, P cores), compute.light d8 measures 15.2 / 15.2 / 16.4 µs (rounds 15.4-16.7) and d32 15.2 / 15.0 / 15.1, with submit CPU the same in each (14.3-14.6 µs). So the change and the Mesa flag leave light submits unchanged. The measured submits carry no semaphore waits, so they never take the firmware-source input path anyway.
On that stack the hop is 21.4 µs (timeline) and 24.8 µs (binary) unpinned, from 153. dEQP s0, mq-tl and mq-v1 pass with 0 Fail.
🤖 Generated with Claude Code