Skip to content

soc: apple: mailbox: receive T6050 GPU mailboxes by interrupt - #1844

Merged
DjDeveloperr merged 2 commits into
integration/aurorafrom
j716c/t6050-gpu-mailbox-irq
Nov 13, 2026
Merged

DjDeveloperr merged 2 commits into
integration/aurorafrom
j716c/t6050-gpu-mailbox-irq

Conversation

@erikwb

@erikwb erikwb commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Problem

The T6050 GPU mailboxes (apple,t6050-ascwrap-v6, apple,t6050-x1n1-ascwrap-v6 and apple,t6050-j775d-ascwrap-v6) have received by polling since #1601. After each send, a 25 us hrtimer polls the receive FIFO for 1 ms; otherwise a once-per-jiffy delayed work polls it. A full send FIFO is waited out by sleeping polls.

ASC mailboxes are interrupt driven. On the 16-inch M5 Max MacBook Pro (J716c), polling costs about 5% of GPU throughput in an inference engine's decode step.

Change

The T6050 GPU mailboxes take the same start, stop, receive and send paths as the T8140, T8142 and T6040 GPU mailboxes (the apple,asc-mailbox-v4 registers behind the ASCWrap v6 wrapper offset). Receive and send are interrupt driven, as in the original Asahi ASC mailbox driver, with the threaded GPU receive handler those mailboxes already use:

  • The recv-not-empty line is a threaded, oneshot interrupt (IRQF_ONESHOT | IRQF_NO_AUTOEN | IRQF_NO_SUSPEND). apple_mbox_start() enables it and apple_mbox_stop() disables it.
  • Its handler drains the I2A FIFO until it is empty, one receive budget per pass.
  • A non-atomic sender that finds the A2I FIFO full waits for the send-empty interrupt and its completion.

Two commits:

  • soc: apple: mailbox: receive T6050 GPU mailboxes by interrupt once it fires (41c359d). It enabled the receive interrupt beside the pollers.
  • soc: apple: mailbox: receive T6050 GPU mailboxes by interrupt. It removes the polling for these mailboxes:
    • the rx_poll_until_irq hardware flag;
    • the 25 us receive hrtimer and its kick from apple_mbox_send();
    • the jiffy receive poller and watchdog, and the proof state;
    • the storm guard's extension to these mailboxes;
    • the sleeping poll of a full send FIFO.
      It also drops the KUnit cases for that machinery. The apple_mbox_g17_stall suite keeps its receive budget, send stall and receive utilization cases.

Against integration/aurora:

The ASCWrap v8 mailboxes (T8152) keep their poller, poll-only default, ascwrap_v8_rx_irq parameter and storm guard; changing them is a separate change. No exported interface changes. apple_mbox_set_rx_util_min() takes effect on these mailboxes again, since their receive runs in the threaded handler. #1739 (the fast poller's arming flag) no longer applies, because the fast poller is gone.

Testing

  • Hardware, M5 Max MacBook Pro (J716c), erikwb.

    • Setup: the first commit (receive interrupt active, pollers still beside it) against the same daily tree without it. An inference engine's decode of Qwen3.8-Flash-Next quantized to 4 bits, after a 1024-token prompt, with greedy sampling, the same engine, Mesa and model on both kernels, two runs each. The full method is in this PR's history.
    • GPU time of the speculative-decode verification pass (median per run): 21.93 / 21.93 ms → 21.02 / 21.04 ms.
    • Speculative decode: 84.2 / 87.1 → 89.9 / 89.7 tok/s. Single-token decode: 62.4 / 63.8 → 66.4 / 67.2 tok/s. Output tokens were identical.
    • Recv interrupt counts with the first commit, before → after the runs: 2302600000.mailbox-recv 962 → 6409, 2302e00000.mailbox-recv 17 → 245 (0 without it, which does not enable the line).
  • Hardware, M5 Pro MacBook Pro (J714s): the default kernel predates soc: apple: mailbox: receive T6050 GPU mailboxes by polling #1601, so it runs this PR's mailbox.c. During a dEQP run, the recv interrupts count:

    • 2302600000.mailbox-recv went 1,322 → 1,341 in 5 s, spread across all 18 CPUs;
    • 2302e00000.mailbox-recv read 34;
    • both send-empty lines stayed at 0.

    A merge-gate run of this branch (gpubench, glmark2 and a synchronization CTS subset, against the default kernel) follows in a comment.

  • CI: the checks on this head.

🤖 Generated with Claude Code

… fires

The T6050 GPU mailboxes receive by polling, because the recv-not-empty
interrupt does not reach the AIC on the 14-inch M5 Max MacBook Pro: a
25 us hrtimer for 1 ms after each send, otherwise a once-per-jiffy
poller, and the interrupt is never enabled. On the 16-inch M5 Max
MacBook Pro the interrupt works, and there polling costs about 5% of
GPU throughput in an inference engine's decode step.

Enable the recv-not-empty interrupt on these mailboxes again, and keep
both pollers running beside it until it has delivered a message. From
then on the fast poller stops and sends no longer arm it: the interrupt
receives, as on the other ASC mailboxes. The line is level triggered
and the GPU handler is oneshot, so a message that arrives while it runs
fires it again. Each mailbox start proves the line again.

The jiffy poller keeps running as a watchdog. Where the interrupt
delivers, the poller finds the FIFO empty. A message it does find was
not delivered by the interrupt (or its thread had not run yet): the
poller hands it on and clears the proof, so that the sends after it arm
the fast poller again until the interrupt delivers a message again. A
line that never fires keeps polling exactly as before. A line that
delivers once and then stops, or fires only spuriously, loses no
message: the watchdog takes one that it misses on a following pass,
normally the next, subject to workqueue scheduling and to its budget of
64 messages per pass.

The ASCWrap v8 storm guard now covers these mailboxes too: a line that
fires 1000 times in a row with nothing to read is disabled at the AIC
and receive falls back to polling.

The hardware flag rx_poll_only becomes rx_poll_until_irq. A full send
FIFO is still waited out by sleeping polls, without the send-empty
line, as before.

Tested with KUnit (arm64, QEMU, suite apple_mbox_g17_stall): new cases
cover proof by a delivered message, the watchdog taking the proof back
when the line is silent with a message queued, the storm guard, and
start and stop. They wait on what the pollers receive, and read state
after cancelling them or under rx_lock. On an M5 Max MacBook Pro (J716c,
whose interrupt fires), an inference engine's decode, two runs each
without and with this commit: GPU time of its verification pass 21.93 /
21.93 -> 21.02 / 21.04 ms, single-token decode 62.4 / 63.8 -> 66.4 /
67.2 tok/s, output identical; the GPU mailboxes' recv interrupts count
in /proc/interrupts, the watchdog took the proof back 64 times in ~100 s
of decode (each time the next message proved the line again), and the
storm guard did not trip.

Developed with Claude Code.
Signed-off-by: Erik Bourget <erik@ebourget.net>
@DjDeveloperr

DjDeveloperr commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Independent review of 41c359da found no blocker. That head is now superseded by the follow-up commit, which makes these mailboxes interrupt driven like the other ASC mailboxes (see the updated description).

What was verified at that head:

  • Scope: only the three T6050 ascwrap-v6 compatibles. T8140 takes none of the changed branches, and the v8 mailboxes see only a log wording change.
  • Delivery: every receive path drained under rx_lock, so each message arrived once and in order.
  • IRQ depth: enable/disable stayed balanced through the storm guard, stop and start.

Notes that still apply after the follow-up commit:

ASC mailboxes are interrupt driven; T6050's GPU mailboxes now use the
same paths as the other ASCWrap v6 GPU mailboxes.

The three T6050 GPU compatibles (apple,t6050-ascwrap-v6,
apple,t6050-x1n1-ascwrap-v6 and apple,t6050-j775d-ascwrap-v6) describe
the ASC v4 mailbox block behind the ASCWrap v6 wrapper offset, as the
T8140, T8142 and T6040 GPU mailboxes do. Their hardware descriptions now
differ from those only in the wrapper windows, and they take the same
start, stop, receive and send paths:

 - the recv-not-empty line is the threaded, oneshot GPU receive
   interrupt (IRQF_ONESHOT | IRQF_NO_AUTOEN | IRQF_NO_SUSPEND), enabled
   by apple_mbox_start() and disabled by apple_mbox_stop();
 - its handler drains the I2A FIFO until it is empty, one receive
   budget per pass;
 - a non-atomic sender that finds the A2I FIFO full waits for the
   send-empty interrupt and its completion.

Remove what received them by polling instead:

 - the rx_poll_until_irq hardware flag (rx_poll_only before);
 - the 25 us receive hrtimer, its kick from apple_mbox_send() and its
   deadline;
 - the jiffy receive poller and its watchdog for these mailboxes, and
   the rx_irq_proven state;
 - the storm guard's extension to them;
 - the sleeping poll of a full send FIFO, which left the send-empty
   interrupt unused.

This undoes, for these mailboxes, the receive and send changes of
commits 43c6ca8 ("soc: apple: mailbox: receive T6050 GPU mailboxes
by polling"), a23755c ("soc: apple: mailbox: poll T6050 GPU
receive quickly while a reply is outstanding") and 41c359d ("soc:
apple: mailbox: receive T6050 GPU mailboxes by interrupt once it
fires").

The T8152 ASCWrap v8 provider keeps its receive poller, its poll-only
default, the ascwrap_v8_rx_irq parameter and its storm guard, unchanged.

Drop the KUnit cases that tested the removed receive polling. The
apple_mbox_g17_stall suite keeps its receive budget, send stall and
receive utilization cases. No exported interface changes.

Signed-off-by: Diljit Singh <hey@djdev.me>
@DjDeveloperr DjDeveloperr changed the title soc: apple: mailbox: receive T6050 GPU mailboxes by interrupt once it fires soc: apple: mailbox: receive T6050 GPU mailboxes by interrupt Nov 13, 2026
@DjDeveloperr

Copy link
Copy Markdown
Contributor

@erikwb, a change of direction from the project owner on this one. Mailboxes are meant to be driven by interrupts. So the T6050 GPU mailboxes should receive like every other ASC mailbox, with no poll fallback.

On top of your commit, I pushed a commit that does that for the three T6050 compatibles, modelled on the original Asahi ASC mailbox driver, with the threaded GPU receive handler that T8140, T8142 and T6040 use:

  • a threaded oneshot receive interrupt, enabled by start and disabled by stop;
  • the handler drains the FIFO;
  • a full send FIFO waits on the send-empty interrupt;
  • no hrtimer, jiffy poller, watchdog, proof state or storm guard for them.

The net effect is that mailbox.c matches what it was before #1601. The ASCWrap v8 (T8152) poller stays for now. I updated the title and description to match, keeping your J716c numbers. Thanks for this: your commit showed the line delivers on J716c, and your decode measurements are the case for the change.

@DjDeveloperr DjDeveloperr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving and merging at 788441e0890e.

M5 Pro (J714s) merge gate, slot hw2.

  • Arms:
    • A = the default kernel (7.3.0-m5-intg6). It predates #1601, so its mailbox.c is byte-identical to this PR's.
    • B = 7.3.0-m5-mbox2: integration/aurora a34d26c2 with this branch, the same config, dtb, Stage 2 and command line as the default.
  • Recv interrupts on both arms (/proc/interrupts):
    • A, 165 s after boot: 2302600000.mailbox-recv 1,323, spread over all 18 CPUs; 2302e00000.mailbox-recv 25.
    • B, 46 s after boot: 811 and 14.
    • Both send-empty lines stayed at 0 on both arms.
  • Health: no GPU timeouts, recoveries or faults, no oops, and the desktop stayed up throughout. That held over boot, gpubench, glmark2, a timeline-semaphore CTS sample and three idle/load cycles.
  • gpubench (the gpubench lane's harness and placement, same Mesa on both arms), A → B, median us per submit:
    • compute.light d1 159.4 → 158.1 (gpu_gap 149.7 → 149.3), d16 14.69 → 14.59, d64 14.63 → 14.42;
    • render.light tri d16 26.14 → 26.19, d64 26.60 → 24.05;
    • sync.signal host 129.5 → 123.9 us;
    • compute.heavy bw and copy.bw within 1%;
    • two mid-depth rows (compute.light d4, render.light tri d2) +7%.
    • The two arms run identical mailbox code, so these differences come from the rest of the integration tree or run-to-run noise, not from this PR.
  • glmark2-es2, off-screen 1080p: 4,364 → 4,471.
  • Sync CTS:
    • On B: dEQP-VK.synchronization2.op.multi_queue.timeline_semaphore.*, 1,855 cases at 16 jobs, all Pass, 0 GPU or mailbox fault lines. My harness cut the run short (a runner-detection race); the cases that ran all passed.
    • On the default kernel, i.e. this PR's receive path: the full 4,269-case list gave 4,266 Pass / 0 Fail / 3 NotSupported in earlier runs (mesa#264's gate).

CI: all 7 checks green on this head, on "Merge 788441e into 6882c13". integration/aurora has not touched the mailbox files since.

@DjDeveloperr
DjDeveloperr merged commit adb5a7e into integration/aurora Nov 13, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants