Skip to content

drm/asahi: g15: renumber the TA/FS and compute stamps before they wrap - #287

Merged
chriskearney merged 4 commits into
omacom:integration/aurorafrom
samucarre:g15-stamp-renumber
Oct 30, 2026
Merged

chriskearney merged 4 commits into
omacom:integration/aurorafrom
samucarre:g15-stamp-renumber

Conversation

@samucarre

Copy link
Copy Markdown

Problem. The G15 stamps step by 0x100 per job from fixed seeds, so they wrap after a fixed count since the driver loaded: the TA stamp after 0x860000 renders (about six hours of desktop), the compute stamp after 0x3f0000 computes (4,128,768: about 390 image generations with ggml-vulkan at one node per submission). On a J516C, with the stamp_seed() from before 7ccdfe8 (which skipped zero), the TA stamp wrapped at render 8,781,824 and the firmware stopped with DAG: Non Sequential Stamp Updates seen ... stampValue 0x100; the GPU was lost until reboot. 7ccdfe8 now wraps the stamps through zero; I have not tried whether the firmware takes that wrap.

Change. Renumber the stamps from their seeds while the GPU is idle, shortly before they would wrap, so they never reach the wrap whatever the firmware does there. Four commits:

  1. TA/FS: at the first publication with nothing in flight within g15_stamp_reseed_margin renders of the TA wrap, write the seeds to the user and firmware stamps, reset the engine targets and move an offset, modulo the 2^24 period of stamp_seed(), so the next render is ordinal 1, as after boot.
  2. Inside that margin nothing is published behind a flight, so a client that keeps the GPU busy cannot hold the renumbering off until the wrap. A non-zero margin is clamped to at least one prefix and at most half the stamp range.
  3. The same for the compute stamp, with g15_compute_reseed_margin: the stamps go back to what ComputeGraph::init() writes with no stamp base (user COMPUTE_SEED, firmware 0).
  4. A stamps: line in asahi-g15/stats: computes and renders left before each wrap, and both margins.

g15_stamp_base and g15_compute_stamp_base (testing only) start the counts near the wraps, so this can be tested in seconds. Independent of #261: it applies to integration/aurora as it is and merges cleanly with it and with #258.

Enabling it. Every new parameter defaults to 0, and with both margins at 0 nothing changes: no offset, no hold, the same stamps as now. I run both margins at 100000:

options asahi g15_stamp_reseed_margin=100000 g15_compute_reseed_margin=100000

(or asahi.g15_stamp_reseed_margin=100000 asahi.g15_compute_reseed_margin=100000 on the kernel command line). They are read when the module loads; asahi-g15/stats shows them (render_margin, compute_margin) with what is left, and the kernel logs G15: TA/FS stamps renumbered ... or G15: compute stamp renumbered ... when it happens, with these margins about once every 8.7M renders or 4.0M computes.

Checked on a MacBook Pro 16" M3 Max (J516C, 64 GB), always on top of local patches, among them HDMI on dcpext0 and J516C panel brightness:

Not checked. Whether the firmware takes the wrap through zero: with a margin of 0 and a stamp base near the wrap this series makes it a test of seconds, but a stop costs a reboot and I have not run it. Other boards; this branch alone on hardware.

Model of the TA targets
# TA stamp targets with the renumbering of this series, over 26M renders with a margin of 100000,
# renumbering at the first idle publication inside the margin (prefixes of 8 renders).
TA, M32 = 0x7a000000, 1 << 32
RENDER_WRAP = 0xffffff - TA // 0x100 + 1  # 0x860000


def stamp_seed(seed, n):  # as in encode.rs: seed + (n as u32) * 0x100, wrapping
    return (seed + (n % M32) * 0x100) % M32


def run(period, renders=26_000_000, margin=100_000, prefix=8):
    draw, base, prev, bad, renumbered = 0, 0, TA, 0, []
    while draw < renders:
        if (draw + base) % period + margin >= RENDER_WRAP:
            base = (period - draw % period) % period  # renumber_render_stamps()
            prev = TA  # the seed written to the firmware stamp
            renumbered.append(draw)
        for _ in range(prefix):
            draw += 1
            t = stamp_seed(TA, draw + base)
            if (t - prev) % M32 != 0x100:
                bad += 1
                print(f"  render {draw}: target {t:#010x} after {prev:#010x}")
            prev = t
    print(f"period {period:#x}: renumbered at {renumbered}, targets out of sequence: {bad}")


run(1 << 24)    # this series
run(0xffffff)   # the period of the earlier stamp_seed()
period 0x1000000: renumbered at [8681824, 17363648], targets out of sequence: 0
  render 8681825: target 0x7a000000 after 0x7a000000
  render 17363649: target 0x79ffff00 after 0x7a000000
period 0xffffff: renumbered at [8681824, 17363648], targets out of sequence: 2

🤖 Generated with Claude Code

Samuel Carreño added 4 commits October 30, 2026 00:59
The TA stamp of render ordinal n is stamp_seed(TA_SEED, n): it steps by
0x100 and wraps after 0x860000 renders, about six hours of desktop. With
the stamp_seed() from before commit 7ccdfe8 ("drm/asahi: wrap G15
stamps through zero"), which skipped zero and went from 0xffffff00 back
to 0x100 there, the firmware stopped at render 8781824 with "DAG: Non
Sequential Stamp Updates seen ... stampValue 0x100" and the GPU was lost
until reboot. Since that commit the stamps wrap through zero; whether
the firmware takes that wrap has not been tried.

Add an offset to the render ordinal used for the TA/FS stamps and, when
g15_stamp_reseed_margin is set, renumber both stamps from their seeds at
the first publication with nothing in flight inside that margin: write
the seeds to the user and firmware stamps, reset the engine targets and
move the offset so the next render counts as ordinal 1, as after boot.
The offset is taken modulo the 2^24 ordinals after which stamp_seed()
repeats (STAMP_PERIOD). The stamps then never reach the wrap.
g15_stamp_base starts the count near the wrap, to test this in seconds
instead of hours.

Both parameters default to 0, which keeps the stamps exactly as before.

Tested on a J516C, together with the next commit, on integration/aurora
with the earlier stamp_seed() and the offset taken modulo its period of
0xffffff ordinals: from g15_stamp_base 2000 renders before the wrap with
a margin of 1000, the stamps were renumbered at render 1000 and 3003
renders retired; in desktop use, one boot renumbered at renders 8681824
and 17363648. On this base, modulo 0xffffff would give the render after
the first renumbering ordinal 0, whose targets the seeds already meet,
and the one after the second ordinal -1, a TA target below the seed.

Generated-by: LLM
Signed-off-by: Samuel Carreño <soyminero11yt@gmail.com>
Renumbering only happens at a publication with nothing in flight. A
client that keeps the GPU busy (no vsync) can keep a flight in the air
through the whole margin, and the TA stamp would then wrap as before.
Inside the margin, publish nothing behind a flight, so the flights drain
and the next publication renumbers while idle (once every ~8.7M renders
with a margin of 100000).

Also clamp a non-zero margin to at least one prefix of renders, so that
the last prefix published outside it cannot cross the wrap, and to at
most half the stamp range, so that it cannot renumber on every
publication.

Found while reviewing the previous commit, with a model of prefixes of
1-8 renders: a fully overlapped client wrapped 199/199 times without
this and 0/199 with it. On a J516C, with the earlier stamp_seed() (see
the previous commit), from g15_stamp_base 500000 renders before the
wrap with a margin of 100000 and the GPU kept busy (286875 publications
behind a flight), the stamps were renumbered at render 400000 and
rendering went on past the wrap point to render 526113.

Generated-by: LLM
Signed-off-by: Samuel Carreño <soyminero11yt@gmail.com>
The compute stamp of ordinal n is stamp_seed(COMPUTE_SEED, n): it steps
by 0x100 from 0xc1000000 and, after 0x3f0000 computes (4128768: about
390 FLUX.3 1024x1024 images at one node per submission), goes from
0xffffff00 back to 0. Whether the firmware takes that wrap has not been
tried either; the TA/FS renumbering of the previous commits does not
cover it.

Renumber the compute stamp the same way. Add an offset to the compute
ordinal used for the stamp and, when g15_compute_reseed_margin is set,
at the first publication with nothing in flight inside that margin
write the compute stamps back as ComputeGraph::init() leaves them with
no stamp base (user COMPUTE_SEED, firmware 0), reset the compute target
and move the offset so the next compute counts as ordinal 1. Inside the
margin nothing is published behind a flight, as for TA/FS, so the
flights drain; the margin is at least one prefix of computes
(COMPUTE_SLOTS) and at most half the stamp range. The ordinal itself,
which the compute image also carries, keeps counting as the render draw
does.

g15_compute_stamp_base starts the count near the wrap, to test this in
seconds instead of hours. Both parameters default to 0, which keeps the
stamp exactly as before.

On a J516C, with g15_compute_stamp_base=4023768 (105000 computes before
the wrap) and g15_compute_reseed_margin=100000, the stamp was renumbered
at compute 5000, the GPU kept running computes up to 159899, past the
point where the stamp would have wrapped, and a FLUX.3 1024x1024 image
generated fine afterwards.

Generated-by: LLM
Signed-off-by: Samuel Carreño <soyminero11yt@gmail.com>
Add a "stamps:" line to asahi-g15/stats with the computes and renders
the current numbering has left before the compute and TA stamps wrap,
and both renumbering margins (0: off). A script that limits compute
work before the wrap can then tell a kernel that renumbers the compute
stamp from one that does not, and count from the stamp rather than
from boot (g15_compute_stamp_base moves the start).

Generated-by: LLM
Signed-off-by: Samuel Carreño <soyminero11yt@gmail.com>

@chriskearney chriskearney left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed all six changed files at exact head c3fad65647f0e46223b91fb7a562d2f1a7c3e945, the drain/retirement and failure paths, and the independent exact-method host controls (26M renders and 10M computes, two renumberings each, default/boundary/write-failure cases and five caught semantic mutants). No blocking source finding for landing this as a default-off experimental facility: all four new parameters default to zero, preserving current stamp generation and publication policy.

This approval does not qualify enabled firmware reseeding or a new board/firmware cohort. The current modulo-2^24 TA/FS path has not reached the wrap boundary on hardware; older TA/FS and compute observations are on different/local stacks. Before calling enabled behavior qualified or making it default-on, run a bounded near-wrap qualification on an already-supported/admitted G15 stack with the existing stop/restore route. A wrap-to-zero crash control or new Max/firmware admission is not required.

Current required CI and full merge-group CI remain required. The C compile job skips Rust; the actual queue-tree package build must retain enabled RUST/DRM_ASAHI/G15 config and successful aarch64 Asahi module build evidence. Normal full packaging can provide that native build; an earlier package run is evidence of the mechanism, not a build of this head.

@chriskearney chriskearney reopened this Oct 30, 2026
@chriskearney chriskearney reopened this Oct 30, 2026
@chriskearney
chriskearney added this pull request to the merge queue Oct 30, 2026
Merged via the queue into omacom:integration/aurora with commit 31748f0 Oct 30, 2026
15 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants