Scene damage and adaptive CPU rendering

[huacnlee]

Summary

Adds scene damage, adaptive CPU rendering on Linux, and partial GPU redraws on macOS and Windows to reduce rendering work for small changes:

1. Scene damage (crates/gpui/src/fast/damage.rs, all platforms): each finished scene carries Scene::damage, the device-pixel rectangles where it can draw different pixels than the previous scene. The damage is exact. It diffs each draw-order group: byte-equal groups are skipped; otherwise primitives are matched by value and checked for relative order. It also covers primitive extents (shadow blur, transformed sprites, paths), dirty scroll-layer tiles, and path batches that split or join. Nearby rectangles merge (8 px). Past 16 rectangles the cheapest pair merges, and damage over half the window becomes the whole window.
2. Adaptive CPU rendering (crates/gpui_wgpu/src/fast/{cpu,adaptive}/, Linux wgpu): a CPU rasterizer reproduces the WGSL shaders and the GPU's 8-bit blending, and is checked pixel by pixel against WgpuHeadlessRenderer. How a fragment becomes an 8-bit level depends on the GPU vendor (NVIDIA truncates to 12 bits, Intel to 16), and the rasterizer handles both. The atlas keeps a CPU mirror. A policy draws small damage on the CPU and sends large changes, expensive bursts, surfaces and the first frame to the GPU. Long bursts of small changes, such as a spinner, catch the canvas up once and then stay on the CPU.
   • Presenters (crates/gpui_linux/src/fast/cpu_present/): Wayland wl_shm on a synchronized subsurface, because the explicit-sync WSI surface forbids shm. While the subsurface shows, wp_alpha_modifier_v1 makes the window surface transparent, so translucent windows show no ghosting. Without that protocol, CPU frames go through blit. X11 uses PutImage, with MIT-SHM for large rectangles.
   • There is also a blit mode: upload the region and draw one full-screen triangle.
   • Default: native, with blit where no presenter is installed.
3. Partial redraws (crates/gpui_apple/src/fast/partial.rs, crates/gpui_windows/src/fast/partial.rs, macOS and Windows): the GPU renderers keep the last frame in a window-sized canvas and redraw only the damage rectangles into it, scissored, then copy the canvas into the drawable / back buffer. Windows presents with Present1 and the damage as dirty rectangles. A frame with nothing to draw (same scene, or empty damage) is neither drawn nor presented. GPUI_PARTIAL_REDRAW=0 turns it off.

Upstream files get hooks only. script/check-upstream reports 0 violations. The API differences are listed in docs/upstream-sync.md. The reference doc is docs/adaptive-rendering.md and the design is in docs/superpowers/specs/2026-10-08-damage-and-adaptive-cpu-design.md.

Controls: GPUI_CPU_RENDER=0|always, GPUI_CPU_PRESENT=native|blit, GPUI_PARTIAL_REDRAW=0, GPUI_RENDER_STATS=1, GPUI_CPU_VERIFY=1.

What this means

On Linux, at the same or lower CPU cost, the application stops using the GPU for small updates. In every small-update scenario the process CPU stays flat or drops: caret 0.4% → 0.2%, quotes 1.9% → 1.4%, spinner 8.3% → 4.3%. The application's GPU time drops to 0.

• Linux CPU does not go up in these small-update measurements. A CPU frame of a few hundred to a few thousand pixels takes tens of µs, which is less than recording and submitting a whole-window GPU frame.
• What it buys: the GPU stays idle while the app is nearly still. A trading app spends most of its day in that state: quotes ticking, clocks, carets, hovers. Keeping the GPU idle mainly saves power and heat. The gain is largest on laptops (battery, fans) and discrete GPUs, where each wake-up jumps to a high clock state, and with many windows or apps open.
• What it does not do:
  • It does not make the app faster. Frame rate, latency and scrolling are unchanged, and large changes still go to the GPU.
  • The compositor still composites the window, so its GPU time drops only for continuous animations (spinner 164 → 101 ms/s).
• To be honest about magnitude: on this desktop with an integrated GPU the absolute saving is small. A few tens of ms/s of iGPU time are saved, while the idle desktop already keeps the iGPU ~35% busy. Power has not been measured yet: RAPL needs root, and the sampling code is parked for later. So "saves power" is still an inference. The convincing test is a laptop with a discrete GPU, on battery, running the real Longbridge UI.

Performance

Measured with script/measure-adaptive --modes gpu,adaptive,blit --reps 2, 20 s per run. Hardware: Intel Arrow Lake iGPU, 4K at scale 1.5, Hyprland. The GPU columns are the GPU time of the application's and the compositor's own DRM clients, in ms per second.

Wayland	Process CPU gpu → adaptive	App GPU gpu → adaptive → blit	Compositor GPU gpu → adaptive → blit
CaretBlink	0.4% → 0.2%	24.6 → 0 → 1.6	45.8 → 46.7 → 44.8
Hover	1.3% → 1.1%	52.1 → 0 → 3.7	36.8 → 36.4 → 39.6
Quotes	1.9% → 1.4%	64.7 → 0 → 5.1	32.0 → 35.4 → 31.8
Spinner	8.3% → 4.3%	356 → 0 → 38.7	164 → 101 → 162
Scroll (stays on GPU)	16.1% → 16.4%	377 → 368 → 331	173 → 176 → 173

X11 (XWayland) shows the same pattern. For example, Spinner: 8.3% → 4.3% CPU, 320 → 0 ms/s app GPU, and 237 → 138 ms/s compositor GPU. Full tables are in the doc.

Native beats blit on every measure. Blit keeps 5–10% of the GPU time. It also makes the compositor recomposite the whole window, while native recomposites only the damage.

Re-measured after the review fixes (same command, Wayland and X11): every mode keeps 60 fps and the gpu vs adaptive picture is unchanged. Wayland: caret 0.4% → 0.3%, hover 1.4% → 1.1%, quotes 1.9% → 1.6%, spinner 9.1% → 5.0% CPU, and app GPU time 19–358 ms/s → 0 for every small update. X11: spinner 6.9% → 3.5%. The desktop was not in the same state as in the morning run (the idle compositor drew 110 ms/s instead of 28 on Wayland, and nothing on X11, likely a blanked screen), so compare modes within a run, not absolute numbers across runs.

Per-frame CPU cost of this PR (headless gpui_perf --headless --frames 300, retention on, 1440×900; the PR against its merge base edbd764, median of 3 alternating runs, run-to-run spread mostly 1–4%). Headless frames are not drawn, so this shows the cost only, not the rendering savings. Almost all of it is the damage diff:

Scenarios	Frame CPU, base → PR	Damage diff
Forms, settings, scroll lists	+1–6% (mostly within noise)	0.001–0.03 ms
Workspace (trading UI)	+5–7%, ~2.2 → 2.4 ms	0.09–0.10 ms
table-ticks-many	+7.8%, 7.31 → 7.88 ms	0.11 ms
list-select	+13.6%, 0.82 → 0.93 ms	0.095 ms
list-variable-scroll (worst)	+31.7%, 0.275 → 0.362 ms	0.079 ms (~2400 changed primitives a frame)

At most ~0.1 ms a frame, about 0.6% of a core at 60 fps. In real windows it is more than paid back on small updates (the CPU column above drops), but frames that are redrawn whole anyway (scrolling, GPUI_PARTIAL_REDRAW=0, headless) pay it for nothing. A follow-up can stop diffing once the damage covers most of the window: 60–91% of the frames in workspace-scroll and list-select end up whole.

macOS partial redraws, re-measured after the empty-frame fix (2026-10-08, commit 907863e, MacBook Pro M3 / 24 GB, 1512×836 pt window at 2×). Same release build, gpui_perf --idle --duration 15 --warmup 2, GPUI_RENDER_STATS=1, three rounds alternating whole and partial redraws (full/partial, partial/full, full/partial). Whole frames (GPUI_PARTIAL_REDRAW=0) → partial (GPUI_PARTIAL_REDRAW=1); each value is the median of three runs. Process CPU is a percentage of one core. GPU time is the change in each process's ioreg accumulatedGPUTime, divided by the measured duration, excluding startup and warmup.

Scenario	Process CPU	App GPU ms/s	WindowServer GPU ms/s	Update frames/s
CaretBlink	1.84% → 1.86%	8.20 → 5.53 (-32.5%)	77.16 → 76.08	2.00 → 2.00
Clock	1.46% → 1.40%	4.20 → 2.51 (-40.2%)	78.03 → 70.83	1.00 → 1.00
Hover	3.62% → 3.69%	15.71 → 10.46 (-33.4%)	75.07 → 77.40	4.00 → 4.00
Quotes	5.23% → 5.21%	20.19 → 14.59 (-27.7%)	90.07 → 90.73	5.00 → 5.00
Spinner	34.89% → 42.67%	369.49 → 245.30 (-33.6%)	282.16 → 348.85	120.00 → 120.00
Scroll	40.45% → 45.57%	402.53 → 304.08 (-24.5%)	244.07 → 302.68	120.00 → 119.98
Idle	0.95% → 1.02%	0.00 → 0.00	71.60 → 69.38	0.00 → 0.00

The application's GPU time drops 24–40%, with update frequency unchanged. Small-update CPU is approximately flat, but continuous animations have a measurable cost: Spinner's CPU median rises from 34.89% to 42.67%, and Scroll's from 40.45% to 45.57%. CPU varies across runs (Spinner: whole 34.18–43.14%, partial 39.58–43.34%; Scroll: whole 40.30–44.14%, partial 41.90–46.39%), so these results do not support a claim of no CPU regression on macOS.

WindowServer GPU time also rises for both continuous animations in every paired round. Its counters include the whole desktop, with about 70 ms/s of background activity even in Idle, so they cannot all be attributed to this application. Lower application GPU time is not evidence of lower total power consumption here; power and battery life were not measured. Apple's partial path still loads/stores the canvas and copies the whole canvas to the drawable; Core Animation composites each presented drawable whole.

Scroll includes expected whole-frame fallbacks after glyph atlas writes. Across each complete partial-mode run (including warmup and scenario transitions), there were 43 whole frames: one initial canvas frame and 42 atlas-write fallbacks. No frame was skipped in these measurements: Quotes had visible changes at this wider viewport, while Idle already requested no ongoing draws in either mode. The empty-frame fix's correctness is covered by the Metal pixel tests, but its performance benefit in a narrower window has not been measured. These M3 results replace the earlier M4 table; hardware and viewport differ, so absolute values should not be compared across the two runs. Windows has not been measured on real hardware.

CPU rasterizer (this machine): 1920×1080 whole frame in 7.7 ms on 1 thread and 2.3 ms on 8 threads. A 200×40 text region takes 55 µs and a caret 14 µs.

Correctness checks

• GPUI_CPU_VERIFY=1 redraws each CPU frame's scene whole and compares it pixel by pixel with the incrementally drawn canvas, so it catches any missed damage, stale region or atlas write. It was run on every gpui_perf --idle scenario: tiled and 1600×1000 windows, Wayland and X11, native and blit, ~2600 CPU frames. Result: verify_bad=0 everywhere.
  • It found one real bug, now fixed: the raster panicked on path batches beside a narrow region.
  • It also settles the old Quotes TODO: frames with empty damage are genuinely unchanged. In narrower windows the price columns are clipped.
• CPU vs GPU frames, real window: the Idle workspace was drawn whole on the GPU and on the CPU (native and blit) and screenshotted in place. Out of 3.5 M pixels, 42–64 differ by more than 3 levels, on Wayland and on X11. The differences are the clock (a different second), path antialiasing edges (≤5 levels), and twice a single path MSAA sample.
• Opacity ghosting: at window opacity 0.5, native and blit frames now differ in 36 pixels. Without wp_alpha_modifier_v1 they differed in 54,095.
• Partial redraws: CI draws scenes partially and whole on the runners' real Metal and Direct3D devices and compares them byte for byte (paths, glyphs, images, shadows, transparent windows, several rectangles, overlapping damage, consecutive partial frames, skipped frames).

Review fixes

A five-way review (damage, adaptive policy, CPU raster, Linux presenters, macOS/Windows) found no missed damage in the core diff. Fixed:

• X11: a CPU frame's PutImage could land before a queued GPU PresentPixmap and be overwritten at vblank. The presenter now selects PresentCompleteNotify and re-uploads what CPU frames drew since the GPU's present. Expose rectangles are uploaded too (no compositing manager: uncovered areas stayed black).
• Device recovery dropped the platform presenter (CPU frames fell back to blit for good); it is now carried over.
• Atlas writes taken by a frame that was not presented were lost; they are kept until a frame presents them.
• can_draw now checks paths and surfaces inside the scroll-layer tiles it draws.
• The fragment-bits flag is no longer process-global (flaky tests when two vendors' settings ran in parallel); it is passed through the draw.
• Windows: a background appearance change now draws the frame whole; path batches are rasterized once per frame, not once per damage rectangle.

Tests

• gpui: damage tests and property tests over random scenes.
• gpui_wgpu: CPU vs GPU pixel comparisons on NVIDIA and Intel, the policy (including catch-up), atlas writes across unpresented frames, device recovery, blit, and verify.
• gpui_linux: presenters, including the X11 re-upload bookkeeping.
• gpui_apple / gpui_windows: partial redraw pixel tests, run in CI on macOS and Windows. Local M3 validation at 907863e: cargo test -p gpui_apple --lib 23 passed, including 8 partial-redraw tests on the real Metal device; cargo test -p gpui --features test-support --lib fast::tests::damage 15 passed; script/check-upstream 0 violations. All six performance runs completed successfully with no panic, ERROR or WARN logs. Manual occlusion, window resizing and cross-display checks were not performed.
• Clippy (-D warnings) and rustfmt are clean; the Windows and macOS crates are also cross-checked from Linux.
• wasm32, x11-only and wayland-only builds succeed.

Notes

• The vendor's fragment bits are a const generic chosen once per span, so the raster keeps a constant-shift path for both (whole 1080p frame 7.6–7.9 ms on 1 thread, 2.1–2.3 ms on 8).
• On Intel there are two rounding edges, both allowed for in the tests there: a rotated sprite's filtering can differ by 3 levels, and a checkerboard cell edge that falls exactly on a pixel center can move one pixel. Neither shows within a frame, because the canvas is redrawn whole when the frame switches between CPU and GPU.
• Not tested on AMD: there the fragment conversion is assumed to be 16-bit.

🤖 Generated with Claude Code