feat(native): add direct image pixel updates

.changeset/dynamic-img-pixels.md

@@ -0,0 +1,8 @@
+---
+'@gpuix/native': minor
+'@gpuix/react': minor
+---
+
+Add desktop `updateImage(elementId, width, height, bgra)` and `clearImage(elementId)` for existing `<img>` nodes. Pixels are copied from a tightly packed BGRA Uint8Array, rendered through ordinary GPUI img, and released on clear, src change, resize, or destruction. Same-size updates preserve native image identity. Sampling remains linear. Uploads are limited to 4096 pixels per axis (64 MiB) before copying; GPU failures fall back to src and are not retried until a new version is submitted. Upstream atlas search also found [zed-industries/zed#54659](https://github.com/zed-industries/zed/issues/54659) (allocation failures) and [#57516](https://github.com/zed-industries/zed/pull/57516) (tile lifetime); these remain upstream work, not fork changes in this PR.
+
+Bump GPUI to merged [remorses/zed#4](https://github.com/remorses/zed/pull/4) (`81c99f816b4a5f69d3c014774068034c24d1d7af`) for `RenderImage::from_bgra` and `Window::update_image`. Upstream searches for `from_bgra` and `update_image` found no matching issues/PRs; upstream nearest-neighbor [zed-industries/zed#57393](https://github.com/zed-industries/zed/pull/57393) remains open. Ordinary GPUI Img has no sampling builder at this revision, so that feature is deferred rather than reimplementing its painting in GPUIX.

README.md

@@ -146,6 +146,7 @@ gpuix completions install
 | **timeline** | `bun --hot timeline.tsx` | A video-editor timeline: clip dragging, edge trimming with snapping, playhead scrubbing, marquee selection, zoom under the pointer, and a two-axis pan with a frozen ruler and track column |
 | **mail** | `bun --hot mail.tsx` | A Superhuman-style mail client: three panes, thread list, and a Framer newsletter |
 | **native-text** | `bun --hot native-text.tsx` | The three native text components with a tab switcher |
+| **dynamic-image** | `bun --hot dynamic-image.tsx` | Animated BGRA pixels in one ordinary `<img>`, without React commits per frame |
 | **counter** | `bun --hot counter.tsx` | The smallest possible app: state, events, hover |
 | **diff** | `bun --hot diff.tsx` | A diff viewer composed from `<div>` and `<text>` in JS, for comparison |
 | **web** | `bun run web` from the repository root | The ChatGPT example rendered in a browser canvas with WebGPU |
@@ -2107,6 +2108,61 @@ child. Put the radius on the image.
 />
 ```
 
+### Dynamic pixels (desktop)
+
+Use the renderer from `useGpuixRequired()` (or `useGpuix().renderer`) to update
+an existing `<img>` without a React render or a JSON/base64 pixel payload:
+
+```tsx
+const renderer = useGpuixRequired()
+const image = useRef<{ id: number }>(null)
+
+useLayoutEffect(() => {
+  const bgra = new Uint8Array([0, 0, 255, 255]) // one opaque red pixel
+  renderer.updateImage!(image.current!.id, 1, 1, bgra)
+}, [renderer])
+
+return <img ref={image} style={{ width: 128, height: 128 }} />
+```
+
+- `updateImage(elementId, width, height, bgra: Uint8Array): void` copies the
+  supplied view before returning. Reuse or modify it afterwards; native retains
+  no JS pointer. `Buffer` and offset `Uint8Array` views also work.
+- Pixels are **8-bit BGRA, straight (unpremultiplied) alpha**, matching GPUI's
+  decoded raster images. Rows run top to bottom, pixels left to right, with
+  exactly `width * 4` bytes per row and no padding. Alpha `0` is transparent;
+  alpha `255` is opaque. There is no stride, partial-update, or format option.
+- Width and height must be positive integers no larger than **4096** each
+  (at most **64 MiB** per image). This ingress limit is checked before copying
+  pixels; it is not a guarantee that GPU memory is available. The view length
+  must equal **exactly `width * height * 4`**. Invalid dimensions, overflow,
+  lengths, or IDs throw without replacing the current pixels. An ID must be a
+  non-negative safe integer identifying a live `<img>` in this renderer.
+- Call after React commits, for example from `useLayoutEffect`, `useEffect`, or
+  an event handler. Earlier React mutations are committed before effects run.
+  Updates schedule the next native frame; multiple calls before that frame
+  keep only the latest pixels. The test renderer requires `flush()` to paint.
+- Same-size updates keep native image identity and use GPUI's atlas update.
+  A size change creates a new identity and retires the old atlas entry. GPUI
+  still performs its normal repaint; no cached-frame presentation is exposed.
+- Pixels override `src`. An actual subsequent `src` change (including removing
+  it) clears the override. Rerendering with the same `src` preserves the pixels.
+  `clearImage(elementId): void` returns to the current `src`, or the usual
+  placeholder if absent. Clearing a live img twice is safe; a destroyed ID throws.
+- Clear, src changes, resize, and unmount release obsolete atlas entries on the
+  next native frame. The window owns uploaded images; nothing is shared across
+  renderers. Unmount needs no manual cleanup beyond stopping your producer.
+- This is ordinary `<img>` rendering: `objectFit`, intrinsic sizing, styles,
+  corner clipping, accessibility, and events still apply. Sampling is currently
+  **linear only**; nearest-neighbor awaits a GPUI Img builder API.
+- A GPU upload failure is logged once for that submitted version and falls
+  back to `src` (or an empty image). Other images continue uploading. Unrelated
+  frames do not retry it; a new `updateImage` call allows another attempt.
+- These two methods are desktop-only, and absent on the browser renderer.
+
+See [`examples/dynamic-image.tsx`](./examples/dynamic-image.tsx) for a small
+animated color field that reuses one buffer and never commits React per frame.
+
 ### `<svg>`
 
 `<svg>` uses GPUI's **monochrome icon renderer**. Raw `source` works on desktop

examples/dynamic-image.test.tsx

@@ -0,0 +1,30 @@
+import fs from 'node:fs'
+import React from 'react'
+import { afterEach, expect, it, vi } from 'vitest'
+import { createTestRoot, hasNativeTestRenderer } from '@gpuix/react/testing'
+import { DynamicImage } from './dynamic-image'
+
+const itNative = hasNativeTestRenderer ? it : it.skip
+
+afterEach(() => vi.useRealTimers())
+
+itNative('animates one img without React commits and stops on unmount', () => {
+  vi.useFakeTimers()
+  const root = createTestRoot({ width: 304, height: 304 })
+  const uploads = vi.spyOn(root.renderer, 'updateImage')
+  const commits = vi.spyOn(root.renderer, 'applyBatch')
+  root.render(<DynamicImage />)
+  const committed = commits.mock.calls.length
+  fs.mkdirSync('screenshots', { recursive: true })
+  root.renderer.captureScreenshot('screenshots/dynamic-image-first.png')
+  const first = fs.readFileSync('screenshots/dynamic-image-first.png')
+  vi.advanceTimersByTime(200)
+  root.renderer.captureScreenshot('screenshots/dynamic-image-animated.png')
+  expect(fs.readFileSync('screenshots/dynamic-image-animated.png')).not.toEqual(first)
+  expect(root.renderer.findByType('img')).toHaveLength(1)
+  expect(uploads).toHaveBeenCalledTimes(6)
+  expect(commits).toHaveBeenCalledTimes(committed)
+  root.unmount()
+  vi.advanceTimersByTime(200)
+  expect(uploads).toHaveBeenCalledTimes(6)
+})

examples/dynamic-image.tsx

@@ -0,0 +1,44 @@
+import { useEffect, useRef } from 'react'
+import { render, useGpuixRequired } from '@gpuix/react'
+
+export function DynamicImage() {
+  const renderer = useGpuixRequired()
+  const image = useRef<{ id: number }>(null)
+
+  useEffect(() => {
+    const id = image.current!.id
+    const pixels = new Uint8Array(32 * 32 * 4)
+    let frame = 0
+    const paint = () => {
+      for (let y = 0; y < 32; y++) {
+        for (let x = 0; x < 32; x++) {
+          const i = (y * 32 + x) * 4
+          pixels[i] = (x * 8 + frame) % 256
+          pixels[i + 1] = (y * 8 + frame) % 256
+          pixels[i + 2] = 180
+          pixels[i + 3] = 255
+        }
+      }
+      renderer.updateImage!(id, 32, 32, pixels)
+      frame += 4
+    }
+    paint()
+    const timer = setInterval(paint, 40)
+    // Unmount releases the image; no explicit clearImage is needed here.
+    return () => clearInterval(timer)
+  }, [renderer])
+
+  return (
+    <div style={{ padding: 24, backgroundColor: '#202030', height: '100%' }}>
+      <img ref={image} testId="pixels" alt="Animated color field"
+        style={{ width: 256, height: 256, borderRadius: 24 }} />
+    </div>
+  )
+}
+
+if (import.meta.main) {
+  render(<DynamicImage />, {
+    title: 'Dynamic image', width: 304, height: 304,
+    focus: process.env.GPUIX_BACKGROUND !== '1',
+  })
+}

packages/native/Cargo.toml

@@ -92,6 +92,7 @@ napi-build = "2"
 # shipped addon: the wire format is still JSON until the bench says otherwise.
 [dev-dependencies]
 rmp-serde = "1.3"
+ctor = "0.6"
 
 [[example]]
 name = "hello"

packages/native/index.d.ts

@@ -26,6 +26,10 @@ export declare class GpuixRenderer {
    * Acquires the tree mutex ONCE for the entire batch.
    */
   applyBatch(json: string): Array<number>
+  /** Copy tightly packed BGRA pixels into an existing img. Paints on the next frame. */
+  updateImage(elementId: number, width: number, height: number, bgra: Uint8Array): void
+  /** Release an img's pixel override on the next frame and return to its src. */
+  clearImage(elementId: number): void
   /** Pump the native event loop. Returns false after the last window closes. */
   tick(): boolean
   isInitialized(): boolean
@@ -149,6 +153,10 @@ export declare class GpuixRenderer {
  */
 export declare class TestGpuixRenderer {
   constructor(width?: number | undefined | null, height?: number | undefined | null)
+  /** Copy tightly packed BGRA pixels into an existing img. Call flush to paint. */
+  updateImage(elementId: number, width: number, height: number, bgra: Uint8Array): void
+  /** Release an img's pixel override on the next flush and return to its src. */
+  clearImage(elementId: number): void
   /**
    * How many elements the retained tree holds, reachable from the root or
    * not. `getTreeJson` walks from the root, so it cannot see a node that was

packages/native/src/custom_elements/img.rs

@@ -184,12 +184,16 @@ impl CustomElement for ImgElement {
     ) -> gpui::AnyElement {
         use gpui::prelude::*;
 
-        let el = match &self.source {
-            ImgSource::Path(path) => gpui::img(path.clone()),
-            ImgSource::Uri(uri) => gpui::img(uri.clone()),
-            ImgSource::Data(image) => gpui::img(image.clone()),
-            ImgSource::Empty => return img_fallback(&ctx, &self.alt, "img: no src"),
-            ImgSource::Invalid => return img_fallback(&ctx, &self.alt, "img: load failed"),
+        let el = if let Some(image) = &ctx.image {
+            gpui::img(image.clone())
+        } else {
+            match &self.source {
+                ImgSource::Path(path) => gpui::img(path.clone()),
+                ImgSource::Uri(uri) => gpui::img(uri.clone()),
+                ImgSource::Data(image) => gpui::img(image.clone()),
+                ImgSource::Empty => return img_fallback(&ctx, &self.alt, "img: no src"),
+                ImgSource::Invalid => return img_fallback(&ctx, &self.alt, "img: load failed"),
+            }
         };
         // The id is what makes gpui's `ImgState` persist. Without it `Img` has no
         // `GlobalElementId`, so the animated-GIF frame index and the delayed

packages/native/src/custom_elements/mod.rs

@@ -54,6 +54,8 @@ pub struct CustomRenderContext<'a> {
     pub highlight_set: Option<std::sync::Arc<crate::text::HighlightContext>>,
     /// Retained custom props, including `role` and `aria-*`.
     pub props: &'a HashMap<String, serde_json::Value>,
+    /// Direct pixel override for an img, never serialized through custom props.
+    pub image: Option<std::sync::Arc<gpui::RenderImage>>,
 }
 
 impl CustomRenderContext<'_> {