Skip to content

Add optional NativeDawn plugin for WebGPU on Windows - #1781

Open
CedricGuillemet wants to merge 56 commits into
BabylonJS:masterfrom
CedricGuillemet:nativeDawn
Open

CedricGuillemet wants to merge 56 commits into
BabylonJS:masterfrom
CedricGuillemet:nativeDawn

Conversation

@CedricGuillemet

@CedricGuillemet CedricGuillemet commented Jul 16, 2026 •

Copy link
Copy Markdown
Collaborator

Babylon Native renders WebGPU through bgfx, which cannot expose the full WebGPU API that Babylon.js's WebGPUEngine expects. This adds an optional NativeDawn plugin that implements the JavaScript WebGPU API directly on top of Dawn, so WebGPUEngine can run on Babylon Native on Windows. The bgfx path is unchanged and remains the default.

The goal is for Dawn to behave as an ordinary plugin: it registers its own WebGPU surface and uses the shared polyfills (canvas, URL, XMLHttpRequest, window) rather than patching the runtime. Playground gains a WebGPU configuration that runs the existing validation scenes against it.

Only Windows has a surface and is exercised end to end. The other platform jobs compile the plugin but do not render with it.

Worth a look:

  • The plugin keeps a small number of deliberate accommodations: a V8 finalizer drain (JsRuntimeHost does not drain pending finalizers yet) and the software-adapter (WARP) resource setting used on CI. Both are documented in the source and should move upstream.
  • Validation scenes still excluded on WebGPU in Apps/Playground/Scripts/config.json each carry a reason. The main groups are 17 Atmosphere and OpenPBR scenes, which stall and then lose the device on the CI software adapter (no shader error; they pass on a real GPU) and so have no CI coverage until a GPU runner is available, 5 IBL-shadow scenes (bgfx throws Unsupported alpha mode: 19; Dawn hits a 2D/3D texture binding mismatch), and 3 SPZ scenes that need ReadableStream in JsRuntimeHost.
  • 14 SPZ/CSG scenes that were wrongly excluded everywhere now run via the es-module-import workaround (excluded on JavaScriptCore, where the SPZ/Manifold WASM never loads).
  • CI runs the WebGPU validation with --generate-references, so it gates on loading and rendering failures, not pixel comparison. Pixel comparison against references has only been run locally.
  • Context2D shadows are stored but not rendered, matching the bgfx canvas.
  • The Lite parity and URL validation reports live in Plugins/NativeDawn. Their results are a point-in-time record, and some links point at local build artifacts.

@matthargett

Copy link
Copy Markdown
Contributor

can we coordinate a little more on these kinds of changes? I spent months on the wgpu-native backing for this integration, and you all said you wouldn't be switching off of bgfx. I feel like I'm being as transparent and inclusive as I can, but I'm getting a bit of whiplash with this sudden change in policy (even if it favors my goals).

@CedricGuillemet

Copy link
Copy Markdown
Collaborator Author

can we coordinate a little more on these kinds of changes? I spent months on the wgpu-native backing for this integration, and you all said you wouldn't be switching off of bgfx. I feel like I'm being as transparent and inclusive as I can, but I'm getting a bit of whiplash with this sudden change in policy (even if it favors my goals).

This is only an experiment. Taking a closer look at #1605 is also on my todolist. We are also investing time on CI testing and harness. No need to worry , we won't forget you :)

@bghgary

bghgary commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

We are still in the middle of planning with some prototyping/experiments. See here for some idea of what we are up to: BabylonJS/Babylon-Lite#10.

@CedricGuillemet

CedricGuillemet commented Jul 21, 2026 •

Copy link
Copy Markdown
Collaborator Author

Babylon-Lite bundles on NativeDawn/WebGPU

Ran all 216 Babylon-Lite scene bundles through the Playground on the NativeDawn/WebGPU backend. Tested with the V8 JS engine (the WebGPU bootstrap is V8-only; ChakraCore fails).

~124 scenes render correctly — PBR/glTF, IBL/HDR, shadows (incl. CSM), skinning/morph/VAT, most NME materials, frame-graph post-processes (DoF/TAA/bloom), gizmos, geospatial camera, .babylon (Sponza), grid material, floating-origin, etc.

~92 don't work, grouped by reason:

Reason Scenes
WASM not fetchable — Havok physics 40-42,44-49,100-106,209
WASM — Recast navigation 170,172,173
WASM — manifold CSG 91
WASM — basis/KTX2, meshopt, text-shaper (silent black) 36,112,211,180,181
Web Worker (Blob URL) — Gaussian Splatting, gizmos 120-129,222
Blob.stream/ReadableStream missing 66,72,140
Canvas 2D API incomplete (sprites, DynamicTexture) 18,24,25,50-57,90,92-98,117,118,145,146,179,205,206
Multi-canvas (getContext null) 227,228
Missing local lab assets (fetch) — fixable, just copy assets 58,59,73,81,83,160,171,174,175,210,254,255,257,258,259,260,266
Silent black, not yet isolated (still black post texture-fix — unrelated cause) 12,30,63,149,153,263,264

Physics/nav/gaussian-splatting fail as expected (no web workers + feature .wasm can't be fetched). The biggest fixable buckets are the Canvas-2D API and the missing-local-assets.

How to test a lite bundle

  1. In the Babylon-Lite repo, build the lib and bundle a scene as a single self-contained IIFE classic script (the default build:bundle-scenes output is ESM+code-split, which the Playground's classic LoadScript can't load):
    pnpm build:lib
    # esbuild: bundle:true, splitting:false, format:"iife",
    #   alias babylon-lite -> packages/babylon-lite/build/lib
    # -> lab/public/bundle/sceneN.playground.js
    
  2. Copy sceneN.playground.js into the Playground Scripts/ dir.
  3. Run it with the V8 build:
    Playground.exe --headless app:///Scripts/sceneN.playground.js
    
    (drop --headless to see the window). Any root-relative asset (/foo.png) is redirected to app:///Scripts/foo.png, so place sibling assets there.

How to test the glTF texture fix

# the exact bug case (renders textured, not black):
Playground.exe --headless --test "BoomBox with Unlit" app:///Scripts/validation_native.js
# -> "No pixel difference!"  /  "GLTF BoomBox with Unlit Material validated"

# full validation suite:
Playground.exe --headless app:///Scripts/validation_native.js
# -> Run complete. ran=259 passed=259 failed=0

@laurentdiazfr

laurentdiazfr commented Jul 21, 2026 •

Copy link
Copy Markdown

@CedricGuillemet What are your thoughts on https://github.com/wcandillon/react-native-webgpu as a Dawn backend? Would being tied to React Native be a limitation for what you're trying to achieve? React native and Expo excosystem is huge nowadays, it will increase the adoption of babylon native.

@CedricGuillemet

Copy link
Copy Markdown
Collaborator Author

@laurentdiazfr https://github.com/wcandillon/react-native-webgpu looks great and being able to use Lite directly is awesome. BabylonNative corresponds to different needs (same for BRN). For example, usage of XR or tight integration of native libraries with Babylon. The more possibilities, the better, I think. For example, fills the gap when a user wants to integrate 3D in a native android app.

@matthargett

Copy link
Copy Markdown
Contributor

FYI, expanded my wgpu branch suite based on the report above. A few fixes is that branch, but there's several more WHATWG/W3C API compliance issues that need fixing to expand it to the fuller screenshot suite on browsers.

I submitted those PRs into JsRuntimeHost, and validated them locally on macOS 26, iOS 18, and Android 12 (Pico XR). I tested across JSC and QuickJS in that matrix, but ran out of time to go further today (I spot check a lot of the screenshots etc manually so I can dig deeper into pixel differences.)

This has been a great driver to make BabylonNative competitive as a 3D-forward app runtime that stands shoulder to shoulder with other peers like deno, bun, NodeJS, and electron! Very exciting!

@CedricGuillemet

CedricGuillemet commented Aug 26, 2026 •

Copy link
Copy Markdown
Collaborator Author

NativeDawn follow-up: workarounds and shims to replace

This is a cleanup checklist, not a list of merge blockers. The current implementations are useful for proving and testing the backend, but these should be replaced with explicit APIs or fixes in the appropriate layer.

Updated 2026-10-05 against cbb2f4ee. Checked items are complete; partially addressed items remain unchecked.

Runtime/bootstrap

  • Replace the BABYLON.Tools.LoadScript monkey patch with explicit native-host integration. Partially addressed: it no longer lives in NativeDawn. Apps/Playground/Scripts/dawn_playground.js explicitly supplies Playground shader-script loading through shared native fetch, validates response status, and reports load/evaluation failures. A supported Babylon.js loader capability/injection API is still needed to remove the host-level override itself; synthetic <script>.onload is not used.

  • Replace the BABYLON.NativeEngine alias and private bootstrap globals. Partially addressed: NativeDawn no longer intercepts BABYLON, constructs an engine, or installs __dawnPendingEngine/__dawnEngine. Playground explicitly creates its WebGPUEngine and exposes _playgroundWebGPUEngine / _playgroundWebGPUReady; validation and snippet startup await that readiness contract. The BABYLON.NativeEngine singleton alias remains only in Playground for compatibility, so removing that constructor override still needs a supported host/backend contract. Windows Lite and the standalone runtime regression run without the Babylon.js bootstrap.

  • Expose a supported backend identifier instead of detecting NativeDawn through _nativeDawnClear. Validation and the snippet runner use TestUtils.getGraphicsApiName() === "WebGPU". _nativeDawnClear and its milestone clear/rendering path have been removed.

  • Replace magic JS globals used by the host frame/resize bridge. Embedding calls typed NativeDawn::Tick(env) and NativeDawn::ResizeSurface(env, width, height) directly. The plugin no longer installs or looks up global frame / __dawnResize hooks. Typed ticking owns animation callbacks, deferred readback, submitted-surface presentation, Dawn event processing, finalizers, and resource retirement; engine-specific resize handling belongs to the host.

JavaScript-host / Web platform gaps

  • Remove the synchronous WebAssembly.instantiate override. Removed. The retained master JsRuntimeHost pin 33e4a233dea178712352ec90a647683084bfbf1d includes BabylonJS/JsRuntimeHost#232, which dispatches V8 foreground/delayed tasks. Native asynchronous WASM is covered by the standalone runtime regression. Dawn GPU event processing remains required and is separate from V8 task pumping.

  • Move UTF-16 support into the shared TextDecoder polyfill. Removed the plugin decoder override. The shared pinned implementation contains BabylonJS/JsRuntimeHost#230 and BabylonJS/JsRuntimeHost#235 for UTF-16LE/BE and malformed-input handling; UTF-8 aliases are also supported upstream. Runtime regressions cover the shader-helper requirements. This is not a claim of complete TextDecoder streaming/fatal/DataView support.

  • Consolidate the DOM/canvas shims into shared polyfills. NativeDawn still creates or patches document, getElementById, querySelector, window, and Canvas compatibility behavior. The old JS-global __dawnCanvas cache has been replaced by an internal presentation-canvas reference with width/height accessors that work without Babylon.js. Moving the remaining duplicated DOM/Canvas behavior into one shared, tested compatibility layer is still pending.

  • Fix URL/Blob/object-URL behavior in the shared host polyfills. Partially addressed: the Blob byte stash and custom object-URL-to-data-URL bridge have been removed; native Blob/object URLs now use shared JsRuntimeHost/UrlLib implementations from BabylonJS/JsRuntimeHost#108 and BabylonJS/JsRuntimeHost#207. A narrow URL constructor coercion wrapper remains because the current host constructor does not coerce URL-object/non-string arguments. Expanded Blob behavior (slice, stream, multiple parts) and native Blob identity fixes remain upstream follow-ups; basic native Blob/object-URL support does not imply complete browser compatibility.

  • Replace the global root-relative fetch rewrite with an application asset resolver. Removed the plugin-wide fetch rewrite. Only Apps/Playground/Scripts/lite_native.js redirects bundled lab root-relative URLs to app:///Scripts; unrelated NativeDawn applications keep the shared host fetch semantics.

  • Move Image / createImageBitmap compatibility into a shared image polyfill (Plugins/NativeDawn/Source/NativeDawn.cpp). Native image decoding is required functionality, but implementing the browser API surface inside the graphics backend makes it unavailable or divergent for other backends.

Dawn/platform workaround

  • Remove the WARP MSAA lazy-clear workaround after the Dawn issue is fixed (Plugins/NativeDawn/Source/NativeDawn.cpp). The code enables nonzero_clear_resources_on_creation_for_testing and detects WARP using adapter type, Microsoft vendor ID and device-name substrings. This should be tracked upstream in Dawn; string/vendor sniffing and a testing-only toggle should not be permanent.

Test and CI accommodations

  • Restore actual NativeDawn pixel validation in CI (.github/workflows/build-win32.yml). The full suite currently runs with --generate-references because WARP output differs from hardware-generated references, so it gates load/render/crash behavior but not image regressions. Use WARP-specific references or a stable hardware runner; keep Babylon-Lite's WARP pixel test as the current narrow guard until then.

  • Pin the Babylon-Lite revision used by CI (.github/workflows/build-win32.yml). Cloning the default branch tip makes this PR's CI depend on unrelated future Babylon-Lite changes. Pin a commit/tag and update it deliberately.

  • Fix shader-transpiler readiness upstream instead of pre-warming it (Apps/Playground/Scripts/validation_native.js). The first GLSL effect has order-dependent isReady() behavior until glslang/twgsl is loaded. The proper Babylon.js fix should make readiness deterministic; then remove the harness pre-warm.

  • Replace the growing per-test global reset list with real test isolation (Apps/Playground/Scripts/validation_native.js). The harness manually restores Math.random, loader observers, reverse depth, snapshot rendering and every ForceGLSL static because playgrounds leak process-global state. Prefer a fresh engine/context per test or a supported Babylon.js reset API.

  • Turn WebGPU exclusions and widened tolerances into tracked issues (Apps/Playground/Scripts/config.json). The reasons document real gaps well (unsupported LINE_LOOP/TRIANGLE_FAN conversion, decoder/runtime issues, platform raster differences), but issue links/owners are needed so exclusions do not become permanent silently.

Validation status

Local Windows validation passed: standalone runtime 49 checks, Playground host 18 checks, Babylon-Lite 80/240000 differing pixels (0.033%), focused Babylon.js WebGPU 13/13, broader indices 0-175 161/161 with 15 existing catalog skips, and shared native/JavaScript coverage 6 native tests / 128 JS checks. The full-suite GUI Slate stall remains and reproduces with the original PR bindings; selected-scope success is not a full-suite pass. The rebase preserved the exact tested file tree. CI for the rebased head is still pending, not yet confirmed green.

Not included in this debt list

The WebGPU constant globals, OffscreenCanvas, Image, createImageBitmap, requestAnimationFrame, backend-specific CMake selection, and platform guards are not inherently hacks: a native WebGPU host needs those capabilities. The debt is where they are implemented today (backend-local patches/private globals), incomplete standards behavior, or temporary CI bypasses—not the existence of the APIs themselves.

@bkaradzic-microsoft

Copy link
Copy Markdown
Member

Good list. Four of these are already root-caused and fixed on my shotgun branch — the fixes are in JsRuntimeHost, which is where you're asking for them to live, so they turn into deletions here rather than rewrites.

Already fixed in JsRuntimeHost (not yet PR'd — on bkaradzic-microsoft/JsRuntimeHost@shotgun)

Remove the synchronous WebAssembly.instantiate override (NativeDawn.cpp:4426-4443) — your diagnosis is exactly right. AppRuntime_V8 creates a default v8::Platform but nothing ever called PumpMessageLoop, so V8 finished the compile on a background thread, posted a foreground task to resolve the promise, and that task was never run. (Sync new WebAssembly.Module was unaffected, which made it look like a hang rather than a failure. AppRuntime_JSI already did the equivalent via TaskRunnerAdapter; the direct-V8 path was simply missing it.) DrainMicrotasks now pumps the queue, and the platform is wrapped in a DispatchingPlatform whose foreground task runner nudges the app dispatcher so a pump also happens when the app is otherwise idle. The override can just be deleted.

Move UTF-16 support into the shared TextDecoder polyfill (NativeDawn.cpp:4394-4423) — done in Polyfills/TextDecoder: utf-16le/utf-16be including the leading-BOM strip. The JS wrapper can go.

Same commit also fixes a third one you'd have hit next: XMLHttpRequest.responseText/response truncated at the first NUL byte. ResponseString() returns a string_view and only data() was handed to Napi, so it was measured with strlen — Emscripten inlines the .wasm payload as a JS string literal, so spz.js arrived as 1955 of 805507 characters.

Together those three are what let me re-enable validation tests 100, 104, 157 and 188 (Gaussian Splatting SPZ, CSG2/manifold), which were excluded precisely because loading a script as an ES module didn't work on Native.

Already fixed on master's current JsRuntimeHost pin

The Blob stash / object-URL → data: URL conversion (NativeDawn.cpp:4675-4714) is redundant as of JsRuntimeHost #207, which is in cb988ba. The URL polyfill now registers a blob: scheme resolver with UrlLib, so fetch, XMLHttpRequest and any other UrlLib consumer resolve blob: URLs directly — including the forceBitmapOverHTMLImageElement XHR path that motivated the hack. createObjectURL/revokeObjectURL are backed by a real store, and the shim here is already guarded by if (!url.Get("createObjectURL").IsFunction()).

The URL coercion wrapper (NativeDawn.cpp:4736-4781) is still needed though — URL::URL does info[0].As<Napi::String>() with no USVString conversion, so a URL object as url or base throws. That's a genuine small JsRuntimeHost fix.

Agreed, not fixed

The DOM/canvas/Image/createImageBitmap consolidation — my Batches 77/78 add Image, PointerEvent, screen, DOM event dispatch and KeyboardEvent to validation_native.js, which is the same debt you're flagging at validation_native.js:772-802, not a resolution. Worth doing once in a shared Window/DOM/Canvas layer rather than twice badly.

Happy to split the JsRuntimeHost commit into PRs (V8 pumping / TextDecoder / XHR NUL are independent) so this PR can bump the pin and drop the three shims.

CedricGuillemet added a commit to CedricGuillemet/BabylonNative that referenced this pull request Oct 5, 2026
Integrate PR BabylonJS#1781 with current master, remove superseded runtime shims, and use typed Dawn frame/resize APIs. Move Babylon engine bootstrap into Playground and cover standalone Lite, runtime, and host behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CedricGuillemet and others added 5 commits October 12, 2026 15:19
Preserve upstream per-test validation workaround metadata and scene cleanup while retaining Dawn bootstrap and canvas behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Resolve CI and catalog conflicts while preserving separate Dawn paths and including Dawn jobs in the aggregate CI gate. Record complete Windows Lite and WebGPU results without hiding existing dependency, packaging or capability failures.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Integrate Babylon.js 9.31.3, validation updates and CI caching while retaining the optional Dawn build routes. Resolve Canvas gradients using the upstream implementation and preserve WebGPU-specific catalog restrictions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep upstream straight RGBA interpolation while restoring nearest-byte packing for both linear and radial ramps. Update the merged exact-byte unit assertions to retain the existing Chrome-backed Dawn quantization contract.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep runtime regressions available locally, expose the configured JavaScript engine through Dawn TestUtils, and record complete pinned-consumer Windows validation results.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
deltakosh pushed a commit to BabylonJS/Babylon.js that referenced this pull request Oct 28, 2026
…9075)

> Supersedes #18562, which GitHub cannot reopen because its head branch
moved after it was closed. Same change, brought up to date with current
master; the measurements below are new.

## Summary

This PR adds an internal render-command batching layer for WebGPU draw
submission, bringing the WebGPU engine into congruence with a pattern
Babylon.js core already ships: the bgfx-based NativeEngine batches its
engine commands through a `CommandBufferEncoder`
(`thinNativeEngine.pure.ts`) and flushes them across the JS/native
bridge in bulk, rather than paying one Node-API crossing per call. The
WebGPU engine has no equivalent layer, so on hosts that run it over a
JS/native boundary every `setPipeline`, `setBindGroup`,
`setVertexBuffer`, `setIndexBuffer` and `draw*` crosses that boundary
individually. That per-draw bridge traffic is the CPU cost profiled in
[this forum
post](https://forum.babylonjs.com/t/solving-cpu-bottleneck-in-nativexr-hill-valley-ar-portal-scene/63558)
(iPhone XS / iPhone 12, Hill Valley AR portal scene).

This is also the architecture browsers use one layer down for the
identical boundary problem: Chromium's Dawn wire protocol serializes
WebGPU commands into batched buffers across the renderer-to-GPU-process
boundary, and Unreal's `FRHICommandList` records on the render thread
and translates on the RHI thread. The boundary here is N-API or JSI
rather than IPC or a thread hop; the remedy is the same.

**Browsers are unaffected, and this is now measured rather than
asserted** (see *Browser cost* below): the no-provider path costs about
5 ns per draw in isolation, and a 3,000-draw-per-frame scene in Chrome
152 shows no measurable frame-time difference between this branch and
master.

**Scope note:** this is transport-level, order-preserving submission
batching, not draw merging. It does not reorder, sort or combine draws,
so it composes with instancing rather than competing with it, and the
stream format reserves multi-draw fields for future multi-draw-indirect
lowering.

## Why the engine is the right layer

Every native WebGPU host for JavaScript binds the render-pass encoder
one call at a time, because none of them owns the draw loop:

- BabylonNative's Dawn backend
([BabylonJS/BabylonNative#1781](BabylonJS/BabylonNative#1781))
binds each `GPURenderPassEncoder` method as its own N-API callback;
`setBindGroup` reads its dynamic offsets into a fresh `std::vector` on
every call
([NativeDawn.cpp#L2295-L2340](https://github.com/BabylonJS/BabylonNative/blob/da15f7ae1a6f5c59a4d6f1fd322145a3d308f70f/Plugins/NativeDawn/Source/NativeDawn.cpp#L2295-L2340)).
- react-native-webgpu binds the same methods as per-call JSI host
functions
([GPURenderPassEncoder.h#L103-L112](https://github.com/wcandillon/react-native-webgpu/blob/24cf75ba084cb23fb477053fc310ecbfceae6bb2/packages/webgpu/cpp/rnwgpu/api/GPURenderPassEncoder.h#L103-L112)).
Its optimizations (prototype caches, import-once native buffers, async
work pumped on the owning instance) target per-object and per-frame
costs; per-draw crossings are out of its reach for the same reason.
- The wgpu-native backend this PR was developed against is the same
shape.

The Babylon Lite native plan treats this as a first-class metric: every
prototype must report "JS↔C crossing count and cost per frame", and
option S0d is a purpose-built JS↔C wire that crosses once per frame
([BABYLON_LITE_PLAN.md](https://github.com/bghgary/BabylonNative/blob/bghgary/babylon-lite-plan/BABYLON_LITE_PLAN.md)).
This PR is that wire for the existing engine: it lives in the transport
layer, behind the standard WebGPU API, and a host opts in by installing
a provider.

The benefit scales with draws per frame, which is the honest way to read
the native numbers below: a frame with ten draws gains nothing; the full
Hill Valley scene at 81 draws per frame gains about 8% on iPhone 12; the
stress scene at thousands of draws per second cuts draw-level bridge
crossings by about 95%.

## Architecture

- `RenderCommandBatcher`: a small generic lower-or-replay adapter. It
tries the optional lowering path; for a draw that cannot lower, it
flushes pending lowered work first, then replays the original command
exactly once. Ordering is preserved unconditionally.
- `WebGPURenderPassCommandStream`: the WebGPU-specific lowerer. It
encodes compatibility-mode draws as opcode words into a reused
`Uint32Array`, state-diffing pipeline, index/vertex buffers and bind
groups so redundant state is elided before it is recorded, then submits
the whole batch through a single `_recordCommands` call on the render
pass.
- Flush boundaries: batched draws are recorded before every direct
render-pass mutation (viewport, scissor, stencil reference, blend
constant, full-quad clears, occlusion query begin/end, debug marker
push/pop/insert, and render-pass end) through a
`ThinWebGPUEngine._flushRenderPassCommands()` seam that is a no-op for
engines that do not batch. Without these boundaries a draw batched
inside an occlusion-query bracket would be recorded after the bracket
closes (the query reports zero samples and the mesh wrongly disappears
as occluded), and GPU captures would attribute draws to the wrong debug
group. The occlusion-query flush sits inside the `_occlusionQueryActive`
guard introduced by #18918.
- The draw hot path is allocation-free: the engine reuses two scratch
draw-command objects (both the lowering and replay paths consume
commands synchronously), and the stream reuses scratch id arrays.
`_draw` now has one apply site, `ApplyWebGPURenderPassDrawCommand`, so
indirect and multi-draw lowering plug in at one place rather than at
each encoder call.
- Failure policy: a protocol-version mismatch or a backend recording
failure logs one error (enriched with the provider's diagnostics) and
permanently falls back to direct encoder calls for that engine; nothing
throws from the render loop. Resource ids returned by the provider are
validated as positive safe integers at the trust boundary; an
out-of-range id makes the draw replay directly.
- Typed indexed/non-indexed draw command variants instead of a magic
numeric draw type.

## Host contract

A host opts in by installing a provider on the engine:

```js
engine.renderPassCommandRecorderProvider = {
    protocolVersion: 1,                       // must equal WebGPURenderPassCommandStreamProtocolVersion in the decoder build
    isEnabled: () => true,                    // host-controlled toggle
    getResourceId: (resource) => /* host handle id for a pipeline/buffer/bind group, or 0 */,
    getLastError: () => /* optional backend diagnostics for failure logs */,
};
```

and exposing a `_recordCommands(words, drawCount, multiDrawCallCount,
multiDrawDrawCount, wordCount)` hook on its render-pass encoders. The
recorder must consume the words synchronously; the buffer is reused
after the call returns. The `protocolVersion` handshake mirrors the
`PROTOCOL_VERSION` guard the bgfx-based NativeEngine uses to keep the JS
encoder and native decoder from drifting, soft-failing here because this
transport is an optimization rather than the engine's only path.

## Browser cost

With no provider installed (always the case in browsers) a
compatibility-mode draw fills the scratch command, calls `submit`, which
returns at the provider null check, and replays through
`ApplyWebGPURenderPassDrawCommand`, which issues the same encoder calls
the old inline code did.

- Isolated microbenchmark of exactly that path against the inline
encoder calls, on a stub render pass (V8 / Node 24, Apple M4 Max,
2,000,000 draws per round, three rounds): inline 8.0–8.3 ns per draw,
no-provider path 13.1–13.4 ns per draw. The added cost is about 5 ns per
draw, or 15 µs for a 3,000-draw frame.
- Chrome 152 (WebGPU on Metal, Apple M4 Max), UMD bundles built from
this branch and from its master base, a scene of 3,000 meshes with 64
materials (2,996 draws per frame), compatibility mode, 600 timed frames
after 150 warm-up frames, measuring main-thread time from `beginFrame()`
through `scene.render()`:

| Bundle | mean | p50 | p90 | p99 |
|---|---:|---:|---:|---:|
| master | 38.9 ms | 44.5 ms | 50.9 ms | 69.9 ms |
| this branch | 38.1 ms | 43.4 ms | 51.9 ms | 59.6 ms |

The two are indistinguishable within run-to-run noise; Chrome's own cost
in that scene is about 13 µs per draw, three orders of magnitude above
the added path.

## Test coverage

- Contract-level unit coverage of the batcher and stream: lowering
absent/present/disabled, fallback replay with flush-before-replay
ordering, state-diff behavior per state kind,
indexed/non-indexed/indirect draws, late and out-of-range resource ids,
command storage growth, render-pass boundary flushing, no-provider
behavior, protocol-version mismatch, and permanent fallback with
diagnostics when backend recording fails.
- Flush-boundary scenario tests
(`webgpuRenderPassFlushBoundaries.test.ts`) drive the real registered
engine prototype methods against a semantic model of the native render
pass: a GPU timeline that executes direct calls and decoded
command-stream batches in submission order, attributing each executed
draw to the occlusion query and debug groups open at execution time.
Assertions are on user-visible outcomes ("which draws did query N
observe", "which draws are inside group X", "did the draws stay on the
batched path"), not on encoder-call ordinality.

## Validation

- `npx vitest run --project=unit` over `packages/dev/core/test/unit`
with this branch merged with current master: 289 files, 4,793 tests pass
plus the one pre-existing expected failure, including 39 focused tests
across `renderCommandBatcher.test.ts`,
`webgpuRenderPassCommandStream.test.ts`,
`webgpuRenderPassFlushBoundaries.test.ts` and `engine.query.test.ts`.
- `npx eslint --quiet` over all changed source and test files: clean.
The `pure.ts` barrel entry is in the generator's order.
- `nx build babylonjs`: passes; the resulting bundle is the one measured
above.

Native integration validation (BabylonNative, WebGPU backed by
wgpu-native):

- macOS Release/LTO Playground, non-AR Hill Valley stress scene,
material parity checks passing (`issues=0`), SSAO2 enabled in
`mode=prepass`. Command stream enabled, steady windows: about 13,920
draws per 2 s grouped into about 720 command-stream bridge calls, about
94.8% fewer draw-level bridge traversals, at 59.5–60.5 native fps.
Command stream disabled: same draw volume, `cmdStreams=0`, about 59–60
native fps. The M4 path is vsync-bound, so this run demonstrates the
bridge-call reduction rather than an fps gain.
- iPhone 12 (A14), full Hill Valley scene forced non-XR (112 active
meshes, 81 draws per frame with SSAO2): 18 fps → 19.5 fps with the
command stream enabled; no change in the pre-placement phase, which
draws about 10 meshes per frame.
- iPhone XS (A12): about 40 fps → 45 fps with improved 1% lows, as
documented in the forum post.

## Related work

This continues the Hill Valley NativeXR AR Portal optimization series
already merged from this validation effort:

- #18539 — Support SSAO2 with world-space normals
- #18521 — Move custom frame requester to AbstractEngine to enable
120fps support in BabylonNative
- #18538 — Fix KTX noMipmap flip for compressed sRGB mipmaps

## Future work

- Multi-draw-indirect lowering through the reserved
`multiDrawCallCount`/`multiDrawDrawCount` stream fields (wgpu-native
exposes MDI; the WebGPU spec does not yet).
- Lower debug markers
(`pushDebugGroup`/`popDebugGroup`/`insertDebugMarker`) into stream
opcodes so that enabling GPU profiler markers does not fragment batches
with boundary flushes; today markers stay correct (flushed) but each
marker costs a bridge crossing and splits the batch, which is acceptable
for profiling sessions only.
- The command-stream types already admit `GPURenderBundleEncoder`, so
the same lowering could batch bundle construction under
`compatibilityMode = false` if profiling justifies it.
CedricGuillemet and others added 7 commits October 28, 2026 11:58
Applies BabylonJS/UrlLib#39 transport fixes to Dependencies/UrlLib, updates NativeDawn to use vendored dependencies, and refreshes validation docs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…iew fixes

- Route data: URLs through fetch/UrlLib instead of a plugin-owned base64 decoder.
- Coerce URL constructor/canParse arguments with ToString() in the URL polyfill and drop the global URL wrapper.
- Fire Image.onerror when decoding fails.
- Implement cancelAnimationFrame with monotonic ids.
- Build JsRuntimeHost/UrlLib unit tests in the Linux NativeDawn job.
- Remove a duplicate Spotlight shadows catalog entry; clarify CI visualization docs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… waits

Atmosphere scenes never become ready on the CI WARP adapter; each waited out the 20-minute default sized for the ASAN job, pushing the Win32 NativeDawn job past its timeout.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… route readback failures to error callback

createScene's promise wait is now capped by the readiness timeout instead of a
fixed 10 minutes, so --readiness-timeout bounds scenes that never resolve.

NativeDawn framebuffer readback failures (e.g. device lost) are now delivered
to the error callback passed to TestUtils.getFrameBufferData, mirroring
TestUtils, instead of throwing uncaught from Tick. The runner treats "lost"
like "cancel", so a lost device aborts the run cleanly with a summary. A
diagnostic-only device-lost callback logs the reason and message.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CedricGuillemet and others added 2 commits October 30, 2026 09:58
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…s on WebGPU

- ContextDawn stores shadowColor/Blur/OffsetX/OffsetY (saved/restored) and warns once instead of throwing, matching the bgfx canvas.
- config.json: exclude Material Plugin (GLSL-only plugin), Node material PBR 1 (intermittent createScene stall), and the Atmosphere/OpenPBR scenes that lose the device on the CI WARP adapter, on WebGPU only, with reasons.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@CedricGuillemet CedricGuillemet changed the title NativeDawn plugin Add optional NativeDawn plugin for WebGPU on Windows Oct 30, 2026
CedricGuillemet and others added 3 commits October 30, 2026 13:25
…master references; move Dawn docs

- Resolve device.lost from the Dawn device-lost callback
- Isolate requestAnimationFrame callbacks, validate handle type tags
- Size readback from the presented texture; define event stubs only if missing
- Restore 32 shared reference PNGs to master versions
- Move Dawn parity/validation docs into Plugins/NativeDawn and update scripts

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Remove the obsolete _LoadScriptModuleAsync exclusions from 14 scenes on all
graphics APIs, relax errorRatio for two Compound scenes, and correct the
reason for the 5 IBL shadow scenes that stay excluded.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The SPZ/Manifold WASM module never finishes loading on Linux JSC, hanging the
validation run (same as the already-opted-in SPZ/CSG scenes).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@CedricGuillemet
CedricGuillemet requested review from bghgary and bkaradzic-microsoft and a balanced review from Copilot and removed request for bghgary November 2, 2026 13:52
@CedricGuillemet
CedricGuillemet marked this pull request as ready for review November 2, 2026 13:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved critical defects can render to obsolete windows, display stale image textures, and falsely report failed or incomplete parity runs as passing.

7 open issues
What changed in this PR

Adds an optional Dawn-backed WebGPU plugin so Babylon.js WebGPUEngine can run on Babylon Native, with Windows rendering support and bgfx remaining the default.

Changes:

  • Integrates NativeDawn with Embedding, shared Canvas helpers and Playground initialization.
  • Adds shared data-URL decoding, byte-preserving POST handling and regression coverage.
  • Adds WebGPU build jobs, Windows rendering validation and Babylon-Lite parity tooling.
File Description
Polyfills/​Window/​Source/​Window.cpp Supplies a pixel-ratio fallback without bgfx.
Polyfills/​Canvas/​Source/​nanovg/​nanovg_dawn.h Declares the Dawn NanoVG backend interface.
Polyfills/​Canvas/​Source/​nanovg/​nanovg_dawn_shader.h Adds WGSL shaders for Canvas rendering.
Polyfills/​Canvas/​Source/​MeasureText.h Declares backend-independent text measurement.
Polyfills/​Canvas/​Source/​MeasureText.cpp Shares measurement through NanoVG contexts.
Polyfills/​Canvas/​Source/​Image.h Exposes decoded image pixels.
Polyfills/​Canvas/​Source/​Image.cpp Implements decoded-pixel access.
Polyfills/​Canvas/​Source/​Gradient.cpp Rounds and clamps gradient color bytes.
Polyfills/​Canvas/​Source/​Colors.h Updates HSL conversion and quantization.
Polyfills/​Canvas/​Source/​CanvasEncoding.h Declares shared PNG encoding.
Polyfills/​Canvas/​Source/​CanvasEncoding.cpp Encodes RGBA pixels as PNG data URLs.
Polyfills/​Canvas/​Source/​Canvas.cpp Uses the shared PNG encoder.
Polyfills/​Canvas/​CMakeLists.txt Builds encoding helpers and optional Dawn rendering.
Plugins/​TestUtils/​Source/​TestUtils.cpp Exposes exit-callback notification.
Plugins/​TestUtils/​Include/​Babylon/​Plugins/​TestUtils.h Declares backend-independent exit notification.
Plugins/​TestUtils/​CMakeLists.txt Publishes the GraphicsDevice dependency.
Plugins/​NativeDawn/​Source/​ImageDecode.cpp Decodes images into RGBA pixels.
Plugins/​NativeDawn/​Source/​ImageDataDawn.h Declares persistent Canvas image data.
Plugins/​NativeDawn/​Source/​ImageDataDawn.cpp Implements clamped pixel buffers and readback.
Plugins/​NativeDawn/​Source/​ContextDawn.h Declares Dawn Canvas2D operations and state.
Plugins/​NativeDawn/​Source/​CanvasDawn.h Declares Dawn canvas rendering and readback.
Plugins/​NativeDawn/​Include/​Babylon/​Plugins/​NativeDawn.h Exposes plugin lifecycle and surface APIs.
Plugins/​NativeDawn/​CMakeLists.txt Defines plugin dependencies and runtime integration.
Plugins/​CMakeLists.txt Registers the optional NativeDawn plugin.
Embedding/​Source/​View.cpp Routes frames, resizing and input through Dawn.
Embedding/​Source/​RuntimeImpl.h Adds Dawn lifecycle and frame-dispatch state.
Embedding/​Source/​Runtime.cpp Initializes and tears down Dawn on the JS thread.
Embedding/​CMakeLists.txt Selects Dawn and links Compression support.
Dependencies/​UrlLib/​Tests/​UrlRequestErrorReporting.cpp Adds HTTP body, header and lifecycle tests.
Dependencies/​UrlLib/​Tests/​SchemeResolver.cpp Tests cancellation before resolver invocation.
Dependencies/​UrlLib/​Tests/​Main.cpp Initializes WinRT for transport tests.
Dependencies/​UrlLib/​Tests/​DataUrl.cpp Tests data-URL decoding and lifecycle behavior.
Dependencies/​UrlLib/​Tests/​CMakeLists.txt Registers new transport tests and entry point.
Dependencies/​UrlLib/​Source/​UrlRequest_Windows_Shared.h Preserves POST bytes and handles optional headers.
Dependencies/​UrlLib/​Source/​UrlRequest_Shared.h Routes shared schemes through cancellable resolution.
Dependencies/​UrlLib/​Source/​UrlRequest_Base.h Integrates built-in data-URL resolution.
Dependencies/​UrlLib/​Source/​UrlRequest_Apple.mm Preserves raw POST body bytes.
Dependencies/​UrlLib/​Source/​UrlRequest_Android.cpp Writes POST bodies as byte-array chunks.
Dependencies/​UrlLib/​Source/​DataUrl.h Implements data-URL parsing and decoding.
Dependencies/​UrlLib/​README.md Documents data URLs and transport behavior.
Dependencies/​UrlLib/​Include/​UrlLib/​UrlLib.h Documents shared scheme-resolution contracts.
Dependencies/​UrlLib/​CMakeLists.txt Adds the base-n decoding dependency.
Dependencies/​JsRuntimeHost/​Polyfills/​URL/​Source/​URL.cpp Coerces URL arguments to strings.
CMakeLists.txt Configures optional Dawn dependencies and backends.
BUILDING.md Documents WebGPU builds, validation and limitations.
Apps/​UnitTests/​Source/​Tests.Canvas.Readback.cpp Tests gradient rounding and HSL colors.
Apps/​Playground/​Win32/​App.cpp Resolves script paths and selects bootstrap behavior.
Apps/​Playground/​Shared/​PlaygroundScripts.h Documents Dawn bootstrap loading.
Apps/​Playground/​Shared/​PlaygroundScripts.cpp Loads Dawn bootstrap and forwards readiness options.
Apps/​Playground/​Shared/​Diagnostics.cpp Adds symbolized Windows crash stacks.
Apps/​Playground/​Shared/​CommandLine.h Adds a readiness-timeout option.
Apps/​Playground/​Shared/​CommandLine.cpp Parses and validates readiness timeouts.
Apps/​Playground/​Scripts/​run_snippet.js Runs individual Playground snippets.
Apps/​Playground/​Scripts/​playground_runner.js Waits for WebGPU engine initialization.
Apps/​Playground/​Scripts/​lite_parity_native.js Bridges native evaluation and screenshots to parity tests.
Apps/​Playground/​Scripts/​lite_native.js Adds a Lite rendering and pixel-comparison harness.
Apps/​Playground/​Scripts/​dawn_playground.js Initializes the Playground WebGPU engine.
Apps/​Playground/​Scripts/​dawn_playground_native.js Tests bootstrap, resizing and framebuffer readback.
Apps/​Playground/​CMakeLists.txt Packages harnesses and Windows runtime support.
Apps/​Playground/​Android/​BabylonNative/​build.gradle Enables plugin-only Dawn builds.
.github/​workflows/​ci.yml Adds NativeDawn platform jobs.
.github/​workflows/​build-win32.yml Builds Dawn and runs Lite/WebGPU validation.
.github/​workflows/​build-macos.yml Adds Dawn plugin build coverage.
.github/​workflows/​build-linux.yml Adds Dawn build dependencies and coverage.
.github/​workflows/​build-ios.yml Adds Dawn plugin build coverage.
.github/​workflows/​build-android.yml Adds plugin-only Dawn build coverage.
.github/​scripts/​prepare-lite-parity.mjs Generates isolated native parity suites.
.github/​scripts/​lite-parity-tools.test.mjs Tests serialization and reporting helpers.
.github/​scripts/​lite-parity-reporter.mjs Records schedules, attempts and runner outcomes.
.github/​scripts/​lite-parity-records.mjs Summarizes cases and associates attempt metrics.
.github/​scripts/​lite-parity-fixture.mjs Adapts native and browser parity fixtures.
.github/​scripts/​diagnose-lite-parity.mjs Replays failed native scenes in browsers.
.github/​scripts/​complete-lite-parity.mjs Exercises remaining catalog scenes.
.github/​scripts/​capture-dawn-provenance.mjs Records source and executable fingerprints.
.github/​scripts/​bundle-lite-scene.mjs Bundles one Lite scene for native execution.
.github/​scripts/​bundle-lite-parity.mjs Bundles the Lite parity catalog.
.github/​instructions/​babylon-native-debugging.instructions.md Documents the readiness-timeout flag.

Review effort: Balanced

Comment on lines +23 to +25
const status = !invocationStatus || pending > 0 ? "IN PROGRESS / incomplete"
: cases.some(record => record.status !== "passed") ||
(!cases.length && invocationStatus !== "passed") ? "failed" : "passed";
Comment on lines +13 to +16
writeFileSync(join(this.output, "scheduled-tests.json"), JSON.stringify(
suite.allTests().map(test => ({
id: test.id, title: test.titlePath(), location: test.location,
})), null, 2));
Comment thread Embedding/Source/View.cpp
Comment on lines +155 to +156
m_runtime.m_appRuntime->Dispatch([lw, lh](Napi::Env env) {
Babylon::Plugins::NativeDawn::ResizeSurface(env, lw, lh);
std::shared_ptr<arcana::cancellation_source> m_cancellationSource{};
JsRuntimeScheduler m_runtimeScheduler;

std::unordered_map<const NativeCanvasImage*, int> m_nvgImageIndices;
.replace(/'\/textures\//g, "'" + pgRoot + "/textures/")
.replace(/"textures\//g, '"' + pgRoot + "/textures/")
.replace(/'textures\//g, "'" + pgRoot + "/textures/")
.replace(/\/scenes\//g, pgRoot + "/scenes/")
Comment on lines +128 to +137
std::string m_lineCap{}; // 'butt', 'round', 'square'
std::string m_lineJoin{}; // 'round', 'bevel', 'miter'

// Dash pattern from setLineDash. Retained only so getLineDash() round-trips;
// strokes are always drawn solid (nanovg has no dashed stroke).
std::vector<double> m_lineDash{};
std::string m_filter{};
std::string m_direction{"ltr"}; // 'ltr', 'rtl'
float m_miterLimit{0.f};
float m_lineWidth{0.f};
Comment on lines +156 to +163
struct SavedStyle
{
std::variant<std::string, GradientStyle> fillStyle;
std::variant<std::string, GradientStyle> strokeStyle;
Font font;
int currentFontId;
ShadowState shadow;
};
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants