Skip to content

V3: Animation clips for the 3D Model layer - #191

Merged
git-chad merged 124 commits into
git-chad/shader-lab-v3-planfrom
git-chad/v3-model-animation
Oct 1, 2026
Merged

git-chad merged 124 commits into
git-chad/shader-lab-v3-planfrom
git-chad/v3-model-animation

Conversation

@git-chad

Copy link
Copy Markdown
Collaborator

Step 3 of roadmap 4.4 (4.3 in Fase 4): animation clips for the 3D Model layer. It's stacked on the three.js upgrade (#190).

What you get

  • Clip picker. When a .glb has clips, the Model section shows Animation: Clip (each clip, All clips, None), Play, Speed (0–4×), Repeat (Loop, Ping-Pong, Once) and Start.
  • Sensible default. If each clip moves different objects (per-object actions exported from Blender), all of them play. Otherwise, the first one plays, so Idle/Walk/Run on one skeleton don't blend together.
  • Tied to the timeline. Clip time is Start + composition time × Speed, so the preview, PNG and video match frame by frame. While paused, the layer shows the Start frame.
  • Timeline length. Importing an animated model sets the timeline to the clip's length, the same way video does (the last import wins).
  • Stays framed. Framing, floor height and shadows are computed over 24 poses across the clip, so a moving model stays in frame and on its floor.
  • Everything follows the pose. Exact depth, shadows and the replacement materials follow skinned and morphed meshes. The Brushed Metal and Iridescent noise sticks to the rest pose, so it doesn't slide over a moving surface.
  • Saved with the project. Clip settings save, undo and duplicate like any other setting. The clip list comes from the file's glTF JSON, so older projects get it too.

Tests

  • model-animation.mjs adds 44 GPU checks: timing math, default clips, parsing, poses over time, clip-wide framing, animated depth, speed, pause, the repeat modes, Start, None, All clips, skinning, timeline seeding, save/reopen and exported frames.
  • model-layer-ui.mjs, run against the dev server, imports an animated .glb and checks the clip list, the timeline length and the Play toggle.
  • tsc, tsgo and lint are clean apart from the two existing warnings. The full composition suite passed on the upgrade commit below this one. The final full run on this stack was stopped at your request, so it hasn't been run on this exact commit.
  • All GPU runs used SwiftShader. Nothing was checked on a hardware GPU.

Manual test: the Animation section in tests/composition/MODEL-LAYER-MANUAL-QA.md.

🤖 Generated with Claude Code

@vercel

vercel Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
shader-lab Ready Ready Preview Oct 1, 2026 2:04pm UTC

@greptile-apps

greptile-apps Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 1/5

[Medium risk] Adds animation clip playback to the 3D model layer.

The PR is not ready to merge because valid animated models can have incorrect clip selection, timeline length, or framing.

Summary

Adds glTF animation clip discovery, playback controls, timeline-duration seeding, pose-aware model fitting, and composition tests. The clip picker has identifier collisions, some playable clips lose duration metadata, replacement skips timeline seeding, and sampled fitting can miss animated bounds.

Reviews (1) · Last reviewed commit: 8a3530a

Comment thread src/renderer/model-pass.ts Outdated
Comment thread src/lib/editor/model-animation.ts
Comment thread src/store/asset-store.ts
Comment thread src/lib/editor/model-animation.ts
Comment thread src/renderer/model-pass.ts
…roup failures

Media load callbacks now ignore replaced or removed passes and pending loads are cleared on disposal. Strict-mode pass failures are reported once at the originating pass instead of at every group boundary.
Publishing now drops every descendant of a hidden group and clears the
parentId of any retained layer (the audio source) whose parent was removed,
so the published scene passes hierarchy validation on reopen.
Cap the neighbor row and column search to two cells each side so the
worst-case cellAt count per fragment is 25 instead of unbounded in the
packed minimum-size, maximum-stroke configuration. Mirrored in the
runtime package.
…e on buttons

Clear the active paint layer when the layer stops being paintable (selection,
mode, visibility or lock change) so Paint never resumes without Edit Paint.
Only preventDefault on Space for non-interactive targets so focused buttons
and links still activate.
…est, video benchmark

- Image background keeps the input alpha instead of forcing opaque (editor and runtime passes)
- Duplication check edits the real mode param and asserts both layers
- Add a --dots scenario set (graph, blobs, plexus) to the effect video benchmark
…trols

Mesh drift is scaled to keep each point inside its cell, and each quad
splits along whichever diagonal is interior. Dot and link controls that
Mesh ignores are hidden, via a new oneOf visibility condition.
… style paper, duplicate test

- Un-premultiply ink color in transparent-paper output so alpha coverage is not applied twice (app and SDK pass)
- Show Angle for hatch and squiggle (new oneOf visibility condition); show Cross Angle only in hatch
- Plotter styles also set paper to color
- Duplication test now edits the real mode param and checks both layers
- Scale transparent-paper toner by input alpha so empty pixels stay clear
- Mix Amount in premultiplied space so partial amounts do not darken toner
- Skip the blur pyramid when Amount is zero
- Update a real parameter in the duplicate-independence check
- Mirror the pass changes into the React runtime
…ate test

Seed small shapes at their top-left tip so Cut Line and Cloud style them.
Raise the flood reach cap to 4096 so distant rings survive.
Let Scalloped draw inside for negative offsets.
Make the duplicate-independence check update a real parameter.
…terials, shadows, exact depth and Blender-style transforms

A new 3D Model source layer renders a .glb (or self-contained .gltf) into
its own HalfFloat MSAA target and composites it as a source over the layers
below. Draco, meshopt and KTX2 decoders are served from public/three. The
model is fitted to a tight vertex sphere with a margin; focal length changes
perspective without changing size.

Lighting: five CC0 Poly Haven studios or a custom .hdr, each normalized to
the same mean irradiance so switching studios keeps the exposure. The
environment goes through one persistent pmremTexture node set as
scene.environmentNode, because r183 bakes the environment texture node into
compiled materials and swapping scene.environment left them sampling the
disposed PMREM. A camera-relative key light casts PCF soft shadows onto an
invisible floor, and a blurred height pass under the model draws the
contact shadow. Tone mapping runs in the composite (Neutral by default).

Materials: the file's PBR materials or Chrome, Brushed Metal, Glass, Clay,
Rubber and Iridescent overrides. Anisotropy was dropped (flat white on
derivative tangents); iridescence is a view-angle spectral tint because
thin-film interference was invisible on metals.

Depth: an override pass writes near=1 depth normalized to the model's
bounding sphere, composited over any incoming scene depth, so Blur, Glass,
Relief, Input -> Depth and depth masks read the model, also inside groups.

Transform UX (asked for Blender basics): Location/Rotation/Scale as XYZ
fields with scrubbable axis letters, Move/Rotate/Scale gizmos, G/R/S modal
transforms with X/Y/Z locks, Alt+G/R/S clear, middle-drag orbit and
Shift+middle-drag shift. Modal keys are captured before other handlers so a
focused toolbar tooltip cannot swallow Escape.

Workarounds for three r183, found by tests:
- NodeMaterialObserver is shared per node builder state and its renderId
  fast path skips the world-matrix cache, so a mesh returning to an exact
  earlier pose kept a stale GPU matrix. A node-valued property on file
  materials makes the observer refresh every render.
- compileAsync allocates the transmission backdrop copy with the canvas
  format, breaking glass in HalfFloat targets; transmissive models compile
  on first draw instead.
- ShadowNode updates once per node frame per camera and frames only advance
  on three's own rAF, so the pass advances the node frame before each draw
  (hidden-tab exports and several draws per tick froze shadows).
- WebGPU only allows 1 or 4 samples; MSAA turns off above 4K so 8192 px
  exports stay within memory.

Exports wait up to 30 s for model and HDR loads.
The picker already uses the cube for 3D Model; the layer row fell back to
the generic image placeholder like effect layers do.
…, load tracking

- Offset the floor, contact shadow and key light target by Location Y
- Render the depth pass with per-material alpha-tested variants so cutouts stay empty
- Carry alpha test, alpha map, map alpha, transparency and depth write into material presets
- Track model and environment loads per request so stale loads cannot clear readiness
- Keep model and environment errors in separate slots and flag a missing custom HDR
three 0.183.2 -> 0.185.1 and @types/three 0.185.4. The published runtime now
asks for three ^0.185.0, so projects on 0.183 have to update three with it
(changeset included). @webgpu/types is now an explicit dev dependency
referenced from src/types/webgpu.d.ts, because @types/three stops pulling
it in with 0.186.

0.186.1, the latest release, was tried first and set aside. It has a TSL
codegen bug (three #34339, still open): a shared math value first read
inside an If/select branch is cached in a variable assigned only in that
branch, so every other read sees 0. It broke the Photographic Cells outline
at gap 0 and Focus Blur, and a scan of every WGSL shader the composition
suite generates found the same uninitialized reads in Glass, Connected Dots,
Photocopy and Plotter, with none on 0.185.1. r186 started counting usage
through ContextNode, so far more nodes get cached than before. Turning that
cache off inside branches and loops fixed Cells but made glass and blur
shaders compile several times slower and still failed a Focus Blur check,
so it is not worth shipping. r187 rewrites this cache; retry then with the
full suite and the WGSL scan.

The 3D layer no longer skips compileAsync for transmissive models: 0.183
still throws the transmission CopyTextureToTexture format error without the
skip, 0.185 does not. The exact-pose uniform refresh stays, since three only
fixes that in r186 (0.185 without it fails the pose-return test). The shadow
node-frame advance and the persistent environment node stay as well.

Visible changes: environment rotation follows three r184 (same direction as
rotating an object), so Studio Rotation turns the other way than on 0.183;
the layer is unreleased, so there is no migration. The r185 premultiplied-
alpha output change only touches transparent output pixels and the canvas
is opaque; all alpha checks pass.

three now reports uncaptured GPU errors through console.error. SwiftShader
cannot import decoded video frames, which produces the same 64 validation
errors on 0.183 (Chrome printed them as warnings, so they were never
counted). The composition suite now ignores exactly that ExternalTexture
chain, prints how many it skipped, and still fails on any other GPU error.

The roadmap records the new order the user asked for on September 29: three
upgrade, animation clips, SVG to 3D, and the path-traced Render mode last.
The shader reference lists every three/tsl export so the MCP can tell agents
which TSL functions exist. three 0.185 adds packNormalToRGB,
unpackRGBToNormal, transformNormalByInverseViewMatrix and the other names
from the r184 and r185 releases, so CI's snapshot check failed until the
generated file was refreshed.
Step 3 of roadmap 4.4 (4.3 in Fase 4). The Model section lists the clips of
a .glb, read straight from the glTF JSON (names and durations from the
animation input accessors), so the list shows without loading the model and
works for projects saved before this change.

Clip picks one clip, All clips plays every clip together, None shows the
rest pose. The automatic default plays every clip when each clip moves
different objects (per-object actions from Blender) and otherwise the first
one, so a character with Idle/Walk/Run on one skeleton does not blend them.

Play, Speed (0-4x), Repeat (Loop, Ping-Pong, Once) and Start map composition
time to clip time: Start + time * Speed, wrapped by the repeat mode, set on
each action and evaluated with mixer.update(0). That keeps preview, PNG and
video export identical frame by frame; paused shows the Start frame. The
clip settings are hidden registry params rendered in the Model section, so
they save, undo and duplicate like any other param. Importing an animated
model seeds the timeline length from the playing clips, like video.

Framing, floor height and shadows are computed over 24 poses spread across
the selected clips, so a moving model stays in frame and on its floor. The
depth pass, shadows and replacement materials follow skinned and morphed
meshes; Brushed Metal and Iridescent noise uses positionGeometry so it stays
on the surface instead of sliding while bones move. Floor sampling now
evaluates only the vertices it keeps.

model-animation.mjs adds 44 GPU checks: clip timing math, default clips,
parsing, poses over time, clip-wide framing, animated depth, speed, pause,
repeat modes, Start, None, All clips, skinning, timeline seeding, save and
reopen, and exported frames. The editor UI test imports an animated .glb
and checks the clip list, the timeline length and the Play toggle.
- Fit animated bounds from keyframe-aware pose samples with a larger per-pose vertex limit
- Give each clip a unique selection value so duplicate or reserved names resolve correctly
- Read the input accessor data when its max is missing so clip duration is not zero
- Seed the timeline duration when replacing a model
- Stop requesting renders once a Once clip has finished
@git-chad
git-chad force-pushed the git-chad/v3-three-upgrade branch from 08db20e to 30df7bb Compare October 1, 2026 14:03
@git-chad
git-chad force-pushed the git-chad/v3-model-animation branch from 9932ffb to 2f89143 Compare October 1, 2026 14:03
@git-chad
git-chad changed the base branch from git-chad/v3-three-upgrade to git-chad/shader-lab-v3-plan October 1, 2026 15:14
@git-chad
git-chad merged commit 2f89143 into git-chad/shader-lab-v3-plan Oct 1, 2026
4 of 5 checks passed

This branch was successfully deployed

1 active deployment
Preview — 2f89143e Deployed Oct 1, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant