Skip to content

fix(ios): render outline traits in XRLiveRealityKitSession - #3

Closed
shaaaaawn wants to merge 1 commit into
mainfrom
fix/ios-session-outline
Closed

shaaaaawn wants to merge 1 commit into
mainfrom
fix/ios-session-outline

Conversation

@shaaaaawn

@shaaaaawn shaaaaawn commented Nov 15, 2026 •

Copy link
Copy Markdown
Owner

Problem

userData.xr.outline never renders in RealityKit hosts that drive an XRLiveRealityKitSession. Outline shells are only created by RealityKitHelper.applyXRTraits → configureEntity → applyOutline, and session hosts don't call applyXRTraits; the iOS demo calls it only when no session can be created. On web, three-live/src/plugins/postprocessing.ts draws the outline whenever the trait is set.

On current main (9ea5712), the green GripperTarget orb in the Robotic Arm IK preset (outline: { color: '#22c55e', thickness: 0.025 }) shows no rim in the iOS demo. With this change it does.

Change

  • XRLiveRealityKitSession.bindEntities calls RealityKitHelper.applyOutline for each bound node whose session.traits(at:) has an enabled outline. It goes in node order: compiled GLBs list parents before children, so a child's own outline wins. It doesn't go through configureEntity, which would re-add the physics bodies bindEntities strips.
  • Shells are decoration, so the session keeps them out of scene-content paths (new internal RealityKitHelper.isOutlineShell):
    • bindEntities never binds a shell as a node.
    • recolor leaves the unlit rim alone on material changes (cycleMaterialColor, setMaterialUniform).
    • installInputTargets removes the collision that generateCollisionShapes(recursive: true) gives shells, so taps hit the model instead of the slightly larger rim.
    • XRSimRealityKitBridge.setFlash and color(of:) skip shells, so a hit flash can't swap in a material without front-face culling.

Validation

9ea5712 plus this commit, Xcode 27.2, Swift 6.4.2:

  • swift test (macOS): 111 tests pass. The new testRealityKitSessionRendersOutlineTraits checks:

    • the rim's color;
    • that a cycleMaterialColor select recolors the orb but not the rim;
    • that shells get no collision;
    • that re-binding neither duplicates shells nor changes the bound entities.

    It fails on unpatched main ("Session must add an outline shell to the outlined node").

  • xcodebuild test on an iPhone 17 Pro simulator (iOS 26.5): pass.

  • xcodebuild test on an Apple Vision Pro simulator (visionOS 27.2): pass, including the outline test, which runs the visionOS code path.

  • iOS demo, XR-Live → Robotic Arm IK: no rim on main, #22c55e rim with this change (below). With a temporary onSelect on the orb, a tap reports Tapped GripperTarget. Without the collision removal it reported Tapped __xr_outline_shell.

Robotic Arm IK in the iOS demo. Left, on main: the green orb has no outline. Right, with this PR: the orb has a bright green outline rim

visionOS: the same preset in a minimal volumetric RealityView app (not part of this PR) that binds it with XRLiveRealityKitSession. It steps the runtime a fixed 45 frames, so both captures share a pose.

Robotic Arm IK on the Apple Vision Pro simulator, visionOS 27.2. Left, on main: the green orb has no outline. Right, with this PR: the orb has a bright green outline rim

Scope

  • Only the RealityKit layer in mobile/ios changes. Web, Android, Rust and the trait schema are unchanged.
  • Not changed here:
    • Android doesn't render outline either (SceneCoreComponentApplier.onAttachOutline is a no-op hook).
    • The existing applyOutline rim is static, so pulse isn't animated, and skinned or blend-shape meshes keep their bind shape.
  • Shells need front-face culling (iOS 18 / macOS 15 / visionOS 2 SDKs). Older systems keep applyOutline's fallback.

@shaaaaawn
shaaaaawn force-pushed the fix/ios-session-outline branch from e4d9d60 to 2bc5620 Compare November 15, 2026 19:24
`userData.xr.outline` shells were only created by
`RealityKitHelper.applyXRTraits` -> `configureEntity`, which hosts that
drive an `XRLiveRealityKitSession` never call (the iOS demo only calls it
when no session can be created). So every outlined node, such as the
GripperTarget orb in the Robotic Arm IK preset, rendered with no rim on
iOS and visionOS.

The session now applies enabled `outline` traits to its bound entities
when it binds the scene, matching web, where the post-processing plugin
draws outlines whenever the trait is set. It reads the traits through
`session.traits(at:)`, like `enablePostProcessing` does for bloom, and
does not go through `configureEntity`, so it does not re-add the physics
bodies `bindEntities` strips or repeat `installInputTargets` work.

Now that sessions carry shells, keep them out of paths that would treat
them as scene content:
- `recolor` skips shells, so material changes (e.g. cycleMaterialColor)
  no longer repaint the rim with the model's color.
- `bindEntities` skips shells, so re-binding a hierarchy cannot bind a
  shell as a node's fallback mesh entity.
- `installInputTargets` removes the collision that
  `generateCollisionShapes(recursive: true)` gives shells, so taps hit
  the model instead of the enlarged rim (and hosts never see a shell as
  the hit entity).
- `XRSimRealityKitBridge.setFlash` and `color(of:)` skip shells, so a hit
  flash does not swap in a non-culled material that hides the model.

Adds `testRealityKitSessionRendersOutlineTraits`, which fails without the
`recolor` guard and checks that shells take no collision.
@shaaaaawn
shaaaaawn force-pushed the fix/ios-session-outline branch from 2bc5620 to 3b96968 Compare November 15, 2026 19:59
@shaaaaawn

Copy link
Copy Markdown
Owner Author

Merged upstream as google#102 (5585eb7).

@shaaaaawn shaaaaawn closed this Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant