Skip to content

Interactive static assets imported as SVG, MP3 or MP4 never load because the service worker rejects their data URLs #1796

Open

Description

@joshunrau

The interactive iframe's service worker cannot serve SVG, MP3 or MP4 static assets, though the bundler supports importing all three. instrument-bundler loads .svg, .mp3 and .mp4 with esbuild's dataurl loader, which (esbuild 0.28) produces data:image/svg+xml,<svg …> (percent-encoded text, not base64), data:audio/mpeg;base64,… and data:video/mp4;base64,…. worker.js only accepts data:image/*;base64,… and data:text/*,…, so it throws for all three. The throw happens before respondWith is called, so the request falls through to the network, which has no such file. An author who writes import tone from './tone.mp3' and staticAssets: { '/tone.mp3': tone }, as the instrument guidelines describe, gets audio that never plays and SVG stimuli that never appear, with nothing reported to the host.

Where

packages/runtime-internal/src/interactive/worker.js:22-51:

function dataUrlToResponse(dataUrl) {
  // Try base64 image format first
  const base64Match = /^data:(image\/[a-zA-Z+]+);base64,([A-Za-z0-9+/=]+)$/.exec(dataUrl);
  // ...
  // Try plain text format
  const textMatch = /^data:(text\/[a-zA-Z+]+),(.*)$/.exec(dataUrl);
  // ...
  throw new Error('Invalid or unsupported data URL: must be a base64-encoded image or plain text');
}

and the loader that produces those URLs, packages/instrument-bundler/src/utils.ts:24:

    .with(P.union('.jpeg', '.jpg', '.png', '.svg', '.webp', '.mp3', '.mp4'), () => 'dataurl' as const)

Reproduce

  1. In the playground, copy the "Interactive Instrument with Static Assets" example and add a tone.mp3 and an icon.svg.
  2. Import both (import tone from './tone.mp3', import icon from './icon.svg'), add them to staticAssets as '/tone.mp3' and '/icon.svg', and reference them from fragment.html (<audio src="tone.mp3" controls>, <img src="icon.svg">).
  3. Open the instrument in apps/web.

Actual: the service worker logs Invalid or unsupported data URL: must be a base64-encoded image or plain text for each request, both requests go to the server and fail, the audio element cannot play and the SVG image is broken. The PNG in the same instrument loads.
Expected: every data URL the bundler can produce for an import is served with its own MIME type.

Tests

packages/runtime-internal/src/interactive/__tests__/worker.test.ts: it('should serve every data URL the bundler emits, so svg, mp3 and mp4 static assets load'). Use the exact forms esbuild produces: data:image/svg+xml,<svg …>, data:audio/mpeg;base64,… and data:video/mp4;base64,…. Assert the Content-Type and the decoded body of each.

e2e, in testing/src/specs/instrument-completion.spec.ts (or a playground spec): render an interactive instrument whose staticAssets include an SVG and assert the image's naturalWidth is greater than 0.

Suggested fix

Parse any data:<mime>[;base64],<payload> generically. Take the MIME type from the URL, base64-decode when ;base64 is present, and otherwise decodeURIComponent the payload (esbuild's text form is percent-escaped). Then respond with those bytes. Keep throwing only for strings that are not data URLs at all.

Activity

  1. added
    BugType: existing behavior is wrong
    Difficulty: LowIsolated change, about 2 hours or less
    Area: InstrumentsInstrument authoring API, bundling, runtime, rendering, measures and the built-in library
    Good First IssueWell-scoped, low-risk and self-contained; a good entry point for a new contributor
    on Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Area: InstrumentsInstrument authoring API, bundling, runtime, rendering, measures and the built-in libraryBugType: existing behavior is wrongDifficulty: LowIsolated change, about 2 hours or lessGood First IssueWell-scoped, low-risk and self-contained; a good entry point for a new contributorPriority: MediumReal value; schedule normally

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions