From be01cdf4f152cfa1889f1d24b501d746d2cf7df3 Mon Sep 17 00:00:00 2001 From: Mick van Dijke <12992260+mickvandijke@users.noreply.github.com> Date: Thu, 22 Oct 2026 17:27:04 +0200 Subject: [PATCH 1/2] perf(media): start playback sooner and keep Firefox's worker running createMediaSource() opened its reader in the core's streaming mode, which starts fetching ahead, and the file's last record, alongside the first block a player waits for. On the live network that delayed the first frame. Media sources now open an ordinary reader, which still reads ahead once playback's reads follow each other, and keep priming the head and tail. The media service worker installs while the reader opens, and priming starts as soon as the reader is open rather than once the worker controls the page; errors surface in the same order as before. While any media source is open the page messages the worker every 10 s, because Firefox stops a worker that has had no new event for 30 s, even one still answering a request. The worker waits up to 120 s for a block, beyond the core's retries of a missing record after 15 and 45 s, and marks its responses nosniff and sandboxed, since their type comes from the file's metadata. Co-Authored-By: Claude Opus 5.5 --- README.md | 38 ++++++---- .../ADR-0003-adaptive-browser-downloads.md | 28 ++++++++ public/autonomi-stream-sw.js | 14 +++- src/client.ts | 31 ++++++--- src/internal/media.ts | 47 +++++++++++-- test/client.test.ts | 69 ++++++++++++++++++- test/limits.test.ts | 7 ++ test/media.test.ts | 41 +++++++++++ 8 files changed, 244 insertions(+), 31 deletions(-) diff --git a/README.md b/README.md index 056d41f..3907fde 100644 --- a/README.md +++ b/README.md @@ -957,7 +957,8 @@ source.close(); For audio and video content types, and common media extensions, creating the source also reads the first `SDK_LIMITS.mediaPrimeHeadBytes` (1 MiB) and the -last `SDK_LIMITS.mediaPrimeTailBytes` (64 KiB) at once, in the background. A +last `SDK_LIMITS.mediaPrimeTailBytes` (64 KiB) at once, in the background, as +soon as its reader is open and while the service worker is still installing. A player reads the header, then seeks to the index MP4 files keep at the end, and only then plays; with both already in the reader's cache, playback starts as soon as the element is attached instead of after two sequential network @@ -970,17 +971,26 @@ page-owned authenticated reader; it does not connect to Autonomi nodes itself. The page and client must therefore remain open while the media URL is in use. Multiple clients can serve independent media sources on the same page. -A media source's reader streams: from wherever playback starts or seeks to, the -shared core fetches up to four records of the next 32 MiB in parallel. Playback -therefore does not wait for each record's discovery. It also fetches the file's -last record when playback starts, because MP4 and WebM usually keep their index -there. It keeps at most about 24 of the file's records cached, releasing those -behind playback. Readers from `openFile()` read ahead only once a read -continues a previous read, so a single header read does not fetch the window. +A media source's reader is an ordinary random-access reader, like one from +`openFile()`: it reads ahead once a read continues a previous one, so playback +fetches the following records while it plays, but the first block a player +waits for has the connections to itself. The core's streaming mode, which +fetches up to four records of the next 32 MiB and the file's last record as soon +as playback starts, delayed the first frame on the live network, so media +sources no longer use it; priming reads the head and the tail instead. See +[ADR-0003](docs/adr/ADR-0003-adaptive-browser-downloads.md). + +While a media source is open, the page messages the service worker every 10 +seconds. Firefox stops a worker that has had no new event for 30 seconds, even +one still answering a request, which otherwise cut off a block that took longer +or a response the player read for that long. The worker waits up to two minutes +for a block, longer than the core's retries of a missing record, after 15 and 45 +seconds. If the application already has a root-scoped service worker, merge the `autonomi-file-range` fetch and message-handling logic from the packaged worker -into it and pass that worker's URL: +into it, including its `message` listener, which the keep-awake messages need, +and pass that worker's URL: ```ts const source = await client.createMediaSource(address, { @@ -1133,11 +1143,11 @@ console.log(SDK_LIMITS.maxFileBytes, SDK_LIMITS.maxRangeBytes); The limits and capability reports are immutable. Unsupported input sizes are rejected before copying upload byte arrays or staging Blobs. Reads learn the -verified size from the core; media setup checks it before registering a worker -or creating a URL. Uploads retain the upstream 1 GB cap. Buffered downloads need -sufficient memory and default to a 1 GB output budget; disk streams need -destination capacity, and upload staging needs quota for at least one encrypted -record. +verified size from the core; media setup checks it before creating a URL. Its +service worker installs while the reader opens. Uploads retain the upstream 1 GB +cap. Buffered downloads need sufficient memory and default to a 1 GB output +budget; disk streams need destination capacity, and upload staging needs quota +for at least one encrypted record. `getBrowserCapabilities()` safely runs outside a browser and makes no network requests, allocations of workers, or permission prompts. `features` reports each diff --git a/docs/adr/ADR-0003-adaptive-browser-downloads.md b/docs/adr/ADR-0003-adaptive-browser-downloads.md index f18dc5b..747fafe 100644 --- a/docs/adr/ADR-0003-adaptive-browser-downloads.md +++ b/docs/adr/ADR-0003-adaptive-browser-downloads.md @@ -143,3 +143,31 @@ attached afterwards is served from the reader's cache. The option `primePlayback` forces or disables it. The reads are never awaited and never fail the media source; a failure is left for the player to report. No core or wire change: the reads use the ordinary streaming reader. + +## Media readers open without streaming mode (2026-10-22) + +The first frame waits on one fetch: the record holding the start of the +file. The core's streaming mode starts fetching up to four records of the +next 32 MiB, and the file's last record, at the same moment, so those +compete with it for the same connections. `createMediaSource()` now opens +an ordinary reader, which reads ahead only once a read continues a previous +one, and keeps priming the head and the tail. It also stops waiting on +itself: the media service worker installs while the reader opens, and +priming starts as soon as the reader is open rather than once the worker +controls the page. Errors surface in the same order as before. + +Against the live network, ten alternating rounds in fresh Chromium contexts +with the same 54.9 MB video gave a median first frame of 6.6 s against +9.1 s, faster in 7 of 10 pairs; one baseline sample failed to load. Seeking +to the middle after ten seconds of playback slowed, median 6.5 s against +3.4 s: by then streaming mode had already fetched the window holding the +middle. The trade favours a fast start. A core change that serves the +first requested range before streaming mode's read-ahead would keep both. + +The page also messages the worker every 10 s while a media source is open, +because Firefox stops a worker that has had no new event for 30 s even while +it answers a request. The worker waits up to 120 s for a block, beyond the +core's retries of a missing record after 15 and 45 s, and marks its +responses `nosniff` and sandboxed, since their type comes from the file's +metadata. No core or wire change. Evidence is in the +[2026-10-22 audit](../audits/2026-10-22-media-first-frame.md). diff --git a/public/autonomi-stream-sw.js b/public/autonomi-stream-sw.js index fd0f3d1..c5180e0 100644 --- a/public/autonomi-stream-sw.js +++ b/public/autonomi-stream-sw.js @@ -1,11 +1,19 @@ const STREAM_MARKER = "/__autonomi_stream/"; // Keep standalone-worker limits aligned with SDK_LIMITS; test/limits.test.ts checks them. const STREAM_BLOCK_BYTES = 1024 * 1024; -const RANGE_REQUEST_TIMEOUT_MS = 45_000; +// A block can wait on a record the client must first find, and the shared core +// retries a missing record after 15 and again after 45 seconds, so a shorter +// wait gave up on blocks that were still coming. +const RANGE_REQUEST_TIMEOUT_MS = 120_000; const MAX_STREAM_FILE_BYTES = Number.MAX_SAFE_INTEGER; self.addEventListener("install", () => self.skipWaiting()); self.addEventListener("activate", (event) => event.waitUntil(self.clients.claim())); +// While a media source is open its page messages this worker every 10 s, only +// so that Firefox keeps it running: Firefox stops a worker that has had no new +// event for 30 s, even one still answering a request. Receiving the message is +// all it takes. +self.addEventListener("message", () => {}); self.addEventListener("fetch", (event) => { const url = new URL(event.request.url); @@ -52,6 +60,10 @@ async function handleStreamRequest(event, url) { "Content-Disposition": `inline; filename*=UTF-8''${encodeURIComponent(name)}`, "Content-Length": String(responseLength), "Content-Type": contentType, + // The type comes from the file's own metadata. Never let the bytes be taken + // for another type, or run as a page on this origin if opened directly. + "X-Content-Type-Options": "nosniff", + "Content-Security-Policy": "sandbox; default-src 'none'", }); if (partial) headers.set("Content-Range", `bytes ${start}-${end}/${size}`); if (event.request.method === "HEAD") { diff --git a/src/client.ts b/src/client.ts index 126bfa6..cea8ab8 100644 --- a/src/client.ts +++ b/src/client.ts @@ -891,22 +891,33 @@ export class AutonomiClient { try { this.#assertOpen(); report("Opening an Autonomi random-access media reader"); - // Playback reads sequentially from wherever it starts or seeks to. - reader = await this.#openReader(file, { + this.#media ??= new MediaBridge(); + // Nothing waits on anything it need not: the service worker installs while + // the reader opens, and the head and tail are read the moment it is open. + const worker = this.#media.prepare({ ...options, signal: operation.signal }); + // Awaited after the reader, so a failure surfaces in the same order as + // when one waited for the other. + worker.catch(() => undefined); + // The reader opens without streaming mode. That mode starts fetching ahead, + // and the file's last record, alongside the first block a player waits + // for, which on the live network delayed the first frame (ADR-0003); a + // plain reader still reads ahead once playback's reads follow each other. + const opened = await this.#openReader(file, { ...(options.onProgress ? { onProgress: options.onProgress } : {}), parentOperationId: operation.id, signal: operation.signal, - }, true); - this.#media ??= new MediaBridge(); - source = await this.#media.attach(reader, { + }, false); + reader = opened; + if (options.primePlayback ?? isPlayableMedia(opened)) { + report(`Priming playback of ${opened.name}`, { phase: "media" }); + // Not awaited: the reads only warm the reader's cache for the player. + void primePlayback(opened); + } + await worker; + source = await this.#media.attach(opened, { ...options, signal: operation.signal, }); - if (options.primePlayback ?? isPlayableMedia(reader)) { - report(`Priming playback of ${source.file.name}`, { phase: "media" }); - // Not awaited: the source is usable now and the reads only warm its cache. - void primePlayback(reader); - } report(`Media source ready for ${source.file.name}`, { phase: "complete" }); return source; } catch (error) { diff --git a/src/internal/media.ts b/src/internal/media.ts index b9623ce..98980ab 100644 --- a/src/internal/media.ts +++ b/src/internal/media.ts @@ -10,6 +10,16 @@ interface MediaSession { source: MediaSource; } +/** + * Firefox stops a service worker that has had no new event for 30 seconds, even + * one still answering a range request, at most 30 seconds later. Every message + * from the page is a new event, so the page sends one this often while any of + * its media sources is open. + */ +const KEEP_AWAKE_MS = 10_000; +/** What the page sends the worker to keep it running; the worker need only receive it. */ +const KEEP_AWAKE_MESSAGE = { type: "autonomi-stream-awake" } as const; + /** File extensions treated as playable media when the content type says nothing. */ const MEDIA_EXTENSION = /\.(mp4|m4v|m4a|webm|ogv|ogg|oga|mov|mp3|wav|flac|aac|mkv)$/i; @@ -38,14 +48,16 @@ export class MediaBridge { #workerUrl?: string; #scope?: string; #messageListenerAttached = false; + #keepAwake: ReturnType | undefined = undefined; constructor() {} - async attach(reader: PublicFileReader, options: MediaOptions): Promise { + /** + * Install the media service worker and wait until it controls the page. + * Independent of any file, so a caller can run it while the file opens. + */ + async prepare(options: Pick): Promise { throwIfAborted(options.signal); - if (!Number.isSafeInteger(reader.size) || reader.size < SDK_LIMITS.minFileBytes || reader.size > SDK_LIMITS.mediaMaxFileBytes) { - throw new AutonomiError("MEDIA_FAILED", `Media file size must be from ${SDK_LIMITS.minFileBytes} through ${SDK_LIMITS.mediaMaxFileBytes} bytes`); - } const { features } = getBrowserCapabilities(); if (!features.secureContext || !features.serviceWorker) { throw new AutonomiError("MEDIA_FAILED", "Media streaming requires a secure context (HTTPS; localhost is allowed) with service-worker support"); @@ -55,8 +67,20 @@ export class MediaBridge { location.href, ).href; const scope = normalizeScope(options.scope ?? "/"); + // Once this worker controls the page there is nothing left to do. + if (this.#workerUrl === workerUrl && this.#scope === scope && navigator.serviceWorker.controller?.scriptURL === workerUrl) return; await this.#ensureWorker(workerUrl, scope, options.signal); throwIfAborted(options.signal); + } + + async attach(reader: PublicFileReader, options: MediaOptions): Promise { + throwIfAborted(options.signal); + if (!Number.isSafeInteger(reader.size) || reader.size < SDK_LIMITS.minFileBytes || reader.size > SDK_LIMITS.mediaMaxFileBytes) { + throw new AutonomiError("MEDIA_FAILED", `Media file size must be from ${SDK_LIMITS.minFileBytes} through ${SDK_LIMITS.mediaMaxFileBytes} bytes`); + } + // Already done when the caller prepared the worker; checking again is cheap. + await this.prepare(options); + const scope = normalizeScope(options.scope ?? "/"); if (!this.#messageListenerAttached) { navigator.serviceWorker.addEventListener("message", this.#onMessage); this.#messageListenerAttached = true; @@ -81,10 +105,12 @@ export class MediaBridge { if (closed) return; closed = true; this.#sessions.delete(sessionId); + this.#updateKeepAwake(); reader.close(); }, }; this.#sessions.set(sessionId, { reader, source }); + this.#updateKeepAwake(); return source; } @@ -97,6 +123,7 @@ export class MediaBridge { } } this.#sessions.clear(); + this.#updateKeepAwake(); if (this.#messageListenerAttached) { navigator.serviceWorker.removeEventListener("message", this.#onMessage); this.#messageListenerAttached = false; @@ -189,6 +216,18 @@ export class MediaBridge { await waitForController(workerUrl, signal); } + // Messages the worker while any media source is open, and only then. + #updateKeepAwake(): void { + if (this.#sessions.size > 0 && this.#keepAwake === undefined) { + this.#keepAwake = setInterval(() => { + navigator.serviceWorker.controller?.postMessage(KEEP_AWAKE_MESSAGE); + }, KEEP_AWAKE_MS); + } else if (this.#sessions.size === 0 && this.#keepAwake !== undefined) { + clearInterval(this.#keepAwake); + this.#keepAwake = undefined; + } + } + #assertWorkerLocation(workerUrl: string, scope: string): void { if (this.#workerUrl && (this.#workerUrl !== workerUrl || this.#scope !== scope)) { throw new AutonomiError( diff --git a/test/client.test.ts b/test/client.test.ts index 3e7cb8f..28a66f1 100644 --- a/test/client.test.ts +++ b/test/client.test.ts @@ -1125,14 +1125,79 @@ describe("terminal operation events", () => { } }); + it("installs the media worker while the reader opens, and primes before the worker is ready", async () => { + const client = await AutonomiClient.connect(endpoint); + const raw = state.networks.at(-1)!; + const readRange = vi.fn(async () => new Uint8Array()); + let opened!: (reader: unknown) => void; + raw.openPublicFile.mockReturnValue(new Promise((resolve) => { opened = resolve; })); + const worker = { scriptURL: "https://app.example/autonomi-stream-sw.js" } as ServiceWorker; + const registration = { scope: "https://app.example/", active: worker, installing: null, waiting: null, update: vi.fn(), unregister: vi.fn() } as unknown as ServiceWorkerRegistration; + let registrations!: (value: ServiceWorkerRegistration[]) => void; + const getRegistrations = vi.fn(() => new Promise((resolve) => { registrations = resolve; })); + vi.stubGlobal("isSecureContext", true); + vi.stubGlobal("location", { href: "https://app.example/player", origin: "https://app.example" }); + vi.stubGlobal("navigator", { serviceWorker: { + getRegistrations, getRegistration: vi.fn(async () => registration), + register: vi.fn(async () => registration), ready: Promise.resolve(registration), controller: worker, + addEventListener: vi.fn(), removeEventListener: vi.fn(), + } }); + const settle = () => new Promise((resolve) => setTimeout(resolve, 0)); + try { + const creating = client.createMediaSource(file.address); + await settle(); + // Both started: neither the worker nor the reader waits for the other. + expect(getRegistrations).toHaveBeenCalledOnce(); + expect(raw.openPublicFile).toHaveBeenCalledWith(expect.anything(), expect.any(Function), { streaming: false }); + opened({ size: 50_000_000, name: "movie.mp4", contentType: "video/mp4", readRange, close: vi.fn(), free: vi.fn() }); + await settle(); + // The head and tail are being read while the worker is still installing. + expect(readRange.mock.calls).toEqual([ + [0, SDK_LIMITS.mediaPrimeHeadBytes], + [50_000_000 - SDK_LIMITS.mediaPrimeTailBytes, SDK_LIMITS.mediaPrimeTailBytes], + ]); + registrations([registration]); + const source = await creating; + expect(source.url).toContain("https://app.example/__autonomi_stream/"); + source.close(); + } finally { + vi.unstubAllGlobals(); + client.close(); + } + }); + + it("reports a failed media worker once the reader has opened, and closes the reader", async () => { + const client = await AutonomiClient.connect(endpoint); + const raw = state.networks.at(-1)!; + const close = vi.fn(); + let opened!: (reader: unknown) => void; + raw.openPublicFile.mockReturnValue(new Promise((resolve) => { opened = resolve; })); + // No service worker support: the worker fails at once, while the reader is opening. + vi.stubGlobal("isSecureContext", true); + vi.stubGlobal("navigator", {}); + try { + let settled = false; + const creating = client.createMediaSource(file.address).finally(() => { settled = true; }); + await new Promise((resolve) => setTimeout(resolve, 0)); + // The open-file step ends before the media step that started it. + expect(settled).toBe(false); + opened({ size: 50_000_000, name: "movie.mp4", contentType: "video/mp4", readRange: vi.fn(async () => new Uint8Array()), close, free: vi.fn() }); + await expect(creating).rejects.toMatchObject({ code: "MEDIA_FAILED" }); + expect(close).toHaveBeenCalledOnce(); + } finally { + vi.unstubAllGlobals(); + client.close(); + } + }); + it("ends both media setup and its open-file child when opening fails", async () => { const events: import("../src/types.js").ProgressEvent[] = []; const onProgress = (event: import("../src/types.js").ProgressEvent) => events.push(event); const client = await AutonomiClient.connect(endpoint, { onProgress }); events.length = 0; await expect(client.createMediaSource(file, { onProgress })).rejects.toMatchObject({ code: "OPEN_FILE_FAILED" }); - // Playback reads sequentially, so media opens a streaming reader that reads ahead. - expect(state.networks.at(-1)!.openPublicFile).toHaveBeenCalledWith(expect.anything(), expect.any(Function), { streaming: true }); + // Media opens a plain reader: streaming mode delayed the first frame (ADR-0003). + expect(state.networks.at(-1)!.openPublicFile).toHaveBeenCalledWith(expect.anything(), expect.any(Function), { streaming: false }); const terminal = events.filter((event) => event.status !== "running"); expect(terminal.map((event) => event.operation)).toEqual(["open-file", "media"]); expect(terminal[0]!.parentOperationId).toBe(terminal[1]!.operationId); diff --git a/test/limits.test.ts b/test/limits.test.ts index 50fbf1e..3b8ef9a 100644 --- a/test/limits.test.ts +++ b/test/limits.test.ts @@ -30,6 +30,13 @@ it("keeps standalone media-worker limits aligned with the exported SDK limits", expect(range.status).toBe(206); expect(range.headers.get("content-range")).toBe(`bytes ${start}-${end}/${size}`); expect(range.headers.get("content-length")).toBe("16"); + // The type comes from the file's metadata; the bytes never run as a page here. + expect(range.headers.get("x-content-type-options")).toBe("nosniff"); + expect(range.headers.get("content-security-policy")).toBe("sandbox; default-src 'none'"); + // The page's keep-awake messages need a listener to be events at all. + expect(handlers.has("message")).toBe(true); + // Outlasts the shared core's retries of a missing record, after 15 and 45 s. + expect(runInContext("RANGE_REQUEST_TIMEOUT_MS", context)).toBeGreaterThan(45_000 + 15_000); }); it("enforces the advertised range limit and stream default before calling the core", async () => { diff --git a/test/media.test.ts b/test/media.test.ts index 5d9fa26..a313331 100644 --- a/test/media.test.ts +++ b/test/media.test.ts @@ -100,6 +100,47 @@ describe("MediaBridge service-worker registration", () => { expect(listeners.size).toBe(0); }); + it("does nothing more once its worker controls the page", async () => { + const worker = serviceWorker(defaultWorkerUrl); + const sdkRegistration = registration(rootScopeUrl, worker); + const container = serviceWorkerContainer({ registrations: [sdkRegistration], ready: sdkRegistration, controller: worker }); + stubBrowser(container); + const bridge = new MediaBridge(); + await bridge.prepare({}); + await bridge.attach(fileReader(), {}); + await bridge.prepare({}); + expect(container.getRegistrations).toHaveBeenCalledOnce(); + bridge.close(); + }); + + // Firefox stops a worker that has had no new event for 30 s, even mid-answer. + it("messages the worker every 10 s while a media source is open, and only then", async () => { + vi.useFakeTimers(); + try { + const postMessage = vi.fn(); + const worker = { scriptURL: defaultWorkerUrl, postMessage } as unknown as ServiceWorker; + const sdkRegistration = registration(rootScopeUrl, worker); + stubBrowser(serviceWorkerContainer({ registrations: [sdkRegistration], ready: sdkRegistration, controller: worker })); + const bridge = new MediaBridge(); + const first = await bridge.attach(fileReader(), {}); + const second = await bridge.attach(fileReader(), {}); + await vi.advanceTimersByTimeAsync(30_000); + expect(postMessage).toHaveBeenCalledTimes(3); + expect(postMessage).toHaveBeenCalledWith({ type: "autonomi-stream-awake" }); + first.close(); + await vi.advanceTimersByTimeAsync(10_000); + expect(postMessage).toHaveBeenCalledTimes(4); + second.close(); + await vi.advanceTimersByTimeAsync(30_000); + expect(postMessage).toHaveBeenCalledTimes(4); + await bridge.attach(fileReader(), {}); + bridge.close(); + await vi.advanceTimersByTimeAsync(30_000); + expect(postMessage).toHaveBeenCalledTimes(4); + } finally { + vi.useRealTimers(); + } + }); }); function stubBrowser(container: ServiceWorkerContainer): void { From a4f4e8c807ae98d64979791a568c559a89850874 Mon Sep 17 00:00:00 2001 From: Mick van Dijke <12992260+mickvandijke@users.noreply.github.com> Date: Thu, 22 Oct 2026 17:27:04 +0200 Subject: [PATCH 2/2] docs: record the media first-frame measurements Ten alternating live rounds of createMediaSource() with try's featured video, a real-Firefox check of the worker keep-awake with lowered idle limits, playback through the packaged worker in Chromium, Firefox and WebKit, and the SDK checks. Co-Authored-By: Claude Opus 5.5 --- docs/audits/2026-10-22-media-first-frame.md | 109 ++++ .../2026-10-22-media-first-frame/adr.log.gz | Bin 0 -> 74 bytes .../2026-10-22-media-first-frame/bridge.html | 2 + .../checksums.json | 11 + .../comparison-1.json | 478 ++++++++++++++++++ .../comparison-2.json | 448 ++++++++++++++++ .../2026-10-22-media-first-frame/drive.mjs | 49 ++ .../example-builds.log.gz | Bin 0 -> 692 bytes .../firefox-keepalive.jsonl | 2 + .../firefox-keepalive.mjs | 56 ++ .../2026-10-22-media-first-frame/harness.js | 56 ++ .../2026-10-22-media-first-frame/index.html | 8 + .../2026-10-22-media-first-frame/pack.json.gz | Bin 0 -> 1758 bytes .../2026-10-22-media-first-frame/pack.log.gz | Bin 0 -> 168 bytes .../playback.jsonl | 3 + .../2026-10-22-media-first-frame/playback.mjs | 40 ++ .../pr-checker.log.gz | Bin 0 -> 594 bytes .../sdk-check.log.gz | Bin 0 -> 667 bytes .../verify-wasm.log.gz | Bin 0 -> 239 bytes 19 files changed, 1262 insertions(+) create mode 100644 docs/audits/2026-10-22-media-first-frame.md create mode 100644 docs/audits/2026-10-22-media-first-frame/adr.log.gz create mode 100644 docs/audits/2026-10-22-media-first-frame/bridge.html create mode 100644 docs/audits/2026-10-22-media-first-frame/checksums.json create mode 100644 docs/audits/2026-10-22-media-first-frame/comparison-1.json create mode 100644 docs/audits/2026-10-22-media-first-frame/comparison-2.json create mode 100644 docs/audits/2026-10-22-media-first-frame/drive.mjs create mode 100644 docs/audits/2026-10-22-media-first-frame/example-builds.log.gz create mode 100644 docs/audits/2026-10-22-media-first-frame/firefox-keepalive.jsonl create mode 100644 docs/audits/2026-10-22-media-first-frame/firefox-keepalive.mjs create mode 100644 docs/audits/2026-10-22-media-first-frame/harness.js create mode 100644 docs/audits/2026-10-22-media-first-frame/index.html create mode 100644 docs/audits/2026-10-22-media-first-frame/pack.json.gz create mode 100644 docs/audits/2026-10-22-media-first-frame/pack.log.gz create mode 100644 docs/audits/2026-10-22-media-first-frame/playback.jsonl create mode 100644 docs/audits/2026-10-22-media-first-frame/playback.mjs create mode 100644 docs/audits/2026-10-22-media-first-frame/pr-checker.log.gz create mode 100644 docs/audits/2026-10-22-media-first-frame/sdk-check.log.gz create mode 100644 docs/audits/2026-10-22-media-first-frame/verify-wasm.log.gz diff --git a/docs/audits/2026-10-22-media-first-frame.md b/docs/audits/2026-10-22-media-first-frame.md new file mode 100644 index 0000000..e7b652b --- /dev/null +++ b/docs/audits/2026-10-22-media-first-frame.md @@ -0,0 +1,109 @@ +# Media first frame: plain reader, no serial waits, Firefox keep-awake — 2026-10-22 + +[try.autonomi.com](https://github.com/WithAutonomi/try/pull/18), which uses the +same core directly, started its video sooner than `createMediaSource()`. This +change brings what it does differently into the SDK, measured with the SDK's +own media path. + +## Changes + +- `createMediaSource()` opens its reader without the core's streaming mode. + That mode starts fetching up to four records of the next 32 MiB, and the + file's last record, alongside the first block a player waits for. The plain + reader still reads ahead once a read continues a previous one. +- The media service worker installs while the reader opens, and the head and + tail are primed as soon as the reader is open, instead of each waiting for + the one before. An open failure still surfaces first; a worker failure + surfaces once the reader has opened, which is then closed. +- While any media source is open the page messages the worker every 10 s, and + the worker listens. Firefox stops a worker that has had no new event for + 30 s, even one still answering a request. +- The worker waits up to 120 s for a block instead of 45 s: the core retries a + missing record after 15 and after 45 s. Its responses carry + `X-Content-Type-Options: nosniff` and `Content-Security-Policy: sandbox; + default-src 'none'`, since their type comes from the file's metadata. + +No public type, core or wire change. Applications that merged the packaged +worker into their own should merge its new `message` listener too. + +## Revisions + +- Baseline: SDK `8c00e820feea25d344c9b83bad8466740c52fbdb` (PR #7, 0.1.3). +- Candidate: this branch, built from the same commit plus this change; the + built `client.js`, `internal/media.js` and worker hashes in + [`checksums.json`](2026-10-22-media-first-frame/checksums.json) are those + measured. +- WASM in both: ant-client `8b19c1499360ebd8166a1ef3dd65b88db914dd7f`, + `0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8`. +- Video: try's featured 54,903,201-byte MP4, read from its manifest + (`content/manifests/video.ant`, embedded root map), so no DataMap is fetched. +- macOS 26.5.2 arm64; Node 22.23.1; Playwright 1.62.1; Chromium + 151.0.7922.34, Firefox 153.0, WebKit 26.5. + +## First frame on the live network + +[`harness.js`](2026-10-22-media-first-frame/harness.js), identical in both +builds, connects with `AutonomiClient.connect()` on mainnet defaults when the +page opens, waits 3 s, then creates a media source for the video, plays it +muted, and records the first decoded frame (`requestVideoFrameCallback`), ten +seconds of playback, a seek to the middle and every read the reader served. +[`drive.mjs`](2026-10-22-media-first-frame/drive.mjs) ran ten alternating rounds, +each in a fresh Chromium context. + +| Build | First frame (s) | Median | Seek to middle, median | +| --- | --- | ---: | ---: | +| Baseline | 6.1, 11.0, 5.8, 9.1, 7.3, 10.6, failed, 11.3, 8.4, 9.6 | 9.1 | 3.4 s | +| Candidate | 7.2, 4.5, 13.3, 5.2, 5.6, 6.7, 9.6, 6.5, 7.7, 5.4 | 6.6 | 6.5 s | + +The candidate was faster in 7 of 10 paired rounds. One baseline sample failed: +the player reported no supported source. Every other sample played at least +ten seconds in ten, with two `waiting` events each. + +In every sample the first frame is the read of the first 1 MiB, which waits +for the file's first record; every block after it came from the reader's cache +in about 30 ms. The spread is that one record's fetch on the live network, +which was slower in the afternoon than the morning's try runs, plausibly +because the day's runs made it hot for its holders. + +The seek is slower because streaming mode had, after ten seconds, already +fetched the window holding the middle of this file; the plain reader fetches +it when asked. The change favours a fast start. A core change serving the +first requested range before streaming mode's read-ahead would keep both. + +The same comparison in try's own page, with the same core, gave a median +first frame of 5.0 s with a plain reader against 8.4 s in streaming mode, +three rounds each. + +## Firefox + +[`firefox-keepalive.mjs`](2026-10-22-media-first-frame/firefox-keepalive.mjs) +([results](2026-10-22-media-first-frame/firefox-keepalive.jsonl)) lowers +Firefox's worker idle limits to 15 s and 1 s and serves, through the +SDK's `MediaBridge` and packaged worker, a reader that takes 22 s per block. The +baseline answered `206` with an empty body after 15 s: the worker was stopped +mid-answer. The candidate answered the ten requested bytes after 22 s, after +two keep-awake messages, and sent none once the source closed. + +[`playback.mjs`](2026-10-22-media-first-frame/playback.mjs) plays a test-pattern +MP4 through the same bridge and worker in Chromium, Firefox and WebKit +([results](2026-10-22-media-first-frame/playback.jsonl)): each +played past half a second, with the new response headers present. + +## Automated validation + +- `npm run check`: build, TypeScript and **262 tests in 23 files passed**. New + tests: the worker installs while the reader opens and priming starts before + it is ready, with a plain reader; a failed worker surfaces once the reader has + opened and closes it; the bridge does nothing more once its worker controls + the page; keep-awake messages every 10 s only while a source is open; the + worker's message listener, timeout and headers. Five of them fail against the + baseline source; the failed-worker ordering test passes on both and guards + the new parallel setup. +- `npm run build:examples`, `npm pack --dry-run` (97 files), `npm run + verify:wasm`, ADR governance (8 ADRs) and all 52 PR-checker cases passed. + +## Limits + +Ten samples per build from one machine and network, Chromium only for live +timings, one video. Live node load and versions are unknown. The seek trade-off +above stands until the core change it names. diff --git a/docs/audits/2026-10-22-media-first-frame/adr.log.gz b/docs/audits/2026-10-22-media-first-frame/adr.log.gz new file mode 100644 index 0000000000000000000000000000000000000000..71c935d1aef06a7934e6132795e5069300594635 GIT binary patch literal 74 zcmV-Q0JZ-giwFqqD%xrS17T!xE^KdS0CRK+Qb^A)OD)Pv%u7yHC`c?WPEAqJuuyOW gN~C4xq-qpvDkNv5CTFLnXzFnR0G*%D`!4_h0N;fheE +Media bridge only diff --git a/docs/audits/2026-10-22-media-first-frame/checksums.json b/docs/audits/2026-10-22-media-first-frame/checksums.json new file mode 100644 index 0000000..2320a6a --- /dev/null +++ b/docs/audits/2026-10-22-media-first-frame/checksums.json @@ -0,0 +1,11 @@ +{ + "/site-baseline/sdk/client.js": "1b62355d096b83ab69e70b5f85670a16957646cd8ae2d114a4ee0fe590161d8b", + "/site-baseline/sdk/internal/media.js": "59f151a25423dc74065bb9243df47f5fb862d4786bf56bf7b82928d0e10e2581", + "/site-baseline/autonomi-stream-sw.js": "7862bcd999b201383224313a675c512299399a1a10d3c428468617033d359683", + "/site-candidate/sdk/client.js": "8fce06261e2eb86e5fba64a7df1ca66dce73c085ae26a81725909cb2ad63bb33", + "/site-candidate/sdk/internal/media.js": "b3fc5174f1d5af8ed0696278639edc93c52a243fbb3b8cd535f98f489686d6e4", + "/site-candidate/autonomi-stream-sw.js": "d92e3d61913e6433103e54ee82aaf4910395ca74c5759e95ab342239a5e0ea06", + "/site-candidate/sdk/wasm/ant_core_bg.wasm": "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8", + "/site-candidate/video.ant": "cea3ab028b454d6ff00e26119e3f46f0a57f3bcbbf9b9794c67fbc37d1aba7f4", + "/site-baseline/harness.js": "33d49a9672c2f3e206f4ffb5552e429fba4955e71e0cf98307d9f98f5c0208ce" +} diff --git a/docs/audits/2026-10-22-media-first-frame/comparison-1.json b/docs/audits/2026-10-22-media-first-frame/comparison-1.json new file mode 100644 index 0000000..26cb3b0 --- /dev/null +++ b/docs/audits/2026-10-22-media-first-frame/comparison-1.json @@ -0,0 +1,478 @@ +{ + "startedAt": "2026-10-22T15:12:34.351Z", + "browser": "151.0.7922.34", + "variants": [ + { + "name": "baseline", + "url": "http://127.0.0.1:39830" + }, + { + "name": "candidate", + "url": "http://127.0.0.1:39831" + } + ], + "runs": 5, + "scriptSha256": "78b1f3be7b8b1573e35be54fb3a368427a527fa47e5951ac0012fdb876689e93", + "samples": [ + { + "round": 1, + "variant": "baseline", + "waiting": 2, + "sourceReadyMs": 17.400000005960464, + "firstFrameMs": 6077.39999999702, + "readsBeforeFirstFrame": [ + { + "start": 0, + "length": 1048576, + "at": 9, + "ms": 5961 + }, + { + "start": 54837665, + "length": 65536, + "at": 9, + "ms": -1 + }, + { + "start": 0, + "length": 1048576, + "at": 21, + "ms": 5949 + }, + { + "start": 1048576, + "length": 1048576, + "at": 5971, + "ms": 27 + }, + { + "start": 2097152, + "length": 1048576, + "at": 6005, + "ms": 29 + }, + { + "start": 3145728, + "length": 1048576, + "at": 6044, + "ms": -1 + } + ], + "playedIn10s": 9.958324, + "seekMs": 3345.699999997021, + "status": "passed", + "pageErrors": [], + "wasm": [ + "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8" + ] + }, + { + "round": 1, + "variant": "candidate", + "waiting": 2, + "sourceReadyMs": 17.299999997019768, + "firstFrameMs": 7189.299999991059, + "readsBeforeFirstFrame": [ + { + "start": 0, + "length": 1048576, + "at": 3, + "ms": 7127 + }, + { + "start": 54837665, + "length": 65536, + "at": 3, + "ms": 2203 + }, + { + "start": 0, + "length": 1048576, + "at": 19, + "ms": 7111 + }, + { + "start": 1048576, + "length": 1048576, + "at": 7132, + "ms": 31 + }, + { + "start": 2097152, + "length": 1048576, + "at": 7166, + "ms": 28 + } + ], + "playedIn10s": 9.990967999999999, + "seekMs": 9631.49999999702, + "status": "passed", + "pageErrors": [], + "wasm": [ + "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8" + ] + }, + { + "round": 2, + "variant": "candidate", + "waiting": 2, + "sourceReadyMs": 17, + "firstFrameMs": 4464.79999999404, + "readsBeforeFirstFrame": [ + { + "start": 0, + "length": 1048576, + "at": 3, + "ms": 4404 + }, + { + "start": 54837665, + "length": 65536, + "at": 3, + "ms": 2884 + }, + { + "start": 0, + "length": 1048576, + "at": 19, + "ms": 4388 + }, + { + "start": 1048576, + "length": 1048576, + "at": 4408, + "ms": 30 + }, + { + "start": 2097152, + "length": 1048576, + "at": 4440, + "ms": 28 + } + ], + "playedIn10s": 9.986528999999999, + "seekMs": 2636.7000000029802, + "status": "passed", + "pageErrors": [], + "wasm": [ + "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8" + ] + }, + { + "round": 2, + "variant": "baseline", + "waiting": 2, + "sourceReadyMs": 17.400000005960464, + "firstFrameMs": 11030.900000002981, + "readsBeforeFirstFrame": [ + { + "start": 0, + "length": 1048576, + "at": 10, + "ms": 10965 + }, + { + "start": 54837665, + "length": 65536, + "at": 10, + "ms": 3919 + }, + { + "start": 0, + "length": 1048576, + "at": 21, + "ms": 10954 + }, + { + "start": 1048576, + "length": 1048576, + "at": 10975, + "ms": 29 + }, + { + "start": 2097152, + "length": 1048576, + "at": 11006, + "ms": 29 + } + ], + "playedIn10s": 9.986566, + "seekMs": 1559.199999994038, + "status": "passed", + "pageErrors": [], + "wasm": [ + "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8" + ] + }, + { + "round": 3, + "variant": "baseline", + "waiting": 2, + "sourceReadyMs": 21.399999991059303, + "firstFrameMs": 5830.199999997019, + "readsBeforeFirstFrame": [ + { + "start": 0, + "length": 1048576, + "at": 12, + "ms": 5763 + }, + { + "start": 54837665, + "length": 65536, + "at": 12, + "ms": 3238 + }, + { + "start": 0, + "length": 1048576, + "at": 25, + "ms": 5750 + }, + { + "start": 1048576, + "length": 1048576, + "at": 5776, + "ms": 27 + }, + { + "start": 2097152, + "length": 1048576, + "at": 5805, + "ms": 29 + } + ], + "playedIn10s": 9.989638999999999, + "seekMs": 7418.4000000059605, + "status": "passed", + "pageErrors": [], + "wasm": [ + "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8" + ] + }, + { + "round": 3, + "variant": "candidate", + "waiting": 2, + "sourceReadyMs": 23.200000002980232, + "firstFrameMs": 13271.29999999106, + "readsBeforeFirstFrame": [ + { + "start": 0, + "length": 1048576, + "at": 6, + "ms": 13208 + }, + { + "start": 54837665, + "length": 65536, + "at": 6, + "ms": 3499 + }, + { + "start": 0, + "length": 1048576, + "at": 25, + "ms": 13189 + }, + { + "start": 1048576, + "length": 1048576, + "at": 13215, + "ms": 29 + }, + { + "start": 2097152, + "length": 1048576, + "at": 13245, + "ms": 26 + } + ], + "playedIn10s": 9.984762, + "seekMs": 7879, + "status": "passed", + "pageErrors": [], + "wasm": [ + "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8" + ] + }, + { + "round": 4, + "variant": "candidate", + "waiting": 2, + "sourceReadyMs": 23, + "firstFrameMs": 5232.700000000001, + "readsBeforeFirstFrame": [ + { + "start": 0, + "length": 1048576, + "at": 6, + "ms": 5210 + }, + { + "start": 54837665, + "length": 65536, + "at": 6, + "ms": 2794 + }, + { + "start": 0, + "length": 1048576, + "at": 25, + "ms": 5190 + }, + { + "start": 1048576, + "length": 1048576, + "at": 5216, + "ms": 31 + } + ], + "playedIn10s": 9.959088, + "seekMs": 2456.400000002981, + "status": "passed", + "pageErrors": [], + "wasm": [ + "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8" + ] + }, + { + "round": 4, + "variant": "baseline", + "waiting": 2, + "sourceReadyMs": 18.700000002980232, + "firstFrameMs": 9135.399999994039, + "readsBeforeFirstFrame": [ + { + "start": 0, + "length": 1048576, + "at": 10, + "ms": 9068 + }, + { + "start": 54837665, + "length": 65536, + "at": 11, + "ms": 3538 + }, + { + "start": 0, + "length": 1048576, + "at": 22, + "ms": 9056 + }, + { + "start": 1048576, + "length": 1048576, + "at": 9079, + "ms": 29 + }, + { + "start": 2097152, + "length": 1048576, + "at": 9113, + "ms": 27 + } + ], + "playedIn10s": 9.985812000000001, + "seekMs": 4147.199999988079, + "status": "passed", + "pageErrors": [], + "wasm": [ + "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8" + ] + }, + { + "round": 5, + "variant": "baseline", + "waiting": 2, + "sourceReadyMs": 21.399999991059303, + "firstFrameMs": 7326.799999999999, + "readsBeforeFirstFrame": [ + { + "start": 0, + "length": 1048576, + "at": 10, + "ms": 7256 + }, + { + "start": 54837665, + "length": 65536, + "at": 11, + "ms": 4314 + }, + { + "start": 0, + "length": 1048576, + "at": 26, + "ms": 7240 + }, + { + "start": 1048576, + "length": 1048576, + "at": 7271, + "ms": 29 + }, + { + "start": 2097152, + "length": 1048576, + "at": 7302, + "ms": 28 + } + ], + "playedIn10s": 9.982589999999998, + "seekMs": 4634.300000008941, + "status": "passed", + "pageErrors": [], + "wasm": [ + "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8" + ] + }, + { + "round": 5, + "variant": "candidate", + "waiting": 2, + "sourceReadyMs": 19.5, + "firstFrameMs": 5565.6, + "readsBeforeFirstFrame": [ + { + "start": 0, + "length": 1048576, + "at": 4, + "ms": 5507 + }, + { + "start": 54837665, + "length": 65536, + "at": 4, + "ms": 1274 + }, + { + "start": 0, + "length": 1048576, + "at": 21, + "ms": 5489 + }, + { + "start": 1048576, + "length": 1048576, + "at": 5511, + "ms": 29 + }, + { + "start": 2097152, + "length": 1048576, + "at": 5542, + "ms": 27 + } + ], + "playedIn10s": 9.993248000000001, + "seekMs": 4231.5999999940395, + "status": "passed", + "pageErrors": [], + "wasm": [ + "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8" + ] + } + ] +} diff --git a/docs/audits/2026-10-22-media-first-frame/comparison-2.json b/docs/audits/2026-10-22-media-first-frame/comparison-2.json new file mode 100644 index 0000000..f09cb43 --- /dev/null +++ b/docs/audits/2026-10-22-media-first-frame/comparison-2.json @@ -0,0 +1,448 @@ +{ + "startedAt": "2026-10-22T15:18:43.796Z", + "browser": "151.0.7922.34", + "variants": [ + { + "name": "candidate", + "url": "http://127.0.0.1:39831" + }, + { + "name": "baseline", + "url": "http://127.0.0.1:39830" + } + ], + "runs": 5, + "scriptSha256": "78b1f3be7b8b1573e35be54fb3a368427a527fa47e5951ac0012fdb876689e93", + "samples": [ + { + "round": 1, + "variant": "candidate", + "waiting": 2, + "sourceReadyMs": 20.299999997019768, + "firstFrameMs": 6709.10000000298, + "readsBeforeFirstFrame": [ + { + "start": 0, + "length": 1048576, + "at": 4, + "ms": 6589 + }, + { + "start": 54837665, + "length": 65536, + "at": 4, + "ms": -1 + }, + { + "start": 0, + "length": 1048576, + "at": 22, + "ms": 6570 + }, + { + "start": 1048576, + "length": 1048576, + "at": 6593, + "ms": 30 + }, + { + "start": 2097152, + "length": 1048576, + "at": 6624, + "ms": 28 + }, + { + "start": 3145728, + "length": 1048576, + "at": 6653, + "ms": -1 + } + ], + "playedIn10s": 9.958454, + "seekMs": 14960.79999999702, + "status": "passed", + "pageErrors": [], + "wasm": [ + "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8" + ] + }, + { + "round": 1, + "variant": "baseline", + "waiting": 2, + "sourceReadyMs": 17.599999994039536, + "firstFrameMs": 10609.799999991059, + "readsBeforeFirstFrame": [ + { + "start": 0, + "length": 1048576, + "at": 8, + "ms": 10546 + }, + { + "start": 54837665, + "length": 65536, + "at": 8, + "ms": 8222 + }, + { + "start": 0, + "length": 1048576, + "at": 21, + "ms": 10533 + }, + { + "start": 1048576, + "length": 1048576, + "at": 10555, + "ms": 27 + }, + { + "start": 2097152, + "length": 1048576, + "at": 10584, + "ms": 27 + } + ], + "playedIn10s": 9.989044999999999, + "seekMs": 8765.899999994042, + "status": "passed", + "pageErrors": [], + "wasm": [ + "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8" + ] + }, + { + "round": 2, + "variant": "baseline", + "status": "failed", + "error": "page.evaluate: NotSupportedError: Failed to load because no supported source was found.", + "pageErrors": [], + "wasm": [ + "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8" + ] + }, + { + "round": 2, + "variant": "candidate", + "waiting": 2, + "sourceReadyMs": 16.299999997019768, + "firstFrameMs": 9634.09999999404, + "readsBeforeFirstFrame": [ + { + "start": 0, + "length": 1048576, + "at": 3, + "ms": 9571 + }, + { + "start": 54837665, + "length": 65536, + "at": 3, + "ms": 9337 + }, + { + "start": 0, + "length": 1048576, + "at": 18, + "ms": 9556 + }, + { + "start": 1048576, + "length": 1048576, + "at": 9576, + "ms": 31 + }, + { + "start": 2097152, + "length": 1048576, + "at": 9609, + "ms": 30 + } + ], + "playedIn10s": 9.980536, + "seekMs": 11478.299999994037, + "status": "passed", + "pageErrors": [], + "wasm": [ + "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8" + ] + }, + { + "round": 3, + "variant": "candidate", + "waiting": 2, + "sourceReadyMs": 20.400000005960464, + "firstFrameMs": 6483.800000002981, + "readsBeforeFirstFrame": [ + { + "start": 0, + "length": 1048576, + "at": 4, + "ms": 6423 + }, + { + "start": 54837665, + "length": 65536, + "at": 4, + "ms": 3557 + }, + { + "start": 0, + "length": 1048576, + "at": 23, + "ms": 6405 + }, + { + "start": 1048576, + "length": 1048576, + "at": 6428, + "ms": 28 + }, + { + "start": 2097152, + "length": 1048576, + "at": 6458, + "ms": 25 + } + ], + "playedIn10s": 9.987957999999999, + "seekMs": 4106.79999999404, + "status": "passed", + "pageErrors": [], + "wasm": [ + "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8" + ] + }, + { + "round": 3, + "variant": "baseline", + "waiting": 2, + "sourceReadyMs": 22.299999997019768, + "firstFrameMs": 11295.70000000894, + "readsBeforeFirstFrame": [ + { + "start": 0, + "length": 1048576, + "at": 12, + "ms": 11227 + }, + { + "start": 54837665, + "length": 65536, + "at": 13, + "ms": 2954 + }, + { + "start": 0, + "length": 1048576, + "at": 26, + "ms": 11213 + }, + { + "start": 1048576, + "length": 1048576, + "at": 11240, + "ms": 28 + }, + { + "start": 2097152, + "length": 1048576, + "at": 11270, + "ms": 29 + } + ], + "playedIn10s": 9.986059000000001, + "seekMs": 65, + "status": "passed", + "pageErrors": [], + "wasm": [ + "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8" + ] + }, + { + "round": 4, + "variant": "baseline", + "waiting": 2, + "sourceReadyMs": 22.299999997019768, + "firstFrameMs": 8426.20000000596, + "readsBeforeFirstFrame": [ + { + "start": 0, + "length": 1048576, + "at": 12, + "ms": 8359 + }, + { + "start": 54837665, + "length": 65536, + "at": 12, + "ms": 4328 + }, + { + "start": 0, + "length": 1048576, + "at": 26, + "ms": 8344 + }, + { + "start": 1048576, + "length": 1048576, + "at": 8372, + "ms": 27 + }, + { + "start": 2097152, + "length": 1048576, + "at": 8401, + "ms": 26 + } + ], + "playedIn10s": 9.987123, + "seekMs": 3379.800000002979, + "status": "passed", + "pageErrors": [], + "wasm": [ + "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8" + ] + }, + { + "round": 4, + "variant": "candidate", + "waiting": 2, + "sourceReadyMs": 22.600000008940697, + "firstFrameMs": 7738.20000000298, + "readsBeforeFirstFrame": [ + { + "start": 0, + "length": 1048576, + "at": 6, + "ms": 7674 + }, + { + "start": 54837665, + "length": 65536, + "at": 6, + "ms": 2597 + }, + { + "start": 0, + "length": 1048576, + "at": 25, + "ms": 7655 + }, + { + "start": 1048576, + "length": 1048576, + "at": 7680, + "ms": 31 + }, + { + "start": 2097152, + "length": 1048576, + "at": 7713, + "ms": 28 + } + ], + "playedIn10s": 9.988992, + "seekMs": 5214.9999999970205, + "status": "passed", + "pageErrors": [], + "wasm": [ + "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8" + ] + }, + { + "round": 5, + "variant": "candidate", + "waiting": 2, + "sourceReadyMs": 21.799999997019768, + "firstFrameMs": 5396.5, + "readsBeforeFirstFrame": [ + { + "start": 0, + "length": 1048576, + "at": 4, + "ms": 5335 + }, + { + "start": 54837665, + "length": 65536, + "at": 4, + "ms": 2784 + }, + { + "start": 0, + "length": 1048576, + "at": 24, + "ms": 5315 + }, + { + "start": 1048576, + "length": 1048576, + "at": 5340, + "ms": 29 + }, + { + "start": 2097152, + "length": 1048576, + "at": 5371, + "ms": 27 + } + ], + "playedIn10s": 9.989819, + "seekMs": 7699.20000000596, + "status": "passed", + "pageErrors": [], + "wasm": [ + "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8" + ] + }, + { + "round": 5, + "variant": "baseline", + "waiting": 2, + "sourceReadyMs": 21.599999994039536, + "firstFrameMs": 9580.70000000298, + "readsBeforeFirstFrame": [ + { + "start": 0, + "length": 1048576, + "at": 12, + "ms": 9491 + }, + { + "start": 54837665, + "length": 65536, + "at": 12, + "ms": 6105 + }, + { + "start": 0, + "length": 1048576, + "at": 25, + "ms": 9477 + }, + { + "start": 1048576, + "length": 1048576, + "at": 9522, + "ms": 31 + }, + { + "start": 2097152, + "length": 1048576, + "at": 9570, + "ms": 28 + } + ], + "playedIn10s": 9.997207000000001, + "seekMs": 2686.4000000059605, + "status": "passed", + "pageErrors": [], + "wasm": [ + "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8" + ] + } + ] +} diff --git a/docs/audits/2026-10-22-media-first-frame/drive.mjs b/docs/audits/2026-10-22-media-first-frame/drive.mjs new file mode 100644 index 0000000..0b03b44 --- /dev/null +++ b/docs/audits/2026-10-22-media-first-frame/drive.mjs @@ -0,0 +1,49 @@ +// Alternating fresh-context samples of the harness on each served variant. +// Usage: node drive.mjs --out results.json --runs 3 --variant name=url ... +import { chromium } from "/Users/mick/RustroverProjects/try-autonomi-manifests/node_modules/playwright/index.mjs"; +import { mkdir, readFile, writeFile } from "node:fs/promises"; +import { createHash } from "node:crypto"; +import { dirname, resolve } from "node:path"; +import { parseArgs } from "node:util"; + +const { values } = parseArgs({ options: { variant: { type: "string", multiple: true }, runs: { type: "string", default: "3" }, out: { type: "string" }, browser: { type: "string", default: "chromium" } } }); +const variants = values.variant.map(item => { const [name, url] = item.split("="); return { name, url }; }); +const browser = await chromium.launch({ headless: true, args: ["--force-fieldtrials=WebRTC-NoSdpMangleUfrag/Enabled/", "--autoplay-policy=no-user-gesture-required"] }); +const report = { startedAt: new Date().toISOString(), browser: browser.version(), variants, runs: Number(values.runs), + scriptSha256: createHash("sha256").update(await readFile(new URL(import.meta.url))).digest("hex"), samples: [] }; +const out = resolve(values.out); +await mkdir(dirname(out), { recursive: true }); +try { + for (let round = 1; round <= report.runs; round++) { + for (const variant of round % 2 ? variants : [...variants].reverse()) { + const context = await browser.newContext(); + const page = await context.newPage(); + const errors = []; + const wasm = new Set(); + page.on("pageerror", error => errors.push(error.message)); + page.on("response", async response => { if (new URL(response.url()).pathname.endsWith(".wasm")) wasm.add(createHash("sha256").update(await response.body()).digest("hex")); }); + const sample = { round, variant: variant.name }; + try { + await page.goto(variant.url); + await page.waitForFunction(() => window.__harness && window.__harness.state !== "connecting", null, { timeout: 60_000 }); + if (await page.evaluate(() => window.__harness.state) !== "connected") throw Error(await page.evaluate(() => window.__harness.error)); + // As a visitor reads the page: the remaining seeds connect in the background. + await page.waitForTimeout(3000); + Object.assign(sample, await page.evaluate(() => window.__harness.run({ manifestUrl: "./video.ant" }))); + sample.status = sample.playedIn10s >= 5 ? "passed" : "failed"; + } catch (error) { + sample.status = "failed"; + sample.error = String(error.message ?? error); + } finally { + sample.pageErrors = errors; + sample.wasm = [...wasm]; + await context.close(); + } + console.log(JSON.stringify(sample)); + report.samples.push(sample); + await writeFile(out, JSON.stringify(report, null, 2) + "\n"); + } + } +} finally { + await browser.close(); +} diff --git a/docs/audits/2026-10-22-media-first-frame/example-builds.log.gz b/docs/audits/2026-10-22-media-first-frame/example-builds.log.gz new file mode 100644 index 0000000000000000000000000000000000000000..9ea9a9bf5b1055e5856aaafc47e01f2fce1c3f06 GIT binary patch literal 692 zcmV;l0!#fLiwFqhD%xrS17&z&ZE$R5En;M^X@CIzAm}g1CSWASW+YK6N!dx3y+XU}>2i`%vJ+Gm6R0pZ)k+$QkHg{9XmW&( z*Gw%*qbgpNY)ZItu0^$$R5&urk7t0vL$#pS4J)!kdPmB-pmK7AcwHjVaMTS}bIqkww-pYRi_3|+gou}~)dc_n)s~B8xH}e@^ptN9=D@6HTs!Nw9k*feksMIfMJbE6d8B7G_85JFS+3;Mp6Vj@z z8`Z`Z53EL#JfXF1t_1R$6r4>?qL_j;A0(q z8BV06xhbKA)?!67=eN;hek+->%QjV^|Ye zA`~1XNeF@8&(S?_BMdkk1MlkKyisTJb5J{wO4%!b-m0MhGYF&c@RR)fbU~J%o;JR{ zd>lhCfaAb}#B1T51Fc00F^)XfE2VA0;+bW^s9I7XN6LjE#-`e8?FY5pg7*=tkFY3J zA8b)5BRKkU?#ydEkST)LH^~&ZLz%D}T9+w^6U#EK$wGJMaW1(qGrANT&TGTnN2MOY zqDeijMUi^kG-b6jpWigIZ}M_Sqpp%ekJ{BdJSyKLP~QAvS?++nI0QU6A&kRqzB_(; ziT!35+4i#*Y(*4xa(X|y<0Ydl{hPjzs|JomV-0kR-o8kmtN8x?&Dm_w1^vz6W2Xzc z5Qe5M82ksn)RORq6i!X<+cZYE!^HRzV{cgGodx!B&5W|Bn|T3?#=W+{*UKPf7pv1< a>0H;;+5&&mNC(pSTDm{G*9 { + const { MediaBridge } = await import("./sdk/internal/media.js"); + const bytes = Uint8Array.from({ length: 4096 }, (_, index) => index & 255); + let awake = 0; + const post = ServiceWorker.prototype.postMessage; + ServiceWorker.prototype.postMessage = function (message, ...rest) { + if (message?.type === "autonomi-stream-awake") awake++; + return post.call(this, message, ...rest); + }; + const reader = { + address: "ab".repeat(32), name: "slow.mp4", size: bytes.length, contentType: "video/mp4", + async read(start, length) { + await new Promise(resolve => setTimeout(resolve, 22_000)); + return bytes.slice(start, start + length); + }, + close() {}, + }; + const bridge = new MediaBridge(); + const source = await bridge.attach(reader, {}); + const started = performance.now(); + let answer; + try { + const response = await fetch(source.url, { headers: { Range: "bytes=100-109" } }); + answer = { status: response.status, body: [...new Uint8Array(await response.arrayBuffer())] }; + } catch (error) { + answer = { error: String(error) }; + } + answer.seconds = (performance.now() - started) / 1000; + source.close(); + const whileOpen = awake; + await new Promise(resolve => setTimeout(resolve, 12_000)); + bridge.close(); + return { answer, whileOpen, afterClose: awake - whileOpen }; + }); + results.push({ name, browser: browser.version(), ...result }); + console.log(JSON.stringify(results.at(-1))); + } finally { + await browser.close(); + } +} diff --git a/docs/audits/2026-10-22-media-first-frame/harness.js b/docs/audits/2026-10-22-media-first-frame/harness.js new file mode 100644 index 0000000..0c4b79c --- /dev/null +++ b/docs/audits/2026-10-22-media-first-frame/harness.js @@ -0,0 +1,56 @@ +// The SDK's own createMediaSource() against the live network: connect on load, +// then on request stream a manifest entry and time its first frame, playback +// and a seek. window.__harness drives it from Playwright. +import { AutonomiClient, PublicFileReader, decodeManifest } from "./sdk/index.js"; + +// Every read the reader serves, for the timeline. +const reads = []; +const read = PublicFileReader.prototype.read; +PublicFileReader.prototype.read = async function (start, length, ...rest) { + const entry = { start, length, at: performance.now() }; + reads.push(entry); + try { return await read.call(this, start, length, ...rest); } finally { entry.ms = performance.now() - entry.at; } +}; + +const harness = window.__harness = { state: "connecting", error: null }; +const video = document.getElementById("player"); +let client; +try { + client = await AutonomiClient.connect(); + harness.state = "connected"; +} catch (error) { + harness.state = "failed"; + harness.error = String(error?.message ?? error); +} + +harness.run = async ({ manifestUrl }) => { + const entry = (await decodeManifest(new Uint8Array(await (await fetch(manifestUrl)).arrayBuffer()))).entries[0]; + const result = { waiting: 0 }; + reads.length = 0; + video.addEventListener("waiting", () => result.waiting++); + const started = performance.now(); + const source = await client.createMediaSource(entry); + result.sourceReadyMs = performance.now() - started; + const first = new Promise(resolve => video.requestVideoFrameCallback(at => resolve(at - started))); + video.src = source.url; + await video.play(); + result.firstFrameMs = await first; + result.readsBeforeFirstFrame = reads.map(entry => ({ start: entry.start, length: entry.length, at: Math.round(entry.at - started), ms: Math.round(entry.ms ?? -1) })); + const playbackStart = video.currentTime; + await new Promise(resolve => setTimeout(resolve, 10_000)); + result.playedIn10s = video.currentTime - playbackStart; + const target = video.duration / 2; + const seekStarted = performance.now(); + const seek = new Promise(resolve => { + const onFrame = (at, metadata) => (Math.abs(metadata.mediaTime - target) < 1 ? resolve(at - seekStarted) : video.requestVideoFrameCallback(onFrame)); + video.requestVideoFrameCallback(onFrame); + }); + video.currentTime = target; + await video.play(); + result.seekMs = await Promise.race([seek, new Promise((_, reject) => setTimeout(() => reject(new Error("seek frame deadline")), 60_000))]); + video.pause(); + video.removeAttribute("src"); + video.load(); + source.close(); + return result; +}; diff --git a/docs/audits/2026-10-22-media-first-frame/index.html b/docs/audits/2026-10-22-media-first-frame/index.html new file mode 100644 index 0000000..72da4af --- /dev/null +++ b/docs/audits/2026-10-22-media-first-frame/index.html @@ -0,0 +1,8 @@ + + +SDK media first frame + + + + + diff --git a/docs/audits/2026-10-22-media-first-frame/pack.json.gz b/docs/audits/2026-10-22-media-first-frame/pack.json.gz new file mode 100644 index 0000000000000000000000000000000000000000..dd37e6de4f2b1227a24ea9e680bf3fbdf0a9e2ca GIT binary patch literal 1758 zcmV<41|j($iwFqqD%xrS18`wuYc6VYZ*Bm!TH8+CI2gX?DI%`6DhaV2A5y7OSzrsx z7FcGGimIwIiJ3{695(harPV&g-tN=wldLo63^quOhZ{7`@9(oe|Ni~^u;X1_pL^Z| z3SdCJ{c9;|xu0jr{r=Ths=9m-^<`at6^WAjyiopaT~4%(FVlNJ7iq@*yR4SVzpHsJ zc6Pk~{{6>OLaP00d^^e#pI2GX{0@}#ej(FO@|6d3xn`R1ir;3VOl3l@jM2gvwTvtN zm>Xrv;8D1(sJTU#RsBZS+j*Th)nQ?X0+|*^nO*BciBj+?7{T=lP(TL za$l91Zn#ust=san25r@78k`k_l^I<$q-{6K@=UGC1{Y?`C|fNOkyYx&G0}EqolYAo z4B((Ga?Xn}ANfzj1HjyxdR(Y1UjxR=&b50tbukkj2wQf|80fz7bbXAM9zmeNB}D3v zt5M0*HTZBj*rcsqGje)nteGys0c>kA18szECeNd>@i1;Th!66$X#$pGLdmAg+1=FY zX^Uap#!N<8Ybe;#lAyN6LJciliviwfr_9&vRx>ohpfU8OraVbRr42N!HyaYKtW;JO zv!dS7nOAWsh5lJiODdk+>lu-H+U11jp)hK^WykAlbniDkveNS)npH}*6QWlQmHulC znr{TIJ`yB46$7eA0A3rp?^V z(%m=S?z>5*hJ38bqRg|+Q2H(mK!A4lW)%t@N?)w9FxEaMb&neR7KqHgHy{vWgd#dG z92bq&BJIsS#v#Qx#)}$5F30&42;YWr3IIqb5`c;bpfqAEiABN^l0=-|=|K|7f~%Pv zgE~`>O=S&(0rc+&CzBiY`lC9$IaB?EdwzI+bB*5**xV086<5Os@Q>5O&tZgQE|}L7p0LDF`X~S>QC&(|;+kP7DOkeK`rfN&^$oUFbyA zN(gj(mPVNnbw^_C2LjY7MUeB#Bv;H>H=0?fw+KZDiCn-NCBk|dP&eonbz-efH+e)| zm-RD#6eAb#MtJ}TnHzA6Ixy}y)#i)8MI06b7w8ska0K0;+r&W>c5DB7T50p22r*|I zMxkOxLx&6&u>cFhxD$=es>L^(OfZUFpc@5Z6gh#mD1&fBJL2oTL%~P;{I;y^aT;Nl zEMG070Nj#TG>c-)+z`!nfVRm87;#Bs3*y8vi!ESU%m~4yA zx&{E~h_WsRwUE}elEBz8ky%5;-1FEXUqm8jT&mO?YMXdE)eapqnFAmL%sG!mJqD;F zZ*LvO+4IDe3(F*0%Bx%Cb_8y$9<)-fa?_YB1_}GqCwa z#3mcWu~Qz~GT@rUcJ>E5F5@q_8G^(KyhWytJF^%43L)GrA}$}J*z~~^yCJYd^sWJM z4d12_4N*LsGLaKx>+Mz4 z@!)2@WoSa2E%Ae><00kjotO0}o;U!v>{S7918%*zKs4$|AoJ#l)0gHA*q7Tu&=eR| zib>adVfBLvKRLZR@-Gjr`ZxahMgR2hxEH8z%KVV3!)?Xr0M_9^QUY&(WoB}%WDHo#!n80MiX> AD*ylh literal 0 HcmV?d00001 diff --git a/docs/audits/2026-10-22-media-first-frame/pack.log.gz b/docs/audits/2026-10-22-media-first-frame/pack.log.gz new file mode 100644 index 0000000000000000000000000000000000000000..be5ffafac4ffd06ff38b0bcabc86ddf4b9038bd1 GIT binary patch literal 168 zcmV;Z09XGXiwFqlD%xrS18`wuYc6bWX8?_lI}XDj5JY=Uv6VX@QF`Pi2f=`3;_wl> z#LDesh~l(#yQ7(>8Kz*~J*~P@iYa)hBk`rA9bMsQxA`Kj;s&|6tj)$0#2ld12xfK1 z&)8W0>;9*g>Yo;-=@jUlT{rzia!AAYj%)a29SDi0_)I9fEW!haL1&G_PiTPinC`)k WH|ez%ynH*_GR8h2mlK0y0RRA`YEt|F literal 0 HcmV?d00001 diff --git a/docs/audits/2026-10-22-media-first-frame/playback.jsonl b/docs/audits/2026-10-22-media-first-frame/playback.jsonl new file mode 100644 index 0000000..1bb44e5 --- /dev/null +++ b/docs/audits/2026-10-22-media-first-frame/playback.jsonl @@ -0,0 +1,3 @@ +{"browser":"chromium","version":"151.0.7922.34","played":0.511293,"status":206,"nosniff":"nosniff","csp":"sandbox; default-src 'none'"} +{"browser":"firefox","version":"153.0","played":0.528215,"status":206,"nosniff":"nosniff","csp":"sandbox; default-src 'none'"} +{"browser":"webkit","version":"26.5","played":0.5045102076654525,"status":206,"nosniff":"nosniff","csp":"sandbox; default-src 'none'"} diff --git a/docs/audits/2026-10-22-media-first-frame/playback.mjs b/docs/audits/2026-10-22-media-first-frame/playback.mjs new file mode 100644 index 0000000..942c89b --- /dev/null +++ b/docs/audits/2026-10-22-media-first-frame/playback.mjs @@ -0,0 +1,40 @@ +// A real