Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 24 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -957,7 +957,8 @@ source.close();

For audio and video content types, and common media extensions, creating the
source also reads the first `SDK_LIMITS.mediaPrimeHeadBytes` (1 MiB) and the
last `SDK_LIMITS.mediaPrimeTailBytes` (64 KiB) at once, in the background. A
last `SDK_LIMITS.mediaPrimeTailBytes` (64 KiB) at once, in the background, as
soon as its reader is open and while the service worker is still installing. A
player reads the header, then seeks to the index MP4 files keep at the end,
and only then plays; with both already in the reader's cache, playback starts
as soon as the element is attached instead of after two sequential network
Expand All @@ -970,17 +971,26 @@ page-owned authenticated reader; it does not connect to Autonomi nodes itself.
The page and client must therefore remain open while the media URL is in use.
Multiple clients can serve independent media sources on the same page.

A media source's reader streams: from wherever playback starts or seeks to, the
shared core fetches up to four records of the next 32 MiB in parallel. Playback
therefore does not wait for each record's discovery. It also fetches the file's
last record when playback starts, because MP4 and WebM usually keep their index
there. It keeps at most about 24 of the file's records cached, releasing those
behind playback. Readers from `openFile()` read ahead only once a read
continues a previous read, so a single header read does not fetch the window.
A media source's reader is an ordinary random-access reader, like one from
`openFile()`: it reads ahead once a read continues a previous one, so playback
fetches the following records while it plays, but the first block a player
waits for has the connections to itself. The core's streaming mode, which
fetches up to four records of the next 32 MiB and the file's last record as soon
as playback starts, delayed the first frame on the live network, so media
sources no longer use it; priming reads the head and the tail instead. See
[ADR-0003](docs/adr/ADR-0003-adaptive-browser-downloads.md).

While a media source is open, the page messages the service worker every 10
seconds. Firefox stops a worker that has had no new event for 30 seconds, even
one still answering a request, which otherwise cut off a block that took longer
or a response the player read for that long. The worker waits up to two minutes
for a block, longer than the core's retries of a missing record, after 15 and 45
seconds.

If the application already has a root-scoped service worker, merge the
`autonomi-file-range` fetch and message-handling logic from the packaged worker
into it and pass that worker's URL:
into it, including its `message` listener, which the keep-awake messages need,
and pass that worker's URL:

```ts
const source = await client.createMediaSource(address, {
Expand Down Expand Up @@ -1133,11 +1143,11 @@ console.log(SDK_LIMITS.maxFileBytes, SDK_LIMITS.maxRangeBytes);

The limits and capability reports are immutable. Unsupported input sizes are
rejected before copying upload byte arrays or staging Blobs. Reads learn the
verified size from the core; media setup checks it before registering a worker
or creating a URL. Uploads retain the upstream 1 GB cap. Buffered downloads need
sufficient memory and default to a 1 GB output budget; disk streams need
destination capacity, and upload staging needs quota for at least one encrypted
record.
verified size from the core; media setup checks it before creating a URL. Its
service worker installs while the reader opens. Uploads retain the upstream 1 GB
cap. Buffered downloads need sufficient memory and default to a 1 GB output
budget; disk streams need destination capacity, and upload staging needs quota
for at least one encrypted record.

`getBrowserCapabilities()` safely runs outside a browser and makes no network
requests, allocations of workers, or permission prompts. `features` reports each
Expand Down
28 changes: 28 additions & 0 deletions docs/adr/ADR-0003-adaptive-browser-downloads.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,3 +143,31 @@ attached afterwards is served from the reader's cache. The option
`primePlayback` forces or disables it. The reads are never awaited and
never fail the media source; a failure is left for the player to report.
No core or wire change: the reads use the ordinary streaming reader.

## Media readers open without streaming mode (2026-10-22)

The first frame waits on one fetch: the record holding the start of the
file. The core's streaming mode starts fetching up to four records of the
next 32 MiB, and the file's last record, at the same moment, so those
compete with it for the same connections. `createMediaSource()` now opens
an ordinary reader, which reads ahead only once a read continues a previous
one, and keeps priming the head and the tail. It also stops waiting on
itself: the media service worker installs while the reader opens, and
priming starts as soon as the reader is open rather than once the worker
controls the page. Errors surface in the same order as before.

Against the live network, ten alternating rounds in fresh Chromium contexts
with the same 54.9 MB video gave a median first frame of 6.6 s against
9.1 s, faster in 7 of 10 pairs; one baseline sample failed to load. Seeking
to the middle after ten seconds of playback slowed, median 6.5 s against
3.4 s: by then streaming mode had already fetched the window holding the
middle. The trade favours a fast start. A core change that serves the
first requested range before streaming mode's read-ahead would keep both.

The page also messages the worker every 10 s while a media source is open,
because Firefox stops a worker that has had no new event for 30 s even while
it answers a request. The worker waits up to 120 s for a block, beyond the
core's retries of a missing record after 15 and 45 s, and marks its
responses `nosniff` and sandboxed, since their type comes from the file's
metadata. No core or wire change. Evidence is in the
[2026-10-22 audit](../audits/2026-10-22-media-first-frame.md).
109 changes: 109 additions & 0 deletions docs/audits/2026-10-22-media-first-frame.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Media first frame: plain reader, no serial waits, Firefox keep-awake — 2026-10-22

[try.autonomi.com](https://github.com/WithAutonomi/try/pull/18), which uses the
same core directly, started its video sooner than `createMediaSource()`. This
change brings what it does differently into the SDK, measured with the SDK's
own media path.

## Changes

- `createMediaSource()` opens its reader without the core's streaming mode.
That mode starts fetching up to four records of the next 32 MiB, and the
file's last record, alongside the first block a player waits for. The plain
reader still reads ahead once a read continues a previous one.
- The media service worker installs while the reader opens, and the head and
tail are primed as soon as the reader is open, instead of each waiting for
the one before. An open failure still surfaces first; a worker failure
surfaces once the reader has opened, which is then closed.
- While any media source is open the page messages the worker every 10 s, and
the worker listens. Firefox stops a worker that has had no new event for
30 s, even one still answering a request.
- The worker waits up to 120 s for a block instead of 45 s: the core retries a
missing record after 15 and after 45 s. Its responses carry
`X-Content-Type-Options: nosniff` and `Content-Security-Policy: sandbox;
default-src 'none'`, since their type comes from the file's metadata.

No public type, core or wire change. Applications that merged the packaged
worker into their own should merge its new `message` listener too.

## Revisions

- Baseline: SDK `8c00e820feea25d344c9b83bad8466740c52fbdb` (PR #7, 0.1.3).
- Candidate: this branch, built from the same commit plus this change; the
built `client.js`, `internal/media.js` and worker hashes in
[`checksums.json`](2026-10-22-media-first-frame/checksums.json) are those
measured.
- WASM in both: ant-client `8b19c1499360ebd8166a1ef3dd65b88db914dd7f`,
`0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8`.
- Video: try's featured 54,903,201-byte MP4, read from its manifest
(`content/manifests/video.ant`, embedded root map), so no DataMap is fetched.
- macOS 26.5.2 arm64; Node 22.23.1; Playwright 1.62.1; Chromium
151.0.7922.34, Firefox 153.0, WebKit 26.5.

## First frame on the live network

[`harness.js`](2026-10-22-media-first-frame/harness.js), identical in both
builds, connects with `AutonomiClient.connect()` on mainnet defaults when the
page opens, waits 3 s, then creates a media source for the video, plays it
muted, and records the first decoded frame (`requestVideoFrameCallback`), ten
seconds of playback, a seek to the middle and every read the reader served.
[`drive.mjs`](2026-10-22-media-first-frame/drive.mjs) ran ten alternating rounds,
each in a fresh Chromium context.

| Build | First frame (s) | Median | Seek to middle, median |
| --- | --- | ---: | ---: |
| Baseline | 6.1, 11.0, 5.8, 9.1, 7.3, 10.6, failed, 11.3, 8.4, 9.6 | 9.1 | 3.4 s |
| Candidate | 7.2, 4.5, 13.3, 5.2, 5.6, 6.7, 9.6, 6.5, 7.7, 5.4 | 6.6 | 6.5 s |

The candidate was faster in 7 of 10 paired rounds. One baseline sample failed:
the player reported no supported source. Every other sample played at least
ten seconds in ten, with two `waiting` events each.

In every sample the first frame is the read of the first 1 MiB, which waits
for the file's first record; every block after it came from the reader's cache
in about 30 ms. The spread is that one record's fetch on the live network,
which was slower in the afternoon than the morning's try runs, plausibly
because the day's runs made it hot for its holders.

The seek is slower because streaming mode had, after ten seconds, already
fetched the window holding the middle of this file; the plain reader fetches
it when asked. The change favours a fast start. A core change serving the
first requested range before streaming mode's read-ahead would keep both.

The same comparison in try's own page, with the same core, gave a median
first frame of 5.0 s with a plain reader against 8.4 s in streaming mode,
three rounds each.

## Firefox

[`firefox-keepalive.mjs`](2026-10-22-media-first-frame/firefox-keepalive.mjs)
([results](2026-10-22-media-first-frame/firefox-keepalive.jsonl)) lowers
Firefox's worker idle limits to 15 s and 1 s and serves, through the
SDK's `MediaBridge` and packaged worker, a reader that takes 22 s per block. The
baseline answered `206` with an empty body after 15 s: the worker was stopped
mid-answer. The candidate answered the ten requested bytes after 22 s, after
two keep-awake messages, and sent none once the source closed.

[`playback.mjs`](2026-10-22-media-first-frame/playback.mjs) plays a test-pattern
MP4 through the same bridge and worker in Chromium, Firefox and WebKit
([results](2026-10-22-media-first-frame/playback.jsonl)): each
played past half a second, with the new response headers present.

## Automated validation

- `npm run check`: build, TypeScript and **262 tests in 23 files passed**. New
tests: the worker installs while the reader opens and priming starts before
it is ready, with a plain reader; a failed worker surfaces once the reader has
opened and closes it; the bridge does nothing more once its worker controls
the page; keep-awake messages every 10 s only while a source is open; the
worker's message listener, timeout and headers. Five of them fail against the
baseline source; the failed-worker ordering test passes on both and guards
the new parallel setup.
- `npm run build:examples`, `npm pack --dry-run` (97 files), `npm run
verify:wasm`, ADR governance (8 ADRs) and all 52 PR-checker cases passed.

## Limits

Ten samples per build from one machine and network, Chromium only for live
timings, one video. Live node load and versions are unknown. The seek trade-off
above stands until the core change it names.
Binary file added docs/audits/2026-10-22-media-first-frame/adr.log.gz
Binary file not shown.
2 changes: 2 additions & 0 deletions docs/audits/2026-10-22-media-first-frame/bridge.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
<!doctype html>
<html lang="en"><head><meta charset="utf-8"><title>Media bridge only</title></head><body></body></html>
11 changes: 11 additions & 0 deletions docs/audits/2026-10-22-media-first-frame/checksums.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"<scratch>/site-baseline/sdk/client.js": "1b62355d096b83ab69e70b5f85670a16957646cd8ae2d114a4ee0fe590161d8b",
"<scratch>/site-baseline/sdk/internal/media.js": "59f151a25423dc74065bb9243df47f5fb862d4786bf56bf7b82928d0e10e2581",
"<scratch>/site-baseline/autonomi-stream-sw.js": "7862bcd999b201383224313a675c512299399a1a10d3c428468617033d359683",
"<scratch>/site-candidate/sdk/client.js": "8fce06261e2eb86e5fba64a7df1ca66dce73c085ae26a81725909cb2ad63bb33",
"<scratch>/site-candidate/sdk/internal/media.js": "b3fc5174f1d5af8ed0696278639edc93c52a243fbb3b8cd535f98f489686d6e4",
"<scratch>/site-candidate/autonomi-stream-sw.js": "d92e3d61913e6433103e54ee82aaf4910395ca74c5759e95ab342239a5e0ea06",
"<scratch>/site-candidate/sdk/wasm/ant_core_bg.wasm": "0c01cd4377a4b4846ae989691317738b909a83ce95e81ea62ce1833391a794a8",
"<scratch>/site-candidate/video.ant": "cea3ab028b454d6ff00e26119e3f46f0a57f3bcbbf9b9794c67fbc37d1aba7f4",
"<scratch>/site-baseline/harness.js": "33d49a9672c2f3e206f4ffb5552e429fba4955e71e0cf98307d9f98f5c0208ce"
}
Loading
Loading