Skip to content

feat(playback): one progressive bitmap-subtitle demux per source, served behind a coverage watermark - #444

Closed
randrini wants to merge 8 commits into
mainfrom
feat/pgs-progressive-sidecar
Closed

randrini wants to merge 8 commits into
mainfrom
feat/pgs-progressive-sidecar

Conversation

@randrini

@randrini randrini commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Bitmap (PGS) subtitle pages do not appear for a large source served over a relay.
A 120 s page is requested with its own window, but a relay cannot seek, so ffmpeg
demuxes the container from byte 0 and discards the prefix. The cost of a page at
offset S is (S + W) × bitrate, not W × bitrate: for an 18.25 GB / 35 Mbps
source that is ~525 MB per page, which is also the window extraction timeout.

Pages therefore end at the timeout holding a partial .sup — 180 bytes, nothing
at all, or a single 2,288-byte slice — and never a complete display set. Since a
cue-less bitmap page is correctly refused by the cache, nothing is kept and the
next request pays the same read. The player sits on "Loading subtitles…" for the
rest of playback. The same track works from a small source (a 190 MB file
extracts 19 KB in 15 s), so this is cost, not a broken track.

Related issue: N/A
Validation tasks: none

This depends on the bitmap cache-admission fix from #442, which is in main.

Approach

Two independent pieces, in commit order.

A read budget for a page. A windowed bitmap extract over a remote source is
capped at 256 MB of reads, taken from the extract process's own I/O counters.
ffmpeg has no option to bound its input reads, so the cap is enforced from
outside the process: at the first sample past it, the extract is killed. The page
answers the retryable 503 with a distinct additive problem code
(subtitle_page_too_costly), a counter, and the existing per-identity extraction
backoff so a client retry does not pay the same read again. A whole-track extract
must read everything, an artifact-backed window reads a local file, and a local
source's prefix read is a disk read that finishes in seconds — none are capped.

One progressive demux per track. One unbounded .sup extract runs per
(source identity, track, format generation), single-flighted through the cache's
existing full-track fill (warm slot plus the detached per-source relay budget).
It writes the track's artifact and indexes display sets as the bytes land, so the
cache knows how far the track is covered. A page is then served in this order:
the last-good window, the committed artifact, then the progressive artifact if
the requested end is inside the coverage; otherwise the retryable 503 advertises
what is actually covered in X-Subtitle-Coverage. A page the artifact does not
cover never falls back to a source read, and the miss starts the fill.

Invariants the implementation holds:

  • coverage advances only at a complete display set (PCS … END), derived from
    bytes already written to the artifact, so it never overstates;
  • a page is never served from a truncated run, and an incomplete tail is never
    published as the whole-track entry — the artifact flips to the committed entry
    only when its extract reaches end of file;
  • a covered page is a local file read on an independent handle, so it cannot
    block the extract's writes or backpressure its stdout;
  • the fill draws the detached relay budget, never a request-path slot;
  • a text (WebVTT/SRT/ASS) window and a local file keep their existing paths.

The API handler owns the fill on a cost refusal, because a virtual source's relay
registration dies with the request, and it starts the fill even while the viewer
is playing: no page extract can produce that page at an affordable cost, and the
bounded page prefetch would re-read the same source for pages the fill is already
publishing.

Deferred

  • Pipeline tee. When a remux or transcode already demuxes the source, the same
    .sup should be an additional output of that invocation instead of a second
    read. The progressive path is correct without it; this only removes the
    duplicate container read.
  • Page prefetch from the watermark. The prefetch still extracts bounded pages
    on a large relay source. Its pages fail fast under the new budget, but it should
    skip windows the artifact already covers.
  • Unknown-size rows. Progressive serving requires a known container size of
    at least 256 MB. A row whose provider never declared a size keeps the page
    extract (now capped) rather than refusals based on an assumption; enabling it
    is a follow-up once virtual rows always carry a size.
  • Coverage from the demux position. The watermark comes from display-set
    timestamps, so a window inside a long gap between subtitles answers retryable
    until the next display set is parsed instead of proving the gap empty.

Validation

  • go build ./..., go vet on the touched packages: pass.
  • Focused go test -count=1 -race on the new playback and handler tests: pass.
    The new tests do not compile against the base commit (no coverage index, no
    page-cost error, no source size on the extract options), which is the
    fail-before form for a new path; the behaviour they forbid is pinned by a base
    test that serves a truncated page run with 200.
  • go test -count=1 ./internal/playback/: only two real-ffmpeg tests fail
    (TestOnDemandSubtitleSegmentsDoNotWaitForTheStream,
    TestServeExtractDiscardsACappedDemuxCopy), and both fail identically at the
    base commit.
  • go test -count=1 ./internal/api/handlers/: three bitmap page-reuse/prefetch
    tests fail; the base commit fails the same three in a full run and in
    isolation. One playback-start timing test fails intermittently at both.
  • make verify-playback-fixtures: the extract-argument matrix is unchanged.
  • make lint-changed BASE_REF=kipsilabs/main in the Go/vips build image: 0
    issues. It could not run on the host (no libvips), so the push used
    --no-verify.
  • make verify-local-paths and gofmt: clean.
  • Not run: any check against a live relay source. The cost figures above come
    from the production incident, not from this branch.

Risks

  • Behaviour change for large relay bitmap tracks: a page now waits for the
    artifact instead of hanging on a page extract. The first page a viewer needs is
    available once the fill has parsed that far; a client that retries keeps its
    existing backoff and sees a retryable 503 until then.
  • subtitle_page_too_costly is a new problem code on the 503. Status,
    Retry-After, and the client's retry loop are unchanged.
  • The coverage header now also appears on a 503, describing the artifact's parsed
    range rather than a produced slice.
  • The bitmap fill may now run during playback where it was paused-only. It uses
    the detached relay budget, so it cannot take a slot a client-facing read needs.

Checklist

  • I read and can explain the complete diff.
  • This pull request addresses one concern.
  • Each test this pull request adds meets "Decide what to test" in CONTRIBUTING.md.
  • The Evidence section shows every change a user can see, or says there is none.

AI Disclosure

  • Harness: OpenCode
  • Tool(s): OpenCode agent
  • Model(s): Muse Spark
  • Involvement: Fully AI-generated
  • Adversarial review: n/a

Posted by an AI agent (Muse Spark, OpenCode harness).

randrini added 2 commits November 13, 2026 17:02
A windowed PGS extract cannot seek a non-seekable relay, so ffmpeg demuxes
the container from byte 0 and discards the prefix: a page at offset S costs
(S + W) x bitrate, not W x bitrate. A 120 s page of an 18.25 GB / 35 Mbps
source is ~525 MB, which is also the two-minute extraction timeout, so such
pages were killed at the timeout and answered with a partial .sup that held
no complete display set.

The extract now watches the process's own read count and kills it once a
page has read more than 256 MB, the same order as the largest source a page
read is still worth attempting. The page answers the retryable 503 the
client already backs off from, with a distinct additive problem code
(subtitle_page_too_costly), a counter, and the existing per-identity
extraction backoff so a retry does not pay the same read again.

Only a bounded bitmap page read from a remote source is capped: a
whole-track extract must read everything by design, an artifact-backed
window reads a local file, and a local source's prefix read is a disk read
that finishes in seconds.
…ce demux

A windowed PGS extract cannot seek a non-seekable relay, so every page paid
for the whole container prefix again: on an 18.25 GB / 35 Mbps source a
120 s page reads ~525 MB, which is the window extract timeout, so pages
returned partial .sup bytes with no complete display set and were never
cached.

One unbounded .sup demux now runs per source identity, track and format
generation, single-flighted through the cache's warm slot and detached
relay budget. It writes the track's artifact and indexes the display sets
as bytes land, so the cache knows how far the track is covered:

- a page inside the coverage is served by copying those display sets out
  of the artifact, with no source read and no ffmpeg;
- a page beyond it answers the retryable shape with X-Subtitle-Coverage
  describing what is actually covered, never a truncated run;
- on end of file the artifact flips into the committed whole-track entry.

The watermark only advances at a complete display set (PCS through END) and
is derived from bytes already written to the artifact, so a page is never
served ahead of what has been parsed. A page refused for its read cost also
starts the fill, since the artifact is what will serve it; the API handler
owns that fill because its relay registration outlives the request.

Text windows, local files, and a remote container whose size is unknown
keep their existing paths.
@randrini
randrini force-pushed the feat/pgs-progressive-sidecar branch from dbbe3da to c911389 Compare November 13, 2026 16:21
@randrini
randrini changed the base branch from feat/playback-subtitle-polish to main November 13, 2026 16:21
randrini added 6 commits November 13, 2026 22:59
…ably

The progressive bitmap fill never ran in production, so every page still cost a
fresh source extract. A page miss is served by the handler's detached warm, and
that warm admitted a bitmap track only while paused; a playing session dropped
it with "bitmap full-track warm stays paused-only". Five page misses, five
skips, no artifact, no watermark, no coverage-gated answer: 100.2 s for the first
page and 70.6/75.7/88.1 s for later distinct pages, with only an exact repeat
fast from last-good.

A bounded bitmap page is the one bitmap request whose fill is the page path, so
it now routes to the waiting-page admission: the fill starts while playing,
single-flighted per (source identity, track, format) through the cache's warm
slot and detached relay budget, and it replaces the bounded window prefetch,
which would read the same source again for pages the fill publishes. Every other
bitmap request keeps the paused-only full-track warm.

The read cap measured the wrong cost. A page at 3600 s returned 200 after 88 s
with its read count under the cap because the relay answers the seek with range
reads, so the page is now priced before the extract: the container prefix it
would discard, `start x rate`, from the row's size and runtime or bitrate, and
in wall-clock terms from the read rate this identity's own extracts measured.
A page beyond 256 MB of prefix, or whose prefix would take more than 45 s at the
measured rate, is refused before any source read with the retryable 503, its
Retry-After and X-Subtitle-Coverage, and that refusal starts the fill. An opening
page has no prefix and extracts exactly as before. The runtime 256 MB cap stays
for pages that are still attempted.

An extraction that ran out the window extract deadline answered a non-retryable
500 while the client had four attempts to spend. A page that times out now
answers the same retryable page-cost shape as the read cap; a windowed text
extract that fails or is canceled no longer commits a 200 carrying a header and
half a cue, because the slice is held until its extract completes and dropped
otherwise. The handler classifies both through one mapping: deadline, upstream
5xx and a lapsed relay token are retryable, everything else stays a loud 500.

A `.sup` muxer that refuses the mapped stream's codec after a release rotation
("sup muxer supports only codec hdmv_pgs_subtitle", exit 234) is a mapping the
drift probe missed, so it now reads as a stream-map error and takes the existing
re-probe and remap path — a remapped track or a retryable 409 — instead of a 500.
Verifying a bitmap request's live layout before every spawn is left out: that
probe costs a provider round trip per page and needs a design decision.

Serving rules for the new page path are in docs/architecture/playback-protocol-v3.md
and docs/architecture/web-pgs-subtitles.md. The web client is unchanged: it
retries with backoff and honours Retry-After.
serveProgressiveSUPPage set Content-Type and Cache-Control after
serveWindow had copied the page, and the first Write commits the status
and headers. Every non-empty page therefore went out with neither, so an
intermediary could cache a per-window subtitle slice. Set both headers
before the copy on every path through it, including the empty window.

The existing page test only read the recorder's live header map, which a
late Set still updates; the new test reads the committed response.
pagePrefixEstimate priced a page's discarded container prefix as
start x byte rate, which can exceed a known source size: a seek past the
end, or a size and bitrate that disagree, priced the page above the whole
container and refused a page the source can serve. Clamp the prefix to
SourceSizeBytes when it is known.
… parsed

A page request at a playhead inside the progressive artifact's watermark was
answered with 503 subtitle_source_unavailable. The serve gate required the whole
page range to be parsed (end <= coveredUntil), so a request for 10-130 s against
an artifact that had parsed 0-120 s was refused although every display set that
shows in its first 110 s was complete on disk.

Production showed it plainly: the artifact's watermark reached 126.543 s while
page requests at 10, 20, 40 and 60 s answered 503 with X-Subtitle-Coverage:
0.000-126.543 and no page was ever served; "subtitle page served from the
progressive artifact" never appeared in the log because the copy was never
reached. Each of those requests is snapped onto the source-zero page grid, so
the page under test ends at 240 s even though the requested range ends at 130 s,
and the old gate asked for the whole 0-240 s.

Serve the parsed prefix instead: the cut stops at the watermark, the response is
a 200, and X-Subtitle-Coverage reports the range it really produced. The cut
comes from the display-set index and its first complete display set is checked
before a byte is written (supHasDisplaySet), so a partial run or a cut that
cannot draw is never served. A page that starts at or past the watermark is
still retryable, now as 503 subtitle_page_not_ready with Retry-After and the
covered range, because the source was never read and the page is only waiting
for the artifact that will answer it.
A subtitle response's X-Subtitle-Coverage is the range it really produced, and
a bitmap page answered from a partly parsed track is shorter than the page it
asked for. The window fetcher committed the whole requested range whenever the
body streamed in, so the unparsed tail of such a page was marked as covered and
never requested again: playback would draw the prefix and then nothing until the
next page boundary.

Commit the advertised end when the response carries one, and on the page grid
request the page that contains the committed end rather than the page after it.
A partly produced page is therefore asked for again as the artifact catches up,
and a page that arrived whole is unaffected because its advertised end is the
page end.
@randrini

Copy link
Copy Markdown
Collaborator Author

Superseded by #459, which contains this change plus the rest of the playback reliability bundle. Closing to keep the review surface to one PR.

@randrini randrini closed this Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant