feat: webvtt subtitles

.github/workflows/ci.yml

@@ -25,3 +25,18 @@ jobs:
 
       - name: Run validation suite
         run: make ci
+
+  unused-dependencies:
+    name: Unused dependencies
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out repository
+        uses: actions/checkout@v4
+
+      - name: Install cargo-machete
+        uses: taiki-e/install-action@v2
+        with:
+          tool: cargo-machete
+
+      - name: Fail on dependencies nothing uses
+        run: cargo machete

CONTRIBUTING.md

@@ -23,8 +23,11 @@ explains how the code is organised and [testing](docs/internals/testing.md) how
 - **Bounded by construction.** Anything read from a file or a mapper is untrusted. Check a length
   against the bytes available before allocating, use checked arithmetic, and keep the limits in
   `[limits]` meaningful. `unsafe` is forbidden.
-- **Lint clean.** Clippy runs with `pedantic` and warnings are errors. Prefer fixing the code to
-  allowing the lint, and say why when you allow one.
+- **Lint clean.** `make lint` runs Clippy with `pedantic` and the extra lints in `Cargo.toml`, and
+  warnings are errors. That includes code nothing calls (dead code), unused imports and variables,
+  and leftover `dbg!`, `todo!`, and `unimplemented!`. Delete unused code instead of silencing the
+  warning; when you must allow a lint, use `#[allow(..., reason = "...")]` and say why. CI also
+  runs `make unused-deps` (cargo-machete) to fail on dependencies nothing uses.
 - **Documented.** Behaviour that operators or mapper authors see belongs in the handbook. A change
   that alters an accepted design belongs in a design document first; see below.
 

Cargo.toml

@@ -52,10 +52,39 @@ strip = "debuginfo"
 [lints.rust]
 missing_docs = "warn"
 unsafe_code = "forbid"
+# Code that is never used is deleted, not left behind: dead_code is on by default, and CI turns
+# every warning into an error.
+unused_qualifications = "warn"
+unused_import_braces = "warn"
+unused_lifetimes = "warn"
+trivial_numeric_casts = "warn"
+redundant_lifetimes = "warn"
+single_use_lifetimes = "warn"
+unit_bindings = "warn"
 
 [lints.rustdoc]
 broken_intra_doc_links = "deny"
 
 [lints.clippy]
 all = "warn"
 pedantic = "warn"
+# Leftovers from debugging and unfinished work must not reach main.
+dbg_macro = "warn"
+todo = "warn"
+unimplemented = "warn"
+# Correctness and clarity picks from the restriction group.
+allow_attributes_without_reason = "warn"
+mem_forget = "warn"
+rc_mutex = "warn"
+str_to_string = "warn"
+unused_result_ok = "warn"
+verbose_file_reads = "warn"
+empty_drop = "warn"
+if_then_some_else_none = "warn"
+needless_raw_strings = "warn"
+redundant_type_annotations = "warn"
+# Rust's own `Result<_, ()>` and friends aside, these catch code that compiles but is dead weight.
+redundant_clone = "warn"
+useless_let_if_seq = "warn"
+unnecessary_self_imports = "warn"
+unnecessary_wraps = "warn"

Makefile

@@ -3,7 +3,7 @@
 CARGO ?= cargo
 MDBOOK ?= mdbook
 
-.PHONY: bench test-scripts bench-enforce conformance book book-serve book-test build check ci clean doc doc-open fixtures fmt fmt-check fuzz fuzz-check install-doc-tools lint serve demo site test deny
+.PHONY: unused-deps bench test-scripts bench-enforce conformance book book-serve book-test build check ci clean doc doc-open fixtures fmt fmt-check fuzz fuzz-check install-doc-tools lint serve demo site test deny
 
 help: ## Show the available targets
 	@printf '%s\n' \
@@ -29,6 +29,7 @@ help: ## Show the available targets
 		'lint       Run Clippy with warnings denied' \
 		'serve      Run the example HLS service' \
 		'demo       Serve the web player demo on http://127.0.0.1:8080' \
+		'unused-deps Fail on dependencies that nothing uses (needs cargo-machete)' \
 		'deny       Check dependencies for advisories and licences (needs cargo-deny)' \
 		'site       Build mdBook with rustdoc under /api' \
 		'test       Run unit and integration tests'
@@ -75,9 +76,12 @@ fuzz: ## Run media pipeline fuzzing with nightly
 	cp tests/fixtures/*.mp4 fuzz/corpus/media-pipeline/
 	$(CARGO) +nightly fuzz run media-pipeline fuzz/corpus/media-pipeline
 
-lint: ## Run Clippy with warnings denied
+lint: ## Run Clippy with warnings denied (dead code, unused imports, and the lints in Cargo.toml)
 	$(CARGO) clippy --all-features --all-targets -- -D warnings
 
+unused-deps: ## Fail on dependencies nothing uses (needs `cargo install cargo-machete --locked`)
+	cargo machete
+
 test: ## Run unit and integration tests
 	$(CARGO) test --all-features --all-targets
 

README.md

@@ -22,24 +22,23 @@ proportional to the segment asked for, not to the length of the video.
 - **Packages on demand.** Playlists and manifests are built when an asset loads and segments when
   they are requested, from a sample index kept in memory. There is no packaging step and no output
   to store.
-- **Reads what encoders produce.** Progressive and fragmented MP4, M4A, and QuickTime files; H.264,
-  HEVC, VP9, and AV1; AAC, HE-AAC, AC-3, E-AC-3, Opus, and FLAC; edit lists, several audio tracks,
-  and audio-only files. See [Supported input](#supported-input).
+- **Reads what encoders produce.** Progressive and fragmented MP4, M4A, and QuickTime files with
+  H.264, HEVC, VP9, or AV1 video and AAC, AC-3, E-AC-3, Opus, or FLAC audio. See
+  [supported input](docs/supported-input.md).
 - **Finds media through a mapper.** A mapper service answers "where is asset X?" with a file or an
   HTTP location, so the catalog lives wherever you already keep it. Remote origins are read with
   ranged requests and the server never downloads a whole file.
 - **Is bounded by construction.** Every length read from a file or a mapper is checked before it is
   allocated, and the limits are configuration. `unsafe` code is forbidden. The parser is run against
   corrupted input on every test run, and has a fuzz target.
-- **Is built to be operated.** Prometheus metrics, request IDs, readiness and liveness endpoints,
-  load shedding, graceful shutdown, immutable content-versioned URLs for CDNs, and byte-range and
-  conditional requests.
+- **Is built to be operated.** Prometheus metrics, health endpoints, load shedding, graceful
+  shutdown, and immutable content-versioned URLs for CDNs.
 
 ## What it does not do
 
 - **No transcoding.** The codecs must already suit the protocol, and there is no bitrate ladder
-  unless you supply the renditions. Adaptive renditions, WebVTT subtitles, and HLS I-frame
-  playlists are [planned](docs/technical-design/0006-trick-play-subtitles-and-renditions.md).
+  unless you supply the renditions. Adaptive renditions are
+  [planned](docs/technical-design/0006-trick-play-subtitles-and-renditions.md).
 - **No DRM, no live streaming, no MPEG-TS output.** Segments are fragmented MP4. DRM is planned
   after the features above.
 - **No TLS or authentication.** Run it behind a reverse proxy or CDN that provides both; see
@@ -72,16 +71,15 @@ Then play it:
 ffplay http://127.0.0.1:3000/hls/sample/master.m3u8
 ```
 
-or, in a second terminal, `make demo` and open <http://127.0.0.1:8080> for a player with live
-server metrics next to it. `make help` lists everything else.
+`make demo` starts a web player with live server metrics, and `make help` lists everything else; see
+[Using segmentor](docs/usage.md).
 
 ## Install
 
 - **Release binaries.** Each [GitHub release](https://github.com/includeamin/segmentor/releases)
   attaches Linux binaries for x86-64 and arm64 with checksums and build attestations.
   [`install.sh`](install.sh) downloads one, verifies its checksum, and copies it into place
-  (`curl -fsSL https://raw.githubusercontent.com/includeamin/segmentor/main/install.sh | sh`; read
-  it first, it is short).
+ ).
 - **Container image.** `ghcr.io/includeamin/segmentor`, for the same architectures:
 
   ```sh
@@ -93,135 +91,29 @@ server metrics next to it. `make help` lists everything else.
 - **From source.** `cargo install --git https://github.com/includeamin/segmentor`, which needs a
   recent stable Rust toolchain and a C compiler.
 
-[Verifying a download](docs/releasing.md#verifying-a-release) explains how to check the
-attestations and the image signature. [Deploying](docs/deployment.md) has a Docker Compose file, a
-Kubernetes manifest, and a hardened systemd unit, and says what to put in front of it: segmentor has
-no TLS or authentication of its own.
-
-## Supported input
-
-Progressive and fragmented MP4, M4A, and QuickTime `.mov` files, with `moov` first or last. Nothing is decoded or re-encoded, so the codecs must already suit the protocol:
-
-| | Supported |
-| --- | --- |
-| Video | H.264, HEVC (`hvc1`/`hev1`), VP9, AV1 |
-| Audio | AAC-LC, HE-AAC and HE-AACv2 (explicit signaling), AC-3, E-AC-3, Opus, FLAC |
-| Layout | one video track and any number of audio tracks, or audio only; edit lists of one edit, optionally after one empty edit |
-| Skipped | tracks that are not audio or video (timecode, metadata, subtitles) |
-| Rejected | encrypted media, samples in `moov` mixed with fragments, external data references, more than one sample description per track, other codecs (each error names what was found) |
-
-Whether a player can decode a codec is a separate question: HEVC and the Dolby codecs need Safari or a platform decoder, for instance. See [TDD 0004](docs/technical-design/0004-broader-mp4-input-support.md) for what was verified where.
-
-## Using it
-
-Start the example service with:
-
-```sh
-make serve
-```
-
-The example asset is available at `http://127.0.0.1:3000/hls/sample/master.m3u8`. Configuration is loaded from `vod.example.toml`; asset IDs map to files beneath one canonical media root, and paths cannot escape that root.
-
-Logging is configured in the same file:
-
-```toml
-[logging]
-level = "info"          # trace, debug, info, warn, or error
-format = "json"         # json or compact
-buffer_capacity = 8192
-```
-
-Logs are written by a dedicated worker thread through a bounded, lossy queue. If the logger cannot keep up, log lines are dropped instead of blocking media requests. Request and segment timing events use `debug`, so the default `info` level records lifecycle and asset-loading events without logging every media request.
-
-Each configured asset exposes:
-
-```text
-/hls/{asset}/master.m3u8
-/hls/{asset}/video/index.m3u8
-/hls/{asset}/audio-{n}/index.m3u8   (one per audio track, numbered from 1)
-/hls/{asset}/{track}/init.mp4
-/hls/{asset}/{track}/segments/{index}/media.m4s
-/dash/{asset}/manifest.mpd
-/dash/{asset}/{track}/init.mp4
-/dash/{asset}/{track}/segments/{index}/media.m4s
-/health   liveness
-/ready    readiness (503 once shutdown begins)
-/metrics  Prometheus text
-```
-
-Initialization and media responses support single and suffix byte ranges, `If-Range`, strong ETags, and immutable content-versioned URLs. Media URLs must carry the `v` query parameter the playlists emit; a missing or stale version is a `404`. Media payloads are read through a bounded backpressured stream instead of buffering the complete segment in each HTTP request.
-
-Assets can also be resolved on demand from an external mapper service, including media held on remote HTTP origins; see the [Mapper API reference](docs/mapper-api.md) and [docs/operations.md](docs/operations.md).
-
-CORS, shutdown behavior, concurrency limits, and metrics are configurable in the same file; see [docs/operations.md](docs/operations.md) for production guidance.
-
-## Web player demo
-
-`demo/index.html` is a single-file player (hls.js and dash.js, loaded from a CDN) that plays an asset over HLS or DASH and shows live server metrics parsed from `/metrics` next to it: request rate, throughput, per-route latency, errors, and resolver and cache events. It also shows player-side stats such as buffer, bandwidth, and dropped frames.
-
-```sh
-make serve   # terminal 1: the origin on :3000
-make demo    # terminal 2: the player on http://127.0.0.1:8080
-```
-
-The page reads `/metrics` cross-origin, so keep `[cors]` enabled in the config, as in `vod.example.toml`.
-
-## Packaging from the command line
-
-The packager parses a local MP4, creates keyframe-aligned segment plans, and writes separate fragmented MP4 audio and video tracks:
-
-```sh
-cargo run -- package \
-	--input tests/fixtures/h264-aac.mp4 \
-	--output target/package-test
-```
-
-Regenerate the synthetic parser fixture and its FFprobe packet manifest with `make fixtures`. FFmpeg is used only for fixture generation and output validation, not by the application.
-
-## Releases
-
-Merges to `main` are tagged automatically with a semantic version derived from [Conventional Commits](https://www.conventionalcommits.org), so use `feat:`, `fix:`, or `type(scope)!:` in commit and pull request titles. Publish a release, with a generated changelog and a `latest` or `preview` flag, from the **Release** workflow. See [docs/releasing.md](docs/releasing.md).
+See [Deploying](docs/deployment.md) for Compose, Kubernetes and systemd files, and
+[Verifying a release](docs/releasing.md#verifying-a-release) for checking downloads.
 
 ## Documentation
 
-The project handbook uses the same `mdBook` interface as the Rust Book. Install the pinned documentation tool and serve the book with live reload:
-
-```sh
-make install-doc-tools
-make book-serve
-```
+The handbook, built with mdBook, covers everything beyond this page:
 
-Build the complete static site, including the `rustdoc` API reference, with `make site`. The book starts at `target/book/index.html`, and its API Reference chapter links to the generated Rust documentation under `target/book/api/`.
+- [Using segmentor](docs/usage.md): endpoints, the web player demo, the packaging command
+- [Supported input](docs/supported-input.md), [Deploying](docs/deployment.md), and
+  [Operating the origin](docs/operations.md)
+- [Mapper API](docs/mapper-api.md), [architecture](docs/architecture.md), and the
+  [technical designs](docs/technical-design/README.md)
+- [Releasing](docs/releasing.md), including how versions and tags are made
 
-See [docs/README.md](docs/README.md) for the documentation layout and authoring workflow.
+Build it locally with `make book-serve` (see [docs/README.md](docs/README.md)).
 
 ## Contributing and security
 
 Contributions are welcome; start with [CONTRIBUTING.md](CONTRIBUTING.md), which covers building,
-testing, design documents, and the sign-off every commit needs. Files that will not load or play
-are the most useful reports there are: there is an issue form for them. Please report security
-problems privately, as described in [SECURITY.md](SECURITY.md). Everyone taking part is expected to
-follow the [Code of Conduct](CODE_OF_CONDUCT.md).
-
-## Development
-
-The repository uses stable Rust with `rustfmt` and Clippy. Run the complete local validation suite with:
-
-```sh
-make ci
-```
-
-Run `make help` to list the individual build, check, format, lint, test, and documentation targets.
-
-`make conformance` runs the black-box HLS and DASH conformance suite, and `make bench` measures the performance budgets (see [docs/benchmarks.md](docs/benchmarks.md)).
-
-Compile the media-pipeline fuzz target on stable with `make fuzz-check`. To run a fuzz campaign, install the separate nightly tooling. `make fuzz` copies the generated MP4 fixtures into an ignored, writable corpus before starting libFuzzer:
-
-```sh
-rustup toolchain install nightly
-cargo install cargo-fuzz --locked
-make fuzz
-```
+testing, and the sign-off every commit needs. Files that will not load or play are the most useful
+reports there are: there is an issue form for them. Report security problems privately, as
+described in [SECURITY.md](SECURITY.md). Everyone taking part is expected to follow the
+[Code of Conduct](CODE_OF_CONDUCT.md).
 
 ## License
 

benches/budgets.rs

@@ -23,7 +23,8 @@
     clippy::large_futures,
     clippy::redundant_closure_for_method_calls,
     clippy::too_many_lines,
-    clippy::trivially_copy_pass_by_ref
+    clippy::trivially_copy_pass_by_ref,
+    reason = "harness code, not the production crate"
 )]
 
 use std::path::{Path, PathBuf};

docs/SUMMARY.md

@@ -5,6 +5,8 @@
 # Design
 
 - [Architecture](architecture.md)
+- [Using segmentor](usage.md)
+- [Supported input](supported-input.md)
 - [Deploying](deployment.md)
 - [Operating the origin](operations.md)
 - [Performance budgets](benchmarks.md)