Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/deploy/migrating-to-hyperframes-lambda.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -73,9 +73,9 @@ HyperFrames refuses `data-gpu-mode="hardware"` in distributed mode — hardware

### webm uses closed-GOP VP9

webm distributed renders go through libvpx-vp9 with `-g <chunkSize>`, `-keyint_min <chunkSize>`, `-auto-alt-ref 0`, and `-cpu-used 2`. The alt-ref disable is the load-bearing bit: libvpx-vp9's default non-displayable alt-ref frames can land anywhere in a GOP, which breaks concat-copy at chunk seams. Closed-GOP forces a keyframe at every chunk boundary so `ffmpeg -f concat -c copy` round-trips losslessly. Output is `yuva420p` to preserve alpha. Audio is muxed as Opus.
webm distributed renders go through libvpx-vp9 with `-g <chunkSize>`, `-keyint_min <chunkSize>`, `-auto-alt-ref 0`, and `-cpu-used 4` by default. The alt-ref disable is the load-bearing bit: libvpx-vp9's default non-displayable alt-ref frames can land anywhere in a GOP, which breaks concat-copy at chunk seams. Closed-GOP forces a keyframe at every chunk boundary so `ffmpeg -f concat -c copy` round-trips losslessly. Output is `yuva420p` to preserve alpha. Audio is muxed as Opus.

Distributed webm files are typically ~10-25% larger than the same composition rendered in-process at the same CRF, because closed-GOP forces more keyframes than the in-process single-pass would emit. Per-chunk encode is also slower than libvpx-vp9's default speed/quality tradeoff (`-cpu-used 2` is more conservative than the default for `-deadline good`). The single-machine in-process renderer remains the right choice for short webm renders; distributed pays for itself once a render's wall-clock exceeds what one machine delivers.
Distributed webm files are typically ~10-25% larger than the same composition rendered in-process at the same CRF, because closed-GOP forces more keyframes than the in-process single-pass would emit. VP9 encode speed is controlled by `PRODUCER_VP9_CPU_USED` (`-8` to `8`); use lower values for quality-sensitive or long-form WebM, and higher values when wall-clock encode time matters more than compression efficiency. The single-machine in-process renderer remains the right choice for short webm renders; distributed pays for itself once a render's wall-clock exceeds what one machine delivers.

### State files are local by default

Expand Down
20 changes: 20 additions & 0 deletions docs/guides/performance.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,26 @@ npx hyperframes render --quality draft --output preview.mp4

Draft quality renders fast and is visually close to the final render for everything except encoder-level detail.

## WebM encode speed

Transparent WebM output uses FFmpeg's `libvpx-vp9` encoder. VP9 is CPU-heavy, so short overlay renders can spend most of their wall time in the encode stage even when frame capture is fast.

HyperFrames sets `-cpu-used 4` for VP9 by default. On a devbox check using an 8s 1280×720, 15fps VP9-alpha encode, explicit `-cpu-used 4` cut encode time from 6.3s to 2.6s versus libvpx's default, with SSIM 0.9986 and PSNR 50.5dB against the default encode. Your composition and host CPU will move those numbers, but the direction is consistent: higher values trade some compression efficiency for faster WebM encodes.

Tune per deployment with:

```bash Terminal
PRODUCER_VP9_CPU_USED=2 npx hyperframes render --format webm --output overlay.webm
```

For one-off local renders, pass the same value directly:

```bash Terminal
npx hyperframes render --format webm --vp9-cpu-used 2 --output overlay.webm
```

Valid values are integers from `-8` to `8`; HyperFrames clamps out-of-range values before passing them to FFmpeg.

## Next Steps

<CardGroup cols={2}>
Expand Down
4 changes: 2 additions & 2 deletions docs/guides/remove-background.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -333,7 +333,7 @@ After running any of these externally, encode the output as a HyperFrames-compat
ffmpeg -i frames-%04d.png -c:v libvpx-vp9 \
-pix_fmt yuva420p \
-metadata:s:v:0 alpha_mode=1 \
-auto-alt-ref 0 -b:v 0 -crf 30 \
-auto-alt-ref 0 -cpu-used 4 -b:v 0 -crf 30 \
transparent.webm
```

Expand Down Expand Up @@ -389,7 +389,7 @@ Chrome only reads the alpha plane when the WebM is encoded as `yuva420p` with th
ffmpeg -i in.webm -c:v libvpx-vp9 \
-pix_fmt yuva420p \
-metadata:s:v:0 alpha_mode=1 \
-auto-alt-ref 0 \
-auto-alt-ref 0 -cpu-used 4 \
out.webm
```

Expand Down
1 change: 1 addition & 0 deletions packages/cli/src/background-removal/pipeline.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ describe("background-removal/pipeline — buildEncoderArgs", () => {
const args = buildEncoderArgs("webm", 1920, 1080, 30, "/tmp/out.webm");
expect(args).toContain("libvpx-vp9");
expect(args).toContain("yuva420p");
expect(args[args.indexOf("-cpu-used") + 1]).toBe("4");
// The alpha_mode metadata must be present; without it Chrome ignores the alpha plane.
const idx = args.indexOf("-metadata:s:v:0");
expect(idx).toBeGreaterThan(-1);
Expand Down
3 changes: 3 additions & 0 deletions packages/cli/src/background-removal/pipeline.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import { extname } from "node:path";
import { findFFmpeg, findFFprobe, getFFmpegInstallHint } from "../browser/ffmpeg.js";
import { createSession, type Session } from "./inference.js";
import { type Device, type ModelId } from "./manager.js";
import { DEFAULT_VP9_CPU_USED } from "@hyperframes/engine";

export type OutputFormat = "webm" | "mov" | "png";

Expand Down Expand Up @@ -159,6 +160,8 @@ export function buildEncoderArgs(
"good",
"-row-mt",
"1",
"-cpu-used",
String(DEFAULT_VP9_CPU_USED),
"-auto-alt-ref",
"0",
"-pix_fmt",
Expand Down
17 changes: 17 additions & 0 deletions packages/cli/src/commands/render.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -312,6 +312,23 @@ describe("renderLocal browser GPU config", () => {
});
});

it("forwards vp9CpuUsed into resolveConfig when set", async () => {
await renderLocal("/tmp/project", "/tmp/out.webm", {
fps: { num: 30, den: 1 },
quality: "standard",
format: "webm",
gpu: false,
browserGpuMode: "software",
hdrMode: "auto",
quiet: true,
vp9CpuUsed: 2,
});

expect(producerState.resolveConfigCalls[0]).toMatchObject({
vp9CpuUsed: 2,
});
});

it("omits pageNavigationTimeout from resolveConfig when --browser-timeout is not set", async () => {
await renderLocal("/tmp/project", "/tmp/out.mp4", {
fps: { num: 30, den: 1 },
Expand Down
32 changes: 31 additions & 1 deletion packages/cli/src/commands/render.ts
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,12 @@ import { buildDockerRunArgs, resolveDockerPlatform } from "../utils/dockerRunArg
import { normalizeErrorMessage } from "../utils/errorMessage.js";
import { runEnvironmentChecks } from "../browser/preflight.js";
import type { ProducerLogger, RenderJob } from "@hyperframes/producer";
import { isVideoFrameFormat, type VideoFrameFormat } from "@hyperframes/engine";
import {
MAX_VP9_CPU_USED,
MIN_VP9_CPU_USED,
isVideoFrameFormat,
type VideoFrameFormat,
} from "@hyperframes/engine";
import {
normalizeResolutionFlag,
parseFps,
Expand Down Expand Up @@ -220,6 +225,11 @@ export default defineCommand({
type: "string",
description: "Target video bitrate such as 10M. Mutually exclusive with --crf.",
},
"vp9-cpu-used": {
type: "string",
description:
"libvpx-vp9 -cpu-used value for WebM encodes (-8 to 8). Higher is faster with a larger quality/size tradeoff. Env: PRODUCER_VP9_CPU_USED.",
},
gpu: { type: "boolean", description: "Use GPU encoding", default: false },
"browser-gpu": {
type: "boolean",
Expand Down Expand Up @@ -569,6 +579,20 @@ export default defineCommand({
crf = parsed;
}

let vp9CpuUsed: number | undefined;
if (args["vp9-cpu-used"] != null) {
const raw = args["vp9-cpu-used"];
const parsed = Number(raw);
if (!Number.isInteger(parsed) || parsed < MIN_VP9_CPU_USED || parsed > MAX_VP9_CPU_USED) {
errorBox(
"Invalid vp9-cpu-used",
`Got "${raw}". Must be an integer between ${MIN_VP9_CPU_USED} and ${MAX_VP9_CPU_USED}.`,
);
process.exit(1);
}
vp9CpuUsed = parsed;
}

if (args["video-bitrate"] != null && !videoBitrate) {
errorBox(
"Invalid video-bitrate",
Expand Down Expand Up @@ -730,6 +754,7 @@ export default defineCommand({
browserGpuMode,
hdrMode,
crf,
vp9CpuUsed,
videoBitrate,
quiet: batchQuiet,
browserPath,
Expand Down Expand Up @@ -786,6 +811,7 @@ export default defineCommand({
browserGpuMode,
hdrMode: args.sdr ? "force-sdr" : args.hdr ? "force-hdr" : "auto",
crf,
vp9CpuUsed,
videoBitrate,
videoFrameFormat,
quiet,
Expand All @@ -809,6 +835,7 @@ export default defineCommand({
browserGpuMode,
hdrMode: args.sdr ? "force-sdr" : args.hdr ? "force-hdr" : "auto",
crf,
vp9CpuUsed,
videoBitrate,
videoFrameFormat,
quiet,
Expand Down Expand Up @@ -845,6 +872,7 @@ interface RenderOptions {
browserGpuMode?: "auto" | "hardware" | "software";
hdrMode: "auto" | "force-hdr" | "force-sdr";
crf?: number;
vp9CpuUsed?: number;
videoBitrate?: string;
videoFrameFormat?: VideoFrameFormat;
quiet: boolean;
Expand Down Expand Up @@ -1088,6 +1116,7 @@ async function renderDocker(
browserGpu: options.browserGpuMode === "hardware",
hdrMode: options.hdrMode,
crf: options.crf,
vp9CpuUsed: options.vp9CpuUsed,
videoBitrate: options.videoBitrate,
videoFrameFormat: options.videoFrameFormat,
quiet: options.quiet,
Expand Down Expand Up @@ -1195,6 +1224,7 @@ export async function renderLocal(
: {}),
...(options.protocolTimeout != null && { protocolTimeout: options.protocolTimeout }),
...(options.playerReadyTimeout != null && { playerReadyTimeout: options.playerReadyTimeout }),
...(options.vp9CpuUsed != null ? { vp9CpuUsed: options.vp9CpuUsed } : {}),
}),
hdrMode: options.hdrMode,
crf: options.crf,
Expand Down
1 change: 1 addition & 0 deletions packages/cli/src/docs/rendering.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ Requires: Docker installed and running.
- `-w, --workers` — Parallel workers 1-8 (default: auto)
- `--crf` — Override encoder CRF (mutually exclusive with `--video-bitrate`)
- `--video-bitrate` — Target video bitrate such as `10M` (mutually exclusive with `--crf`)
- `--vp9-cpu-used` — WebM VP9 speed/quality tradeoff (`-8` to `8`, default: `4`). Higher values encode faster with larger output / quality tradeoff.
- `--video-frame-format` — Source video frame extraction format: `auto`, `jpg`, or `png` (default: `auto`). Use `png` for UI recordings, screen captures, and color-sensitive source videos.
- `--gpu` — Use GPU encoding (NVENC, VideoToolbox, AMF, VAAPI, QSV)
- `--browser-gpu` / `--no-browser-gpu` — Force host GPU or software (SwiftShader) for Chrome/WebGL capture. Default for local renders is `auto` — probe WebGL availability on first launch and fall back to software if no GPU is reachable. Docker mode always uses software.
Expand Down
3 changes: 3 additions & 0 deletions packages/cli/src/utils/dockerRunArgs.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,7 @@ describe("buildDockerRunArgs", () => {
browserGpu: false,
hdrMode: "force-hdr",
crf: 16,
vp9CpuUsed: 2,
videoBitrate: undefined,
videoFrameFormat: "png",
quiet: true,
Expand All @@ -182,6 +183,8 @@ describe("buildDockerRunArgs", () => {
expect(args).toContain("8");
expect(args).toContain("--crf");
expect(args).toContain("16");
expect(args).toContain("--vp9-cpu-used");
expect(args).toContain("2");
expect(args).toContain("--video-frame-format");
expect(args).toContain("png");
expect(args).toContain("--quiet");
Expand Down
2 changes: 2 additions & 0 deletions packages/cli/src/utils/dockerRunArgs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ export interface DockerRenderOptions {
browserGpu: boolean;
hdrMode: "auto" | "force-hdr" | "force-sdr";
crf?: number;
vp9CpuUsed?: number;
videoBitrate?: string;
videoFrameFormat?: "auto" | "jpg" | "png";
quiet: boolean;
Expand Down Expand Up @@ -121,6 +122,7 @@ export function buildDockerRunArgs(input: DockerRunArgsInput): string[] {
...(options.gifLoop != null ? ["--gif-loop", String(options.gifLoop)] : []),
...(options.workers != null ? ["--workers", String(options.workers)] : []),
...(options.crf != null ? ["--crf", String(options.crf)] : []),
...(options.vp9CpuUsed != null ? ["--vp9-cpu-used", String(options.vp9CpuUsed)] : []),
...(options.videoBitrate ? ["--video-bitrate", options.videoBitrate] : []),
...(options.videoFrameFormat && options.videoFrameFormat !== "auto"
? ["--video-frame-format", options.videoFrameFormat]
Expand Down
23 changes: 23 additions & 0 deletions packages/engine/src/config.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ describe("resolveConfig", () => {
expect(config.browserGpuMode).toBe("software");
expect(config.enableStreamingEncode).toBe(true);
expect(config.streamingEncodeMaxDurationSeconds).toBe(240);
expect((config as Record<string, unknown>).vp9CpuUsed).toBe(4);
expect(config.audioGain).toBe(1);
expect(config.debug).toBe(false);
});
Expand Down Expand Up @@ -84,6 +85,28 @@ describe("resolveConfig", () => {
expect(config.streamingEncodeMaxDurationSeconds).toBe(0);
});

it("reads VP9 cpu-used from env", () => {
setEnv("PRODUCER_VP9_CPU_USED", "6");

const config = resolveConfig();
expect((config as Record<string, unknown>).vp9CpuUsed).toBe(6);
});

it("falls back to the VP9 cpu-used default for invalid env values", () => {
setEnv("PRODUCER_VP9_CPU_USED", "fast");

const config = resolveConfig();
expect((config as Record<string, unknown>).vp9CpuUsed).toBe(4);
});

it("clamps VP9 cpu-used env values to libvpx's supported range", () => {
setEnv("PRODUCER_VP9_CPU_USED", "99");
expect((resolveConfig() as Record<string, unknown>).vp9CpuUsed).toBe(8);

process.env.PRODUCER_VP9_CPU_USED = "-99";
expect((resolveConfig() as Record<string, unknown>).vp9CpuUsed).toBe(-8);
});

it("treats non-'true' boolean env vars as false", () => {
setEnv("PRODUCER_DISABLE_GPU", "yes");

Expand Down
19 changes: 18 additions & 1 deletion packages/engine/src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import {
isLowMemorySystem,
LOW_MEMORY_TOTAL_MB_THRESHOLD,
} from "./services/systemMemory.js";
import { DEFAULT_VP9_CPU_USED, normalizeVp9CpuUsed } from "./services/vp9Options.js";

/**
* Full engine configuration. All fields are wired through the config
Expand Down Expand Up @@ -101,6 +102,11 @@ export interface EngineConfig {
enablePageSideCompositing: boolean;

// ── Encoding ─────────────────────────────────────────────────────────
/**
* libvpx-vp9 speed/quality tradeoff. Higher values encode faster with a
* larger quality/size tradeoff. FFmpeg accepts integer values from -8 to 8.
*/
vp9CpuUsed: number;
enableChunkedEncode: boolean;
chunkSizeFrames: number;
enableStreamingEncode: boolean;
Expand Down Expand Up @@ -220,6 +226,7 @@ export const DEFAULT_CONFIG: EngineConfig = {
lowMemoryMode: false,
enablePageSideCompositing: true,

vp9CpuUsed: DEFAULT_VP9_CPU_USED,
enableChunkedEncode: false,
chunkSizeFrames: 360,
enableStreamingEncode: true,
Expand Down Expand Up @@ -277,6 +284,11 @@ export function resolveConfig(overrides?: Partial<EngineConfig>): EngineConfig {
if (raw === undefined) return fallback;
return raw === "true";
};
const envVp9CpuUsed = (): number => {
const raw = env("PRODUCER_VP9_CPU_USED");
if (raw === undefined || raw === "") return DEFAULT_CONFIG.vp9CpuUsed;
return normalizeVp9CpuUsed(Number(raw));
};
const envBrowserGpuMode = (): EngineConfig["browserGpuMode"] => {
const raw = env("PRODUCER_BROWSER_GPU_MODE");
if (raw === "hardware" || raw === "software" || raw === "auto") return raw;
Expand Down Expand Up @@ -326,6 +338,7 @@ export function resolveConfig(overrides?: Partial<EngineConfig>): EngineConfig {
DEFAULT_CONFIG.enablePageSideCompositing,
),

vp9CpuUsed: envVp9CpuUsed(),
enableChunkedEncode: envBool(
"PRODUCER_ENABLE_CHUNKED_ENCODE",
DEFAULT_CONFIG.enableChunkedEncode,
Expand Down Expand Up @@ -392,9 +405,13 @@ export function resolveConfig(overrides?: Partial<EngineConfig>): EngineConfig {
// Remove undefined values so they don't override defaults
const cleanEnv = Object.fromEntries(Object.entries(fromEnv).filter(([, v]) => v !== undefined));

return {
const merged = {
...DEFAULT_CONFIG,
...cleanEnv,
...overrides,
};
return {
...merged,
vp9CpuUsed: normalizeVp9CpuUsed(merged.vp9CpuUsed),
};
}
6 changes: 6 additions & 0 deletions packages/engine/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,12 @@ export type {

// ── Configuration ──────────────────────────────────────────────────────────────
export { resolveConfig, DEFAULT_CONFIG, type EngineConfig } from "./config.js";
export {
DEFAULT_VP9_CPU_USED,
MAX_VP9_CPU_USED,
MIN_VP9_CPU_USED,
normalizeVp9CpuUsed,
} from "./services/vp9Options.js";
export {
getSystemTotalMb,
isLowMemorySystem,
Expand Down
14 changes: 12 additions & 2 deletions packages/engine/src/services/chunkEncoder.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -918,7 +918,7 @@ describe("buildEncoderArgs lockGopForChunkConcat", () => {
expect(args[args.indexOf("-g") + 1]).toBe("240");
expect(args[args.indexOf("-keyint_min") + 1]).toBe("240");
expect(args[args.indexOf("-auto-alt-ref") + 1]).toBe("0");
expect(args[args.indexOf("-cpu-used") + 1]).toBe("2");
expect(args[args.indexOf("-cpu-used") + 1]).toBe("4");
expect(args[args.indexOf("-deadline") + 1]).toBe("good");
expect(args.indexOf("-x264-params")).toBe(-1);
expect(args.indexOf("-x265-params")).toBe(-1);
Expand All @@ -934,14 +934,24 @@ describe("buildEncoderArgs lockGopForChunkConcat", () => {
);
expect(args).not.toContain("-g");
expect(args).not.toContain("-keyint_min");
expect(args).not.toContain("-cpu-used");
expect(args[args.indexOf("-cpu-used") + 1]).toBe("4");
// The non-locked, non-alpha VP9 path leaves `-auto-alt-ref` at the
// libvpx default. Alpha branches still emit `-auto-alt-ref 0` for an
// unrelated reason (alpha + alt-ref is unsupported), but that's a
// separate test below.
expect(args).not.toContain("-auto-alt-ref");
});

it("honors the resolved engine VP9 cpu-used override", () => {
const args = buildEncoderArgs(
{ ...baseOptions, codec: "vp9", preset: "good", quality: 23, vp9CpuUsed: 6 },
inputArgs,
"out.webm",
);

expect(args[args.indexOf("-cpu-used") + 1]).toBe("6");
});

it("true with alpha pixel format keeps alpha metadata and emits -auto-alt-ref once", () => {
// Regression: alpha + closed-GOP must NOT double-push `-auto-alt-ref 0`.
// Both paths want it disabled; the encoder branch emits it exactly once.
Expand Down
Loading
Loading