Skip to content

fix(export): align exported audio by restoring the AAC encoder delay - #219

Merged
0xsline merged 1 commit into
0xsline:mainfrom
autober:pr/export-audio-offset
Oct 18, 2026
Merged

0xsline merged 1 commit into
0xsline:mainfrom
autober:pr/export-audio-offset

Conversation

@autober

@autober autober commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

fix(export): exported audio plays 42.7 ms late

中文摘要:服务端导出的 MP4/MOV 音频比画面晚约 42.7 ms(48 kHz 下 2048 个采样)。原因是 Remotion 用 libfdk_aac 把音轨编码成裸 ADTS 再直接封装,而 ADTS 无法记录编码器的预滚(priming)。修复方式:renderMedia 之后无损重封装一次,把音频按编码器延迟前移,由容器以 edit list 记录。在 v0.2.17 上实测:导出的每个提示音都晚 42.7 ms,修复后为 0.0 ms。

The problem

Every server export (mp4/mov via renderMedia) has its audio 42.7 ms behind the picture at 48 kHz. It's most visible on talking heads (lips lead the voice) and on sound effects cut to a beat or a motion graphic.

Cause. Remotion 4.0.509 encodes the render's AAC track with libfdk_aac into a raw ADTS stream (compress-audio.js: -c:a libfdk_aac -f adts), then stream-copies it into the container. FDK's AAC-LC encoder emits 2048 samples of priming before the first real sample, and ADTS has no way to signal that. So the container gets audio at pts 0 with no edit list, and every player hears the whole track 2048 samples late. Decode and mixing are sample-exact; a WAV render of the same timeline is aligned. Only the ADTS encode adds the lag.

The fix

After renderMedia, remux the file losslessly with the audio shifted back by the encoder delay (-itsoffset -2048/sampleRate, -c copy). The priming packets then carry negative timestamps, which the mp4/mov muxer records as an edit list (media_time = 2048). That's the same signalling ffmpeg's own AAC encoder writes for ordinary media.

  • Video packets are copied untouched (the test checks their md5).
  • Safe no-op when the audio isn't AAC, when the priming is already signalled (negative first pts), or when there's no audio, so running it twice is harmless.
  • Uses Remotion's bundled ffmpeg/ffprobe (RenderInternals.callFf), with the same binaries directory and abort signal as the render, so it works in the asar-packaged app.
  • Only the renderMedia path changes. The direct-hardware path already re-encodes audio with ffmpeg's native AAC encoder, which signals priming correctly, so it's left as is. directHardwareRenderer keeps the current attempt.ffmpegOverride check.

New file remotion/aac-encoder-delay.mjs; small hook in remotion/render.mjs.

Evidence

End-to-end export on v0.2.17. I exported a 3 s timeline with a beep track (six 1 kHz beeps) through the real renderTimeline pipeline (Remotion + headless Chromium + libx264/AAC), then measured where each beep lands in the exported file:

Beep onsets vs. source
main (v0.2.17) +42.7 ms on all 6 beeps
With this PR 0.0 ms on all 6 beeps

New check remotion/aac-encoder-delay.verify.mjs (registered in test:serial). It first asserts its premise, that Remotion still writes AAC as libfdk_aac ADTS, so a future Remotion change that makes this unnecessary is caught. Then it rebuilds Remotion's audio encode and mux argument for argument and measures beep onsets against the source PCM: 42.7 ms before → 0.0 ms after. It also checks:

  • the video packets are byte-identical;
  • both streams still start at 0;
  • a second pass is a no-op;
  • already-signalled AAC is left alone;
  • a silent render isn't touched;
  • non-AAC/webm/muted renders are skipped.

Required checks. npm run lint, npx tsc -b and npm run build pass. Running every check from pretest and test:serial individually on Node 24: 583/592 pass. The other 9 fail identically on untouched main in my environment (Chrome-based render checks, no network for fonts and Hugging Face, running as root). They're unrelated to this change.


Written with help from Claude (Anthropic); I reviewed and tested the changes.

Remotion encodes a render's AAC track with libfdk_aac into raw ADTS and
stream-copies it into the mp4/mov. FDK's 2048-sample priming cannot be
signalled in ADTS, so exports played audio 42.7 ms late at 48 kHz.

After renderMedia, remux losslessly with the audio shifted back by the
encoder delay so the container records it as an edit list. No-op when the
audio is not AAC, priming is already signalled, or there is no audio.
Beep-fixture export: 40-41 ms lag before, -2 ms after (fixture: -1 ms).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@0xsline
0xsline merged commit 5ed2e37 into 0xsline:main Oct 18, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants