Repository navigation
fix(export): align exported audio by restoring the AAC encoder delay - #219
Merged
Merged
Conversation
Remotion encodes a render's AAC track with libfdk_aac into raw ADTS and stream-copies it into the mp4/mov. FDK's 2048-sample priming cannot be signalled in ADTS, so exports played audio 42.7 ms late at 48 kHz. After renderMedia, remux losslessly with the audio shifted back by the encoder delay so the container records it as an edit list. No-op when the audio is not AAC, priming is already signalled, or there is no audio. Beep-fixture export: 40-41 ms lag before, -2 ms after (fixture: -1 ms). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix(export): exported audio plays 42.7 ms late
中文摘要:服务端导出的 MP4/MOV 音频比画面晚约 42.7 ms(48 kHz 下 2048 个采样)。原因是 Remotion 用 libfdk_aac 把音轨编码成裸 ADTS 再直接封装,而 ADTS 无法记录编码器的预滚(priming)。修复方式:renderMedia 之后无损重封装一次,把音频按编码器延迟前移,由容器以 edit list 记录。在 v0.2.17 上实测:导出的每个提示音都晚 42.7 ms,修复后为 0.0 ms。
The problem
Every server export (mp4/mov via
renderMedia) has its audio 42.7 ms behind the picture at 48 kHz. It's most visible on talking heads (lips lead the voice) and on sound effects cut to a beat or a motion graphic.Cause. Remotion 4.0.509 encodes the render's AAC track with
libfdk_aacinto a raw ADTS stream (compress-audio.js:-c:a libfdk_aac -f adts), then stream-copies it into the container. FDK's AAC-LC encoder emits 2048 samples of priming before the first real sample, and ADTS has no way to signal that. So the container gets audio at pts 0 with no edit list, and every player hears the whole track 2048 samples late. Decode and mixing are sample-exact; a WAV render of the same timeline is aligned. Only the ADTS encode adds the lag.The fix
After
renderMedia, remux the file losslessly with the audio shifted back by the encoder delay (-itsoffset -2048/sampleRate,-c copy). The priming packets then carry negative timestamps, which the mp4/mov muxer records as an edit list (media_time = 2048). That's the same signalling ffmpeg's own AAC encoder writes for ordinary media.RenderInternals.callFf), with the same binaries directory and abort signal as the render, so it works in the asar-packaged app.renderMediapath changes. The direct-hardware path already re-encodes audio with ffmpeg's native AAC encoder, which signals priming correctly, so it's left as is.directHardwareRendererkeeps the currentattempt.ffmpegOverridecheck.New file
remotion/aac-encoder-delay.mjs; small hook inremotion/render.mjs.Evidence
End-to-end export on v0.2.17. I exported a 3 s timeline with a beep track (six 1 kHz beeps) through the real
renderTimelinepipeline (Remotion + headless Chromium + libx264/AAC), then measured where each beep lands in the exported file:main(v0.2.17)New check
remotion/aac-encoder-delay.verify.mjs(registered intest:serial). It first asserts its premise, that Remotion still writes AAC as libfdk_aac ADTS, so a future Remotion change that makes this unnecessary is caught. Then it rebuilds Remotion's audio encode and mux argument for argument and measures beep onsets against the source PCM: 42.7 ms before → 0.0 ms after. It also checks:Required checks.
npm run lint,npx tsc -bandnpm run buildpass. Running every check frompretestandtest:serialindividually on Node 24: 583/592 pass. The other 9 fail identically on untouchedmainin my environment (Chrome-based render checks, no network for fonts and Hugging Face, running as root). They're unrelated to this change.Written with help from Claude (Anthropic); I reviewed and tested the changes.