Progressive H.264 MP4 without CTTS judders on the native path

[orut34iop]

What happened?

On a progressive H.264 MP4 VOD with reordered pictures but no composition-time table, the native stream-copy path produces continuous forward/backward judder from the first frame. Seeking is not required to trigger it. The same source plays smoothly in another FFmpeg-based player.

The container presents every sampled video packet with PTS == DTS even though codecpar.video_delay == 1. A bounded libavcodec decode of the same head sample shows the contradiction directly: raw decoded AVFrame.pts regresses 16 times while best_effort_timestamp advances 58 times with zero regressions. Because the PTS values are present rather than AV_NOPTS_VALUE, the existing +genpts open flag and the #407 fallback do not repair them. Stream-copying those packet timestamps into fMP4 preserves decode order as presentation order, so AVPlayer displays future reference pictures before the B pictures that precede them.

Routing only this confirmed shape through the software decoder and scheduling decoded frames on best_effort_timestamp removes the judder.

Steps to reproduce

1. Open a seekable MP4 containing progressive H.264 with B-frame reordering, but whose sample table has no ctts composition offsets.
2. Play through the normal native VOD path.
3. Observe continuous small forward/backward motion from playback start.
4. Inspect the first 64 video packets and decoded frames. The reporting source produced: valid_pairs=64, PTS==DTS for all pairs, nonkey=62, decoded=59, best_effort_advances=58, best_effort_regressions=0, raw_pts_regressions=16.

A tiny 96x64 H.264 Main, 30000/1001 fps, 3-B-frame synthetic fixture and its healthy ctts twin are ready for a regression test in the accompanying PR.

AetherEngine version or commit SHA

6.39.0 / f70d4aee595f0947ef2354fbae01c76b41a7857b

Host app

Custom integration

Platform

tvOS

OS version

tvOS 26.6 beta (23L773)

Device / chip

Apple TV 4K (3rd generation)

Playback path

Native AVPlayer

Source media (for playback bugs)

MP4 / progressive H.264 with B-frame reordering / SDR. The source has no usable composition offsets: sampled packet PTS and DTS are equal despite video_delay=1. Audio is not involved in the failure. The synthetic regression fixture is MP4 / H.264 Main / 96x64 / 30000/1001 fps / no audio / SDR.

Error codes / log lines

No AVPlayer or VideoToolbox error is emitted. The diagnostic signature is:

video_packets=64 valid_pairs=64 nonkey=62 decoded_frames=59
best_advances=58 best_regressions=0 raw_regressions=16
raw_best_differences=16

Anything else

This is adjacent to #407 but not the same case. #407 repairs only AVFrame.pts == AV_NOPTS_VALUE; here the raw PTS is always present, but it is the wrong decode-order axis. A focused engine patch with bounded fail-closed detection, automatic per-session software routing, and good/bad MP4 fixtures is being prepared.

[superuser404notfound]

Confirmed, and thanks for a report that came with the asset to prove it. The twin fixtures from #411 are now the regression test.

I reproduced the failure on the Apple decoder rather than only in libavcodec, decoding both twins through AVAssetReader and matching each decoded frame's content against the healthy twin: with ctts removed, 45 of 66 pictures are presented at a time that belongs to a different picture, and the content order steps backwards 30 times, in a repeating +67 ms / -100 ms shuffle per B-group. That is the judder, and no seek is needed to provoke it, exactly as reported.

The fix is #413, and it takes a different route than routing the session to software decode.

The container lost the composition offsets, but the bitstream did not: every slice header carries a picture order count, which is display order. libavcodec's H.264 parser reads it without decoding a pixel, and it takes MP4's length-prefixed AVCC payload directly, so the repair costs one slice header parse per video packet. The demuxer then rewrites each packet to what the muxer should have written:

PTS = (decode time of the picture that opened this coded video sequence) + shift + rank * step
DTS = DTS + shift - reorderDelay * step

Pulling the decode time back by the reorder delay is what keeps PTS >= DTS, and a healthy file carries the same negative head. The shift is 0 or one reorder delay depending on whether the writer left the sample ladder on the presentation axis or kept the edit list that trims the reorder head, read from the container and clamped to that range, so both writer shapes land on the same axis.

Because the rewrite happens once at the demuxer boundary, the fMP4 producer, the segment plan, the software decoder and the still extractor all read one axis, and the source keeps the native path with hardware decode: a missing table no longer costs a route change.

Measurements on the branch:

• 432 packets across three fixture pairs (both edit-list shapes, seven IDR boundaries): every repaired packet matches its healthy twin's PTS and DTS exactly.
• served loopback output: all 301 frames present at the same times the healthy twin's output does, 0 mismatches, 0 backward steps.
• segverify 3/3 segments independently decodable, and a 13-seek burst lands identically on both arms, which is what exercises the re-anchoring after a jump.
• healthy files stay untouched: the sample leaves on the first real PTS/DTS offset, usually the first packet.

Detection is fail-closed: a source is only repaired when every sampled pair is equal, the decode ladder is uniform, the picture order regresses, and the ranks it produces are distinct and fill the sampled window. Variable frame timing without composition offsets, a picture order that does not advance one rank per picture, or a sample that cannot be anchored all leave the stream exactly as the container delivered it.

Leaving this open for a retest on the reporting asset once it ships in a release.

[superuser404notfound]

Merged as f08db21. It is on main now, not in a release yet, so this stays open until it can be retested on the reporting asset.

Two things worth naming from the verification pass, since both would have shipped silently:

• The segment plan is built from the container's index entries and then filled with these packets, so moving the decode timestamps without moving the index cut one segment past its keyframe. The index is now folded onto the same ladder.
• The verdict has to be settled while the demuxer still stands at the head. Taken later it lands wherever the cue prewarm left the source, which handed the producer a head six seconds into the file.

The picture-order parser is also dropped at a seek now, the way libavformat drops its own stream parser (ff_read_frame_flush). Landing on an IDR hides that, because the parser resets its state there, but an open-GOP entry point does not.

[superuser404notfound]

Released in 6.41.0. Your fixture pair is in the regression suite, and the release notes carry the twin-pair measurement in full, including why best_effort_timestamp could not be the axis. Leaving this open for your retest on the original file.