MVC 3D Blu-ray remux (stereo_mode=block_lr): left eye not played on iOS

[TheyCallMeSpy]

What happened?

Playing a 3D Blu-ray MVC remux (MKV, single H.264 track with both eyes in the same track, Matroska
stereo_mode=block_lr). The left eye is not what plays.

I expected the engine to fall back to plain 2D by playing the base view (left eye) and ignoring the
MVC dependent view, which is what every non-3D player does with these files.

The file has one video track. Both views ride in the same packets: packet cadence is 41.7 ms
(23.976 fps), so there is no separate stream or doubled packet rate to select from. The dependent
view slices reference a subset SPS the plain H.264 decoder does not have, which is why ffprobe and
ffmpeg print sps_id 1 out of range when they open it.

ffmpeg itself handles this fine. Decoding 3 seconds of the file gives 5 sps_id 1 out of range
lines and otherwise decodes the base view normally. So the software path already does the right
thing with this content, it is the native path that does not.

I could not find any MVC or multiview handling in the engine (only BDMV in a UDF reader test), and
docs/formats.md lists H.264 / HEVC / VC-1 for Blu-ray with no mention of MVC or 3D, so I think this
is simply not handled rather than broken.

Steps to reproduce

1. Play a 3D Blu-ray MVC remux, MKV with one H.264 High profile video track and Matroska
   stereo_mode=block_lr (both eyes laced into the same blocks).
2. Video is black.

Tested with TRON: Legacy (2010) 3D Remux 1080p, DTS-HD MA 7.1, AVC.

I have a second remux from a different group, Avatar: The Way of Water (2022) 3D Remux 1080p,
TrueHD Atmos 7.1, AVC, and ffprobe shows the same layout (one H.264 track, stereo_mode=block_lr,
same sps_id 1 out of range), so I expect it behaves the same, but I have not played that one yet.

AetherEngine version or commit SHA

2e388e3

Host app

Custom / my own integration
Moonfin for iOS 2.5.0

Platform

iOS

OS version

iOS 26.6

Device / chip

iPhone 15 Pro Max (A17 Pro)

Playback path

Native AVPlayer

Source media (for playback bugs)

MKV, H.264 High profile (MVC 3D Blu-ray remux), 1920x1080, 23.976 fps, DTS-HD MA 7.1, PGS subs,
13 streams, 40.6 GB, 2h05m.

Video stream, one only:

index=0
codec_name=h264
profile=High
width=1920
height=1080
TAG:title=MPEG-4 AVC Video / 13156 - 24283 kbps / 1080p / 23.976 fps / 16:9 / High Profile 4.1
TAG:stereo_mode=block_lr

stereo_mode=block_lr is Matroska StereoMode 13, both eyes laced in one block, left eye first.

Video packet timing for the first 2 seconds, one packet per frame time, so both views are inside the
same packets rather than in separate ones:

0.000000,134387,K__
0.042000,50373,___
0.083000,10163,___
0.125000,22714,___
0.167000,18170,___
0.209000,10182,___

Error codes / log lines

[h264 @ 0x55a2201634c0] sps_id 1 out of range


Printed by ffprobe on open and by ffmpeg during decode (5 times in a 3 second decode). This is the
MVC dependent view's subset SPS. No AVPlayer, CoreMedia, or VideoToolbox error was surfaced on my
side, the session plays, it just does not play the left eye.

The session was direct play. The Jellyfin server wrote no transcode or remux log for it, so the
engine got the original file untouched, nothing on the server side changed the stream.

I don't have the engine's own dispatch line for this session. The app's diagnostic log only has app
level stuff in it, and I'm on Windows so I can't read the iOS console. Let me know if you need it.

Anything else

Since both views are in the same packets, a stream copy to fMP4 hands VideoToolbox samples that
contain base view and dependent view NALs together, which I think is where this goes wrong. The
software path already does the right thing with the same file, so this looks like a routing question
rather than a decode one.

The container already says it. libavformat gives the Matroska StereoMode as the stereo_mode stream
tag, and block_lr / block_rl are the two values that mean both eyes are laced into one block, so
it can be read at load time without probing. The frame packed modes (side by side, top
bottom, anaglyph, interleaved) are single pictures that decode fine and should keep their normal
route, so only those two values would need to change anything.

That would work the same way as the rules already there for interlaced H.264, H.264 High 4:2:2 and
VP9, where the format looks native on paper but doesn't actually come out right, so it goes to
libavcodec instead. Stripping the dependent view NALs on the remux path would work too and
would keep hardware decode, but it is more work for the same visible result.

Real MVC 3D output would be great but is not what I am asking for here. I just want the left eye as
normal 2D, like every other player does with these files.

Things I have not checked yet. I didn't see the engine's dispatch line, so native path is from
reading the routing rules, not from a log. I only know the left eye isn't what plays, I don't know
what is showing instead. I haven't tried the same file transcoded, which would rule the source out
completely. And the second file I only checked with ffprobe, I didn't play it. I can do the last two.

Happy to test a build or provide a sample.

[superuser404notfound]

The reading in this report is right on every step, including which half is the routing question. The engine never read the stereo declaration at all, so a file like this took the native path on the strength of being progressive H.264, and VideoToolbox then got samples with both views' NALs inside and rendered nothing.

On main in 3819496:

• Routing reads the container's stereo declaration at load. H.264 declaring either both-eyes-in-one-block carriage (Matroska StereoMode 13 / 14, which libavformat reports as stream-level AV_STEREO3D_FRAMESEQUENCE and spells stereo_mode=block_lr / block_rl in the stream metadata) goes to the software path, where libavcodec skips the extension NALs and decodes the base view. That is the left eye as plain 2D, the same thing every non-3D player shows. No probe and no decode sample: the container says it before a packet is read.
• The frame-packed modes keep the native path with hardware decode, exactly as suggested. Side by side, top / bottom, checkerboard, row or column interleaved and anaglyph are single self-contained pictures, and cropping an eye out of one of those is a host decision, not the engine's.
• MV-HEVC is exempt for the same reason it should be: Apple's own format, native path plays its base layer.
• The dispatch now names the decision, so the line that was missing here exists: H.264 carries both stereo views in one track ... routing to the software path so the left eye plays as 2D. A host gets these by setting EngineLog.handler; everything at info level goes through it, including the dispatch line right above it.

Verified with aetherctl on synthesized clips: block_lr and block_rl dispatch software and decode, an untagged clip and a left_right clip stay native. The second remux should behave the same, since ffprobe already showed it carries the same declaration.

Two things worth naming rather than leaving implicit:

The rule is declaration-driven, so an MVC file that carries the views without declaring the carriage still takes the native path and still goes black. If one turns up, that is worth its own report with the ffprobe output, and a bitstream check is the answer there.

The software path means no hardware decode for that title. The alternative is the other approach mentioned here, stripping the dependent-view NALs during the remux so VideoToolbox gets a clean base-view stream and keeps hardware decode. That is the better outcome for an Apple TV in particular, and it is bitstream surgery that should not be written against a guess at the NAL layout. The offer of a sample is very welcome for exactly that: 30 to 60 seconds is plenty, video track only is fine.

ffmpeg -i input.mkv -t 60 -map 0:v:0 -c copy sample.mkv
ffprobe -v error -show_entries stream_tags=stereo_mode -of csv sample.mkv

The second line matters, since a cut that loses the declaration would test the wrong thing. A stream copy keeps it here.

Thanks for the report. The packet cadence, the container tag and the ffmpeg decode narrowed this to routing before anyone had to guess at it.

[superuser404notfound]

Released in 6.49.0. The routing rule is in it, so the TRON remux should play the left eye as 2D, and the dispatch line naming the decision is in the engine log if the host mirrors EngineLog.handler. The Avatar remux is worth playing too, since ffprobe showed it carries the same declaration. The sample offer stands for the follow-up question, whether the dependent-view NALs can be stripped during the remux so these files keep hardware decode.