Jump to content

Multi-Angle / Multi-Video Track MKV Support and Recommendations


funwithmedia

Recommended Posts

funwithmedia

As I'm slowing converting some of my library to MKV, I'm encountering multi-angle / multi-video-track DVD's, and I'm wondering how folks tend to handle these, as well as what the current (and likely future) support is in Emby (particularly Emby Theater) for being able to recognize and switch between multiple video tracks (or angles) contained in a single MKV file? I thought I came across a post indicating that Emby does not currently provide a way to switch between multiple angles/video-tracks, but I can't seem to find that post so I may have imagined it.

 

I realize that one option is to just rip 2 versions of the content, one with one angle/track, and the other with the other. The upside is that this route should circumvent any compatibility issues there might be with Emby or other players. The downside of course is potentially a lot of duplicated space (eg, in the case of a movie with just a few alternate scenes).

 

How well does Emby handle multi-angle MKV's?

 

And have any of you created multi-angle MKV's, and if so, what is your process? It appears to me that MakeMKV let's you pick one angle or the other, but it does not appear to me that it gives an option to combine them into a single file containing just the differing portions of each video track (and the ability, in supporting players, to switch between video tracks).

 

I can probably live with going the route of creating multiple complete files, each with a different video track / angle, but figured it is worth checking if there are more efficient, supported ways of going about all this.

 

Thanks!

:)

Link to comment
Share on other sites

Waldonnis

Matroska (mkv) containers support ordered chapters (which can be used for branching), but I'm not sure what software out there will actually write out or play such a file (never looked into it, honestly) or if Emby can deal with it.  I have a few branching movies, so I may play with this a bit when I get some time.

 

Many streaming protocols support multi-angle streaming, so custom DASH or HLS manifests can probably be created to do it as well.

Link to comment
Share on other sites

Waldonnis

Update: Just tried a simple segment linking test and the resulting files properly played in sequence using VLC and MPC-HC (yay).  ffmpeg seemed to ignore any streams not contained in the first mkv file, though (boo).  There was a branch for ffmpeg to add segment linking support, but I have no clue what the disposition was.  I would guess, based on light research, that playback support would be limited as well, as many devices likely just implement basic Matroska support and this is a more advanced (read: obscure outside of certain anime circles) feature of the container.

 

After that, I tried using ordered chapters in two different editions (which basically creates two "titles"/chapter lists in the same file).  This was an interesting test, since I concatenated all of the branched segments into one hunk of video and just used chapters as playlists and the chapter start/end times to segment them rather than using segment linking.  I used the first two segments of Rogue One for the test since it has several intro segments (one for each language provided). Here are the basics of the scenario:

 

Before - three files:

  • Segment 1: English text intro (~41sec)
  • Segment 2: French text intro (~41secs, identical runtime to seg 1)
  • Segment 3: Planet fly-by sequence (~7mins)
  • No chapters in any file

After - one file:

  • One video stream, combining all three segments in numerical order, so each segment's video starts where the previous segment's ended (e.g. seg 3 begins at ~82secs)
  • Two editions (ordered chapters flag set) and chapter sets:
  • First edition chapters - ch1 starts at 0sec, ch2 starts at ~82secs
  • Second edition chapters - ch1 starts at 41secs, ch2 starts at ~82secs

This setup worked really well for straight playback locally for the most part (requires a keyframe at the start of each chapter and a few other quirks to consider for it to do so, but it worked).  This time, VLC played fine, but MPC-HC ignored the second edition entirely and just played the entire video stream with wacky chapter positions (ffplay did the same).  Transcoding it on-the-fly would be problematic for a few reasons that I won't get into - most notably, ffmpeg utterly ignored everything except the first edition and dropped the ordered chapter flag, which essentially broke the nicely-ordered playback of even the first edition...as well as a few other oddities.  It would also need to seek a lot to transcode any particular "edition"...doable, but silly.  Segment linking would be clearly better, but that seems like a no-go as noted before.

 

Ultimately, I'd say it could be done, but it's a ton of work to set these files up properly (can be scripted, but still) and some work on the server side would probably be needed as well to work around ffmpeg's inability to deal with these features of the Matroska container.  As much as I'd love to see it, I just don't think it would be worth the work server-side until proper support is included in ffmpeg (there's a wishlist "bug" on the tracker for this, but no status updates in over nine months).  Also, direct playing these types of files would be equally spotty as evidenced by my two most-used players yielding different results (and ffplay completely ignoring the editions entirely as well), and remuxes aren't going to work either (again, due to ffmpeg).

 

I may still play with this a bit and look at the spec again to see if I can come up with other non-streaming-manifest ideas, but I'm not too hopeful.  I may have missed some option in ffmpeg as well, so I'll review the docs/code a bit as well.

  • Like 2
Link to comment
Share on other sites

Vidman

Btw you are discussing something different than the op. Branching to different chapters is different than switching to a different angle of the same scene

  • Like 1
Link to comment
Share on other sites

Waldonnis

Btw you are discussing something different than the op. Branching to different chapters is different than switching to a different angle of the same scene

 

How so?  At least on blu-rays, they're often implemented the same way as branching entire chapters (via mpls), and you're just swapping playlists when chosing a different angle.  I'm using chapter files to test things since segment linking wasn't working with ffmpeg, but segment linking is independent of chapters entirely and even chapters are a fluid concept (there can be hidden chapters to denote scenes or other such divisions...which can be used with segment UIDs).

 

As an example, my test file has two hidden subchapters to handle the linking of the two segments, and the parent chapter (which is visible on the UI) encompasses both of them (per edition).  If I were to treat those two different intro segments as different angles (which the disc actually does, btw), swapping between them is simply swapping which "edition" I'm using.  Same effect and basically the same mechanism...except I'm calling it edition swapping instead of playlist swapping.  In both cases, the individual "angle" segments are independent video files (m2ts on the BD, mkv in my tests) as well.

 

Small edit: There are implementation details when it comes to blu authoring that do differentiate the two.  In practice, though (at least when ripping), there aren't many (or any?) options to preserving that outside of structure copies.  I'm just trying to figure out a way to preserve that without the bulk....and most of the multi-angle discs that I've seen use branching anyway to handle angles since they're not truly angle changes.  It's honestly all a bit out of my proverbial wheelhouse (I'm more used to straight encodes or branching), so if such ideas wouldn't solve the problem, then just ignore me  :P

Edited by Waldonnis
Link to comment
Share on other sites

Waldonnis

Bah, I'll hang my head in shame here...Vidman's right.  I just did a little more research and my limited collection of "multi-angle" stuff isn't actually multi-angle, but mimicking it with seamless branching instead.  In real multi-angle, there are strict bitrate/GOP requirements and the other angles are stored within the same video stream (they just skip the other angles' data when reading the stream).  GIven the restrictions, I can see why I haven't seen this on any of my discs and why branching is used instead (then just labeled as angles and presented as such via software).

  • Like 1
Link to comment
Share on other sites

funwithmedia

I agree -- it's all very confusing! But great work, @@Waldonnis, on your testing! This all will undoubtedly be helpful to any other folks who decide to explore these things in the future.

 

If you feel like continuing to play with this you might want to have a look at the MPV testing going on, as it sounds like Emby Theater will be transitioning to that for media processing, and it is possible it might handle some of this stuff (regardless of our mutual confusion on what are actual multi-angle discs, and what are branching/segments/whatever discs) better:

https://emby.media/community/index.php?/topic/46763-theater-mpv-test/

Link to comment
Share on other sites

ShoutingMan

For special features that are multi-angle, I use MKVtoolnix to concatenate the individual videos to preserve the content. For a complex branching system like the Cinderella blu-ray, I just gave up and kept the ISO to play properly.

Link to comment
Share on other sites

Waldonnis

mpv might help for HTPC/Theater, but I doubt it would help for some other platforms (like Roku, since it's mostly a closed system with a proprietary OS/API), so they would need transcoding...and without the linking support in ffmpeg, it's a no-go for those.  I have some uses for segment linking personally, though, so I'm glad I looked into it, even if it was born from me trying to solve an entirely different problem  :P 

 

Similar methodology may be useful for mimicking the functionality, but some conversion would be needed first (like breaking out the individual angles' data into their own streams).  Really, I wasn't far off in trying to find a solution - I was just basing it on branching segments rather than true multi-angle implementations, the latter of which brings some additional wrinkles with it.  I noticed a few things that I want to look into more in the Matroska spec anyway to see what other wacky things I can try.  I'm also digging through my DVDs to find one that has a real multi-angle presentation to play with, since my BDs don't seem to.

 

I stopped compiling mplayer and mpv locally since I rarely used them and didn't feel like resolving a missing dependency in my build environment, but may have to start again just to see how mpv's coming along and if it can handle these types of files.

Link to comment
Share on other sites

Waldonnis

Just a quick follow-up.  I demuxed and split out the individual angles from a special feature on the Hannibal DVD (it had five angles).  I then re-encoded and segmented each of those, and created an HLS master playlist and respective angle playlists (along with an extra that alternated angles periodically), which worked well as expected.  I've done alternate video tracks in HLS before, so I knew it would work, but hadn't had to extract individual angles from a DVD VOB like that.  I'm sure the same could be done with a DASH manifest, but I haven't tried it before.  Audio is independent of angle, so I didn't have to worry about that and just encoded the default language audio stream for testing.

 

I have no clue how to adapt something like this to Emby, though, and it required quite a bit of manual work just to get the angle streams extracted and HLS playlists created with the proper segment order/timing, but it was an interesting experiment.  If it supported/detected HLS playlists as media files and ignored the copious number of segment files, this could be workable - albeit still a pain to set up per movie.  The benefit of something like this would be some space savings to store multiple angles, as it would only store the alternate angle segments themselves rather than duplicating the whole file as an independent movie per angle.  Downside is it's definitely tedious and not an easy workflow to deal with per movie.

 

It was a fun experiment, though, and obviously a learning experience wrt angles.  I just wish I had come up with something easier to set up and that was usable in the Emby context.  I'll add it to my list of stuff to look at in my spare time, though, in case something changes or I think of another idea to try.

Edited by Waldonnis
Link to comment
Share on other sites

  • 6 months later...
Waldonnis

Apparently multi-track video in MKV is supported in Kodi.

https://forum.kodi.tv/showthread.php?tid=152756&pid=2567118#pid2567118

 

If implemented well, it could be a nice feature for Emby to support theatrical / directors / anniversary cut variations. Possibly also 2D / 3D releases.

 

Huh, never noticed that before.  I'll have to play with it some time, since I have a few files that could be merged easily (usually colour variants or scene order changes).

 

I would assume that it would be useful for multiple angles or simple variants (Logan and Logan Noir, for example), and I could see how 2D/3D could work as well.  Storing different cuts of a film, though...that could be messy, since they would likely be different lengths and require different subtitles/audio streams.  That would probably mean you would have to manually select the correct audio and subtitle streams after swapping video streams, though.

Link to comment
Share on other sites

  • 2 weeks later...
ShoutingMan

Huh, never noticed that before.  I'll have to play with it some time, since I have a few files that could be merged easily (usually colour variants or scene order changes).

 

I would assume that it would be useful for multiple angles or simple variants (Logan and Logan Noir, for example), and I could see how 2D/3D could work as well.  Storing different cuts of a film, though...that could be messy, since they would likely be different lengths and require different subtitles/audio streams.  That would probably mean you would have to manually select the correct audio and subtitle streams after swapping video streams, though.

 

I have no idea how it works, and haven't tried to install Kodi to experiment. I just do the multiple-folders dance to manage different versions for now. If this feature was implemented, I'd start spending time with MKVToolNix to see how it behaves. :)

Link to comment
Share on other sites

Waldonnis

I have no idea how it works, and haven't tried to install Kodi to experiment. I just do the multiple-folders dance to manage different versions for now. If this feature was implemented, I'd start spending time with MKVToolNix to see how it behaves. :)

 

I also hadn't tried it in Kodi before.  My usual array of software players have long supported multiple video tracks, but I hadn't thought of trying Kodi before you brought it up.

 

I gave it a shot by combining Logan and Logan Noir into one file and it worked quite well. The nice thing about this type of arrangement is that it would allow simple/language variants to share the same audio track selection...and given that some audio tracks are 7-12GB or larger, the space savings alone could be worthwhile.  I also tried multiple resolutions (albeit not using Kodi this time; all of my usual software players support multiple video tracks as well), which was generally fine (see below for the "but").

 

Multiple cuts...things get weird when you have tracks with different lengths in the same file.  I didn't try it with Kodi, but it seemed like each player my usual toolbox had its own way of dealing with it, and none of them were good.  It also effectively doubled the audio and sub track count, so I had to tag them well so I could figure out which audio/subs matched which video track.  I won't even get into how many steps it took to swap video, audio, and subtitles during playback...  In short, it was a big mess and something I don't personally think is worth the hassle to do or support.  Simple or language variants make sense and works well, but different "editions" is just too much of a PITA from a playback perspective.

 

The downside of all of this is that it's likely that some playback devices won't support this nearly as easily as Kodi or MPC-HC/BE do, and may require some remuxing or other trickery to work.    It's worthwhile noting that while players like MPC-HC and -BE handle these things well, VLC acted very strangely in every situation. This could be because I don't really use VLC for anything and I never bothered delving into the configuration, but it's not a good sign. If behaviour between common software players isn't even consistent, I shudder to think of how different client devices' behaviour could be.

  • Like 1
Link to comment
Share on other sites

  • 7 months later...
h.buckle

I have a few of these files for films with multiple versions (e.g. Apocalypse Now/Redux) so it would be good to see support. I use chapterEditor (https://forum.doom9.org/showthread.php?t=169984) to create them from seamless branching blu rays.

On the server side you'd need to update the media info provider. ffprobe does not properly detect multiple editions or hidden chapters - the only tool I've seen that does is mkvinfo from the mkvtoolnix suite.

On the player side LavFilters and MPV support choosing between editions and ordered chapters, I don't know if any others do.

Link to comment
Share on other sites

Waldonnis

I have a few of these files for films with multiple versions (e.g. Apocalypse Now/Redux) so it would be good to see support. I use chapterEditor (https://forum.doom9.org/showthread.php?t=169984) to create them from seamless branching blu rays.

On the server side you'd need to update the media info provider. ffprobe does not properly detect multiple editions or hidden chapters - the only tool I've seen that does is mkvinfo from the mkvtoolnix suite.

On the player side LavFilters and MPV support choosing between editions and ordered chapters, I don't know if any others do.

 

Unfortunately, support for ordered chapters in Matroska is iffy or entirely unsupported for many playback devices.  As I noted above, HLS and DASH can handle multi-angle easily, but creating the manifests/playlists can be tricky to automate.  The Matroska spec has quite a few nifty features like this, but hardly anyone bothers supporting them on the playback side.

 

Also, bear in mind that the ordered chapter enhancement request for ffmpeg is five years old and still open, so if the file needed to be transcoded for resolution, bitrate, or for playback on devices that don't support ordered chapters, you'd have to find some other way to perform the transcode.  This is something that I've been hoping to see support for in ffmpeg for years now, but haven't had the energy to write the code for it myself or port a patch from related projects like mpv.  A couple years back, someone submitted a patch to the ffmpeg folks that (kinda) worked, but I don't think it was ever accepted and I don't recall the reason.  Until ffmpeg/ffprobe support it, the only way to really play these back is via mpv/Kodi/whatever that supports ordered chapters directly.  If you also had other platforms like Roku that don't support it, you'd basically be stuck with an unplayable file that couldn't be transcoded either because ffmpeg can't deal with it properly.

Link to comment
Share on other sites

h.buckle

MPV could potentially be used on the server seeing as their internal demuxer can handle ordered chapters as well as still having all the options of ffmpeg. Their notes say direct stream copy doesn't currently work, but once that is fixed it could be an option

Link to comment
Share on other sites

Guest asrequested

That would be an interesting development. Even if mpv were used to decode and then passed to ffmpeg for re-encoding (no idea of that's even possible). That would have so much potential.

Link to comment
Share on other sites

Guest asrequested

And if it's just for transcoding on the fly to watch in real time, would it need to be repacked? Couldn't the output be directly streamed? You wouldn't need w temp transcode location. Just feed mpv the output parameters and let it go to work. But it would need to be an HLS, and be repacked, right? Damn!

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...