More unnecessary transcoding

[tgurath 7]

Server version 4.7.10.0  with Vivaldi as client |  Theater version 3.0.18  |  Running everything on Windows 11

This doesn't seem related to the other recent transcoding problem post so I opted for a new topic.

I've been playing with video in AV1 format along with audio encoded with varying codecs. I tried mkv, mp4 and webm containers.

1. If audio needs transcoding to play in browser, Server appears to always also transcode the video to h264 even if not needed. This is a performance killer with 4k videos.

2. If there are multiple audio streams, Server isn't smart about picking one it can play without transcoding. I added opus and aac to a video that had dts-hd. All 3 were marked as default. Server managed to pick dts-hd even when it wasn't the first audio stream. If opus is the only one marked as default Server selects it correctly.

I'm no expert in content mastering but it seems if there are multiple languages, some in multiple formats, you'd want to mark all of the primary language streams as default and let the playback code choose the best one it's capable of playing. In my opinion, for Server to send to a browser, to find the best audio stream: find streams marked as default that don't need transcoding, sort by channels descending and then bit rate descending. A ranked list of codecs is another possibility. What doesn't need transcoding varies by browser, unfortunately. Vivaldi lets me use opus with 7.1 channels. Edge lets me use opus but fails with 7.1 channels.

Ideally, if I have a video with a dts-hd stream and an opus or aac stream, I want Server with a web client to pick opus or aac and I want Theater to pick dts-hd.

3 If audio needs transcoding, as mentioned in #1 it causes video transcoding. Switching to a different audio stream, which doesn't need transcoding, doesn't make the video transcoding stop. Either that or I wasn't patient enough in choppy perf to wait for it to eventually switch to direct play. Of course, fixing #1 seems it would also prevent #3 but I thought I'd mention it in case there's issues with stream switching such as also triggering video transcoding. It should be tested when #1 is fixed.

4. Looking at transcode logs, h264 max bit rate is being set to match the bit rate of the source video. That made sense some years ago when h264 was likely the more efficient codec of the two. These days it's limiting the inferior but more compatible codec to the bit rate of more efficient codecs like hevc and av1. That doesn't make sense. For those codecs, their supposed % efficiency improvement could be applied. For instance, hevc is reported to be 25% to 50% more efficient. Perhaps h264 max bit rate should be 150% of the hevc bit rate. Or 135% to pick a number in the middle. And some higher % for av1.

Theater HDR -> SDR problem

To add more fun to my day, Emby Theater stopped correctly playing HDR content with tone mapping for SDR display. It displays a bunch of vertical bands in colors relevant to what should be on screen. I'm guessing this may be caused by the recent mpv update as the problem just started today. Yesterday I played an HDR source fine. Today continuing the same video from yesterday is broken. This only affects Theater, not Server.

 

To end on a positive note: putting av1 + opus in an mp4 or webm container (with no other streams) correctly direct plays in the browser. Also, av1 + opus + other audio streams with only opus marked as default direct plays correctly.

Edited December 8, 2022 by tgurath
Added more detali

[tgurath 7]

I looked at a couple of bluray sources and they have only one of each type of stream (i.e. one audio, one subtitle) set as default. It's consistently the best one, which is great for Theater but with current code will always transcode for Server with a web client. If the bug causing video to transcode when only audio needs transcoding is fixed it'll solve my use case since I'm down to only a stereo setup for a while. (I moved to different country and I'm living in furnished condo with no surround sound.)

In the long term, it'd be nice to see some effort to transcode to the playback platforms' capabilities rather than always dropping to the lowest common denominator of stereo mp3. AAC supports 8 channels. Opus supports more than that. Some browsers are supporting 8+ channels. Dolby says DD+ is supported in some browsers. https://developer.dolby.com/platforms/html5/browser-support/

 

[Luke 42085]

Hi there, we're happy to help. 

 Let's look at an example. Please attach the information requested in how to report a media playback issue. Thanks!

 

[tgurath 7]

Theater version 3.0.19 fixed the broken HDR playback in Theater version 3.0.18, so thanks for that. Now I can watch other content using Theater to avoid this transcoding issue while I'm encoding stuff in AV1 in the background.

Logs and screenshots of "stats for nerds" from Theater and Vivaldi are attached. You can see the reason given for transcoding is audio but it's also unnecessarily transcoding the video. Also, if the transcode is truly using my gpu I'm surprised the playback is so terrible. Not just stuttery but long pauses. My CPU is maxed out encoding AV1 in the background but my GeForce should be idle.

I restarted the server so you'd get clean logs. The movie I tested was The Fast and the Furious if you want to search for that but there's nothing else in there anyway.

embyserver-63806231364.txt ffmpeg-transcode-22e158f5-896c-4aae-b951-fa0ce0d05b2e_1.txt hardware_detection-63806231393.txt

[ebr 16190]

2 hours ago, tgurath said:

You can see the reason given for transcoding is audio but it's also unnecessarily transcoding the video

Hi.  Once we have to convert audio, then it has to be re-packaged into HLS which then restricts what video formats can be delivered.

[tgurath 7]

h265 and AV1 can both be sent via HLS. There's no reason to transcode those for clients that support them.

Evidence of legit support, not just some guy's hack:
https://developers.cloudflare.com/stream/viewing-videos/av1-playback/

Edited December 9, 2022 by tgurath

[tgurath 7]

@Luke @ebr Is that it then? We're just going to pretend it's still 2016 (birth of the HLS spec) and keep forcing everything to h264 and on top of that keep setting a max bit rate that is too low for that codec when coming from better codecs?

Apple has supported h265 over HLS since iOS 11, 4 years ago.
Amazon supports serving h265 over HLS.
For AV1, aside from the Cloud Flare link I shared above, this reddit post from early 2022 may be useful: