Bitrate when tonemapping far too high

[MSI2017 39]

21 minutes ago, GrimReaper said:

It doesn't - it just serves you pre-encoded media with lower bitrate than your connection can handle, they don't transcode in real time. Maybe this can shed some light how they do it:

Yeah but even so my point still stands, it chooses well and switches over to the best quality when it can. Wether those files are pre-existing or need to be transcoded on the fly. I don't get @cayars saying Emby server has it 100x harder than Netflix. It should check what the device can handle (for web for example use this) and transcode if it cannot, direct play when it can. When direct playing isn't working out (poor buffer health or dropped frames for a prolonged time) it should drop down the bitrate. I think at times Emby is making it harder for itself than needed. For example, I have certain files that have EAC3 or AC3 audio, which on a MacBook (or all Chrome instances at least) needs to be transcoded. Instead of just taking the audio, it also starts transcoding the video which for HDR files also means losing HDR.

@softworkzI am genuinly quite interested in how this stuff works, how does Emby server choose how to handle playback? Also if I say something dumb in my post please point it out!

Edited January 30, 2023 by MSI2017

[ebr 14904]

6 hours ago, MSI2017 said:

don't get @cayars saying Emby server has it 100x harder than Netflix

Because they have exactly one copy of a media item that they totally control and can create pre-encoded versions of it at whatever steps of bitrate they desire and then very easily switch between them at playback.  We, on the other hand, have to handle thousands of variations of the same item without pre-encoded versions at varying bitrates.  Instead, we have to create those on the fly at the time of playback.

6 hours ago, MSI2017 said:

When direct playing isn't working out (poor buffer health or dropped frames for a prolonged time) it should drop down the bitrate

That can be a double-edged sword because you cannot just have that side of the equation.  What if the drop in bandwidth was temporary?  We have to somehow discover that and then raise the bitrate again.

There are definitely improvements that can be made but they are really much more complex than you may realize.

[MSI2017 39]

10 minutes ago, ebr said:

Because they have exactly one copy of a media item that they totally control and can create pre-encoded versions of it at whatever steps of bitrate they desire and then very easily switch between them at playback.  We, on the other hand, have to handle thousands of variations of the same item without pre-encoded versions at varying bitrates.  Instead, we have to create those on the fly at the time of playback.

This is basically what Cayars was saying what I don't understand (but want to so an explanation would be cool). Transcoding speed is more than fast enough to act as pre-encoded files. Say netflix has 10 on hand at a different bitrate. They still have to make a choice which one to serve, just like Emby. The only difference is Emby needs to transcode to get that file, whereas netflix uses extra HDD space to have those files ready to go. Transcoding is fast enough, its just making the wrong transcoding decisions. And if the connection is a bit wonky and there's doubt which bitrate to drop down two, you could just have two ffmpg instances going at two different outputs untill they playback and therefore connection quality indication advances a bit. 

 

10 minutes ago, ebr said:

That can be a double-edged sword because you cannot just have that side of the equation.  What if the drop in bandwidth was temporary?  We have to somehow discover that and then raise the bitrate again.

There are definitely improvements that can be made but they are really much more complex than you may realize.

Yeah that's fair but not when it's dropping frames or just outright crashing for a prolonged time? I can make the playback be total shit for 5 minutes and yet emby wouldn't decide to do anything about it. Dropping the bitrate the moment a frame gets dropped would be stupid yeah but after a while a decision needs to be made, and at the moment no decision is made. I get that it is very complex but the current approach is not working.  

 

Don't get me wrong, I love Emby but that's why I am so passionate about this. I see the LTT video of today all about Jellyfin and I want the next video him choosing Emby, because I think its good. That's why it is a bit tough for me to see the "there are definitely improvements that can be made" without much concrete stuff. That's why I start a thread like this. I love the honest conversation, the back and forth and hope it brings everyone and Emby a few steps forward.

Edited January 30, 2023 by MSI2017

[softworkz 3326]

2 hours ago, MSI2017 said:

Transcoding speed is more than fast enough to act as pre-encoded files.

There's no "act as pre-encoded files". Either you have those pre-encoded  already (that's the meaning of "PRE") or you transcode them when needed - which is "live transcoding".

2 hours ago, MSI2017 said:

They still have to make a choice which one to serve, just like Emby

No. They offer all and their client chooses.

2 hours ago, MSI2017 said:

Transcoding is fast enough, its just making the wrong transcoding decisions. And if the connection is a bit wonky and there's doubt which bitrate to drop down two, you could just have two ffmpg instances going at two different outputs untill they playback and therefore connection quality indication advances a bit. 

It's so much more complicated than that.

2 hours ago, MSI2017 said:

Dropping the bitrate the moment a frame gets dropped

Frame drops are not an indication for low bandwidth.

[MSI2017 39]

8 hours ago, softworkz said:

There's no "act as pre-encoded files". Either you have those pre-encoded  already (that's the meaning of "PRE") or you transcode them when needed - which is "live transcoding".

So it is not possible to sortof simulate that? Say the max file is a 4K 40mbps BR rip and the connection cannot handle it, or the client cannot handle 4k. In this case Netflix would fetch one of the pre-encoded files (1080p 8mpbs for example) and serve that (although Netflix starts with a lower bitrate and works its way up, which would be a nogo for Emby I guess). Emby would have to start with the highest quality one (or prefebly be able to tell in advance if that's going to work or not but I get that is hard), and transcode the file to a 1080p 8mbps, which it can do at faster than realtime. And if the connection is really doubtfull you could even have two transcodes going, one at a lower bitrate still and more easily switch over is needed, or stop the second transcode when the connection holds and it is not needed. 

8 hours ago, softworkz said:

It's so much more complicated than that.

But why, the above is my thinking which is apparently wrong, but I'd love to know some more details instead of "its complicated" or "there is room for improvement"

8 hours ago, softworkz said:

No. They offer all and their client chooses.

Ok but wether it is server or client making that choice, at least a choice is being made, whereas I can get a device to have the worst playback experience ever and it would do absolutely nothing about it. 

 

Regarding the framedrops, yeah that probably isn't a good indicator for bandwith problems, but in my case (both on the Chromecast and the laptop I tested on later) the drops were in the thousand of frames which would all drop at once. So more like massive playback stutters. The Chromecast plays for a bit than completely stops playback (but not subs) and that's it. On the PC I later tested it drops like multiple seconds of playback with CPU, GPU, RAM all being barely used. Leaving connection I guess? This is with tonemapping btw.

[ebr 14904]

2 hours ago, MSI2017 said:

I can get a device to have the worst playback experience ever and it would do absolutely nothing about it. 

Exactly how?  If you use our "automatic" setting the app will test bitrate before playback so the only way this should happen is if you have widely fluctuating available bandwidth.  In that case, you would want to just manually set a lower bitrate.

2 hours ago, MSI2017 said:

but in my case (both on the Chromecast and the laptop I tested on later) the drops were in the thousand of frames

Perhaps you are not having a bandwidth problem at all because frame drops are an indication of the player not being able to keep up not bandwidth limitations.  Buffering would indicate bandwidth issues.

[MSI2017 39]

6 hours ago, ebr said:

Exactly how?  If you use our "automatic" setting the app will test bitrate before playback so the only way this should happen is if you have widely fluctuating available bandwidth.  In that case, you would want to just manually set a lower bitrate.

Leaving it on auto it transcodes and tonemaps my 4k video to SDR 1080p, and it cannot keep up, the server sees it (on the dashboard check how to playback position jumps backwards) and happily keeps it stuttering. https://youtu.be/SzRz81zSugU

6 hours ago, ebr said:

Perhaps you are not having a bandwidth problem at all because frame drops are an indication of the player not being able to keep up not bandwidth limitations.  Buffering would indicate bandwidth issues.

Chromecast itself would just playback after a few seconds. It works for a few seconds, than stops (but subs keep going) and never ever recovers. That's the screenshot that started this post. And again, its making a bad decision because (for one thing choosing to turn h265 in 264 rather than keeping it in265 which Chromecast supports) it just keeps going and does nothing about it. 

Edited January 31, 2023 by MSI2017

[softworkz 3326]

On 1/31/2023 at 11:43 AM, MSI2017 said:

Regarding the framedrops, yeah that probably isn't a good indicator for bandwith problems, but in my case (both on the Chromecast and the laptop I tested on later) the drops were in the thousand of frames which would all drop at once. So more like massive playback stutters. The Chromecast plays for a bit than completely stops playback (but not subs) and that's it. On the PC I later tested it drops like multiple seconds of playback with CPU, GPU, RAM all being barely used. Leaving connection I guess? This is with tonemapping btw.

 

20 hours ago, MSI2017 said:

On 1/31/2023 at 1:47 PM, ebr said:

Perhaps you are not having a bandwidth problem at all because frame drops are an indication of the player not being able to keep up not bandwidth limitations.  Buffering would indicate bandwidth issues.

Chromecast itself would just playback after a few seconds. It works for a few seconds, than stops (but subs keep going) and never ever recovers. That's the screenshot that started this post. And again, its making a bad decision because (for one thing choosing to turn h265 in 264 rather than keeping it in265 which Chromecast supports) it just keeps going and does nothing about it. 

Okay, then you are talking about something totally different from "frame drops". 

On 1/31/2023 at 11:43 AM, MSI2017 said:

the drops were in the thousand of frames which would all drop at once

Most likely nothing is getting dropped at all. Probably not a single frame. Playback just stops waiting for data and once it gets the data it just continues at the position where it stopped.

[MSI2017 39]

16 minutes ago, softworkz said:

 

Okay, then you are talking about something totally different from "frame drops". 

Most likely nothing is getting dropped at all. Probably not a single frame. Playback just stops waiting for data and once it gets the data it just continues at the position where it stopped.

That's more likely yes, but still my main issue. If this happens, why does Emby let it keep happening? It will never ever drop down the bitrate.

[MSI2017 39]

On 2/1/2023 at 4:43 PM, MSI2017 said:

That's more likely yes, but still my main issue. If this happens, why does Emby let it keep happening? It will never ever drop down the bitrate.

@Luke,is this expected behavior or a bug? Sortof similar, if I actually play content like 4K stuff on an iPad or MacBook which cannot handle 4K, it will actually drop frames (so not a connection problem) and will forever keep that happening, not dropping down.  To tie in with my feature request for different playback quality names, when I told my parents that their iPad cannot handle 4K and should choose 1080p they still experienced dropped frames because choosing they chose the first 1080p option below 4k, meaning the bitrate was still higher than original, meaning it did nothing, despite there being 1080p in the name. Understandable for me, totally confusing for them

[Luke 37022]

We'll look at lowering the bitrate. Thanks.

[MSI2017 39]

3 minutes ago, Luke said:

We'll look at lowering the bitrate. Thanks.

Cool thanks! Any thoughts on the other issues in this topic?