Force Higher Bitrates

[softworkz 3338]

That is definitely an improvement. But it is still just fixing the symptoms instead of the cause.

 

 

The prober solution would of course be full blown adaptive streaming with client sending statistical data back to the server, changing bandwidth on the fly.

 

But lacking that a simpler bandwidth ladder system would go a long way. Something like this.

 

 

Today I've done a very detailed internal assessment regarding a similar concept.

I can't copy paste the results here. Quick summary: No

 

 

that flow chart wouldn't work anyway,    every Transcode at N, or Reduce to N would cause the stream to stop while ffmpeg builds a new stream.   They need to look into "on-the-fly bitrate changes"

 

@@pir8radio - Excellent! I came up with a long list o technical details and examples, but in the end it always burns down to this: Playback would stop unexpectedly and the user would need to wait again for the restart. That's just unacceptable.

 

Here's only a few selected key points:

• That approach works in one direction only: Reduction

  => So start with which value?

  • If the start value is low, it wouldn't optimize for higher available bandwidths
  • But for all lower available bandwidth values, it would mean that streaming will stop and restart at some time (or multiple times)

    This can happen after 10s or after more than a minute

• A player's 'buffering' state is not always an indication of low bandwidth (can have different reasons)

  .

• We can't calculate an estimated bandwidth from the download duration because it often includes time until transcoding of the segment is finished

  (and we might not even get that data from the player)

  .

Or in other words: A concept that is based on letting playback failures happen is a really bad idea.

[softworkz 3338]

I agree. It is a stop gap to make the best of what is possible with the current playback scheme in Emby. But with some very minimal statistic information from the client, it would be possible to make the <Reduce N> part smarter. So that you would get only one, maybe two restarts at most. And regardless much better then the current solution (or shall we say lack of solution..).

 

You cannot make that smart enough. What you haven't considered is how long it could take only until the the decision can be made that a restart may be required?

If the initially assumed bandwidth is 5 times higher than the actual one, it would take 30s until the first two segments are downloaded.

 

Then, the player might either wait for a 3rd segment or start playback.

 

A player's initial state on start is already 'buffering', so  we would always need to wait until the player's state changes to 'playing' and only from then on, wait until it switches to buffering.

Downloads are done by the players or hls implementations and we usually don't get at the individual download times, and even if we would, we wouldn't know how much of that time is from waiting until the segment is transcoded and ready and how long the actual download had taken, so we cannot even calculate a bandwidth from that.

 

There's even more to consider, but I'll stop here, that should be enough to get an impression why these things are not as they might seem at first sight.

[softworkz 3338]

So everyone that defaults to 1.5Mbps on Auto, means the bandwidth detection system failed to detect their proper bandwidth?  As that would appear to be the case in multiple topic.

 

Yes I see that it has been increased in Beta to 3.0Mbps.

 

 

That is definitely an improvement. But it is still just fixing the symptoms instead of the cause.

 

I agree that it's not a solution.

 

Anyway, this change has not only winners but losers as well.

[softworkz 3338]

I see lots of posts that say ffmpeg can not change bitrates on the fly:  https://stackoverflow.com/questions/55079295/how-to-change-encoding-bitrate-adaptively-to-bandwidth-during-live-rtmp-publish

 

And I see this ONE post that says you can, sort of maybe:  https://ffmpeg.org/pipermail/ffmpeg-devel/2016-August/197518.html

 

Adaptive streaming typically includes changing video resolutions as well, now google for that.

Oh, well and then bitrate changes, resolution changes in combination with hardware acceleration....

 

But first, let's see how adaptive streaming usually works:

• It doesn't operate on playback start

  .

• [....] 

  (leaving out a few things not needed to get to my point)

  .

• The purpose of adaptive streaming is to provide a streaming experience without interruption (opposed to the suggestion above)

  But a certain bandwidth needs to be chosen to start with. Now, there are several options:

  • Perform bandwidth detection and use a safe value lower than the detection result

    .

  • Use some platform API that provides a bandwidth value

    ,

  • (if the content is pre-encoded to multiple bitrates, start with the lowest variant to be safe)

    .

  • Fall back to some some pre-defined value

 

Recognize something? Even with adaptive streaming, we'd have almost the same situation.

 

Only difference is, that the pre-defined value is less important because quality will automatically be increased after a while.

Edited January 15, 2020 by softworkz

[cptlores 23]

You cannot make that smart enough. What you haven't considered is how long it could take only until the the decision can be made that a restart may be required?

If the initially assumed bandwidth is 5 times higher than the actual one, it would take 30s until the first two segments are downloaded.

 

Then, the player might either wait for a 3rd segment or start playback.

 

A player's initial state on start is already 'buffering', so  we would always need to wait until the player's state changes to 'playing' and only from then on, wait until it switches to buffering.

Downloads are done by the players or hls implementations and we usually don't get at the individual download times, and even if we would, we wouldn't know how much of that time is from waiting until the segment is transcoded and ready and how long the actual download had taken, so we cannot even calculate a bandwidth from that.

 

There's even more to consider, but I'll stop here, that should be enough to get an impression why these things are not as they might seem at first sight.

 

My impression from reading this is that there seem to be some inherent weaknesses in the streaming model used for Emby, that makes it not very well suited for streaming in 'difficult' environments like mobile etc. And as I said earlier, your number one job above anything else, is streaming video to the client.

 

I could also mention that Plex does dynamic adaptation (since 2017) just to rub it in, but I guess you are already aware of this.

[Carlo 4330]

On 1/15/2020 at 4:46 PM, cptlores said:

I could also mention that Plex does dynamic adaptation (since 2017) just to rub it in, but I guess you are already aware of this.

Commenting because this thread was recently linked from another thread.  I can tell you how this works at the mile high level because I showed them how to do it.

What they do is remember different speeds for different conditions to have them available to use as defaults. So for example the client can remember the last n number of C class blocks it's used and saves the last known speed.  This way it knows the default speed to start with in your home network, your work network, the cell network you use to drive to work, the network speed from the hotel you stayed at, etc...  It of course isn't perfect but right in the ballpark as it remembers and uses your last known bitrate setting from the last time you used that C block.

The app doesn't need to use any bandwidth tests but simply watches the speed of it filling it's own buffer.  If it can fill it's buffer quickly it has "excess bandwidth" and can ask for a bitrate increase.  If it's keeping the buffer just about filled it's right at the correct bitrate, and lastly if the buffer is decreasing then the bitrate is too high and needs to be reduced to the point it's just staying filled properly.  So it uses it's own metrics of bandwidth availability by watching the performance of "filling it's buffer" on the fly.

This only works when going through ffmpeg and does nothing for direct play. If stuttering happens during direct play the user would need to adjust or "fix" the stream to cause it to go through ffmpeg where the HLS packets can be changed on the fly (with a slight delay).

When the app senses a bitrate change is needed it send that request to the Server which uses the new bitrate setting when creating the NEW packets.  This generally works very good but can still have some issues if the quality of network changes drastically but the app catches up rather quickly and adjusts.  Something like this could happen if someone starts downloading a lot turning their 50mb downlink into a 3mb downlink.  Or multiple people start using the WIFI at the same time instead of just you, or you're moving throughout the house getting farther from the hotspot and your link speed decreases.

But that's the beauty of being able to adapt on the fly (or close to it) so the person learns the stuttering clears itself up rather quickly by decreasing the bitrate and as soon as the line clears up they get more bitrates as well (walking back closer to the hotspot).

Sort of a crude mans "adaptive" streaming using ffmpeg without pre-generated files of different bitrates/resolutions like Netflix, Prime, Hulu use.

In addition to being able to adjust the bitrate on the fly you can also enforce the MAX bitrate of the user or system as a whole.  So if you only have 15mb upload link on your Emby Server then that 15mb can influence and help change EVERY stream so everyone get's a share of the bandwidth vs one o two people hogging it all up.  You can also use the MAX bitrate differently.  So if you applied a 10mb limit to the user it would no longer be PER STREAM but to all streams the user has.  So 1 stream could use all 10 or 2 streams 5 each or 3 streams 3.3mb, etc.  Lots of possibilities when you think about how it could be used.

Edited August 6, 2020 by cayars

[MSI2017 40]

On 8/6/2020 at 5:44 AM, cayars said:

Commenting because this thread was recently linked from another thread.  I can tell you how this works at the mile high level because I showed them how to do it.

What they do is remember different speeds for different conditions to have them available to use as defaults. So for example the client can remember the last n number of C class blocks it's used and saves the last known speed.  This way it knows the default speed to start with in your home network, your work network, the cell network you use to drive to work, the network speed from the hotel you stayed at, etc...  It of course isn't perfect but right in the ballpark as it remembers and uses your last known bitrate setting from the last time you used that C block.

The app doesn't need to use any bandwidth tests but simply watches the speed of it filling it's own buffer.  If it can fill it's buffer quickly it has "excess bandwidth" and can ask for a bitrate increase.  If it's keeping the buffer just about filled it's right at the correct bitrate, and lastly if the buffer is decreasing then the bitrate is too high and needs to be reduced to the point it's just staying filled properly.  So it uses it's own metrics of bandwidth availability by watching the performance of "filling it's buffer" on the fly.

This only works when going through ffmpeg and does nothing for direct play. If stuttering happens during direct play the user would need to adjust or "fix" the stream to cause it to go through ffmpeg where the HLS packets can be changed on the fly (with a slight delay).

When the app senses a bitrate change is needed it send that request to the Server which uses the new bitrate setting when creating the NEW packets.  This generally works very good but can still have some issues if the quality of network changes drastically but the app catches up rather quickly and adjusts.  Something like this could happen if someone starts downloading a lot turning their 50mb downlink into a 3mb downlink.  Or multiple people start using the WIFI at the same time instead of just you, or you're moving throughout the house getting farther from the hotspot and your link speed decreases.

But that's the beauty of being able to adapt on the fly (or close to it) so the person learns the stuttering clears itself up rather quickly by decreasing the bitrate and as soon as the line clears up they get more bitrates as well (walking back closer to the hotspot).

Sort of a crude mans "adaptive" streaming using ffmpeg without pre-generated files of different bitrates/resolutions like Netflix, Prime, Hulu use.

In addition to being able to adjust the bitrate on the fly you can also enforce the MAX bitrate of the user or system as a whole.  So if you only have 15mb upload link on your Emby Server then that 15mb can influence and help change EVERY stream so everyone get's a share of the bandwidth vs one o two people hogging it all up.  You can also use the MAX bitrate differently.  So if you applied a 10mb limit to the user it would no longer be PER STREAM but to all streams the user has.  So 1 stream could use all 10 or 2 streams 5 each or 3 streams 3.3mb, etc.  Lots of possibilities when you think about how it could be used.

@softworkz Is there any update regarding this? Normally I use app.emby.media in my local netowrk but when testing out some ssl stuff I used the external domain in incognito and noticed that in safari it would playback at 720p (1080p content) and when trying a 4K file it chose 1080p60mpbs. 

I just assumed that they clicked this and would ask them to choose full quality since it is a bit of a waste to let the cpu transcode. But experiencing this myself (on a 1000 up 1000 down internet connection, this explained it.

Sometimes when checking the dashboard when my family is playing I see: Reducing bitrate due to quality setting.I just assumed that they clicked this and would ask them to choose full quality since it is a bit of a waste to let the cpu transcode. But experiencing this myself (on a 1000 up 1000 down internet connection, this explained it.

[ng4ever 39]

On 1/15/2020 at 3:46 PM, cptlores said:

 

My impression from reading this is that there seem to be some inherent weaknesses in the streaming model used for Emby, that makes it not very well suited for streaming in 'difficult' environments like mobile etc. And as I said earlier, your number one job above anything else, is streaming video to the client.

 

I could also mention that Plex does dynamic adaptation (since 2017) just to rub it in, but I guess you are already aware of this.

 

I agree sadly