Transcode Throttle appears to be broken

[Happy2Play 9780]

Yes we will probably need to know the difference between BySegmentRequest and ByStreamBufferSize or have it documented somewhere.

[Luke 42077]

This isn't meant to be a personal preference and therefore won't be added to the user interface. It's only there for troubleshooting. Once the issue with Chromecast throttling is resolved there won't be any need for an option anymore.

[Carlo 4560]

So what should it be set to or do we not know yet?

[Luke 42077]

Just now, cayars said:

So what should it be set to or do we not know yet?

We are looking into the issue. Thanks.

[softworkz 5066]

• ByStreamBufferSize  
  is how it has always been until a few weeks ago
  => This controls throttling based on the playback position that the clients report to the server
  .
• BySegmentRequest 
  is what is currently being tested in the betas
  => Throttling is based on the HLS segments that a client is requesting
  This may help in cases when the playback position reporting doesn't work for some reason
  But it fails in cases where a client simply downloads all segments it can get. Then there's no throttling effect.

 

[neik 873]

On 10/27/2020 at 9:03 PM, neik said:

Here is a log when the throttle wasn't kicking in.
What I did: Just started the movie, lowered the quality to force transcoding and let it run f or couple of minutes. At the end the buffer was at more than 4 min and kept going.

ffmpeg-transcode-54d4d97f-0c78-481c-b4b8-3ce6d58af9f2_1.txt 150.05 kB · 3 downloads

This is still an issue on 4.6.0.8.

@softworkz, you said you made some changes, should they have tackled this?

[softworkz 5066]

11 hours ago, neik said:

@softworkz, you said you made some changes, should they have tackled this?

Have you made the change to the config file as explained above?

[neik 873]

On 12/16/2020 at 9:18 PM, softworkz said:

Have you made the change to the config file as explained above?

Just tested it with the second throttle method but the result is the same.
So, issue exist with both methods.

Btw. what one should I stick to for daily use? Shall I change it back to the default or do both work fine?

[softworkz 5066]

@neik - Could you please post an ffmpeg log with the change applied?

[neik 873]

20 minutes ago, softworkz said:

@neik - Could you please post an ffmpeg log with the change applied?

Sure.

ffmpeg-remux-4eaa1a98-7fa4-4746-b0f7-7631e0669b0f_1.txt

[softworkz 5066]

@neik - Thanks for the log.

In fact, throttling gets active as can be seen in the log. 

Our throttling works in a way that it doesn't completely stop processing in ffmpeg. If we would do so, this might trigger internal (or depending on the source also external) timeouts.
That's why throttling "only" (but strongly) slows down the processing - as the actual goal is to save server-side resources. In your case, the processing rate with throttling active is at about 10% of the video playback speed.

In the future - with systems becoming more powerful - we may probably need to increase the throttling "strength" or make it configurable.

Edited December 18, 2020 by softworkz

[neik 873]

7 minutes ago, softworkz said:

Our throttling works in a way that it doesn't completely stop processing in ffmpeg. If we would do so, this might trigger internal (or depending on the source also external) timeouts.
That's why throttling "only" (but strongly) slows down the processing - as the actual goal is to save server-side resources. In your case, the processing rate with throttling active is at about 10% of the video playback speed.

Already thought it would be that way.
Just wanted to make sure it is doing what it should.

Thank you for checking and confirming!

[softworkz 5066]

2 hours ago, neik said:

Btw. what one should I stick to for daily use? Shall I change it back to the default or do both work fine?

I consider the "ThrottleByStreamBufferSize" as the preferable mechanism. Its "drawback" is that it requires client applications to properly and regularly report the current playback position. But this is something that all Emby clients are normally doing. 

The other method, "ThrottleBySegmentRequest" works based on the segments that a client is requesting (trying to always be a few segments ahead of the most recently requested segment). As such, it doesn't need to know the playback position. The drawback of that method is, that the throttling behavior is no longer "predictable" and might appear confusing or "non-functional" to users because in that case, it is completely controlled by the clients.
Each client has a different byte-size or number of segments that it will pre-load and cache. Especially in cases where you have a high-bandwidth video at the server and a small-screen client like a phone, the client may choose to completely download all segments without pause because the transcoded version is just like less than a few dozen megabytes in total, while the server would need to run at full power to produce the small-size video from the original video (of course, that's an extreme example for illustration).

In those latter cases, users will come and report that "throttling doesn't work".

That's why I think the former method is better for default - it provides predictable behavior and leaves control at the server side.

[Carlo 4560]

Just for clarification since this thread was recently linked to:

BySegmentRequest

is now the default being used and set on all new Emby Installations now correct?
That's back to being the opposite of this thread.