Online Streaming Bitrate limit wrong calculated

[sooty234 266]

I laughed so hard at that, if I'd been drinking something, I'd have spat it out. Why is it that you make so many wrong assumptions? Maybe try reading what I wrote more carefully, and try to see where you went wrong. 

[ebr 16169]

13 hours ago, sooty234 said:

This 'feature' has never worked, because it shouldn't exist. If I use ffmpeg, DVDfab or any other software like that, and I set a bitrate, I get that bitrate. That's what it's supposed to be.

I'm sorry, but that is a different setting.  Our settings are MAX values, not absolute values.  When using conversion software, then those are absolute values.   Two different things for different purposes.

Seeing an ffmpeg log from your example playback would help clear a lot of this up for us (and find any issue with the logic if there is one).

Thanks.

[Carlo 4560]

3 hours ago, sooty234 said:

I laughed so hard at that, if I'd been drinking something, I'd have spat it out. Why is it that you make so many wrong assumptions? Maybe try reading what I wrote more carefully, and try to see where you went wrong. 

You're making assumptions of how you think it works vs how it does work.  As Luke has said it's a MAX value allowed for the stream, not a target.

If you want to set a target you do that on the client and change from AUTO to a specific setting such as 1080p - 8 Mbps.

[sooty234 266]

Oh boy! I guess nobody reads, anymore. 

The logs are worthless. You can't fix something that shouldn't exist.

[Luke 42077]

I'm happy to help, I'm just asking for a little additional information. Thanks.

[Painkiller88 248]

1 hour ago, Luke said:

I'm happy to help, I'm just asking for a little additional information. Thanks.

So what exactly do you need from me?

I uploaded everything, and for my understanding, if i set a User Limit for Remote Streams to 8 MBit it should not allow a user to get a videobitrate higher than 8MBit but it also should be as near as possible to the 8MBit limit and not 5 or 6 MBit, as my server has a higher upload and so there is no reason the clients "Auto" value is taking a lower bitrate.

And the test i did to set the client on a higher value than the server per user limit, my result was that the user was able to go OVER the server max limit.

I am also a programmer and ether this function is wrong declared or not working well, because if i allow a user to use maximum 8 MBit there is no way he can go higher, i would accept 7.8 to 8,2MBit with a set limit of 8 but not 6.4 or 8.7.

Is there anything you need from me to test or is the Problem clear?

Thanks

[ebr 16169]

17 hours ago, Painkiller8818 said:

but it also should be as near as possible to the 8MBit limit and not 5 or 6 MBit

That is going to depend on a lot of other factors and our auto algorithms have to be conservative because the most important thing is that the item plays smoothly.  If we get too aggressive with the auto calculation it is possible that playback will stutter or not work at all because we pushed the limit of what our bandwidth test told us.

If you are certain that both your clients and server can always handle a certain bitrate, then that is why we also allow you to set the max value in the apps to a specific value.  Of course, if you do this it is possible there will be problems depending on network conditions and server load at any particular point in time.

[sooty234 266]

27 minutes ago, ebr said:

our auto algorithms have to be conservative because the most important thing is that the item plays smoothly. 

This is what shouldn't exist. You're deciding for us what you think it should do. It has never worked. As I showed, it always uses 2.5 Mb/s, in my case. If you don't want to remove it, then we need an option to completely disable it. If I set 6 Mb/s, I want 6 Mb/s. 

[Carlo 4560]

But you do have the ability to set your bitrate and resolution already to whatever you want on the client.

Example for web client:

 

[rbjtech 5284]

As an example below - I have a 32 Mbps file, force the Android client to use 30 Mbps (forcing a transcode) - both the dashboard and stats for nerds locally on the client are reporting it as a 2.5 Mbps bitrate !

The transcode log - the 30 Mbps is being acknowledged as the Max bitrate in the ffmpeg command line.

C:\Emby-Server\system\ffmpeg.exe -ss 00:07:51.000 -f matroska -i file:"\\media\Films 7\2 Guns (2013)\2 Guns (2013) - BluRay 1080p.mkv" -threads 0 -map 0:0 -map 0:1 -sn -c:v:0 libx264 -pix_fmt yuv420p -maxrate 29808000 -bufsize 59616000 -preset veryfast -profile:v:0 high -level:v:0 4.1 -crf 23 -x264opts:0 subme=0:me_range=4:rc_lookahead=10:me=dia:no_chroma_me:8x8dct=0:partitions=none -g:v:0 72 -keyint_min:v:0 72 -sc_threshold:v:0 0 -copyts -vsync -1 -codec:a:0 copy -metadata:s:a:0 language=eng -disposition:a:0 default -f segment -max_delay 5000000 -avoid_negative_ts disabled -map_metadata -1 -map_chapters -1 -start_at_zero -segment_time 3 -individual_header_trailer 0 -segment_format mpegts -segment_write_temp 1 -segment_list_type m3u8 -segment_start_number 157 -segment_list "C:\Emby-Server\programdata\transcoding-temp\31ADBA.m3u8" -y "C:\Emby-Server\programdata\transcoding-temp\31ADBA_%d.ts"

Streaming details are showing it as ~ 2.5 Mbps -

17:11:32.468 frame=12181 fps= 29 q=28.0 size= 147868kB time=00:17:55.17 bitrate=2388.5kbits/s throttle=100 speed=1.21x

If I look at the actual data being sent to my device via network tools - 2.5 Mbit ....  

If I force the bitrate to direct play - then my network tools report it as 32 Mbit/sec.

So I agree with the OP - something is wrong with the transcoding bitrate algorithms - as even with 'headroom' it no way reflects what you would expect it to be.  

-- edit

So looking into this further and copying the emby ffmpeg command line and changing the crf/preset manually it is using much higher bitrates as I expect it to - so it could simply be the selected/default crf/preset is dictating the bitrate used - thus 2.5 mbit/sec is all it can use with those presets ... selecting 30 Mbps is largely irrelevant ..

I'll report back with my findings and update this thread ..

-- edit 2

ok - so this is exactly what the issue is.

If I set my transcoding preset to 'superfast' and CRF to 18 - then it will use UP TO 30 Mbit to do what it needs to do.

18:13:56.875 frame= 324 fps= 39 q=19.0 size= 44980kB time=01:05:19.61 bitrate=29625.3kbits/s dup=1 drop=0 throttle=off speed= 1.5x

So OP - to get your clients to use UP TO the max bandwidth that you/they set - ensure you are setting a suitable CRF and Preset in the transcoding settings to force your server to ignore trying to save bits !

-- edit 3

I believe the note is technically incorrect in this instance.  The x264 preset when using CRF (which we are for transcoding) has little to do with Quality but it has everything to do with bitrate.

The more bitrate you have, then less you need to compress - which would result in a bigger file size obviously, but as we are streaming, that is not relevant.

It could say something like -

'Choose a faster value to reduce CPU load but at the expense of a higher streaming bitrate, or a slower value to decrease the streaming bitrate but at the expense of a higher CPU load.'

 

 

 

 

Edited August 4, 2020 by rbjtech

[sooty234 266]

It's always been this way. It has never worked, and is perpetually broken. 

Just stop beating a dead horse and give us an option to circumvent it.

 

Carlo, what you keep referring to, also doesn't work. None of it works. If you learn to read carefully, you will see I posted the results of all of the possible configurations.

 

Please give us an option to disable the available bandwidth check, and have it strictly obey the bandwidth we select.

[Painkiller88 248]

59 minutes ago, rbjtech said:

As an example below - I have a 32 Mbps file, force the Android client to use 30 Mbps (forcing a transcode) - both the dashboard and stats for nerds locally on the client are reporting it as a 2.5 Mbps bitrate !

The transcode log - the 30 Mbps is being acknowledged as the Max bitrate in the ffmpeg command line.

C:\Emby-Server\system\ffmpeg.exe -ss 00:07:51.000 -f matroska -i file:"\\media\Films 7\2 Guns (2013)\2 Guns (2013) - BluRay 1080p.mkv" -threads 0 -map 0:0 -map 0:1 -sn -c:v:0 libx264 -pix_fmt yuv420p -maxrate 29808000 -bufsize 59616000 -preset veryfast -profile:v:0 high -level:v:0 4.1 -crf 23 -x264opts:0 subme=0:me_range=4:rc_lookahead=10:me=dia:no_chroma_me:8x8dct=0:partitions=none -g:v:0 72 -keyint_min:v:0 72 -sc_threshold:v:0 0 -copyts -vsync -1 -codec:a:0 copy -metadata:s:a:0 language=eng -disposition:a:0 default -f segment -max_delay 5000000 -avoid_negative_ts disabled -map_metadata -1 -map_chapters -1 -start_at_zero -segment_time 3 -individual_header_trailer 0 -segment_format mpegts -segment_write_temp 1 -segment_list_type m3u8 -segment_start_number 157 -segment_list "C:\Emby-Server\programdata\transcoding-temp\31ADBA.m3u8" -y "C:\Emby-Server\programdata\transcoding-temp\31ADBA_%d.ts"

Streaming details are showing it as ~ 2.5 Mbps -

17:11:32.468 frame=12181 fps= 29 q=28.0 size= 147868kB time=00:17:55.17 bitrate=2388.5kbits/s throttle=100 speed=1.21x

If I look at the actual data being sent to my device via network tools - 2.5 Mbit ....  

If I force the bitrate to direct play - then my network tools report it as 32 Mbit/sec.

So I agree with the OP - something is wrong with the transcoding bitrate algorithms - as even with 'headroom' it no way reflects what you would expect it to be.  

-- edit

So looking into this further and copying the emby ffmpeg command line and changing the crf/preset manually it is using much higher bitrates as I expect it to - so it could simply be the selected/default crf/preset is dictating the bitrate used - thus 2.5 mbit/sec is all it can use with those presets ... selecting 30 Mbps is largely irrelevant ..

I'll report back with my findings and update this thread ..

-- edit 2

ok - so this is exactly what the issue is.

If I set my transcoding preset to 'superfast' and CRF to 18 - then it will use UP TO 30 Mbit to do what it needs to do.

18:13:56.875 frame= 324 fps= 39 q=19.0 size= 44980kB time=01:05:19.61 bitrate=29625.3kbits/s dup=1 drop=0 throttle=off speed= 1.5x

So OP - to get your clients to use UP TO the max bandwidth that you/they set - ensure you are setting a suitable CRF and Preset in the transcoding settings to force your server to ignore trying to save bits !

 

 

 

 

 

OK thanks i will try this, my CFR Settings are default  (23).

But nothing at all, this shouldn't happen, for my understanding if i limit a server at 8MBit there should be NO WAY a Client can go higher, and also a client with much  more downstream than my limit should reach my limit very close and not 2 or 3 Mbit less.

Maybe the team will investigate in this, because it is simply not doing what it should.

Thanks

[rbjtech 5284]

The preset will have more impact on the bandwidth usage - lowering the CRF will just increase the CPU/Quality but use the same bandwidth to do it.

[Painkiller88 248]

I did another test now with my mobile phone (NOT IN WIFI)

Server still on 8MBit limit and default CRF
I started the movie Aquaman 4K and set my phone to 60MBit

And it is reporting it as 60MBit so the Server limit isn't working at all.
The reason it is transcoding is that the movie is still over the Bitrate limit

The movie was stucking from the first second. The Server limit per user shouldn't be overruled from the clients setting otherwhise this function is worthless. I want to limit my users so 3-4 Users can watch at the same time.

 

[rbjtech 5284]

so you have set a global limit of 8 Mbps under server > Network > Internet Streaming bitrate limit correct ?

If yes, check the 'Activity' log when you login with your phone - is it actually a public IP address via your 3G/4G provider ?

For some reason, mine is showing as the LAN gateway address, even if it's remote and thus Emby thinks it's LOCAL and therefore does not enforce the streaming limit ... 

 

 

 

 

[Painkiller88 248]

2 minutes ago, rbjtech said:

so you have set a global limit of 8 Mbps under server > Network > Internet Streaming bitrate limit correct ?

If yes, check the 'Activity' log when you login with your phone - is it actually a public IP address via your 3G/4G provider ?

For some reason, mine is showing as the LAN gateway address, even if it's remote and thus Emby thinks it's LOCAL and therefore does not enforce the streaming limit ... 

 

 

 

 

no not a global server limit, i set this as a max limit for the user directly:

[rbjtech 5284]

ok - either should work - but is the IP being used classed as 'local' or 'remote' - that is the key thing here.  What is Emby reporting your remote IP as ?

[Painkiller88 248]

Emby isn't recorgnizing my Android Phone at all, i see it as active device but i don't see me connecting or disconnecting with the phone, so i can't tell you what IP he thinks, but as i am not in the wifi the phone can just have a remote IP.

But the Android Emby Client for the phone isn't recorgnized by the server

[rbjtech 5284]

If Emby is not recognising the device as 'remote' then it will not enforce the restrictions - it clearly says this in the notes.

[Painkiller88 248]

Thats not the actual Problem, and my Phone was just a Test device for now.

But i did another test, set my user to 8MBit limit on the Server, set the Client Limit to 60MBit (Firefox Browser)

And here we go, much more than it should have

This time it is recorgnized and i have an external ip because it is a browser and it has twice the bandwidth of my limit (8MBit)

So it is simply not limiting right.

[sooty234 266]

This is what I've been saying. It's systematically dysfunctional. If you're lucky enough to find one circumstance in which it kinda works, it's completely broken in another. I've tested this since the first time it was implemented. It has never worked. Looking at a log might only show one set of variables in which it doesn't work, but there so many more in which they also don't. It's just a lot of 'kentucky windage'. 

[rbjtech 5284]

Now I have realised why I never got near the Max bitrate (see above) - I cannot reproduce the issues the OP is facing.  If I set the global bandwidth limit - that's what it restricts it to, if I set the local user limit to overwrite this - either under or over the global, then it enforces this as well.

'Auto' on mobile/4G clients has never personally worked for me (it always sets it far too low) - so I set it to max bitrate and let the server do the restricting.

One thing I have noticed is you are using hardware acceleration - try turning this off and see if you get a different result ?

 

[Happy2Play 9780]

All examples still need the context of logs, server and ffmpeg.

[kikinjo 281]

1 hour ago, Happy2Play said:

All examples still need the context of logs, server and ffmpeg.

Thing is really obvious here and its old few years already.

[Painkiller88 248]

3 hours ago, Happy2Play said:

All examples still need the context of logs, server and ffmpeg.

My logs are in the beginning of this thread already and you already responded to it.
So i think everything is here

Edited August 5, 2020 by Painkiller8818