Buffering/Stuttering when playing high bit-rate HVEC on FireTV 4K

[austinchief 2]

After installing Ruckus AP's throughout my house, and ensuring I have north of 250Mbps speed via WiFi connectivity,  I've starting adding higher bit-rate 4K movies to my library.    It seems when I play any moving with, say north of 50Mbps bit-rate (maybe a tad lower), I get what looks like buffering or stuttering.   The movie simply pauses for 1-3 seconds and then resumes.     This happens 1-3x per minute or so.    

According to EMBY server, the move is direct playing, no transcoding, however, my CPU usage is 50-60%.    The emby server is running on Centos 8, using an older AMD quad core processor with an NVidia graphics card (again older, probably 6 years and CPU/GPU were probably 2-4 years old at the time).   My server is connected to my network via 1Gbps hardline.

With direct streaming, I'm not understanding the pausing/stuttering or high CPU usage.   And the process is EmbyServer (not FFMPEG)

The WiFi speed is showing over 300Mbps, and the current moving playing is 75Mbps (log attached) ... so, I don't see a bandwidth issue.    

This happens on any of my FireTV 4K sticks (I have 7 throughout my house).

Thoughts?

embyserver.txt

[ebr 14918]

Hi.  What type of audio?  As a test, if you set the app to "mix down to stereo" does it make a difference?

[rbjtech 4276]

Are the AP's hardwired or are they meshed ?

Remember cpu is used for the disk I/O and network stack - so on high bandwidth files, high cpu usage is expected on older cpu's - even without transcoding.

It may be worth checking the network stack (NIC) has hardware offloading enabled where possible.

[austinchief 2]

3 hours ago, ebr said:

Hi.  What type of audio?  As a test, if you set the app to "mix down to stereo" does it make a difference?

TrueHD 7.1, DTS 7.1, AC3 5.1 ... tried all 3 and pause/stuttering persists!

Set the global setting to downmix to stereo ... pause/stuttering persists!

[ebr 14918]

5 minutes ago, austinchief said:

TrueHD 7.1, DTS 7.1, AC3 5.1 ... tried all 3 and pause/stuttering persists!

Set the global setting to downmix to stereo ... pause/stuttering persists!

Okay, thanks, then it isn't related to that. Must be something bogging down somewhere. 

[austinchief 2]

2 hours ago, rbjtech said:

Are the AP's hardwired or are they meshed ?

Remember cpu is used for the disk I/O and network stack - so on high bandwidth files, high cpu usage is expected on older cpu's - even without transcoding.

It may be worth checking the network stack (NIC) has hardware offloading enabled where possible.

100% hardwired AP's.   I have a Ruckus Unleashed suite of APs deployed (R710 (4), R610 and T310d for my backyard).    All Fire TV 4K have speed tests north of 250Mbps, with some going well into 300Mbps.   I have no reason to believe this issue is WiFi bandwidth related.

My CPU is the i5-8400, which is an older CPU with UHD630.    I do have a GeForce GTX 1060 video card as well.   Not sure if the video card is helping with transcoding or not.

I'll look into the NIC setting and see.

[rbjtech 4276]

15 minutes ago, austinchief said:

100% hardwired AP's.   I have a Ruckus Unleashed suite of APs deployed (R710 (4), R610 and T310d for my backyard).    All Fire TV 4K have speed tests north of 250Mbps, with some going well into 300Mbps.   I have no reason to believe this issue is WiFi bandwidth related.

My CPU is the i5-8400, which is an older CPU with UHD630.    I do have a GeForce GTX 1060 video card as well.   Not sure if the video card is helping with transcoding or not.

I'll look into the NIC setting and see.

If it's stalling and the server is not transcoding - then it's either the server cannot serve the http quickly enough (unlikely, you'd see 100% cpu) or the bandwidth is simply not there.  Just because an internet speed test works, doesn't mean emby will.

I'm not saying this is the issue, but a TRUE 'like for like' wireless throughput test is to run something like iperf3 on the emby server (point it to a media file for 'data') and then run iperf3 on the fireTv stick - and monitor the throughput.

As I said, it may be the tcp/ip 'stack' on the emby server coupled with your hardware - and it's simply not pushing data out the NIC quickly enough.

edit

I just tested my FireTV 4K via wifi (Ubiquiti AP's) with a 90Mbit 4k Remux - zero issues.   I usually run the USB dongle/wired - but just wanted to double check the FireTV was not the issue ..

Edited March 2, 2023 by rbjtech

[austinchief 2]

1 hour ago, rbjtech said:

If it's stalling and the server is not transcoding - then it's either the server cannot serve the http quickly enough (unlikely, you'd see 100% cpu) or the bandwidth is simply not there.  Just because an internet speed test works, doesn't mean emby will.

I'm not saying this is the issue, but a TRUE 'like for like' wireless throughput test is to run something like iperf3 on the emby server (point it to a media file for 'data') and then run iperf3 on the fireTv stick - and monitor the throughput.

As I said, it may be the tcp/ip 'stack' on the emby server coupled with your hardware - and it's simply not pushing data out the NIC quickly enough.

edit

I just tested my FireTV 4K via wifi (Ubiquiti AP's) with a 90Mbit 4k Remux - zero issues.   I usually run the USB dongle/wired - but just wanted to double check the FireTV was not the issue ..

Ok ... here is the iperf results between my NAS server and my EmbyServer:

Connecting to host 192.168.1.10, port 5201
[  5] local 192.168.1.3 port 42338 connected to 192.168.1.10 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   112 MBytes   936 Mbits/sec   45    332 KBytes       
[  5]   1.00-2.00   sec   109 MBytes   916 Mbits/sec  145    218 KBytes       
[  5]   2.00-3.00   sec   108 MBytes   907 Mbits/sec    5    294 KBytes       
[  5]   3.00-4.00   sec   110 MBytes   921 Mbits/sec    1    387 KBytes       
[  5]   4.00-5.00   sec   107 MBytes   897 Mbits/sec   41    366 KBytes       
[  5]   5.00-6.00   sec   109 MBytes   914 Mbits/sec    1    383 KBytes       
[  5]   6.00-7.00   sec   110 MBytes   921 Mbits/sec   85    206 KBytes       
[  5]   7.00-8.00   sec   109 MBytes   915 Mbits/sec   42    322 KBytes       
[  5]   8.00-9.00   sec   110 MBytes   924 Mbits/sec   24    276 KBytes       
[  5]   9.00-10.00  sec   110 MBytes   921 Mbits/sec    0    389 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.07 GBytes   917 Mbits/sec  389             sender
[  5]   0.00-10.00  sec  1.07 GBytes   915 Mbits/sec                  receiver
 

Note:   This is hardline to hardline, so WiFi is not involved.    I'll need to figure out how to setup iperf on my Fire TV 4k to test the same connection between EmbyServer and Fire TV.

[rbjtech 4276]

15 minutes ago, austinchief said:

Ok ... here is the iperf results between my NAS server and my EmbyServer:

Connecting to host 192.168.1.10, port 5201
[  5] local 192.168.1.3 port 42338 connected to 192.168.1.10 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   112 MBytes   936 Mbits/sec   45    332 KBytes       
[  5]   1.00-2.00   sec   109 MBytes   916 Mbits/sec  145    218 KBytes       
[  5]   2.00-3.00   sec   108 MBytes   907 Mbits/sec    5    294 KBytes       
[  5]   3.00-4.00   sec   110 MBytes   921 Mbits/sec    1    387 KBytes       
[  5]   4.00-5.00   sec   107 MBytes   897 Mbits/sec   41    366 KBytes       
[  5]   5.00-6.00   sec   109 MBytes   914 Mbits/sec    1    383 KBytes       
[  5]   6.00-7.00   sec   110 MBytes   921 Mbits/sec   85    206 KBytes       
[  5]   7.00-8.00   sec   109 MBytes   915 Mbits/sec   42    322 KBytes       
[  5]   8.00-9.00   sec   110 MBytes   924 Mbits/sec   24    276 KBytes       
[  5]   9.00-10.00  sec   110 MBytes   921 Mbits/sec    0    389 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.07 GBytes   917 Mbits/sec  389             sender
[  5]   0.00-10.00  sec  1.07 GBytes   915 Mbits/sec                  receiver
 

Note:   This is hardline to hardline, so WiFi is not involved.    I'll need to figure out how to setup iperf on my Fire TV 4k to test the same connection between EmbyServer and Fire TV.

You might have to side load an Android App - the analiti network app used to do this, but I don't see the iperf option any more ..

 

[Luke 37090]

Hi.  Can you try sideloading our standard android app on the same device and see how that compares?

https://emby.media/emby-for-android.html

Thanks.

 

[austinchief 2]

24 minutes ago, Luke said:

Hi.  Can you try sideloading our standard android app on the same device and see how that compares?

https://emby.media/emby-for-android.html

Thanks.

 

No difference in CPU utilization: 57% +/- 3%

Pausing/Stuttering still occurs 2-3x ... pauses for 4-6s each time.

Attaching log showing before and after sideloading app

embyserver (1).txt

[ebr 14918]

4 hours ago, rbjtech said:

then it's either the server cannot serve the http quickly enough (unlikely, you'd see 100% cpu)

Not necessarily as it could be I/O bound at the server end as well...

[austinchief 2]

One other piece of information ... when I force a full library scan, any movie that is playing (720p, 1080p or 2160p) will pause multiple times while the scan is in process.   Playback is actually MUCH worse during a library scan that playback of a high bit-rate 2160p movice.

I have my library scan to occur at 4am in the morning ... but sometimes I'll add a show or movie and want it available immediately.   I've gotten to the point that I will run targeted scans (I created a folder for each letter of the alphabet for my movies ... allows me to run a scan on just one letter at a time).    Or if I add a TV series/show, I'll just force the scan on my TV Shows library.    This doesn't change the pausing of shows, just gets the scan over and done much quicker.

[rbjtech 4276]

So we are back to either a cpu and/or local IO bottleneck.

Where are the mount points for the actual media on the OS - is this local disk or somewhere else ?

 

[austinchief 2]

4 hours ago, rbjtech said:

So we are back to either a cpu and/or local IO bottleneck.

Where are the mount points for the actual media on the OS - is this local disk or somewhere else ?

 

Media is located in my Synology DS1812+ NAS server.

EmbyServer is a Linux box powered by an i5-8400.   The NAS drives are mounted for EmbyServer to manage.

All Emby clients are fire TV 4K, but I can obviously access via my phone or web browser on any number of computers.

[ebr 14918]

41 minutes ago, austinchief said:

EmbyServer is a Linux box powered by an i5-8400.   The NAS drives are mounted for EmbyServer to manage.

So the media drives are across a network connection from the Emby server?

I'm more and more suspecting an I/O bottleneck (which is compounded by the remote mounts).

[austinchief 2]

4 minutes ago, ebr said:

So the media drives are across a network connection from the Emby server?

I'm more and more suspecting an I/O bottleneck (which is compounded by the remote mounts).

That is correct … the iPerf test between NAS and EmbyServer (Linux box) shows plenty of network bandwidth … near 1Gbps.

The EmbyServer hardware is not CPU limited … but does show one CPU had higher utilization … 55-60%.    When transcoding, that CPU utilization jumps to 150-200%.

 

[rbjtech 4276]

As a test, can you copy one of the troublesome media files to the actual emby server itself (local disk) - and then create a test library and point it to that media.

 

 

[austinchief 2]

11 minutes ago, rbjtech said:

As a test, can you copy one of the troublesome media files to the actual emby server itself (local disk) - and then create a test library and point it to that media.

 

 

Good idea!   Yes!

[GrimReaper 3301]

You can also install on your Fire stick a video player that can play off the network (like VLC) and play directly from source, bypassing Emby altogether and testing your WiFi in the process, you might gain some more insight, whether playback is smooth or it's also buffering, hence eliminating (or confirming) network issues. 

[rbjtech 4276]

3 minutes ago, GrimReaper said:

You can also install on your Fire stick a video player that can play off the network (like VLC) and play directly from source, bypassing Emby altogether and testing your WiFi in the process, you might gain some more insight, whether playback is smooth or it's also buffering, hence eliminating (or confirming) network issues. 

Pretty sure it's the emby server interface bogging down trying to stream (smb?) files from the NAS and stream the client via http at the same time - I don't think the wifi or FireTV client is the issue here.

The local file test (ie not using the NAS) will tell us a lot.

Edited March 3, 2023 by rbjtech

[ebr 14918]

4 minutes ago, rbjtech said:

Pretty sure it's the emby server interface bogging down trying to stream (smb?) files from the NAS and stream the client via http at the same time - I don't think the wifi or FireTV client is the issue here.

That would be my main suspicion as well.

[austinchief 2]

2 hours ago, rbjtech said:

Pretty sure it's the emby server interface bogging down trying to stream (smb?) files from the NAS and stream the client via http at the same time - I don't think the wifi or FireTV client is the issue here.

The local file test (ie not using the NAS) will tell us a lot.

Ok ... so the experiment is clear ... streaming the high bit-rate file from SMB/NAS is problematic.    I moved one movie to the local EmbyServer drive, created a temp library, and streamed to the FireTV 4k stick with ZERO pausing/stuttering.

 

Now ... what are my remedies?      My NAS is the Synology DS1812+ which is a decade old (introduced in 2013).    I am only using 1 of the 2 network interfaces.  I could do link aggregation to bump the network bandwidth up, but I'm not convinced this is an actual network problem.

Is the NAS simply unable to keep up with the SATA reads and shipping over the NIC?    I know the newer NAS units (DS1821+) not only have more network interfaces, but their R/W I/O speeds are much much faster.    Is it simply time to upgrade to the DS1821+?

[Q-Droid 650]

Can you restart Emby server and post the hardware detection log?

And which CPU do you have in the Emby server? "an older AMD quad core" or the "i5-8400"

[Q-Droid 650]

Also keep mbit vs mbyte in mind. A single 100mbit stream is still only ~10mbytes and well within even USB thumb drive speeds and a fraction of what even slow HDDs can handle.