Limited bandwidth when movie is on network drive

[psxlover 31]

Hallo,

I have an old and underpowered netbook that I use as a server, and an external drive that is attached to the internet router. The setup was working fine, until I changed the router about a month ago. Now high bitrate movies freeze every few seconds.

In the beginning I thought that perhaps it was the router's fault and it isn't good with external drives, but copying files between the hard drive and the netbook seems max out the bandwidth of the netbook (100Mbps ethernet adapter), so there shouldn't be an issue there.

When I watch a 26mbps movie from emby server to another device the download & upload speed seems to max out at around 14-16Mbps each. This is regardless if using an application to stream or if streaming through DLNA.

In comparison when streaming the same file from the netbooks HDD, upload seems to vary between 20-40 Mbps and there is no freezing.

I thought that perhaps there was an issue when the netbook needs to download and upload at the same time, but when I tried to use the netbook to copy a file from the HDD that is attached to the router to my desktop, upload and download speeds where around 50Mbps. Also when streaming a higher bitrate movie (58,634 kbps) that is on the HDD that is attached to the router the network usage seems to also be limited to that 14-16Mbps value for download and upload.

Any idea what could be wrong on my setup, that could be causing Emby to have such a limitation on the data it pulls from the network drive and sends to the applications?

[sonusfaber 17]

Ciao

your movies are stored on the HDD attached to the router, correct?
In which way does Emby server see the movies on the HDD ?

 

 

[psxlover 31]

I've tried both through mapping the network drive to "D:\" and by using directly the UNC path "\192.168.1.1\fritz.nas\My_Passport\USERFOLDER\Videos\Movies\", both had the same issue.

[rbjtech 5284]

Ensure your laptop has connected to the switch at 100Mbit FULL DUPLEX. 

Due to you having to READ (Receive/Rx) the source from the network, Emby then convert to HTTP, and then SEND (Transmit/Tx) to the clients - the 100Mbit interface needs to run in Full Duplex mode or it can only do one of these at a time.  Note the two sets of identical lines in the first graph - this is the adapter bottlenecking.

When you only do one (Rx or Tx), the Duplex setting does not matter - Half Duplex will work perfectly well.

Note - if using 1Gig, you do not get a choice, it will always negotiate at full duplex, with 10/100Mbit you may get Half Duplex.

Edited February 9, 2022 by rbjtech

[psxlover 31]

1 hour ago, rbjtech said:

Ensure your laptop has connected to the switch at 100Mbit FULL DUPLEX.

I've changed the value on the adapter from auto negotiate to full duplex, and as I wrote on the first message I copied files from the hard drive to my desktop using the netbook, and the speeds were around 50Mbps for down and for up, which is way more than the 15-16Mbps when streaming from that hard drive through Emby.

[rbjtech 5284]

35 minutes ago, psxlover said:

I've changed the value on the adapter from auto negotiate to full duplex, and as I wrote on the first message I copied files from the hard drive to my desktop using the netbook, and the speeds were around 50Mbps for down and for up, which is way more than the 15-16Mbps when streaming from that hard drive through Emby.

When you are just copying - you and not Tx and Rx at the same time - emby does need to do this and I suspect that's what the problem is - OR there are some other network related settings such as TCP offload not set correctly - as your CPU is very high for such a simple single stream playback.

Maybe provide some screenshots of the Laptop Network Stack config/settings.

[sonusfaber 17]

On 2/6/2022 at 8:40 PM, psxlover said:

When I watch a 26mbps movie from emby server to another device

what about watching the same movie playing it on a web browser running on the server?

[psxlover 31]

On 2/9/2022 at 3:28 PM, rbjtech said:

When you are just copying - you and not Tx and Rx at the same time - emby does need to do this and I suspect that's what the problem is - OR there are some other network related settings such as TCP offload not set correctly - as your CPU is very high for such a simple single stream playback.

As I said the netbook is very low spec and very old, so CPU usage is high if you do anything on it (e.g just having taskmanager open with normal update speed for the graphs has 10-20% CPU usage). It's got two cores with base clock being 1 Ghz (and turbo reaching up to 1.3Ghz). When streaming most of the CPU is used by "System Interrupts" followed by Emby:

How is copying from one network drive, to another network drive on a different machine not transmitting and receiving at the same time? Do Windows copy a parts to the PC and then transmit? As I said both transmit and receive are around 50Mbps at the same time, I'm not copying the file to the netbook first and then to the desktop:

I also tried using a USB3 Hub/Ethernet adapter that I have, and copying from the HDD on the router to the Desktop using the netbook was using 250-300Mbps, while streaming using Emby a movie that is on the HDD attached to the router was still capped (first time I tested I think it was around 12Mbps up&down, now that I tested again it was around 20Mbps up&down)

On 2/9/2022 at 3:28 PM, rbjtech said:

Maybe provide some screenshots of the Laptop Network Stack config/settings.

Are you talking about the network adapter options?

ARP Offload: Enabled
Energy Efficient Ethernet: Enabled (tried with disabled as well)
Flow Control: Rx & Tx Enabled
Interrupt Moderation: Enabled
IPv4 Checksum Offload: Rx & Tx Enabled
Large Send Offload v2 (IPv4): Enabled
Large Send Offload v2 (IPv6): Enabled
Maximum Number of RSS Queueus: 4 Queues
Priority & VLAN: Priority & VLAN Enabled
Receive Buffers: 512
Receive Side Scaling: Enabled
Shutdown Wake-On-Lan: Enabled
Speed & Duplex: 100 Mbps Full Duplex
TCP Checksum Offload (IPv4): Rx & Tx Enabled
TCP Checksum Offload (IPv6): Rx & Tx Enabled
Transmit Buffers: 128
UDP Checksum Offload (IPv4): Rx & Tx Enabled
UDP Checksum Offload (IPv6): Rx & Tx Enabled
Wake on Magic Packet: Enabled
Wake on pattern match: Enabled
WOL & Shutdown Link Speed: 10 Mbps First

Another thing I also tried was enabling SMBv1 on the router, and disabling SMBv2 on the netbook, since the previous router supported only SMBv1, but I didn't see any difference.

On 2/10/2022 at 10:00 AM, sonusfaber said:

what about watching the same movie playing it on a web browser running on the server?

That is not an option. The netbook strangled with 720p/1080p playback, it cannot handle 4k HEVC playback let alone through a browser.

[visproduction 315]

Just a guess.  The USB hub is probably not full duplex so no matter what you set your computer, the slow down might be either the hub or some other switch / Ethernet card.  I believe the test request was to watch a video file directly from a browser and not go through Emby.  Also it looks like you are testing access to the server through the router using the remote IP address and not accessing Emby with the local address.  If it was local, you would not need to go through the Ethernet.  Since you are using the remote address to test, your media is transferred out through the router to the nearest host hub which then redirects it back to your notebook.  The outbound speed that your host provides becomes the slowest link in the chain.  5 mbps or 10 mbps is typical.  You have check your host contract or run a speed test to find the outbound speed.  With this limit, any other traffic coming from you house, including any phones, can take up part of that outbound traffic.

1st) test locally not through the Internet with a browser and see if the media playback.
2nd) test the media locally on Emby

If 1 or 2 fail, then the media is suspect.  Try other media or run a test media downloaded from a test site online.
If 1 and 2 playback fine, then it can be anything that the Ethernet goes through between your notebook, any adapter, switch, router or host outbound speed issues.

Hope that helps.

[psxlover 31]

1 hour ago, visproduction said:

Just a guess.  The USB hub is probably not full duplex so no matter what you set your computer, the slow down might be either the hub or some other switch / Ethernet card.  I believe the test request was to watch a video file directly from a browser and not go through Emby.  Also it looks like you are testing access to the server through the router using the remote IP address and not accessing Emby with the local address.  If it was local, you would not need to go through the Ethernet.  Since you are using the remote address to test, your media is transferred out through the router to the nearest host hub which then redirects it back to your notebook.  The outbound speed that your host provides becomes the slowest link in the chain.  5 mbps or 10 mbps is typical.  You have check your host contract or run a speed test to find the outbound speed.  With this limit, any other traffic coming from you house, including any phones, can take up part of that outbound traffic.

1st) test locally not through the Internet with a browser and see if the media playback.
2nd) test the media locally on Emby

If 1 or 2 fail, then the media is suspect.  Try other media or run a test media downloaded from a test site online.
If 1 and 2 playback fine, then it can be anything that the Ethernet goes through between your notebook, any adapter, switch, router or host outbound speed issues.

Hope that helps.

I feel like you didn't read what I wrote.

The USB hub was used as a test to make sure the issue was not the netbooks ethernet adapter was not the issue since it is 100Mbs while the USB hub has a 1Gbps adapter. Every image and text apart from the one paragraph that mentioned the USB hub was done using the onboard ethernet adapter.

All streaming is done using 192.168.1.100 address (or in the the case of the USB hub 192.168.1.30). My internet connection has 1Mbps upload, there is no way I could stream something through the internet. Also the same file is streaming just fine when located on the Netbook's hdd, but has issues when streaming from the HDD attached to the router so it is not a file issue (apart from the fact that smaller files don't run into the bottleneck so are not affected).

The netbook is not capable of 4k HEVC playback, not sure how to stress out that I just can't play videos on the netbook or  even run a browser on it without waiting 5-10 minutes for a single page to be displayed. It simply can't be used for anything more than downloading and streaming.

Edited February 12, 2022 by psxlover

[visproduction 315]

Psx,  Aha,  Sorry,.If you find a solution, I would look forward to reading about what it was.  Good luck.

[psxlover 31]

Just tried with Plex, and playback has no issues.

@Luke any idea what could have gone wrong with my Emby server configuration after I changed router?

[rbjtech 5284]

Can I just confirm the physical setup here.

HDD > USB3 > Router > SMB2 > Laptop (Emby Server) > HTTP > Emby Client (PC)

IF the connection to the router is only USB2 - then you will have a ~30-40Mbit/sec bottleneck here - so I'm assuming it's USB3.

The above laptop network config looks perfectly fine (full duplex) ..

At what point (bitrate) does the streaming become a problem ?  ie is 20Mbit/sec ok, but beyond 30 it's a problem for example ?

IF you copy a file directly from the HDD to your PC (ie HDD > USB3 > Router > SMB2 > PC) - this copies at 200-300Mbit/sec correct ? (which is technically 'slow' - it should be nearing 800-900Mbit/sec, nearing the 1Gb Ethernet interface speed - but I'll put that down to the Router - but even 200-300Mbit should be more more than enough to stream the intended file).

 

[psxlover 31]

7 hours ago, rbjtech said:

HDD > USB3 > Router > SMB2 > Laptop (Emby Server) > HTTP > Emby Client (PC)

HDD > USB3 > Router > SMB2/3 (tried also SMB1 since that is what the previous router was using) > Netbook (Emby Server) using 100Mbit ethernet but also tried a 1Gigabit > Router > Emby Application (or browser) on LG 2021 model connected over ethernet and Samsung 2019 model connected with wifi

Quote

IF the connection to the router is only USB2 - then you will have a ~30-40Mbit/sec bottleneck here - so I'm assuming it's USB3.

That's not true. USB2 has 480Mbit/sec limit which translates to 30-40Mbytes/sec. That is more than enough, and it is more than the ethernet on the netbook. As a matter of fact the previous router was using USB2, and there was no issue. You seem to be confusing Megabits with Megabytes.

Quote

At what point (bitrate) does the streaming become a problem ?  ie is 20Mbit/sec ok, but beyond 30 it's a problem for example ?

I'm not sure, the files I have, either have too big or too small bitrate. But judging just by what I see in taskmanager, 10-12Mbit/s would already be an issue

Quote

IF you copy a file directly from the HDD to your PC (ie HDD > USB3 > Router > SMB2 > PC) - this copies at 200-300Mbit/sec correct ? (which is technically 'slow' - it should be nearing 800-900Mbit/sec, nearing the 1Gb Ethernet interface speed - but I'll put that down to the Router - but even 200-300Mbit should be more more than enough to stream the intended file).

As I've said I'm mainly using the onboard ethernet which is 100Mbit, and that one is maxed out when copying from the hard drive that's connected to the router into another PC through the netbook. When copying directly to the PC from the router instead of through the netbook, it doesn't max out the Gigabit ethernet, but that doesn't matter because the 100Mbit on the netbook is more than enough for the streaming. As a matter of fact I'm not aware of any TV that has Gigabit ethernet.

The connection on the laptop is not the issue, as both copying files from the HDD attached to the router through the netbook uses a lot more bandwidth than Emby, so that means the bandwidth is there, and also playing the same files using Plex has no issue either.

Edited March 13, 2022 by psxlover

[psxlover 31]

As a comparison, same file same server same TV. Left part is Emby limited to 14-16 Mbit/s, and right side is Plex where it even slows down after a while from 40-45 Mbit/s to 32-34 Mbit/s perhaps because it reaches the TV's cache limit (playing without issues though).

Edited March 13, 2022 by psxlover
Forgot image

[Luke 42079]

On 3/12/2022 at 3:26 PM, psxlover said:

Just tried with Plex, and playback has no issues.

@Luke any idea what could have gone wrong with my Emby server configuration after I changed router?

I'm not sure at this point, sorry. Maybe we should look at an example: 

Thanks.

 

[rbjtech 5284]

16 hours ago, psxlover said:

That's not true. USB2 has 480Mbit/sec limit which translates to 30-40Mbytes/sec. That is more than enough, and it is more than the ethernet on the netbook. As a matter of fact the previous router was using USB2, and there was no issue. You seem to be confusing Megabits with Megabytes.

You are correct of course, it was a typo - should have been Mbytes. 

7 hours ago, Luke said:

I'm not sure at this point, sorry. Maybe we should look at an example: 

Thanks.

 

@Luke - there is a lot of detail in the thread with specific examples.

In summary, the LG Emby Client does not want to stream anything past ~20Mbit/sec from the Emby Server.

This has been prooven to not be bandwidth limited - as a direct SMB file copy is fast and Plex has no issues either.

This is Direct Play (or should be)  only - so what can the OP provide to further investigate this ? 

Edited March 14, 2022 by rbjtech

[psxlover 31]

Sorry @Luke I though I had uploaded server logs from when a movie is streamed from the HDD on the router and from when the same movie was streamed from the onboard HDD, but apparently I haven't. I will update the ticket when I find the time to create the logs.

[Luke 42079]

5 hours ago, psxlover said:

Sorry @Luke I though I had uploaded server logs from when a movie is streamed from the HDD on the router and from when the same movie was streamed from the onboard HDD, but apparently I haven't. I will update the ticket when I find the time to create the logs.

Great, thanks.

[psxlover 31]

@Luke here are the logs. It could be that Emby task + "System Interrupts" task are using 100% of one of the two cores, thus the limit is due to hardware resources, but I don't understand what changed with the new router. As I wrote I've tried using SMB1, like the previous router used, in case the SMB2/3 protocols added extra processing overhead, but I didn't see any improvement.

logs.zip

[AxeMan 25]

I haven't read everything here - BUT - can you check that the user that's starting Emby has access to the path that the videos are on? 

My setup had a very similar issue (with content skip/freezing), not so much the bandwidth issue - and it was related to permissions. Emby was dutifully being a broker/proxy between the network location and the client. 

Just a thought ... 

[psxlover 31]

@AxeMan can you explain what you mean by "the user has access to the path"? I don't see how the Emby user would have a direct access to the files, and the TV doesn't have direct access since it can't login to SMB network folders as far as I know. The TV does have access to the files through DLNA on the router, but as far as I know that's not used in any way when Emby is used, and Emby's DLNA has the same issues as the applications.

Edit: Reading you message again I perhaps you mean the user that is executing the Emby process. How would Emby have access to the to the files if the user it runs on can't access them?

Edited March 22, 2022 by psxlover

[rbjtech 5284]

Agreed - the only 'permissions' Emby server needs is to be able to READ the file via it's local file system - could be a local file path, could be a mapped SMB or NFS networked drive - but as long as it can READ the content, then it can convert it to HTTP ready for streaming.   The 'streaming' part does obviously not have 'permissions' as such and will be using HTTP as the transmission protocol.

Only for DFA (Direct File Access) do permissions from the media share to client become necessary - again, just READ is required.   This is in fact one of the benefits of using DFA, as you can use READ ONLY service accounts to stream the media (as I do on my Shield Client).   

[AxeMan 25]

Quote

 Emby Application (or browser) on LG 2021 model connected over ethernet and Samsung 2019 model connected with wifi

This is the part I missed - I thought your client was another Windows machine. 

The scenario I was describing - was using the Emby Theater client on Windows. The Windows logon user did not have access to the network path, so essentially the file was going from the nas to emby server and from emby server back to the client. And it struggled on 4K content, but nothing else. Even though it was a full gigabit adapter. 

 

 

[psxlover 31]

@Axeman more or less that is also my issue, with the exception that the client is on a different machine so I can't avoid traffic going through Emby server. The question is what is different with the new router that could be causing Emby to struggle with content higher than 10-12kbps

Edited March 23, 2022 by psxlover