Jump to content

Unbounded SMB Connection Accumulation Causing NAS System Crash


Recommended Posts

ingmar_ehrig
Posted

Summary

Emby Server running on NVIDIA Shield TV Pro (2019) accumulates SMB/CIFS connections to a network-attached SMB share over time, without releasing them. Once the connection count reaches several hundred, the NAS (WD My Cloud EX2 Ultra) becomes fully unresponsive due to load, requiring a hard power cycle to recover. This has caused two separate NAS-side crashes within roughly one hour of normal media playback usage and several more since I installed emby Server on my ShieldTV.

Note: I have another emby server running on my MyCloudEX2 for years running without any single problem. I just wanted to switch to a newer .NET System instead of mono.

Environment

  • Emby Server version: 4.9.5.0
  • Device: NVIDIA Shield TV Pro (2019), Tegra X1+
  • Android-Version: 11
  • Build-ID: RQ1A.210105.003.7825230_4387.0822
  • NAS / SMB Server: WD MyCloud EX2 Ultra, MyCloud OS 5, Samba (smbd)
  • Network: Gigabit Ethernet, local network only (no remote access)
  • Library storage: SMB share (smb://<NAS-IP>/...), accessed via mapped/attached network storage in Emby

Steps to Reproduce

  • Set up Emby Server on Shield TV with library folders pointing to an SMB share on a separate NAS.
  • Use the server normally over an extended period (library scans, browsing, video playback) — no unusual or heavy load, single concurrent stream.
  • Periodically check the number of active SMB connections from the Shield's IP address on the NAS side:
            netstat -an | grep :445 | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -rn
 
  • Observe that the connection count from the Shield's IP grows continuously over hours/days, rather than staying stable at a low number (1-5).

Observed Behavior

  • At the time of the crash, 393 out of 397 total SMB connections on the NAS originated from the single Shield device running Emby Server.
  • This coincided with the NAS Linux kernel load average spiking to over 400 (on a 2-core system), with dozens of smbd worker processes stuck in uninterruptible sleep (D state), waiting on I/O.
  • The NAS's web dashboard, and eventually most background services (Apache/httpd, nasAdmin, monit), became unresponsive. SSH access remained marginally functional but was severely lagged.
  • Restarting Emby Server on the Shield (via its own web interface restart function, without touching the NAS) immediately dropped the SMB connection count from 393 to 0, and the NAS load began recovering.
  • This is reproducible: the connection count was independently confirmed via netstat before and after the Emby restart, isolating Emby Server (and not any other NAS-side process) as the source of the accumulated connections.

Expected Behavior

Emby Server's SMB client should reuse or properly close SMB connections/sessions after file operations (metadata reads, thumbnail loads, playback) complete, keeping the connection count to the NAS low and stable regardless of how long the server has been running or how many files have been scanned/played.

Additional Notes

  • No equivalent SMB connection accumulation was observed when running Emby Server directly on the NAS itself (WD My Cloud "Emby Server" app, Mono-based build) accessing local storage — this issue appears specific to the Android TV client/server's SMB implementation when accessing a remote SMB share.
  • There is no user-facing setting within Emby Server for Android TV to control SMB connection pooling, reuse, timeout, or maximum concurrent connections.
  • This is a serious issue beyond cosmetic/performance concerns: it results in total NAS unavailability requiring a hard power cycle, affecting all other devices/services relying on that NAS (other Emby libraries, other network clients), not just Emby itself.

Suggested root cause area

Possibly related to how the Android TV media probe / file streaming layer (MediaProbeManager, ffprobe file access via local loopback Items/File proxying seen in server logs) or the underlying SMB/CIFS client library opens a new connection/session per file access request instead of reusing an existing pooled connection to the same share/host.

 

Best regards

Ingmar

embyserver-63921908969.txt embyserver-63921916800.txt embyserver-63921922586.txt

Posted

Hi, the smb library we’re embedding is probably opening a new connection per file access, however they shouldn’t be left open.

  • 3 weeks later...
ingmar_ehrig
Posted

...so? Will there be a fix in the near future?

Posted

Actually I believe everything should be getting cleaned up. When you've observed this, what were the active smb connections?

ingmar_ehrig
Posted

Huh? What info can I provide which I aready didn't? Do the single smb connections have individuel names? I was scanning a library, updated the mety data or watched a movie. after a while the whole NAS freezed (so no infos from this device) and I had to restart the NAS to work again. The info I provided with my first post is all I have and was done via my windows pc.

Since this behaviour prevents the emby server on the shield to be of any use right now, I switched back to the WD-server again until the problem is solved. At least this is what I hoped for...

Posted

What is the output from netstat?

ingmar_ehrig
Posted

Hi Luke,

Thanks for looking into this. I no longer have the Shield's Emby Server running (I've reverted to my WD My Cloud's own Emby Server instance for stability in the meantime), but I have full netstat output and process data captured live during the incident last night, before I restarted the Shield's Emby Server. Here's everything in detail:

Total SMB connections on the NAS at the time of the crash (port 445):

netstat -an 2>/dev/null | grep :445 | wc -l

Result: 397

Breakdown by source IP:

netstat -an 2>/dev/null | grep :445 | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -rn

Result:

    393 192.168.178.27
      1 0.0.0.0
      1 (empty/malformed line)

192.168.178.27 is the Shield TV's static local IP — the sole client running Emby Server. So 393 of 397 total SMB connections on the NAS originated from this single device.

System state at the time (top -n 1), showing the load and blocked smbd workers:

Mem: 1017344K used, 19328K free, 32K shrd, 45728K buff, 134240K cached
CPU:  0.0% usr 76.9% sys  0.0% nic  0.0% idle 23.0% io  0.0% irq  0.0% sirq
Load average: 443.45 342.40 170.92 7/641 21724

Dozens of smbd worker processes were stuck in uninterruptible sleep (D state), e.g.:

15012  5792 nobody   D<   78016    7.5   1  0.0 smbd -D
28608  5792 root     D<   77984    7.5   1  0.0 smbd -D
27096  5792 root     D<   77984    7.5   1  0.0 smbd -D
16220  5792 root     D<   77984    7.5   1  0.0 smbd -D
7304   5792 root     D<   77984    7.5   1  0.0 smbd -D
29096  5792 root     D<   77984    7.5   1  0.0 smbd -D
7374   5792 root     D<   77984    7.5   1  0.0 smbd -D
27912  5792 root     D<   77984    7.5   1  0.0 smbd -D
7335   5792 root     D<   77984    7.5   1  0.0 smbd -D
7241   5792 root     D<   77984    7.5   1  0.0 smbd -D
26611  5792 root     D<   77984    7.5   1  0.0 smbd -D
27812  5792 root     D<   77984    7.5   1  0.0 smbd -D
8485   5792 root     D<   77984    7.5   1  0.0 smbd -D
25262  5792 root     D<   77984    7.5   1  0.0 smbd -D
27848  5792 root     D<   77984    7.5   1  0.0 smbd -D
16258  5792 root     D<   77984    7.5   1  0.0 smbd -D
26505  5792 root     D<   77984    7.5   1  0.0 smbd -D
29201  5792 root     D<   77984    7.5   1  0.0 smbd -D

(This is only a partial list — the full process table had dozens more smbd instances in the same state, all children of the same smbd parent process, PID 5792.)

Confirmation that Emby was the cause, not something else on the NAS: I restarted Emby Server on the Shield via its own web dashboard restart function — nothing was touched on the NAS side. Immediately afterward:

netstat -an 2>/dev/null | grep :445 | wc -l

Result: 0

All 393 connections from the Shield dropped to zero the moment Emby Server restarted, and the NAS's load began recovering shortly after. I did not restart or touch the NAS itself between the "397" measurement and the "0" measurement — only the Emby Server process on the Shield was restarted.

Let me know if you need the RAID status (/proc/mdstat, both arrays were healthy/[UU], no rebuild in progress) — I still have those saved from that session.

Thanks, Ingmar

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...