ShadowKindjal 19 Posted October 24, 2022 Share Posted October 24, 2022 I'm having this weird issue where the Emby interface, regardless of the app, seems to load very slowly and videos take a long time to start. In some cases some apps, like roku, time out before content starts playing. Once the content starts playing it seems to be fine until you try to seek forward or back. This seems to happen when I have about twenty streams going at once but I, unfortunately, do not have any logs to share while this was occurring at this time. My question is in regard to the transcoding-temp directory. Currently my server boots on an NVME SSD. That's also the same directory that the transcoding-temp directory is loaded too. I also have a 20 drive (14TB each) raidz1 zpool that hosts the contents for my server. I'm wondering if I'm reaching any bottlenecks with my boot drive which is leading to the slow Emby interface. What's puzzling is that the server interface (Ubuntu 18.04 GUI) is completely responsive during this time though. The server is running 128GB of RAM, Ryzen 3700X, and a RTX Quadro 4000. Would moving the transcoding-temp directory off the NVME SSD and to the zpool help solve this issue or should I buy a separate SSD (possibly two in a stripe) specifically for temporary transcodes? Or is there something else I'm completely missing? Link to comment Share on other sites More sharing options...
Luke 37232 Posted October 24, 2022 Share Posted October 24, 2022 Hi there, let's look at an example. Please attach the information requested in how to report a media playback issue. Thanks! Link to comment Share on other sites More sharing options...
ShadowKindjal 19 Posted October 24, 2022 Author Share Posted October 24, 2022 Like I previously said, I don't currently have a log of when the issue took place. Until I'm able to get 20 concurrent streams going again all I can do is ask if anyone has run into this issue and find out how they manage their transcoding-temp folder with that many streams. 1 Link to comment Share on other sites More sharing options...
ShadowKindjal 19 Posted October 26, 2022 Author Share Posted October 26, 2022 (edited) I had the problem again last night around 10pm. This time with only nine active streams. Everything was very slow. Interface was slow to populate with content and content was very slow to start playing. It seemed to be fine once the content was playing though. 1312357559_embyserver10_26_22.txt Edited October 26, 2022 by ShadowKindjal Link to comment Share on other sites More sharing options...
ShadowKindjal 19 Posted October 27, 2022 Author Share Posted October 27, 2022 I just had this issue again with only three active streams. Only task running was a library scan. The entire interface was slow to populate no matter the app. The server itself shows plenty of headroom from a performance POV. I can't seem to pinpoint the issue. embyserver (4).txt Link to comment Share on other sites More sharing options...
Luke 37232 Posted October 27, 2022 Share Posted October 27, 2022 Perhaps try stopping the scan? Removing the cover art plugin is also another way to improve the performance of the server. Link to comment Share on other sites More sharing options...
ShadowKindjal 19 Posted October 27, 2022 Author Share Posted October 27, 2022 I did stop the scan. Performance improved about ten minutes later. It seems that the server should not be bottlenecked considering the specs. Checking htop during this previous slow down showed over 40gb of available RAM and one of sixteen threads at 70% utilization. Link to comment Share on other sites More sharing options...
ShadowKindjal 19 Posted October 27, 2022 Author Share Posted October 27, 2022 I'll remove the cover art plugin just in case it's causing issues tho Link to comment Share on other sites More sharing options...
Luke 37232 Posted October 27, 2022 Share Posted October 27, 2022 Great thanks @ShadowKindjal Link to comment Share on other sites More sharing options...
ShadowKindjal 19 Posted November 17, 2022 Author Share Posted November 17, 2022 Removing the cover art plugin does not seem like it has resolved the issue so I'd like to loop back to my original question. I currently use my boot SSD for metadata storage, caching, and temp transcoding. Is it possible that I'm hitting a bandwidth limitation on the boot SSD and that setting up an NVME SSD dedicated to temporary transcoding would resolve the issue? Server Specs CPU: AMD Ryzen 7 3700X GPU: Nvidia Quadro RTX 4000 Motherboard: Gigabyte X570 Aorus Elite RAM: 4 x 32GB G.Skill Ripjaws V Series Boot Drive: 500GB WD Black SN750 NVMe SSD Zpool: 4 x 5 raidz1 w/ 10TB WD Red Drives Link to comment Share on other sites More sharing options...
rbjtech 4324 Posted November 17, 2022 Share Posted November 17, 2022 I run my trancoding on a seperate nvme for this reason. The more you can do in parallel without queues the better. Moving it to non solid state though would I thinka be a backwards step .. if you have the mobo slot, then a 2nd nvme should solve the issue imo. Link to comment Share on other sites More sharing options...
ShadowKindjal 19 Posted November 18, 2022 Author Share Posted November 18, 2022 On 11/17/2022 at 12:54 PM, rbjtech said: I run my trancoding on a seperate nvme for this reason. The more you can do in parallel without queues the better. Moving it to non solid state though would I thinka be a backwards step .. if you have the mobo slot, then a 2nd nvme should solve the issue imo. I'm considering it. I just want to confirm the bottleneck is on the boot SSD. Part of me is wondering if my zpool is the bottleneck. When Emby becomes unresponsive the Ubuntu GUI does not slow down. The computer feels snappy. If the boot drive was the issue one would think the whole system would slow down. As Emby slows down media art starts to populate slowly, navigating the interface takes longer, and playback can take up to 30 seconds to start. All of my media art is stored in the folders with the corresponding content on the zpool. Is it possible that I'm hitting a ceiling on the read speeds with my zpool? I know this question would fall outside the scope of emby but I'm at a loss. I'm also asking because it could make more sense to use the additional SSD for caching on the zpool to increase read speeds. Link to comment Share on other sites More sharing options...
rbjtech 4324 Posted November 19, 2022 Share Posted November 19, 2022 There is a separate cache for recently used metadata/art (from wherever your local emby cache is) - so saving art to the media folders should not be the reason. Of course if it's not in cache, then yes it needs to fetch it. A log of browsing when the slowdown happens would help - as it will show the http response times as part of the log. Link to comment Share on other sites More sharing options...
ShadowKindjal 19 Posted November 20, 2022 Author Share Posted November 20, 2022 Here is said log file. Users noticed streams were buffering. I attempted to browse the site but pages took a long time to load so i restarted the server. embyserver-63804508817.txt Link to comment Share on other sites More sharing options...
Luke 37232 Posted December 12, 2022 Share Posted December 12, 2022 Hi, are you still having an issue with this? Have you updated to Emby Server 4.7.10? Link to comment Share on other sites More sharing options...
ShadowKindjal 19 Posted January 10, 2023 Author Share Posted January 10, 2023 I apologize for the delayed response but yes, I am still dealing with this issue on version 4.7.11. I plan to buy a gen 4 nvme ssd to see if that relieves some of the issues I have been having but I wanted to post my disk speeds for consultation prior to buying the drive. If I'm being honest, the measured speeds from my drives make me feel like an additional drive won't make much of a difference but I don't know what the typical read write speeds are for a transcoded 1080p stream. Currently data is stored on the disk array before being transferred to the boot SSD's transcoding temp directory. Disk Array Boot SSD Link to comment Share on other sites More sharing options...
ShadowKindjal 19 Posted January 10, 2023 Author Share Posted January 10, 2023 I'd like to add that the issue performance isn't an issue. I notice slow downs at CPU utilizations of only 20%. Memory is fine too. On average 40GB of the 128GB of RAM is always free and my SWAP file is 1meg. Lastly, with the measured speeds above it's hard for me to conclude my drives as the culprit. This happened again the other day with about 13 active 1080p streams but then last night I had another 13 streams going with no issue. From personal experience, it seems like the interface slows down but once you get a stream playing it has no issues until you need to skip ahead of the buffer. Restarting the server seems to be the only immediate fix but that halts all active streams. Link to comment Share on other sites More sharing options...
rbjtech 4324 Posted January 10, 2023 Share Posted January 10, 2023 Just looking at the latest log - I see some very large network latency in responding to the clients - are the clients very remote ? As an example - here I see average latency ranging from 930ms to 4500ms Are you sure this is not a network issue ? As a comparison - my remote connection latency is ~15-20ms .. Link to comment Share on other sites More sharing options...
ShadowKindjal 19 Posted January 10, 2023 Author Share Posted January 10, 2023 Yes, the majority of clients are remote but this issue occurs on local client devices as well. Pinging Google's primary DNS server yields a 10-12ms of latency. Link to comment Share on other sites More sharing options...
ShadowKindjal 19 Posted January 10, 2023 Author Share Posted January 10, 2023 (edited) Pinging my server from my work computer yields 9-17ms of latency Edited January 10, 2023 by ShadowKindjal Link to comment Share on other sites More sharing options...
rbjtech 4324 Posted January 10, 2023 Share Posted January 10, 2023 Sure - All I'm suggesting is the latencies in the log are huge - it's possible your network hardware cannot keep up with the concurrency / streaming demands wrt IP queues. Is it a decent router/firewall or is it the ISP supplied equipment ? Is there a certain number of clients when the issue begin to start ? Worth looking at the network queues when you are getting loaded - if the interfaces are being held due to queues, then this will certainly come across as very laggy/unresponsive - but emby itself does not have any issues. Is the LAN/WAN sharing a common interface ? Link to comment Share on other sites More sharing options...
ShadowKindjal 19 Posted January 10, 2023 Author Share Posted January 10, 2023 (edited) I completely agree that the latencies are ridiculously large and I appreciate you pointing that out. My current networking hardware utilizes a Ubiquiti USG-3P. The WAN port connects directly to my my ISP supplied ONT, the LAN port connects to a Unifi switch that handles all my LAN traffic, and the server connects directly to the LAN2 port on the USG. Edited January 10, 2023 by ShadowKindjal Link to comment Share on other sites More sharing options...
ShadowKindjal 19 Posted January 10, 2023 Author Share Posted January 10, 2023 I would like to add that over the past month the USG has not gone above 6% CPU usage and 36% memory usage Link to comment Share on other sites More sharing options...
Luke 37232 Posted January 10, 2023 Share Posted January 10, 2023 1 hour ago, ShadowKindjal said: I apologize for the delayed response but yes, I am still dealing with this issue on version 4.7.11. I plan to buy a gen 4 nvme ssd to see if that relieves some of the issues I have been having but I wanted to post my disk speeds for consultation prior to buying the drive. If I'm being honest, the measured speeds from my drives make me feel like an additional drive won't make much of a difference but I don't know what the typical read write speeds are for a transcoded 1080p stream. Currently data is stored on the disk array before being transferred to the boot SSD's transcoding temp directory. Disk Array Boot SSD Server log? Link to comment Share on other sites More sharing options...
ShadowKindjal 19 Posted January 10, 2023 Author Share Posted January 10, 2023 1 hour ago, Luke said: Server log? I happened to have an event last night. I ended up performing a restart around 9:48pm embyserver-63808915719.txt Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now