Jump to content

Extremely long media library scan and slow unresponsive apps


jameskitt616

Recommended Posts

jameskitt616

Hello fellow Emby Users,

lately i encounter extreme loading times, when i open emby on web/mobile app/desktop app it takes 10 seconds easy to load all the preview. Same for searching or any click, it takes ages. Starting a video file sometimes can take up to 30-60 seconds. If i go into the Database Settings and vacuum the database on next start up and restart Emby, everything is blazing fast again as it should be. But pretty much as soon as i start a library scan it slows down pretty fast till it's not usable anymore.
A library scan takes +18 hours in this state. The VM is installed on SSDs and runs Ubuntu 22.04 minimal version. I set up a new VM and installed Emby again but i run into the same issues. I am trying to troubleshoot this for a couple of weeks now but i sorta run out of ideas. For storage i use cloud storage mounted via rclone and image extraction off.

The VM has 8GB ram and 8 cores of a i9-10850K CPU assigned which is plenty so there shouldn't be any bottleneck. I sadly can't tell when this issue starts if it came with a new Emby version or not since i've got this for a while and only occasionally bothered to check what could cause it, so i have it for a while. 

If it requires me to test different scenarios and or send more files please let me know.

Thanks for any help in advance!

embyserver.txt

Link to comment
Share on other sites

Hi, can you try removing the stats plugin, then restart the server and see if that helps? Thanks.

Link to comment
Share on other sites

jameskitt616

I did, vacuumed the DB file and restarted Emby, started a library scan again and it still became slow. Sadly didn't help.

Link to comment
Share on other sites

jameskitt616

I also noticed after cancelling the library scan it does not become fast again, it stays in that bugged state.

Link to comment
Share on other sites

On 6/20/2022 at 2:03 AM, jameskitt616 said:

I also noticed after cancelling the library scan it does not become fast again, it stays in that bugged state.

Hi @jameskitt616 can you please provide a new log file after making the two above changes? Thanks !

Link to comment
Share on other sites

jameskitt616

Sure, here you go!

P.S. I also started *another* test run with another VM, this time with Ubuntu 20.04. The initial Library scan takes a few days so i will report back when it's done. Also i was not sure if this phenomenon instantly occurs or after certain events, i remember something when restoring the server config via the plugin...but totally unsure about this. As said will keep you posted!

embyserver-63791280000.txt

  • Thanks 1
Link to comment
Share on other sites

  • 2 weeks later...
jameskitt616

Seems like there is an issue with Ubuntu 22.04 and emby. My test with Ubuntu 20.04 seems to be successful, at least so far my first impressions everything seems to be normal again.

Link to comment
Share on other sites

twitchnln

I don't think Ubuntu is the issue, I am seeing similar here running 4.7.5.0 under debian 11. I have a media scan that has been running in excess of 5 days, using 700% of my CPU (AMD FX8320 8-core) and 97% of memory (32GB) according to top. This all began when I recently upgraded to 4.7.5.0 previously this machine had no issues serving Emby to my LAN.  When this happens the time in top process has shown up to 10 min discrepency between screen updates and local system clock.

Link to comment
Share on other sites

jameskitt616

My new Emby install is also starting to slow down again, but only a little bit *yet*. Which is in tolerance. Will keep you posted. 

Link to comment
Share on other sites

jameskitt616

Yes can confirm now. I ran into the same issue again.

After the initial library scan the following took pretty short 30-120 min. Now i am back at 6-10 Hours climbing up per scan basically and the longer Emby is running without reboot the slower it gets.

Link to comment
Share on other sites

Can you please provide a new example server log from 4.7.5? Thanks !

Link to comment
Share on other sites

jameskitt616

Yes, i am back to the original problem, library scan taking almost an day. Everything is not responsive, loading times are very long etc.

So doesnt seem to be a problem with the Ubuntu version.

Link to comment
Share on other sites

Hi, that's strange because I don't see anything obvious here to cause that. How many Emby libraries have you created and what is the content type of each one?

Link to comment
Share on other sites

jameskitt616

I have no explanation for this either, i honestly don't know.

As said, my library is fully in cloud storage via RClone mounted.
I've got A Movie Library with one path (+1 Collection Library)
I've got a TV shows library with 6 paths in there.

I also started another test today installing Plex to see if i run into the same issue's there, to verify that the issue is on Emby or on my side.

Link to comment
Share on other sites

OK, if you'd like to try the beta server version 4.8.0.6, there are some small improvements to the library scan there.

Then in the next build, which will be 4.8.0.7, there will be some more impactful improvements. Since your media is stored on something that will have slower disk I/O, I'd be interested to know if you see any improvements with it.

Link to comment
Share on other sites

jameskitt616

Will look forward to the update!

I/O never was a issue in the past years, i use this it like that for quite a while.

Link to comment
Share on other sites

By the way if you don't use nfo files, then disabling the nfo features on each of your libraries should also help.

Link to comment
Share on other sites

  • 3 weeks later...
jameskitt616

So the past weeks i investigated a bit more and found out interesting things.

So i use ZFS RaidZ1 on 5 SSDs for my Server. The problem is that i used consumer grade SSDs, and the load i was driving on my Server with all the other stuff hosted maxed out their IOPS which caused them to fail and have incredibly high wearout due the parity configuration needs much much higher calculations than a normal stripe or mirror does.

I changed my entire System Raid setup and for testing i installed Emby on one SSD. Now after a few days it is still making the initial library scan but the rest feels fairly snappy again and as it should be (Well so far at least, but i am positive that it will stay like that).

Link to comment
Share on other sites

jameskitt616

I will report back in a few days to a week if everything still is fast and if the library scans are also back to normal time again.

For now i will say sry for all the inconveniences i caused!

Link to comment
Share on other sites

jameskitt616

Alright, after my last message Emby became massively slow again. Slow library scans, slow UI etc.
At that point i wondered what changed.
Everytime i re-setup Emby and did all the initial library scans and everything seemed to work fine, i finished setting up everything, which included adding an API key for Ombi. Apparently i run into the issue that Ombi is hammering Emby with API calls and slowing it down that massively that it's not usable anymore. I removed the connection between the two of them and my library scan this night went down from +15 hours to 1 hour.

Link to comment
Share on other sites

Ah yes. That's actually tougher to spot now in the logs with the reduced http logging, unless debug logging is on. Thanks for the feedback.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...