Long library loading times after every server startup

[Gaba90 4]

Hi emby team,

after every server startup of my nas / the emby server, no matter how many hours later I do the first login to emby, the initial loading of the library's takes quite long, over 10 seconds, sometimes over 20 seconds. After the first login the loading times are normal. 

Is this normal behavior that emby only start to read the libraries when someone logins first? Shouldn't it do that during startup? 

Attached is my server log. 

embyserver.txt

[Luke 37019]

HI, is it possible that coincides with a drive spinning up?

[Gaba90 4]

Hi Luke, no as I said this only happens once every time i first start the server and even if I happen to login 2 hours later (like Today) but the nas is still in use for other stuff, so the drives are not going to sleep

[Carlo 4330]

Hi, What you could do is open up performance monitor then switch to watching specific processes.
Note the exact time when you login.  Don't even do anything except login.  Wait 5 minutes then download the emby server log from the LOG menu.

Have a read if you understand it or upload it here with the time you logged in and we'll look to see what it was doing.

Meanwhile did you notice anything unusual happening looking at the processes?

Edited December 3, 2021 by cayars

[Gaba90 4]

13 hours ago, cayars said:

Hi, What you could do is open up performance monitor then switch to watching specific processes.
Note the exact time when you login.  Don't even do anything except login.  Wait 5 minutes then download the emby server log from the LOG menu.

Have a read if you understand it or upload it here with the time you logged in and we'll look to see what it was doing.

Meanwhile did you notice anything unusual happening looking at the processes?

Do you mean synologys performance monitor? Or does emby have something like that inside? 

[Luke 37019]

Hi, are you still seeing this?

[Gaba90 4]

Yes, sadly. I have attached today's log. Started server at 9am, logged in at 11 am and initial loading took over 20 seconds (can be seen in the log with response time 24000 ms and higher) 

embyserver (1).txt

[gillmacca01 149]

I'm seeing the same on a QNAP.

I presumed that Emby was reading cache files to load things up faster, and at restart there is no cache files, so has to build them, but I could be wrong

[Luke 37019]

4 hours ago, Gaba90 said:

Yes, sadly. I have attached today's log. Started server at 9am, logged in at 11 am and initial loading took over 20 seconds (can be seen in the log with response time 24000 ms and higher) 

embyserver (1).txt 82.2 kB · 0 downloads

And then after that it's fine?

[Gaba90 4]

2 hours ago, Luke said:

And then after that it's fine?

Yes after that first load it's fine. 

@gillmacca01i checked the cache folder but there doesn't seem to be a lot of files written during that time slot. But I will check when I start the next time if the folder is empty. 

 

[Carlo 4330]

On 12/7/2021 at 5:18 AM, Gaba90 said:

Yes, sadly. I have attached today's log. Started server at 9am, logged in at 11 am and initial loading took over 20 seconds (can be seen in the log with response time 24000 ms and higher) 

embyserver (1).txt 82.2 kB · 1 download

I'm seeing some errors in your logs with some Libraries not available.
Error in Directory watcher for: /volume1/music/Hörbücher
Error in Directory watcher for: /volume1/video2/Serien

I'm also seeing errors: The configured user limit (8192) on the number of inotify watches has been reached, or the operating system failed to allocate a required resource.
Synology ran out of inotify resources and it needs to be adjusted.

You need to enable SSH in DSM if not enabled already then SSH into the NAS.

Run these two commands one after the other but first do sudo -i and type the admin password to gain root access.

echo fs.inotify.max_user_watches
echo fs.inotify.max_user_instances

This will probably show 8196 and 100 which need to be raised.

sysctl fs.inotify.max_user_watches=524288
sysctl fs.inotify.max_user_instances=1000

Run the first two command again to verify the numbers changed.

These values will only be valid until the NAS has restarted.  What you want to do now is setup a daily task in Task Scheduler (control panel).
Make it a one time job owned by user root that runs only on startup.
The commands to run at startup are those two lines above that reset the numbers.
Make sure to enable the job so it's active.  You can manually run it now to make sure you don't get an error.

If you want a good test. Run these two commands again from SSH setting the numbers lower.  Then run the schedule task and if everything is working when you recheck them from SSH they will be 52488 and 1000.  You will be in good shape moving forward.

[Gaba90 4]

I have already set this as a command to run at start up from the nas.

Inotfity fix from another thread here:

sh -c '(sleep 90 && echo 204800 > /proc/sys/fs/inotify/max_user_watches)&'

It just seems to overlap with running the command and starting the server. As you can see a little below the error message the directory watcher is being started again without error. I don't think that's what's causing this. 

[Gaba90 4]

Just tried updating to beta 4.7.0.19 with little improvements but still taking a while. 

 

Could these options help in this case? 

[Luke 37019]

6 hours ago, Gaba90 said:

Just tried updating to beta 4.7.0.19 with little improvements but still taking a while. 

 

Could these options help in this case? 

It's possible, yes.

[Gaba90 4]

What are preferable settings for that options? 

[Luke 37019]

You can try raising the two numeric values as high as you're willing to go based on the tradeoffs that the help text describes.

[Happy2Play 8250]

6 minutes ago, Luke said:

You can try raising the two numeric values as high as you're willing to go based on the tradeoffs that the help text describes.

Would EnableSqLiteMmio have any factor as this setting does not appear to have a UI switch?

 

But yes I would expect raising databasecachesize to help.

https://emby.media/community/index.php?/topic/103252-extremely-slow

 

[Carlo 4330]

Normally you would always want that setting on unless it causes a problem as it allows using memory-mapped I/O vs using read/write system calls.  On provides for better performance. The higher the IO is on the system the more this helps.

[Gaba90 4]

I have set it to 1gb for now, seems faster already will test it for a few days. The ram I have in my system is more than enough and I hardly use other applications than the media server, so my ram usage is usually below 10%.

[Carlo 4330]

What exactly did you set to 1GB?
if you have RAM to spare setting "Database cache size (MB)" to 512 is a good idea.  Going past that will have marginal return up to 1024 but this is something you can test for on your own system.

"Attempt to optimize the database" on server shutdowns will cause shutdowns to take longer but will do a quick analysis of the database using the "Analysis row limit". 400 is the default but anything from 100 and 1000 will work.  The higher the number, the more rows are used during shutdown to analyze the database to see if an optimization should take place.

"Vacuum the database on the next Emby Server startup" is a one time use setting.  This will force an optimization to take place on startup so don't be surprised if it takes a while on start.  Just let it go without any interference. 

I manually run the vacuum/optimize anytime I've added media in volume to keep things nice and speedy.  If you haven't done this before or in a while, check that box and restart the server.  It will unmark/disable the option on start up so you don't have to.

To get a rough idea how much memory you need to use for the first option restart your system after doing a vacuum.  Let Emby settle and then look at your memory use for EmbyServer.  Now try doing some searches for media or people timing the results.  Do 5 or so searches and write down the specific order and time it took each search. Now scroll through your movie library a thousand or so entries. When done take note of how much memory is in use for EmbyServer.

You can now adjust the number and repeat the exact same steps. If the memory is being as a cache you should see better times for the search.  Same with the ending memory use.  Sqlite will only use as much memory as it needs to if you had it set to 1024 but it only needed 256, that is all it will allocate.  So it won't hurt to set this higher if you have RAM to spare, but if it is actually using a full 1024 and you hardly see any difference between that and 512 then from a resource standpoint 512 is likely a better choice if the memory is used for other purposes.

Hope that helps

[Gaba90 4]

Hi Cayars, 

i have set database cache to 1GB. 

now after reading your post i also increased the row limit for database optimization and did a vacuum of the database on the next emby server startup. --> this seems to have solved my problem because the last 2 days i started my server and logged in some time later, everything seems to be loaded already thank you !

[Luke 37019]

Thanks for the feedback.

[Carlo 4330]

@Gaba90 that's awesome.

Glad it's working well for you now.

Once in a while or if you add a lot new media, give the optimize a run.  Keeps the database optimized and running quickly.