Emby Home Page + Menu Load Time - NAS Reboot Start-Up or Hibernation

[davidawarner 6]

Emby Friends, 

Question about Emby Home Page + Menu Load Time during NAS Reboot Start-Up or drives coming out of Hibernation.
This occurs if I reboot my NAS or when the drives come out of hibernation after idle of 20 minutes. The main home page
will take roughly 1.5 to 2 minutes to load and I can hear the NAS drives reading the drives to display the thumbnail content. 

Is this expected behavior? 

Here is my setup

Synology NAS DS720+ - Purchased last November 2022
Memory: 18GB
Hard Drives Two IronWolf 18TB Seagate 7200RPM drives. I have both drives as one volume and not running in a RAID formation.
Drives are healthy and no errors

Synology NAS: DSM 7.1.1-42962 Update 4

Emby Server is up to date current: Version 4.7.11.0
Media Size for Emby: Roughly 3TB of photos, video, music. 

I am running Ubiquity Network 1GB Network Switch with Ubiquity Access Point. If I am direct connect to the network switch, same behavior vs. wireless connections. 

So it looks like Emby server is reading the entire folder structure coming out of hibernation or power cycle of the NAS in one pre-load. 

What are my options here? 

Would investing in SSD NVM drives for read/write cache help for quicker access times? 

As a side note, when I access all the different menus - videos, photos, music, etc.. Response time is immediate and no issue's for hours of media access.

Please advise

tvos

 

[Happy2Play 8296]

A guess may be database cache size but would need to see the server log.

Personally, don't hibernate/sleep drives so don't really have any insight on this.  But after reboot would not expect this.

 

[davidawarner 6]

that is what I was thinking too about the hibernation of the drives. Right now, it is set to 20 minutes and they will go into hibernation.

Where can I find the emby logs or can I put a trace on the server and test?

please advise

tvos

[Happy2Play 8296]

1 minute ago, davidawarner said:

Where can I find the emby logs or can I put a trace on the server and test?

You would have to SSH into the NAS or go to Emby App Settings-Logs and view or download.

You can see Emby programdata path in Server Info via the three dot menu on the Dashboard.

 

[davidawarner 6]

Thank You... should I email the logs or attach them here.

I also have a open case with Synology too.

Please advise

tvos

[Happy2Play 8296]

2 minutes ago, davidawarner said:

Thank You... should I email the logs or attach them here.

I also have a open case with Synology too.

Please advise

tvos

You can PM if you like or attach here.  There is an anonymized option when you click on the log via Logs but currently does not catch everything depending on your setup.

[davidawarner 6]

Hello Happy2Play, 

Here is the  hardware and server log. I did see some interesting errors in the log using Notepad ++ but not sure if this is
contributes to the load time. 

When I obtained the logs it was after a complete power cycle of my NAS. I loaded emby server main interface and browse each folder: video, movies, music, etc. 

I have clocked that it takes just under two minutes for the home landing page to draw the thumb nails, content, etc. Of course, I did take in consideration the the NAS is still 
loading other services. But at least you  have fresh logs w/o any other variables being introduced.

Let me know your findings and Thank You

tvos
 

embyserver.txt hardware_detection-63812076065.txt

Edited February 16, 2023 by davidawarner

[Happy2Play 8296]

The inotify error only relates to Real Time Monitoring and you have to adjust it.

 The configured user limit (8192) on the number of inotify watches has been reached, or the operating system failed to allocate a required resource.

But yes you do have some really slow queries.

Should help some but don't know with this issue but increase you Database Cache Size.

And maybe Vacuum your database.

[davidawarner 6]

I will definitely add these performance tweaks per your recommendation.

Now, my synology NAS has 18 GB of memory so what would you suggest for increasing the database cache size? 300MB, 500MB, 1GB?

Is  vacuum the database? Is this in any relation to indexing/defragmentation. 

Please advise and Thank You for the quick response. 

tvos

 

Edited February 16, 2023 by davidawarner

[Happy2Play 8296]

17 minutes ago, davidawarner said:

I will definitely add these performance tweaks per your recommendation.

Now, my synology NAS has 18 GB of memory so what would you suggest for increasing the database cache size? 300MB, 500MB, 1GB?

Is  vacuum the database? Is this in any relation to indexing/defragmentation. 

Please advise and Thank You for the quick response. 

tvos

 

Depends on the size of the library.db but 18TB I would say 256 or 512 but 1024 should be an issue either.  One of my servers has a 297Mb db and I use 512.

Yes vacuum is related to indexing/defragmentation.

[davidawarner 6]

Will let you know my findings with your recommendation for performance and increasing the DB. I did increase to 1024 since I have the memory available. 

Again, Thank You for your quick response and help me out. 

Updates to follow

tvos

[davidawarner 6]

Hello Happy2Play, 

I have increased the DB memory to 1024 (1gb), vacuumed the database, along with the scheduler task. 

Reporting to you that this is now night and day 1000% improvement. I tested both scenarios from hibernation and three times during
cold/warm power cycle. Once the web server started, the home page was immediate accessible along with navigating the folder structures. 

I did include the new logs for your review. A few errors to show for your review if need to be concerned with. 

 

Line 215: 2023-02-15 21:39:27.501 Info VaapiCodecProvider: ProcessRun 'ffdetect_vaencdec' Execute: /var/packages/EmbyServer/target/bin/ffdetect -hide_banner -show_program_version -loglevel 48 -show_error -show_log 40 vaencdec -print_format json

    Line 216: 2023-02-15 21:39:27.974 Error LibraryMonitor: Error in Directory watcher for: /volume1/Photos
    Line 216: 2023-02-15 21:39:27.974 Error LibraryMonitor: Error in Directory watcher for: /volume1/Photos
    Line 217:     *** Error Report ***
    Line 233: 2023-02-15 21:39:27.977 Error LibraryMonitor: Error in Directory watcher for: /volume1/Music
    Line 233: 2023-02-15 21:39:27.977 Error LibraryMonitor: Error in Directory watcher for: /volume1/Music
    Line 234:     *** Error Report ***

    Line 254: 2023-02-15 21:39:28.715 Info QuickSyncCodecProvider: ProcessRun 'ffdetect_qsvencdec' Execute: /var/packages/EmbyServer/target/bin/ffdetect -hide_banner -show_program_version -loglevel 48 -show_error -show_log 40 qsvencdec -print_format json 

    Line 257: 2023-02-15 21:39:28.964 Info NvidiaCodecProvider: ProcessRun 'ffdetect_nvencdec' Execute: /var/packages/EmbyServer/target/bin/ffdetect -hide_banner -show_program_version -loglevel 48 -show_error -show_log 40 nvencdec -print_format json 
 

Thank You

tvos

 

 

embyserver.txt hardware_detection-63812093969.txt

[seanbuff 840]

Glad your load times have improved.

20 minutes ago, davidawarner said:

 Line 233: 2023-02-15 21:39:27.977 Error LibraryMonitor: Error in Directory watcher for: /volume1/Music
    Line 234:     *** Error Report ***

Looks like you're still hitting your iNotify limit -- did you also follow the steps to increase this using the first link in Happy's post above?

[davidawarner 6]

10 hours ago, seanbuff said:

Glad your load times have improved.

Looks like you're still hitting your iNotify limit -- did you also follow the steps to increase this using the first link in Happy's post above?

Unless, I missed a step, I do have RTM enabled for a scheduled task, received the email with no errors reported, SSH into the NAS server and ran the commands per instructions.

Increased my database to 1024GBm Vacuum the database.

Suggestions?

Last but not least, should I Vacuum the database weekly, monthly? Is there a task I can setup to run w/o manual intervention?

Please advise and Thank You

tvos

 

[Luke 37096]

There is no scheduled task for vacuuming but there will be in the next release.

[Happy2Play 8296]

2 hours ago, davidawarner said:

Unless, I missed a step, I do have RTM enabled for a scheduled task, received the email with no errors reported, SSH into the NAS server and ran the commands per instructions.

 

Suggestions?

Looks correct are you still seeing that error in server log?

But someone more familiar with the Synology may have to comment.

[davidawarner 6]

Hello Happy2Play, 

Here is the most recent server log as of 11:45 AM PST. in which it does show the error directory watcher.

I did talk with Synology this morning and they did explain not to put the drivers in hibernation, especially with Seagate IronWolf drives. The other
component is that my drives that I am using is not Gold certified with Synology. (Don't want to go down that path lol). Ran a complete disk check, monitoring health
and purchased two 18TB drives back in November. 

What are your thoughts? Right now, everything is immediate access and no lag when drawing the screen and reading the content. 

tvos

embyserver.txt

[Happy2Play 8296]

9 minutes ago, davidawarner said:

Hello Happy2Play, 

Here is the most recent server log as of 11:45 AM PST. in which it does show the error directory watcher.

I did talk with Synology this morning and they did explain not to put the drivers in hibernation, especially with Seagate IronWolf drives. The other
component is that my drives that I am using is not Gold certified with Synology. (Don't want to go down that path lol). Ran a complete disk check, monitoring health
and purchased two 18TB drives back in November. 

What are your thoughts? Right now, everything is immediate access and no lag when drawing the screen and reading the content. 

tvos

embyserver.txt 37.36 kB · 0 downloads

Did you verify the task is working?

To check that your changes took affect you can optionally run the following commands in Terminal

cat /proc/sys/fs/inotify/max_user_instances
cat /proc/sys/fs/inotify/max_user_watches

 

 

[davidawarner 6]

yes, I did.. went into SSH command prompt, entered the commands and here are the results. the other commands earlier, nothing returned.,

Edited February 16, 2023 by davidawarner

[Happy2Play 8296]

Sorry not a Synology user but appear the task is working but Emby still throw the same error.

The configured user limit (8192) on the number of inotify watches has been reached

@cayarsyou have any insight on this?

[davidawarner 6]

1 hour ago, Happy2Play said:

Sorry not a Synology user but appear the task is working but Emby still throw the same error.

The configured user limit (8192) on the number of inotify watches has been reached

@cayarsyou have any insight on this?

I have reached out to Synology and see if they have insight to this issue.

Updates to follow

tvos

[Carlo 4330]

20 hours ago, Happy2Play said:

Sorry not a Synology user but appear the task is working but Emby still throw the same error.

The configured user limit (8192) on the number of inotify watches has been reached

@cayarsyou have any insight on this?

What I might be wondering is if the current values are simply not high enough for the amount of files present on the system. Music & photos can really burn through the inotify handles needed compared to movies/shows per GB of space utilized since they are smaller. I've had to raise this numbers quite a bit on my system.

I've had to bump these values up on my system quite a bit. These are the settings I'm using:
sysctl fs.inotify.max_user_watches=8392704;
sysctl fs.inotify.max_user_instances=4096;

@davidawarner I'd be happy to remote in and have a look. Send me a PM (hover over my avatar to message me directly) if you would like to do this.
Carlo

[davidawarner 6]

3 hours ago, cayars said:

What I might be wondering is if the current values are simply not high enough for the amount of files present on the system. Music & photos can really burn through the inotify handles needed compared to movies/shows per GB of space utilized since they are smaller. I've had to raise this numbers quite a bit on my system.

I've had to bump these values up on my system quite a bit. These are the settings I'm using:
sysctl fs.inotify.max_user_watches=8392704;
sysctl fs.inotify.max_user_instances=4096;

@davidawarner I'd be happy to remote in and have a look. Send me a PM (hover over my avatar to message me directly) if you would like to do this.
Carlo

Hello Carlo,

I am IT engineer and don't think we need to do a remote session. If you would kindly share with me instructions when I SSH into my NAS and what file to modify that would be helpful. 

So question I have is will I need to increase the values periodically and where do I determine values?

 

Please advise

tvos

 

[Q-Droid 652]

With a 90 second delay for each of those commands to increase the inotify settings it's possible the Emby server is starting before the values are set. Try restarting the Emby server after you verify the values. If the errors go away after the Emby restart then you might have to reduce or remove the delay for the inotify change or delay the Emby startup.

 

[davidawarner 6]

2 hours ago, Q-Droid said:

With a 90 second delay for each of those commands to increase the inotify settings it's possible the Emby server is starting before the values are set. Try restarting the Emby server after you verify the values. If the errors go away after the Emby restart then you might have to reduce or remove the delay for the inotify change or delay the Emby startup.

 

please share how to perform this.

 

thank you

tvos