Enabling real time monitoring creates a high cpu load

[Guest]

Whenever I enable real time monitoring for a small Downloads mixed content library on my SSD, the cpu load increases significantly.

CPU:  51% usr   9% sys   0% nic  37% idle   1% io   0% irq   0% sirq
Load average: 0.96 0.94 0.84
54% /system/EmbyServer -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3

CPU:  51% usr   9% sys   0% nic  37% idle   1% io   0% irq   0% sirq
Load average: 0.96 0.94 0.84 
54% /system/EmbyServer -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3

As a workaround for this, I disabled real time monitoring for that library and setup a scheduled task to scan my library at 1 minute intervals and the server load decreases immensely.

CPU:   0% usr   0% sys   0% nic  98% idle   0% io   0% irq   0% sirq
Load average: 0.14 0.20 0.22 
0% /system/EmbyServer -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3

Now this workaround would work well enough for my needs except I also have 2 other large static libraries that get scanned/updated on startup only so they can idle down until they are needed and enabling this scheduled task every minute would keep those larger 8tb drives spinning every minute without idling down.

When I have a scheduled task to scan my library at startup only and real time monitoring enabled on my downloads folder, everything works the way I want it to, but the CPU load of that one feature isn't something I can overlook easily.

I can recreate these results easily, run commands, or post relevant logs if needed.

Thanks,

[Luke 42078]

Hi, please attach the Emby server log from when this happens, ideally with debug logging enabled. Thanks.

[Guest]

Attached. This was the load at the time I pulled the Emby server log with debug enabled while real time monitoring was active.

CPU:  55% usr   6% sys   0% nic  36% idle   0% io   0% irq   1% sirq
Load average: 1.00 0.91 0.91
59% /system/EmbyServer -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3

embyserver.txt

[Guest]

It appears that due to my large libraries having 256MB cache available on their respective HDD, the large libraries appear to be caching the scans while the scans aren't actually waking up the 8TB HDDs.  This means I can solve my problem by just scheduling a media library scan every minute without enabling "Real time monitoring" and my servers return to a near normal load.  I don't know how much the scans cache but as long as it's under 256MB, im good.  I assume that if the scans ever require more than 256MB that the HDDs will start waking up every minute.

[Guest]

I'm still here to troubleshoot this issue so I can hopefully use the real time monitoring feature someday but scanning my library only once a minute is close to real time for me and reduces the server load significantly.  My problem may be resolved but the core problem wasn't so I will leave my lab and resources open to you so we can resolve this.  Just tell me what you require.

[Happy2Play 9780]

Looks like something is continuously causing changes to your files in your Downloads folder and everything is being Refreshed.

Example looks like this episode was refreshed 18 times.

Spoiler

    Line   672: 2023-03-28 02:26:02.774 Info LibraryMonitor: Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV (/Downloads/Ozarks Folder/Ozark.S01.COMPLETE/Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV.mkv) will be refreshed.
    Line  5420: 2023-03-28 02:32:05.347 Info LibraryMonitor: Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV (/Downloads/Ozarks Folder/Ozark.S01.COMPLETE/Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV.mkv) will be refreshed.
    Line 14237: 2023-03-28 02:40:02.774 Info LibraryMonitor: Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV (/Downloads/Ozarks Folder/Ozark.S01.COMPLETE/Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV.mkv) will be refreshed.
    Line 17273: 2023-03-28 02:43:01.815 Info LibraryMonitor: Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV (/Downloads/Ozarks Folder/Ozark.S01.COMPLETE/Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV.mkv) will be refreshed.
    Line 18760: 2023-03-28 02:45:01.711 Info LibraryMonitor: Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV (/Downloads/Ozarks Folder/Ozark.S01.COMPLETE/Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV.mkv) will be refreshed.
    Line 19487: 2023-03-28 02:47:02.122 Info LibraryMonitor: Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV (/Downloads/Ozarks Folder/Ozark.S01.COMPLETE/Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV.mkv) will be refreshed.
    Line 21794: 2023-03-28 02:49:02.214 Info LibraryMonitor: Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV (/Downloads/Ozarks Folder/Ozark.S01.COMPLETE/Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV.mkv) will be refreshed.
    Line 22724: 2023-03-28 02:51:02.349 Info LibraryMonitor: Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV (/Downloads/Ozarks Folder/Ozark.S01.COMPLETE/Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV.mkv) will be refreshed.
    Line 24878: 2023-03-28 02:53:03.001 Info LibraryMonitor: Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV (/Downloads/Ozarks Folder/Ozark.S01.COMPLETE/Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV.mkv) will be refreshed.
    Line 27612: 2023-03-28 02:56:05.185 Info LibraryMonitor: Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV (/Downloads/Ozarks Folder/Ozark.S01.COMPLETE/Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV.mkv) will be refreshed.
    Line 29619: 2023-03-28 02:58:02.188 Info LibraryMonitor: Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV (/Downloads/Ozarks Folder/Ozark.S01.COMPLETE/Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV.mkv) will be refreshed.
    Line 32533: 2023-03-28 03:03:04.120 Info LibraryMonitor: Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV (/Downloads/Ozarks Folder/Ozark.S01.COMPLETE/Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV.mkv) will be refreshed.
    Line 35970: 2023-03-28 03:05:03.362 Info LibraryMonitor: Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV (/Downloads/Ozarks Folder/Ozark.S01.COMPLETE/Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV.mkv) will be refreshed.
    Line 37214: 2023-03-28 03:07:01.264 Info LibraryMonitor: Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV (/Downloads/Ozarks Folder/Ozark.S01.COMPLETE/Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV.mkv) will be refreshed.
    Line 40522: 2023-03-28 03:10:03.713 Info LibraryMonitor: Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV (/Downloads/Ozarks Folder/Ozark.S01.COMPLETE/Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV.mkv) will be refreshed.
    Line 43960: 2023-03-28 03:13:03.170 Info LibraryMonitor: Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV (/Downloads/Ozarks Folder/Ozark.S01.COMPLETE/Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV.mkv) will be refreshed.
    Line 47299: 2023-03-28 03:16:04.264 Info LibraryMonitor: Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV (/Downloads/Ozarks Folder/Ozark.S01.COMPLETE/Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV.mkv) will be refreshed.
    Line 49583: 2023-03-28 03:18:02.661 Info LibraryMonitor: Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV (/Downloads/Ozarks Folder/Ozark.S01.COMPLETE/Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV.mkv) will be refreshed.

 

[Guest]

I've never set anything up to constantly change my files so I have no idea how to verify that or make it stop.

[Happy2Play 9780]

Don't know the technical side of this but something appears to be touching the files and the filesystem is reporting change to Emby via RTM and Refreshing these items.  At least that is what I am interpreting from the log.

Are the file timestamps changing on for example the episode I listed above?

[Guest]

They are not.  Apr 27  2020 Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV.mkv. 

I'm investigating transmission now, the only other program that has write access to the folder as well as verifying the permissions and user:group of everything in my library just to make sure something doesn't have a weird property that causes a rescan.

[Happy2Play 9780]

Just now, kufkis said:

They are not.  Apr 27  2020 Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV.mkv. 

I'm investigating transmission now, the only other program that has write access to the folder as well as verifying the permissions and user:group of everything in my library just to make sure something doesn't have a weird property that causes a rescan.

That would be be file created date I am guessing but what about modified date?  Sorry don't know the platform.

example of Modified data will trigger RTM.

2023-03-28 18:36:08.124 Info LibraryMonitor: Atomic Blonde (C:\Users\Media\Desktop\Videos\Movies - nfo\Atomic Blonde (2017)\Atomic Blonde (2017).mkv) will be refreshed.

[Guest]

It's openwrt linux.  I was using "ls -l" but "date -r" does the trick.

display the last modification time of FILE

date -r Ozark.S01E01.720p.NF.WEBRip.x264-GalaxyTV.mkv

Mon Apr 27 01:08:40 MDT 2020

[Happy2Play 9780]

Honestly don't know but something via the filesystem is reporting changes and triggering Emby via RTM.

Devs or fellow platform users will have to assist more.

[Guest]

Thanks for the help.  I got a possible lead to investigate now which is more than i had.

[Guest]

I think it is or was a helper script for transmission that helps me clear the history from the queue that have already been downloaded and moved.  It doesn't run constantly but clears my download queue once a minute. I changed it to clear my queue every 15 minutes and the cpu load seems to go back to normal.

Side note: The helper script was only enabled because the feature "Script done" in transmission has never worked for me in any fashion so I've been using a workaround for 10 years now. I'm going to go bug them now for this feature so I can remove the helper script completely so that it doesn't even bother Emby every 15 minutes.

[Luke 42078]

Thanks for the feedback. Generally speaking i would suggest that if you have a folder that is constantly being written to, then I would either adjust that behavior or not use the realtime monitor on that particular library.

[Guest]

yep. I agree, but I didn't know something was spamming the folder.

This project help me locate a script that was spamming my folder.

Thanks.