playback pauses, web ui unresponsive during library scans

[psybertech 33]

This has been happening on and off for a while, but I am noticing it more lately.

 

During a library scan, playback on my Fire Tv will pause - not actually pause, meaning there is no OSD pop up, it just stops and sits there for 30 seconds or more.

During that time, the web UI will not load.

 

If I am watching everything, this happens at around 69% of the library scan.

 

I've seen this on direct play, direct stream and transcoding.

 

CPU usage does jack up to 50% during the can at points for most CPUs (8 total cores running) and a few spike to 100% but not all... maybe 2 or 3.

 

Attached is my log from right now.

 

the time to look at is in-between 20:25 and 20:27 on 6/29/2018 while I was watching GLOW S2E3. The title is direct playing according to the dashboard.

 

I really don't understand why this is happening.

 

The server is running on an Intel NUC NUC6i7KYK Core i7 with 8GB of RAM.

I can transcode 4k up to 35MBps on it with no problems. 

 

It is not a PC issue.

 

It just stops working normally during library scans.

 

I have had to come up with a custom library scan script that now I only run over night so as to not allow playback issues.

I really want to be able to update the library when file changes are detected automatically 24/7 and not have to manually do it when needed.

 

I am running Linux's 3.4.1 and I don't recall this being a factor as much before, but maybe I was not watching when scans were happening at that magical 69%-ish point.

 

Two logs are attached.

• the error_trimmmed cuts out most bits after the log start until before and after the issue pops up (I know it happens more overall, but I know the approximate times in this one since it happened while starting this topic).
• full log is everything from midnight until I made a copy to attach.

 

I do see traktv being logged near some of the times of the issues and no, I have not tried to disable it since it is important (not for viewing status, but for a way to tell Plex I have watched a throw-a-way episode and Plex will delete it since Emby doesn't support delete after X days for shows... I can remove it for testing of course)

 

Anwyay.............

 

Please help.

Thanks!!!

 

J

 

last edit to include CPU graph during the time of the scan

error_trimmed.txt

full log.txt

Edited June 30, 2018 by psybertech

[Luke 42092]

If you're seeing resource usage that high then i would try cutting down the number of installed plugins. Try removing these plugins:

	TimeLordMovies 18.6.20.923
	WebHooks 2.0.0.0
	CoverArt 4.0.72.0
	Reports 1.0.8.0
	Playback Reporting 1.0.15.0
	Statistics 2.0.4.5
	Email Notifications 3.1.3.0

[psybertech 33]

Luke,

 

I have removed webhooks and email notifications now, but these seem to be based on demand or via scheduled task triggers...

• WebHooks 2.0.0.0 -REMOVED
• Email Notifications 3.1.3. DISBABLED FOR ALL USERS 

But could these impact SQLite access and hamper playback and GUI response???

• Reports 1.0.8.0 REMOVED
• Playback Reporting 1.0.15.0 REMOVED
• Statistics 2.0.4.5  REMOVED

I have disabled those if you think that will help. They just seem to be DB analyzers unless they add a ton to the scraping process. (???)

 

OK.. uninstalled/disabled:

• reports
• playback reporting
• statistics
• webhooks
• email notifications (not removed; just disabled for all users for now)

I am leaving the cover art plugin active since it has been there since day 1 for me. I just can't see this being an issue... I may be wrong and will remove it if needed.

I also left TimeLordMovies since it should only run on demand.

 

I know I am not disabling or removing all you mentioned, so please chime in if you feel I need to remove the last two.

 

So far, running a scan now.... CPU is still spiking as normal (not worried yet) but UI is still sluggish and slow. Just one try means nothing of course.

 

I will report back with more time using the lack of the above plugins.

 

Thank you, Luke! 

 

Jay

[psybertech 33]

Just wanted to report back....

 

For whatever reason, I tried everything Luke asked in pieces and eventually all at once and nothing really changed.

 

BUT... I disabled the Auto-Box-Sets plugin and all seems to be so much better now.

 

I noticed it was taking about 1 minute to complete in my logs and saw that my issue was near the end of the scan so I figured I'd try to disable it and so far, it seems to be the cause of my woes.

 

Anyway.... I will turn it on if I think I need it again, but for now, I can manually add titles so I don't need it.

 

Anyway.....................

 

j

[psybertech 33]

Another followup...

 

Although with Auto-box-sets removed and things acting much better, I still am experiencing pausing during library scans - more specifically, near the end of the scan (usually ~69% or a little more - depends how fast I can get to the dashboard to see the progress).

 

It is no where near as bad as before, but happens at least once, if not more times, a day.

 

I can't say for sure if that number being lower is now due to my new script to scan the library which monitors the network folder and only runs the library scan if something was changed, rather than every 45 minutes all the time.

My scans now are only run between 5 and 10 times a day compared to a fixed 32 with the old time based scan.

So maybe the problem hasn't gone away, rather just masked due to decreased scans.

 

I don't know what is going on.

 

This does happen on direct play, direct stream and transcoding (most of my transcoding is old DVD rips that are fully transcoded at fast rates, so they aren't running real-time and I don't transcode often - 90++% of my stuff direct plays or direct streams). 

 

I need to figure this out. It sucks when it happens and embarrassing when with people - "Yeah... so.... my library is being updated right now... everything will be fine in like 30 seconds or so.... so, how was your day?.. Cool! YAY! Its playing again....... Beer?"

 

 

 

 

[Luke 42092]

Thanks for the info.