library scans went from 5 minutes to over 45

[psybertech 33]

I don't know what happened.

Within the past week (a bit before v3.5 was released) my library scans are just extremely long (went from 5 minutes to over 45).

I had some issues on scan times before that hung around 69%, but now its around the 88-90% time. This time it hangs so much longer....

 

Attached are my logs from just today's log changes (and reboots in-between). I aborted some after a while, but i hope they indicate something.

I see nothing really out of the norm and I an't think of any changes to the backend.

 

I've done the disable plugin thing a bunch of times before today, but, of course, I will try them again.

 

Server is Ubuntu server 64 but

media is mounted via CIFS (tried NFS mounting too but that caused timestamp issues) to my Synology DS1812+ NAS...

 

Attached logs are all over the place... try the largest one and go from there if that helps.

Debug logging was enabled for most, if not all, of these. (I turned debug logging on a few days ago to see if I could see what is what).

Also note that debug logging has not changed the scan time significantly when enabled. 

 

I know there are a handful of videos that fail ffprobe since they were corrupt and I have replace all of them (or deleted now).

 

 

Not sure if this log is in the ZIP attached, but this is my current log which ran a scan for 38 minute... (Log.txt)

 

Anyway.... 

 

Any help is GREATLY appreciated!

I just want sub 5 min scans again!!!

 

-for support.zip

Log.txt

Edited July 20, 2018 by psybertech

[Luke 42085]

Hi, please try removing these plugins and see how that compares.

2018-07-16 10:08:51.871 Info App: Loading MediaBrowser.Plugins.TimeLordMovies, Version=18.6.20.923, Culture=neutral, PublicKeyToken=null from /var/lib/emby/plugins/MediaBrowser.Plugins.TimeLordMovies.dll
2018-07-16 10:08:51.871 Info App: Loading CoverArt, Version=4.0.72.0, Culture=neutral, PublicKeyToken=null from /var/lib/emby/plugins/CoverArt.dll
2018-07-16 10:08:51.871 Info App: Loading MediaBrowser.Plugins.Trailers, Version=1.1.6.0, Culture=neutral, PublicKeyToken=null from /var/lib/emby/plugins/Mediabrowser.Plugins.Trailers.dll
2018-07-16 10:08:51.871 Info App: Loading EmbyReports, Version=1.0.8.0, Culture=neutral, PublicKeyToken=null from /var/lib/emby/plugins/EmbyReports.dll
2018-07-16 10:08:51.871 Info App: Loading statistics, Version=2.0.4.5, Culture=neutral, PublicKeyToken=null from /var/lib/emby/plugins/Statistics.dll
2018-07-16 10:08:51.872 Info App: Loading MediaBrowser.Plugins.SmtpNotifications, Version=3.1.3.0, Culture=neutral, PublicKeyToken=null from /var/lib/emby/plugins/MediaBrowser.Plugins.SmtpNotifications.dll
 

Thanks.

[psybertech 33]

OK, disabled plugins and restarted server (hardware, not just emby service), and went straight to triggering a scan. No new files were to be added (I am pretty sure).

Once the scan was complete, I restarted the Emby service from the dashboard and collected the log (renamed just to add a note on what's what for me).

 

Attached is the log from that scan.

It took 42 minutes.

 

Debug logging was on, but even off, there is little to no difference.

 

Again, the hangup starts around the 88-92% mark.

Once it gets to about that range, which is somewhat quick to get to within a few minutes, it just then seems to then stall out.

 

Can I toggle on more debugging info with a different start command rather than using the service's default?

 

And this is just interesting... when searching the log for the word 'scan', once the scan starts (line 1 below) it is then found next after about 39.5 minutes (line 2 below). Then the next time is 3.5 minutes later once the scan is complete. If you remove the time from line 1 to line 2, the total time would be kinda close to what I was getting before (4:30 - 5:30 on average) Just interesting? Anyway............. 

2018-07-20 11:20:35.216 Info TaskManager: Executing Scan media library
2018-07-20 12:00:02.667 Info App: App Activity: app: My Library Scan, version: 3.3.1.0, deviceId: 28c8c33fab8844f7ae75a40e5df0a41c, deviceName: virtuntu
2018-07-20 12:03:28.091 Info TaskManager: Scan media library Completed after 42 minute(s) and 52 seconds

Any suggestions would be great.

 

Thanks!

 

J

embyserver-63667685216restart-scan-42-mins.txt

[Luke 42085]

If you'd like to try the beta server, that would be helpful, thanks.

[psybertech 33]

Luke,

 

I have no problem testing and providing feedback, but I don't currently have a setup to compile. Sorry... I'm just a web dev so my toolkit is limited.

 

I am trying to figure out the versioning on GIT using 'releases'...

I don't see any beta releases on GIT for 3.5.x like I do for 3.4 (which may not be what I am thinking anyway - I am more of a add/commit/push guy - not so much a pull and compile --- web-dev here mainly).

If needed, I will get the tools and figure it out, but if you could point me in the right starting direction, that would be great!

 

Thanks!

J

[Luke 42085]

Everything you need is on our website: https://emby.media/download

 

Thanks.

[embyuser24 53]

Hi,

 

The compiled releases can be found here;

 

https://github.com/MediaBrowser/Emby.Releases/releases

 

and if you are using docker you can use "emby/embyserver:beta" or "emby/embyserver:3.6.0.1" if you want a specific build in your docker run command. You can find emby docker tags here;

 

https://hub.docker.com/r/emby/embyserver/tags/

[Luke 42085]

Yes but the instructions are on our website. You never need to go anywhere else directly. Our download page has everything you need:

 

https://emby.media/download

 

Thanks.

[psybertech 33]

Wow.. I never thought to check the main DL page again.. I never scrolled below my needed version... just a few inches away! 

 

OK.. Giving it a go now, but won't be able to report back until tomorrow. Beer is good and sleep is needed.

 

Thanks again, Luke and misc random user-guy for the alternative releases link...

 

More to follow...

 

Thanks

J

[Luke 42085]

Thanks for the feedback.

[psybertech 33]

I wanted to report back... sorry it took a bit....

 

I upgraded to the Beta release 3.6.0.1 and then 0.2....

My scans dropped from 40++ minutes to about 15. So a nice change!

I decided to try the stable release after a bunch of scans and, to my surprise, they too are around 15 minutes now. ???

I re-enabled all plugins... same... went back to Beta... same... so I am not sure what happened with the change to the Beta release, but it did change my scan times for the better.

 

I am back on the Beta release with all plugins (and a couple more) that I had installed before and the scan times are consistent. Still not around the 5 minute mark, but I am much happier now since I check for network file changes and possibly scan every 45 minutes and didn't want a scan to be overlooked if a previous scan was still ongoing.

 

Anyway... I can provide new logs, but I see nothing now that wasn't there before. But I may be wrong.

 

Thanks for the Beta suggestion, Luke!

Whatever changed remained and for now I am sticking with the Beta releases.

Now.... to start posting questions about changes in the Beta dashboard-ui js scripts.

 

j

[Luke 42085]

Thanks for the feedback.