Slow library scan

[charles-martel 0]

Hi,

 

I meant : between version 3.5.3.0 and version 4.0.1.0 ?

 

Best regards.

[Luke 42077]

No, 4.1.0.5.

[charles-martel 0]

Hi,

 

No, no, I want to said : Why there is a regression with the speed of library scan between the 3.5.3.0 and the 4.0.1.0 ?

Because the 3.5.3.0 is fast and the 4.0.1.0 slow.

 

Best regards.

[Carlo 4560]

Try testing the latest BETA which is really fast!

As Luke pointed out this was improved in 4.1.0.5.

[charles-martel 0]

I know the beta 4.1.0.5 correct the issue, I just want understand why there was a regression before.

[Carlo 4560]

Likely because of the new DB format combined with new code and even possibly new scanner methods.

 

Bugs happen during upgrades and this was a massive upgrade from a code standpoint.

[brunovon 1]

Glad to know that 4.1.0.5 is fixing this - Thanks charles-martel for confirming this. Any idea when this will be live? And where can I download and manually install the beta? Since mine is a home server, I do not mind installing the beta.

[Luke 42077]

Hopefully soon. Instructions are on the download page of our website. Thanks.

[MindBullet 93]

Hey guys.

I've been abscent quite a while, but I've done the initial library scan which took ~12 hours, now I'm doing another manual scheduled task library scan and it's already scanning for 4hrs 33 minutes and seems to be stuck at 93.6%. Is there an issue with the scanner atm? I remember this task was blazingly fast with the same amount of movies and TV shows, when I was on Emby half a year ago.

Logs are attached, hope there's something I/you can do. 

 

Best regards

embyserver.txt

[Luke 42077]

Hey guys.

 

I've been abscent quite a while, but I've done the initial library scan which took ~12 hours, now I'm doing another manual scheduled task library scan and it's already scanning for 4hrs 33 minutes and seems to be stuck at 93.6%. Is there an issue with the scanner atm? I remember this task was blazingly fast with the same amount of movies and TV shows, when I was on Emby half a year ago.

 

Logs are attached, hope there's something I/you can do. 

 

Best regards

 

Well you can see extra activity coming from plugins you installed all throughout the log. Try removing these:

2019-11-30 10:55:12.198 Info App: Loading Lastfm, Version=1.0.21.0, Culture=neutral, PublicKeyToken=null from C:\Users\donat\AppData\Roaming\Emby-Server\programdata\plugins\Lastfm.dll
2019-11-30 10:55:12.198 Info App: Loading MovieThemeSongs, Version=1.6.6.4, Culture=neutral, PublicKeyToken=null from C:\Users\donat\AppData\Roaming\Emby-Server\programdata\plugins\moviethemesongs.dll
2019-11-30 10:55:12.198 Info App: Loading MovieThemeVideos, Version=1.6.6.4, Culture=neutral, PublicKeyToken=null from C:\Users\donat\AppData\Roaming\Emby-Server\programdata\plugins\MovieThemeVideos.dll
2019-11-30 10:55:12.198 Info App: Loading ThemeSongs, Version=1.6.6.4, Culture=neutral, PublicKeyToken=null from C:\Users\donat\AppData\Roaming\Emby-Server\programdata\plugins\themesongs.dll
2019-11-30 10:55:12.198 Info App: Loading ThemeVideos, Version=1.6.6.4, Culture=neutral, PublicKeyToken=null from C:\Users\donat\AppData\Roaming\Emby-Server\programdata\plugins\ThemeVideos.dll
2019-11-30 10:55:12.198 Info App: Loading YTrailers, Version=1.6.6.0, Culture=neutral, PublicKeyToken=null from C:\Users\donat\AppData\Roaming\Emby-Server\programdata\plugins\YTrailers.dll

Or keep them, and recognize that they provide you with something but each comes with it's own cost. When you install plugins you are notified about this during the installation process.

[MindBullet 93]

Hey Luke!

Thank you for your answer and this solved the issue. 

I uninstalled all the plugins 1 by 1 and it turns out it was the last.fm plugin. Thank you so much Luke, made my day! (And Emby Server ).

 

Big love!

 

[Luke 42077]

Thanks for the feedback.

[MindBullet 93]

Oh wait. Now it's stuck at 94.6% instead of 93.6. Hell no!

[MindBullet 93]

Yes it's again stuck, but only 1% farther :/ All the plugins in question are still uninstalled.

EDIT: It's a bit weird. It jumped to almost 97% and fell back to 93% and is now sitting on 94.6% for ~15 minutes. I only added like...15 episodes

Edited November 30, 2019 by MindBullet

[Luke 42077]

It's not stuck. I would just be patient and allow it to finish.

[MindBullet 93]

Hey again.

I finally managed it! It was like you said, that it might be an old-ish plug in, but it was not one from your list, but the trakt.tv plug in. I didn't uninstall it at first, but noticed it was still active. Library scan now finished in under 2 minutes <3 Thank you, solved!

[MindBullet 93]

Hey once again.

I would like not to bother you again, but it doesn't seem to be plugin related at all. Yesterday the library scan started to become unresponsive again, today I uninstalled all plugins which are from 3rd parties and it's stuck for the last 45 minutes at 93.6% again. Could anybody look into the logs again please?

EDIT: It seems like only the scheduled task is broken for me. Scanning manually on the library level always works

embyserver.txt

Edited December 1, 2019 by MindBullet

[Happy2Play 9780]

To me it looks like Emby purged about 1600 episodes from the database and is still processing episodes.

 

starting purge around here but I am not seeing what started it.

2019-12-01 11:29:36.199 Info App: Removing item from database, Type: Episode, Name: xxxxxxxxxxxxxxxxxxxxx, Path: , Id: 159644
2019-12-01 11:29:36.199 Info App: Deleting path Y:\Emby\metadata\library\77\77db730d24a62a89a361fe6e40a441b6

[MindBullet 93]

Oh okay! But what does that mean and how can I stop it from purging?

[MindBullet 93]

Could that be a permission issue? Because not everything of my Emby installation is on the local OS drive. I have a dedicated NVMe 2.0 2TB Samsung EvoPlus drive only for metadata and .bif files etc. That's where the drive letter Y: comes from

Edited December 1, 2019 by MindBullet

[Luke 42077]

Some files are not there that were there before.

[MindBullet 93]

Oh yes, that's true, I moved a few seasons from different HDDs and merged them together, e.g. S01-S02 of Chicago PD were on hard drive "X:" and I moved them to the rest of the TV show on "E:", let's say. Is this a problem?

 

Or let's take this example. S03-S04 of Criminal Intent were on XX and I moved them to the rest of the TV show on YY, did a scan and now the seasons are like they're not on my HDD, like in the screenshots. As if they wouldn't exist...Is the only way around this to move everything back where it came from? I didn't know you couldn't move files to different storages, if that is the case.

They're on the same HDD like the other seasons now, but don't show up. Weird thing is that my Google Chrome notifications get triggered when I added S03 and S04, but the episodes are missing though.

 

Edited December 1, 2019 by MindBullet

[MindBullet 93]

I ran another scan (I think the 4th today) and this time the seasons of Criminal Intent are there. That's weird (to me). Can you see anything in the logs that might help me for future instances? And: is this not applicable on Emby to move files to whichever location attached to the Emby installation?

Here's a log a final time I hope Thank you all <3 

[Luke 42077]

 

 

Is this a problem?

No, it's not a problem. Just give it time and allow the scan to finish.

[MindBullet 93]

Okay whew! Thank you sir <3