Emby Scan Slow

[lstryhrn 7]

So I have very slow scans when I am rebuilding my Emby library. I had the same issues when I first set up Emby a long time ago. My question is, is there somewhere in the scan settings that I scan each folder one at a time. I think the system is trying to scan every folder at one time. The scan speeds up when I manually go to each folder and identify what tv show it is. I dont really have the issue with Movies. 

Edited June 21, 2022 by lstryhrn

[rbjtech 4248]

check to see if you have thumbnails being created - if yes, then turn it off until the library is built.

[Luke 37046]

It generally comes down to what features are enabled. If you can attach an emby server log, when we can probably offer some advice to improve the performance of the scans:;

Thanks.

[lstryhrn 7]

12 hours ago, Luke said:

It generally comes down to what features are enabled. If you can attach an emby server log, when we can probably offer some advice to improve the performance of the scans:;

Thanks.

Sorry about that. I thought I attached the log files. 

embyserver.txt

[Luke 37046]

HI, to improve the performance of your scans, try removing the anime and theme songs plugins, and then restart the server and see how that compares.

[lstryhrn 7]

25 minutes ago, Luke said:

HI, to improve the performance of your scans, try removing the anime and theme songs plugins, and then restart the server and see how that compares.

Thanks for look at my log file but the scan is still so slow after uninstalling those plugins,. I did notice that its trying to scan every since folder and file at the same time. Is there a option where it can scan one folder at a time? I ask because when I go manually force it to find the metadata it takes no time. 

[Luke 37046]

It does go one at at a time. Why do you think it's trying to do all at once?

[lstryhrn 7]

1 minute ago, Luke said:

It does go one at at a time. Why do you think it's trying to do all at once?

Because I go to different folders\tv shows  while scanning and each one of them I go to has shows I can see the episode increasing. For example if I go to X-Files, if you keep refreshing the page the episodes number goes up. While X-files is loading up, I may go to Supernatural and its doing that same thing. 

[Luke 37046]

@lstryhrn OK I'm not sure why it happened that way but I'll take a look. I can assure it's one at a time though. It puts them all into a background queue and steps through it one at a time.

[lstryhrn 7]

I want to buy a  lifetime membership but one thing is stopping me  is that the scans are a pure joke. Anytime  you add alot of media to the server it takes ages for the scan to complete. Everywhere I look on this forum there is no answer to the problem. I even tried the scan the same media on my Jellyfin and Plex server to see if its was my setup and they were alot faster. I tired to siable all of my plugins and its still very slow. I thought it was my google drive mapping BUT I dont use that anymore. I store all of my media locally now.  

embyserver.txt

[Luke 37046]

Hi, the next release of the server should have some improved library scan performance. Thanks.

[rbjtech 4248]

19 hours ago, lstryhrn said:

I want to buy a  lifetime membership but one thing is stopping me  is that the scans are a pure joke. Anytime  you add alot of media to the server it takes ages for the scan to complete. Everywhere I look on this forum there is no answer to the problem. I even tried the scan the same media on my Jellyfin and Plex server to see if its was my setup and they were alot faster. I tired to siable all of my plugins and its still very slow. I thought it was my google drive mapping BUT I dont use that anymore. I store all of my media locally now.  

embyserver.txt 7.63 MB · 1 download

You have 'create thumbnails during library scan and when media added' turned on - thus for every episode/film added - it is sequentially adding these thumbnails before moving onto the next item.  Depending on your cpu/disk speed - this is adding a huge overhead.

It states this very clearly in the text.

If you want to quickly scan your library (like JF/Plex is doing) then opt to create the Video preview thumbnails as a scheduled task ONLY - done out of hours or when you specify in the scheduler.    Obviously the preview thumbnails won't show up until it's complete ..

 

The new version which @Luke is referring to probably doesn't do the tasks sequentially, it does the library scan first, THEN goes back and does the thumbnails - that would make much more sense ...  or he may be referring to the image extraction speed using keyframes - but I still think on a new install (or where you add lots of new media) to run these as separate tasks.

 

Edited September 1, 2022 by rbjtech

[lstryhrn 7]

40 minutes ago, rbjtech said:

You have 'create thumbnails during library scan and when media added' turned on - thus for every episode/film added - it is sequentially adding these thumbnails before moving onto the next item.  Depending on your cpu/disk speed - this is adding a huge overhead.

It states this very clearly in the text.

If you want to quickly scan your library (like JF/Plex is doing) then opt to create the Video preview thumbnails as a scheduled task ONLY - done out of hours or when you specify in the scheduler.    Obviously the preview thumbnails won't show up until it's complete ..

 

The new version which @Luke is referring to probably doesn't do the tasks sequentially, it does the library scan first, THEN goes back and does the thumbnails - that would make much more sense ...  or he may be referring to the image extraction speed using keyframes - but I still think on a new install (or where you add lots of new media) to run these as separate tasks.

 

I dont have that set to any of my libraries. its set to never. I dont know where you see that I have that turned on but its not. 

 

[rbjtech 4248]

4 minutes ago, lstryhrn said:

I dont have that set to any of my libraries. its set to never. I dont know where you see that I have that turned on but its not. 

 

The logs say differently ..

For every item being scanned - it has a 'quick-extract-image' task - that is creating the thumbnails.

So emby is either ignoring your config - or that log is before you turned it off ...?

Edited September 1, 2022 by rbjtech

[lstryhrn 7]

13 minutes ago, rbjtech said:

The logs say differently ..

For every item being scanned - it has a 'quick-extract-image' task - that is creating the thumbnails.

So emby is either ignoring your config - of that log is before you turned it off ...?

Well that's a great find on your part. I always have it turned off. Learned my lesson in the past.  Also I think its turned off by default when you create a library.  I have rebuilt my system 3 or 4 times and have always have that option turned off. Its only slow scans on tv shows. Im on server version 4.7.6.0. I dont think this matters because I have always had this problem. Its crazy that its saying its creating thumbnails because I dont  see thumbnails at all on the tv shows.

Edited September 1, 2022 by lstryhrn

[rbjtech 4248]

Hmm very odd.

Maybe roll the log (in schedule)

Then create a brand new test library, leave it as defaults, put a couple of episodes/movies in there - and lets see if it creates thumbnails for them ..

Once scanned - add the log and I'll take another look ...

Maybe it's a bug and that is what Luke alluded to above ... ?

[lstryhrn 7]

33 minutes ago, rbjtech said:

Hmm very odd.

Maybe roll the log (in schedule)

Then create a brand new test library, leave it as defaults, put a couple of episodes/movies in there - and lets see if it creates thumbnails for them ..

Once scanned - add the log and I'll take another look ...

Maybe it's a bug and that is what Luke alluded to above ... ?

Thanks for looking into this. It seem like it only happen when I have a massive import to the folder. But say when I add one or two shows, it takes no time. For example. My batman animated series has 117 shows . It took no time to scan those shows 

[rbjtech 4248]

Post the new log if you like and we can confirm if it created thumbnails for the new batman episodes ..

It may be be creating them, but just not using them (as you have that option turned off) - in which case, Luke should be made aware (or maybe he already is ..)

 

[lstryhrn 7]

2 minutes ago, rbjtech said:

Post the new log if you like and we can confirm if it created thumbnails for the new batman episodes ..

It may be be creating them, but just not using them (as you have that option turned off) - in which case, Luke should be made aware (or maybe he already is ..)

 

embyserver.txt

[rbjtech 4248]

It is - but strangely, only for items in the Featurettes sub folders ..

Very odd - hopefully @Luke can comment further ..

[lstryhrn 7]

1 minute ago, rbjtech said:

It is - but strangely, only for items in the Featurettes sub folders ..

Very odd - hopefully @Luke can comment further ..

Thanks for looking into this. I knew something goofy was going but I couldnt figure it out. I will wait for @Luke to comment.