Extremely slow scans on fresh install?

[famulor 1]

Hi. So i have a server with quickbox installed and i decided to install emby on it. For some reason it takes +24 hours to scan 1067 movies? Ive never seen something like that. its taken 5 hours to scan 150 tv shows and half an hour to scan 6 remux movies. Is there any obvious mistakes im making? 

 

Ive attached some server logs. If they are not the correct logs then tell me and i'll see if i can get the correct ones  

 

 

server-63637835326.txt

server-63637835333.txt

[Luke 42077]

It looks like fanart.tv is currently down and that will definitely cause scans to take longer if fanart metadata is enabled, which it is by default.

[Jdiesel 1431]

It is also worth mentioning that chapter image extraction is enabled which can be resource intensive and increase scanning times. Once the initial library scan is complete future scans should be quite quick even with chapter image extraction enabled.

[famulor 1]

It looks like fanart.tv is currently down and that will definitely cause scans to take longer if fanart metadata is enabled, which it is by default.

Where do i change what metadata agent emby uses?

 

It is also worth mentioning that chapter image extraction is enabled which can be resource intensive and increase scanning times. Once the initial library scan is complete future scans should be quite quick even with chapter image extraction enabled.

Wait how? I didnt enable it when i was adding my librarys(i even double checked). Where do i turn it off then?

[Jdiesel 1431]

It may just have kicked off a scheduled task to run at 2:00am. I didn't look too closely at the log file but did notice that there was mention of chapter image extraction taking place.

[Luke 42077]

It is also worth mentioning that chapter image extraction is enabled which can be resource intensive and increase scanning times. Once the initial library scan is complete future scans should be quite quick even with chapter image extraction enabled.

 

It's off by default, but will still check the locations to see if chapter images are there.

[famulor 1]

So is my issue related to tvdb? and if so is that something i can change?

[Luke 42077]

if you go into the server dashboard -> library -> metadata, you can go through the various types and ensure that fanart is disabled for each type. that will stop the server from trying to download from it.

[famulor 1]

if you go into the server dashboard -> library -> metadata, you can go through the various types and ensure that fanart is disabled for each type. that will stop the server from trying to download from it.

So i did that and tried again and its still kinda "stuck" on 69-70%

[Luke 42077]

@@famulor if you can attach a new server log we'll take a look. Please keep in mind, the very first library scan will take a little time and once you get through it it will be much faster after that. Thanks.

[famulor 1]

@@famulor if you can attach a new server log we'll take a look. Please keep in mind, the very first library scan will take a little time and once you get through it it will be much faster after that. Thanks.

I mean its on 70% now. Im guessing it shouldnt be this slow? New server log attached.

server-63637920000.txt

[Luke 42077]

We're probing every file for media info with ffprobe. What you can also do is disable the screen grabber under library -> metadata. You'll need to do it for each type individually, but doing it for episodes will have the most benefit.

[famulor 1]

@@Luke - What about movies then? There must be a reason why the scanning is so extremely slow? Or is it intended?

[Luke 42077]

Your recent log is scanning episodes and since you're at 70% i would guess the movies are already done. But you can certainly go back into library -> metadata and disable more metadata related features. You can disable the screen grabber for all video types. You can disable some additional secondary metadata/image providers such as fanart.tv.

 

But the main reason the first scan takes a bit of time is that we are probing every media file for media info with ffprobe.

[Luke 42077]

Once you get to 75, the primary scanning related functions are done and it moves onto post-scan activities, so given that you're at 70, it is probably almost done.

[famulor 1]

Your recent log is scanning episodes and since you're at 70% i would guess the movies are already done. But you can certainly go back into library -> metadata and disable more metadata related features. You can disable the screen grabber for all video types. You can disable some additional secondary metadata/image providers such as fanart.tv.

 

But the main reason the first scan takes a bit of time is that we are probing every media file for media info with ffprobe.

Only reason it was done with movies is because i only did the remux category which is 6 movies. Im doing the main movie library now and will report back with how long it took. its 22:56 here now. It seems strange that i need to disable all sorts of stuff to get any decent scanning speed.

[mastrmind11 722]

Only reason it was done with movies is because i only did the remux category which is 6 movies. Im doing the main movie library now and will report back with how long it took. its 22:56 here now. It seems strange that i need to disable all sorts of stuff to get any decent scanning speed.

Think about what it's doing...   all major metadata providers limit API calls per time period, so Emby has to account for that.  On top of that, ffprobe has to look at every single piece of media to read it's metadata in order to index it.  On top of that, it has to take frame grabs of every chapter.  On top of that, it has to account for your server, which may or may not be a piece of shit.  Like I tell my 4 year old, have some patience.

[Luke 42077]

Yea unfortunately everytime a topic like this is created, i then have to go look for more things to disable by default, so it looks like fanart.tv might be the odd man out, and also, we could possibly just live without the screen grabber for episodes since the series thumb image can always be used.

[famulor 1]

Think about what it's doing...   all major metadata providers limit API calls per time period, so Emby has to account for that.  On top of that, ffprobe has to look at every single piece of media to read it's metadata in order to index it.  On top of that, it has to take frame grabs of every chapter.  On top of that, it has to account for your server, which may or may not be a piece of shit.  Like I tell my 4 year old, have some patience.

I know what its doing. i also know how much faster the competition works. (dont even get started on the "go use that then" argument).. I have patience but it still does not make sense.

 

 

 

Yea unfortunately everytime a topic like this is created, i then have to go look for more things to disable by default, so it looks like fanart.tv might be the odd man out, and also, we could possibly just live without the screen grabber for episodes since the series thumb image can always be used.

Well plex uses fanart aswell and dosent deal with this issue.

[mastrmind11 722]

I know what its doing. i also know how much faster the competition works. (dont even get started on the "go use that then" argument).. I have patience but it still does not make sense.

 

 

 

Well plex uses fanart aswell and dosent deal with this issue.

You get to choose the platform, that's the beauty.  

 

However, considering Plex was first to market, and etc etc, I'd imagine there might be some QoS going on there.  Just saying...

Edited August 10, 2017 by mastrmind11

[famulor 1]

So the movie library took 1 hour and 30 minutes. (1067 movies). Thats more what i expected dno what changed but it seems to be faster now.

Now i just need to figure out why its so flaky with getting metadata in my local language (Danish). I thought plex and emby used the same metaagents

Edited August 10, 2017 by famulor

[Luke 42077]

What does flaky mean? As a test I just set my preferred language to Danish and then refreshed a title. Notice the Danish genres and overview.

 

[famulor 1]

Flaky means hit and miss like these for example: 

 

http://i.imgur.com/UlcfuC3.jpg

http://i.imgur.com/J1B3v5J.jpg

 

http://i.imgur.com/CrF3EHU.jpg

http://i.imgur.com/fO6PEzz.jpg

 

http://i.imgur.com/s9cEM4a.jpg

http://i.imgur.com/POxDSgE.jpg

 

And metadata settings: 

http://i.imgur.com/GQFXIR8.png

 

 

 

What does flaky mean? As a test I just set my preferred language to Danish and then refreshed a title. Notice the Danish genres and overview.

 

[Luke 42077]

When did you configure that?

[famulor 1]

When did you configure that?

Yesterday