Any way to quantify what causes a long library scan?

[podonnell 51]

I've recently been catching my library scans running, which I never have in the past. When I do. I watch to see how long it takes to complete. Currently I've had one running for at least 2 minutes prior to me starting this thread, and it is still be running when I finished typing this message.

I did check to confirm there is only a single stream happening, the CPU of the host machine is about as low as it normally is, and there are no current transcodes.

Curious if there is any way to quantify what might be causing the longer library scans as of late. Is there any way that this can easily be done?

Thanks!

[Luke 43172]

Hi, the log is the best way to see what it's doing. 

Library scan times will depend on what has changed since the last scan, what features you've enabled, what plugins you've installed, and what else might be consuming resources on the server at that time.

[bkettering 2]

Adding to what Luke said, the quick way to see it without reading the whole log: open Dashboard, then Scheduled Tasks while a scan is running, and look at which task is actually active and its progress. In my experience the scan itself is quick and the long part is the add-ons that run with it, like chapter image or thumbnail extraction and intro detection, especially right after an update when they reprocess things. Turning those off for a day is an easy test.

[podonnell 51]

3 hours ago, bkettering said:

Adding to what Luke said, the quick way to see it without reading the whole log: open Dashboard, then Scheduled Tasks while a scan is running, and look at which task is actually active and its progress. In my experience the scan itself is quick and the long part is the add-ons that run with it, like chapter image or thumbnail extraction and intro detection, especially right after an update when they reprocess things. Turning those off for a day is an easy test.

Oh that is very helpful, thank you. Haven't visited this section of the Dashboard in a bit.

So I manually ran:
Scan media library

It had said the last time it ran it took over two minutes, but this manual run was very fast:
Last ran 1 second ago, taking 12 seconds.

 

Would there be anything different on how that happens? When I caught it running for the few minutes the other day, I did not think my library had changed all that much -- so I was confused why it was running so long. It does make sense if I had a lot of changes in the library.

Anyway, while it was running it didn't show anything other than it was running.
It looks like the other functions you mention are their own scheduled tasks, and the media library scan runs on its own.

[Lessaj 579]

Intro detection and video thumbnails can also run during library scans, it just depends what settings you're using on your libraries. It's not uncommon for scans to take a long time if for example you have it set to refresh metadata every so often.

[podonnell 51]

I just caught another automated one kicking off, and this one went way longer than my manual one. Not crazy long, but 3 minutes and 20 seconds compared to my manual run which took seconds.

It sat on 53% for about two minutes alone. In the log at the moment was:
fmpegManager: ProcessRun 'CreateTitleFingerprint' Process exited with code 0

It looks like in general it is going through the audio fingerprinting process, which is intro detection I believe? bkettering mentioned this could be the cause, but I did not think that was running since it is its own section in scheduled tasks (below), but it seems like the automated library scan includes this, and perhaps my manual one does not? That would explain the longer automated library scans if so.

[marriedman 111]

Does Windows have a comparable command to the Linux utility tail ? Someone on here told me about that and I use it ALL the time! Very informative.

[Uncle_Frank 12]

I have mine set to everything under Scheduled Tasks is off it still loads meta after each movie gets added but it don't wake you up from hard drives grinding at 4am scanning for subtitles. I have hard drives in the room behind the tv screwed to a sheet of plywood with a fan running can't see anything but the 100inch monitor / tv screen hanging on the wall and with settings off you can't hear nothing either. It use to scan for subtitles like crazy before shutting it all off.

[bkettering 2]

yep, CreateTitleFingerprint is the audio fingerprinting intro detection uses, so that's your 2 minutes. my guess on the manual vs automatic difference is that the manual run just had nothing new to fingerprint, since the scan only does that work for new or changed episodes. if you'd rather keep scans quick, check your TV library settings for the intro detection option and see if you can have it skip the library scan and just let the scheduled task handle it overnight instead.
@marriedman on windows, powershell does the same thing as tail: Get-Content "C:\path\to\embyserver.txt" -Wait -Tail 50. swap in wherever your emby logs folder is and it'll stream new lines as they're written

[Lessaj 579]

What settings do you have set on your libraries? Specifically what is set for the following options:

"After the initial import, automatically refresh metadata from the internet"
"Automatically refresh metadata for episodes with placeholder titles such as TBA"
"Generate video preview thumbnails"
"Generate intro video markets"

[podonnell 51]

20 minutes ago, Lessaj said:

What settings do you have set on your libraries? Specifically what is set for the following options:

"After the initial import, automatically refresh metadata from the internet"
"Automatically refresh metadata for episodes with placeholder titles such as TBA"
"Generate video preview thumbnails"
"Generate intro video markets"

I do have most of these enables on my big library, so I think this does add up.

[Lessaj 579]

3 hours ago, podonnell said:

I do have most of these enables on my big library, so I think this does add up.

I was asking for specifics.  Because there are options for "as a scheduled task" or "as a scheduled task and when media is added" for example. In the case of metadata refresh (I have mine set for 30 days) if you for example added a whole show of 100 episodes, then at that 30 day mark it's going to take some extra time to make the API calls to refresh the metadata during the library scan. I see this a lot with my music since I add in bulk.