Excessive library scan duration - reason unclear

[sscheib 11]

Hello community,

 

since two stable versions I am experiencing excessive library scan durations. To give you an idea, what I mean by excessive:

Before the issues started to appear: ~1h scan duration

At the moment: >8h scan duration

 

The content has not significantly been increased however, that is why I was wondering what's going on.

I took a look at the log file and quickly found the issue why my library scan seems to take that much time (for obvious reasons I censored both the URL and the path to the files, but they are different! ):

2019-09-04 17:19:52.467 Info MediaEncoder: ProcessRun 'ffprobe' Started.
2019-09-04 17:19:54.275 Info MediaEncoder: ProcessRun 'ffprobe' Process exited with code 0
2019-09-04 17:19:54.331 Info MediaEncoder: ProcessRun 'ffprobe' Execute: /opt/emby-server/bin/ffprobe -i file:"/some/file/here" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2019-09-04 17:19:54.406 Info MediaEncoder: ProcessRun 'ffprobe' Started.
2019-09-04 17:19:55.394 Info MediaEncoder: ProcessRun 'ffprobe' Process exited with code 0
2019-09-04 17:19:55.431 Info MediaEncoder: ProcessRun 'ffprobe' Execute: /opt/emby-server/bin/ffprobe -i file:"/some/file/here" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2019-09-04 17:19:55.522 Info MediaEncoder: ProcessRun 'ffprobe' Started.
2019-09-04 17:19:56.523 Info MediaEncoder: ProcessRun 'ffprobe' Process exited with code 0
2019-09-04 17:19:56.557 Info MediaEncoder: ProcessRun 'ffprobe' Execute: /opt/emby-server/bin/ffprobe -i file:"/some/file/here" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2019-09-04 17:19:56.623 Info MediaEncoder: ProcessRun 'ffprobe' Started.
2019-09-04 17:19:58.056 Info MediaEncoder: ProcessRun 'ffprobe' Process exited with code 0
2019-09-04 17:19:58.091 Info MediaEncoder: ProcessRun 'ffprobe' Execute: /opt/emby-server/bin/ffprobe -i file:"/some/file/here" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2019-09-04 17:19:58.172 Info MediaEncoder: ProcessRun 'ffprobe' Started.
2019-09-04 17:20:02.403 Info HttpServer: HTTP POST http://my.tld:8096/emby/Sessions/Playing/Progress. UserAgent: Mozilla/5.0 (Web0S; Linux/SmartTV) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.34 Safari/537.36 WebAppManager
2019-09-04 17:20:02.406 Info HttpServer: HTTP Response 204 to 46.5.77.71. Time: 3ms. http://my.tld:8096/emby/Sessions/Playing/Progress
2019-09-04 17:20:02.778 Info MediaEncoder: ProcessRun 'ffprobe' Process exited with code 0
2019-09-04 17:20:02.817 Info MediaEncoder: ProcessRun 'ffprobe' Execute: /opt/emby-server/bin/ffprobe -i file:"/some/file/here" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2019-09-04 17:20:02.902 Info MediaEncoder: ProcessRun 'ffprobe' Started.
2019-09-04 17:20:08.615 Info MediaEncoder: ProcessRun 'ffprobe' Process exited with code 0
2019-09-04 17:20:08.656 Info MediaEncoder: ProcessRun 'ffprobe' Execute: /opt/emby-server/bin/ffprobe -i file:"/some/file/here" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2019-09-04 17:20:08.727 Info MediaEncoder: ProcessRun 'ffprobe' Started.
2019-09-04 17:20:10.245 Info MediaEncoder: ProcessRun 'ffprobe' Process exited with code 0
2019-09-04 17:20:10.286 Info MediaEncoder: ProcessRun 'ffprobe' Execute: /opt/emby-server/bin/ffprobe -i file:"/some/file/here" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2019-09-04 17:20:10.376 Info MediaEncoder: ProcessRun 'ffprobe' Started.
2019-09-04 17:20:11.645 Info MediaEncoder: ProcessRun 'ffprobe' Process exited with code 0
2019-09-04 17:20:11.684 Info MediaEncoder: ProcessRun 'ffprobe' Execute: /opt/emby-server/bin/ffprobe -i file:"/some/file/here" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2019-09-04 17:20:11.755 Info MediaEncoder: ProcessRun 'ffprobe' Started.
2019-09-04 17:20:12.394 Info HttpServer: HTTP POST http://my.tld:8096/emby/Sessions/Playing/Progress. UserAgent: Mozilla/5.0 (Web0S; Linux/SmartTV) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.34 Safari/537.36 WebAppManager

So I went ahead and tried to figure out, why exactly emby is calling ffprobe for basically every media file I have, but I couldn't find a reason for it.

- 'Create thumbnail images during the library scan' is not checked in the library settings (however I have checked 'Enable thumbnail image creation', which, however, should (according to the help text) independently run from the library scan with its own scheduled task. Is this still correct?

- I unticked 'Screen Grabber' as 'Episode Image Fetchers'

 

Is there anything else I could do?

I don't see a reason why ffprobe should run during a library scan (but maybe I am simply not understanding what a library scan in emby does in the first place).

 

For me it seems pretty clear, that it is slowing my library scan massively, which it didn't do in previous versions.

 

 

Software information:

Emby 4.2.1.0 on latest Debian oldstable (Stretch)

 

Thanks!

 

 

[Luke 37360]

Hi there, please attach the emby server log. Thanks.

[sscheib 11]

Hello Luke,

 

please find the logs attached.

emby_logs.tar.gz

Edited September 4, 2019 by sscheib

[Luke 37360]

Hi, this is a whole folder full of log files. Which one is from the time frame that the problem was experienced?

[sscheib 11]

Hi, embyserver.txt (which is the latest). These are the logs from 1 day only however, so basically all of them are relevant. Just look for the mentioned line within the log files (MediaEncoder: ProcessRun 'ffprobe' Started.)

Edited September 5, 2019 by sscheib

[Luke 37360]

Ok, there's nothing obvious here. You could take a look at this thread where there are similar reports:

https://emby.media/community/index.php?/topic/75628-tv-shows-library-scan-takes-longer-with-newer-beta-servers/

 

Although now they're saying it's fixed and I didn't make any changes for it.

[sscheib 11]

well, from what I could see, they talked about version 4.3.x (beta) - I am on stable right now.

can you estimate when 4.3.x will become stable? I don't want to switch to beta, as I had a lot of issues earlier with the betas

[Luke 37360]

I don't have an ETA at this time, sorry, but it should be similar to our typical release intervals.