ffprobe timeout error - library scan

[IkeTaylor11 60]

I built a new Emby server on Ubuntu 18.04. My libraries are coming from a webdav directory mounted on the server at boot. When I do a library scan, ffprobe starts, gives an error in the emby server log and will not advance any further. It gets stuck at 86.4% This is the error i'm seeing in the emby log. 

 

2019-12-03 14:49:00.307 Error MediaProbeManager: ProcessRun 'ffprobe' Error Output:
    ffprobe version 4.3.0-emby_2019_11_09 Copyright © 2007-2019 the FFmpeg developers and softworkz for Emby LLC
     built with gcc 8.3.0 (crosstool-NG 1.24.0)
2019-12-03 14:49:00.319 Error App: Error in ffprobe
    *** Error Report ***
    Version: 4.3.0.30
    Command line: /opt/emby-server/system/EmbyServer.dll -programdata /var/lib/emby -ffdetect /opt/emby-server/bin/ffdetect -ffmpeg /opt/emby-server/bin/ffmpeg -ffprobe /opt/emby-server/bin/ffprobe -restartexitcode 3 -updatepackage emby-server-deb_{version}_amd64.deb
    Operating system: Unix 4.15.0.29
    64-Bit OS: True
    64-Bit Process: True
    User Interactive: True
    Runtime: file:///opt/emby-server/system/System.Private.CoreLib.dll
    Processor count: 2
    Program data path: /var/lib/emby
    Application directory: /opt/emby-server/system
    System.Exception: System.Exception: Error running ffprobe for file:"/mnt/emby/Movies/Aquaman (2018)/Aquaman (2018).mkv" -
     at Emby.Server.MediaEncoding.Probing.MediaProbeManager.GetMediaInfoInternal(String inputPath, String primaryPath, MediaProtocol protocol, Boolean extractChapters, String probeSizeArgument, Boolean isAudio, Boolean forceEnableLogging, CancellationToken cancellationToken)
     at MediaBrowser.Providers.MediaInfo.FFProbeVideoInfo.ProbeVideo[T](T item, MetadataRefreshOptions options, LibraryOptions libraryOptions, CancellationToken cancellationToken)
     at MediaBrowser.Providers.Manager.MetadataService`2.RunCustomProvider(ICustomMetadataProvider`1 provider, TItemType item, String logName, MetadataRefreshOptions options, RefreshResult refreshResult, CancellationToken cancellationToken)
    Source: Emby.Server.MediaEncoding
    TargetSite: Void MoveNext()

[Luke 37054]

Hi there, please attach the emby server log. thanks.

[IkeTaylor11 60]

Log attached

emby_log_12_3_19.log

[Luke 37054]

Strange. The log suggests that either the server doesn't have permission to read the contents of the files, or there is some kind of I/O problem.

[IkeTaylor11 60]

These are the permissions that are set for the emby user. 

 

root@ikemedia:~# sudo -u emby ls -al /mnt/emby/

total 17

drwxr-xr-x   9 root root   288 Dec  3 09:47  .

drwxr-xr-x   2 root root     0 Nov 27 13:07 'Camera Uploads'

drwxr-xr-x   2 root root     0 Dec  3 11:06  cloudflare

drwx------   2 root root     0 Dec  3 14:44  lost+found

drwxrwxrwx 266 root root 13360 Dec  1 12:40  Movies

drwxr-xr-x   2 root root     0 Dec  2  2018  Music

drwxr-xr-x  58 root root  2784 Dec  3 12:45 'TV Series'

drwxr-xr-x  25 root root     0 Nov 27 21:33  UFC

root@ikemedia:~# groups emby

emby : emby sudo video davfs2

root@ikemedia:~# sudo -u emby ls -al /var/lib/emby/

total 48

drwxr-xr-x  9 emby emby 4096 Dec  3 11:18 .

drwxr-xr-x 32 root root 4096 Dec  3 10:59 ..

-rw-r--r--  1 emby emby  220 Aug 31  2015 .bash_logout

-rw-r--r--  1 emby emby 3771 Aug 31  2015 .bashrc

drwxr-xr-x 10 emby emby 4096 Dec  3 14:43 cache

drwxr-xr-x  4 emby emby 4096 Dec  3 12:23 config

drwxr-xr-x  4 emby emby 4096 Dec  3 18:25 data

drwxr-xr-x  2 emby emby 4096 Dec  3 18:25 logs

drwxr-xr-x  4 emby emby 4096 Dec  3 14:33 metadata

drwxr-xr-x  4 emby emby 4096 Dec  3 12:33 plugins

-rw-r--r--  1 emby emby  807 Apr  4  2018 .profile

drwxr-xr-x  3 emby emby 4096 Dec  3 10:59 root

root@ikemedia:~# sudo -u emby ls -al /opt/emby-server/

total 60

drwxrwxrwx  8 root root  4096 Dec  3 10:59 .

drwxr-xr-x  3 root root  4096 Dec  3 10:59 ..

drwxrwxrwx  2 root root  4096 Dec  3 10:59 bin

drwxrwxrwx  4 root root  4096 Dec  3 10:59 etc

drwxrwxrwx  4 root root 16384 Dec  3 10:59 lib

drwxrwxrwx 30 root root  4096 Dec  3 10:59 licenses

drwxrwxrwx  3 root root  4096 Dec  3 10:59 share

drwxrwxrwx  5 root root 20480 Dec  3 10:59 system

root@ikemedia:~#

 

 

I have uploaded another emby log. It looks like ffprobe is erroring even when not scanning the library. 

 

 

emby_log_12_3_19_log_2.log

[Luke 37054]

Why do you feel that way?

[IkeTaylor11 60]

Why do you feel that way?

jeez, do you always have to ask that? Can't you just analyze the logs and the info I have provided and tell me what to do next to try and resolve this? 

[Luke 37054]

I asked that because I looked in the log and saw no evidence of what you were saying. And I also asked because you didn't indicate what led you into thinking that.

[IkeTaylor11 60]

I asked that because I looked in the log and saw no evidence of what you were saying. And I also asked because you didn't indicate what led you into thinking that.

saw this in the log and it said ffmpeg and ffprobe, so i thought that was an issue

 

2019-12-03 18:25:16.011 Debug FfmpegManager: ProcessRun 'ffmpeg -hide_banner -version' Started.

2019-12-03 18:25:16.024 Error TaskManager: Error

*** Error Report ***

Version: 4.3.0.30

Command line: /opt/emby-server/system/EmbyServer.dll -programdata /var/lib/emby -ffdetect /opt/emby-server/bin/ffdetect -ffmpeg /opt/emby-server/bin/ffmpeg -ffprobe /opt/emby-server/bin/ffprobe -restartexitcode 3 -updatepackage emby-server-deb_{version}_amd64.deb

Operating system: Unix 4.15.0.29

64-Bit OS: True

64-Bit Process: True

User Interactive: True

Runtime: file:///opt/emby-server/system/System.Private.CoreLib.dll

Processor count: 2

Program data path: /var/lib/emby

Application directory: /opt/emby-server/system

System.Net.Http.HttpRequestException: System.Net.Http.HttpRequestException: Resource temporarily unavailable ---> System.Net.Sockets.SocketException: Resource temporarily unavailable

   at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)

   --- End of inner exception stack trace ---

   at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsyncInternal(HttpRequestOptions options, String httpMethod)

   at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsync(HttpRequestOptions options, String httpMethod)

   at Emby.Server.Implementations.Updates.InstallationManager.GetAvailablePackagesWithoutRegistrationInfo(Boolean enableCache, CancellationToken cancellationToken)

   at Emby.Server.Implementations.Updates.InstallationManager.GetAvailablePluginUpdates(Version applicationVersion, CancellationToken cancellationToken)

   at Emby.Server.Implementations.ScheduledTasks.PluginUpdateTask.Execute(CancellationToken cancellationToken, IProgress`1 progress)

   at Emby.Server.Implementations.ScheduledTasks.ScheduledTaskWorker.ExecuteInternal(TaskOptions options)

Source: System.Net.Http

TargetSite: Void MoveNext()

InnerException: System.Net.Sockets.SocketException: Resource temporarily unavailable

Source: System.Private.CoreLib

TargetSite: Void Throw()

   at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)

[Luke 37054]

That's a timeout related to checking for available plugin updates.

[IkeTaylor11 60]

@@Luke what is involved with a library scan? I removed my other libraries and added a smaller one to test with. It is 137GB and has 23 movies in it. The library scan has been running for 10 hours and is at 91%. I am seeing lots of write I/O on my server during the scanning process. The library I am adding is already a library on a different emby server so all the images and nfo files are already in the library folders. My connection to the library is over the web and only has a 20Mbps connection. the scan is pegging that bandwidth since the scan started. 

[Luke 37054]

Depends on what features you've enabled. We'd have to look at the server log. Thanks.

[IkeTaylor11 60]

Depends on what features you've enabled. We'd have to look at the server log. Thanks.

Library settings and emby log during the scan last night. 

library_settings.zip

emby_log_12_04_19.log

[Luke 37054]

For faster library scans, consider disabling the thumbnail extraction features for your libraries.

[IkeTaylor11 60]

For faster library scans, consider disabling the thumbnail extraction features for your libraries.

I thought thumbnail extraction was a separate task? Are you saying this happens with the library scan as well? Why is there 2 task in Emby then?

 

 

Sent from my iPhone using Tapatalk

[Luke 37054]

I thought thumbnail extraction was a separate task? Are you saying this happens with the library scan as well? Why is there 2 task in Emby then?

 

 

Sent from my iPhone using Tapatalk

 

Based on the settings for each library, the server is doing exactly what you told it to do.

[IkeTaylor11 60]

Based on the settings for each library, the server is doing exactly what you told it to do.

I don’t doubt that Emby is doing what I’m asking it to. The issue is the ffprobe timing out and taking forever to complete a scan, even on a small library.

 

Also the amount of I/O and data being generated during the scan seems excessive. Especially since all the thumbnails and nfo files exist.

 

 

Sent from my iPhone using Tapatalk

[Luke 37054]

Right but you were warned about this when you enabled the feature, and you were given a choice of when to have it run.

[IkeTaylor11 60]

Right but you were warned about this when you enabled the feature, and you were given a choice of when to have it run.

 

Thumbnail extraction is a separate task. It shouldn’t take place during a library scan. If it’s in a library scan what’s the point of having a separate task for it?

 

The extraction task fails as well btw, gives up after 4 hours.

 

 

Sent from my iPhone using Tapatalk

[Luke 37054]

 

 

Thumbnail extraction is a separate task. It shouldn’t take place during a library scan.

 

What happens when other users disagree? For example, they want their thumbnails available immediately when the media is added.

 

That's why I've been asking if you explored your settings for each library. Have you done that? Because as part of this troubleshooting I'm looking to find out what prevented you from discovering the options so that we can figure out what to adjust so that this doesn't come up again.

[IkeTaylor11 60]

What happens when other users disagree? For example, they want their thumbnails available immediately when the media is added.

 

That's why I've been asking if you explored your settings for each library. Have you done that? Because as part of this troubleshooting I'm looking to find out what prevented you from discovering the options so that we can figure out what to adjust so that this doesn't come up again.

I was asking if thumbnail creation was part of the scan library task, you haven’t given an answer.

 

I don’t understand why there would be 2 tasks if thumbnail creation is included in scan library.

 

I have disabled thumbnail creation on the library and nothing has changed. Ffprobe still times out on each media file it tries to scan in the library.

 

 

Sent from my iPhone using Tapatalk

[Luke 37054]

 

 

I was asking if thumbnail creation was part of the scan library task, you haven’t given an answer.

 

I've been trying to answer that this in your control in library settings. Sorry, I should have just come out and said it sooner, but i was trying to get a feel for why you didn't know about this.

[IkeTaylor11 60]

I've been trying to answer that this in your control in library settings. Sorry, I should have just come out and said it sooner, but i was trying to get a feel for why you didn't know about this.

So your saying the any time a library scan runs it does a thumbnail creation task as well? Why is there a separate task for that very thing then? It’s redundant and misleading.

 

Also, as I said, I disabled the thumbnail creation on the library and it still has timeout errors and takes forever. Over 2 hours per file for it to move on to the next one.

 

 

Sent from my iPhone using Tapatalk

[Luke 37054]

 

 

I have disabled thumbnail creation on the library and nothing has changed. Ffprobe still times out on each media file it tries to scan in the library.

 

We'd have to look at an example. It could be that it was already in progress at the time you disabled it.

[Luke 37054]

 

 

So your saying the any time a library scan runs it does a thumbnail creation task as well? Why is there a separate task for that very thing then? 

 

I'm saying it's however you want it to be.