Emby Slow Scan Recently

[MrJohnW1ck 1]

Hi Team,

Recently Library scan is taking 40 mins before it was faster,

is there anything i can do to speedup the library faster.

Attaching the Log for your reference.

 

embyserver (3).txt

[Neminem 807]

It looks like TheMovieDB is timing out.

It might be a TheMovieDB issue that needs to be fixed by TheMovieDB.

Timeouts like these will give you longer scan times.

Error in TheMovieDb    *** Error Report ***
    Version: 4.8.11.0
    Command line: /opt/emby-server/system/EmbyServer.dll -programdata /var/lib/emby -ffdetect /opt/emby-server/bin/ffdetect -ffmpeg /opt/emby-server/bin/ffmpeg -ffprobe /opt/emby-server/bin/ffprobe -restartexitcode 3 -updatepackage emby-server-deb_{version}_arm64.deb
    Operating system: Linux version 6.8.0-1024-oracle (buildd@bos03-arm64-040) (aarch64-linux-gnu-gcc-13 (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0, GNU ld (GNU Binutils for Ubu
    Framework: .NET 6.0.36
    OS/Process: arm64/arm64
    Runtime: opt/emby-server/system/System.Private.CoreLib.dll
    Processor count: 4
    Data path: /var/lib/emby
    Application path: /opt/emby-server/system
    MediaBrowser.Model.Net.HttpException: MediaBrowser.Model.Net.HttpException: The SSL connection could not be established, see inner exception.
     ---> System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
     ---> System.IO.IOException: Unable to read data from the transport connection: Connection reset by peer.
     ---> System.Net.Sockets.SocketException (104): Connection reset by peer
       --- End of inner exception stack trace ---
       at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
       at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
       at System.Net.Security.SslStream.<FillHandshakeBufferAsync>g__InternalFillHandshakeBufferAsync|189_0[TIOAdapter](TIOAdapter adap, ValueTask`1 task, Int32 minSize)
       at System.Net.Security.SslStream.ReceiveBlobAsync[TIOAdapter](TIOAdapter adapter)
       at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean isApm)
       at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)
       --- End of inner exception stack trace ---
       at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)
       at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
       at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
       at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage request)
       at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
       at System.Net.Http.HttpConnectionPool.GetHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
       at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
       at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
       at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
       at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
       at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
       at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsyncInternal(HttpRequestOptions options, String httpMethod)
       --- End of inner exception stack trace ---
       at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsyncInternal(HttpRequestOptions options, String httpMethod)
       at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsync(HttpRequestOptions options, String httpMethod)
       at MovieDb.MovieDbProviderBase.GetMovieDbResponse(HttpRequestOptions options)
       at MovieDb.MovieDbSeriesProvider.FetchMainResult(String id, String language, CancellationToken cancellationToken)
       at MovieDb.MovieDbSeriesProvider.DownloadSeriesInfo(String id, String preferredMetadataLanguage, CancellationToken cancellationToken)
       at MovieDb.MovieDbSeriesProvider.GetMetadata(SeriesInfo info, CancellationToken cancellationToken)
       at Emby.Providers.Manager.MetadataService`2.ExecuteRemoteProviders(MetadataResult`1 temp, LibraryOptions libraryOptions, String logName, TIdType id, IRemoteMetadataProvider`2[] providers, MetadataRefreshOptions options, CancellationToken cancellationToken)
    Source: Emby.Server.Implementations
    TargetSite: Void MoveNext()
    InnerException: System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
    Source: System.Net.Http
    TargetSite: Void MoveNext()
       at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)
       at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
       at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
       at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage request)
       at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
       at System.Net.Http.HttpConnectionPool.GetHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
       at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
       at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
       at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
       at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
       at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
       at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsyncInternal(HttpRequestOptions options, String httpMethod)
    InnerException: System.IO.IOException: Unable to read data from the transport connection: Connection reset by peer.
    Source: System.Net.Sockets
    TargetSite: Void ThrowException(System.Net.Sockets.SocketError, System.Threading.CancellationToken)
       at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
       at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
       at System.Net.Security.SslStream.<FillHandshakeBufferAsync>g__InternalFillHandshakeBufferAsync|189_0[TIOAdapter](TIOAdapter adap, ValueTask`1 task, Int32 minSize)
       at System.Net.Security.SslStream.ReceiveBlobAsync[TIOAdapter](TIOAdapter adapter)
       at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean isApm)
       at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)
    InnerException: System.Net.Sockets.SocketException: Connection reset by peer
    Source: 
    TargetSite: 
    

[MrJohnW1ck 1]

31 minutes ago, Neminem said:

TheMovieDB is timing out

So if i disable the following plugin will it be faster

[Neminem 807]

I think so but then you might miss some metadata form TheMovieDB when its working again.

To be honest I don't see this issue with them when importing movies.

Do you have something blocking.

AV, firewall, VPN or the like ?  

[MrJohnW1ck 1]

Hi @NeminemVPN is turned on i just checked, i didn't change AV or firewall as this mounted in a server

Should i turn off the vpn and try once?

[MrJohnW1ck 1]

its happening to mostly for the TV shows part instead of Movies and im stuck here for 2hrs

embyserver (4).txt

[MrJohnW1ck 1]

Hi The reason why im highlighting this is because VPN was on from the start of Emby Server Creation it was taking media library scan very less time like 8 mins to 10mins if any new media added it takes 15mins to complete but now it takes 2hrs plus even if nothinng is added

[Neminem 807]

29 minutes ago, MrJohnW1ck said:

Should i turn off the vpn and try once?

Yes try that, sometimes VPN providers ip get blacklisted and can't access.

[MrJohnW1ck 1]

Hi The reason why im highlighting this is because VPN was on from the start of Emby Server Creation it was taking media library scan very less time like 8 mins to 10mins if any new media added it takes 15mins to complete but now it takes 2hrs plus even if nothinng is added

[MrJohnW1ck 1]

Plus i just checked with server side there's no VPN installed i can confirm that

[Neminem 807]

I'm not sure why you are having all these errors.

Devs will have to answer.

MediaBrowser.Model.Net.HttpException: MediaBrowser.Model.Net.HttpException: The SSL connection could not be established, see inner exception.
     ---> System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
     ---> System.IO.IOException: Unable to read data from the transport connection: Connection reset by peer.
     ---> System.Net.Sockets.SocketException (104): Connection reset by peer

It looks like you are blocking or you are being blocked

[Neminem 807]

It also looks like you have issues with your google drive, that will also add to prolonged scan times.

ProcessRun 'quick-extract-image' Error Output:    ffmpeg version 5.1-emby_2023_06_25 Copyright (c) 2000-2022 the FFmpeg developers and softworkz for Emby LLC
      built with gcc 10.3.0 (crosstool-NG 1.25.0)
    Execution Date: 2025-05-10 10:29:55
    -vsync is deprecated. Use -fps_mode
    Passing a number to -vsync is deprecated, use a string argument as described in the manual.
    [libdav1d @ 0x16cc4800] libdav1d 1.0.0
    Input #0, matroska,webm, from 'file:/usr/share/google-drive-share/Animation , Cartoons/Series/Mr.Bean Animated Series/Mr.Bean.The.Animated.Series.S02.1080p.NF.WEB-DL.AAC2.0.AV1-Saon/Mr.Bean.The.Animated.Series.S02E34.Valuable.Lessons.1080p.NF.WEB-DL.AAC2.0.AV1-Saon.mkv':
      Metadata:
        encoder         : libebml v1.4.2 + libmatroska v1.6.4
      Duration: 00:11:01.89, start: 0.000000, bitrate: 1062 kb/s
      Stream #0:0: Video: av1 (Main), yuv420p10le(tv), 1920x1080, SAR 1:1 DAR 16:9, Level 8, 25 fps, 25 tbr, 1k tbn (default)
        Metadata:
          BPS             : 932470
          DURATION        : 00:11:01.800000000
          NUMBER_OF_FRAMES: 16545
          NUMBER_OF_BYTES : 77138653
      Stream #0:1(eng): Audio: aac (HE-AAC), 48000 Hz, stereo, fltp (default)
        Metadata:
          title           : English [HE-AAC 2.0]
          BPS             : 127999
          DURATION        : 00:11:01.888000000
          NUMBER_OF_FRAMES: 15513
          NUMBER_OF_BYTES : 10590207
      Stream #0:2(eng): Subtitle: subrip (hearing impaired)
        Metadata:
          title           : English [SDH]
          BPS             : 52
          DURATION        : 00:10:59.160000000
          NUMBER_OF_FRAMES: 222
          NUMBER_OF_BYTES : 4314
    [libdav1d @ 0x16cd0d40] libdav1d 1.0.0
    Stream mapping:
      Stream #0:0 -> #0:0 (av1 (libdav1d) -> mjpeg (native))
    Press [q] to stop, [?] for help
    [swscaler @ 0x16e095b0] [swscaler @ 0x16e16ca0] deprecated pixel format used, make sure you did set range correctly
    [swscaler @ 0x16e095b0] [swscaler @ 0x16e5d7d0] deprecated pixel format used, make sure you did set range correctly
    [swscaler @ 0x16e095b0] [swscaler @ 0x16ea40f0] deprecated pixel format used, make sur
 

[MrJohnW1ck 1]

Any idea how to fix these cuz i m new to this

[MrJohnW1ck 1]

I removed the movieDB and will try restarting the server instance completely if it fixes illl keep this forum updated

[Neminem 807]

[MrJohnW1ck 1]

Can it be cause my database cache size is limited to 16Gigs.

also can anyone clear me if database cache is gonna be in terms of storage or RAM

[Neminem 807]

I do not thing those settings will help you with all the errors in your logs.

Look through you logs in a text editor, and solve 1 at a time.

[MrJohnW1ck 1]

Which Text editor are u usinng as urs shows sources and etc

[Neminem 807]

Notepad++ and a set of filters.

Or 

Scarfsail/AdvancedLogViewer: Advanced Log Viewer also with filters.

Look for "log Level" = Error

Or Just search for Error "ctrl+f"

Edited Saturday at 03:36 PM by Neminem

[MrJohnW1ck 1]

Thanks most part it s showing movie db socket error somekind

[Neminem 807]

Yes and I think Emby is waiting to get the timeout before trying again.

That will lead to a lot of timeouts and longer scan times.

So you will need to fix the socket errors and SSL errors you have in your logs.

I don't see these errors in my logs, so that rules out metadata end points.

So I guess its local to your system.

Be it Router / Firewall / DNS / ISP or something is blocking it. 

[Happy2Play 9384]

3 hours ago, MrJohnW1ck said:

Hi The reason why im highlighting this is because VPN was on from the start of Emby Server Creation it was taking media library scan very less time like 8 mins to 10mins if any new media added it takes 15mins to complete but now it takes 2hrs plus even if nothing is added

But that will have absolutely nothing to do with Emby as provider sites control their api access and sometimes as mentioned VPN users abuse the system and they fight back by blacklisting said IP address/ranges.  So yes all the TMDB queries will slow down the scan process.  Technically every provider enabled can/will.

Does TMDB work without your VPN?

 

[MrJohnW1ck 1]

its on a server and there s no VPN activated in there but it takes 2hrs sometimes to scan the media, i have 35tb worth of data

[MrJohnW1ck 1]

I dont have ssl activated for my domain, and im noticing this delay in scan media after i imported Friends DV 2160p Hevc Remux worth 2TB

[MrJohnW1ck 1]

Repeatedly im seeing only TheMovieDB socket Error is there any fix for that