Jump to content
Karl Blixt

ANSWERED Emby server crashes when using search.

Recommended Posts

Karl Blixt

I just upgraded to 3.5.2.0 on my ubuntu 16.04 machine and wanted to check out if the new search feature that finds original title as well as the title was included. and to my delight when searching for "shaw" it very correctly found the movie shawshank redemption even though I've titled it "nyckeln till friheten" which is the swedish title for the movie.

 

However, I quickly found out that the server had crashed for some reason. By the looks of it it crashed after the search because but while fetching the posters because some of the posters for the results shows up while others are just unloaded black.

 

I've done some testing and I've found that the server dosen't crash as long as you search for movies that have the same title as original title. I'm not sure if it has to be exactly the same or just simmilar enough to not cause a crash for some reason. anyway, whenever the search result contains a movie with an altarantive title the server crashes.

 

I'll attach the logs of a session that experienced a crash but when looking into the server loggs there are a few entries called "unhandled_[hex based hash i presume].txt with the following exception:

 

System.IO.IOException: Unable to read data from the transport connection: Connection reset by peer. ---> System.Net.Sockets.SocketException: Connection reset by peer

--- End of inner exception stack trace ---
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request)
at System.Net.Security.SslState.ThrowIfExceptional()
at System.Net.Security.SslState.CheckThrow(Boolean authSuccessCheck, Boolean shutdownCheck)
at System.Net.Security.SslState.CheckOldKeyDecryptedData(Memory`1 buffer)
at System.Net.Security.SslState.HandleQueuedCallback(Object& queuedStateRequest)
at System.Net.Security.SslState.FinishHandshakeRead(Int32 newState)
at System.Net.Security.SslState.FinishHandshake(Exception e, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.RehandshakeCompleteCallback(IAsyncResult result)
at System.Net.LazyAsyncResult.Complete(IntPtr userToken)
at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken)
at System.Net.AsyncProtocolRequest.CompleteUserWithError(Exception e)
at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
at System.Threading.ThreadPoolWorkQueue.Dispatch()
System.IO.IOException
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request)
at System.Net.Security.SslState.ThrowIfExceptional()
at System.Net.Security.SslState.CheckThrow(Boolean authSuccessCheck, Boolean shutdownCheck)
at System.Net.Security.SslState.CheckOldKeyDecryptedData(Memory`1 buffer)
at System.Net.Security.SslState.HandleQueuedCallback(Object& queuedStateRequest)
at System.Net.Security.SslState.FinishHandshakeRead(Int32 newState)
at System.Net.Security.SslState.FinishHandshake(Exception e, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.RehandshakeCompleteCallback(IAsyncResult result)
at System.Net.LazyAsyncResult.Complete(IntPtr userToken)
at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken)
at System.Net.AsyncProtocolRequest.CompleteUserWithError(Exception e)
at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
at System.Threading.ThreadPoolWorkQueue.Dispatch()
InnerException: System.Net.Sockets.SocketException

System.Net.Sockets.SocketException (104): Connection reset by peer 

embyserver-63668706532.txt

Share this post


Link to post
Share on other sites
Luke

It's because you still have audiodb urls saved in your database unfortunately. audiodb has been removed in 3.5.2 but since you already have urls saved, what you'll need to do is remove your music library, run a scan, and then add it back.

  • Like 1

Share this post


Link to post
Share on other sites
Karl Blixt

Thank you, this resolved my issue.

Share this post


Link to post
Share on other sites
Luke

Thanks for the feedback.

Share this post


Link to post
Share on other sites
letterman

I have the same problem.

 

Is removing and re-adding the ONLY solution???

Everything worked very well till 3.4.2

 

It is incredible that you change core engines in that deep way, without having a working library update tool.

 

Deleting and re-adding all libraries takes about 2 weeks and what about my collections??? The content is linked to the library and it will be lost after deleting and re-adding.

 

sorry, your suggestion is not satisfying.

Edited by letterman

Share this post


Link to post
Share on other sites
Luke

I have the same problem.

 

Is removing and re-adding the ONLY solution???

I have 39(!) libraries and everything worked very well till 3.4.2

 

It is incredible that you change core engines in that deep way, without having a working library update tool.

 

Deleting and re-adding all libraries takes about 2 weeks and what about my collections??? The content is linked to the library and it will be lost after deleting and re-adding.

 

sorry, your suggestion is not satisfying.

 

Can you please attach the emby server log? you can learn how to do that here:

https://emby.media/community/index.php?/topic/739-how-to-report-a-problem/

Thanks.

Share this post


Link to post
Share on other sites
letterman

Attached my logs.

 

Crashing seems to happen after search clicking on search results, e.g. a song

Edited by letterman

Share this post


Link to post
Share on other sites
letterman

Crash is not only after search, but also, when browsing.

Attached a 2nd crash log.

Edited by letterman

Share this post


Link to post
Share on other sites
evil_beast666

Hi!

 

Sometimes, Emby service crash when trying to scan my music library.

 

This is because sometimes "The Audio DB" provider response with "null" value and I think you can control this behavior by "twice" lines of code :)

 

Real example:

2018-08-07 16:11:53.001 Info HttpClient: GET http://www.theaudiodb.com/api/v1/json/49jhsf8248yfahka89724011/album-mb.php?i=3a2714a8-12f7-4f00-ad0a-8cec1315ced2

Result:

{"album":null}

Workaround:

 

I've disabled temporarily this provider in my music library and work good by the moment :)

 

 

Attaching my complete log file anonymized.

 

Thx!

embyserver-63669255277.txt

Share this post


Link to post
Share on other sites
Luke

That's not going to cause a crash though. The server can handle that and continue onward.

Share this post


Link to post
Share on other sites
evil_beast666

It should but sadly crash :-\

 

I've done more test:

@@evil_beast666

 

While I browse for web interface I've clicked in some music artist link and also crash with next docker log messages:

UnhandledException
UnhandledException
System.IO.IOException: Unable to read data from the transport connection: Connection reset by peer. ---> System.Net.Sockets.SocketException: Connection reset by peer
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
   at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request)
   at System.Net.Security.SslState.ThrowIfExceptional()
   at System.Net.Security.SslState.CheckThrow(Boolean authSuccessCheck, Boolean shutdownCheck)
   at System.Net.Security.SslState.CheckOldKeyDecryptedData(Memory`1 buffer)
   at System.Net.Security.SslState.HandleQueuedCallback(Object& queuedStateRequest)
   at System.Net.Security.SslState.FinishHandshakeRead(Int32 newState)
   at System.Net.Security.SslState.FinishHandshake(Exception e, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.RehandshakeCompleteCallback(IAsyncResult result)
   at System.Net.LazyAsyncResult.Complete(IntPtr userToken)
   at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken)
   at System.Net.AsyncProtocolRequest.CompleteUserWithError(Exception e)
   at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
System.IO.IOException
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
   at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request)
   at System.Net.Security.SslState.ThrowIfExceptional()
   at System.Net.Security.SslState.CheckThrow(Boolean authSuccessCheck, Boolean shutdownCheck)
   at System.Net.Security.SslState.CheckOldKeyDecryptedData(Memory`1 buffer)
   at System.Net.Security.SslState.HandleQueuedCallback(Object& queuedStateRequest)
   at System.Net.Security.SslState.FinishHandshakeRead(Int32 newState)
   at System.Net.Security.SslState.FinishHandshake(Exception e, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.RehandshakeCompleteCallback(IAsyncResult result)
   at System.Net.LazyAsyncResult.Complete(IntPtr userToken)
   at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken)
   at System.Net.AsyncProtocolRequest.CompleteUserWithError(Exception e)
   at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
InnerException: System.Net.Sockets.SocketException
System.Net.Sockets.SocketException (104): Connection reset by peer


System.IO.IOException: Unable to read data from the transport connection: Connection reset by peer. ---> System.Net.Sockets.SocketException: Connection reset by peer
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
   at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request)
   at System.Net.Security.SslState.ThrowIfExceptional()
   at System.Net.Security.SslState.CheckThrow(Boolean authSuccessCheck, Boolean shutdownCheck)
   at System.Net.Security.SslState.CheckOldKeyDecryptedData(Memory`1 buffer)
   at System.Net.Security.SslState.HandleQueuedCallback(Object& queuedStateRequest)
   at System.Net.Security.SslState.FinishHandshakeRead(Int32 newState)
   at System.Net.Security.SslState.FinishHandshake(Exception e, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.RehandshakeCompleteCallback(IAsyncResult result)
   at System.Net.LazyAsyncResult.Complete(IntPtr userToken)
   at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken)
   at System.Net.AsyncProtocolRequest.CompleteUserWithError(Exception e)
   at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
System.IO.IOException
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
   at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request)
   at System.Net.Security.SslState.ThrowIfExceptional()
   at System.Net.Security.SslState.CheckThrow(Boolean authSuccessCheck, Boolean shutdownCheck)
   at System.Net.Security.SslState.CheckOldKeyDecryptedData(Memory`1 buffer)
   at System.Net.Security.SslState.HandleQueuedCallback(Object& queuedStateRequest)
   at System.Net.Security.SslState.FinishHandshakeRead(Int32 newState)
   at System.Net.Security.SslState.FinishHandshake(Exception e, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.RehandshakeCompleteCallback(IAsyncResult result)
   at System.Net.LazyAsyncResult.Complete(IntPtr userToken)
   at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken)
   at System.Net.AsyncProtocolRequest.CompleteUserWithError(Exception e)
   at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
InnerException: System.Net.Sockets.SocketException
System.Net.Sockets.SocketException (104): Connection reset by peer


[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] syncing disks.
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.

Share this post


Link to post
Share on other sites
Happy2Play

Merging topics.

Share this post


Link to post
Share on other sites
Luke

Crash is not only after search, but also, when browsing.

Attached a 2nd crash log.

 

@@letterman Unfortunately this defect occurred in 3.5.0, and even with Audiodb disabled in 3.5.2, you still have Audiodb urls saved in your database. Your choices to recover are:

  • Remove your music library, run a library scan, then add the music library back
  • Use a database editing tool to remove or update the saved audiodb urls
  • Like 1

Share this post


Link to post
Share on other sites
Happy2Play

Is this a non-Windows issue as I am not seeing this on my servers.

Share this post


Link to post
Share on other sites
letterman

@Luke:

  • Remove your music library, run a library scan, then add the music library back
    This is impossible - it would take weeks and it was running very stable till update...
    In case of remove & re-adding. What will happen to my collections ([boxsets])? Will they be lost?
  • Use a database editing tool to remove or update the saved audiodb urls
    Where is the database stored (QNAP)? I found the "library.db"-file in .../programdata/data? Is this the one you mean? Gosh, The size is 2.3 GB...
    Which string do I have to search & replace for?
    Which database format is it and which 
    editor do you suggest? (QNAP)


    This is a real bug - why don't you publish a tool or plugin for this (if you know what to do) easy procedure?
Edited by letterman

Share this post


Link to post
Share on other sites
evil_beast666

 

@@letterman Unfortunately this defect occurred in 3.5.0, and even with Audiodb disabled in 3.5.2, you still have Audiodb urls saved in your database. Your choices to recover are:

  • Remove your music library, run a library scan, then add the music library back
  • Use a database editing tool to remove or update the saved audiodb urls

 

 

I choose second option :)

 

1. Download sqlite browser tool: https://github.com/sqlitebrowser/sqlitebrowser/releases

2. Stop Emby server.

3. Search "library.db" file in your "data" dir. (/opt/emby/config_beta/data in my case).

4. (OPTIONAL) Perform a backup.

5. Open "library.db" with sqlitebrowser and go to "Execute SQL".

6. Paste the next, run and save changes:

UPDATE TypedBaseItems SET Images = null WHERE Images LIKE '%theaudiodb%';

7. Substitute your new modified "library.db".

8. Start Emby server again.

9. It works for me!

 

Enjoy!

  • Like 3

Share this post


Link to post
Share on other sites
evil_beast666

 

@Luke:

  • Remove your music library, run a library scan, then add the music library back

    This is impossible - too many collections & libraries

  • Use a database editing tool to remove or update the saved audiodb urls

    Which string do I have to search & replace for?

    Where is the database stored (QNAP)

    Which editor do you suggest? (QNAP)

     

     

    This is a real bug - why don't you publish a tool or plugin for this (if you know what to do) easy procedure? It's really a big mess.... Please help.

 

 

Dude...keep calm and search in Google for 5 minutes...or you can enjoy my instructions ;)

Share this post


Link to post
Share on other sites
evil_beast666

I choose second option :)

 

1. Download sqlite browser tool: https://github.com/sqlitebrowser/sqlitebrowser/releases

2. Stop Emby server.

3. Search "library.db" file in your "data" dir. (/opt/emby/config_beta/data in my case).

4. (OPTIONAL) Perform a backup.

5. Open "library.db" with sqlitebrowser and go to "Execute SQL".

6. Paste the next, run and save changes:

UPDATE TypedBaseItems SET Images = null WHERE Images LIKE '%theaudiodb%';

7. Substitute your new modified "library.db".

8. Start Emby server again.

9. It works for me!

 

Enjoy!

 

After run that sql query, the server works again. But it curious, because when I trying to identify artist metadata still using "theaudiodb" provider but it's manually disabled in my library :-\

 

Also I tried to set the "musicbrainz" id manually but the next it did is get in "theaudiodb" as you can see:

2018-08-07 21:45:34.278 Info App: Setting provider id's to item 946e1eca-9e4b-9f0f-bbfd-2f89aed298d5-GRiZ: {"MusicBrainzArtist":"c62327d3-df81-46ce-bd94-5bd3040f8699"}
2018-08-07 21:45:34.392 Info HttpClient: GET http://www.theaudiodb.com/api/v1/json/49jhsf8248yfahka89724011/artist-mb.php?i=c62327d3-df81-46ce-bd94-5bd3040f8699
2018-08-07 21:45:34.478 Error Main: UnhandledException
        *** Error Report ***
        Version: 3.6.0.2

bla bla bla the same above errors

Maybe is better idea set null Images and ProviderIds...

UPDATE TypedBaseItems SET ProviderIds = null,Images = null WHERE Images LIKE '%theaudiodb%';

What you think @@Luke? Some definitive solution to avoid "theaudiodb"?

 

 

Thx in advance :)

Edited by evil_beast666
  • Like 1

Share this post


Link to post
Share on other sites
Luke

Yes that's a good way of wiping out existing audiodb data.

  • Like 1

Share this post


Link to post
Share on other sites
letterman

It worked for me. Emby is stable again and it seems to be much faster than 3.4.x

Thanks for the help @@evil_beast666

  • Like 1

Share this post


Link to post
Share on other sites
maegibbons

Well done @evil_beast666 for a well documented solution.

 

Krs

 

Mark

  • Like 1

Share this post


Link to post
Share on other sites
Luke

Well done.

  • Like 1

Share this post


Link to post
Share on other sites
Luke

@@letterman please let us know if this helps. Thanks !

Share this post


Link to post
Share on other sites
Luke

Thanks for the feedback.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...