Server Hanging/Crash at least once a day.

[plazma 13]

Since the big database upgrade (2 stable releases back). Ive had the server crash once and at least twice a day it dies and becomes a zombie (refusing to answer clients).

:v3.0.5871

 

Host is windows 7 x86 and upto two builds back its been solid as a rock.

 

Crash (although i believe this may have been on the last build):

Object reference not set to an instance of an object.

System.NullReferenceException

   at System.Diagnostics.Process.Close()

   at System.Diagnostics.Process.Dispose(Boolean disposing)

   at System.ComponentModel.Component.Dispose()

   at MediaBrowser.MediaEncoding.Encoder.MediaEncoder.ProcessWrapper.Process_Exited(Object sender, EventArgs e)

   at System.Diagnostics.Process.OnExited()

   at System.Diagnostics.Process.RaiseOnExited()

   at System.Diagnostics.Process.CompletionCallback(Object context, Boolean wasSignaled)

   at System.Threading._ThreadPoolWaitOrTimerCallback.WaitOrTimerCallback_Context(Object state, Boolean timedOut)

   at System.Threading._ThreadPoolWaitOrTimerCallback.WaitOrTimerCallback_Context_f(Object state)

   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)

   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)

   at System.Threading._ThreadPoolWaitOrTimerCallback.PerformWaitOrTimerCallback(Object state, Boolean timedOut)

 

 

And the latest server log from right before when it hung this last time doesn't show much exciting towards the end, apart from (below) shortly before it snuffs it:

 

2016-02-18 17:47:25.4050 Info MediaEncoder: Killing ffmpeg process

2016-02-18 17:47:25.4206 Error App: Error in ffprobe

*** Error Report ***

Version: 3.0.5871.0

Command line: C:\Users\plazma\AppData\Roaming\Emby-Server\System\MediaBrowser.ServerApplication.exe

Operating system: Microsoft Windows NT 6.1.7601 Service Pack 1

Processor count: 3

64-Bit OS: False

64-Bit Process: False

Program data path: C:\Users\plazma\AppData\Roaming\Emby-Server

Application Path: C:\Users\plazma\AppData\Roaming\Emby-Server\System\MediaBrowser.ServerApplication.exe

ffprobe failed - streams and format are both null.

System.ApplicationException

  at MediaBrowser.MediaEncoding.Encoder.MediaEncoder.<GetMediaInfoInternal>d__4.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

  at MediaBrowser.Providers.MediaInfo.FFProbeAudioInfo.<GetMediaInfo>d__6.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

  at MediaBrowser.Providers.MediaInfo.FFProbeAudioInfo.<Probe>d__0`1.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

  at MediaBrowser.Providers.Manager.MetadataService`2.<RunCustomProvider>d__5b.MoveNext()

 

 

Im also seeing plenty of this while flicking back through the various logs:

 

2016-02-17 20:05:10.4992 Error ServerManager: SessionWebSocketListener failed processing WebSocket message Context

*** Error Report ***

Version: 3.0.5870.0

Command line: C:\Users\plazma\AppData\Roaming\Emby-Server\system\MediaBrowser.ServerApplication.exe

Operating system: Microsoft Windows NT 6.1.7601 Service Pack 1

Processor count: 3

64-Bit OS: False

64-Bit Process: False

Program data path: C:\Users\plazma\AppData\Roaming\Emby-Server

Application Path: C:\Users\plazma\AppData\Roaming\Emby-Server\system\MediaBrowser.ServerApplication.exe

Value cannot be null.

Parameter name: id

System.ArgumentNullException

  at MediaBrowser.Server.Implementations.Library.LibraryManager.GetItemById(Guid id)

  at MediaBrowser.Server.Implementations.Session.SessionManager.ReportNowViewingItem(String sessionId, String itemId)

  at MediaBrowser.Server.Implementations.Session.SessionWebSocketListener.ProcessContextMessage(WebSocketMessageInfo message)

  at MediaBrowser.Server.Implementations.Session.SessionWebSocketListener.ProcessMessage(WebSocketMessageInfo message)

  at MediaBrowser.Server.Implementations.ServerManager.ServerManager.<>c__DisplayClass3.<>c__DisplayClass5.<<ProcessWebSocketMessageReceived>b__2>d__7.MoveNext()

 

2016-02-17 20:05:10.4992 Info HttpServer: HTTP GET http://xxx.xxx.xxx.xxx:8096/mediabrowser/Items/f9f13b71b62aaec354936f6024e22055/ThemeMedia?InheritFromParent=False&UserId=2c78c85c31db41a7a13c420422821175&format=Json. UserAgent: 

2016-02-17 20:05:10.4992 Info HttpServer: HTTP Response 200 to xxx.xxx.xxx.xxx. Time: 0ms. http://xxx.xxx.xxx.xxx:8096/mediabrowser/Items/f9f13b71b62aaec354936f6024e22055/ThemeMedia?InheritFromParent=False&UserId=2c78c85c31db41a7a13c420422821175&format=Json

2016-02-17 20:05:10.7023 Error ServerManager: SessionWebSocketListener failed processing WebSocket message Context

*** Error Report ***

Version: 3.0.5870.0

Command line: C:\Users\plazma\AppData\Roaming\Emby-Server\system\MediaBrowser.ServerApplication.exe

Operating system: Microsoft Windows NT 6.1.7601 Service Pack 1

Processor count: 3

64-Bit OS: False

64-Bit Process: False

Program data path: C:\Users\plazma\AppData\Roaming\Emby-Server

Application Path: C:\Users\plazma\AppData\Roaming\Emby-Server\system\MediaBrowser.ServerApplication.exe

Value cannot be null.

Parameter name: id

System.ArgumentNullException

  at MediaBrowser.Server.Implementations.Library.LibraryManager.GetItemById(Guid id)

  at MediaBrowser.Server.Implementations.Session.SessionManager.ReportNowViewingItem(String sessionId, String itemId)

  at MediaBrowser.Server.Implementations.Session.SessionWebSocketListener.ProcessContextMessage(WebSocketMessageInfo message)

  at MediaBrowser.Server.Implementations.Session.SessionWebSocketListener.ProcessMessage(WebSocketMessageInfo message)

  at MediaBrowser.Server.Implementations.ServerManager.ServerManager.<>c__DisplayClass3.<>c__DisplayClass5.<<ProcessWebSocketMessageReceived>b__2>d__7.MoveNext()

 

 

And there is also lots of this (opened a of few of the xmls manually and they all appear fine):

 

2016-02-17 23:00:58.5612 Error App: Error parsing xml file \\xxx\movie.xml

2016-02-17 23:00:58.5612 Error App: Error in Emby Xml

*** Error Report ***

Version: 3.0.5871.0

Command line: C:\Users\plazma\AppData\Roaming\Emby-Server\system\MediaBrowser.ServerApplication.exe

Operating system: Microsoft Windows NT 6.1.7601 Service Pack 1

Processor count: 3

64-Bit OS: False

64-Bit Process: False

Program data path: C:\Users\plazma\AppData\Roaming\Emby-Server

Application Path: C:\Users\plazma\AppData\Roaming\Emby-Server\system\MediaBrowser.ServerApplication.exe

The 'Synopsis' start tag on line 38 position 3 does not match the end tag of 'Plot'. Line 40, position 234.

System.Xml.XmlException

  at System.Xml.XmlTextReaderImpl.Throw(Exception e)

  at System.Xml.XmlTextReaderImpl.Throw(String res, String[] args)

  at System.Xml.XmlTextReaderImpl.ThrowTagMismatch(NodeData startTag)

  at System.Xml.XmlTextReaderImpl.ParseEndElement()

  at System.Xml.XmlTextReaderImpl.ParseElementContent()

  at System.Xml.XmlTextReaderImpl.Read()

  at System.Xml.XmlTextReaderImpl.Skip()

  at MediaBrowser.Controller.Providers.BaseItemXmlParser`1.FetchDataFromXmlNode(XmlReader reader, MetadataResult`1 itemResult)

  at MediaBrowser.LocalMetadata.Parsers.BaseVideoXmlParser`1.FetchDataFromXmlNode(XmlReader reader, MetadataResult`1 result)

  at MediaBrowser.Controller.Providers.BaseItemXmlParser`1.Fetch(MetadataResult`1 item, String metadataFile, XmlReaderSettings settings, Encoding encoding, CancellationToken cancellationToken)

  at MediaBrowser.Controller.Providers.BaseItemXmlParser`1.Fetch(MetadataResult`1 item, String metadataFile, CancellationToken cancellationToken)

  at MediaBrowser.LocalMetadata.Providers.MovieXmlProvider.Fetch(MetadataResult`1 result, String path, CancellationToken cancellationToken)

  at MediaBrowser.LocalMetadata.BaseXmlProvider`1.<GetMetadata>d__0.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)

  at MediaBrowser.Providers.Manager.MetadataService`2.<RefreshWithProviders>d__3e.MoveNext()

 

Actually looking at the xml closely it does appear its right, it starts "<Synopsis>" and ends "</Plot>"

 

Going to have a to do a trawl through and correct them, guess at some point my meta manager had an issue... doh although i dont think its this causing the crash (i would hope) as that meta has been like that for over 3 years on the one i looked at.

Edited February 18, 2016 by plazma

[Luke 42083]

Please always supply the complete log files. thanks.

[plazma 13]

The unhandled (crash, first one) was the full log.

 

Ive wiped the existing logs (for now so im not trawling through days worth looking for clues if it happens again), corrected the xml files and blown an in place manual install over the top for good measure, will see what occurs over the next 24 hours and post back with a sanitized log if it continues, ive not managed more than 24 hours before hitting an issue previously so, i should know if ive still got an issue pretty quick.

 

:-)

[houseofbugs 0]

I am having this exact same issue. My logs look similar. Can you tell me how to get them and I can post mine? This does it on the 5871 release and the 5885 beta.

[Luke 42083]

Hi, instructions are here:

 

http://emby.media/community/index.php?/topic/739-how-to-report-a-problem/

 

Thanks.

[houseofbugs 0]

server-63591609600.txt

 

Thanks for the reply. A log from when a crash took place is attached.

 

 

Edited February 23, 2016 by houseofbugs

[Luke 42083]

server-63591609600.txt

 

Thanks for the reply. A log from when a crash took place is attached.

 

Yea there are some fundamental issues there. Since you're running a beta version, i would first try updating to the latest.

[plazma 13]

Ok well since my last post ive been 100% stable for days now (sorry for the delayed reply, personal life gets in the way).

 

What i did :

 

Installed in place (over the top) stable build (same version i already had).

Corrected the few files where meta was not correct (tags not matching... not sure how that happened its been like it for years apparently).

Installed :

KB3118401 - Universal C Runtime Update

KB3121255 - Fix FsRtlNotifyFilterReportChange

 

Not sure which one fixed it, but one did, as i said i only had one crash (and that was a previous build), what i was largely getting was getting the server turning into a zombie and sitting there doing nothing and responding to nothing (sort of like it got stuck on something).

 

Sorry i cant be more help, but if your getting the same message in the logs about xml tags (go and correct them), make sure the Universal C runtime updates installed and hopefully it may help you

 

Update: just having a flick through your logs i see a few things:

 

1. Its having major issues with meta (people) collection, have you had a play with any of the meta options in emby (specifically the people), i personally dont use emby for meta collection (i used media centre master instead), but i just had a quick look at the default options on mine in emby, make sure the default options are set and emby.xml is ticked (change it and hit save if its not).

 

Its also throwing a similar error on subtitles so you may want to double check this.

 

2. I also see errors in there for : Queueing task SyncConvertScheduledTask, Are you a premium subscriber, if not im not even sure why that would be running... if you dont use it you could just go into the scheduled task and delete all the trigger events (you could re-create them if you decided to use it at a later date).

 

Im guessing your database update went ok... ? 

Edited February 23, 2016 by plazma

[Luke 42083]

Well done.