Jump to content

Ubuntu Server - Crashing Overnight

darrenkdean

By darrenkdean
in Linux

 Share

Recommended Posts

darrenkdean

darrenkdean   29

Posted
Posted

Good Morning,

 

I'm running into an issue with ubuntu emby server, whereby when we wake up in the mornings the emby server is unresponsive.  Can't ssh into the machine & the terminal is unresponsive (frozen) as well.  Only way to fix it is to restart the machine.  In an effort to begin troubleshooting, I have timestamped the logs & attached them from the point of restart to the last known working before the crash.  Any thoughts or guidance you have on how to correct this issue would be greatly appreciated.

 

Best-

server-63637169157, 7.30am.txt

server-63637169187, 7.26am.txt

unhandeled_e934ea41-9b6b-423f-b666-ba8490d724f9, 7.26am.txt

server-63637142400, 4.35am.txt

ffmpeg-transcode-b5e17cd1-5ee0-41b9-b120-c4db2b94fcbc, 4.35am.txt

Link to comment
Share on other sites
davidacarter1984

davidacarter1984   0

Posted
Posted (edited)

Hi there

 

I am also having an issue with emby server crashing. looking at my syslog file it seems to get stuck and shutdown on a null reference error this is not only over night it can happen within an hour of starting the services back up. extract from logfile included below.

 

Aug  1 11:59:21 MediaServer emby-server[9350]: System.NullReferenceException
Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.IO.InotifyWatcher.ProcessEvents (System.Byte[] buffer, System.Int32 length) [0x000ac] in <5641e4edad4f4464ba58c620a7b8ea48>:0
Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.IO.InotifyWatcher.Monitor () [0x0003c] in <5641e4edad4f4464ba58c620a7b8ea48>:0
Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) [0x00017] in <dbb16e0bacdc4a0f87478e401bc29b6c>:0
Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x0008d] in <dbb16e0bacdc4a0f87478e401bc29b6c>:0
Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <dbb16e0bacdc4a0f87478e401bc29b6c>:0
Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) [0x00031] in <dbb16e0bacdc4a0f87478e401bc29b6c>:0
Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.Threading.ThreadHelper.ThreadStart () [0x0000b] in <dbb16e0bacdc4a0f87478e401bc29b6c>:0
Aug  1 11:59:21 MediaServer emby-server[9350]: [ERROR] FATAL UNHANDLED EXCEPTION: System.NullReferenceException: Object reference not set to an instance of an object
Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.IO.InotifyWatcher.ProcessEvents (System.Byte[] buffer, System.Int32 length) [0x000ac] in <5641e4edad4f4464ba58c620a7b8ea48>:0
Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.IO.InotifyWatcher.Monitor () [0x0003c] in <5641e4edad4f4464ba58c620a7b8ea48>:0
Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) [0x00017] in <dbb16e0bacdc4a0f87478e401bc29b6c>:0
Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x0008d] in <dbb16e0bacdc4a0f87478e401bc29b6c>:0
Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <dbb16e0bacdc4a0f87478e401bc29b6c>:0
Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) [0x00031] in <dbb16e0bacdc4a0f87478e401bc29b6c>:0
Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.Threading.ThreadHelper.ThreadStart () [0x0000b] in <dbb16e0bacdc4a0f87478e401bc29b6c>:0
Aug  1 11:59:21 MediaServer tvheadend[2062]: htsp: 192.168.0.3 [ carter | TVHclient4Emby-3.2.26.0 ]: Disconnected
Aug  1 11:59:21 MediaServer systemd[1]: emby-server.service: Main process exited, code=exited, status=1/FAILURE
Aug  1 11:59:21 MediaServer systemd[1]: emby-server.service: Unit entered failed state.
Aug  1 11:59:21 MediaServer systemd[1]: emby-server.service: Failed with result 'exit-code'.
 

Edited by davidacarter1984
Link to comment
Share on other sites
Luke Emby Team

Luke   37180

Posted
Posted

@@darrenkdean, after the server shut down, the OS kept a hold on the server port and that's why you couldn't start back up again. Rebooting is generally the simplest way to solve this, although in many cases the port will also be released after a period of time.

 

As far as why it is happening in the first place, can you please try reducing your set of installed plugins to just the following:


	Dropbox 3.0.6143.21237
	Folder Sync 3.0.6149.23393
	Genre Cleaner 1.0.2.5
	Google Drive 3.0.6153.28260
	Server Configuration Backup 1.0.1.15
	Trailers 1.0.6399.530
	Studio Cleaner

Once you've done this, let us know if the issue persists. Thanks.

  • Like 1
Link to comment
Share on other sites
darrenkdean

darrenkdean   29

Posted
  • Author
Posted

@@darrenkdean, after the server shut down, the OS kept a hold on the server port and that's why you couldn't start back up again. Rebooting is generally the simplest way to solve this, although in many cases the port will also be released after a period of time.

 

As far as why it is happening in the first place, can you please try reducing your set of installed plugins to just the following:


	Dropbox 3.0.6143.21237
	Folder Sync 3.0.6149.23393
	Genre Cleaner 1.0.2.5
	Google Drive 3.0.6153.28260
	Server Configuration Backup 1.0.1.15
	Trailers 1.0.6399.530
	Studio Cleaner

Once you've done this, let us know if the issue persists. Thanks.

 

Thank you for the feedback.  I have reduced it down to the suggested plugins & will watch it for the next couple of days to see if it shuts down again.  More to come.

 

Best-

Link to comment
Share on other sites
Luke Emby Team

Luke   37180

Posted
Posted

Hi there

 

I am also having an issue with emby server crashing. looking at my syslog file it seems to get stuck and shutdown on a null reference error this is not only over night it can happen within an hour of starting the services back up. extract from logfile included below.

 

Aug  1 11:59:21 MediaServer emby-server[9350]: System.NullReferenceException

Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.IO.InotifyWatcher.ProcessEvents (System.Byte[] buffer, System.Int32 length) [0x000ac] in :0

Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.IO.InotifyWatcher.Monitor () [0x0003c] in :0

Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) [0x00017] in :0

Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x0008d] in :0

Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in :0

Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) [0x00031] in :0

Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.Threading.ThreadHelper.ThreadStart () [0x0000b] in :0

Aug  1 11:59:21 MediaServer emby-server[9350]: [ERROR] FATAL UNHANDLED EXCEPTION: System.NullReferenceException: Object reference not set to an instance of an object

Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.IO.InotifyWatcher.ProcessEvents (System.Byte[] buffer, System.Int32 length) [0x000ac] in :0

Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.IO.InotifyWatcher.Monitor () [0x0003c] in :0

Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) [0x00017] in :0

Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x0008d] in :0

Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in :0

Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) [0x00031] in :0

Aug  1 11:59:21 MediaServer emby-server[9350]:   at System.Threading.ThreadHelper.ThreadStart () [0x0000b] in :0

Aug  1 11:59:21 MediaServer tvheadend[2062]: htsp: 192.168.0.3 [ carter | TVHclient4Emby-3.2.26.0 ]: Disconnected

Aug  1 11:59:21 MediaServer systemd[1]: emby-server.service: Main process exited, code=exited, status=1/FAILURE

Aug  1 11:59:21 MediaServer systemd[1]: emby-server.service: Unit entered failed state.

Aug  1 11:59:21 MediaServer systemd[1]: emby-server.service: Failed with result 'exit-code'.

 

 

@@davidacarter1984, try turning off the realtime monitor for your Emby libraries. Unfortunately there's a crash coming from the mono runtime that we're not able to catch and handle. We will be looking to switch to the microsoft .net core 2.0 runtime in the near future.

Link to comment
Share on other sites
darrenkdean

darrenkdean   29

Posted
  • Author
Posted

Happy to report, that after removing all but the suggested plugins, Emby did not crash overnight.  I'm guessing I should add the plugins back slowly, 1 by 1, until the crashing resumes?  Any thoughts on which plugin might be the culprit?

 

Best-

Link to comment
Share on other sites
Luke Emby Team

Luke   37180

Posted
Posted

It's hard to say because some of those come from community developers and I don't even know very much about what the plugins are doing. i would start with autoboxsets. that one should be safe to add back.

Link to comment
Share on other sites
  • 2 weeks later...
darrenkdean

darrenkdean   29

Posted
  • Author
Posted

It's hard to say because some of those come from community developers and I don't even know very much about what the plugins are doing. i would start with autoboxsets. that one should be safe to add back.

 

Wanted to double back on this thread.  After testing one by one, the issue appears to be with the Emby Statistics plugin.  Once removed, the server no longer appears to crash.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing
×
×
  • Create New...