Jump to content

Emby crashing - Ubuntu server


daniellog

Recommended Posts

richardbigg

Sorry about all of that, I'm looking to see if there is an easy way yo post larger files... Maybe an attachment??

Link to comment
Share on other sites

Well again i would remove all plugins until the issues are resolved, then the logs will be smaller.

Link to comment
Share on other sites

There are definitely some issues with the Ubuntu package that need to be worked out. For some reason it's just not as stable as the other distros right not.

 

For testing purposes, if you could try the manual linux installation steps from the website, that would help us isolate whether it's package related or just in server code.

Link to comment
Share on other sites

Taloth Saldono

Hey guys, I'm on of the devs over at Sonarr. We (probably) ran into the same problem. Better part is that we isolated the problem in the linux kernel.

Would cause all kinds of native SIGSEGV and NullReferenceExceptions, took me almost a month and 100+ hours to isolate it.

 

Anyway, the problem I found only occurs in a virtual machine, afaik anyway, so if you're not running a vm then you can probably stop reading.

 

The fix for ubuntu is committed in their repository, but we'll have to wait patiently for it to get released.

Or more precisely, the commits needed were backported from linux 4.0.

 
Happened to all mono versions tested, and started with Ubuntu kernel 3.13.0-48. At first we thought it was user-specific, but as more and more reports started to accrue we had to investigate.
In hindsight it was rather obvious, slowly users started to install and boot into 3.13.0-48 in late March explaining the slow increase in affected users.
 

Some links:

https://forums.sonarr.tv/t/native-mono-crashes/4985

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1450584

 

I haven't reported it yet to the debian team.

 

I kindly ask that someone over at Emby coordinates this across the forum threads and github issues.

 

Of course one of you will have to test if the proposed fix actually solves the problem for Emby, which involves compiling a kernel.

 

 

Cheers guys and have fun!

Edited by Taloth Saldono
  • Like 1
Link to comment
Share on other sites

FingerlessGlovs

Hi guys,

 

Hope you can help me find out why I am crashing. It crashed when kodi went to update. But sometimes if can update over 10 times with out emby crashing.

 

http://pastebin.com/vFUwkjS0

Link to comment
Share on other sites

there's no evidence of a crash there. perhaps start by describing your problem in detail. thanks.

Link to comment
Share on other sites

FingerlessGlovs

I got another crash here as well. http://pastebin.com/2atraxkr

 

It seems to me that it crashes when the libary gets queryed by user either by kodi plugin or the web ui. I went to click next up and it crashed. Thats what that log is from.

 

This only started to happen since Emby updated. In the past 2 months has the need for reset the Emby libary been needed? (I would like to avoid this due to the played counter.)

 

Mono Version 3.10.0 64-Bit

Link to comment
Share on other sites

Taloth Saldono

  1.  System.NullReferenceException

  2.           at (wrapper unknown) System.Threading.Monitor:FastMonitorEnterV4 (object,bool&)

          at System.Threading.EventWaitHandle.Set () [0x00000] in <filename unknown>:0

 

Pretty tell-tale sign of the weird crashes we saw, NullRefs in places where there never should/could be a nullref. (http://emby.media/community/index.php?/topic/19955-emby-crashing-ubuntu-server/?p=207271)

 

But I've had reports of it happening on non-VMs as well, so I'm not sure if the issue I found in the kernel is the only one causing this, I hope the ubuntu kernel team gets to release the new version soon so some ppl can start testing it.

 

Ow, and one user had damaged physical memory, with the exact same symptoms... wasted 16h on that one :)

 

  • Like 1
Link to comment
Share on other sites

FingerlessGlovs

So what do I do wait for a fix or backdate by kernal?

 

I have created a bash script to check if its running and start it back up if it crashes now :)

 

EDIT: Just install the stable 4.0.2 Kernal I shall see if this fixes it. I read up a bit and it might be fixed it the lastest version.

 

EDIT 2: Removed auto restart script and shall see how long this lasts.

Edited by MrJonny
Link to comment
Share on other sites

Nodja

Did your kernel upgrade ended up fixing it?

 

I was having the same problem but the mainline kernel doesn't detect my network card on my NUC for some reason so I decided to upgrade to 15.04/3.19.0-16.and it seems to be a lot more stable now. (hasn't crashed so far)

 

I'm using the docker container btw.

Link to comment
Share on other sites

coudy

Hi,

I have this from the author of this bug report https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=784960

 

 

> how is this stress test acting ? I run it on kernel 4.0.2 and on
> 4.1.rc3, but it finish after 3 second without error, nothing printed. So
> I don't know if it is ok or not.

If the test runs to completion, it has been successful. When it fails,
mono will crash and generate an error of some sort. The exact error
differs between executions:

andy@shadow-sonarr:~$ uname -a; COUNT=1; while true; do echo $COUNT;
COUNT=$((COUNT+1)); mono ./bug-18026.exe || break; done
Linux shadow-sonarr 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt9-3~deb8u1
(2015-04-24) x86_64 GNU/Linux
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

Unhandled Exception:
System.NullReferenceException: Object reference not set to an instance
of an object
  at Test.Main () [0x00000] in <filename unknown>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.NullReferenceException: Object
reference not set to an instance of an object
  at Test.Main () [0x00000] in <filename unknown>:0

andy@shadow-sonarr:~$ COUNT=1; while true; do echo $COUNT;
COUNT=$((COUNT+1)); mono ./bug-18026.exe || break; done
1
2
3
4
Stacktrace:


Native stacktrace:

        mono() [0x4b3f7c]
        mono() [0x50c30f]
        mono() [0x423637]
        /lib/x86_64-linux-gnu/

 libpthread.so.0(+0xf8d0) [0x7f81c0c0a8d0]
        mono() [0x5f8c8b]
        mono() [0x5f7b59]
        mono() [0x58400d]
        mono() [0x5cdf5e]
        mono() [0x5d2d90]
        mono() [0x5d3672]
        mono() [0x5f0101]
        mono() [0x5f02eb]
        [0x41f58a8e]

Debug info from gdb:


=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.
=================================================================

Aborted

It seems to be particularly likely to happen inside a virtual machine
that has multiple cores assigned to i

 

try this:

wget https://github.com/mono/mono/raw/master/mono/tests/bug-18026.cs
mcs bug-18026.cs
mono bug-18026.exe
uname -a; COUNT=1; while true; do echo 1661; COUNT=1662; mono ./bug-18026.exe || break; done

if it crash, you are in affected kernel version and you should upgrade kernel. I have compiled kernel 4.x on debian, and my MB does not crash for 6h.

Edited by coudy
  • Like 1
Link to comment
Share on other sites

xelar

I was on Debian 3.16 Kernel and had the same issues. Tried the Docker version but didn't help either.

 

I now downgraded to 3.2.68-1+deb7u1 and will test the latest Emby version 5607.2

Link to comment
Share on other sites

xelar

I downgraded to Debian 3.2 and this bug test runs without crashing there. Let's see if that is also true for emby :)

Link to comment
Share on other sites

Nodja

try this:

wget https://github.com/mono/mono/raw/master/mono/tests/bug-18026.cs
mcs bug-18026.cs
mono bug-18026.exe
uname -a; COUNT=1; while true; do echo 1661; COUNT=1662; mono ./bug-18026.exe || break; done

if it crash, you are in affected kernel version and you should upgrade kernel. I have compiled kernel 4.x on debian, and my MB does not crash for 6h.

 

Thanks, this confirms it. It's not crashing for me anymore on 15.04 with the 3.19.0-16 kernel.

Link to comment
Share on other sites

thefirstofthe300

YAY!!! It might be FIXED!

 

What an interesting bug. I knew it had to be something weird.

Link to comment
Share on other sites

thefirstofthe300

Just an FYI, I'm not running it in a VM and its still happening.

 

You might still try rolling back your kernel to see what happens. The guys in the other thread seem to be having some success with doing so.

Link to comment
Share on other sites

Taloth Saldono

Thanks, this confirms it. It's not crashing for me anymore on 15.04 with the 3.19.0-16 kernel.

 

FYI, 3.19.0-16 does not contain the fixes I mentioned. So if 3.19.0-16 is truly stable then it's possibly another reason or fix. Which kernel were you running before?

When I debugged on ubuntu the testcase would sometimes not crash for several dozen of cycles, and I was running a heavier version of the testcase.

Link to comment
Share on other sites

FingerlessGlovs

Sorry Guys for late reply but want to make sure its works.

Since Kernal 4.0.1 It has been more stable then it has ever been for me.

 

Due to a fix with the rtc timer I believe

Edited by MrJonny
  • Like 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...