Crash during media scan ('out of memory' error in ffprobe?)

[dncrjim 0]

Hello,

 

Originally had this problem on:

Version: 3.2.25.0

SBC: RPi 3 B

DietPi: V153

 

Posted about it here:  https://github.com/MediaBrowser/Emby/issues/2755

 

Have now upgraded to:

Version: 3.2.26.0

DietPi: V154

(brand new fresh install)

 

Still have the problem. to summarize:

Any type of media scan (library scan or individual folder scan) will work for a period of time (non-consistent or predictable from what I can tell) then locks up the pi for around 20 minutes, and then emby-server disappears from the active process list and the pi returns to normal running. emby-server does not self restart, but can be restarted and appears to return to running normally. This is especially a problem since I am attempting to scan 1700+ movies and 120+ shows, as this start and stop process appears to only be scanning an average of 100 files before it crashes. I was eventually able to get all of the show episodes scanned on the original install, but never the movies.

 

Surely there is a better option than just restarting emby-server from the command line every several hours and hoping it gets a bit further each time...

 

Both original and new server logs attached.

 

Just in case it's relevant, files are on smb share on network; metadata and images are stored locally on the pi.

 

Thanks for any help!

2017.07.13 08.38.log.txt

2017.07.13 13.46.log.txt

2017.07.28 19.13 Emby Crash Log.txt

2017.07.28 22.01 Emby Crash Log.txt

Edited July 29, 2017 by dncrjim

[Luke 37191]

Hi, I notice you're running mono 5.0. We are not yet supporting that because we've found some defects with it. Are you able to revert back to 4.8.1? Thanks.

[dncrjim 0]

From my forum searching I had wondered if that was the problem. While probably able, I don't trust that the instructions I've cobbled together from several google results would work without breaking something I will not be able to fix. Is there a resource you could direct me to with instructions?

 

The best I found was: https://forum.fivem.net/t/server-latest-mono-5-0-0-will-break-the-server-read-for-fix/19393 but I believe it would need some tweaks that I do not trust myself to make.

 

:-/

[dncrjim 0]

Managed to downgrade mono to 4.8.1, same problem exists. Please advise.

2017.07.29 03.40 Emby Server Log.txt

[dncrjim 0]

Still having the same issue during scans. Now have a different out of memory error as well. Set up systemd to auto-restart service after crash. Restart is immediate upon emby-server crash. To show the time it spends locked up, this log ends at 6:32am and the next log does not begin until 9:17am.
 
Please help.

2017.07.31 06.32 Emby Crash Log.txt

Edited July 31, 2017 by dncrjim

[smerrikin 3]

I too am having a shit ton of issues with Emby crashing on scans. I can't even keep the server running cause as soon as I restart it, it starts a scan and crashes again.

 

I am also on Mono v5 and don't know much about Linux to be able to downgrade.

 

This is on a fresh server running 3.2.26.0

My server that is running 3.2.20.0 is perfectly fine. Also running Mono v5.

[dncrjim 0]

Things I have learned in the last 2 days...

• downgrading to Mono v4.8.1 does not help.
• auto-restarting emby-server upon crash does work and reduces necessary user-intervention a lot through the first 24-48 hrs of attempted scanning. (service is set to auto-scan library on startup) but doesn't do much good after that.
• scanning individual libraries is much more successful than letting emby do a full library scan, but does not solve problem.
• reducing swappiness from 10 (seeming default on my system?) to 1 helps. a lot. but absolutely does not solve problem.
• I don't seem to be able to make the system save the setting to swappiness=0 (I know it's not recommended, but it was worth a try)
• system still regularly becomes non-responsive (for 2-8 hours at a time) when it maxes out the memory and swap, and in the end it crashes anyway requiring a reboot to clear both ram and swap. At the point that a decent portion of the media files have been scanned, the system will only be responsive for 5-10 minutes after reboot before the memory maxes out and the system becomes completely unusable.
• running comparison testing on 3 different pi's to try and get around this problem is not a particularly entertaining hobby.

If someone with a bit more experience with this could address the issue I would really appreciate it.

 

System has been stuck here for an hour:

 

Auto-restart service:

 

# sudo systemctl enable emby-server.service

to confirm success:

# ln /etc/systemd/system/multi-user.target.wants/

 

edit the file:

# sudo nano /etc/systemd/system/multi-user.target.wants/emby-server.service

and change the line with Restart=* to Restart=always

save and exit; reboot.

 

Change Swappiness:

 

# sudo nano /proc/sys/vm/swappiness

 

mine was originally set to 10, reducing it to 1 definitely helped. if I set it to 0, the setting seems to go back to 1 on reboot, so I haven't been able to test that option.

Edited August 2, 2017 by dncrjim

[Luke 37191]

@@dncrjim, thanks for the info! I would be interested to see how your experience compares when running under the new Microsoft .NET Core runtime as opposed to mono.

 

If you'd like to test that out then please check out this topic:

 

https://emby.media/community/index.php?/topic/50012-emby-server-for-net-core/

 

Thanks !