EmbyServer dying.

[frankie64 9]

So I rebuilt everything on a NEW server (Ryzen 7 -3900). Had some problems with the server Program Dying. So I rebuilt on a almost 100% identical box.  Noticed the same problem.  I have attached the end of the log file plus a status report from the system.  Hoping you guys can provide some insight into this. 

emby-log-info.txt

[Lessaj 467]

You system doesn't seem to have enough memory, it's being killed by the OOM (out of memory) killer. How much memory and swap space is configured?

Quote

Jun 10 05:46:48 emby2-rebuild systemd[1]: emby-server.service: A process of this unit has been killed by the OOM killer.
Jun 10 05:46:52 emby2-rebuild systemd[1]: emby-server.service: Main process exited, code=killed, status=9/KILL
Jun 10 05:46:52 emby2-rebuild systemd[1]: emby-server.service: Failed with result 'oom-kill'.
Jun 10 05:46:52 emby2-rebuild systemd[1]: emby-server.service: Consumed 3h 32min 22.666s CPU time, 2.0G memory peak, 0B memory swap peak.

 

[frankie64 9]

32 GB ram and 8GB swap....

[Lessaj 467]

Hm, well considering the uptime is only 2 days and you've dipped over 1GB into the swap to me this suggests that you did have a low memory condition and possibly the 2GB peak reported by systemd is inaccurate, or rather it's accurate but if emby server was the highest memory consuming process during the low memory condition then it could have been chosen to be killed. It could also be due to memory fragmentation, if there wasn't enough pages free when the memory allocation request was made. What other services are you running on this system? Do you have any kind of monitoring tool to view historical memory usage? My system has 32GB of memory, most of which is used for zfs cache, and with an uptime of 8 days hasn't touched the swap.

[frankie64 9]

dmesg | grep -i 'killed process' [165200.130509] Out of memory: Killed process 1216 (EmbyServer) total-vm:47639428kB, anon-rss:29498488kB, file-rss:588kB, shmem-rss:0kB, UID:997 pgtables:85668kB oom_score_adj:0

ChatGPT

The dmesg output shows that the Emby server process (EmbyServer) was killed due to running out of memory. The process had a very high memory usage (total virtual memory: ~47GB, anonymous resident set size: ~29GB), which is consistent with the OOM killer's decision to terminate it.

[Lessaj 467]

That makes a lot more sense than the 2gb reported by systemd, not sure what that's about. Please attach a complete log file.

[frankie64 9]

OK, I think i know what it is, it looks like there is a memory leak, during a "scan media library process" the system went from using 6 GB of ram to almost 16GB, fair enough, however when the process stopped it did not give it back.  running it again RAM usage hit 23.6 GB, again not giving any back, (emby virtual ram 24.8 GB), each run of the task caused more memory to be consumed:

[frankie64 9]

BTW, running Version 4.8.8.0. on ubuntu 24.04 LTS, This only started to happen on the AMD processor, I was running on an I7 before. 

[Luke 42080]

Hi, please attach the complete Emby server log file. Thanks.

[frankie64 9]

emby_logs.zip. I included all of the log files from the past few days, the one that has a timestamp of 05:46 this morning is when i think it died this morning. Let me know if you want me to provide anything else.

 

[Lessaj 467]

Your logs contain a lot of errors related to opensubtitles, have you correctly configured this plugin?

Quote

2024-06-10 00:00:08.486 Error SubtitleManager: Error searching for subtitles on Open Subtitles
    MediaBrowser.Model.Net.HttpException: MediaBrowser.Model.Net.HttpException: Unauthorized

    MediaBrowser.Model.Net.HttpException: MediaBrowser.Model.Net.HttpException: TooManyRequests

There's also several errors related to access being denied to paths for sync jobs (bunch of fallout episodes and others), and I guess even a whole library path is missing too.

Quote

System.UnauthorizedAccessException: System.UnauthorizedAccessException: Access to the path '/mnt/nas2/current_tv/Fallout/Season 1/Fallout - S01E01 - The End WEBDL-2160p Proper - mobile.mp4' is denied.

System.UnauthorizedAccessException: System.UnauthorizedAccessException: Access to the path '/mnt/win-nas2/TV/Cagney & Lacey/Season 7/Cagney & Lacey - S07E22 - A Fair Shake (2) SDTV.nfo' is denied.

    System.IO.DirectoryNotFoundException: System.IO.DirectoryNotFoundException: Could not find a part of the path '/mnt/write/recordings'.

 

[frankie64 9]

2 hours ago, Lessaj said:

Your logs contain a lot of errors related to opensubtitles, have you correctly configured this plugin?

There's also several errors related to access being denied to paths for sync jobs (bunch of fallout episodes and others), and I guess even a whole library path is missing too.

 

As far as the opensubtitles , i uninstalled the plugin, as for the access errors, i forgot to mount the remote drives, ok, so now the process does not crash the system and take up the memory. the problem resolved around the fact that the restore process going to the new machine restored the plugin but it looks like it did not restore the authentication. Somehow this cause the media scan to do this. seems to be fine now.

 

Edited June 10, 2024 by frankie64

[Lessaj 467]

I wasn't sure if it would actually be related but it was more that it makes the log really difficult to read so it's better to resolve it either by fixing the authentication or removing the plugin. Glad it's improved, certainly keep us posted if problems arise again.