Jump to content

EmbyServer consuming all available memory

ivaat

By ivaat
in Linux

 Share

Recommended Posts

ivaat

ivaat   0

Posted
Posted

HI

 

I am helping one person out. He has constantly issue where embyserver consumes all available server memory. 

He has server with 64G of memory. Running unraid os with embyserver latest via docker. Version 3.5.3.0

He tried to disable emyserver addons. Still issue

Just in case analysed processes with strace. log attached in gzip due the fact, unzipped over 400MB file size.

 

What could cause this?

 

emby-strace-27-10-2018.log.gz

post-372862-0-49117900-1540641880_thumb.jpg

Link to comment
Share on other sites
ivaat

ivaat   0

Posted
  • Author
Posted

HI.

 

It is filled with users ip-s. I filtered out Error and noticed there is two types of errors. In total. Errors occurred in previous crash: grep Error embyserver-63676264150.txt |wc -l

6843
 

 

2018-10-27 19:09:10.975 Info Main: Application path: /system/EmbyServer.dll
2018-10-27 19:09:11.007 Info Main: Emby
        Command line: /system/EmbyServer.dll -programdata /config -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
        Operating system: Unix 4.14.49.0
        64-Bit OS: True
        64-Bit Process: True
        User Interactive: True
        Processor count: 32
        Program data path: /config
        Application directory: /system
2018-10-27 19:09:11.376 Info App: Application version: 3.5.3.0
2018-10-27 19:09:11.378 Info App: Loading assemblies
2018-10-27 19:09:11.440 Info App: File /config/plugins/OpenSubtitles.dll has version 1.0.2.0
2018-10-27 19:09:11.442 Info App: File /system/plugins/OpenSubtitles.dll has version 1.0.2.0
2018-10-27 19:09:11.521 Info App: Loading OpenSubtitles, Version=1.0.2.0, Culture=neutral, PublicKeyToken=null from /config/plugins/OpenSubtitles.dll
2018-10-27 19:09:11.521 Info App: Loading MediaBrowser.Plugins.TuneIn, Version=1.1.9.0, Culture=neutral, PublicKeyToken=null from /config/plugins/MediaBrowser.Plugins.TuneIn.dll
2018-10-27 19:09:11.521 Info App: Loading CoverArt, Version=4.0.79.0, Culture=neutral, PublicKeyToken=null from /config/plugins/CoverArt.dll
2018-10-27 19:09:11.521 Info App: Loading playback_reporting, Version=1.1.3.0, Culture=neutral, PublicKeyToken=null from /config/plugins/playback_reporting.dll
2018-10-27 19:09:11.521 Info App: Loading MediaBrowser.Api, Version=3.5.3.0, Culture=neutral, PublicKeyToken=null
2018-10-27 19:09:11.521 Info App: Loading MediaBrowser.WebDashboard, Version=3.5.3.0, Culture=neutral, PublicKeyToken=null
2018-10-27 19:09:11.521 Info App: Loading MediaBrowser.Model, Version=3.5.3.0, Culture=neutral, PublicKeyToken=null
2018-10-27 19:09:11.521 Info App: Loading MediaBrowser.Common, Version=3.5.3.0, Culture=neutral, PublicKeyToken=null
2018-10-27 19:09:11.521 Info App: Loading MediaBrowser.Controller, Version=3.5.3.0, Culture=neutral, PublicKeyToken=null
2018-10-27 19:09:11.521 Info App: Loading MediaBrowser.Providers, Version=3.5.3.0, Culture=neutral, PublicKeyToken=null
2018-10-27 19:09:11.521 Info App: Loading Emby.Photos, Version=3.5.3.0, Culture=neutral, PublicKeyToken=null
2018-10-27 19:09:11.521 Info App: Loading Emby.Server.Implementations, Version=3.5.3.0, Culture=neutral, PublicKeyToken=null
2018-10-27 19:09:11.521 Info App: Loading Emby.Server.MediaEncoding, Version=3.5.3.0, Culture=neutral, PublicKeyToken=null
2018-10-27 19:09:11.521 Info App: Loading Emby.Dlna, Version=3.5.3.0, Culture=neutral, PublicKeyToken=null
2018-10-27 19:09:11.521 Info App: Loading MediaBrowser.LocalMetadata, Version=3.5.3.0, Culture=neutral, PublicKeyToken=null
2018-10-27 19:09:11.521 Info App: Loading Emby.Notifications, Version=3.5.3.0, Culture=neutral, PublicKeyToken=null
2018-10-27 19:09:11.521 Info App: Loading MediaBrowser.XbmcMetadata, Version=3.5.3.0, Culture=neutral, PublicKeyToken=null
2018-10-27 19:09:11.521 Info App: Loading Emby.Server.Connect, Version=3.5.3.0, Culture=neutral, PublicKeyToken=null
2018-10-27 19:09:11.521 Info App: Loading Emby.Server.Sync, Version=3.5.3.0, Culture=neutral, PublicKeyToken=null
2018-10-27 19:09:11.803 Info SqliteUserRepository: Sqlite version: 3.24.0
2018-10-27 19:09:11.805 Info SqliteUserRepository: Sqlite compiler options: COMPILER=gcc-6.3.0,ENABLE_COLUMN_METADATA,ENABLE_DBSTAT_VTAB,ENABLE_FTS3,ENABLE_FTS3_PARENTHESIS,ENABLE_FTS3_TOKENIZER,ENAB
LE_FTS5,ENABLE_JSON1,ENABLE_PREUPDATE_HOOK,ENABLE_RTREE,ENABLE_SESSION,ENABLE_UNLOCK_NOTIFY,ENABLE_UPDATE_DELETE_LIMIT,LIKE_DOESNT_MATCH_BLOBS,MAX_SCHEMA_RETRY=25,MAX_VARIABLE_NUMBER=250000,OMIT_LOOK
ASIDE,SECURE_DELETE,THREADSAFE=1
2018-10-27 19:09:11.848 Info SqliteUserRepository: Default journal_mode for /config/data/users.db is wal
2018-10-27 19:09:11.852 Info SqliteUserRepository: PRAGMA synchronous=1
2018-10-27 19:09:11.874 Info AuthenticationRepository: Default journal_mode for /config/data/authentication.db is wal
2018-10-27 19:09:11.874 Info AuthenticationRepository: PRAGMA synchronous=1
2018-10-27 19:09:12.017 Info ActivityRepository: Default journal_mode for /config/data/activitylog.db is wal
2018-10-27 19:09:12.017 Info ActivityRepository: PRAGMA synchronous=1
2018-10-27 19:09:12.023 Info SqliteDisplayPreferencesRepository: Default journal_mode for /config/data/displaypreferences.db is wal
2018-10-27 19:09:12.024 Info SqliteDisplayPreferencesRepository: PRAGMA synchronous=1
2018-10-27 19:09:12.522 Info SqliteItemRepository: Default journal_mode for /config/data/library.db is wal
2018-10-27 19:09:12.522 Info SqliteItemRepository: PRAGMA synchronous=1
2018-10-27 19:09:12.652 Info PlaybackReporting - UserActivityAPI: UserActivityAPI Loaded
2018-10-27 19:09:12.743 Info HttpServer: Calling ServiceStack AppHost.Init
2018-10-27 19:09:13.242 Info HttpServer: Adding HttpListener prefix http://+:8096/
2018-10-27 19:09:13.242 Info HttpServer: Adding HttpListener prefix https://+:8920/
2018-10-27 19:09:13.491 Info Skia: SkiaSharp version: 1.60.0.0
2018-10-27 19:09:13.500 Info PlaybackReporting - TaskCleanDb: TaskCleanDb Loaded
2018-10-27 19:09:13.502 Info PlaybackReporting - TaskCleanDb: TaskCleanDb Loaded
2018-10-27 19:09:13.535 Info TaskManager: Daily trigger for Playback Reporting Trim Db set to fire at 10/28/18 12:00:00 AM, which is 290.774408303333 minutes from now.
2018-10-27 19:09:13.560 Info TaskManager: Daily trigger for Download missing subtitles set to fire at 10/28/18 4:00:00 AM, which is 530.773995166667 minutes from now.
2018-10-27 19:09:13.562 Info TaskManager: Daily trigger for Chapter image extraction set to fire at 10/28/18 2:00:00 AM, which is 410.773961385 minutes from now.
2018-10-27 19:09:13.580 Info TaskManager: Daily trigger for Rotate log file set to fire at 10/28/18 12:00:00 AM, which is 290.77365505 minutes from now.
2018-10-27 19:09:13.586 Info TaskManager: Daily trigger for Cloud & Folder Sync set to fire at 10/28/18 4:00:00 AM, which is 530.77355099 minutes from now.
2018-10-27 19:09:13.593 Info MediaEncoder: FFMpeg: /bin/ffmpeg
2018-10-27 19:09:13.593 Info MediaEncoder: FFProbe: /bin/ffprobe
2018-10-27 19:09:13.593 Info MediaEncoder: Validating media encoder at /bin/ffmpeg
2018-10-27 19:09:13.597 Info MediaEncoder: Running /bin/ffmpeg -decoders
2018-10-27 19:09:13.635 Info MediaEncoder: Decoder available: mpeg2video
2018-10-27 19:09:13.636 Info MediaEncoder: Decoder available: h264_cuvid
2018-10-27 19:09:13.636 Info MediaEncoder: Decoder available: hevc_cuvid
2018-10-27 19:09:13.636 Info MediaEncoder: Decoder available: mpeg2_cuvid
2018-10-27 19:09:13.636 Info MediaEncoder: Decoder available: mpeg4_cuvid
2018-10-27 19:09:13.636 Info MediaEncoder: Decoder available: vc1_cuvid
2018-10-27 19:09:13.636 Info MediaEncoder: Decoder available: vp8_cuvid
2018-10-27 19:09:13.636 Info MediaEncoder: Decoder available: vp9_cuvid
2018-10-27 19:09:13.637 Info MediaEncoder: Decoder available: ac3
2018-10-27 19:09:13.637 Info MediaEncoder: Decoder available: aac
2018-10-27 19:09:13.637 Info MediaEncoder: Decoder available: mp3
2018-10-27 19:09:13.637 Info MediaEncoder: Decoder available: h264
2018-10-27 19:09:13.637 Info MediaEncoder: Decoder available: hevc
2018-10-27 19:09:13.639 Info MediaEncoder: Running /bin/ffmpeg -encoders

2018-10-27 16:09:30.322 Error HttpServer: Error processing request
        *** Error Report ***
        Version: 3.5.3.0
        Command line: /system/EmbyServer.dll -programdata /config -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
        Operating system: Unix 4.14.49.0
        64-Bit OS: True
        64-Bit Process: True
        User Interactive: True
        Processor count: 32
        Program data path: /config
        Application directory: /system
        System.ArgumentNullException: Value cannot be null.
        Parameter name: username
           at Emby.Server.Implementations.Library.UserManager.AuthenticateUser(String username, String password, String hashedPassword, String remoteEndPoint, Boolean isUserSession)
           at Emby.Server.Implementations.Session.SessionManager.AuthenticateNewSessionInternal(AuthenticationRequest request, Boolean enforcePassword)
           at MediaBrowser.Api.UserService.Post(AuthenticateUserByName request)
--
2018-10-27 16:09:30.428 Error HttpServer: Error processing request
        *** Error Report ***
        Version: 3.5.3.0
        Command line: /system/EmbyServer.dll -programdata /config -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
        Operating system: Unix 4.14.49.0
        64-Bit OS: True
        64-Bit Process: True
        User Interactive: True
        Processor count: 32
        Program data path: /config
        Application directory: /system
        System.FormatException: Input string was not in a correct format.
           at ServiceStack.Text.StringSegmentExtensions.ParseGeneralStyleGuid(StringSegment value, Int32& len)
           at ServiceStack.Text.StringSegmentExtensions.ParseGuid(StringSegment value)
           at ServiceStack.Text.Common.DeserializeBuiltin`1.<>c.<GetParseStringSegmentFn>b__7_14(StringSegment value)
           at ServiceStack.Text.Jsv.JsvReader.<>c__DisplayClass2_0.<GetParseFn>b__0(String v)
Link to comment
Share on other sites
Luke Emby Team

Luke   37024

Posted
Posted

But first i would suggest removing these plugins and see if it helps:

2018-10-27 19:09:11.521 Info App: Loading MediaBrowser.Plugins.TuneIn, Version=1.1.9.0, Culture=neutral, PublicKeyToken=null from /config/plugins/MediaBrowser.Plugins.TuneIn.dll
2018-10-27 19:09:11.521 Info App: Loading CoverArt, Version=4.0.79.0, Culture=neutral, PublicKeyToken=null from /config/plugins/CoverArt.dll
2018-10-27 19:09:11.521 Info App: Loading playback_reporting, Version=1.1.3.0, Culture=neutral, PublicKeyToken=null from /config/plugins/playback_reporting.dll

Thanks.

Link to comment
Share on other sites
ivaat

ivaat   0

Posted
  • Author
Posted (edited)

Hi

 

Plugins disabled. 2h running and 10G memory consumed.

 

edit:

 

weird just dropped to 6806M

this might be progress

 

Also weird thing with Cloud & Folder Sync. Does not go over 1.5%. Not related?

 

Thanks

Edited by ivaat
Link to comment
Share on other sites
EODCrafter

EODCrafter   179

Posted
Posted (edited)

Server + Theater me alone watching consumes this......

5bd5c25a36c22_Screenshotfrom201810280904

 

In Comparison, Kodi doing the Exact same Activity....

 

5bd5ca258dd75_Screenshotfrom201810280937

Edited by EODCrafter
Link to comment
Share on other sites
gene0915

gene0915   37

Posted
Posted (edited)

HI

 

I am helping one person out. He has constantly issue where embyserver consumes all available server memory. 

He has server with 64G of memory. Running unraid os with embyserver latest via docker. Version 3.5.3.0

He tried to disable emyserver addons. Still issue

Just in case analysed processes with strace. log attached in gzip due the fact, unzipped over 400MB file size.

 

What could cause this?

 

I've posted about this for a while now. There's just something about Emby and Linux (not sure if I ever saw this problem with Windows) where it consumes virtually all available RAM over the course of several hours. I thought when I switched to Antergos that the problem went away. It did not. I'd like to dump Plex but Emby's run-away memory consumption is preventing me. Every time there's a server update, i grab the latest .DEB file and give it a shot hoping that the problem has finally been fixed but it still remains.

 

I noticed a way I could trigger some of the memory hogging. If I go into the console and click on 'Movies' to see all my movies, Emby will jump 50-150 megs of RAM usage. Close the browser and do it again.... another 50-100 megs of memory usage is consumed. Something as silly as just opening the admin console and clicking on your Movie library causes a spike in memory usage that is never returned to the system when you exit the console.

 

I have about 2,000 movies and a couple hundred TV shows with thousands of episodes. With Emby, after 6-12 hours, it's using 98% of my system memory. Stopping/restarting Emby resets memory usage back to 100-200? megs but let it sit for 12 hours, back to 98% memory usage.

 

With Plex, I don't think I've ever seen it use more than a couple hundred megs. Plex can run for weeks and memory usage doesn't move. Emby, not so much.

 

I currently run Mint 19 but have tested the latest version of Emby with Ubuntu, Debian, Antergos and the weird memory leak happens in all of them. Since I see the problem with different flavors of Linux, there's got to be something with Emby.

 

EDIT: Actually, I'm running Mint 19, not 18 as I originally typed.

Edited by gene0915
Link to comment
Share on other sites
mastrmind11

mastrmind11   717

Posted
Posted

Try the docker image.  My server has been up for months and emby hasn't touched 300MB.  (I'm not saying it shouldn't be looked into, just that if that's the only reason to stay w/ Plex, check out the docker :))

Link to comment
Share on other sites
gene0915

gene0915   37

Posted
Posted

Try the docker image.  My server has been up for months and emby hasn't touched 300MB.  (I'm not saying it shouldn't be looked into, just that if that's the only reason to stay w/ Plex, check out the docker :))

 

There's something about dockers I could just never get working properly. I -DID- try messing with them a while ago but I needed to get Plex up and running ASAP and decided I wasted enough time getting it to work on a Saturday morning so abandoned that effort. :)

 

Luke, I posted some server logs in this old post: https://emby.media/community/index.php?/topic/54307-emby-consuming-practically-all-available-ram/ and appreciated all your help but if I recall correctly, turning off the images (from your last note in my thread) made no difference. The memory issue, and massive CPU hit with multiple transcodes, forced me to stick with Plex for the time being.

Link to comment
Share on other sites
Luke Emby Team

Luke   37024

Posted
Posted

Logs from 7 months to 1 year ago will no longer be relevant, so we'll need to take a fresh look. Thanks.

Link to comment
Share on other sites
mastrmind11

mastrmind11   717

Posted
Posted

There's something about dockers I could just never get working properly. I -DID- try messing with them a while ago but I needed to get Plex up and running ASAP and decided I wasted enough time getting it to work on a Saturday morning so abandoned that effort. :)

 

Luke, I posted some server logs in this old post: https://emby.media/community/index.php?/topic/54307-emby-consuming-practically-all-available-ram/ and appreciated all your help but if I recall correctly, turning off the images (from your last note in my thread) made no difference. The memory issue, and massive CPU hit with multiple transcodes, forced me to stick with Plex for the time being.

I dunno how long ago "a while ago" is, but the ecosystem and community has come a long way.  I bet you can be up and running in under 30 mins https://hub.docker.com/r/emby/embyserver/

Link to comment
Share on other sites
gene0915

gene0915   37

Posted
Posted

Logs from 7 months to 1 year ago will no longer be relevant, so we'll need to take a fresh look. Thanks.

 

Oh, I didn't post that in hope you'd look at them, it was mainly for reference. I tried the latest release of Emby that came out a few weeks ago(?) and still experience the exact same memory leak issue......... the same issue I've seen since switching to Linux about two years ago and probably 5+ Emby releases.

 

When I see threads pop up about the same problem, I'll lurk and see if a solution is found. I also update my server with every new Emby release hope that THAT version finally resolves everything.

 

If I get some time this weekend, I'll post some new logs using the latest version of Emby.

 

Thanks for always being there for us!

Link to comment
Share on other sites
ivaat

ivaat   0

Posted
  • Author
Posted

Hi

 

Just to confirm. Already using docker. Also for couple days was normal but yesterday turning docker image export i had stop docker and start again. Today all memory consumed. What more can be done?

 

5bdd81c101348_imagedfsssd.jpg

Link to comment
Share on other sites
Luke Emby Team

Luke   37024

Posted
Posted

Please try again with the next release of emby server. You can try the beta server if you like. Thanks.

Link to comment
Share on other sites
wizardbuilder2017

wizardbuilder2017   0

Posted
Posted

Please try again with the next release of emby server. You can try the beta server if you like. Thanks.

Howe can install beta on docker?

Link to comment
Share on other sites
ivaat

ivaat   0

Posted
  • Author
Posted

Please try again with the next release of emby server. You can try the beta server if you like. Thanks.

 

Using version Version 3.5.3.0. Latest. After recommended disabling plugins it is much better but it still happens.

Not going to use beta. Not for this system

Link to comment
Share on other sites
wizardbuilder2017

wizardbuilder2017   0

Posted
Posted

Hello Luke,

is it possible to use a script on docker thats restarting the emby server like for example every 4 hours becouse of this memory problem?

if yes, do u have a script that i can use?
 

Thank you

Link to comment
Share on other sites
ivaat

ivaat   0

Posted
  • Author
Posted

Thanks for the feedback.

hmm strace should help developers to identify problem. it was not just feedback but also ongoing request to get it sorted. the strace log what i initially give you.. did not help?

Link to comment
Share on other sites
EODCrafter

EODCrafter   179

Posted
Posted (edited)

Hello Luke,

is it possible to use a script on docker thats restarting the emby server like for example every 4 hours becouse of this memory problem?

if yes, do u have a script that i can use?

 

Thank you

Cron job

Docker restart {containerID}

doesn't work?

Edited by EODCrafter
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing
×
×
  • Create New...