*Resolved* RAM Utilization Balloons in Emby Server v. 4.7.11.0

[Edrock200 38]

Not sure if it's my config or an issue with 4.7.11.0. I've noticed over the last few days 2 instances of emby on 2 different servers using massive amounts of RAM, like 60+GB. A restart brings it back down to normal levels (< 4GB), but it slowly creeps up over a few days. I did modify my DB cache size to 1024MB, however, its been at this setting for months, and the RAM balloon issue didn't seem to happen with previous versions.

Edited May 2, 2023 by Edrock200

[Luke 37144]

Hi there, please attach the emby server log from when the problem occurred. Thanks.

[rbjtech 4300]

From experience, key culprits here include run away strm files (that loop across libraries) or items that appear in more than one library (over lapping libraries).

Also watch 'when' the memory starts increasing - is it during library scans ?

[Edrock200 38]

I rotated my logs and I was waiting for it to happen again but think I found the culprits on both, which interestingly appear to be different root causes.

 

Emby1 - someone had accidentally queued a large amount of items for transcode and download.

Emby2 - one large library unintentionally had almost every metadata providers enabled to include screen grabber. For reasons I'm still unsure of, this ffprobe/ffmpeg function to grab a screen was failing/crashing on a large number of episodes. However, I don't want screen grabber on anyway.

 

Turning off screen grabber and deleting the conversions/transfers scheduled tasks seemed to resolve the issue on both. I'll monitor and if it happens again grab and attach some clean logs. 

With regards to overlapping Libraries, I do not have overlapping paths or media across libraries nor do I utilize strm files.

Thank you both for the prompt replies and apologies for the false alarm.

Edited May 2, 2023 by Edrock200

[Edrock200 38]

I should note, I would highly recommend setting conversion tasks to default to auto threads and not tick "full speed conversion". Presently it defaults to max threads and full speed conversion. I have a decently beefy box (i9-9900, nvme, 128gb ram) and even with this conversions absolutely hammer the box and make emby nearly unusable until conversions are done. Adjusting to auto and unticking full speed conversions seemed to make things usable in parallel. I can't imagine most would want conversions to have such priority by default. @LukeJust my 2 cents fwiw.

Edited May 2, 2023 by Edrock200

[legierk 6]

I have the issue. I restarted the Win11 PC to solve it. So far RAM usage is about 100MB. Been sitting at 14k MB though and that is where this log is taken from. I see above about turning stuff off etc......could someone re-explain what to turn off to a novice?

embyserver-63819596633.txt

[Luke 37144]

1 hour ago, legierk said:

I have the issue. I restarted the Win11 PC to solve it. So far RAM usage is about 100MB. Been sitting at 14k MB though and that is where this log is taken from. I see above about turning stuff off etc......could someone re-explain what to turn off to a novice?

embyserver-63819596633.txt 2.61 MB · 0 downloads

Hi, as a test, if you remove your pictures library from emby server, does that resolve the issue?

[legierk 6]

Luke, I think it did. It has been staying steady at about 400MB RAM usage with two simultaneus streams going.

[Luke 37144]

55 minutes ago, legierk said:

Luke, I think it did. It has been staying steady at about 400MB RAM usage with two simultaneus streams going.

Thanks, That's your workaround for now, and the issue is resolved in the upcoming 4.8 release.

[legierk 6]

As an update, it is holding at ~400mb. It was still scanning the library this morning (starts at midnight), so I checked the log and it appears it is the Music library, which is one folder, that is causing the slowdown. Almost done (93%) after 9 hours. No issue though since the grandkids can still watch movies without hiccups. Was thinking of getting 16gb more ram, but now I'm thinking everything is OK. The Music folder is quite large, although I'm sure some of you have me beat.

[legierk 6]

FYI it finally finished scanning the music library. Nothing is streaming. Seems the memory usage is still abnormally high to me. IDK. Maybe I should get more RAM. (I accidently installed the AMD GPU, instead of the Nvidia it had, when I rebuilt this server a few days ago. I think that is why it has some errors on loading the Nvidia Codecs. If you guys think it would "work" better with the Nvidia 1050ti then I can swap it out no problem. Haven't had any issues streaming though since I removed the photo library.) That's a big txt file...maybe I shouldn't upload a file that big....lmk.

embyserver.txt

[Luke 37144]

Try restarting the server and see how things compare.

[legierk 6]

Restarted Emby this morning and went to work. No accessing the server today. RAM usage hovering at ~150mb. In all fairness, I believe when the ram usage was so high, it was the first time scanning the music (and photos prior to removing) library. (For context, I use Windows Storage Spaces for the storage pool. It's about 30tb total, so I think 15 usable...at any rate, I repurpose my older PCs for the "server" when I upgrade. So I recently moved the pool to a new PC. Actually I just swapped out the motherboards and OS drive. Reinstall Emby and try to remember my settings to put them in. This is one main reason I need to upgrade to Premiere. I have the folder art/pics in the folders so it doesn't have to keep redownloading everything. This only happens once every 2-3 years though....). Thanks for all the help. Will still probably add another 16gb ram, as it is constantly using about 40%. You guys are very responsive and design a very nice product. More reason for me to get Premiere.

[Edrock200 38]

On 5/13/2023 at 8:42 PM, Luke said:

Hi, as a test, if you remove your pictures library from emby server, does that resolve the issue?

Is it possible that the same bug could be in play if one uses side loaded posters/backgrounds/art work?

[Luke 37144]

53 minutes ago, Edrock200 said:

Is it possible that the same bug could be in play if one uses side loaded posters/backgrounds/art work?

No, it's only applicable to photo libraries.

[TMCsw 123]

21 hours ago, legierk said:

Will still probably add another 16gb ram, as it is constantly using about 40%.

Hu?? Why?

There is no benefit in having more free RAM available.

Although ‘swapping’ should be avoided both Windows and Linux do a rather good job of preemptively swapping out idle apps first (among the worst offenders are Web Browsers with many tabs open (just close them when done for now)).

Instead spoil your grand kids with the money you would have spent on the RAM…

[legierk 6]

2 hours ago, TMCsw said:

Hu?? Why?

There is no benefit in having more free RAM available.

Although ‘swapping’ should be avoided both Windows and Linux do a rather good job of preemptively swapping out idle apps first (among the worst offenders are Web Browsers with many tabs open (just close them when done for now)).

Instead spoil your grand kids with the money you would have spent on the RAM…

Hmm. Maybe I'm looking at it wrong. Using over 40% ram total right now, and PC ("server") is idle. Emby itself is idle and hogging ~400mb....it creeped back up, but seems to hang out at 400 or so. I guess that is where it is happy, just still seems like alot (compared to other apps). 16gb more of Gskill is $40, so it's not too bad. As far as the swapping goes, I only have to do that when the "hand-me-down" MB/CPU/RAM start to fizzle out and have hardware issues. The "server" runs 24/7....doesn't sleep. 

Side note, is there a way to have it sleep and when I access it, to "wake it up"? I see there is an option on the Emby Roku app to "Wake up server", but I've never tried it. Besides, I think my wife would have a fit if she had to wait 30 secs for the server to "wake up".

Edited May 17, 2023 by legierk
add photo

[TMCsw 123]

Well it’s up to you but at 40% when idle you are showing EmbyServer is using about 400+MB so if it was really busy it wouldn’t (short of a memory leek) use any where near 10x (4GB) more RAM keeping you under 100% RAM usage. (best way is check usage when servers is the busiest) So just close programs your not using, and if you are so inclined Google for how to de-bloat Windows.

To get the server to sleep then wake you need to enable ‘wake on LAN’ on the server bios (uefi).

If you do go ahead with the RAM upgrade make sure the kit is compatible with your motherboard (check there web support page) and current RAM.

Edited May 18, 2023 by TMCsw
typo