RAM memory leak in the Embe process.

[Santrex 7]

2 hours ago, ebr said:

Can you just try it as a test so we can eliminate it as a possible issue?

Yes, I'm talking to management right now.

[Santrex 7]

We have removed the plugin. The EMBY service was restarted and we began to observe the dynamics of RAM. Unfortunately, it didn't help.
So far, the only reason we think of is the frequent update of the entire MediaLibrary EMBY database (once every 5 minutes), with an average scan of up to 2.30 minutes.
We will try to switch to a "point" update of the folder directories "inside" the libraries, but this will take time.
While the real reason is not visible. We will continue to watch.
It seemed to us that removing the plugin reduced the load on RAM, but did not completely remove it.

LogEmbyRAM4.7z

[Luke 42078]

Are you able to update to Emby Server 4.7.9 and see if that helps?

[Santrex 7]

2 hours ago, Luke said:

Are you able to update to Emby Server 4.7.9 and see if that helps?

We updated one of our servers to 4.7.9 as part of the test (we will observe)
Were there any significant changes? We also do not have the oldest version...

Edited November 17, 2022 by Santrex

[joshhuggins 22]

I just built a new server and I think I was seeing the same thing with 4.7.9 on Win11 22H2 (22621.819). For me it seems like it's tied to something with the scanning of the media library, and possibly related to the issue previously reported back in this post regarding Photo Libraries? I removed my Photo Library and the issue went away.

TL;DR version

I did my initial install / setup of 4.7.9, added in my Premiere key info and then loaded the Server Config Backup plugin. It's the only plug-in I add in these days. I restored my server settings with the plugin. The server has been on 1-2 days which normally should be long enough to process my library after a reload. I noticed earlier today that several libraries were not displaying everything but I didn't have time to look into it & thought it might be a library path that needed to be fixed.

Came back to the system this evening and noticed the audio cutting out on my wireless headset while watching some YouTube. Trying to figure out what was going on, I noticed that my main OS SSD was peaking at 100% Active time in Task Manager. Switched over to Processes to see what was pegging the SSD and it was the main System Process (PID:4) , then noticed that my RAM was maxed out at 98%-100% usage and found Emby Server maxing it out.

Did a little poking at things and noticed in the Scheduled Tasks that the Scan Media Library was running and seemed to be stuck at 92%. Tried to stop the scan but while it did show it was trying to stop, it never did. So I crashed the server and restarted it. Watched the Scheduled Tasks page and the Task Manager and sure enough as soon as the Scan Media library task started the RAM usage started to rapidly shoot up into over 10G range and the progress halted at 92% so I shut it down again. I repeated this a couple times and then I removed the 12hr Task Trigger and restarted the server and the RAM usage never ramped up again unless I manually started the scan media library again.

Took a look at the logs, and after a few library path cleanups and the issue still sticking around, I remembered the previous issue with Photo Libraries mentioned above. I then removed my Photo Library and when I started the Scan, it launched fine and hovers right at 300-400 MB of RAM usage and the progress bar has started to report steady progress past 92%.

I came here to report it when I saw this post and figured I would add on here instead of a new post in case it might be Santrex's issue as well. I had been on the beta builds since the old post above and did not have issues with my Photo Library since then, so not sure where this might have crept in and showed up only after my reload unless this is just another bug that looks like the first one. Anywho TL:DR goal reached. 

embyserverlogs.rar

Edited November 19, 2022 by joshhuggins
Spelling

[rbjtech 5284]

3 hours ago, joshhuggins said:

I just built a new server and I think I was seeing the same thing with 4.7.9 on Win11 22H2 (22621.819). For me it seems like it's tied to something with the scanning of the media library, and possibly related to the issue previously reported back in this post regarding Photo Libraries? I removed my Photo Library and the issue went away.

TL;DR version

I did my initial install / setup of 4.7.9, added in my Premiere key info and then loaded the Server Config Backup plugin. It's the only plug-in I add in these days. I restored my server settings with the plugin. The server has been on 1-2 days which normally should be long enough to process my library after a reload. I noticed earlier today that several libraries were not displaying everything but I didn't have time to look into it & thought it might be a library path that needed to be fixed.

Came back to the system this evening and noticed the audio cutting out on my wireless headset while watching some YouTube. Trying to figure out what was going on, I noticed that my main OS SSD was peaking at 100% Active time in Task Manager. Switched over to Processes to see what was pegging the SSD and it was the main System Process (PID:4) , then noticed that my RAM was maxed out at 98%-100% usage and found Emby Server maxing it out.

Did a little poking at things and noticed in the Scheduled Tasks that the Scan Media Library was running and seemed to be stuck at 92%. Tried to stop the scan but while it did show it was trying to stop, it never did. So I crashed the server and restarted it. Watched the Scheduled Tasks page and the Task Manager and sure enough as soon as the Scan Media library task started the RAM usage started to rapidly shoot up into over 10G range and the progress halted at 92% so I shut it down again. I repeated this a couple times and then I removed the 12hr Task Trigger and restarted the server and the RAM usage never ramped up again unless I manually started the scan media library again.

Took a look at the logs, and after a few library path cleanups and the issue still sticking around, I remembered the previous issue with Photo Libraries mentioned above. I then removed my Photo Library and when I started the Scan, it launched fine and hovers right at 300-400 MB of RAM usage and the progress bar has started to report steady progress past 92%.

I cam here to report it when I saw this post and figured I would add on here instead of a new post in case it might be Santrex's issue as well. I had been on the beta builds since the old post above and did not have issues with my Photo Library since then, so not sure where this might have crept in and showed up only after my reload unless this is just another bug that looks like the first one. Anywho TL:DR goal reached. 

embyserverlogs.rar 3.44 MB · 0 downloads

Great work - I too had this issue and removed my Photo Library to resolve it.  This was still an issue on 4.8.0.16 beta - I've since upgraded to .18 beta.

@Luke I believe you are aware of this issue ? - I understand it to be an issue with a 3rd party library on scanning images/photos ?   

Crashing servers is not good - please can you advise if you believe this is fixed as I did not see anything in the release notes.

If it's an outstanding/unresolved issue - then maybe add some logic to kill the scan it if it gets in this loop ?

[Luke 42078]

That's resolved in 4.8. thanks.

[Santrex 7]

In my case, all my "MediaLibrary" are "TYPE" - "Video clips"
I can't be sure exactly what "Materials" are in the "Scans" folders. So I don't fill them up.
Perhaps there is "Photo-materials" or something like that.
Maybe I need to change the "TYPE" for my "MediaLibrary" so that only "Video Materials" gets there. My main format is "MXF".
Which "TYPE" should I choose?In my case all my "MediaLibrary" are "Video clips".
I can't be sure exactly what "Materials" are in the "Scans" folders. So I don't fill them up.
Perhaps there is "Photo-materials" or something like that.
Maybe I need to change the "TYPE" for my "MediaLibrary" so that only "Video Materials" gets there. My main format is "MXF".
Which "TYPE" should I choose?

[Luke 42078]

7 hours ago, Santrex said:

In my case, all my "MediaLibrary" are "TYPE" - "Video clips"
I can't be sure exactly what "Materials" are in the "Scans" folders. So I don't fill them up.
Perhaps there is "Photo-materials" or something like that.
Maybe I need to change the "TYPE" for my "MediaLibrary" so that only "Video Materials" gets there. My main format is "MXF".
Which "TYPE" should I choose?In my case all my "MediaLibrary" are "Video clips".
I can't be sure exactly what "Materials" are in the "Scans" folders. So I don't fill them up.
Perhaps there is "Photo-materials" or something like that.
Maybe I need to change the "TYPE" for my "MediaLibrary" so that only "Video Materials" gets there. My main format is "MXF".
Which "TYPE" should I choose?

You could do home videos and uncheck the option to import photos.

[Flieger42 1]

So I´m running 4.7.10 fresh installed yesterday on a win11 (server). Have the same problem of filling the RAM above 90% when scanning Photo-Library. 

Is it possible to upgrade to a beta to get rid of this issue?
 

 

[Luke 42078]

45 minutes ago, Flieger42 said:

So I´m running 4.7.10 fresh installed yesterday on a win11 (server). Have the same problem of filling the RAM above 90% when scanning Photo-Library. 

Is it possible to upgrade to a beta to get rid of this issue?
 

 

Hi, yes, instructions can be found here: https://emby.media/windows-server.html

Thanks.

[Flieger42 1]

Ok. installed 4.8.0.19 Beta and it worked! Thanks.

[legierk 8]

I jsut posted in another thread re this RAM leakage/usage. I just deleted my photo library folder to see if that helps. As I type, RAM usage is at 343MB and holding, as my grandkids are watching two different movies in two different rooms. It is also doing a library scan (I guess since I just deleted the Photos folder?)  I am on 4.7.11.0. Not sure if I want to go to beta. Perhaps 4.8 will be released soon?

[Luke 42078]

1 hour ago, legierk said:

I jsut posted in another thread re this RAM leakage/usage. I just deleted my photo library folder to see if that helps. As I type, RAM usage is at 343MB and holding, as my grandkids are watching two different movies in two different rooms. It is also doing a library scan (I guess since I just deleted the Photos folder?)  I am on 4.7.11.0. Not sure if I want to go to beta. Perhaps 4.8 will be released soon?

Hi, we are working hard on getting 4.8 ready for release. Thanks.