Jump to content


Photo

Emby OUT OF MEMORY ERRORS


Best Answer pir8radio , 16 April 2015 - 07:18 PM

What exactly is involved in the "hack"?  I've never investigated it.  Just gimme a link if you have one.

 

@ebr here is a link to the program that edits the exe for you:

http://www.ntcore.com/4gb_patch.php

 

I think this is what flag the above program is setting not 100% sure though:

https://msdn.microso...y/wz223b1z.aspx

 

This may help too, its C# specific:

http://stackoverflow...n-visual-studio

Go to the full post


  • Please log in to reply
94 replies to this topic

#41 pir8radio OFFLINE  

pir8radio

    NGINX

  • Members
  • 2927 posts
  • Local time: 04:13 AM
  • LocationChicago

Posted 15 April 2015 - 07:41 PM

How many items are in your library? 

@primecut My stats as of a few months ago are in my signature at the bottom of this post.

 

Yeah, once an OOM exception has been thrown, it is going to get re-thrown by anything that attempts to grab another chunk of memory (while the condition still exists) and images are one of the things most likely to need memory.

 

However, if you look at your last log, the very first OOM that was thrown had nothing to do with images.  It was attempting to do a recursive search of some part of your library related to doing an actual search via the API.

 

Knowing that your library is on the ridiculous side in terms of size (number of items) you may simply be hitting a physical limit with our current design and the fact we are a 32bit app.  Have you tried using the hack that allows a 32bit app to run in a larger memory space?

 

@ebr I have used it in the past...  I will certainly try it again But i monitor Emby it i never see it go above 1.2 gigs of ram.. even when these OOM errors happen i quickly check.   What is the upper limit, i thought 2?    Usually after it calms down it settles around 950 megs of ram usage.   I used to get OOM errors when the app pushed the 2 gig mark... and the 4gig hack worked then..  but this seems to throw OOM errors at lower levels of ram usage and sometimes it crashes out, sometimes it pops an OOM window on my server.  If i leave the window alone other OOM windows will pop up and the server keeps running.. some times it crashes out.   Weird..    Ill try the hack first, but let me know what you think of the 1.2 gig usage..     I would actually like to see a 64 bit version..   EBR can you guys just apply the 4 gig hack out of the box, so we don't have to keep re-doing it on our end?  Shouldn't affect 32bit users if i recall correctly that flag is ignored on a 32 bit machine is it?

 

As much as I hate to say it, @Luke and ebr are correct, about maintaining two versions, blah blah...   :)   I also understand you cant tend to the 1%  (though I annoy the piss out of both of them every time I have an issue)  If they jumped on every suggestion out there from every person eMBy would be a mess...      

 

Maybe I should just use a linux VM and run eMBy within that?       

 

I just tried the 4 gig patch and things seem somewhat stable, I'll wait and see when users start loading up the server.    


Edited by pir8radio, 15 April 2015 - 08:14 PM.


#42 Happy2Play OFFLINE  

Happy2Play

    Trial and Error

  • Moderators
  • 16268 posts
  • Local time: 02:13 AM
  • LocationWashington State

Posted 15 April 2015 - 08:30 PM

How many items are in your library? 

My stats as of a few months ago are in my signature at the bottom of this post.

Is it just me, but I don't see any ones signatures. **(found the option in my settings)

 

There must be an issue with small files like mp3's.  I have almost 30Tb of Movies and TV (5585 movies, 580 series with 35,366 episodes) and haven't seen a OOM error.  I don't monitor it continuously but haven't notice the ram usage ever over 400Mb.


Edited by Happy2Play, 15 April 2015 - 09:34 PM.


#43 pir8radio OFFLINE  

pir8radio

    NGINX

  • Members
  • 2927 posts
  • Local time: 04:13 AM
  • LocationChicago

Posted 15 April 2015 - 09:09 PM

Weird i see every ones signatures... @Happy2Play   

 

Movies 335 (full uncompressed rips, vary from 1 gig to 30 gig each)

Music videos about 13,000

MP3's about 143,000

 

Been doing some house cleaning so my actual files are a little below those counts +- 800


Edited by pir8radio, 15 April 2015 - 09:10 PM.


#44 ebr OFFLINE  

ebr

    Chief Bottle Washer

  • Administrators
  • 47681 posts
  • Local time: 05:13 AM

Posted 16 April 2015 - 11:19 AM

Yes the total heap size is 2G but you can have an OOM error even when your total heap usage is well below that.  It is all about getting a contiguous block of memory of the size needed and the heap may be very fragmented and not have it available.  So, the hack should help you out.

 

I'm sure we'll have a 64bit version at some point it just isn't a big enough problem for the user base as a whole right now to pull us off of other things.  Luke has, instead, been concentrating on more efficient use of memory so that we can fit on smaller devices.  

 

If someone else can come up with a low-maintenance way of us building both 32 and 64 bit versions of the server then I'm sure we'd welcome that contribution.


  • Angelblue05 likes this

#45 saitoh183 OFFLINE  

saitoh183

    Advanced Member

  • Members
  • 997 posts
  • Local time: 06:13 AM
  • LocationCanada, St-Eustache, QC

Posted 16 April 2015 - 12:20 PM

Yes the total heap size is 2G but you can have an OOM error even when your total heap usage is well below that.  It is all about getting a contiguous block of memory of the size needed and the heap may be very fragmented and not have it available.  So, the hack should help you out.

 

I'm sure we'll have a 64bit version at some point it just isn't a big enough problem for the user base as a whole right now to pull us off of other things.  Luke has, instead, been concentrating on more efficient use of memory so that we can fit on smaller devices.  

 

If someone else can come up with a low-maintenance way of us building both 32 and 64 bit versions of the server then I'm sure we'd welcome that contribution.

 

Maybe a temporary solution would be to included the Hack as an option in the MB installer because the only annoying thing about the hack is that you have to not forget to apply it after each server patch. In a fresh install, it could have a warning next to the option stating that this can cause instability and is experimental...or anything along that line that will warn uses before hand. For updates since there is no installer, you could maybe add a option in the same place we choose the server branch that will tell the installer to do the hack.


Edited by saitoh183, 16 April 2015 - 12:21 PM.


#46 ebr OFFLINE  

ebr

    Chief Bottle Washer

  • Administrators
  • 47681 posts
  • Local time: 05:13 AM

Posted 16 April 2015 - 03:50 PM

The updates are automatic and unattended so we can't have any options.



#47 saitoh183 OFFLINE  

saitoh183

    Advanced Member

  • Members
  • 997 posts
  • Local time: 06:13 AM
  • LocationCanada, St-Eustache, QC

Posted 16 April 2015 - 04:00 PM

The updates are automatic and unattended so we can't have any options.

 

@ebr

 

Ah ok...oh well just got to live with it :)



#48 ebr OFFLINE  

ebr

    Chief Bottle Washer

  • Administrators
  • 47681 posts
  • Local time: 05:13 AM

Posted 16 April 2015 - 04:43 PM

What exactly is involved in the "hack"?  I've never investigated it.  Just gimme a link if you have one.



#49 pir8radio OFFLINE  

pir8radio

    NGINX

  • Members
  • 2927 posts
  • Local time: 04:13 AM
  • LocationChicago

Posted 16 April 2015 - 07:18 PM   Best Answer

What exactly is involved in the "hack"?  I've never investigated it.  Just gimme a link if you have one.

 

@ebr here is a link to the program that edits the exe for you:

http://www.ntcore.com/4gb_patch.php

 

I think this is what flag the above program is setting not 100% sure though:

https://msdn.microso...y/wz223b1z.aspx

 

This may help too, its C# specific:

http://stackoverflow...n-visual-studio


Edited by pir8radio, 16 April 2015 - 07:23 PM.

  • skidmarks, brainfryd and TonkaTuff like this

#50 ebr OFFLINE  

ebr

    Chief Bottle Washer

  • Administrators
  • 47681 posts
  • Local time: 05:13 AM

Posted 17 April 2015 - 09:46 AM

Luke - that build event looks pretty simple, what do you think about doing this to the server?

 

Are there any downsides to it?



#51 jabbera OFFLINE  

jabbera

    Advanced Member

  • Developers
  • 109 posts
  • Local time: 05:13 AM
  • LocationBoston, MA

Posted 17 April 2015 - 05:40 PM

Some (old) unmanaged code would use the top bit of a 32 bit pointer to actually maintain some program state about the points to object. I've never ran into one of these. I would be shocked if there was any side effect of making Emby large address aware.

#52 pir8radio OFFLINE  

pir8radio

    NGINX

  • Members
  • 2927 posts
  • Local time: 04:13 AM
  • LocationChicago

Posted 18 April 2015 - 12:41 PM

@Luke to save the day.....    :D



#53 pir8radio OFFLINE  

pir8radio

    NGINX

  • Members
  • 2927 posts
  • Local time: 04:13 AM
  • LocationChicago

Posted 23 April 2015 - 07:32 AM

@ebr an update, my server has been running flawlessly since the 4gb "hack"....      If that is something that can get worked into the releases/updates that would be great...  I think that will allow you guys to get way more life out of the 32bit version. 



#54 mediacowboy OFFLINE  

mediacowboy

    Advanced Member

  • Alpha Testers
  • 1825 posts
  • Local time: 05:13 AM
  • LocationTexas, United States

Posted 23 April 2015 - 09:59 PM

I am seeing a bunch of these. Attached are my log's. I am running Emby server 3.0.5588.1. These logs are from 4/23/2015 4:34PM to 8:51 PM.

Attached Files



#55 AgileHumor OFFLINE  

AgileHumor

    Advanced Member

  • Alpha Testers
  • 759 posts
  • Local time: 02:13 AM

Posted 24 April 2015 - 11:29 AM

well i don't think there's any reason to impose an artificial limitations on linux just because of windows. if you want to help do the work for 64-bit then yes it can be done.

You are imposing a limit for 64 bit users (large majority of new Emby installs) unless you change the compiling process (which will still work for 32 bit and 64 bit systems):

http://stackoverflow...bit-windows-os 

 

 

Have you tried using the hack that allows a 32bit app to run in a larger memory space?

 

The 3GB hack only works on 32 bit systems needing to address larger then 2GB of app ram...doesn't work in 64 bit OS (~90% of new Emby installs)

 

I have not tested this utility, but it might help fix the compiler flag in the App without the Dev team making changes.

http://www.maketeche...dows-64-bit-os/

 

 

PS - Amazing that 32 bit because "codecs" are easier to install (which is not my experience in 2015, maybe 2005)...the platform is limited for large libraries. 


Edited by AgileHumor, 24 April 2015 - 03:15 PM.


#56 saitoh183 OFFLINE  

saitoh183

    Advanced Member

  • Members
  • 997 posts
  • Local time: 06:13 AM
  • LocationCanada, St-Eustache, QC

Posted 24 April 2015 - 11:36 AM

PSS - If you are on 32 bit and need to use the hack, you might get some corruption issues on video playback as shown here:

http://www.techrepub...ws-81-systems/ 

 

dead link



#57 saitoh183 OFFLINE  

saitoh183

    Advanced Member

  • Members
  • 997 posts
  • Local time: 06:13 AM
  • LocationCanada, St-Eustache, QC

Posted 24 April 2015 - 11:38 AM

The 3GB hack only works on 32 bit systems needing to adress larger then 2GB of app ram...doesn't work in 64 bit OS (~90% of new Emby installs)

 

I have not tested this utility, but it might help fix the compiler flag in the App without the Dev team making changes.

http://www.maketeche...dows-64-bit-os/

 

 

For the people that were having problems, that was what we were already using and is also what we are discussing in this thread



#58 ebr OFFLINE  

ebr

    Chief Bottle Washer

  • Administrators
  • 47681 posts
  • Local time: 05:13 AM

Posted 24 April 2015 - 12:02 PM

(~90% of new Emby installs)

 

Where are you getting that metric?



#59 AgileHumor OFFLINE  

AgileHumor

    Advanced Member

  • Alpha Testers
  • 759 posts
  • Local time: 02:13 AM

Posted 24 April 2015 - 12:19 PM

dead link

 

Link is correct, the forum added a white space at end which their sever can't accept.  I tried again without whitespace...but if that doesn't work...just delete the extra character to view the page on TechRepub weird web server. 

 

http://www.techrepub...ows-81-systems/


Edited by AgileHumor, 24 April 2015 - 12:30 PM.


#60 AgileHumor OFFLINE  

AgileHumor

    Advanced Member

  • Alpha Testers
  • 759 posts
  • Local time: 02:13 AM

Posted 24 April 2015 - 12:21 PM

For the people that were having problems, that was what we were already using and is also what we are discussing in this thread

 

 

Sorry, I missed pirates post of the links.    Got super anxious that months of server instability could be attributed to such a simple fix while reading the threads from #1....scanned to much at the end.   Was also shocked that large library stability issues we're specifically being skipped.  Wish I knew that before posting logs over many months to have it ignored.   Now that I know, it saves us all time from reading or posting. 


Edited by AgileHumor, 24 April 2015 - 12:40 PM.





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users