jabbera 23 Posted April 15, 2015 Share Posted April 15, 2015 Also with the dependency on imagemagick 64 bit versions will need to be supplied. Link to comment Share on other sites More sharing options...
ebr 14910 Posted April 15, 2015 Share Posted April 15, 2015 Also with the dependency on imagemagick 64 bit versions will need to be supplied. Why? If that were the case, the same thing would be needed for ffmpeg and a number of other components. Link to comment Share on other sites More sharing options...
jabbera 23 Posted April 15, 2015 Share Posted April 15, 2015 ffmpeg is an out of process executable. Imagemagick is loaded in process as a dll in the emby process space. This means bitness has to match. Link to comment Share on other sites More sharing options...
ebr 14910 Posted April 15, 2015 Share Posted April 15, 2015 ffmpeg is an out of process executable. Imagemagick is loaded in process as a dll in the emby process space. This means bitness has to match. It would. We are still a 32 bit app. I'm not suggesting we change that. Link to comment Share on other sites More sharing options...
saitoh183 137 Posted April 15, 2015 Share Posted April 15, 2015 It would. We are still a 32 bit app. I'm not suggesting we change that. Will people that have a large library be plagued with this type of problem always? Link to comment Share on other sites More sharing options...
jabbera 23 Posted April 15, 2015 Share Posted April 15, 2015 It would. We are still a 32 bit app. I'm not suggesting we change that. Ah. I misread your first post. Link to comment Share on other sites More sharing options...
primecut 19 Posted April 15, 2015 Share Posted April 15, 2015 How many items are in your library? Link to comment Share on other sites More sharing options...
ebr 14910 Posted April 15, 2015 Share Posted April 15, 2015 Will people that have a large library be plagued with this type of problem always? I'm sure we will continue to improve but pirate's library size is beyond just "large". It is way into the top 1% of what we will encounter. The vast majority of people should not have issues. Link to comment Share on other sites More sharing options...
saitoh183 137 Posted April 15, 2015 Share Posted April 15, 2015 I'm sure we will continue to improve but pirate's library size is beyond just "large". It is way into the top 1% of what we will encounter. The vast majority of people should not have issues. Ok, but why not move to 64bit...maybe eventually? Link to comment Share on other sites More sharing options...
ebr 14910 Posted April 15, 2015 Share Posted April 15, 2015 Ok, but why not move to 64bit...maybe eventually? Because not everyone has 64bit systems so that would mean we would have to maintain two separate builds across all platforms including separate builds of all the components we utilize. A massive development and maintenance task that is not worth it for the 1% of people it will affect. (sorry, I know when you are in that 1% that sux) Instead, the goal is to get working in as small a footprint as possible but, of course, there will be limitations to that as well when you are dealing with massive library sizes. Link to comment Share on other sites More sharing options...
jabbera 23 Posted April 15, 2015 Share Posted April 15, 2015 (edited) I think we should support 64 bit Windows personally. Many Linux users run 64 bit. (This is your workaround by the way @pir8radio). We could certainly package emby such that there is 1 install that can support both 32 and 64 bit windows if that was the issue. .NET is built to run 64 bit on 64 bit machines, and 32 bit on 32 bit machines. We force 32 bit on 64. (I don't know why historically.) Over 90% of new OEM computers run 64 bit windows. Windows Server 2012 is only available in 64 bit. While windows 10 will have a 32 bit build I truly expect that to be the last revision of 32 bit windows even sold. Edited April 15, 2015 by jabbera Link to comment Share on other sites More sharing options...
saitoh183 137 Posted April 15, 2015 Share Posted April 15, 2015 Because not everyone has 64bit systems so that would mean we would have to maintain two separate builds across all platforms including separate builds of all the components we utilize. A massive development and maintenance task that is not worth it for the 1% of people it will affect. (sorry, I know when you are in that 1% that sux) Instead, the goal is to get working in as small a footprint as possible but, of course, there will be limitations to that as well when you are dealing with massive library sizes. I think we should support 64 bit Windows personally. Many Linux users run 64 bit. (This is your workaround by the way @pir8radio). We could certainly package emby such that there is 1 install that can support both 32 and 64 bit windows if that was the issue. .NET is built to run 64 bit on 64 bit machines, and 32 bit on 32 bit machines. We force 32 bit on 64. (I don't know why historically.) Over 90% of new OEM computers run 64 bit windows. Windows Server 2012 is only available in 64 bit. While windows 10 will have a 32 bit build I truly expect that to be the last revision of 32 bit windows even sold. Yeah it does suck but jabbera has a valid point. Also most people that buy new systems get a 64bit os with it because no OEM PC is shipped with 32bit anymore. So unless there is a valid reason to run 32 bit or you personally installed 32bit, lots of people should be on 64bit. Link to comment Share on other sites More sharing options...
ebr 14910 Posted April 15, 2015 Share Posted April 15, 2015 But most people are not on brand new systems and many, many people who built htpcs used x86 to simplify the codec installs. And we have many servers running on htpcs. In short, we will not be able to abandon x86 systems for quite some time I imagine. Link to comment Share on other sites More sharing options...
jabbera 23 Posted April 15, 2015 Share Posted April 15, 2015 I don't think anyone is suggesting abandoning 32 bit. But you can run 64 bit on Linux. Why not windows? (Or turn off 64 bit in Linux so we truly only have to support 1 bitness) Link to comment Share on other sites More sharing options...
Luke 37053 Posted April 15, 2015 Share Posted April 15, 2015 well i don't think there's any reason to impose an artificial limitations on linux just because of windows. if you want to help do the work for 64-bit then yes it can be done. Link to comment Share on other sites More sharing options...
pir8radio 1292 Posted April 15, 2015 Author Share Posted April 15, 2015 (edited) How many items are in your library? @@primecut My stats as of a few months ago are in my signature at the bottom of this post. Yeah, once an OOM exception has been thrown, it is going to get re-thrown by anything that attempts to grab another chunk of memory (while the condition still exists) and images are one of the things most likely to need memory. However, if you look at your last log, the very first OOM that was thrown had nothing to do with images. It was attempting to do a recursive search of some part of your library related to doing an actual search via the API. Knowing that your library is on the ridiculous side in terms of size (number of items) you may simply be hitting a physical limit with our current design and the fact we are a 32bit app. Have you tried using the hack that allows a 32bit app to run in a larger memory space? @@ebr I have used it in the past... I will certainly try it again But i monitor Emby it i never see it go above 1.2 gigs of ram.. even when these OOM errors happen i quickly check. What is the upper limit, i thought 2? Usually after it calms down it settles around 950 megs of ram usage. I used to get OOM errors when the app pushed the 2 gig mark... and the 4gig hack worked then.. but this seems to throw OOM errors at lower levels of ram usage and sometimes it crashes out, sometimes it pops an OOM window on my server. If i leave the window alone other OOM windows will pop up and the server keeps running.. some times it crashes out. Weird.. Ill try the hack first, but let me know what you think of the 1.2 gig usage.. I would actually like to see a 64 bit version.. EBR can you guys just apply the 4 gig hack out of the box, so we don't have to keep re-doing it on our end? Shouldn't affect 32bit users if i recall correctly that flag is ignored on a 32 bit machine is it? As much as I hate to say it, @@Luke and ebr are correct, about maintaining two versions, blah blah... I also understand you cant tend to the 1% (though I annoy the piss out of both of them every time I have an issue) If they jumped on every suggestion out there from every person eMBy would be a mess... Maybe I should just use a linux VM and run eMBy within that? I just tried the 4 gig patch and things seem somewhat stable, I'll wait and see when users start loading up the server. Edited April 16, 2015 by pir8radio Link to comment Share on other sites More sharing options...
Happy2Play 8279 Posted April 16, 2015 Share Posted April 16, 2015 (edited) How many items are in your library? My stats as of a few months ago are in my signature at the bottom of this post. Is it just me, but I don't see any ones signatures. **(found the option in my settings) There must be an issue with small files like mp3's. I have almost 30Tb of Movies and TV (5585 movies, 580 series with 35,366 episodes) and haven't seen a OOM error. I don't monitor it continuously but haven't notice the ram usage ever over 400Mb. Edited April 16, 2015 by Happy2Play Link to comment Share on other sites More sharing options...
pir8radio 1292 Posted April 16, 2015 Author Share Posted April 16, 2015 (edited) Weird i see every ones signatures... @@Happy2Play Movies 335 (full uncompressed rips, vary from 1 gig to 30 gig each) Music videos about 13,000 MP3's about 143,000 Been doing some house cleaning so my actual files are a little below those counts +- 800 Edited April 16, 2015 by pir8radio Link to comment Share on other sites More sharing options...
ebr 14910 Posted April 16, 2015 Share Posted April 16, 2015 Yes the total heap size is 2G but you can have an OOM error even when your total heap usage is well below that. It is all about getting a contiguous block of memory of the size needed and the heap may be very fragmented and not have it available. So, the hack should help you out. I'm sure we'll have a 64bit version at some point it just isn't a big enough problem for the user base as a whole right now to pull us off of other things. Luke has, instead, been concentrating on more efficient use of memory so that we can fit on smaller devices. If someone else can come up with a low-maintenance way of us building both 32 and 64 bit versions of the server then I'm sure we'd welcome that contribution. 1 Link to comment Share on other sites More sharing options...
saitoh183 137 Posted April 16, 2015 Share Posted April 16, 2015 (edited) Yes the total heap size is 2G but you can have an OOM error even when your total heap usage is well below that. It is all about getting a contiguous block of memory of the size needed and the heap may be very fragmented and not have it available. So, the hack should help you out. I'm sure we'll have a 64bit version at some point it just isn't a big enough problem for the user base as a whole right now to pull us off of other things. Luke has, instead, been concentrating on more efficient use of memory so that we can fit on smaller devices. If someone else can come up with a low-maintenance way of us building both 32 and 64 bit versions of the server then I'm sure we'd welcome that contribution. Maybe a temporary solution would be to included the Hack as an option in the MB installer because the only annoying thing about the hack is that you have to not forget to apply it after each server patch. In a fresh install, it could have a warning next to the option stating that this can cause instability and is experimental...or anything along that line that will warn uses before hand. For updates since there is no installer, you could maybe add a option in the same place we choose the server branch that will tell the installer to do the hack. Edited April 16, 2015 by saitoh183 Link to comment Share on other sites More sharing options...
ebr 14910 Posted April 16, 2015 Share Posted April 16, 2015 The updates are automatic and unattended so we can't have any options. Link to comment Share on other sites More sharing options...
saitoh183 137 Posted April 16, 2015 Share Posted April 16, 2015 The updates are automatic and unattended so we can't have any options. @@ebr Ah ok...oh well just got to live with it Link to comment Share on other sites More sharing options...
ebr 14910 Posted April 16, 2015 Share Posted April 16, 2015 What exactly is involved in the "hack"? I've never investigated it. Just gimme a link if you have one. Link to comment Share on other sites More sharing options...
Solution pir8radio 1292 Posted April 16, 2015 Author Solution Share Posted April 16, 2015 (edited) What exactly is involved in the "hack"? I've never investigated it. Just gimme a link if you have one. @@ebr here is a link to the program that edits the exe for you: http://www.ntcore.com/4gb_patch.php I think this is what flag the above program is setting not 100% sure though: https://msdn.microsoft.com/en-us/library/wz223b1z.aspx This may help too, its C# specific: http://stackoverflow.com/questions/2597790/can-i-set-largeaddressaware-from-within-visual-studio Edited April 16, 2015 by pir8radio 3 Link to comment Share on other sites More sharing options...
ebr 14910 Posted April 17, 2015 Share Posted April 17, 2015 Luke - that build event looks pretty simple, what do you think about doing this to the server? Are there any downsides to it? Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now