Brendon 101 Posted March 28 Share Posted March 28 This has happened about a dozen times since the last update, running Emby on Windows Server 2016, i get up in the morning and the whole thing is non-responsive, i log into the server remotely and Emby is tearing through over 10GB of RAM??? I have attached screenshots of the process tree showing how much its using one two occasions and the appropriate logs, can anyone tell me what the hell is going on, this never happened before 4.8 embyserver-63846178750.txt embyserver.txt embyserver-63846144001.txt Link to comment Share on other sites More sharing options...
Luke 37066 Posted March 28 Share Posted March 28 HI, please try removing these plugins: Actor Web Scraper 1.0.0.325 Episode Refresh 1.0.0.1 Guest Star Cleaner 0.1.0.1 Playback Reporting 2.1.0.5 Statistics 3.2.0.0 TimeLordTV 19.8.3.1105 TMDB People Fix 1.0.0.136 Then restart the server and see how things compare. Link to comment Share on other sites More sharing options...
Ronstang 202 Posted March 28 Share Posted March 28 RAM is cheap, this is why I have 48 Gb in this rig and 40 Gb in my beta machine. RAM is never an issue but hard drive space is if you don't keep an eye on the transcoding temp folder. Link to comment Share on other sites More sharing options...
Brendon 101 Posted March 28 Author Share Posted March 28 3 minutes ago, Ronstang said: RAM is cheap, this is why I have 48 Gb in this rig and 40 Gb in my beta machine. RAM is never an issue but hard drive space is if you don't keep an eye on the transcoding temp folder. Been running fine on what it has for years, buying more is not an answer!! Link to comment Share on other sites More sharing options...
Brendon 101 Posted March 28 Author Share Posted March 28 38 minutes ago, Luke said: HI, please try removing these plugins: Actor Web Scraper 1.0.0.325 Episode Refresh 1.0.0.1 Guest Star Cleaner 0.1.0.1 Playback Reporting 2.1.0.5 Statistics 3.2.0.0 TimeLordTV 19.8.3.1105 TMDB People Fix 1.0.0.136 Then restart the server and see how things compare. Thanks Luke, do i just remove them all and put back one by one and see what hits hard?? Like i said was all fine before 4.8 and i have added none of the above since 4.8 Link to comment Share on other sites More sharing options...
Ronstang 202 Posted March 28 Share Posted March 28 (edited) 7 minutes ago, Brendon said: Been running fine on what it has for years, buying more is not an answer!! Mine was too, until it wasn't. We keep adding jobs for our machines to handle and expecting more and more with the same HW, unfortunately at some point that catches up to you. When I built this new machine last year I ended up with some left over incompatible RAM I had added to the old machine trying to get it to do what I wanted. I stuck that in my beta machine which at the time was doing the lion's share of my encoding and when I did I realized how much I was holding myself back with not much RAM. My encoding needs more RAM than I realized and my speeds increased by over 50%. My current combo came with 16 Gb, and once I found this out I added another 32 Gb and I never have memory issues now. I'm on a tight budget too but for less than $100 I consider that a win. Edited March 28 by Ronstang Link to comment Share on other sites More sharing options...
Brendon 101 Posted March 28 Author Share Posted March 28 2 minutes ago, Ronstang said: Mine was too, until it wasn't. We keep adding jobs for our machines to handle and expecting more and more with the same HW, unfortunately at some point that catches up to you. When I built this new machine last year I ended up with some left over incompatible RAM I had added to the old machine trying to get it to do what I wanted. I stuck that in my beta machine which at the time was doing the lion's share of my encoding and when I did I realized how much I was holding myself back with not much RAM. My encoding needs more RAM than I realized and my speeds increased by over 50%. My current combo came with 16 Gb, and once I found this out I added another 32 Gb and I never have memory issues now. I'm on a tight budget too but for less than $100 I consider that a win. I know how machines work mate, i work in IT, if i felt it needed more RAM i would put it in but i don't feel i do, why is it spinning up and demanding so much RAM when there is literally no-one using it??? I have 15 family members streaming and transcoding throughout the day and evening and it doesn't use that much RAM, throwing RAM at it is not solving the problem it just gives it more room to grow and keep happening. Link to comment Share on other sites More sharing options...
Ronstang 202 Posted March 28 Share Posted March 28 Hopefully disabling the plugins will solve the problem then, but if they are the problem and you like and need them there likely isn't another option unless the authors find a way to plug the memory leaks. I am forced to use a proxy for my Ceton cards. It is a memory pig, but there is no other option if I want to use my tuner. If I leave it running nonstop eventually the plugin shuts itself down and I loose recordings. Even memory didn't solve it so I was forced to use task manager to automatically shut down and restart the proxy once a day to prevent the problem. So each night before bed I have to check upcoming recordings and make sure the time my proxy restarts doesn't conflict, if it does I have to change the trigger each night. It's a pain but what I have to do to use emby. So if it's the plugins and you need or want them you have a choice. I run no plugins except the ones emby installs by default and emby never uses RAM up on either of my two servers. Link to comment Share on other sites More sharing options...
Luke 37066 Posted March 28 Share Posted March 28 2 hours ago, Brendon said: Thanks Luke, do i just remove them all and put back one by one and see what hits hard?? Like i said was all fine before 4.8 and i have added none of the above since 4.8 I would remove them and keep them removed and then just observe. Link to comment Share on other sites More sharing options...
JoLarsson 19 Posted March 28 Share Posted March 28 Weird, I've been running my Emby server on 16GB of ram for years and the EmbyServer process never spikes. It's 14 days since I rebooted my server and EmbyServer process is currently at 210 MB. My users are playing roughly 7 movies and 20-30 tv-episodes per day/total. All transcoding. I'm also on the latest stable version. It must be something else "unique" causing this or there would be hundreds of posts here by now. Of the plugins you have I'm only using Playback Reporting Link to comment Share on other sites More sharing options...
visproduction 122 Posted March 28 Share Posted March 28 From the log File about 12 errors at 4 per second: Quote *** Error Report *** Version: 4.8.3.0 Command line: C:\Users\Administrator\AppData\Roaming\Emby-Server\system\EmbyServer.dll ... Source: Microsoft.AspNetCore.Server.Kestrel.Core TargetSite: Void Throw(Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.RequestRejectionReason) Related? https://stackoverflow.com/questions/61154053/how-to-resolve-microsoft-aspnetcore-server-kestrel-core-badhttprequestexception Link to comment Share on other sites More sharing options...
Brendon 101 Posted March 30 Author Share Posted March 30 On 28/03/2024 at 20:40, JoLarsson said: Weird, I've been running my Emby server on 16GB of ram for years and the EmbyServer process never spikes. It's 14 days since I rebooted my server and EmbyServer process is currently at 210 MB. My users are playing roughly 7 movies and 20-30 tv-episodes per day/total. All transcoding. I'm also on the latest stable version. It must be something else "unique" causing this or there would be hundreds of posts here by now. Of the plugins you have I'm only using Playback Reporting Same mate, this is sudden, hence looking for answers. Link to comment Share on other sites More sharing options...
Brendon 101 Posted March 30 Author Share Posted March 30 On 29/03/2024 at 03:01, visproduction said: From the log File about 12 errors at 4 per second: Related? https://stackoverflow.com/questions/61154053/how-to-resolve-microsoft-aspnetcore-server-kestrel-core-badhttprequestexception Looking into it, thank you. Link to comment Share on other sites More sharing options...
Brendon 101 Posted March 30 Author Share Posted March 30 On 28/03/2024 at 14:40, Luke said: I would remove them and keep them removed and then just observe. I am pulling them out one by one @Luke, some of the ones you mentioned never even run, they have no schedule and are manually controlled. I have taken statistics out first and it spiked over 10GB again this morning and locked up, i will remove another now and keep monitoring. Link to comment Share on other sites More sharing options...
Gilgamesh_48 943 Posted March 31 Share Posted March 31 Could this be related to: Spoiler Link to comment Share on other sites More sharing options...
Brendon 101 Posted March 31 Author Share Posted March 31 @Lukestill testing mate but it happened twice within 12hrs now and this morning i got a new message from the server not seen before. So far i have removed; Actor Web Scraper 1.0.0.325 and Statistics 3.2.0.0 (neither made a difference) and have just now uninstalled Guest Star Cleaner 0.1.0.1 1 Link to comment Share on other sites More sharing options...
JoLarsson 19 Posted April 2 Share Posted April 2 (edited) Maybe it's a broken Visual C++ or something in windows that have been patched or corrupted. It's a longshot but I have seen stranger things than that in our server farms at work. On my emby server I have only Microsoft Visual C++ 2015-2022 Redistributable (both x64 and x86) installed. I'm really not sure what emby server needs but a lot of the C++ redists are end of life so only 2013 and 2015-2022 are still active. Edited April 2 by JoLarsson 1 Link to comment Share on other sites More sharing options...
Brendon 101 Posted April 10 Author Share Posted April 10 @LukeI removed the plugins requested and was still seeing the issue mate, something else i noticed is that my library scans were taking an insane amount of time, i am talking 2 days to get halfway and they were running at the default setting, i disabled that and change it to run once a week on Sunday at midnight, it seemed to get stuck around the 40% mark and spent the next 24hrs for another 4% before i cancelled it, having thought that maybe the problem i disabled it but i am still seeing issues. Twice now this week in the peak evening hours emby has just locked up, everyones streams frozen and my windows server shut the main embyserver process down as it was out of recourses and left the embytray.exe running, I have put a couple of essential plugins back in after i established they weren't the problem but they are not set to autorun at all. When it locked up last night i grabbed the error message from windows event viewer as well as the emby logs (attached) The Event Viewer error.... - <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"> - <System> <Provider Name="Microsoft-Windows-Resource-Exhaustion-Detector" Guid="{9988748E-C2E8-4054-85F6-0C3E1CAD2470}" /> <EventID>2004</EventID> <Version>0</Version> <Level>3</Level> <Task>3</Task> <Opcode>33</Opcode> <Keywords>0x8000000020000000</Keywords> <TimeCreated SystemTime="2024-04-09T10:41:16.295312100Z" /> <EventRecordID>70738</EventRecordID> <Correlation ActivityID="{14105A88-04DA-416C-AD40-EB7A81DF9370}" /> <Execution ProcessID="512" ThreadID="1044" /> <Channel>System</Channel> <Computer>HAF-X-EMBY</Computer> <Security UserID="S-1-5-18" /> </System> - <UserData> - <MemoryExhaustionInfo xmlns="http://www.microsoft.com/Windows/Resource/Exhaustion/Detector/Events"> - <SystemInfo> <SystemCommitLimit>51522088960</SystemCommitLimit> <SystemCommitCharge>51514077184</SystemCommitCharge> <ProcessCommitCharge>49921589248</ProcessCommitCharge> <PagedPoolUsage>265306112</PagedPoolUsage> <PhysicalMemorySize>12867383296</PhysicalMemorySize> <PhysicalMemoryUsage>12642131968</PhysicalMemoryUsage> <NonPagedPoolUsage>111468544</NonPagedPoolUsage> <Processes>59</Processes> </SystemInfo> - <ProcessInfo> - <Process_1> <Name>EmbyServer.exe</Name> <ID>3988</ID> <CreationTime>2024-04-07T10:45:38.125083800Z</CreationTime> <CommitCharge>48340180992</CommitCharge> <HandleCount>1013</HandleCount> <Version>4.8.3.0</Version> <TypeInfo>201</TypeInfo> </Process_1> - <Process_2> <Name>MsMpEng.exe</Name> <ID>2232</ID> <CreationTime>2024-03-14T22:26:30.130850200Z</CreationTime> <CommitCharge>276561920</CommitCharge> <HandleCount>565</HandleCount> <Version>0.0.0.0</Version> <TypeInfo>66</TypeInfo> </Process_2> - <Process_3> <Name>dwm.exe</Name> <ID>7140</ID> <CreationTime>2024-03-31T16:02:32.508125100Z</CreationTime> <CommitCharge>136179712</CommitCharge> <HandleCount>401</HandleCount> <Version>10.0.14393.0</Version> <TypeInfo>211</TypeInfo> </Process_3> - <Process_4> <Name>SearchUI.exe</Name> <ID>2080</ID> <CreationTime>2024-03-14T23:03:56.058606700Z</CreationTime> <CommitCharge>129540096</CommitCharge> <HandleCount>1017</HandleCount> <Version>10.0.14393.2430</Version> <TypeInfo>152</TypeInfo> </Process_4> - <Process_5> <Name /> <ID>0</ID> <CreationTime>1601-01-01T00:00:00.000000000Z</CreationTime> <CommitCharge>0</CommitCharge> <HandleCount>0</HandleCount> <Version>0.0.0.0</Version> <TypeInfo>0</TypeInfo> </Process_5> - <Process_6> <Name /> <ID>0</ID> <CreationTime>1601-01-01T00:00:00.000000000Z</CreationTime> <CommitCharge>0</CommitCharge> <HandleCount>0</HandleCount> <Version>0.0.0.0</Version> <TypeInfo>0</TypeInfo> </Process_6> </ProcessInfo> - <PagedPoolInfo> - <Tag_1> <Name>CM16</Name> <PoolUsed>133894144</PoolUsed> </Tag_1> - <Tag_2> <Name>MmSt</Name> <PoolUsed>21409088</PoolUsed> </Tag_2> - <Tag_3> <Name>Vi52</Name> <PoolUsed>7286240</PoolUsed> </Tag_3> </PagedPoolInfo> - <NonPagedPoolInfo> - <Tag_1> <Name>NVRM</Name> <PoolUsed>10065936</PoolUsed> </Tag_1> - <Tag_2> <Name>Pool</Name> <PoolUsed>6501952</PoolUsed> </Tag_2> - <Tag_3> <Name>ConT</Name> <PoolUsed>6127616</PoolUsed> </Tag_3> </NonPagedPoolInfo> - <ExhaustionEventInfo> <Time>2024-04-09T10:40:31.215517200Z</Time> </ExhaustionEventInfo> </MemoryExhaustionInfo> </UserData> </Event> embyserver-63848292181.txt Link to comment Share on other sites More sharing options...
Luke 37066 Posted April 10 Share Posted April 10 1 hour ago, Brendon said: @LukeI removed the plugins requested and was still seeing the issue mate, something else i noticed is that my library scans were taking an insane amount of time, i am talking 2 days to get halfway and they were running at the default setting, i disabled that and change it to run once a week on Sunday at midnight, it seemed to get stuck around the 40% mark and spent the next 24hrs for another 4% before i cancelled it, having thought that maybe the problem i disabled it but i am still seeing issues. Twice now this week in the peak evening hours emby has just locked up, everyones streams frozen and my windows server shut the main embyserver process down as it was out of recourses and left the embytray.exe running, I have put a couple of essential plugins back in after i established they weren't the problem but they are not set to autorun at all. When it locked up last night i grabbed the error message from windows event viewer as well as the emby logs (attached) The Event Viewer error.... - <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"> - <System> <Provider Name="Microsoft-Windows-Resource-Exhaustion-Detector" Guid="{9988748E-C2E8-4054-85F6-0C3E1CAD2470}" /> <EventID>2004</EventID> <Version>0</Version> <Level>3</Level> <Task>3</Task> <Opcode>33</Opcode> <Keywords>0x8000000020000000</Keywords> <TimeCreated SystemTime="2024-04-09T10:41:16.295312100Z" /> <EventRecordID>70738</EventRecordID> <Correlation ActivityID="{14105A88-04DA-416C-AD40-EB7A81DF9370}" /> <Execution ProcessID="512" ThreadID="1044" /> <Channel>System</Channel> <Computer>HAF-X-EMBY</Computer> <Security UserID="S-1-5-18" /> </System> - <UserData> - <MemoryExhaustionInfo xmlns="http://www.microsoft.com/Windows/Resource/Exhaustion/Detector/Events"> - <SystemInfo> <SystemCommitLimit>51522088960</SystemCommitLimit> <SystemCommitCharge>51514077184</SystemCommitCharge> <ProcessCommitCharge>49921589248</ProcessCommitCharge> <PagedPoolUsage>265306112</PagedPoolUsage> <PhysicalMemorySize>12867383296</PhysicalMemorySize> <PhysicalMemoryUsage>12642131968</PhysicalMemoryUsage> <NonPagedPoolUsage>111468544</NonPagedPoolUsage> <Processes>59</Processes> </SystemInfo> - <ProcessInfo> - <Process_1> <Name>EmbyServer.exe</Name> <ID>3988</ID> <CreationTime>2024-04-07T10:45:38.125083800Z</CreationTime> <CommitCharge>48340180992</CommitCharge> <HandleCount>1013</HandleCount> <Version>4.8.3.0</Version> <TypeInfo>201</TypeInfo> </Process_1> - <Process_2> <Name>MsMpEng.exe</Name> <ID>2232</ID> <CreationTime>2024-03-14T22:26:30.130850200Z</CreationTime> <CommitCharge>276561920</CommitCharge> <HandleCount>565</HandleCount> <Version>0.0.0.0</Version> <TypeInfo>66</TypeInfo> </Process_2> - <Process_3> <Name>dwm.exe</Name> <ID>7140</ID> <CreationTime>2024-03-31T16:02:32.508125100Z</CreationTime> <CommitCharge>136179712</CommitCharge> <HandleCount>401</HandleCount> <Version>10.0.14393.0</Version> <TypeInfo>211</TypeInfo> </Process_3> - <Process_4> <Name>SearchUI.exe</Name> <ID>2080</ID> <CreationTime>2024-03-14T23:03:56.058606700Z</CreationTime> <CommitCharge>129540096</CommitCharge> <HandleCount>1017</HandleCount> <Version>10.0.14393.2430</Version> <TypeInfo>152</TypeInfo> </Process_4> - <Process_5> <Name /> <ID>0</ID> <CreationTime>1601-01-01T00:00:00.000000000Z</CreationTime> <CommitCharge>0</CommitCharge> <HandleCount>0</HandleCount> <Version>0.0.0.0</Version> <TypeInfo>0</TypeInfo> </Process_5> - <Process_6> <Name /> <ID>0</ID> <CreationTime>1601-01-01T00:00:00.000000000Z</CreationTime> <CommitCharge>0</CommitCharge> <HandleCount>0</HandleCount> <Version>0.0.0.0</Version> <TypeInfo>0</TypeInfo> </Process_6> </ProcessInfo> - <PagedPoolInfo> - <Tag_1> <Name>CM16</Name> <PoolUsed>133894144</PoolUsed> </Tag_1> - <Tag_2> <Name>MmSt</Name> <PoolUsed>21409088</PoolUsed> </Tag_2> - <Tag_3> <Name>Vi52</Name> <PoolUsed>7286240</PoolUsed> </Tag_3> </PagedPoolInfo> - <NonPagedPoolInfo> - <Tag_1> <Name>NVRM</Name> <PoolUsed>10065936</PoolUsed> </Tag_1> - <Tag_2> <Name>Pool</Name> <PoolUsed>6501952</PoolUsed> </Tag_2> - <Tag_3> <Name>ConT</Name> <PoolUsed>6127616</PoolUsed> </Tag_3> </NonPagedPoolInfo> - <ExhaustionEventInfo> <Time>2024-04-09T10:40:31.215517200Z</Time> </ExhaustionEventInfo> </MemoryExhaustionInfo> </UserData> </Event> embyserver-63848292181.txt 4.37 MB · 0 downloads Hi, please also try removing these plugins: Playback Reporting 2.1.0.5 TimeLordTV 19.8.3.1105 Then restart the server and see how things compare. Additionally, instead of using our older Android TV app, Can you try sideloading our standard android app on the same device and see how that compares? https://emby.media/emby-for-android.html Thanks. Link to comment Share on other sites More sharing options...
Brendon 101 Posted April 10 Author Share Posted April 10 (edited) 2 hours ago, Luke said: Hi, please also try removing these plugins: Playback Reporting 2.1.0.5 TimeLordTV 19.8.3.1105 Then restart the server and see how things compare. Additionally, instead of using our older Android TV app, Can you try sideloading our standard android app on the same device and see how that compares? https://emby.media/emby-for-android.html Thanks. On what device?? Everything i use emby on is either browser or the apps from the store, there should be no outdated apps in use, i cannot control external streams, but within my house there are 4x NVidia Shields and all are up to date, apart from that emby is only used via WEBGUI, is the app store not up to date? When it froze on me last night i was using WebGUI on Edge but it froze on everyone else as well, Apple, Xbox they all froze. I am happy to do as asked i just dont see how changing emby on one device is going to solve anything, some of my family streaming remotely use apple or xbox, they all froze up as well because the emby process in the server died. It has done this when none of my devices are in use and the only local access to emby is the pc i am typing on now, it has also done it at 2am in the morning with not a soul logged in, so yeah shoot me down if wrong mate but i am confused with pointing at a single device or app. Also i did take those 2 apps out and they made no difference, i put them back as i needed them, i will take them out again though. If it assists at all @Lukei am happy to drop anydesk on the server and give you access so you can investigate directly. Edited April 10 by Brendon Link to comment Share on other sites More sharing options...
Luke 37066 Posted April 10 Share Posted April 10 Quote On what device?? On the android tv devices. Thanks. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now