Carlo 4330 Posted June 29, 2023 Share Posted June 29, 2023 It's quite possible there are a 2 different memory issues. I know the issue I've been trying to chase down happens on bare metal or any type of container. I'm pretty sure I know the general cause but need to narrow it down a bit more specific so it can be found/fixed. If you're sure it's a docker issue you may be able to solve this yourself by switching from Docker to Podman. It's command line interface is identical to Docker’s, including the arguments. You can simply alias the docker command to podman without noticing the difference, making it easy for existing Docker users to transition to Podman: # .bashrc alias docker=podman Podman does a couple things differently which actually make it more reliable I think. Unlike Docker, which uses the dockerd daemon to manage all the containers under its control, Podman is daemonless. Therefore, there’s no persistent connection to some long-living process, removing the single point of failure issue in Docker, where an abrupt crash in the daemon process can kill running containers or cause them to become orphaned. Podman containers are started as child processes of the Podman process, heavily using user namespaces and network namespaces. Podman also differentiates from Docker by using rootless containers by default. Root access is not necessary for launching and operating a container, but it helps to mitigate potential vulnerabilities in the container runtime that can cause privilege escalation. I think that simplifies adjusting privileges and makes it a little easier to understand. A really cool built in feature of Podman that is not yet present in Docker is the ability to create and run pods. A pod is a collection of one or more containers that utilize a shared pool of resources and work closely together to achieve a specific function. Pods are also the smallest execution unit in Kubernetes, making the transition to Kubernetes easier should the need arise. Summing up, it should be fairly strait forward on most OS (NAS aside) to install Podman and even run it onside Docker but I'd suggest for testing to shut down docker, do the alias and bring your whole stack up running in Podman just as it was running under Docker. If the memory issue was truly Docker related we will know shortly. With any luck the issue is fixed and you got a bit of an upgrade in the process. Link to comment Share on other sites More sharing options...
jaycedk 389 Posted June 29, 2023 Share Posted June 29, 2023 Thanks for you valued input But i think I will give Unraid a month to see if they can come up with a fix. Right now unraid uses docker, and I would not know how to use Podman in that environment. I have a HP ProDesk 400 G3 mini laying around, where i can play around with Linux + Podman, if all else failes. I might call on you for advice when and if that time comes Link to comment Share on other sites More sharing options...
seanbuff 842 Posted June 29, 2023 Share Posted June 29, 2023 6 hours ago, jaycedk said: But i think I will give Unraid a month to see if they can come up with a fix. UnRaid 6.12.2 was just released. They have reverted the Docker version back to an older one to address memory usage issues, etc. See how you go with that. Link to comment Share on other sites More sharing options...
jaycedk 389 Posted June 30, 2023 Share Posted June 30, 2023 (edited) Updating right now Will report back tomorrow, when i have tested. Edited June 30, 2023 by jaycedk Link to comment Share on other sites More sharing options...
jaycedk 389 Posted July 3, 2023 Share Posted July 3, 2023 (edited) As promised here is my findings. Unraids docker roll back did not do it. But I have now notices that its only during media streaming, trascoded , streamed or Directplay i see the high mem usage. That usually drops down when the stream is ended. I have attached some log files, I case you want them, to look through. embyserver-63823766400.txt embyserver-63823939199.txt embyserver (9).txt Edited July 3, 2023 by jaycedk 1 Link to comment Share on other sites More sharing options...
Luke 37132 Posted July 3, 2023 Share Posted July 3, 2023 5 hours ago, jaycedk said: As promised here is my findings. Unraids docker roll back did not do it. But I have now notices that its only during media streaming, trascoded , streamed or Directplay i see the high mem usage. That usually drops down when the stream is ended. I have attached some log files, I case you want them, to look through. embyserver-63823766400.txt 2.94 MB · 0 downloads embyserver-63823939199.txt 3.61 MB · 0 downloads embyserver (9).txt 2.85 MB · 0 downloads Hi, please try removing these plugins: 2023-07-02 03:03:06.574 Info App: Loading EpMetaRefresh, Version=1.0.0.1, Culture=neutral, PublicKeyToken=null from /config/plugins/EpMetaRefresh.dll 2023-07-02 03:03:06.574 Info App: Loading Lastfm, Version=1.0.27.0, Culture=neutral, PublicKeyToken=null from /config/plugins/Lastfm.dll 2023-07-02 03:03:06.574 Info App: Loading MediaBrowser.Plugins.SmtpNotifications, Version=3.1.6.0, Culture=neutral, PublicKeyToken=null from /config/plugins/MediaBrowser.Plugins.SmtpNotifications.dll 2023-07-02 03:03:06.574 Info App: Loading MediaBrowser.Plugins.TimeLord, Version=19.8.3.1105, Culture=neutral, PublicKeyToken=null from /config/plugins/MediaBrowser.Plugins.TimeLord.dll 2023-07-02 03:03:06.574 Info App: Loading PhillipsHue, Version=1.1.0.6, Culture=neutral, PublicKeyToken=null from /config/plugins/PhillipsHue.dll 2023-07-02 03:03:06.574 Info App: Loading Restart, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null from /config/plugins/Restart.dll 2023-07-02 03:03:06.574 Info App: Loading statistics, Version=3.0.10.0, Culture=neutral, PublicKeyToken=null from /config/plugins/Statistics.dll 2023-07-02 03:03:06.574 Info App: Loading playback_reporting, Version=2.0.0.3, Culture=neutral, PublicKeyToken=null from /config/plugins/playback_reporting.dll Then restarting the server and see how things compare. Link to comment Share on other sites More sharing options...
jaycedk 389 Posted July 4, 2023 Share Posted July 4, 2023 Ok will do and report back my findings. 1 Link to comment Share on other sites More sharing options...
jaycedk 389 Posted July 4, 2023 Share Posted July 4, 2023 Hmm don't think its a plugin issue. Removed the plugins restartet the emby docker. Watched a movie, observed this. embyserver (10).txt Link to comment Share on other sites More sharing options...
jaycedk 389 Posted July 5, 2023 Share Posted July 5, 2023 Ok I think I had an realization of what is going on. Here is the memory usage from all my dockers. Here we see the 2 ppl streaming. During the same time span, I see this when looking at the server wide mem usage. This leads me to believe that, when streaming, the file touched is being pulled into RAM Cache. If we look at RAM Used server wide its siting nicely at around 19GB, but looking at all dockers usage it's well above 24GB at times. And I see an increase in RAM cached. So in short user starts a stream. Video file loaded into RAM cached. I think, I see combined RAM Used + RAM Cached, in my docker graf, after the kernel was updated with the Unraid update. I think, I only saw RAM Used before this update. Tbh I have not seen oom_kill at any time. I was just confused by what I could see. 1 Link to comment Share on other sites More sharing options...
jaycedk 389 Posted July 5, 2023 Share Posted July 5, 2023 Just for good measure. Can we move my comments to a new topic, since I hijacked this one from op. Link to comment Share on other sites More sharing options...
andrewmcd7 1 Posted October 7, 2023 Author Share Posted October 7, 2023 Can I please raise this issue again. It’s occurring on my server usually once a day. Last about 5-15 min. No scheduled tasks occurring. It logs in and you see the images for the libraries but everything else hangs. I have attached the log file. app2806145776409558552.zip Link to comment Share on other sites More sharing options...
andrewmcd7 1 Posted October 8, 2023 Author Share Posted October 8, 2023 @Luke^^^ Link to comment Share on other sites More sharing options...
Luke 37132 Posted October 9, 2023 Share Posted October 9, 2023 On 10/8/2023 at 9:30 AM, andrewmcd7 said: @Luke^^^ Hi, infuse is hitting the server pretty hard during it's syncing process. Try removing the Infuse plugin and also closing infuse apps on your devices and then see how things compare. Link to comment Share on other sites More sharing options...
andrewmcd7 1 Posted October 9, 2023 Author Share Posted October 9, 2023 14 minutes ago, Luke said: Hi, infuse is hitting the server pretty hard during its syncing process. Try removing the Infuse plugin and also closing infuse apps on your devices and then see how things compare. Will removing the plugin but still having infuse clients still connect, will that make the problem worse or better? I thought the plug in was supposed to lessen the DB hammering on emby. Link to comment Share on other sites More sharing options...
Luke 37132 Posted October 9, 2023 Share Posted October 9, 2023 28 minutes ago, andrewmcd7 said: Will removing the plugin but still having infuse clients still connect, will that make the problem worse or better? I thought the plug in was supposed to lessen the DB hammering on emby. I don't know enough about how InfuseSync works to answer that. I'm just commenting on what I see in the server log. Link to comment Share on other sites More sharing options...
andrewmcd7 1 Posted October 17, 2023 Author Share Posted October 17, 2023 Unfortunately this didn’t resolve the issue. It might have made it less frequent but it’s still happening. Here are logs from a different server… libraries were loading but no media was able to be played. Time was 7:45am. I’m racking my brain here. Any insight would help @Luke embyserver_3.txt Link to comment Share on other sites More sharing options...
Schleudertrauma 4 Posted October 17, 2023 Share Posted October 17, 2023 There still are many issues related to your media in fact that emby can't find the media files in some cases. Have you set the permissions to your media files correctly? In some cases emby can't start it's native services and then crashes but i am not a dev, so i don't know why that happens. In most cases in my history with linux and emby were the permission related to emby. In your case i would try to install emby natively from the debian package for ubuntu systems and test whether the issue persists or not. If not, then there could be a failure in your docker environment. Link to comment Share on other sites More sharing options...
andrewmcd7 1 Posted October 17, 2023 Author Share Posted October 17, 2023 Yeah I realized after a mount dropped in that server… @schleudertrauma Link to comment Share on other sites More sharing options...
Luke 37132 Posted October 17, 2023 Share Posted October 17, 2023 4 hours ago, andrewmcd7 said: Yeah I realized after a mount dropped in that server… @schleudertrauma Has this resolved your issue? Link to comment Share on other sites More sharing options...
andrewmcd7 1 Posted October 17, 2023 Author Share Posted October 17, 2023 22 minutes ago, Luke said: Has this resolved your issue? No this was a second server. Thought it was the same issue. It wasn’t. Link to comment Share on other sites More sharing options...
andrewmcd7 1 Posted December 26, 2023 Author Share Posted December 26, 2023 Looks like the issue is still happening occasionally. Here is another server of mine that just had the same issue. Loads the thumbnails for libraries and that’s it. See attached logs. embyserver__24_.txt Link to comment Share on other sites More sharing options...
Luke 37132 Posted December 29, 2023 Share Posted December 29, 2023 On 12/26/2023 at 4:31 PM, andrewmcd7 said: Looks like the issue is still happening occasionally. Here is another server of mine that just had the same issue. Loads the thumbnails for libraries and that’s it. See attached logs. embyserver__24_.txt 1.25 MB · 1 download OK we're looking into it. Thanks. Link to comment Share on other sites More sharing options...
andrewmcd7 1 Posted January 18 Author Share Posted January 18 Hey Luke any progress on this? It’s a daily issue on many servers we manage. Hoping there’s something we can do Link to comment Share on other sites More sharing options...
Luke 37132 Posted January 18 Share Posted January 18 1 minute ago, andrewmcd7 said: Hey Luke any progress on this? It’s a daily issue on many servers we manage. Hoping there’s something we can do The 4.8 release is coming soon and will be lighter on it's feet than 4.7. Thanks. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now