dotcom 8 Posted January 24, 2020 Share Posted January 24, 2020 (edited) I have a quite beefy server, 2 cpu 8 cores each 128GB RAM running docker. This has been running completely fine and very fast for a while. About 2 weeks ago I upgraded to (current) latest docker image and zero hiccups, still speedy as can be. About 3 days ago randomly the ui would be slow and larger media items would take time. I've done network tests and they seem to be fine, doesn't appear to be server. Load is nothing. example of request log from embyserver.txt Observations Pings on my network to the machine are consistently under 10ms. A secure copy wirelessly to my laptop is about 44MB/s consistently Seems that a restart of the container does speed it up for a short time but rather quickly it bogs down again. Any suggestions where to look further? I don't see any errors or other issues.TIA partial-embyserver.txt Edited January 24, 2020 by dotcom Link to comment Share on other sites More sharing options...
Luke 36886 Posted January 24, 2020 Share Posted January 24, 2020 Hi there, can you please attach the emby server log from when this happened? Thanks. Link to comment Share on other sites More sharing options...
dotcom 8 Posted January 24, 2020 Author Share Posted January 24, 2020 (edited) Attached. If I restart the container things seem decently snappy for a few minutes then everything slows down post 5-10 mins embyserver.txt Edited January 24, 2020 by dotcom Link to comment Share on other sites More sharing options...
dotcom 8 Posted January 24, 2020 Author Share Posted January 24, 2020 (edited) Also attached the partial log from before I kept, thought I attached it in the first place but I forgot to click upload. Edited January 24, 2020 by dotcom Link to comment Share on other sites More sharing options...
dotcom 8 Posted January 26, 2020 Author Share Posted January 26, 2020 Anything I can try on this or to increase verbosity? Link to comment Share on other sites More sharing options...
dotcom 8 Posted January 28, 2020 Author Share Posted January 28, 2020 I started another container from scratch using a different volume for /config and everything is snappy again. I would like to troubleshoot my old config to see why this happened in the first place in case it happens again. Would also be nice to have my customizations and watch history back too. Link to comment Share on other sites More sharing options...
Luke 36886 Posted January 28, 2020 Share Posted January 28, 2020 Does the new volume have better performance than the old one? Link to comment Share on other sites More sharing options...
dotcom 8 Posted January 28, 2020 Author Share Posted January 28, 2020 No, I’m talking a docker volume. Just a different directory on my main system. I only have one 24 TB raid 5 volume that reads and writes around 300mbps. My guess is there is an issue with the Database or something causing the slowness. If I mount my old config in the same container slowdown happens within the first couple minutes of starting. Nothing in any log to indicate why. Link to comment Share on other sites More sharing options...
Luke 36886 Posted January 29, 2020 Share Posted January 29, 2020 For the old config, how old is that installation? Link to comment Share on other sites More sharing options...
Happy2Play 8144 Posted January 29, 2020 Share Posted January 29, 2020 Attached. If I restart the container things seem decently snappy for a few minutes then everything slows down post 5-10 mins This log shows the database being purged at "2020-01-24 04:49:55.501" on version 4.4.0.8. Looks like 542 items being removed along with the perspective /config/metadata/library/ information and possibly re-added but can't tell with the log provided. The slow response appears to happen right after. Don't know what is changing with your setup that would cause this purge though. Anything I can try on this or to increase verbosity? You can enable Debug logging. Dashboard-Logs Note you have to restart your server to start or stop this logging level, when enabled or disabled. Link to comment Share on other sites More sharing options...
dotcom 8 Posted January 29, 2020 Author Share Posted January 29, 2020 (edited) Here is an updated log with debug logging turned on. I removed the old log, started the container, clicked around then waited a few minutes. at 2020-01-29 03:22:12.818 things really slow down with a response clicking on an episode at over 102 seconds. At 2020-01-29 03:25:48.476 I click on a different episode of the same series which takes 78 seconds for just the first response back (nothing but spinner is showing still) then around 2020-01-29 03:27:57.172 everything else for the episode loads. Playback is also affected for higher bitrate shows, as well as downloads.I don't care too much about this anymore because I mapped a different directory to config which is very snappy and happy again. I just lose all my watch history and customizations.I'm happy to help narrow this down so it doesn't happen to me or someone else again if you'd like. embyserver.txt Edited January 29, 2020 by dotcom Link to comment Share on other sites More sharing options...
dotcom 8 Posted January 29, 2020 Author Share Posted January 29, 2020 For the old config, how old is that installation? The config was around may 11th of 2019. I think docker:latest was 4.2.0.5 Link to comment Share on other sites More sharing options...
Luke 36886 Posted January 29, 2020 Share Posted January 29, 2020 Ok that's not so old, but you could try this on your old container and see if this helps: https://emby.media/community/index.php?/topic/75510-42032-vacuum-database-config-switch/ Link to comment Share on other sites More sharing options...
dotcom 8 Posted January 29, 2020 Author Share Posted January 29, 2020 Ok that's not so old, but you could try this on your old container and see if this helps: https://emby.media/community/index.php?/topic/75510-42032-vacuum-database-config-switch/ This helped a little bit actually. Things are still quite slow after 10+ minutes. Updated new log post vacuum. embyserver.txt Link to comment Share on other sites More sharing options...
Happy2Play 8144 Posted January 29, 2020 Share Posted January 29, 2020 Don't know what is going on with your server, but it is still purging items from the database. 2020-01-29 18:42:12.353 Debug BaseItem: Removed item: 2020-01-29 18:42:12.727 Info App: Deleting path 4.5 hours later still 2020-01-29 23:20:24.731 Info App: Deleting path Link to comment Share on other sites More sharing options...
dotcom 8 Posted January 30, 2020 Author Share Posted January 30, 2020 It’s always doing that, some sort of bug was hit they caused this and has it in a loop. My other container works just fine on the same machine with a different config path Link to comment Share on other sites More sharing options...
Luke 36886 Posted January 30, 2020 Share Posted January 30, 2020 Are the libraries setup exactly the same? Link to comment Share on other sites More sharing options...
dotcom 8 Posted January 30, 2020 Author Share Posted January 30, 2020 Yes they are, I have a rather simple library scheme. A folder for movies and a folder for tv Link to comment Share on other sites More sharing options...
Luke 36886 Posted January 30, 2020 Share Posted January 30, 2020 Ok, it's probably related to all of that database activity that happy2play mentioned. Link to comment Share on other sites More sharing options...
dotcom 8 Posted January 30, 2020 Author Share Posted January 30, 2020 Should I just call this a fluke and kill the old one off entirely? I am only continuing this rabbit hole in hopes of helping others that run into the same issue. Link to comment Share on other sites More sharing options...
Luke 36886 Posted January 30, 2020 Share Posted January 30, 2020 If the new container is faster then you may just want to go with that yes. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now