Riezel 15 Posted March 15 Share Posted March 15 (edited) as per title says, my emby keep crashed while doing full scan library (today was the scheduled days, weekly) the scan stuck in 22.3% then crashed (my library is HUGE, probably due this as well??) i've tried to reduce database cache size to 512 (previously 4096, yes i know its my fault, but it was fine before 4.8.x, and it seems elevate the crash for a while, tho it still crash in the end) my synology has 20gb of ram, here is some screen cap of emby usage i attached all my synology log (excluding hardware detection) the emby itself is in idle, only doing full scan lib i think my emby already crashed 6+ today (while i still tweaking db cache size, and reduce it bit a bit) embyserver-log.zip Edited March 15 by Riezel add additional pic Link to comment Share on other sites More sharing options...
Riezel 15 Posted March 15 Author Share Posted March 15 just tried with 128mb, it still crashed i guess, i cant do full lib scan until this memory issue fixed Link to comment Share on other sites More sharing options...
Luke 37071 Posted March 15 Share Posted March 15 Hi, please try removing these plugins: 2024-03-15 08:44:25.004 Info App: Loading ChapterApi, Version=1.3.0.8, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/ChapterApi.dll 2024-03-15 08:44:25.004 Info App: Loading Addic7ed, Version=1.1.1.0, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/Addic7ed.dll 2024-03-15 08:44:25.004 Info App: Loading Podnapisi, Version=1.0.9.0, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/Podnapisi.dll 2024-03-15 08:44:25.004 Info App: Loading SubDb, Version=1.0.7.0, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/SubDb.dll 2024-03-15 08:44:25.004 Info App: Loading Emby.DiagnosticsPlugin, Version=4.8.0.58, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/Emby.DiagnosticsPlugin.dll 2024-03-15 08:44:25.004 Info App: Loading VantagePoint, Version=2.1.0.5, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/VantagePoint.dll 2024-03-15 08:44:25.004 Info App: Loading playback_reporting, Version=2.1.0.5, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/playback_reporting.dll 2024-03-15 08:44:25.004 Info App: Loading EpMetaRefresh, Version=1.0.0.1, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/EpMetaRefresh.dll 2024-03-15 08:44:25.004 Info App: Loading Lastfm, Version=1.0.29.0, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/Lastfm.dll 2024-03-15 08:44:25.004 Info App: Loading Emby.AutoOrganize, Version=1.7.3.0, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/Emby.AutoOrganize.dll 2024-03-15 08:44:25.004 Info App: Loading statistics, Version=3.2.0.0, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/Statistics.dll Then restart the server and see how things compare. Thanks. Link to comment Share on other sites More sharing options...
Riezel 15 Posted March 15 Author Share Posted March 15 1 minute ago, Luke said: Hi, please try removing these plugins: 2024-03-15 08:44:25.004 Info App: Loading ChapterApi, Version=1.3.0.8, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/ChapterApi.dll 2024-03-15 08:44:25.004 Info App: Loading Addic7ed, Version=1.1.1.0, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/Addic7ed.dll 2024-03-15 08:44:25.004 Info App: Loading Podnapisi, Version=1.0.9.0, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/Podnapisi.dll 2024-03-15 08:44:25.004 Info App: Loading SubDb, Version=1.0.7.0, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/SubDb.dll 2024-03-15 08:44:25.004 Info App: Loading Emby.DiagnosticsPlugin, Version=4.8.0.58, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/Emby.DiagnosticsPlugin.dll 2024-03-15 08:44:25.004 Info App: Loading VantagePoint, Version=2.1.0.5, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/VantagePoint.dll 2024-03-15 08:44:25.004 Info App: Loading playback_reporting, Version=2.1.0.5, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/playback_reporting.dll 2024-03-15 08:44:25.004 Info App: Loading EpMetaRefresh, Version=1.0.0.1, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/EpMetaRefresh.dll 2024-03-15 08:44:25.004 Info App: Loading Lastfm, Version=1.0.29.0, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/Lastfm.dll 2024-03-15 08:44:25.004 Info App: Loading Emby.AutoOrganize, Version=1.7.3.0, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/Emby.AutoOrganize.dll 2024-03-15 08:44:25.004 Info App: Loading statistics, Version=3.2.0.0, Culture=neutral, PublicKeyToken=null from /var/packages/EmbyServer/var/plugins/Statistics.dll Then restart the server and see how things compare. Thanks. question before i remove this, is removing them from plugin folder (like delete the .dll manually) will retain the config i've set for them ?? when i restore the plugins ? Link to comment Share on other sites More sharing options...
Riezel 15 Posted March 15 Author Share Posted March 15 thanks for pointing it out @Luke i've disabled only ChapterApi now, since from my emby settings, only that plugins that will run along the full scan media (and revert back db cache to 8192) sofar so good, kept to ~4gb ish (sometimes 6gb ish) it still manageable (and it's already run for 1 hours now, though the scan still stuck in 22.3%) will keep monitoring for now, hope there wont be any crash, will see tomorrow and report it back if its still crashed 1 Link to comment Share on other sites More sharing options...
Riezel 15 Posted March 16 Author Share Posted March 16 update, it seems not working (my assumption is, the memory leak appear when it scans the series with lot of episode) this one, is example of 52 episode in 1 series (i have some series that have hundred episode, 300+ for 1 series too, so this would be problem) the previous post image that get high memory, is when scan started in series that have lot of episode as well, my assumption is emby populate them first then put it into RAM for easier to process ? which lead memory spike ? currently i'm trying to do a single library scan, if it can trigger memory spike or not for now (image below is SS from single lib scans, while previous post is from full scan lib, and memory handling behavior for those type of scans seems different) spiked from 1gb~ish to 4gb when scan 24 ~ 26 episode spiked from 2gb~ish to 5.6gb when scan 50ish episode (noticed initial ram still stays in 2gb, from previous scans) and sometime, it back to under 1 gb again (if found something that need to be processed perhaps?) cc @Luke embyserver-63846161591.txt Link to comment Share on other sites More sharing options...
Riezel 15 Posted March 16 Author Share Posted March 16 it died again and in same series, in prev post (still same single library scans though) Link to comment Share on other sites More sharing options...
Riezel 15 Posted March 17 Author Share Posted March 17 @Lukeafter turning the debug log mode it seems, it's relevant to this, since i noticed memory spike while scans pointing this for example (E2877) [Tsundere] To Aru Kagaku no Railgun - OVA [BDRip h264 1280x720 FLAC][1E2877D4].mkv if this was really the issue, then i can't go scan my lib atm, until this things fixed (since renaming all my files to delete those kind of strings, in my already huge library is kinda unreasonable), attached the log, i forced stop the scan, by restarting the server in the end embyserver (3).txt Link to comment Share on other sites More sharing options...
Luke 37071 Posted March 20 Share Posted March 20 Have you been able to complete a scan? Link to comment Share on other sites More sharing options...
Riezel 15 Posted March 21 Author Share Posted March 21 6 hours ago, Luke said: Have you been able to complete a scan? unfortunately nope, in the end, i'v renamed all my library (even made new library), currently still going with refresh meta data if it is refresh meta data, memory leak wont happens, i'll try going full scans again after i finished refreshed all my lib first (ETA probably 3 ~ 4 more days again ) Link to comment Share on other sites More sharing options...
Riezel 15 Posted March 24 Author Share Posted March 24 finally after fully migrate the lib (ofc after renaming ALL my files to avoid the metadata naming bug) i succeed to full scan my library, memory usage is around 600MB ~ 950MB (with all plugins enabled back, as it is) also this is my lib atm (will grow further later though), hence taking 7 hours (because i havnt finished 1 series full refresh meta data earlier, i think) my hypothesis is, this happens due the file naming recognition which lead to the metadata crawler stuck (?, since it can crawl up to thousand episode for 1 files only with naming like S01E01 - xxxxxxxxxxxx [EBAE2395], so it might query up to episode 2395, while the series itself is only 20 episode for example) or yes as carlo said, maybe its due the SQL error, something like multiple root value or something happens too much (imagine 1 series with 50 episode, and each one has up to thousand query / episode), for now i already prevent them, by auto trim those CRC hash for any further files, also i attach the logs, for the full scans log (no debug), most error is in myanimelist and anilist (no sql error, something with the response i think) or maybe some when i deleted 1 folder to rename it again later, hope this helps to mitigate this bug again later (with different approach file naming recognition, i hope, if it was really due that bug, since in some previous version of emby, i didnt encounter this one, and i always make it daily / weekly routine for full scan) what i did to make this happens 1. delete the old library (yes it's painful) 2. full rename ALL FILES that has possibility to cause metadata bug, like everything (nfo, thumb, bif, and the video files) 3. make new library (instant stop the full scans, after created) 4. instead doing full scans, i instead, go with "Refresh meta data" each folder, for my debugging purpose as well (while monitoring the ram usage, sometimes) 5. after finished full refresh meta data all library, go full scans now 6. wait while monitoring ram usage cc @Luke @Carlo logs.zip 1 Link to comment Share on other sites More sharing options...
Luke 37071 Posted March 26 Share Posted March 26 Thanks for following up ! Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now