SwanStream 59 Posted September 23, 2018 Share Posted September 23, 2018 (edited) Since moving my library management to the synology box (DS2415+) has been rock solid. Some weeks ago, not sure exactly when or what changed.... something updated and since I've been getting some server unreachable messages from the client side (Kodi w/addon). I've figured out the server is shutting itself down, I've looked to make sure real time monitoring is off as suggested in other posts that seem relevant. Sadly, ignoring it hasn't fixed it, so now I need to ask for help sorting it... Log is insanely huge, but can be found here: https://www.dropbox.com/s/jnzkzmmdurd30wv/Log.txt?dl=0 Do I have verbose logging selected somewhere I can't find? I've verified it's NOT set to debug... but as I said, the logs Yesterday I updated the server to latest available and last night we had another shutdown. So the 'fix' is NOT in the 3.5.3.0-1 rev. I should add that the title of the post is totally a wild-ass-guess regarding the cause of the shutdown, It's shutting down, if it's related to scanning I just got lucky diagnosing 'cause I'm stupid when it comes to this stuff. Edited September 23, 2018 by bikesquid Link to comment Share on other sites More sharing options...
Luke 36876 Posted September 23, 2018 Share Posted September 23, 2018 Hi, there's nothing obvious here, but you could try turning off the server's dlna features and see if that helps. You could also try removing the kodi sync plugin. thanks. Link to comment Share on other sites More sharing options...
solabc16 379 Posted September 23, 2018 Share Posted September 23, 2018 Hello @ Two things... 1) If possible, can you run the send logs utility, that will give me a better idea of what's going on : https://github.com/MediaBrowser/Wiki/wiki/Synology-:-How-to-Send-us-Support-Logs 2) Would you consider running a .NET Core test release? https://emby.media/community/index.php?/topic/62285-net-core-package-test-programme-for-synologyxpenology/ Best - James Link to comment Share on other sites More sharing options...
SwanStream 59 Posted September 24, 2018 Author Share Posted September 24, 2018 @@Luke - Thanks a bunch for taking a look and the suggestions. I'll start by turning off DLNA (which I wasn't even aware was still enabled....) DLNA causes me massive excess net traffic I don't need anyway. @@solabc16 - 1) Sure - on the way... 2) Y'all do amazing work to create and support all this, the least us users can do is be willing to invest some time to support your efforts to help us, so absolutely I'm available to test code. let's do it. Cheers guys! Link to comment Share on other sites More sharing options...
SwanStream 59 Posted September 24, 2018 Author Share Posted September 24, 2018 @@solabc16 - Getting HTTP error500 when trying to send logs. It's a large log archive 1.2GB (doesn't sound right to me... but maybe...) so maybe I'm timing out at 700kB/s upload speed... ?? any suggestions? Link to comment Share on other sites More sharing options...
solabc16 379 Posted September 24, 2018 Share Posted September 24, 2018 (edited) Hello @ As you noted earlier, for one reason or another, your logs are 'insanely' large! Suggest, stopping Emby Server in Package Center, then deleting all of the logs in the directory:- /var/packages/EmbyServer/target/var/logs/ Once you've started Emby Server again, let it run for a while then try sending the logs. Best - James Edited September 24, 2018 by solabc16 Link to comment Share on other sites More sharing options...
EagleMitchell 1 Posted September 25, 2018 Share Posted September 25, 2018 You could adjust how frequently your server rolls the log file. Additionally, do you have plugins that are bloating the logs? Link to comment Share on other sites More sharing options...
SwanStream 59 Posted September 25, 2018 Author Share Posted September 25, 2018 @@solabc16 - Hate to be useless but... tried again on sending the log and for some reason I don't understand I keep getting http error 500 when sending (log is now only 79MB... in less than 24 hours) so should be fine, but no luck. My unix commandline knowledge is crap or I'd "just" copy the file off to a PC and attach here, but I don't know how to do that via SSH. Unless someone can walk me through it, I'll have to wait for the system to finish expanding a disk group before I can try to reboot it. Link to comment Share on other sites More sharing options...
solabc16 379 Posted September 26, 2018 Share Posted September 26, 2018 Hello @ It's nothing you're doing wrong, it's just the maximum archive size limit being reached. To give some reference, the 600MB+ file from your first post compresses down to a handful of MB. So it seems your system has already generated a sizeable set of logs; which is definitely not 'normal'. Can you repeat the above process to clear logs, then run the utility after an hour; that should be enough to get something useful that will give us a clue as to what is going on. Best - James Link to comment Share on other sites More sharing options...
SwanStream 59 Posted September 26, 2018 Author Share Posted September 26, 2018 @@solabc16 - Thanks James! So as a 'test' I: stopped emby. deleted logs from the referenced dir. started emby ran util immediately and log size is basically the same as yesterday (76MB) within 30 seconds of starting server....No need to wait an hour to fail! Is that too big for receiving end? If so can it just be compressed and sent manually from putty? Must be pulling some other files to include Link to comment Share on other sites More sharing options...
solabc16 379 Posted September 26, 2018 Share Posted September 26, 2018 Hello @ Yeah, that's definitely not 'normal' and I don't recall seeing or hearing about anything that extreme in my time working on the project. If you haven't already, I suggest stopping the package... then can you run the following and post the output here:- ls -lh /var/packages/EmbyServer/target/var/logs/ Best - James Link to comment Share on other sites More sharing options...
SwanStream 59 Posted September 26, 2018 Author Share Posted September 26, 2018 @@solabc16 - OK, I'm officially dumber than the dog!! I "thought" you meant *.log, not ALL the files in the logs directory!! So done did it again, this time following instructions.... and it worked!! So you've got a log set from immediately after the server has started. If you want fresh logs after a bit, I should be capable now. Thanks for the patients all So, here's the dir listing you just asked for in case it's telling- total 1.5G -rw-r--r-- 1 embysvr users 541 Sep 26 15:11 certsync.log -rw-r--r-- 1 embysvr users 92 Sep 26 15:11 drmsetup.log -rw-r--r-- 1 embysvr users 256K Sep 26 15:28 embyserver.txt -rw-r--r-- 1 embysvr users 1.5G Sep 26 15:28 embysvr.stderr -rw-r--r-- 1 embysvr users 631K Sep 26 15:11 embysvr.stdout -rw-r--r-- 1 embysvr users 2.1K Sep 26 15:28 ESdaemon.log -rw-r--r-- 1 root root 25K Sep 26 15:30 start-stop-status_20180926.log Link to comment Share on other sites More sharing options...
SwanStream 59 Posted September 26, 2018 Author Share Posted September 26, 2018 log is sendlogs_SwanStreamer_synology_avoton_2415+_20180926T223241UTC.tgz Link to comment Share on other sites More sharing options...
solabc16 379 Posted September 26, 2018 Share Posted September 26, 2018 Great, thanks @bikesquid - give me a moment and let me see what is going on here. Best - James Link to comment Share on other sites More sharing options...
solabc16 379 Posted September 26, 2018 Share Posted September 26, 2018 Right, nothing particularly unusual going on in the logs contained in that archive. The file of interest above is embysvr.stderr, which is suspect at 1.5GB. This file in the archive is only a few KB, can you leave the server running briefly and use the directory listing command above to watch for this file growing. If you see it start to grow significantly, then stop the server and run the send logs utility again. Best - James Link to comment Share on other sites More sharing options...
SwanStream 59 Posted September 26, 2018 Author Share Posted September 26, 2018 Again - Thanks for the time and help! I saw that file size... wonder if it's possible it's just more than the server can handle. Seen that particular size be a problem before... thinking about a cache size limit for some 32bit program, though I can't remember what now. Server just finished a scan that previously was basically stalled at 90+%. So more gooder for sure just ditching the log. Now doing a scan of libraries looking for missing metadata and the log in question is doubled, so I'm sending although I haven't seen it seize up and don't know if I'd call it significant other than ratio.... sorry if its a waste of time. new log is here: sendlogs_SwanStreamer_synology_avoton_2415+_20180926T233828UTC.tgz Link to comment Share on other sites More sharing options...
SwanStream 59 Posted September 26, 2018 Author Share Posted September 26, 2018 and as I wrap up that post... logs jumped signifgantly so redone-did-dat-der-util and the file is sendlogs_SwanStreamer_synology_avoton_2415+_20180926T234659UTC.tgz I hope that's helpful!! Link to comment Share on other sites More sharing options...
solabc16 379 Posted September 27, 2018 Share Posted September 27, 2018 Hello @ These logs look ok, nothing out of the ordinary to report. The sizes will grow a fair amount when running a full library scan, especially on a new library and/or when metadata is being added. Saying that, these are more 'normal' than the monster logs that were being generated previously, so we still haven't seen what was causing those. Let me know how your system is looking now and whether there has been any significant change to the log sizes. Best - James Link to comment Share on other sites More sharing options...
EagleMitchell 1 Posted September 27, 2018 Share Posted September 27, 2018 Thanks for the offer @@solabc16 I don't know much about .NET however so I don't know how helpful I can be. Link to comment Share on other sites More sharing options...
SwanStream 59 Posted September 27, 2018 Author Share Posted September 27, 2018 @@solabc16 - Thinks are looking promising. Self healing? In any case, until it blinks out again, probably nothing to be learned. server is 90% through it's rescan looking for missing metadata (what can I say, I'm a hoarder).... Here's the dir listing in case that is informative- total 70M -rw-r--r-- 1 embysvr users 541 Sep 26 15:31 certsync.log -rw-r--r-- 1 embysvr users 92 Sep 26 15:31 drmsetup.log -rw-r--r-- 1 embysvr users 19M Sep 27 00:00 embyserver-63673603200.txt -rw-r--r-- 1 embysvr users 8.1M Sep 27 07:48 embyserver.txt -rw-r--r-- 1 embysvr users 44M Sep 27 07:49 embysvr.stderr -rw-r--r-- 1 embysvr users 17K Sep 26 15:31 embysvr.stdout -rw-r--r-- 1 embysvr users 1.8K Sep 26 15:31 ESdaemon.log -rw-r--r-- 1 root root 11K Sep 26 15:36 start-stop-status_20180926.log Link to comment Share on other sites More sharing options...
solabc16 379 Posted September 27, 2018 Share Posted September 27, 2018 Thanks for the update @, the above looks reasonable given the current library scan / metadata acquisition going on. Do shout if you have any issues... Best - James Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now