sector327 4 Posted January 22, 2018 Posted January 22, 2018 Had something unexpected happen today. While watching a movie, it simply returned the error "Video Error - unexpected problem (but not server timeout or http error) has been detected" I hadn't upgraded anything in a while but when I ssh into the server (Fedora27 Server) and try to access samba, I see the service failed. Notice the super informative "Failed with result 'exit-code'." Haha...thanks for the info! (hey....sometimes you have to laugh). $ sudo systemctl status smb ● smb.service - Samba SMB Daemon Loaded: loaded (/usr/lib/systemd/system/smb.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Mon 2018-01-22 11:37:03 CST; 2min 58s ago Process: 2149 ExecStart=/usr/sbin/smbd --foreground --no-process-group $SMBDOPTIONS (code=exited, status=1/FAILURE) Main PID: 2149 (code=exited, status=1/FAILURE) Jan 22 11:37:03 servername systemd[1]: Starting Samba SMB Daemon... Jan 22 11:37:03 servername systemd[1]: smb.service: Main process exited, code=exited, status=1/FAILURE Jan 22 11:37:03 servername systemd[1]: Failed to start Samba SMB Daemon. Jan 22 11:37:03 servername systemd[1]: smb.service: Unit entered failed state. Jan 22 11:37:03 servername systemd[1]: smb.service: Failed with result 'exit-code'. anyways, next step was to look in journalctl -xe Here I see a TON of messages about file '5' write error: No space left on device [v8.31.0 try http://www.rsyslog.com/e/2027 ] Jan 22 11:37:31 servername rsyslogd[813]: action 'action 0' (module 'builtin:omfile') message lost, could not be processed. Check for additional error messages before this one. [v8.31.0 try http://www.rsyslog.com/e/2 So a quick look at my disk space and I see that /fedora-root is full. df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 1.9G 0 1.9G 0% /dev tmpfs 1.9G 8.0K 1.9G 1% /dev/shm tmpfs 1.9G 18M 1.9G 1% /run tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup /dev/mapper/fedora-root 15G 15G 20K 100% / tmpfs 1.9G 0 1.9G 0% /tmp /dev/sda2 976M 183M 726M 21% /boot /dev/sda1 200M 11M 189M 6% /boot/efi /dev/sdb2 3.7T 1.6T 2.1T 44% /media/usb1 tmpfs 384M 0 384M 0% /run/user/1000 Is this b/c of emby? I'm running "du -hs" and I'm getting this below. If this is a Fedora issue, let me know and I'll go investigate elsewhere. I'm just kind of curious if there's something that is sometimes seen w/ Emby where root gets filled. (Emby or not, any free advice on how to resolve?) [root@servername /]# du -hs du: cannot access './proc/1251/task/1251/fd/3': No such file or directory du: cannot access './proc/1251/task/1251/fdinfo/3': No such file or directory du: cannot access './proc/1251/fd/4': No such file or directory du: cannot access './proc/1251/fdinfo/4': No such file or directory 15G . [root@servername /]# thanks!
Luke 42077 Posted January 22, 2018 Posted January 22, 2018 Hi there, can you please attach the emby server log? you can learn how to do that here: https://emby.media/community/index.php?/topic/739-how-to-report-a-problem/ Thanks.
mastrmind11 722 Posted January 22, 2018 Posted January 22, 2018 (edited) Had something unexpected happen today. While watching a movie, it simply returned the error "Video Error - unexpected problem (but not server timeout or http error) has been detected" I hadn't upgraded anything in a while but when I ssh into the server (Fedora27 Server) and try to access samba, I see the service failed. Notice the super informative "Failed with result 'exit-code'." Haha...thanks for the info! (hey....sometimes you have to laugh). $ sudo systemctl status smb ● smb.service - Samba SMB Daemon Loaded: loaded (/usr/lib/systemd/system/smb.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Mon 2018-01-22 11:37:03 CST; 2min 58s ago Process: 2149 ExecStart=/usr/sbin/smbd --foreground --no-process-group $SMBDOPTIONS (code=exited, status=1/FAILURE) Main PID: 2149 (code=exited, status=1/FAILURE) Jan 22 11:37:03 servername systemd[1]: Starting Samba SMB Daemon... Jan 22 11:37:03 servername systemd[1]: smb.service: Main process exited, code=exited, status=1/FAILURE Jan 22 11:37:03 servername systemd[1]: Failed to start Samba SMB Daemon. Jan 22 11:37:03 servername systemd[1]: smb.service: Unit entered failed state. Jan 22 11:37:03 servername systemd[1]: smb.service: Failed with result 'exit-code'. anyways, next step was to look in journalctl -xe Here I see a TON of messages about file '5' write error: No space left on device [v8.31.0 try http://www.rsyslog.com/e/2027 ] Jan 22 11:37:31 servername rsyslogd[813]: action 'action 0' (module 'builtin:omfile') message lost, could not be processed. Check for additional error messages before this one. [v8.31.0 try http://www.rsyslog.com/e/2 So a quick look at my disk space and I see that /fedora-root is full. df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 1.9G 0 1.9G 0% /dev tmpfs 1.9G 8.0K 1.9G 1% /dev/shm tmpfs 1.9G 18M 1.9G 1% /run tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup /dev/mapper/fedora-root 15G 15G 20K 100% / tmpfs 1.9G 0 1.9G 0% /tmp /dev/sda2 976M 183M 726M 21% /boot /dev/sda1 200M 11M 189M 6% /boot/efi /dev/sdb2 3.7T 1.6T 2.1T 44% /media/usb1 tmpfs 384M 0 384M 0% /run/user/1000 Is this b/c of emby? I'm running "du -hs" and I'm getting this below. If this is a Fedora issue, let me know and I'll go investigate elsewhere. I'm just kind of curious if there's something that is sometimes seen w/ Emby where root gets filled. (Emby or not, any free advice on how to resolve?) [root@servername /]# du -hs du: cannot access './proc/1251/task/1251/fd/3': No such file or directory du: cannot access './proc/1251/task/1251/fdinfo/3': No such file or directory du: cannot access './proc/1251/fd/4': No such file or directory du: cannot access './proc/1251/fdinfo/4': No such file or directory 15G . [root@servername /]# thanks! Happens on nix systems that utilize a package manager and doesn't auto remove old/unnecessary packages. This is usually set to off by default in case you have the need to roll back. A full boot partition has all kinds of weird side effects, and is likely causing your issues too. I use Ubuntu and I run : sudo apt-get autoremove to free up space on my boot partition. Check here: https://unix.stackexchange.com/questions/23330/how-can-i-remove-orphan-packages-in-fedora If you're using dnf as your package manager, the command would be : dnf autoremove or for yum: yum autoremove Edited January 22, 2018 by mastrmind11
Luke 42077 Posted January 22, 2018 Posted January 22, 2018 @@sector327 please let us know if this helps. thanks !
sector327 4 Posted January 22, 2018 Author Posted January 22, 2018 Hi there, can you please attach the emby server log? you can learn how to do that here: https://emby.media/community/index.php?/topic/739-how-to-report-a-problem/ Thanks. Can do! Sort of. The link you specify doesn't address where the emby logs are for Linux systems. I looked in /var/ log.. and didnt see any emby folders. I did find emby-server in /opt. But no folder named "log". Let me know where to find the emby server logs (on linux) and I'll grab em
Luke 42077 Posted January 22, 2018 Posted January 22, 2018 Logs can be accessed directly within the server's web interface by navigating to Help -> Logs.
sector327 4 Posted January 22, 2018 Author Posted January 22, 2018 Logs can be accessed directly within the server's web interface by navigating to Help -> Logs. web page returns "Site can't be reached. refused to connect". I've ssh'd into the server though & can grab them from whereever they are though
sector327 4 Posted January 22, 2018 Author Posted January 22, 2018 (edited) or for yum: yum autoremove good idea. Just tried it and there's not enough room to even do that. dangit... [root@servername ~]# yum autoremove error: db5 error(28) from dbenv->open: No space left on device error: cannot open Packages index using db5 - No space left on device (28) error: cannot open Packages database in /var/lib/rpm Error: Error: rpmdb open failed [root@servername ~]# I'm trying to run "du -h" on / and its returning 15G. Looks like the biggest files are in /usr. Trying to get a log file out that shows it all. Edited January 22, 2018 by sector327
sector327 4 Posted January 22, 2018 Author Posted January 22, 2018 sucess in the fact that I deleted the downloaded emby rpm and then could run "yum autoremove" (which only freed up about 2mb) but still, that allowed a little space and then I could restart the emby server via "systemctl restart emby-server". I was then able to grab the log from the other night when it initially croaked. I've attached it to this post as its pretty long to paste. 2018_01_19 Emby log.txt
Luke 42077 Posted January 22, 2018 Posted January 22, 2018 Hi, before proceeding i would suggest installing the latest version. thanks !
sector327 4 Posted January 22, 2018 Author Posted January 22, 2018 (edited) ok done. (I plugged back in my usb samba drive. edited the fstab to re-enable the automount and then mounted again via "mount -a" to bring it all back online. ) I then downloaded the new package, dumped it in my writable area of the samba share and did an upgrade via "dnf install emby-server-rpm_3.2.70.0_x86_64.rpm". I just checked my free space and it looks like I'm good again. Hopefully it doesn't fill back up. Will let you know if it does. Still not certain if the filling up of root was b/c of me (or my config) or an error w/ Emby. Hopefully my experience can help w/ the dev of the product. It's the best ever!!! [root@servername uploads]# df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 1.9G 0 1.9G 0% /dev tmpfs 1.9G 8.0K 1.9G 1% /dev/shm tmpfs 1.9G 1.1M 1.9G 1% /run tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup /dev/mapper/fedora-root 15G 9.1G 6.0G 61% / tmpfs 1.9G 4.0K 1.9G 1% /tmp /dev/sda2 976M 183M 726M 21% /boot /dev/sda1 200M 11M 189M 6% /boot/efi tmpfs 384M 0 384M 0% /run/user/0 /dev/sdb2 3.7T 1.6T 2.1T 44% /media/usb1 Edited January 22, 2018 by sector327
sector327 4 Posted January 22, 2018 Author Posted January 22, 2018 (edited) annnnnnnnd we're dead. ugh... ok, the df result is the same as post #11 so we're not full. so that's good. Looking in cockpit, I see this about emby failing and something about NOTINSTALLED. Did I not upgrade it correctly? (thought I did). I can start it (systemctl start emby-service) and it will run for a moment or two and then I can check it ((systemctl status emby-service) and see it's failed again. Edited January 22, 2018 by sector327
mastrmind11 722 Posted January 22, 2018 Posted January 22, 2018 try the equivalent of apt-get update && apt-get upgrade for Fedora. Perhaps you're missing some dependencies. Can also check the systemctl logs and last resort dmesg (if there is such a thing on Fedora). tldr, there's not enough info there to figure this out.
sector327 4 Posted January 22, 2018 Author Posted January 22, 2018 try the equivalent of apt-get update && apt-get upgrade for Fedora. Perhaps you're missing some dependencies. Can also check the systemctl logs and last resort dmesg (if there is such a thing on Fedora). tldr, there's not enough info there to figure this out. I totally agree there's not enough info there. Trying to find it. I just attached the dmesg output from the reboot but I don't really see anything helpful there. I did an update though (its just "dnf update" btw) just to be sure. rebooted and the prob was still there. will continue to poke at it as I have time. thanks for helping. dmesg_output.txt 1
mastrmind11 722 Posted January 22, 2018 Posted January 22, 2018 Also, you might give the docker a try as a test. Simple enought to get going (though admittedly I haven't tried since the move to core). Just as a baseline, anyway.
Luke 42077 Posted January 23, 2018 Posted January 23, 2018 annnnnnnnd we're dead. ugh... ok, the df result is the same as post #11 so we're not full. so that's good. Looking in cockpit, I see this about emby failing and something about NOTINSTALLED. Did I not upgrade it correctly? (thought I did). I can start it (systemctl start emby-service) and it will run for a moment or two and then I can check it ((systemctl status emby-service) and see it's failed again. This is resolved for the next release and is on the beta channel for testing. thanks. 1
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now