SuperCurro 4 Posted July 21, 2020 Posted July 21, 2020 (edited) Hello Since 3 weeks ago I have random crash from emby that freeze the Ubuntu Linux 18.04.3 server. Kern.log Jul 20 22:18:28 SERVIDOR2 kernel: [605193.781202] ffmpeg[512]: segfault at ffffffff8bc5a2d6 ip 00007f205b7664e2 sp 00007fff9a8b6328 error 5 in libc-2.27.so[7f205b5f8000+1e7000] Jul 20 22:18:28 SERVIDOR2 kernel: [605193.815515] EmbyServer[338]: segfault at ffffffff894cfc6b ip 00007fb93eee94e2 sp 00007fb8457f8838 error 5 in libc-2.27.so[7fb93ed7b000+1e7000] Jul 20 22:18:28 SERVIDOR2 kernel: [605193.944154] traps: compiz[9066] trap invalid opcode ip:7fe7a4f4b067 sp:7ffdc80b97c0 error:0 in libc-2.27.so[7fe7a4ddd000+1e7000] Jul 20 22:18:28 SERVIDOR2 kernel: [605193.979717] systemd-journal[2578]: segfault at 3535632f ip 00007fdeb87b3536 sp 00007ffe1d296b20 error 6 in libc-2.27.so[7fdeb8725000+1e7000] Jul 20 22:18:28 SERVIDOR2 kernel: [605193.980482] systemd[1]: segfault at ffffffff894cfc56 ip 00007f96350314e2 sp 00007ffd8f89dcd8 error 5 in libc-2.27.so[7f9634ec3000+1e7000] Jul 20 22:18:28 SERVIDOR2 kernel: [605193.983265] gnome-session-b[8676]: segfault at ffffffff894cfc80 ip 00007eff871be4e2 sp 00007ffeafe40f88 error 5 in libc-2.27.so[7eff87050000+1e7000] Jul 20 22:18:28 SERVIDOR2 kernel: [605194.006308] systemd-logind[7697]: segfault at 69d2aa59 ip 00007f9ffc1bf98d sp 00007ffd19644ec0 error 4 in libc-2.27.so[7f9ffc128000+1e7000] Jul 20 22:18:28 SERVIDOR2 kernel: [605194.012190] dbus-daemon[8294]: segfault at 63696669 ip 00007fac26e37d7a sp 00007ffd9f851678 error 4 in libc-2.27.so[7fac26cce000+1e7000] Jul 20 22:18:28 SERVIDOR2 kernel: [605194.028535] polkitd[7783]: segfault at ffffffff894cfc56 ip 00007f79a3ad74e2 sp 00007ffe51d11e68 error 5 in libc-2.27.so[7f79a3969000+1e7000] Jul 20 22:18:28 SERVIDOR2 kernel: [605194.030554] dconf worker[9152]: segfault at 7f9371d094a0 ip 00007f93713db2f0 sp 00007f9368ebc308 error 7 in libc-2.27.so[7f937126d000+1e7000] syslog Jul 20 22:18:28 SERVIDOR2 kernel: [605193.781202] ffmpeg[512]: segfault at ffffffff8bc5a2d6 ip 00007f205b7664e2 sp 00007fff9a8b6328 error 5 in libc-2.27.so[7f205b5f8000+1e7000] Jul 20 22:18:28 SERVIDOR2 kernel: [605193.815515] EmbyServer[338]: segfault at ffffffff894cfc6b ip 00007fb93eee94e2 sp 00007fb8457f8838 error 5 in libc-2.27.so[7fb93ed7b000+1e7000] Jul 20 22:18:28 SERVIDOR2 kernel: [605193.944154] traps: compiz[9066] trap invalid opcode ip:7fe7a4f4b067 sp:7ffdc80b97c0 error:0 in libc-2.27.so[7fe7a4ddd000+1e7000] Jul 20 22:18:28 SERVIDOR2 systemd[1]: emby-server.service: Main process exited, code=killed, status=11/SEGV Jul 20 22:18:28 SERVIDOR2 systemJul 20 22:18:28 ^����e: Failed with result 'signal'. Jul 20 22:18:28 SERVIDOR2 kernel: [605193.979711] systemd-journald[2578]: /var/log/journal/c5533d1adb9e4329a81ca618e26ba7ea/system.journal: Journal file corrupted, rotating. Jul 20 22:18:28 SERVIDOR2 kernel: [605193.979717] systemd-journal[2578]: segfault at 3535632f ip 00007fdeb87b3536 sp 00007ffe1d296b20 error 6 in libc-2.27.so[7fdeb8725000+1e7000] Jul 20 22:18:28 SERVIDOR2 kernel: [605193.980482] systemd[1]: segfault at ffffffff894cfc56 ip 00007f96350314e2 sp 00007ffd8f89dcd8 error 5 in libc-2.27.so[7f9634ec3000+1e7000] Jul 20 22:18:28 SERVIDOR2 kernel: [605193.983265] gnome-session-b[8676]: segfault at ffffffff894cfc80 ip 00007eff871be4e2 sp 00007ffeafe40f88 error 5 in libc-2.27.so[7eff87050000+1e7000] Jul 20 22:18:28 SERVIDOR2 kernel: [605193.996543] rfkill: input handler enabled Jul 20 22:18:28 SERVIDOR2 kernel: [605194.006308] systemd-logind[7697]: segfault at 69d2aa59 ip 00007f9ffc1bf98d sp 00007ffd19644ec0 error 4 in libc-2.27.so[7f9ffc128000+1e7000] Jul 20 22:18:28 SERVIDOR2 kernel: [605194.012190] dbus-daemon[8294]: segfault at 63696669 ip 00007fac26e37d7a sp 00007ffd9f851678 error 4 in libc-2.27.so[7fac26cce000+1e7000] Jul 20 22:18:28 SERVIDOR2 kernel: [605194.028535] polkitd[7783]: segfault at ffffffff894cfc56 ip 00007f79a3ad74e2 sp 00007ffe51d11e68 error 5 in libc-2.27.so[7f79a3969000+1e7000] Jul 20 22:18:28 SERVIDOR2 kernel: [605194.030554] dconf worker[9152]: segfault at 7f9371d094a0 ip 00007f93713db2f0 sp 00007f9368ebc308 error 7 in libc-2.27.so[7f937126d000+1e7000] Jul 20 22:18:28 SERVIDOR2 kernel: [605194.085766] Firewall: *UDP6OUT Blocked* IN= OUT=enp4s0 SRC=fe80:0000:0000:0000:9c18:6831:0ae3:3805 DST=ff02:0000:0000:0000:0000:0000:0000:00fb LEN=254 TC=0 HOPLIMIT=255 FLOWLBL=606266 PROTO=UDP SPT=5353 DPT=5353 LEN=214 UID=111 GID=120 Jul 20 22:18:29 SERVIDOR2 kernel: [605194.357122] WARNING: CPU: 1 PID: 9187 at /build/linux-lF11Cx/linux-4.15.0/mm/page_counter.c:27 page_counter_cancel+0x1b/0x20 Jul 20 22:18:29 SERVIDOR2 kernel: [605194.357124] Modules linked in: btrfs zstd_compress ufs qnx4 hfsplus hfs minix ntfs msdos jfs xfs nf_log_ipv6 ip6t_REJECT nf_reject_ipv6 ip6table_filter nf_log_ipv4 nf_log_common xt_tcpudp xt_recent xt_connlimit xt_owner nf_nat_ftp xt_REDIRECT nf_nat_redirect xt_conntrack nf_conntrack_ftp xt_LOG xt_limit xt_multiport iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 ip6table_mangle ip6table_raw ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 nf_nat nf_conntrack ip6_tables iptable_mangle iptable_raw ipt_REJECT nf_reject_ipv4 iptable_filter binfmt_misc wmi_bmof zfs(PO) zunicode(PO) zavl(PO) icp(PO) zcommon(PO) znvpair(PO) spl(O) snd_hda_codec_realtek edac_mce_amd snd_hda_codec_generic snd_hda_intel kvm_amd snd_hda_codec snd_hda_core kvm snd_hwdep snd_pcm snd_seq_midi nouveau irqbypass Jul 20 22:18:29 SERVIDOR2 kernel: [605194.357151] snd_seq_midi_event crct10dif_pclmul snd_rawmidi crc32_pclmul ghash_clmulni_intel pcbc snd_seq mxm_wmi snd_seq_device video snd_timer ttm aesni_intel drm_kms_helper snd aes_x86_64 crypto_simd glue_helper cryptd soundcore k10temp ccp drm fb_sys_fops syscopyarea sysfillrect sysimgblt shpchp mac_hid wmi sch_fq_codel parport_pc ppdev lp parport ip_tables x_tables autofs4 raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear dm_snapshot dm_bufio igb i2c_algo_bit dca alx ptp i2c_piix4 ahci mdio pps_core libahci gpio_amdpt gpio_generic Jul 20 22:18:29 SERVIDOR2 kernel: [605194.357172] CPU: 1 PID: 9187 Comm: java Tainted: P O 4.15.0-111-generic #112-Ubuntu Jul 20 22:18:29 SERVIDOR2 kernel: [605194.357172] Hardware name: Gigabyte Technology Co., Ltd. AX370-Gaming 5/AX370-Gaming 5, BIOS F25 01/16/2019 Jul 20 22:18:29 SERVIDOR2 kernel: [605194.357174] RIP: 0010:page_counter_cancel+0x1b/0x20 Jul 20 22:18:29 SERVIDOR2 kernel: [605194.357175] RSP: 0018:ffffb1b20b2a3908 EFLAGS: 00010296 Jul 20 22:18:29 SERVIDOR2 kernel: [605194.357176] RAX: ffffd6268c7935a0 RBX: ffffd6268c793360 RCX: 0000000000000000 Jul 20 22:18:29 SERVIDOR2 kernel: [605194.357177] RDX: ffffd6268c7932a0 RSI: 0000000000000001 RDI: ffffd6268c793360 Jul 20 22:18:29 SERVIDOR2 kernel: [605194.357177] RBP: ffffb1b20b2a3908 R08: ffff9a489e812800 R09: 000000000001866e Jul 20 22:18:29 SERVIDOR2 kernel: [605194.357178] R10: 0000000000000001 R11: ffffffffffffffff R12: 0000000000000001 Jul 20 22:18:29 SERVIDOR2 kernel: [605194.357178] R13: ffffb1b20b2a3980 R14: ffffb1b20b2a3a30 R15: ffffffffaec625c8 Jul 20 22:18:29 SERVIDOR2 kernel: [605194.357180] FS: 00007f4cdcdfd700(0000) GS:ffff9a489ec40000(0000) knlGS:0000000000000000 Jul 20 22:18:29 SERVIDOR2 kernel: [605194.357180] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jul 20 22:18:29 SERVIDOR2 kernel: [605194.357181] CR2: 00007f253ceae170 CR3: 000000009d60a000 CR4: 00000000003406e0 Jul 20 22:18:29 SERVIDOR2 kernel: [605194.357181] Call Trace: I though it was the last update of libc6 so I reverse from 2.27-3ubuntu1.2 to 2.27-3ubuntu1, but is still unstable. Package name libc6-dev Update system APT Package description amd64 GNU C Library: Development Libraries and Header Files Current state New version 2.27-3ubuntu1.2 Installed version 2.27-3ubuntu1 Available version 2.27-3ubuntu1.2 Installation source Bionic It is clear that the root cause is the transcoding, but I dont know what to do now, help would be very apreciate. Thanks! ffmpeg-transcode-6bb40eb7-1412-42eb-bebc-75b76b767c75_1.txt embyserver-63730914070.txt Edited July 21, 2020 by SuperCurro
SuperCurro 4 Posted July 24, 2020 Author Posted July 24, 2020 I updated the version of but still the same problem Jul 24 20:14:45 SERVIDOR2 kernel: [124208.766710] traps: sh[19099] general protection ip:7f2b4c946eae sp:7fff8fac3488 error:0 in libc-2.27.so[7f2b4c908000+1e7000] Jul 24 20:14:45 SERVIDOR2 kernel: [124208.781953] traps: sh[19101] general protection ip:7f5e4d7f3eae sp:7ffcc77d23f8 error:0 in libc-2.27.so[7f5e4d7b5000+1e7000] Jul 24 20:14:45 SERVIDOR2 kernel: [124208.783211] traps: sh[19102] general protection ip:7f4c2b246eae sp:7ffdcfe27998 error:0 in libc-2.27.so[7f4c2b208000+1e7000] Jul 24 20:14:45 SERVIDOR2 kernel: [124208.784916] traps: sh[19103] general protection ip:7f0e4e80feae sp:7ffd13e2eb68 error:0 in libc-2.27.so[7f0e4e7d1000+1e7000] Jul 24 20:14:45 SERVIDOR2 kernel: [124208.977529] traps: sh[19109] general protection ip:7fe16929deae sp:7ffcbc3af238 error:0 in libc-2.27.so[7fe16925f000+1e7000] Jul 24 20:14:45 SERVIDOR2 kernel: [124208.980946] traps: sh[19110] general protection ip:7fcb318f0eae sp:7ffc68147db8 error:0 in libc-2.27.so[7fcb318b2000+1e7000] Jul 24 20:14:45 SERVIDOR2 kernel: [124208.984213] traps: sh[19111] general protection ip:7f29b535beae sp:7ffc16b58748 error:0 in libc-2.27.so[7f29b531d000+1e7000] And the server crashs
SuperCurro 4 Posted July 25, 2020 Author Posted July 25, 2020 Tested the beta version emby-server-deb_4.5.0.14_amd64.deb It last 1 hour instead of 10 minutes, but it crash Tested to updtate ffmpeg, but I dont know how to configure emby to use the latest version of ffmpeg. Any idea? If it is software the only test I can do is to recover 6 month old backup imagen and test without update. If it crash then is a hardware problem. If someone has a better idea please suggest it.
Q-Droid 803 Posted July 25, 2020 Posted July 25, 2020 Your log shows segfaults for more than just Emby. If you uninstall Emby or at least stop and disable it do the other apps still crash?
SuperCurro 4 Posted July 25, 2020 Author Posted July 25, 2020 (edited) 34 minutes ago, Q-Droid said: Your log shows segfaults for more than just Emby. If you uninstall Emby or at least stop and disable it do the other apps still crash? Thanks for reply If emby is not working is stable, If emby is decoding a movie crash, and yes crash all the programs, but because the emby. Is like a memory fail and I have to turn off the current to be able to reboot. I have two servers almost identical, same hardware, same software, and is the first time since one year I have stability problems. No OC, and no other crashes. It started One month ago with the update of libc6, I recover the snapshot just before that update but the same problem. Edited July 25, 2020 by SuperCurro
Q-Droid 803 Posted July 25, 2020 Posted July 25, 2020 Are both servers crashing? If only one is crashing and they're nearly identical can you swap the memory between them? I know you think Emby may be causing the crash but too many other processes are also failing. Emby might be the only one with the workload that triggers the event but too many other processes are segfaulting on libc. Can you purge libc then reinstall? I haven't tried this so I don't know what dependents could be impacted, perhaps too many.
SuperCurro 4 Posted July 25, 2020 Author Posted July 25, 2020 (edited) 12 hours ago, Q-Droid said: Are both servers crashing? If only one is crashing and they're nearly identical can you swap the memory between them? I know you think Emby may be causing the crash but too many other processes are also failing. Emby might be the only one with the workload that triggers the event but too many other processes are segfaulting on libc. Can you purge libc then reinstall? I haven't tried this so I don't know what dependents could be impacted, perhaps too many. The servers are in different countries. But I can run with only one RAM memory slot, then test and later do the same with the other Enviado desde mi ELS-NX9 mediante Tapatalk Edited July 26, 2020 by SuperCurro English
Luke 38500 Posted July 27, 2020 Posted July 27, 2020 Yea that would be a useful result to find out. Thanks.
SuperCurro 4 Posted July 27, 2020 Author Posted July 27, 2020 4 hours ago, Luke said: Yea that would be a useful result to find out. Thanks. Ok, is confirmed, it was one memory slot. Now I am running with only 8GB in one slot and is 100% stable. (now 2 days running). With only the bad slot in 10 minutes of streaming it crashed. Is the first time I have this problem, the equipment is a Ryzen 7 1700 with 3200 Mhz memory that has been perfectly stable for the last 2 years. Thanks for the help.
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now