Jump to content

100% CPU Usage

rogerlipscombe

By rogerlipscombe
in Linux

 Share

Recommended Posts

rogerlipscombe

rogerlipscombe   1

Posted
  • Author
Posted

Installing 4.2 seems to have fixed it. Thanks.

 

Nope. Not fixed. Currently using 180% CPU. Two threads both repeatedly doing the following:

stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E02 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=471823660, ...}) = 0
Link to comment
Share on other sites
groenator

groenator   11

Posted
Posted

I think it depends on what type of hardware the user has. For me, from first day the CPU usage is the same, over 100% when the TV guide is refreshed. Emby is not that smart with multithread. I am using an intel I7 8 Core CPU and emby only uses one core. Shame!

Link to comment
Share on other sites
rogerlipscombe

rogerlipscombe   1

Posted
  • Author
Posted (edited)

I think it depends on what type of hardware the user has. For me, from first day the CPU usage is the same, over 100% when the TV guide is refreshed. Emby is not that smart with multithread. I am using an intel I7 8 Core CPU and emby only uses one core. Shame!

 

But my bug's got nothing to do with the TV guide refresh. It's got everything to do with trying to find a suitable filename when making a recording. We can see that from the stack traces that I captured.

 

...which is why I suggested that you raise a different thread (and I apologise for being brusque). You've got a different bug. The fact that it manifests as 100% CPU usage is a symptom, but the cause is almost certainly different. Unfortunately, I can't find a way to update the thread title to be more specific.

 

FWIW, my hardware is an Intel Baby Canyon NUC7I5BNH Core I5-7260U (which is why cooling's a problem).

Edited by rogerlipscombe
Link to comment
Share on other sites
rogerlipscombe

rogerlipscombe   1

Posted
  • Author
Posted

Regarding your findings with EnsureFileIsUnique, it should be resolved for the next release. Thanks.

 

Thanks. I've upgraded to 4.2.1.0. I assume "next release" is newer than that, though...?

 

Also, in case the data point is useful (for "yeah, that was the bug"), this seems to kick off during the day; looking in "dmesg -T", I've see the last 4 "temperature above threshold" incidents start at 16:10, 16:21, 17:30 and 12:30. I'm not sure if that's UTC or local (BST). That implies to me that it's not (?) a scheduled event. Certainly not a nightly one, anyway.

Link to comment
Share on other sites
  • 2 months later...
rogerlipscombe

rogerlipscombe   1

Posted
  • Author
Posted (edited)

I'm on 4.3.0.30 and CPU usage is improved (thrashing at ~60%, rather than 100%), but the underlying problem still exists. 'strace' on the relevant process shows:

sched_yield()                           = 0
stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=445741352, ...}) = 0
stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=445741352, ...}) = 0
stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=445741352, ...}) = 0
stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=445741352, ...}) = 0
stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=445741352, ...}) = 0
stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=445741352, ...}) = 0
stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=445741352, ...}) = 0
stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=445741352, ...}) = 0
stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=445741352, ...}) = 0
stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=445741352, ...}) = 0
stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=445741352, ...}) = 0
stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=445741352, ...}) = 0
stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=445741352, ...}) = 0
stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=445741352, ...}) = 0
stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=445741352, ...}) = 0
stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=445741352, ...}) = 0
stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=445741352, ...}) = 0
stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=445741352, ...}) = 0
stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=445741352, ...}) = 0
stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=445741352, ...}) = 0
futex(0x7fc9fc009050, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
futex(0x7fc9fc009000, FUTEX_WAKE_PRIVATE, 1) = 0
sched_yield()                           = 0

...and it just loops, doing this again and again and again.

 

Note that the named file was written in June.

 

One thing that _might_ be important: the server ran out of disk space. I wonder if it would be possible to detect this (or just the fact that it's not been successfully recording anything for a while) and display a notification on the web UI or (preferably) somewhere prominent in the client app.

Edited by rogerlipscombe
Link to comment
Share on other sites
Luke Emby Team

Luke   37008

Posted
Posted

Hi, yes we do plan to detect this and recycle partial segments throughout the transcoding process in an attempt to avoid this from happening.

Link to comment
Share on other sites
  • 1 month later...
rogerlipscombe

rogerlipscombe   1

Posted
  • Author
Posted (edited)

Any progress on this? Do you need more information from me? It's still happening a couple of times a week.

 

In fact, it literally happened immediately after I posted this. Just more of the same:

stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E07 Mr.ts", {st_mode=S_IFREG|0644, st_size=444175860, ...}) = 0
stat("/mounts/data/emby/livetv/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E07 Mr.ts", ^C{st_mode=S_IFREG|0644, st_size=444175860, ...}) = 0
...

Stack:

OS Thread Id: 0x36be (13)
TEB information is not available so a stack size of 0xFFFF is assumed
Current frame: libc.so.6!_xstat + 0x15 [/build/glibc-OTsEL5/glibc-2.27/io/../sysdeps/unix/sysv/linux/wordsize-64/xstat.c:35]
Child-SP         RetAddr          Caller, Callee
00007F397627AC30 00007f3988c426d0 System.Native.so!SystemNative_Stat2 + 0x40, calling System.Native.so!__xstat64
00007F397627ACF0 00007f3914a8a35a (MethodDesc 00007f39151e9340 + 0x11a DomainBoundILStubClass.IL_STUB_PInvoke(System.String, FileStatus ByRef))
00007F397627AD30 00007f3914a8a35a (MethodDesc 00007f39151e9340 + 0x11a DomainBoundILStubClass.IL_STUB_PInvoke(System.String, FileStatus ByRef))
00007F397627ADB0 00007f39152b378d (MethodDesc 00007f39151e83d8 + 0x2d System.IO.FileSystem.FileExists(System.String, Int32, ErrorInfo ByRef)), calling 00007f3914a89888 (stub for Interop+Sys.Stat(System.String, FileStatus ByRef))
00007F397627ADF0 00007f39152b18be (MethodDesc 00007f39151e8228 + 0x4e System.IO.PathInternal.TrimEndingDirectorySeparator(System.String)), calling (MethodDesc 00007f39151e81f8 + 0 System.IO.PathInternal.EndsInDirectorySeparator(System.ReadOnlySpan`1<Char>))
00007F397627AE50 00007f39152aa2a6 (MethodDesc 00007f39151e78c0 + 0x66 System.IO.File.Exists(System.String)), calling (MethodDesc 00007f39151e83d8 + 0 System.IO.FileSystem.FileExists(System.String, Int32, ErrorInfo ByRef))
00007F397627AE70 00007f3917b4ec83 (MethodDesc 00007f3917089790 + 0x63 Emby.LiveTV.EmbyTV.FileExists(System.String, System.String))
00007F397627AEA0 00007f3917b4ebea (MethodDesc 00007f3917089780 + 0xea Emby.LiveTV.EmbyTV.EnsureFileUnique(System.String, System.String)), calling (MethodDesc 00007f3917089790 + 0 Emby.LiveTV.EmbyTV.FileExists(System.String, System.String))
00007F397627AF00 00007f3917977e37 (MethodDesc 00007f3917181ea0 + 0x9b7 Emby.LiveTV.EmbyTV+<RecordStream>d__102.MoveNext()), calling 00007f3916b9f4d0 (stub for Emby.LiveTV.EmbyTV.EnsureFileUnique(System.String, System.String))
00007F397627B160 00007f39144a47fd (MethodDesc 00007f3914038138 + 0x8d System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object))

 

I've got monit to restart the emby process whenever it does, but that's not ideal -- e.g. if it happened in the middle of a scheduled recording.

Edited by rogerlipscombe
Link to comment
Share on other sites
rogerlipscombe

rogerlipscombe   1

Posted
  • Author
Posted

Ah. Just noticed the 4.3.1.0 upgrade; the changelist has at least one library scanning fix in it. Have applied the upgrade. Will watch to see if this happens again.

 

Thanks.

Link to comment
Share on other sites
  • 2 months later...
ArnoRoggo

ArnoRoggo   1

Posted
Posted

Ich hatte die Auslastung von 99 % eine Woche lang. Als ich die Erzeugung von Vorschaubildern (Bibliothek - Bearbeiten) deaktivierte, normalisierte sich die CPU-Auslastung.

 

I had a 99% load for a week. When I deactivated the generation of thumbnails scenes (Library - Edit), the CPU utilization normalized. 

Link to comment
Share on other sites
Luke Emby Team

Luke   37008

Posted
Posted

Yes thumbnail generation is a cpu intensive task. You are warned about this when you enable it. Thanks.

Link to comment
Share on other sites
  • 3 months later...
rogerlipscombe

rogerlipscombe   1

Posted
  • Author
Posted (edited)
On 1/3/2020 at 7:30 PM, Luke said:

Let us know how you get on. Thanks.

Running 4.4.3.0; still using an entire core. CPU temp currently 96C. Restarting emby does _not_ fix it.

Still stuck in a loop stat'ing the same file over and over again.

Edit: Aha. It's looping with 

stat("/mounts/data/emby/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E01 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=467150760, ...}) = 0

...and it's currently attempting to record (finished about 2 minutes ago, as I type this):

Brooklyn Nine-Nine - Season 4 - 1. Coral Palms Pt.1

I already have the following related files: 

'Brooklyn Nine-Nine S04E01 Coral Palms Pt.1.nfo'
'Brooklyn Nine-Nine S04E01 Coral Palms Pt.1.ts'
'Brooklyn Nine-Nine S04E01 Coral Palms Pt.nfo'
'Brooklyn Nine-Nine S04E01 Coral Palms Pt.ts'
'Brooklyn Nine-Nine S04E02 Coral Palms Pt. 2.nfo'
'Brooklyn Nine-Nine S04E02 Coral Palms Pt. 2.ts'
'Brooklyn Nine-Nine S04E02 Coral Palms Pt.nfo'
'Brooklyn Nine-Nine S04E02 Coral Palms Pt.ts'
'Brooklyn Nine-Nine S04E03 Coral Palms Pt.3.nfo'
'Brooklyn Nine-Nine S04E03 Coral Palms Pt.3.ts'
'Brooklyn Nine-Nine S04E03 Coral Palms Pt.nfo'
'Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts'

I've got "record duplicates" (or whatever it's called) turned on for this series.

Random thought: is it getting confused by the dots in the episode name? I also see the following: 

Brooklyn Nine-Nine S04E07 Mr.nfo
Brooklyn Nine-Nine S04E07 Mr. Santiago.nfo
Brooklyn Nine-Nine S04E07 Mr. Santiago.ts
Brooklyn Nine-Nine S04E07 Mr.ts

...which are all presumably supposed to be the same episode.

Edited by rogerlipscombe
Link to comment
Share on other sites
Luke Emby Team

Luke   37008

Posted
Posted
11 hours ago, rogerlipscombe said:

Running 4.4.3.0; still using an entire core. CPU temp currently 96C. Restarting emby does _not_ fix it.

Still stuck in a loop stat'ing the same file over and over again.

Edit: Aha. It's looping with 


stat("/mounts/data/emby/recordings/Brooklyn Nine-Nine (2013)/Season 4/Brooklyn Nine-Nine S04E01 Coral Palms Pt.ts", {st_mode=S_IFREG|0644, st_size=467150760, ...}) = 0

...and it's currently attempting to record (finished about 2 minutes ago, as I type this):

Brooklyn Nine-Nine - Season 4 - 1. Coral Palms Pt.1

I already have the following related files: 


'Brooklyn Nine-Nine S04E01 Coral Palms Pt.1.nfo'
'Brooklyn Nine-Nine S04E01 Coral Palms Pt.1.ts'
'Brooklyn Nine-Nine S04E01 Coral Palms Pt.nfo'
'Brooklyn Nine-Nine S04E01 Coral Palms Pt.ts'
'Brooklyn Nine-Nine S04E02 Coral Palms Pt. 2.nfo'
'Brooklyn Nine-Nine S04E02 Coral Palms Pt. 2.ts'
'Brooklyn Nine-Nine S04E02 Coral Palms Pt.nfo'
'Brooklyn Nine-Nine S04E02 Coral Palms Pt.ts'
'Brooklyn Nine-Nine S04E03 Coral Palms Pt.3.nfo'
'Brooklyn Nine-Nine S04E03 Coral Palms Pt.3.ts'
'Brooklyn Nine-Nine S04E03 Coral Palms Pt.nfo'
'Brooklyn Nine-Nine S04E03 Coral Palms Pt.ts'

I've got "record duplicates" (or whatever it's called) turned on for this series.

Random thought: is it getting confused by the dots in the episode name? I also see the following: 


Brooklyn Nine-Nine S04E07 Mr.nfo
Brooklyn Nine-Nine S04E07 Mr. Santiago.nfo
Brooklyn Nine-Nine S04E07 Mr. Santiago.ts
Brooklyn Nine-Nine S04E07 Mr.ts

...which are all presumably supposed to be the same episode.

Can you please attach the emby server log from when this happened? Thanks.

Link to comment
Share on other sites
  • 2 weeks later...
  • 2 weeks later...
rogerlipscombe

rogerlipscombe   1

Posted
  • Author
Posted

This hasn't happened recently. I'm going to assume that it's because I turned off Record Duplicates for this particular series (or maybe they stopped showing repeats of Brooklyn Nine-Nine). I've got monit monitoring it, so if it happens again, I'll know.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing
×
×
  • Create New...