High CPU usage with no streaming in progress

[Thuzad 50]

Hello,

For some time now, I have noticed that Emby uses a lot of CPU when there is no streaming going on. I've looked at the logs and it doesn't do anything at all at that time.

Screenshot monitoring the host server: Screenshot_20210611_181159.png

Screenshot monitoring of the Emby container: Screenshot_20210611_181443.png

Logs sent by PM to @Luke

[Luke 42078]

The realtime monitor is reacting to lots of changed files reported by your file system, so try turning that off and then restarting the server. Also see here:

https://support.emby.media/support/solutions/articles/44002210894-corrupt-database

[Carlo 4561]

I see the same thing in your logs. If you search for "Info LibraryMonitor:" in the log you will see all the files being refreshed.

You also have what looks like a locked database and the link Luke provided will show you an easy way to fix this.

Let us know how you make out.

[Thuzad 50]

Thanks for the information and the analyze.
I will try to fix the SQLite database when I have less streaming users.

Concerning the refresh of my media, it seems normal since I just restarted the server. So it does a scan at startup. Unless I'm wrong?

Is it normal that it refreshes all my media and that I have so many logs concerning "Info LibraryMonitor:" ? Maybe I'm doing something that changes my media...

I have indeed seen the SQLite error but I would like to understand the other errors I have so I can solve them.

Could you tell me what each of these errors are?

2021-06-11 17:42:00.798 Error TaskManager: Error
	*** Error Report ***
	Version: 4.6.1.0
	Command line: /opt/emby-server/system/EmbyServer.dll -programdata /var/lib/emby -ffdetect /opt/emby-server/bin/ffdetect -ffmpeg /opt/emby-server/bin/ffmpeg -ffprobe /opt/emby-server/bin/ffprobe -restartexitcode 3 -updatepackage emby-server-deb_{version}_amd64.deb
	Operating system: Linux version 5.4.0-74-generic (buildd@lgw01-amd64-038) (gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)) #83-Ubuntu SMP Sat May 8 02:35:39 UTC 2021
	Framework: .NET Core 3.1.13
	OS/Process: x64/x64
	Runtime: opt/emby-server/system/System.Private.CoreLib.dll
	Processor count: 4
	Data path: /var/lib/emby
	Application path: /opt/emby-server/system
	System.InvalidOperationException: System.InvalidOperationException: Sequence contains no elements
	   at System.Linq.ThrowHelper.ThrowNoElementsException()
	   at System.Linq.Enumerable.First[TSource](IEnumerable`1 source)
	   at Emby.Server.Sync.SyncManager.GetTargetsWithReadySyncItems()
	   at Emby.Server.Sync.DownloadNotificationScheduledTask.Execute(CancellationToken cancellationToken, IProgress`1 progress)
	   at Emby.Server.Implementations.ScheduledTasks.ScheduledTaskWorker.ExecuteInternal(TaskOptions options)
	Source: System.Linq
	TargetSite: Void ThrowNoElementsException()

 

2021-06-11 18:01:06.329 Error App: Error in Screen Grabber
	*** Error Report ***
	Version: 4.6.1.0
	Command line: /opt/emby-server/system/EmbyServer.dll -programdata /var/lib/emby -ffdetect /opt/emby-server/bin/ffdetect -ffmpeg /opt/emby-server/bin/ffmpeg -ffprobe /opt/emby-server/bin/ffprobe -restartexitcode 3 -updatepackage emby-server-deb_{version}_amd64.deb
	Operating system: Linux version 5.4.0-74-generic (buildd@lgw01-amd64-038) (gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)) #83-Ubuntu SMP Sat May 8 02:35:39 UTC 2021
	Framework: .NET Core 3.1.13
	OS/Process: x64/x64
	Runtime: opt/emby-server/system/System.Private.CoreLib.dll
	Processor count: 4
	Data path: /var/lib/emby
	Application path: /opt/emby-server/system
	System.Exception: System.Exception: ffmpeg image extraction failed for file:"/media/storage4/films/The Girl from Monaco (2008)/LA_FILLE_DE_MONACO.Title3.mkv" (output file not found)
	   at Emby.Server.MediaEncoding.ImageExtraction.ImageExtractionManager.ExtractImageInternal(ReadOnlyMemory`1 inputPath, ReadOnlyMemory`1 container, MediaStream videoStream, Nullable`1 streamIndex, Nullable`1 threedFormat, Nullable`1 offset, Boolean useThumbNailFilter, CancellationToken cancellationToken)
	   at Emby.Server.MediaEncoding.ImageExtraction.ImageExtractionManager.ExtractImage(ReadOnlyMemory`1 inputPath, ReadOnlyMemory`1 container, MediaStream videoStream, Nullable`1 streamIndex, MediaProtocol protocol, Boolean isAudio, Nullable`1 threedFormat, Nullable`1 offset, CancellationToken cancellationToken)
	   at MediaBrowser.Providers.MediaInfo.VideoImageProvider.GetVideoImage(Video item, CancellationToken cancellationToken)
	   at MediaBrowser.Providers.Manager.ItemImageProvider.RefreshFromProvider(BaseItem item, IDynamicImageProvider provider, Boolean providerReportsChanges, ImageRefreshOptions refreshOptions, TypeOptions savedOptions, LibraryOptions libraryOptions, List`1 downloadedImages, RefreshResult result, CancellationToken cancellationToken)
	Source: Emby.Server.MediaEncoding
	TargetSite: Void MoveNext()

 

2021-06-11 18:04:22.923 Error App: Error in Proxer
	*** Error Report ***
	Version: 4.6.1.0
	Command line: /opt/emby-server/system/EmbyServer.dll -programdata /var/lib/emby -ffdetect /opt/emby-server/bin/ffdetect -ffmpeg /opt/emby-server/bin/ffmpeg -ffprobe /opt/emby-server/bin/ffprobe -restartexitcode 3 -updatepackage emby-server-deb_{version}_amd64.deb
	Operating system: Linux version 5.4.0-74-generic (buildd@lgw01-amd64-038) (gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)) #83-Ubuntu SMP Sat May 8 02:35:39 UTC 2021
	Framework: .NET Core 3.1.13
	OS/Process: x64/x64
	Runtime: opt/emby-server/system/System.Private.CoreLib.dll
	Processor count: 4
	Data path: /var/lib/emby
	Application path: /opt/emby-server/system
	System.Net.WebException: System.Net.WebException: The remote server returned an error: (403) Forbidden.
	   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
	   at System.Net.WebClient.GetWebResponse(WebRequest request, IAsyncResult result)
	   at System.Net.WebClient.GetWebResponseTaskAsync(WebRequest request)
	   at System.Net.WebClient.DownloadBitsAsync(WebRequest request, Stream writeStream, AsyncOperation asyncOp, Action`3 completionDelegate)
	   at MediaBrowser.Plugins.Anime.Providers.Proxer.Api.WebRequestAPI(String link)
	   at MediaBrowser.Plugins.Anime.Providers.Proxer.Api.Search_GetSeries(String title, CancellationToken cancellationToken, Boolean bettersearchresults)
	   at MediaBrowser.Plugins.Anime.Providers.Proxer.Api.FindSeries(String title, CancellationToken cancellationToken)
	   at MediaBrowser.Plugins.Anime.Providers.Proxer.ProxerSeriesProvider.GetMetadata(SeriesInfo info, CancellationToken cancellationToken)
	   at MediaBrowser.Providers.Manager.MetadataService`2.ExecuteRemoteProviders(BaseItem originalItem, MetadataResult`1 temp, LibraryOptions libraryOptions, String logName, TIdType id, IEnumerable`1 providers, CancellationToken cancellationToken)
	Source: System.Net.Requests
	TargetSite: System.Net.WebResponse EndGetResponse(System.IAsyncResult)

 

 

[Luke 42078]

No, that's not the library scan. It's exactly what I said - you have the realtime monitor enabled and it is reacting to lots of changes in your files.

Try also removing the anime plugin and restarting the server.

[Thuzad 50]

Hello,

I have noticed locked in the logs again. And it affects the record of where a person stopped playing media.

I tried to continue to follow the documentation but the dump command doesn't seem to work:

root@emby:~ # sqlite3 library.db ".recover" | sqlite3 library-recovered.db
Error: unknown command or invalid arguments:  "recover". Enter ".help" for help

Another question, if I reset my database. Will I lose the status of my read media?

Edited June 15, 2021 by Floflobel

[Happy2Play 9780]

26 minutes ago, Floflobel said:

Hello,

I have noticed locked in the logs again. And it affects the record of where a person stopped playing media.

I tried to continue to follow the documentation but the dump command doesn't seem to work:

root@emby:~ # sqlite3 library.db ".recover" | sqlite3 library-recovered.db
Error: unknown command or invalid arguments:  "recover". Enter ".help" for help

Another question, if I reset my database. Will I lose the status of my read media?

There is no need to recover your database as that has nothing to do with locked database.

Locked database happens when the database is not shutdown properly on the previous session and the new session can not take control of the database.

You have to ensure Emby is shutdown, then delete the shm and wal files and start Emby.

The hard part is figuring out why/what is shutting down and starting Emby to cause this issue.  I know there is a topic in the Synology section that the NAS shutdown task appears to be causing this issue as it appear it is not shutting down Emby properly it would appear to just kill it instead of a graceful shutdown.

[Thuzad 50]

Thank you for all this information. Indeed I confused a database corruption with a database lock.

It could surely happen because of an OOM. I already had several OOM due to a lack of memory. I do a daily restart of Emby (systemctl restart emby-server), but I don't think this is the cause.

I will try to understand when the locked happens and I will keep you informed. I'm also trying to see if the CPU usage is still high when the server is not doing anything.

Another question, I removed the "Anime" plugin and I don't have the errors anymore. What can I do to put it back and avoid these errors.

[Luke 42078]

What do you think the anime plugin is doing for you?

[Thuzad 50]

I was using this plugin to get the info from all my animations. But I think it is not necessary...

[Luke 42078]

On 6/17/2021 at 5:14 PM, Floflobel said:

I was using this plugin to get the info from all my animations. But I think it is not necessary...

Was it actually getting anything for you?

[Thuzad 50]

I think so, on manga/anime. But I will check in the next weeks when I will have new media.

[Thuzad 50]

After an analysis over several days, the problem no longer occurs (high CPU usage). Thank you for your help.

[Luke 42078]

Thanks for the feedback.

[Thuzad 50]

Hello,

I have very regularly locked my database. I can't figure out where the problem comes from. The server restarts only at night and I restart it properly via systemctl.

Could you help me to analyze the problem? Where do I start?

[Thuzad 50]

Is there a way to detect locked except by looking at the logs? (via api, or other)

[Happy2Play 9780]

2 hours ago, Floflobel said:

Hello,

I have very regularly locked my database. I can't figure out where the problem comes from. The server restarts only at night and I restart it properly via systemctl.

Could you help me to analyze the problem? Where do I start?

This is a defect that will be resolved in 4.6.4.0 as mentioned in other topics, if you need a fix right now you can update to beta 4.7.0.3 then downgrade when 4.6.4.0 is released.

But you can remove the database lock to your current system by removing the shm and wal files with Emby shutdown per the KB.

2 hours ago, Floflobel said:

Is there a way to detect locked except by looking at the logs? (via api, or other)

Not that I know of as it is a detect of previous running session not shutting down properly causing the new session to not be able to take control of the database.  But only dev can say the cause that was identified.