Jump to content

'Scan Media Library' stuck on scanning playlists files, 100% cpu usage


sir_brickalot

Recommended Posts

sir_brickalot

I had a problem with a couple of files. Library scan was stuck at 94.6% and cpu usage of EmbyServer stayed at 100%. No streaming possible. When I stopped the media library scan, cpu usage went down.

All drives have been checked for file system errors using fsck and none have been found.

I rebooted the system and restarted the emby-server service a couple of times. Always the same where it gets stuck.

In the logs were no errors but the last messages have always been about some playlists:

2019-05-24 11:00:48.470 Debug App: PlaylistItemsProvider reports change to 140669 - /zsb0/zmedia/Music/_Playlists/osx/Haldern Pop 2005.m3u8
2019-05-24 11:00:48.470 Debug App: Running PlaylistItemsProvider for /zsb0/zmedia/Music/_Playlists/osx/Haldern Pop 2005.m3u8
2019-05-24 11:00:49.205 Debug App: Running PlaylistXmlProvider for /zsb0/zmedia/Music/_Playlists/osx/Haldern Pop 2005.m3u8
2019-05-24 11:00:49.205 Debug App: PlaylistXmlProvider returned no metadata for /zsb0/zmedia/Music/_Playlists/osx/Haldern Pop 2005.m3u8
2019-05-24 11:00:49.205 Debug App: PlaylistItemsProvider reports change to 140670 - /zsb0/zmedia/Music/_Playlists/osx/Haldern Pop 2011.m3u8
2019-05-24 11:00:49.205 Debug App: Running PlaylistItemsProvider for /zsb0/zmedia/Music/_Playlists/osx/Haldern Pop 2011.m3u8

I deleted the '_Playlists/' folder and after the scan went through smoothly.

 

There seems to be a bug in emby in handling these files. I attached them, maybe it helps.

 

My setup is:

Description:    Ubuntu 18.04.2 LTS
Release:    18.04
Codename:    bionic

 

Emby Version: 4.1.1.0

 

Internal drive: formatted with zfs.

2 external drives: formatted with exfat.

 

Logs are attached.

Archive.zip

hardware-detection.txt

embyserver.txt

Link to comment
Share on other sites

sir_brickalot

Hi, are these really big playlists?

Between 20 and 30 entries.

 

You can check the playlists in the archive.zip file. (Sorry, I forgot to mention I attached the playlists.)

 

I might be special that the playlists are with relative paths like this: '../../path/to/album'

Edited by sir_brickalot
Link to comment
Share on other sites

  • 2 weeks later...

What we're doing is looking up existing items in the library with the same path, and we have to do it case insensitive, which can come with a high cost.

@@MikePlanet any thoughts on performance of doing this with like vs collate nocase?

Link to comment
Share on other sites

MikePlanet

What we're doing is looking up existing items in the library with the same path, and we have to do it case insensitive, which can come with a high cost.

@@MikePlanet any thoughts on performance of doing this with like vs collate nocase?

Hi @@Luke, you can avoid the high cost for lookups when you use the "collate nocase" either for the field itself in the table or the created index - should be much (! factor 50-100) faster.

So create an index like this:

CREATE INDEX "idx_MediaItems1a" ON "MediaItems" (
	"Path" collate nocase
); 

And a request like this will be much faster:

select * from MediaItems
where path = 'path to check' collate nocase 

as it will be using the precreated index.

I made a test and it was 1,5 seconds versus 15 ms.

I assume you do not want to partial compare (i.e. like '%partialpath%') - this will be slow again and should better be done my FTS engine if possible.

Link to comment
Share on other sites

MikePlanet

note: if you also want to keep fast case sensitive comparisons keep the current index, the optimizer will automatically select the correct one depending on your statement (with or without 'collate nocase')

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...