Monitor Library Scan Progress

[aerogems 0]

Mainly because the Kodi integration looks to be much better compared to Plex, I set up a (native) Emby server on my DS916+ for testing. I added my movie collection with no real issues, but when it comes to adding my TV Show collection... I started running it around 4-5pm yesterday, it's about quarter past 11am the following day now after letting the process run all night long, and when I refresh the library view not a single show appears.

 

Depending on where I look in Emby I'm either at 14%-- same as it was when I went to bed last night -- or 36.5% completed -- which appeared at some point during the night and hasn't moved in the 4 or so hours since I got up. The only indications I have of anything being done are hearing the HDD in my NAS, the performance monitor widget saying my CPU is at 99% perpetually, and occasional bursts of network activity.

 

I get that with somewhere on the order of 10K+ episodes, it's going to take a while to process all of them, but shouldn't I have seen at least SOME kind of visual update in the library view on the server? I would have expected that after 12+ hours it would have completed at least a few shows. Am I missing something somewhere? With Plex you have the Alerts section on the dashboard that gives you at least a vague idea of what is going on right then. Is there anything comparable with Emby?

[PenkethBoy 2063]

have a look at your logs - will be fairly obvious what its doing - yes more feed back would be good - but on a library build you would get swamped with info and slow things down even more

 

first scan always takes a long time - lots of threads on this

 

also depends what you asked emby to do re thumbnails/metadata lookup etc for each library

 

just be patient 10k+ on even an powerful pc will take hours - on a NAS - usually low power - lots longer

 

once done it will be very quick to do another and subsequent library scans

[aerogems 0]

The problem with the logs tends to be way too much noise for very little signal when you're just looking to check up on scanning agent progress... but here's the other issue. I just checked the log, and it's full of references to my movie library, but not a single one for my TV library. I even checked the previous log that was created at midnight... still nothing.

 

It's not the amount of time it's taking to do the scan that's the issue, it's that I can't tell for certain it isn't just caught in a loop or that the scanning process didn't go zombie. There's zero direct evidence that it's doing anything except pegging my NAS CPU and HDD. About 3 hours since my last post and it's still at 14 and 36.5% respectively depending on where I look. Still not a single entry in my TV Show library. I would have expected it to at least tick through a tenth of a percent by now. If I could at least see it was adding one or two new shows to my library, that would at least indicate it was still working properly.

[PenkethBoy 2063]

is the log file growing in size

[aerogems 0]

Yes, but when I pay close attention to the time stamps, the only entries seem to be when I go to a different section on the server. I go up a ways and I see a lot of things like this (sanitized to remove potentially sensitive info)

2020-04-01 03:28:12.409 Info App: [LinuxMount] Checking we can attempt to mount [smb://[NAS]/Movies/Movies/Dragon Ball Z Movies & Specials/Dragon_Ball_Z_Movie_-_13.mkv], Extension = [.mkv], Operating System = [Linux], Executables Available = [True].
2020-04-01 03:28:12.409 Info App: [LinuxMount] Checking we can attempt to mount [smb://[NAS]/Movies/Movies/Dragon Ball Z Movies & Specials/Dragon_Ball_Z_Movie_-_13.mkv], Extension = [.mkv], Operating System = [Linux], Executables Available = [True].
2020-04-01 03:28:12.411 Info App: ProcessRun 'quick-extract-imageseries' Execute: /var/packages/EmbyServer/target/ffmpeg/bin/ffmpeg -f matroska -threads 1 -skip_interval 10 -copyts -i "http://127.0.0.1:8096/Items/File?Id=smb%3A%2F%2F[NSD]%2FMovies%2FMovies%2FDragon+Ball+Z+Movies+%26+Specials%2F%5B%5DDragon_Ball_Z_Movie_-_13.mkv" -an -sn -s 320x180 -vsync cfr -r 0.1 -f image2 "/var/packages/EmbyServer/target/var/cache/temp/de4e068c967341669b620341e426a2e6/img_%05d.jpg"
2020-04-01 03:28:12.782 Info HttpServer: HTTP GET http://127.0.0.1:8096/Items/File?Id=smb%3A%2F%2Fscottsnas%2FMovies%2FMovies%2FDragon+Ball+Z+Movies+%26+Specials%2F%5BCleo%5DDragon_Ball_Z_Movie_-_13_(Dual+Audio_10bit_BD1080p_x265).mkv. UserAgent: Lavf/58.31.104
2020-04-01 03:28:12.794 Info HttpServer: HTTP Response 500 to 127.0.0.1. Time: 12ms. http://127.0.0.1:8096/Items/File?Id=smb%3A%2F%2F[nas]%2FMovies%2FMovies%2FDragon+Ball+Z+Movies+%26+Specials%2F%5B%5DDragon_Ball_Z_Movie_-_13.mkv
2020-04-01 03:28:12.893 Info App: ProcessRun 'quick-extract-imageseries' Process exited with code 1
2020-04-01 03:28:13.044 Error ChapterImagesTask: Error creating thumbnails for smb://[nas]/Movies/Movies/Dragon Ball Z Movies & Specials/Dragon_Ball_Z_Movie_-_13.mkv
	*** Error Report ***
	Version: 4.3.1.0
	Command line: /var/packages/EmbyServer/target/server/EmbyServer.exe -package synology -programdata /var/packages/EmbyServer/target/var -ffmpeg /var/packages/EmbyServer/target/ffmpeg/bin/ffmpeg -ffprobe /var/packages/EmbyServer/target/ffmpeg/bin/ffprobe -ffdetect /var/packages/EmbyServer/target/ffmpeg/bin/ffdetect -restartexitcode 121
	Operating system: Unix 3.10.105.0
	64-Bit OS: True
	64-Bit Process: True
	User Interactive: False
	Mono: 5.18.0.240 (tarball Fri Jan 18 10:51:45 UTC 2019)
	Runtime: file:///volume1/@appstore/EmbyServer/3rdparty/mono/5.18.0.240/lib/mono/4.5/mscorlib.dll
	Processor count: 4
	Program data path: /var/packages/EmbyServer/target/var
	Application directory: /volume1/@appstore/EmbyServer/releases/4.3.1.0
	System.Exception: System.Exception: No thumbnails created for smb://[nas]/Movies/Movies/Dragon Ball Z Movies & Specials/Dragon_Ball_Z_Movie_-_13.mkv 320
	  at MediaBrowser.Providers.MediaInfo.ThumbnailGenerator.CreateThumbnailSet (MediaBrowser.Controller.Entities.Video item, MediaBrowser.Model.Configuration.LibraryOptions libraryOptions, System.Int32 width, System.Threading.CancellationToken cancellationToken) [0x003f1] in <b715abeb47a74caa86d135cf64129f07>:0 
	  at MediaBrowser.Providers.MediaInfo.ThumbnailGenerator.CreateThumbnailSets (MediaBrowser.Controller.Entities.Video item, MediaBrowser.Model.Configuration.LibraryOptions libraryOptions, System.Threading.CancellationToken cancellationToken) [0x0009d] in <b715abeb47a74caa86d135cf64129f07>:0 
	  at MediaBrowser.Providers.MediaInfo.ThumbnailGenerator.RefreshThumbnailImages (MediaBrowser.Controller.Entities.Video item, MediaBrowser.Model.Configuration.LibraryOptions libraryOptions, MediaBrowser.Controller.Providers.IDirectoryService directoryService, System.Collections.Generic.List`1[T] chapters, System.Boolean extractImages, System.Boolean saveChapters, System.Threading.CancellationToken cancellationToken) [0x0035b] in <b715abeb47a74caa86d135cf64129f07>:0 
	Source: mscorlib
	TargetSite: Void Throw()

I see absolutely nothing like that related to a single TV show.

Edited April 1, 2020 by aerogems

[PenkethBoy 2063]

ignore the TV shows for now - its not got to those yet

 

its failing on creating thumbnails for movies 

 

you have mounted - a smb path - which emby is having trouble with e.g. HTTP Response 500 to 127.0.0.1. Time: 12ms. http://127.0.0.1:8096/Items/File?Id=smb%3A%2F%2F[nas]......

 

so you files are not on your syno?

 

where is emby server installed - the syno?

[aerogems 0]

The files are on my NAS, but I had to use the network paths so I could later use native paths with Kodi. The embysrv user has read/write access to the folder where the movies are stored.

[PenkethBoy 2063]

well - as a test - create a library that points directly to a test Movies/tv show folder with just a few files and see if emby then behaves - if necessary remove the other libraries so you only have a smaller set of movies to scan etc to limit the issue(s) - and its not working anyway

 

if it does work as i suspect - then you need to sort out the smb path issue - might be permissions of the subfolders and files - do you have a user/password setup on the smb paths as IIRC emby only works without a user/pwd for authentication as there is nowhere to enter these details

 

essentially sort the smb issue and emby will behave and scan your files - keep things simple - try network paths later once you have a working environment and then experiment

 

TIP - for the first scan - disable creating thumbnails per library - with the library scan and things will go much quicker - you can turn it on again later after the first scan has completed - its a cpu hog depending on what options you choose later

[aerogems 0]

I think I figured out the issue, and I'll mention it here on the off chance someone else does the same thing.

 

When setting up the library, I didn't pay close enough attention to the documentation about the optional network paths. I deleted all the existing libraries and created a new one based on a folder where I put a single show and left all the scanning options default. I then gave Emby the internal "/volume1/path/to/tv/" path and then also manually entered in the SMB share name. Now it seems to be happy. It worked with the one show folder I created for testing purposes and I've actually seen it update the status when I added my main TV show. It's gone up about 10% since I started writing this post. We'll see what I end up with after it's finished chewing on the rest of my TV shows.

[PenkethBoy 2063]

thats what i was going to suggest next - the optional paths for Kodi

 

It will still take a long time to do the first scan - but should be happier now

 

just check the logs once its done a few shows and make sure you dont have anymore errors

[aerogems 0]

Yeah, I'm not expecting that the progress indicator is necessarily accurate, but this time I'm seeing a full set of entries in the library and if I hit refresh after a few minutes I see one or two more with cover art. So it's at least progress from before.

 

This is one area Plex handles things much better and it would be nice if the Emby devs could copy them.

 

Anyway, thanks for at least attempting to help. Turns out it was my own dumb ass that was at fault, but probably wouldn't have figured that out as quickly without you. So thanks.

[PenkethBoy 2063]

np - always happy to help a dumbass   

 

glad its sorted

 

and yes the % complete is some odd bit of code - will get to 80ish and grind to a halt - dont panic thats normal - but the % will slowly tick up in .1's

 

look in the logs folder as if you have thumbnails being created you will see lots of files like these 

 

extract-multi-image-8b926a37-825c-40f4-84dc-93f8da639a54_1.txt which is the result of ffmpeg doing its thing

 

Good Luck and be patient