Thumbnail Image Extraction Running Incredibly Slow

[Fawkesguy 1]

Fresh Emby (4.8.1.0) install on MacOS.  M2 Mini with 16GB RAM.  Media scan runs fine.  Initial took several hours, but daily scans now finish in about 20 minutes.  3300 movies, 300 TV shows on a Synology NAS.

Other scheduled tasks run as expected in a reasonable time.

Thumbnail image extraction is running very very slowly.  1% per day.  I would imagine that this is not normal, since at its current rate, this task will take over three months to complete.

Any help would be greatly appreciated.  Log attached.

 

 

embyserver.txt

[Abobader 3464]

Hello Fawkesguy,

** This is an auto reply **

Please wait for someone from staff support or our members to reply to you.

It's recommended to provide more info, as it explain in this thread:

Thank you.

Emby Team

[rbjtech 5284]

According to the log, it's doing them about 2 seconds per media file - over a NAS, this is not that slow.  Ensure your temp folder is put on a fast ssd/nvme if possible - as this process writes 100's of images per media item and them compiles them into the .bif file - so a LOT of I/O.    if the temp file is not local, this may be the root cause.

[Fawkesguy 1]

15 minutes ago, rbjtech said:

According to the log, it's doing them about 2 seconds per media file - over a NAS, this is not that slow.  Ensure your temp folder is put on a fast ssd/nvme if possible - as this process writes 100's of images per media item and them compiles them into the .bif file - so a LOT of I/O.    if the temp file is not local, this may be the root cause.

Thank you for the reply.  When you say "temp file", are you referring to the Cache folder?  When I look under Server Info --> Folders, I see:

Program Data
Cache
Metadata
Logs
Transcoding Temporary Files

 

 

[rbjtech 5284]

Yes, pretty sure it's the cache folder 

[Lessaj 467]

If you search for QuickImageSeriesExtractor in your logs you'll see the command including the temp path it's writing the images to. You can pretty much replicate the command as written as long as you make the necessary library links for ffmpeg (IE export LD_LIBRARY_PATH on linux, not sure if you need to do anything on windows). I've played around with trying different places to dump the image files or increasing threads, but I couldn't really improve the speed of it in any meaningful way. It is a lot of small IO for sure though, hundreds of files are created if doing 10 second intervals.

[Fawkesguy 1]

ok, my cache was on an external NVME, but I've switched it to its default location.  Hopefully that helps.

[rbjtech 5284]

13 minutes ago, Fawkesguy said:

ok, my cache was on an external NVME, but I've switched it to its default location.  Hopefully that helps.

If it was nvme anyway, then it's likely not going to help.   In reality though, this is not being 'slow'.   

Remember emby has to read the entire file - skipping every 10 seconds to then grab the image, scale it, write it to local disk, then move to the next.  

If you turn on debug logging, you will actually get a full log for each item - maybe do for a few to see the process it goes though.

The dev's (softworkz in particular) has spent a lot of time on perfecting this - and as the poster above says, it's pretty much at max throughput, short of doing multiple libraries at once and then you simply move the bottleneck to other parts of your system.

Edited February 22, 2024 by rbjtech

[Fawkesguy 1]

Damn.  OK, well I guess I'll check it again in three months and see where it's at.   

[Fawkesguy 1]

Probably a silly question, but what the hell:  Is there any way to copy my thumbnails from Plex to Emby?

 

[rbjtech 5284]

2 minutes ago, Fawkesguy said:

Probably a silly question, but what the hell:  Is there any way to copy my thumbnails from Plex to Emby?

 

are they .bif files ?

[Fawkesguy 1]

17 minutes ago, rbjtech said:

are they .bif files ?

Actually, the Thumbnails folder in my Plex metadata folder it empty.  Now I'm not sure where they reside.

For Emby, under cache/images/resized-images I see .webp and .jpg

 

[rbjtech 5284]

ah ok - if you haven't opted to keep the thumbnail/bif files with the media on the nas (recommended) - then it's not going to do you much good anyway as you won't know where to put them.

If it's not too late  I'd suggest keeping the bif files on the nas, next to the media - that way, they can be re-used and don't need to be regenerated ...

Edited February 22, 2024 by rbjtech

[Happy2Play 9782]

5 minutes ago, Fawkesguy said:

For Emby, under cache/images/resized-images I see .webp and .jpg

These are just device/browser cached images unrelated to interval/bif files.

If you do not save interval images with media then I believe they are in metadata/library per item folder.

But you can wildcard search the folder to find the bif files.

Edited February 22, 2024 by Happy2Play

[Fawkesguy 1]

OK, I'm gonna stop the scan and change the setting to store bif files with media.