Image collages are being recreated on every library scan

[SamES 893]

I don't know when this started, but image collages appear to be recreated on every library scan.  This seems unnecessary as it takes ages and really ties up the server.  This results in delays for clients.

 

server version 3.0.5768.4

 

...

2015-10-21 19:01:41.7789 Info - ImageProcessor: Creating image collage and saving to \\MYBOOKLIVEDUO\Movies\MediaBrowser\cache\temp\1dab9260-af58-45c3-ab20-b7d8d326da1b.png

2015-10-21 19:01:45.2948 Info - ImageProcessor: Completed creation of image collage and saved to \\MYBOOKLIVEDUO\Movies\MediaBrowser\cache\temp\1dab9260-af58-45c3-ab20-b7d8d326da1b.png

2015-10-21 19:01:45.4198 Info - ImageProcessor: Creating image collage and saving to \\MYBOOKLIVEDUO\Movies\MediaBrowser\cache\temp\ca47ebce-c53c-433b-b8cb-307efd7b2e0e.png

2015-10-21 19:01:47.0920 Info - ImageProcessor: Completed creation of image collage and saved to \\MYBOOKLIVEDUO\Movies\MediaBrowser\cache\temp\ca47ebce-c53c-433b-b8cb-307efd7b2e0e.png

2015-10-21 19:01:47.3264 Info - ImageProcessor: Creating image collage and saving to \\MYBOOKLIVEDUO\Movies\MediaBrowser\cache\temp\88a9c9f5-b117-4dd3-87fb-88a549e637d5.png

2015-10-21 19:01:49.5924 Info - ImageProcessor: Completed creation of image collage and saved to \\MYBOOKLIVEDUO\Movies\MediaBrowser\cache\temp\88a9c9f5-b117-4dd3-87fb-88a549e637d5.png

...

 

There also appears to be a lot of FileNotFoundExceptions from the Dynamic Image Provider

 

...

2015-10-21 19:46:55.5767 Error - App: Error in Dynamic Image Provider

*** Error Report ***

Version: 3.0.5768.4

Command line: C:\Users\Home\AppData\Roaming\Emby-Server\System\MediaBrowser.ServerApplication.exe

Operating system: Microsoft Windows NT 6.2.9200.0

Processor count: 4

64-Bit OS: True

64-Bit Process: False

Program data path: C:\Users\Home\AppData\Roaming\Emby-Server

Application Path: C:\Users\Home\AppData\Roaming\Emby-Server\System\MediaBrowser.ServerApplication.exe

Could not find file '\\MYBOOKLIVEDUO\Movies\MediaBrowser\cache\temp\149c97bc-47e8-4e00-b3e2-41f7cdc85960.png'.

System.IO.FileNotFoundException

  at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)

  at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)

...

 

Full log file attached.

 

Thanks

server-63580992648.txt

Edited October 21, 2015 by SamES

[Luke 37269]

that's not true. that only happens in the library contents are changing. you can avoid dynamic images altogether by just assigning your own images if you want.

[SamES 893]

that's not true. that only happens in the library contents are changing. you can avoid dynamic images altogether by just assigning your own images if you want.

Hi @@Luke.  In the log there are several library scans, (I currently have them manually set to several times a day) and every scan is causing the images to be updates.  Nothing has been added to the library in this time, and definitely not in these folders.

 

As a clue, if I open these images directly, they are all collages from my photos library.  Is there a reason why the server might be falsely detecting changes in photo libraries and triggering this?

 

Last scan, 1,944 collage images were made.  This is the same as the last scan only a few hours earlier.

 

If I look at the first and last image created in each scan, they are always of the same folder. 

 

How are changes to photo libraries detected during a scan?

Edited October 22, 2015 by SamES

[MEB 26]

I too am having this issue and it keeps filling up my temp folder with magick files some delete on their own but every so often one sticks and wont delete, I only have 13GB free on my OS drive so these magick files keep crashing my system. How can I turn off these collages for my photo folders? Also how can I change where magick makes its temp files?

 

You said set own images? That's a lot of work to go through 5 years of photos and set each folder its own image.

Log.zip

Edited October 23, 2015 by MEB

[MEB 26]

Any ideas on this?

[Happy2Play 8360]

Any ideas on this?

What temp folder is filling up?

 

Since your log shows the server can find files in your cache that were supposedly just saved.

2015-10-23 01:23:42.1253 Info - ImageProcessor: Creating image collage and saving to M:\Media Browser Cache\temp\3d8f2559-cb2c-40b1-90d1-55218d68f4ef.png 

Could not find file 'M:\Media Browser Cache\temp\3d8f2559-cb2c-40b1-90d1-55218d68f4ef.png'

Edited October 24, 2015 by Happy2Play

[MEB 26]

I was on Dev but switched to Beta after this problem started but all that did was change the temp directory from C:\Windows\Temp to C:\Users\%username%\AppData\Local\Temp

 

I have turned off library scans until this is figured out but for some reason it still runs randomly even with no triggers defined.

 

So anyway to move where the temp magick files are created or to turn off collages for photo folders? I may just remove photos altogether until this is fixed.

 

Edit: Moved my TEMP folders off the SSD, ill see what happens now. Maybe I just need more space to let it finish before it crashes my system.

Edited October 24, 2015 by MEB

[SamES 893]

I was on Dev but switched to Beta after this problem started but all that did was change the temp directory from C:\Windows\Temp to C:\Users\%username%\AppData\Local\Temp

 

I have turned off library scans until this is figured out but for some reason it still runs randomly even with no triggers defined.

 

So anyway to move where the temp magick files are created or to turn off collages for photo folders? I may just remove photos altogether until this is fixed.

 

Edit: Moved my TEMP folders off the SSD, ill see what happens now. Maybe I just need more space to let it finish before it crashes my system.

 

Under Server Settings->Advanced it is possible to set the location of the whole cache folder, which contains the temp folder.

 

I would suggest shutting down the server, then copying (don't move) the whole cache folder to a new drive.

 

Restart the server, then change the Cache path in Server Settings->Advanced.

 

Again, I would suggest restarting the server.

 

It should now be using the remote folder for all cache data, including temp files.

 

Monitor it for a few days before deleting the folder from c drive.

 

Note, I have always had my cache folder on a remote share.  I have never tried moving an existing folder this way but I expect this is the safest way to try and move it.  

[MEB 26]

Under Server Settings->Advanced it is possible to set the location of the whole cache folder, which contains the temp folder.

 

I would suggest shutting down the server, then copying (don't move) the whole cache folder to a new drive.

 

Restart the server, then change the Cache path in Server Settings->Advanced.

 

Again, I would suggest restarting the server.

 

It should now be using the remote folder for all cache data, including temp files.

 

Monitor it for a few days before deleting the folder from c drive.

 

Note, I have always had my cache folder on a remote share.  I have never tried moving an existing folder this way but I expect this is the safest way to try and move it.  

 

That wasn't the folder that was being filled, I said it was 

C:\Windows\Temp

and

C:\Users\%username%\AppData\Local\Temp

 

Those are not the cache folders, the cache folders haven't been on the OS drive in a long time. 

I did change the location of the temp folders within environmental variables and so far since there has been no crash since there is 3TB free for it to fill up. But image magick is still leaving files behind.

But Scan Media Library has been running for 6 hours it keeps restarting.

Im guessing that Media Scan fails at some point and whatever files imagemagick has been working with just leaves those files behind and starts making new ones.

 

I had removed my picture folders and had no issues for a few hours, the scan finished and there were no leftover magick files. So a good fix for me would be a way to disable collage on photo folders since they dont really add much functionality, most picture folders have more than 6 pictures in them anyways.

 

Then there is the random crashes when playing from Android client... makes it hard to see if the Scan Media will ever complete.

Edited October 24, 2015 by MEB

[SamES 893]

Sorry, my bad on picking the wrong temp folder.  I've never noticed anything from the server in my system temp folders, so you may be right that something is crashing and leaving bits behind.

[MEB 26]

Let the Scan run all night, woke up to a crashed Emby and 200MB of magick files in my temp folder. Log attached, its a big one. 50MB uncompressed.

Log (2).zip

[MEB 26]

Any clues in that log file? Should I just remove my photos again and not be able to use that feature?

[MEB 26]

-

Edited October 25, 2015 by MEB

[Happy2Play 8360]

Adding Photos to my library the only temp folder I see with images from the imageprocessor is Emby-Server\cache\temp nothing is going to C:\Windows\Temp or C:\Users\%username%\AppData\Local\Temp.  When library scan was complete Emby-Server\cache\temp had 276 images.

 

Your log also shows (ImageProcessor started with 8 max concurrent image processes) instead of (4 Max).

[MEB 26]

Adding Photos to my library the only temp folder I see with images from the imageprocessor is Emby-Server\cache\temp nothing is going to C:\Windows\Temp or C:\Users\%username%\AppData\Local\Temp.  When library scan was complete Emby-Server\cache\temp had 276 images.

 

Your log also shows (ImageProcessor started with 8 max concurrent image processes) instead of (4 Max).

Well magick files are going to the windows temp and I dont know how to change that, but that is not really the issue, the issue is the scan can never finish for some reason, it runs and runs until it crashes emby and it then leaves the magick files behind. I just want to be able to turn off collages, They add nothing to the picture folders, only place they are nice is on collections.

[SamES 893]

that's not true. that only happens in the library contents are changing. you can avoid dynamic images altogether by just assigning your own images if you want.

 

Hi @@Luke, version 3.0.5781.0 appears to have resolved this issue.  No more new dynamic images from photo library every scan. Library scans are also now back under 10 minutes from 2 hours.

 

Thanks, you can mark this as completed

Edited November 5, 2015 by SamES