4.6 - How are Collections stored and other changes?

[blade005 172]

5 hours ago, Happy2Play said:

In the end you could copy from data/collection to metadata/collection then delete the collection.xml from metadata\collection as it is obsolete (but will have to rename all folders dropping [boxset].  Then on a refresh a nfo will be written if option is enabled.

FYI. I did as you suggested above with my collection. Made a copy of data/collection folders and stripped out collection.xml files and removed [boxset] from each folder name. Tested a few items by moving them into the new metadata/collection location. Ran a REFRESH METADATA with SEARCH FOR MISSING METADATA selection on the test collection item. It did create the collection.nfo file in the new location, but if the folder with collection images were still in the old data/collection location the nfo file would continue to point to that location for images. Tested again, but removed the collection folder from old data/collection location and then a REFRESH METADATA would create collection.nfo file and it pointed to images in new metadata/collection location. If you want to keep your previous images for collections it is a time consuming task to move the folders with images to the new location, remove them from the old location and then run a REFRESH METADATA on each collection item. I wasn't sure what the SCAN METADATA FOLDER selection in Scheduled Tasks menu would do, so I grunted out the REFRESH METADATA method using the multi-select option in collections. Did groups at a time and then would check the newly created folders for accuracy until all collection items were updated. Removed the old COLLECTION item in library, restarted the server and ran a full library scan. Checked collections with all folders updated in new metadata/collection location and all folders removed from old data/collection location. All seems to be displaying as expected. It is great if all this collection information is now in the database and nfo files of individual movies for future updates and/or migrations. Just nice to also have the collection data and corresponding images in the updated metadata location as well.

Thank you for the migration suggestion. 

[Happy2Play 8296]

5 minutes ago, blade005 said:

FYI. I did as you suggested above with my collection. Made a copy of data/collection folders and stripped out collection.xml files and removed [boxset] from each folder name. Tested a few items by moving them into the new metadata/collection location. Ran a REFRESH METADATA with SEARCH FOR MISSING METADATA selection on the test collection item. It did create the collection.nfo file in the new location, but if the folder with collection images were still in the old data/collection location the nfo file would continue to point to that location for images. Tested again, but removed the collection folder from old data/collection location and then a REFRESH METADATA would create collection.nfo file and it pointed to images in new metadata/collection location. If you want to keep your previous images for collections it is a time consuming task to move the folders with images to the new location, remove them from the old location and then run a REFRESH METADATA on each collection item. I wasn't sure what the SCAN METADATA FOLDER selection in Scheduled Tasks menu would do, so I grunted out the REFRESH METADATA method using the multi-select option in collections. Did groups at a time and then would check the newly created folders for accuracy until all collection items were updated. Removed the old COLLECTION item in library, restarted the server and ran a full library scan. Checked collections with all folders updated in new metadata/collection location and all folders removed from old data/collection location. All seems to be displaying as expected. It is great if all this collection information is now in the database and nfo files of individual movies for future updates and/or migrations. Just nice to also have the collection data and corresponding images in the updated metadata location as well.

Thank you for the migration suggestion. 

I just tested Moving from data\collections to metadata\collections renaming dropping [boxset] and deleting the collection.xml, then did a Scan library files on the collection and the moved images were used and nfo was create.  No Refreshing involved.

[Diedrich 355]

13 minutes ago, Happy2Play said:

I just tested Moving from data\collections to metadata\collections renaming dropping [boxset] and deleting the collection.xml, then did a Scan library files on the collection and the moved images were used and nfo was create.  No Refreshing involved.

Question: but are individual movies' nfo files pointing to the correct metadata/collections location?

[Happy2Play 8296]

5 minutes ago, Diedrich said:

Question: but are individual movies' nfo files pointing to the correct metadata/collections location?

It is just a <set> tag for the items not a location.

All 451 collections were still present, but it looks like the only thing that is happening is the images are being recached but are all present.

Not completely sure of the check for providerid to create <set> tag though.

 <set tmdbcolid="91361">
    <name>Halloween Collection</name>
  </set>
  <uniqueid type="TmdbCollection">91361</uniqueid>
  <tmdbcollectionid>91361</tmdbcollectionid>

 

Edited May 21, 2021 by Happy2Play

[Diedrich 355]

3 minutes ago, Happy2Play said:

It is just a <set> tag for the items not a location.

Ah yes. Right, right....gotta get my thinking away from the old method and remember that this is now just tag linking and not absolute path linking. Thanks.

[Happy2Play 8296]

43 minutes ago, Happy2Play said:

I just tested moving from data\collections to metadata\collections renaming dropping [boxset] and deleting the collection.xml, then did a Scan library files on the collection and the moved images were used and nfo was create.  No Refreshing involved.

Correction I had to run the Scan Metadata Folder task also, as I did notice broken images in image editor on Collections.

[blade005 172]

8 hours ago, Happy2Play said:

I just tested Moving from data\collections to metadata\collections renaming dropping [boxset] and deleting the collection.xml, then did a Scan library files on the collection and the moved images were used and nfo was create.  No Refreshing involved.

Cool. I wasn't sure about 'moving' all of those folders from the old location before building them out in the new metadata\collections location. I left them in the old location for testing and the newly created collection.nfo file would continue to reference the old image locations. Good to know you can move all the folders over, do the requisite naming maintenance and have them refresh. I will likely forget all of this when something changes in a few years and we have to migrate again!!

[blade005 172]

15 hours ago, Luke said:

Your movie nfo files, if you use nfo metadata.

With the 4.6 update do movie.nfo files in local folders now contain the appropriate reference data if they are in a collection? So, if you were to install Emby Server on new hardware and have a new database built it would read the movie.nfo files during the library scan and also create the appropriate collections and collection.nfo file? I understand that collection images would have to be refreshed in a new install, but it is nice if collections can now follow with the movie.nfo file.

[tmb4016 86]

Don't forget the option in the Collection Library Settings to 'Save Artwork to Media folder' - For me this is required for new collections to ensure that the artwork and the collection.nfo file are created in the metadata\collections folder.

When this is unticked only the collection.nfo file is created. 

I am pretty sure I had this ticked for the refresh of the collection during the Beta testing and my artwork was stored corrrectly in the new metadata path. ( may be wrong as it was done in the early testing when there were a few bugs)

 

Edited May 22, 2021 by tmb4016

[blade005 172]

14 minutes ago, tmb4016 said:

Don't forget the option in the Collection Library Settings to 'Save Artwork to Media folder' - For me this is required for new collections to ensure that the artwork and the collection.nfo file are created in the metadata\collections folder.

When this is untick only the collection.nfo file is created. 

I am pretty sure I had this ticked for the refresh of the collection during the Beta tresting and my artwork was stoered corrrectly in the new metadata path. ( may be wrong as it was done in the early testing when there were a few bugs)

 

That is not an available option for the COLLECTION Library after the 4.6 stable release. @Lukenoted earlier in the thread that it is now a Virtual Grouping and not a traditional Library with all of the setting options.

18 hours ago, Luke said:

Because it's not a library. It's just a virtual grouping.

 

 

[Carlo 4330]

Check out your options for the COLLECTIONS LIBRARY

[blade005 172]

13 minutes ago, cayars said:

Check out your options for the COLLECTIONS LIBRARY

Ah. Okay. DOOOOH!!!

I didn't realize that selecting the entry in the LIBRARY section would pull that up. I was used to selecting the 3-dot menu, like for all other library items to get to settings. When I only saw EDIT and EDIT IMAGE, I thought there were no SETTING options available. Guess I should have just clicked on EDIT. Such a doofus.  

Apologies @tmb4016 .

[Diedrich 355]

Anybody else having issues with adding more than one image (any type) after adding the first one?

1. Add a poster image
2. Add a Thumb (it's usually here where the web UI screen will go blank)
3. Add a background (or it'll happen here)

Either way, it's happening to me on different collections but almost always after trying to add a second or third image.

embyserver-1.txt

[Happy2Play 8296]

1 hour ago, Diedrich said:

Anybody else having issues with adding more than one image (any type) after adding the first one?

1. Add a poster image
2. Add a Thumb (it's usually here where the web UI screen will go blank)
3. Add a background (or it'll happen here)

Either way, it's happening to me on different collections but almost always after trying to add a second or third image.

embyserver-1.txt 2.98 MB · 0 downloads

Is this the example?

2021-05-21 20:41:52.337 Error Server: Error processing request
	*** Error Report ***
	Version: 4.6.0.50
	Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
	Operating system: Linux version 5.4.0-72-generic (buildd@lcy01-amd64-019) (gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)) #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021
	Framework: .NET Core 3.1.13
	OS/Process: x64/x64
	Runtime: system/System.Private.CoreLib.dll
	Processor count: 12
	Data path: /config
	Application path: /system
	System.IO.DirectoryNotFoundException: System.IO.DirectoryNotFoundException: Could not find a part of the path '/config/data/collections/The Best Movies [boxset]/landscape.png'.
	   at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func`2 errorRewriter)
	   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode)
	   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
	   at Emby.Server.Implementations.IO.ManagedFileSystem.GetFileStream(String path, FileOpenMode mode, FileAccessMode access, FileShareMode share, Int32 bufferSize, FileOpenOptions fileOpenOptions)
	   at Emby.Server.Implementations.IO.ManagedFileSystem.GetFileStream(String path, FileOpenMode mode, FileAccessMode access, FileShareMode share, FileOpenOptions fileOpenOptions)
	   at Emby.Server.Implementations.IO.IOManager.GetResponse(HttpRequestOptions options, MediaProtocol protocol)
	   at Emby.Server.Implementations.IO.IOManager.GetResponse(HttpRequestOptions options)
	   at Emby.Server.Implementations.HttpServer.HttpResultFactory.GetContent(StaticFileResultOptions options, Int64 offset, Int64 length, CancellationToken cancellationToken)
	   at Emby.Server.Implementations.HttpServer.FileWriter.SetContentResponseHeaders(Nullable`1 reportedTotalContentLength, CancellationToken cancellationToken)
	   at Emby.Server.Implementations.HttpServer.HttpResultFactory.GetStaticResult(IRequest requestContext, StaticResultOptions options)
	   at MediaBrowser.Api.Images.ImageService.GetImage(ImageRequest request, Int64 itemId, BaseItem item, Boolean isHeadRequest)
	   at Emby.Server.Implementations.Services.ServiceController.GetTaskResult(Task task)
	   at Emby.Server.Implementations.Services.ServiceHandler.ProcessRequestAsync(HttpListenerHost appHost, IRequest httpReq, IResponse httpRes, RestPath restPath, String responseContentType, CancellationToken cancellationToken)
	   at Emby.Server.Implementations.HttpServer.HttpListenerHost.RequestHandler(IRequest httpReq, ReadOnlyMemory`1 urlString, ReadOnlyMemory`1 localPath, CancellationToken cancellationToken)
	Source: System.Private.CoreLib
	TargetSite: Void ThrowExceptionForIoErrno(ErrorInfo, System.String, Boolean, System.Func`2[Interop+ErrorInfo,Interop+ErrorInfo])

2021-05-21 20:41:53.777 Debug Skia: SKFileStream had a problem reading /config/data/collections/The Best Movies [boxset]/poster.png. Attempting workaround...
2021-05-21 20:41:53.778 Error ImageProcessor: Error encoding image
	*** Error Report ***
	Version: 4.6.0.50
	Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
	Operating system: Linux version 5.4.0-72-generic (buildd@lcy01-amd64-019) (gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)) #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021
	Framework: .NET Core 3.1.13
	OS/Process: x64/x64
	Runtime: system/System.Private.CoreLib.dll
	Processor count: 12
	Data path: /config
	Application path: /system
	System.IO.DirectoryNotFoundException: System.IO.DirectoryNotFoundException: Could not find a part of the path '/config/data/collections/The Best Movies [boxset]/poster.png'.
	   at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func`2 errorRewriter)
	   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode)
	   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
	   at System.IO.FileSystem.CopyFile(String sourceFullPath, String destFullPath, Boolean overwrite)
	   at Emby.Drawing.Skia.SkiaEncoder.GetSKFileStream(String path)
	   at Emby.Drawing.Skia.SkiaEncoder.EncodeImage(String inputPath, String outputPath, Boolean autoOrient, Nullable`1 orientation, Int32 quality, ImageProcessingOptions options, ImageFormat selectedOutputFormat)
	   at Emby.Drawing.ImageProcessor.ProcessImage(ImageProcessingOptions options, CancellationToken cancellationToken)
	Source: System.Private.CoreLib
	TargetSite: Void ThrowExceptionForIoErrno(ErrorInfo, System.String, Boolean, System.Func`2[Interop+ErrorInfo,Interop+ErrorInfo])

 

[Diedrich 355]

4 minutes ago, Happy2Play said:

Is this the example?

yes. the The Best Movies Collection images

[Happy2Play 8296]

1 minute ago, Diedrich said:

yes. the The Best Movies Collection images

They are still using the old location according to the log. @Luke

[Diedrich 355]

1 minute ago, Happy2Play said:

They are still using the old location according to the log.

The images may be referenced in a location but those files do not physically exist because I deleted them.

My solution was to do a metadata refresh with image replacement ticked. I then uploaded my images again and all is well.

[Happy2Play 8296]

3 minutes ago, Diedrich said:

The images may be referenced in a location but those files do not physically exist because I deleted them.

My solution was to do a metadata refresh with image replacement ticked. I then uploaded my images again and all is well.

Basically image paths in the database since you deleted them.

[pünktchen 1258]

For the brave of you, here's the promised plugin: https://emby.media/community/index.php?/topic/98353-migration-plugin/

[Ceten 27]

I found this topic because I have 2 collection libraries on my server (Synology DS916+) after the update to 4.6.1.0, both pointing to the data/collections folder:

 

The collection with mixed contents seems to be a folder view with no movies in the collections anymore, the other one seems to be a correct copy of my old collections library.
Is my assumption correct that I could fix this by moving the images using @pünktchen's plugin and then removing the library with mixed content and the data/collections folder?
I was already using the Auto Boxsets plugin but don't want to lose my custom collections.

Edited May 25, 2021 by Ceten
typo

[Luke 37094]

3 minutes ago, Ceten said:

I found this topic because I have 2 collection libraries on my server (Synology DS916+) after the update to 4.6.1.0, both poting to the data/collections folder:

 

The collection with mixed contents seems to be a folder view with no movies in the collections anymore, the other one seems to be a correct copy of my old collections library.
Is my assumption correct that I could fix this by moving the images using @pünktchen's plugin and then removing the library with mixed content and the data/collections folder?
I was already using the Auto Boxsets plugin but don't want to lose my custom collections.

I'm still trying to hunt down the conditions causing the second library to be created, although it is safe for you to remove. But other than that I don't think there is anything for you to fix. Simply use the new one that was created.

[Diedrich 355]

1 hour ago, Ceten said:

and then removing the library with mixed content and the data/collections folder?

Just copy them somewhere else before you delete them in case you need to reference them, use their images, or restore them.

Edited May 25, 2021 by Diedrich

[Carlo 4330]

I think Luke was referring to removing the library itself in Emby not contents on disc.

[frylich 5]

On 5/20/2021 at 10:33 PM, Luke said:

Because it's not a library. It's just a virtual grouping.

 

 

Hello everyone, is there a way to remove the virtual group "Collections" from the main menu? I don't need collections. Thank you very much!

[pwhodges 1532]

In your user's settings, on the "Home Screen" page, you can turn it off in the listing near the bottom:

Paul