Collections Conundrum

[MarcusJ 18]

Hi guys. My Collections saga continues...

As a reminder, I have two Emby Servers running separately. One is for my home, another is for a friend that I'm helping to build his media center. Both servers are completely up to date, running 4.6.4.0, and will have identical media, However, my Home Emby Server has hundreds of Collections, all of which I have built manually over the years. I thought it would be a rather simple copy and paste of the collections folder from my Home Server to the new Emby Server to get those collections onto the new server, but for some reason it hasn't work out that way

I have tried everything I can think of to achieve something that I thought was simple, and nothing works.

Most recently, I ran the punktchen migration tool on my Home Server to convert the file structure to be more in line with the new 4.6.4.0 structure that existed on the new server. I then scanned the Home Server to ensure it remained healthy after the migration. Then I copied 100% of the media from the Home server (NAS) to locally attached USB hard drives of the new server. After confirming the media copied perfectly, I ran a scan on the new emby server. It found 100% of the media perfectly, but instead of finding the 327 movie collections that existed on the Home server, the new server had a whopping 600+ movie collections. Some, but not all, of the original 327, but an additional pile of collections that were obviously system-generated during the scan.

Next, I deleted the 600+ collections folders in the new server's metadata>collections folder. Then I selected the "Scan metadata folder" on the new server. When it completed, the 600+ collections were still in my library. Then I ran a full scan of the new server.  Not only did the 600 collections remain in my library, but interestingly, the metadata>collections folder remained empty.

Totally perplexed, but morbidly curious, I deleted the entire media from the local drives and started over. After 2 days of file copying, the local drives were once again 100% identical to the Home server's files on the NAS.

This time, I completely disconnected the new server from the internet. Then I copied the content of the home server's metadata>collections folder into the same folder on the new server. I then ran a full scan on the new server. The results - zero collections. So I ran the "Scan metadata folder" on the new server. Results - no change in the collections library. Then I ran yet another full scan of the new server. The results - still zero collections.

Its maddening to waste so much time doing what should take 5 mins, but nothing I do seems to help. I have followed the recommendations given to me for the library settings on the new server, and confirmed they match the settings on the home server. The home server remains fine, and dutifully performs its nightly scheduled scan and retains the exactly correct number of collections (327).

Can anyone help me understand why this is happening and what to do to achieve the simple goal of replicating the collections from the home server onto the new server?

 

Edited September 23, 2021 by MarcusJ

[Carlo 4561]

This may or may not work.

In 4.6 and beyond collection information is kept in the database and not in xml files.  Emby should store the collection info in NFO files which can be imported on the other system.

So what I'd ask you to do is check the files that make up any collection that's missing and see if there is an NFO generated and if it has the collection info in it.
It's possible you turned on NFO writing at some point after the library was already in use so you would not have some NFO files (until the metadata is changed).

[MarcusJ 18]

Again, my thanks for responding.

Because I got frustrated, at one point I deleted the media on the local drives altogether and recopied it all from the NAS back onto the local drives. It took two days, but in so doing it copied all the media, NFO files, subtitle files, fanart, and the metadata together as I keep everything related to a movie in the same folder. So, if I understand your question, yes all of the NFO files are there and are identical to the home server's. I looked at a few of them that belong in collections and they di have the requisite "<set>" lines that I expected. For some reason, the scans are ignoring them.

[MarcusJ 18]

Also remember, at this point the new server has zero collections. I deleted the contents of the collections folder when it had 600+ collections in it, and replaced them with the 328 collections that I wanted. After that it recognizes zero collections.

[Carlo 4561]

If you deleted the collection on the new computer it should then remove the collection from the NFO file. Then if it's not in the NFO file it won't get added back.

You have to be careful how you work with collections like that.

If you like I could do a remote session with you and help you get back on tract with this.

[MarcusJ 18]

Yea, I'd welcome that. How do we do that?

[Carlo 4561]

Sending you a PM momentarily.

[Luke 42083]

Are you still having an issue with this?

[MarcusJ 18]

Yes, still a problem.

Cayars has been great and has helped a lot, but I had to stop the progress for some traveling. I'm back now and will be reaching out to cayars again very soon to pick up where we left off.

[Carlo 4561]

Hit me up when you're ready.

[Luke 42083]

Isn't this just a matter of enabling saving to movie nfo files so that both servers can use the contents?

[MarcusJ 18]

I'll let Cayars answer the root technical cause, but I'll give you my perspective...

I like the new mechanism for storing collections information, but because I wasn't aware of the change, I managed (thru user error) to lose all of my carefully built collections and had them replaced with system-generated collections. Now, apparently the best result I can achieve is to get both servers having this same set of system-generated collections. Cayars is helping me through that process.

I still have the original "collections" folder from the old folder structure with all of my custom collections inside, but apparently there is no mechanism to convert those collections into the new nfo format.

Thank you for following up.

[Carlo 4561]

Kind of a tough one to explain but it's a combination of things from manually modifying meta-data and refreshing that to not having NFO on to removing Emby built collections and creating custom collections.

The two servers have very different paths as well which doesn't help.

So we reset the 1st server up to store NFOs and forced it to create the collections again.  Now he is editing these to recreate the custom collections.
Then we will be able to copy the NFOs over to the other system.