Jump to content

Conversion from Collections XML to NFO format to capture <set> tag


swagardson
Go to solution Solved by Happy2Play,

Recommended Posts

swagardson

Hello,

I've been using Emby premium for a few years now and have generally had little-to-no issues, which has been great.

After upgrading to 4.7 (beta), I noticed items getting kicked out of my collections -- hundreds of items that I painstakingly put together. I understand that this is due to "better future compatibility" and standard format, but I do wish old collections would have remained untouched in a minor version update.

Anyways, I read these couple threads to attempt troubleshooting and playing around with various options:
https://emby.media/community/index.php?/topic/99087-latest-update-is-randomly-removing-tv-shows-from-collections-also-creating-new-collections/
https://emby.media/community/index.php?/topic/99945-films-and-tv-shows-randomly-keep-being-removed-from-collections/page/5/

And after multiple failures to have it resolved non-manually, I downgraded back to 4.5.3, which put everything back in its proper place.

Avoiding veering more off-topic, I'm wondering:

  1. How are collections being defined in the most recent versions? Purely in the item's NFO file and no longer using `data/collections/*`?
  2. If an individual item's NFO file is the new path forward, is there is a built-in Emby utility or general means to add <set> tags to an item based on each collection it is in?
  3. Related to above, if multiple <set> tags are defined in an item's NFO file, will it show up in each collection it is defined in?
    1. Reason I ask is because I have a script which iterates through items in a collection and adds a <set> tag for each collection that item is in
  4. Which options are most pertinent to accurately storing NFO metadata (e.g. "Metadata savers: [X] NFO" in a library's settings) AND reading that info in to properly populate collections?
  5. Will the "Auto Box Set" plugin wreak havoc on random items if enabled? I noticed that the only collections being made were ones resulting from that plugin tagging the NFO files, while everything else was being kicked out.

I really do appreciate the time that everyone puts into the software and answering these questions.

Thanks for any information in advance.

Link to comment
Share on other sites

Hello swagardson,

** This is an auto reply **

Please wait for someone from staff support or our members to reply to you.

It's recommended to provide more info, as it explain in this thread:

Thank you.

Emby Team

Link to comment
Share on other sites

21 hours ago, swagardson said:

Avoiding veering more off-topic, I'm wondering:

  1. How are collections being defined in the most recent versions? Purely in the item's NFO file and no longer using `data/collections/*`?
  2. If an individual item's NFO file is the new path forward, is there is a built-in Emby utility or general means to add <set> tags to an item based on each collection it is in?
  3. Related to above, if multiple <set> tags are defined in an item's NFO file, will it show up in each collection it is defined in?
    1. Reason I ask is because I have a script which iterates through items in a collection and adds a <set> tag for each collection that item is in
  4. Which options are most pertinent to accurately storing NFO metadata (e.g. "Metadata savers: [X] NFO" in a library's settings) AND reading that info in to properly populate collections?
  5. Will the "Auto Box Set" plugin wreak havoc on random items if enabled? I noticed that the only collections being made were ones resulting from that plugin tagging the NFO files, while everything else was being kicked out.

I really do appreciate the time that everyone puts into the software and answering these questions.

Thanks for any information in advance.

Did you restore the system back to 4.5.3 and not just install that version over top 4.6.4?  If you just installed 4.5.3 over top you are bound to have issues as the database was changed in 4.6.

Previously, collections were stored in XML files and should be written to the NFO file.  In the new version of Emby the XML files are no longer used and this information is now stored in the database itself and of course written to the NFO.

The BoxSet plugin is no longer used either as collection functionality is now built into each library where you can configure it.

How you proceed is going to depend on a few things.  Can you check a handful of your custom collection a pick out some of the movies in them and then look at the NFO files.
Do you see your custom collection in the NFO file?

If so that's good as it can be read back later to recreate the collection.  If it's not present and update to a newer version and full library scan will remove those collection. 
So we want to verify you have this info in the NFO.

Let's start with that.

Link to comment
Share on other sites

swagardson
6 hours ago, cayars said:

Did you restore the system back to 4.5.3 and not just install that version over top 4.6.4?  If you just installed 4.5.3 over top you are bound to have issues as the database was changed in 4.6.

Hey, Cayars. I did downgrade on top, which I know will cause potential incompatibility (grievances about non-semantic versioning aside), but the collections issue was my main concern. I'm looking to resolve the issue now so I can upgrade again.
 

6 hours ago, cayars said:

Previously, collections were stored in XML files and should be written to the NFO file.  In the new version of Emby the XML files are no longer used and this information is now stored in the database itself and of course written to the NFO.

The BoxSet plugin is no longer used either as collection functionality is now built into each library where you can configure it.

That's all good and understandable. Thank you for the info.

6 hours ago, cayars said:

How you proceed is going to depend on a few things.  Can you check a handful of your custom collection a pick out some of the movies in them and then look at the NFO files.
Do you see your custom collection in the NFO file?

If so that's good as it can be read back later to recreate the collection.  If it's not present and update to a newer version and full library scan will remove those collection.

So that's my key issue. The NFO files don't have any reference to my custom collections, so the server kicks them out little by little over time.

For clarification:

- What component or setting of Emby should have it writing the information to the NFO files automatically?
- Can one item (NFO file) be a part of multiple "<set>"s (AKA collections)?
- Are collections automatically made when reading from the NFO file's <set> tags?
- How important is the database in determining what is a collection item and what is not? Is the NFO file populating the database in this regard?

Thanks again for your time in advance.

Link to comment
Share on other sites

He's what I would try. Pick a movie that is in a collection that presently does not have that info in the NFO.

In Emby do a mete-refresh on just that movie.  Check the NFO.  Is the collection info there now?
If so you're probably in good shape.  Try a few more.  If what you test all get this info written to the NFOs you should be safe to do a library wide meta-refresh.

Let us know how this works.
If it didn't work, try another test on an individual movie or two.  Turn of NFO reading in the library then perform the above.
Any difference?
 

Link to comment
Share on other sites

Quote

- Can one item (NFO file) be a part of multiple "<set>"s (AKA collections)?
- Are collections automatically made when reading from the NFO file's <set> tags?

Yes to both.

Link to comment
Share on other sites

swagardson
14 hours ago, Luke said:

Yes to both.

Awesome. Thank you for confirming.

22 hours ago, cayars said:

He's what I would try. Pick a movie that is in a collection that presently does not have that info in the NFO.

In Emby do a mete-refresh on just that movie.  Check the NFO.  Is the collection info there now?
If so you're probably in good shape.  Try a few more.  If what you test all get this info written to the NFOs you should be safe to do a library wide meta-refresh.

Let us know how this works.
If it didn't work, try another test on an individual movie or two.  Turn of NFO reading in the library then perform the above.
Any difference?
 

Didn't seem to work, unfortunately.

Oddly enough I don't seem to have the option to turn off NFO reading for the library (even with advanced settings enabled), but I do remember that setting being there before. I only have the "Metadata savers" section. If this is a side-effect of the on-top downgrade, I apologize.

Link to comment
Share on other sites

Go to Library menu in admin and check the type being shown for you libs.  For example does you movie library use type Movie or Mixed Content?
This is where it will show that.

If the library is setup as the wrong type your options will be different and of course scanning will be different as well.  If this is set wrong it can be correctly but let us know.
image.png.a40500c78b71a153cd89f75f11242f00.png

Link to comment
Share on other sites

swagardson

Thanks for the quick response.

Everything should be set correctly. Here's a sample screenshot of my libraries.

For reference, everything we've been doing so far I've been relegating only to the "Movies" library, as that's the only library I have collections in.

1690166354_ScreenShot2021-11-08at09_30_26.thumb.png.7fbf4f98825a4fcf3793b1d6ab302231.png

Link to comment
Share on other sites

swagardson

Yes, there is. By "everything should be set correctly", I meant that all of my videos are categorized correctly, as they've always been. Not sure why some options are not showing up.

I'm awaiting next steps, or being told that I should run my custom script.

Link to comment
Share on other sites

swagardson

Yup. I've been in the same state since the beginning of the thread. I will upgrade again eventually, but I want to try and upgrade legitimately first.

Link to comment
Share on other sites

swagardson

I'm confused. Are we not still trying to resolve why my NFO metadata does not get updated with my collection info (AKA the <set> tags), which then starts slowly kicking my items out of my collections upon upgrade to >= 4.6?

Link to comment
Share on other sites

Try refreshing the metadata on a title and see if that updates the nfo file. Please see if that helps. Thanks.

Link to comment
Share on other sites

swagardson
22 hours ago, swagardson said:

Didn't seem to work, unfortunately.

I already did this when prompted awhile ago. I'm even more confused now. Lol.

Is @cayars not helping any more?

Link to comment
Share on other sites

He is, but the instructions I gave you are the right ones. If that's not happening, then we need to look at why, e.g. your library options, server log from an example, etc.

 

Link to comment
Share on other sites

30 minutes ago, swagardson said:

Is @cayars not helping any more?

Still here but Luke jumped in and I didn't want to distract or make what he was asking/saying confusing.

Link to comment
Share on other sites

swagardson

Here are the server logs, with the last item being the failed refresh, which could be the issue with the NFO file not populating since the transaction gets cancelled overall.

embyserver.txt

Unrelated, but there's also a lot of UPnP/SSDP socket errors that populate the majority of the log. If you know what the issue with that may be, let me know. If it's not simple, I can open another topic.

Link to comment
Share on other sites

19 hours ago, swagardson said:

Here are the server logs, with the last item being the failed refresh, which could be the issue with the NFO file not populating since the transaction gets cancelled overall.

embyserver.txt 24.84 MB · 1 download

Unrelated, but there's also a lot of UPnP/SSDP socket errors that populate the majority of the log. If you know what the issue with that may be, let me know. If it's not simple, I can open another topic.

It looks like your connection to MovieDb is timing out. Typical causes of this are firewall, security software, or VPN. Can you please look into that? Thanks.

Link to comment
Share on other sites

23 hours ago, swagardson said:

Here are the server logs, with the last item being the failed refresh, which could be the issue with the NFO file not populating since the transaction gets cancelled overall.

embyserver.txt 24.84 MB · 2 downloads

Unrelated, but there's also a lot of UPnP/SSDP socket errors that populate the majority of the log. If you know what the issue with that may be, let me know. If it's not simple, I can open another topic.

Try turning off DLNA in Emby menu.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...