Jump to content

Multiple Emby Servers with the same source library - Part 2


adminExitium

Recommended Posts

adminExitium

Hi,

Apologies again for the long post.

 

This is a follow-up to my post from the previous year: 

The discussions there helped me finalise the working of multiple Emby servers that read from the same library with one acting as the metadata writer for generating all the nfo/jpg etc. and the others (referred to as metadata readers) reading those with their respective internet metadata downloaders disabled or uninstalled altogether.

 

Now that 4.6.1 has been released with quite a few changes to how Emby stores and processes the metadata (especially with Collections & Audiobooks), I am planning to wipe the existing metadata and re-generate it with a fresh Emby install so that there are no issues due to upgrades etc.

Before going ahead with it, I have a few questions that I would like clarified:

* Are the metadata generated by 4.5.4 & 4.6.1 servers inter-compatible (barring Collections, which is stored in a different location altogether)? To clarify, can the metadata writer be in 4.6.1 and the metadata readers be in 4.5.4 or vice-versa? Any issues with client compatibilities?
* How are multiple collections stored in the media nfo file? Multiple set tags?
* I see that TV shows also have the "Minimum automatic collection size" setting (like Movies) but no option for "Automatically import collections from the enabled metadata downloaders" like Movies. Is that by design? What does changing the size affect then, if it's not automatically importing the collections?

* What all content has been moved to the metadata folder, apart from Collections? From a brief search, I see that the People folder was also moved there? Anything else?
* Following up on the above, why wasn't the movement of the People folder mentioned in the release notes? The move of Collections is also similarly lacking mention in the release notes but for that, at least there was a mention of the storage rework which helped me to track down the various changes w.r.t the media nfo files & the metadata folder

* I see that the people images are only downloaded on-demand now which becomes problematic in my scenario where the metadata folder is mounted as read-write only on the metadata writer & as read-only on the metadata readers with no streaming occurring on the writer. Is there any API call to force a download of those images?

Link to comment
Share on other sites

Hi, what I would do is shutdown the test server.

Then update your production server, do a full scan, then do metadata refresh of movies.

If you have a music library you may want to start with only this.  You could for example standup a portable Emby version and only create a music library so you can see how good your tagging is and make any changes first before updating your production server.

After you get your production server upgraded and verified working, then try to do the same with your test server.

If you have custom TV or music collections those will not upgrade and will have to be rebuilt manually.

I believe it's only the people and collections that migrated locations.

I personally would NOT do what you were doing in 4.5 with two servers pointed at the same meta-data as that's asking for problems.  Each server should have it's own meta-data location but multiple servers can point to the same media files.

Link to comment
Share on other sites

adminExitium

Ah, sorry, I didn't clarify the metadata paths properly. All the servers indeed have their own unique metadata folders defined, it's just the metadata folder of the metadata writer is also mounted as read-only on the metadata readers so that they can reference the same location for images that get downloaded and hence updated in the nfo(s).

I would prefer upgrading both simultaneously but that won't be possible easily which is why I am enquiring about the backwards or forwards compatibility both between servers (w.r.t nfo files) and clients.

Link to comment
Share on other sites

Do you mean you have the media mounted read only so one system is forced to load data you've already looked at/corrected on the main system?

BTW, why the need to two identical systems?

Link to comment
Share on other sites

adminExitium

Yep, exactly.

Media & Metadata "folder1" is mounted as R/W for the metadata writer. Those are mounted as R/O for the metadata readers but they have their own separate metadata folder as R/W, which I want to minimise the size of.

I have different streaming servers at different locations so that it's closer to the end-users but I want to avoid needing to manage any metadata changes at multiple locations which is why the central metadata writer.

Link to comment
Share on other sites

If the only thing in common between the systems is the media itself then you should be able to update them in series one after another but start with the one that does the NFO/graphic writing.

Let us know how you make out,

Link to comment
Share on other sites

adminExitium

Yep, that's the plan once I have the other queries answered too.

 

6 hours ago, adminExitium said:

* How are multiple collections stored in the media nfo file? Multiple set tags?
* I see that TV shows also have the "Minimum automatic collection size" setting (like Movies) but no option for "Automatically import collections from the enabled metadata downloaders" like Movies. Is that by design? What does changing the size affect then, if it's not automatically importing the collections?

* What all content has been moved to the metadata folder, apart from Collections? From a brief search, I see that the People folder was also moved there? Anything else?
* Following up on the above, why wasn't the movement of the People folder mentioned in the release notes? The move of Collections is also similarly lacking mention in the release notes but for that, at least there was a mention of the storage rework which helped me to track down the various changes w.r.t the media nfo files & the metadata folder

* I see that the people images are only downloaded on-demand now which becomes problematic in my scenario where the metadata folder is mounted as read-write only on the metadata writer & as read-only on the metadata readers with no streaming occurring on the writer. Is there any API call to force a download of those images?

 

Link to comment
Share on other sites

8 hours ago, adminExitium said:

* I see that TV shows also have the "Minimum automatic collection size" setting (like Movies) but no option for "Automatically import collections from the enabled metadata downloaders" like Movies. Is that by design? What does changing the size affect then, if it's not automatically importing the collections?

@Luke is that I glitch that shouldn't be there?

Link to comment
Share on other sites

Because there is no internet metadata provider that supports it for TV, so that's why it doesn't show up. The min size option is still there and will apply when importing data from your existing nfo's.

Link to comment
Share on other sites

adminExitium

So it's meant for any custom collections? That makes sense.

How are multiple collections stored in the media nfo file then? Multiple set tags with different name sub-tags?

 

Hopefully, you can also spend some time and answer these pending queries too:

* What all content has been moved to the metadata folder, apart from Collections? From a brief search, I see that the People folder was also moved there? Anything else?

* I see that the people images are only downloaded on-demand now which becomes problematic in my scenario where the metadata folder is mounted as read-write only on the metadata writer & as read-only on the metadata readers with no streaming occurring on the writer. Is there any API call to force a download of those images?

 

Thanks.

Link to comment
Share on other sites

They use the same <set tag as any other collection.

There's no single api to force download of images for all people.

Link to comment
Share on other sites

Happy2Play
1 minute ago, adminExitium said:

What all content has been moved to the metadata folder, apart from Collections?

Technically only Collections has, as other folders have always been there and were removed over time, and have returned again.  But the names/case have changed.

metadata.jpg.30dc35bba173762f96be8ffcbdbdc984.jpg

People for example started there, then new items went to /library, and now back to /people but a flat structure instead of a-z subfolders.

genres are back for custom images.

8 minutes ago, adminExitium said:

I see that the people images are only downloaded on-demand now which becomes problematic in my scenario where the metadata folder is mounted as read-write only on the metadata writer & as read-only on the metadata readers with no streaming occurring on the writer. Is there any API call to force a download of those images?

Do you  have Download in advance enabled on the library?

Link to comment
Share on other sites

1 minute ago, Happy2Play said:

 

Do you  have Download in advance enabled on the library?

It does not apply to people. The sites hosting these images have asked us in the past not to hammer them, and that's why there's no way to configure this to happen.

Link to comment
Share on other sites

adminExitium
5 minutes ago, Luke said:

They use the same <set tag as any other collection.

 

Just to confirm, each collection will get a separate set tag in the nfo file?

3 minutes ago, Happy2Play said:

Technically only Collections has, as other folders have always been there and were removed over time, and have returned again.  But the names/case have changed.

 

I get that and I accept this as a consequence of different design decisions at different points of time. My point was that I would have expected this info in the release notes rather than being buried in multiple threads piecemeal. Hopefully, this can be improved upon in future releases.

 

8 minutes ago, Luke said:

There's no single api to force download of images for all people.

I am fine with scripting a way to do it per person too. Is a GET to "Items/{Person ID}/Images/Primary" still the right API then so that Emby populates its metadata directory?

Link to comment
Share on other sites

1 minute ago, adminExitium said:

I am fine with scripting a way to do it per person too. Is a GET to "Items/{Person ID}/Images/Primary" still the right API then so that Emby populates its metadata directory?

yes you could do that

Link to comment
Share on other sites

Happy2Play
8 minutes ago, adminExitium said:

Just to confirm, each collection will get a separate set tag in the nfo file?

Yes the individual item metadata will show multiple <set> nodes/tags

  <set>
    <name>1 TEST</name>
  </set>
  <set>
    <name>1 Custom</name>
  </set>

 

Link to comment
Share on other sites

23 minutes ago, adminExitium said:

I am fine with scripting a way to do it per person too. Is a GET to "Items/{Person ID}/Images/Primary" still the right API then so that Emby populates its metadata directory?

But since we've been asked not to do this, we would prefer if you didn't. :)

Link to comment
Share on other sites

adminExitium

Excellent, thanks for the answers.

 

16 hours ago, cayars said:

But since we've been asked not to do this, we would prefer if you didn't. :)

Yep, won't be doing it in bulk for the existing people, will only be doing it for any new media additions or replacements to avoid any undue load.

Link to comment
Share on other sites

Edrock200

Don't mean to hijack the thread by any means but can you point me to any documentation on how 4.6 handles metadata differently from 4.5 please?

Link to comment
Share on other sites

adminExitium

Unfortunately, there isn't a single source of it.

 

Depending on exactly what you are looking for, other than Collections, the other changes were mostly superficial in that the folders (people) were merely moved from the data subdirectory to the metadata subdirectory and the genres folder seems to have been added for custom images.

 

As for collections, there are plenty of threads discussing those changes that you can find in the testing category on this forum and this was the release announcement for it in the beta: 

 

  • Like 1
Link to comment
Share on other sites

  • 1 month later...
adminExitium

In this scenario with 2 servers, where the second is only reading the NFO files generated by the first, is there any benefit to enabling periodic full refresh on the second? I assume any changes to the files or NFO will be picked up automatically as the file timestamps change. Is there anything that won't be picked up without a full refresh?

Link to comment
Share on other sites

rodainas
1 minute ago, adminExitium said:

In this scenario with 2 servers, where the second is only reading the NFO files generated by the first, is there any benefit to enabling periodic full refresh on the second? I assume any changes to the files or NFO will be picked up automatically as the file timestamps change. Is there anything that won't be picked up without a full refresh?

When you edit the nfos, the scan should pick-up all the changes you made to that movie/episode/video automatically, even if you change images files on the source it should reflect after the scan.

  • Like 2
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...