Some Observations About Metadata

[HouseOfCards 101]

Hello all,

I want to make some observations about the metadata management in Emby.  I'm not trying to nit-pick, and I appreciate all the efforts by the developers, and help they provide.  I left Plex a long time ago, and never looked back.  But I did so because they focused more on becoming a streaming service, than a personal media server.  I love Emby, and I don't want to see it make the same mistakes, since they seem to focus a lot of resources on Live TV and DVR features, rather than fixing basic things which are essential to storing and curating your own media.

Again, not trying to beat anyone up, I just want to express my opinion because others may share it and I want Emby to succeed long term.

I think metadata should be the core focus of a personal media platform.  After all, the whole point of putting all your movies, TV, home videos, audio books, family photos, etc... in one place, is to make a single location with all the information about your media.  That's content, and metadata... all in one place.

Let's take one music example I have been experimenting with, because it's a simple "one artist/album/track" example.

I have a CD Single from the group "XTC" called "The Ballad Of Peter Pumpkinhead".  The CD has a few songs, but I only ripped the main song because it's all I cared about.  So what I have is an artist ("XTC"), with one song off the album ("The Ballad Of Peter Pumpkinhead").

Now first, both TheAudioDB and MusicBrainz has the main genre for the artist as "New Wave", but Emby chose to set the genre as "Alternative", because that's embedded in the one track I have (used Picard long ago, genre may have changed since).  So I edited the artist and added all the genres the sites reported.  These edits saved to the NFO, but only three show in the GUI.  I also added a custom image to the song as you can see in the first screen shot.

Now I have two servers, and because Emby does not save or read the changes at the song level, this information is never noticed by the second server.  The custom image is not saved alongside the MP3 ("The Ballad Of Peter Pumpkinhead.jpg" for example), so the image is never picked up by the second server.  In fact, the edited genres never propagate to the other server either, even after a scan library or refresh metadata.  And vice versa.  All the genres are in the NFO, but Emby doesn't pull them in on refresh.

In fact, a refresh of the metadata on the server with the correct genres, doesn't even create the additional genres in the library.  "XTC" should show under all five genres from the screenshot above, but it's still only under the first... "New Wave".

There are similar inconsistencies and annoyances all across the different library types...

My point with all this is that Emby needs to incorporate metadata all the way, and it doesn't.  It covers the average user who drops an album into a folder, and it pulls in the artist and album data.  But even then in some cases, it doesn't save the data to the NFO correctly, although @Luke helped me a bit with this and it's better than before.  While some of it is API issues, or other things outside the developers control, most of this could be solved if we committed to embracing the idea that "all data makes it to disk".

Because aside from the complication of running two servers which most don't, this also affects someone rebuilding a server for whatever reason.  All that support cost when people lose what they had because paths are different or whatever.  The endless complication THAT creates is fixed if Emby acted like this...

The server notices an album a user added.

  ---> Is there an "album.nfo"?  If so, read it and pull in what's there.  Use providers to fill missing information.
  ---> No NFO?  Use the providers and write one.
  ---> Is there a "folder.jpg"?  If so, read it and pull in what's there, otherwise get images from providers.

  ---> Is there an "artist.nfo"?  If so, read it and pull in what's there.  Use providers to fill missing information.
  ---> No NFO?  Use the providers and write one.
  ---> Is there a "folder.jpg" and others?  If so, read them and pull in what's there, otherwise get artist images from providers.

  Now for the missing part...

  ---> Is there a "The Ballad Of Peter Pumpkinhead.nfo"?  If so, pull it in, otherwise write one.  If no API provides this, write a blank one and we can customize if we want.
  ---> Is there a "The Ballad Of Peter Pumpkinhead.jpg"?  If so, pull it in.  If not, write the album image from its parent album.

I even think that we should be able to write multiple images for things like movie covers, music album covers, etc... where it makes sense.  Just like the backgrounds for movies and television, they can change every so often.  Why limit a user to one movie poster when there are so many awesome pieces of artwork you can collect?

  ---> For movies, "poster1.jpg", "poster2.jpg", "poster3.jpg", etc...
  ---> Same for TV shows, music albums, etc...  Where it makes the most sense.

Being complete like this means that any customization we do, stays.  On multiple systems, across rebuilds, everywhere...  There's so much valuable information we could add to our media, but we can't, or don't because we're afraid we'll just lose it all one day anyway, so why bother?  Things like...

  ---> Custom artwork for movies, TV shows, music tracks, etc...
  ---> NFO's written from image names in a photo library can store descriptions of what the photo is from, location, dates, etc...
  ---> Music tracks can contain descriptions of the inspiration of a song, or other interesting facts that can end up in the GUI down the road.
  ---> Etc...  Etc...

The TLDR is that EVERY library should be allowed to follow the same guidelines if the appropriate options are selected in the settings.  If there's an NFO and images, pull them in.  If there's not, write them.  Stored permanently, right with your media.  Every customization saved, every time.  This is a fundamental function for anyone doing more than downloading stuff online and dropping it into a folder.  Some of us actually want to curate the family photos so we can do what a media server is supposed to... share it with family and friends.

My mother should be able to see the Christmas photos in the family photo album, and have a caption about it explaining things.  She should be able to stream the home movies with a caption about what the video is about.  And on and on.

Now again, I'm not beating anyone up.  I know this stuff isn't always simple.  But in this particular case, it's FUNDAMENTAL to what a media server is for...  presenting your media.  If nothing else, your customizations should stick.  Consistently.  Without fear of them being wiped by accident somehow.  I love the project, and I'd be willing to help one-on-one with any of the developers with ideas.  I'm not a programmer, so all I can do is report bugs and offer opinions from a user perspective.  That's why I'm submitting this...  These are real issues for me, and a lot of what I read from feature requests and such, are similar issues.  I'd be happy to submit more details and answer questions about it.

I'll wrap up by saying that I think the consistency of metadata should be given focus.  For me, it's the weak point.  Others may not think it's a huge issue, but for me, it does matter.  A lot.

Thank you, and I appreciate the hard work that goes into everything.

 

[Luke 42375]

Quote

  ---> Is there an "album.nfo"?  If so, read it and pull in what's there.  Use providers to fill missing information.
  ---> No NFO?  Use the providers and write one.
  ---> Is there a "folder.jpg"?  If so, read it and pull in what's there, otherwise get images from providers.

  ---> Is there an "artist.nfo"?  If so, read it and pull in what's there.  Use providers to fill missing information.
  ---> No NFO?  Use the providers and write one.
  ---> Is there a "folder.jpg" and others?  If so, read them and pull in what's there, otherwise get artist images from providers.

Hi, this is exactly how the server behaves with nfo files. Albums and artists are an exception but we've covered that elsewhere. You just have to use the provided nfo plugin build.

However, tracks can have embedded information about the album as well and this is where things can get messy because users expect that to be utilized.

[Luke 42375]

Quote

"XTC" should show under all five genres from the screenshot above, but it's still only under the first... "New Wave".

Have you explored the display options for your emby user?

[HouseOfCards 101]

18 minutes ago, Luke said:

Have you explored the display options for your emby user?

Don't know how I missed that.  Now they all show on the item, but I was more concerned that even with 5 genres for that artist, the genres aren't being created here...

I went through on one server and set custom artwork, because as I mentioned in another post, the automatically generated collages weren't happening for some, or updating on all of them.  This is much better than the missing art on some.

[HouseOfCards 101]

25 minutes ago, Luke said:

Hi, this is exactly how the server behaves with nfo files. Albums and artists are an exception but we've covered that elsewhere. You just have to use the provided nfo plugin build.

However, tracks can have embedded information about the album as well and this is where things can get messy because users expect that to be utilized.

Oh, I get it...  You gotta be careful not to break all the random setups out there.  I'm posting this with all due respect.  You guys are always helpful.  And I see you don't sleep either.  LOL

I would say that for music, the server would be starting with an album getting added.  We don't create the artists on our own.  So the server would see an album get added.  I think it just comes down to the order the refresh of metadata takes.

---> Read the individual tracks (MP3, FLAC...) and extract the artist/album information.  Write the NFO's for artist/album as you do currently (with updated plugin)...
---> Write the "The Ballad Of Peter Pumpkinhead.nfo" using what Emby already does a great job of extracting and compiling.  

But as I was discussing, in this case a "Soundtrack" album has caused all the songs on this album to be "Soundtrack" for a genre.  No problem.  Nothing would change from how you do it now INITIALLY, because the API's probably aren't pulling that detail anyhow.  The tracks first get written with "Soundtrack" as a genre.  That's fine.  What I'm saying is that any information I add from the GUI for the track (genres, descriptions, etc...), should be respected by Emby.  This album has songs which are "Rock & Roll", "Doo-Wop", "Instrumental", etc...  If I go into the track metadata in the GUI and add those genres, the respective songs should appear in the genre categories.

As it is now, if I shuffled, say, the Disco genre... no Disco songs from a soundtrack album would be included.  The most detail is always going to come from the most granular item, which is a track for the case of a music library.  In the above album, playing "Beauty School Dropout" should display "Various Artists" as the performer.  It should show "Frankie Avalon".  I hope I'm making sense...

I did just confirm that if I add genres at the track level in the GUI, the genre gets created/respected in the genre category screen.  The issue is that the information is not written to "track_name.nfo" to be preserved.  A server failure, and I'm at the mercy of restoring somehow to get all those customizations back.  That's why I'm proposing extending NFO and JPG support at the track level.  Custom metadata and images needs to be saved to the disk, not just in an enormous database on the server.

Also, the second server still doesn't pull in the genres on the item page from the existing NFO.  On the second server, those 5 genres don't display on the artist screen, even though the "artist.nfo" file has them all.  In that case, a metadata refresh doesn't pull in what's there.  It's all inconsistent, which is the reason I'm starting a discussion about it.  I'm calling it a night.

Have a good night, and thanks.

[HouseOfCards 101]

So for music at least, I can confirm that the track level appears to be the key.  I know that we could always go in and manually edit the files directly using tagging software, but that is a huge undertaking when we already have a GUI for editing in Emby, and we have thousands and thousands of tracks.

I added "Doo-Wop" and "Rock & Roll" as genres to one song ("Blue Moon") using the GUI to the "Grease" album, and the genres did populate upwards for the album, and write to the "album.nfo" file.

That also created the corresponding genres on the genre tab.

So for music at least, a good start would be to have the metadata scanners do this...

---> No "Song Name.nfo", write the NFO using what's in the tracks tags, as already expected.
---> Existing "Song Name.nfo"?  Import that information since it was created previously, and may have been customized by the user.

Then treat images the same way...

---> No "Song Name.jpg"?  Write the embedded image if present in the file, otherwise write the image from the parent album as "Song Name.jpg".
---> Existing "Song Name.jpg"?  Import that image, since it may have been customized.

This scenario would make it so that the tags and images were written to disk in an NFO or image file, and anything that gets customized later by a user would be permanently stored.  It would fix issues when a server needs to be rebuilt, and allow for the storing of images and metadata outside the internal server database making it easier for being backed up and restored.

I already mentioned this here...
 

This seems like a rational place to start for music.

Have a great weekend.

[MediaIntelNUC 37]

@HouseOfCardsI dont use my Emby for music at all, so I have nothing to add to this topic but OT i have to say that your posts is an awesome example of a user pointing out what could be better AND detailing how that improvment could look like.

A great example of respectful collaboration between a user and devs with a common goal of getting a better experience for all of us that use Emby.

Its easy to point out what you dont like and should be better, its something else to work with the ones who actually can make that change, an example for all of us.

My 2 cents..

N.

Edited 1 hour ago by MediaIntelNUC
Spelling