Syncing Meta Data --- a Development Discussion about how to better sync data

[Lighthammer 80]

Hey All,

I wanted to start a sort of "pre-development" discussion about Meta Data and thinking about better ways to work with it.

I make *A LOT* of effort to make sure my series have clean entries, filled series/episode descriptions, proper date airings, a relevant list of cast, etc.

Whenever I look back at the work I've put into this hobby/side-project I always sit back and wonder to myself if there's not a much better way for the community to sync the good work we've done.

One of the most obvious answers to fixing these sorts of things is to go back to the sources where Emby scrapped the data from and try to fix it. However, one of the biggest problems we've seen from this practice is some people will get very protective over entries they manage or some services require a near act of God to get proper access to it to update the information.

I've been thinking for a very long time that Emby needs a way to share cleaned metadata amongst its server admins and have some powerful options to allow the community to decide what is the most popular way to save the information and by default use the most popular metadata with, of course, the option to go your own direction per entry as you feel necessary. (IE There are a gazillion ways American Dad's episode order is to be or how weather to sort Power Rangers into one series or several different series).

 

There's gotta be a better solution for how this is handled and I very much would like to consider a direction that helps figure it out.

[Luke 37281]

Hi there, what do you suggest?

[Lighthammer 80]

There's big solutions and small solutions to this.

Honestly, I think the MOST ideal method would be to build a backend system featured as a plugin for Emby that Emby Server can access. It would sync data from each server out in the wild back to "home" (to the Emby repository). Users of Emby Server would then be able to like,dislike,rate entries with very low rated items being slowly delisted. 

Personally, I think this should be something that would remain accessible only via Emby Server; so I'm not talking about another yet another IMDB public database; the biggest reason I'm suggesting to not take this public outside of Emby are three fold:

 

#1.) Reduce bandwidth requirements of such a project so that it can be easily financially managed through something simple such as a 1 time payment for the plugin.
#2.) Reduce the amount of "fighting" that often occurs on sites like IMDB or TVDB for who has the "right" information.

#3.) Reduce the information "scraping"/"download" requests down to a minimum, again, to help manage the financial needs of such a server.

 

I think, ultimately, their a plethora of other projects that could potentially make use of such a repository, including, at some point, going live with an IMDB alternative if it's viable.

 

The problem I find with scrapping data from sources such as IMDB is they are often incomplete and difficult to get updated.

[Luke 37281]

Ok, so now the next question is who pays for it

[Lighthammer 80]

Ok, so now the next question is who pays for it

 

Hey Luke,

 

I CLEARLY answered that.

 

I even went out of my way to explain those points.

 

Mind rebutting any thoughts you have point for point?

Edited January 20, 2020 by Lighthammer

[Deathsquirrel 741]

Hey Luke,

 

I CLEARLY answered that.

 

I even went out of my way to explain those points.

 

Mind rebutting any thoughts you have point for point?

 

No, you really didn't.  A one time payment for a plugin isn't remotely going to pay for an ongoing service.

 

I don't see the value, regardless of who pays for it.  Those people that picky about this content probably won't like your take on things any more than you'll like theirs.  You are, by definition, already not happy with the content produced by the people that go to all the effort to sign up on the metadata provider sites and contribute there.  Why would you like my edits to their work?

 

Edited January 20, 2020 by Deathsquirrel

[Lighthammer 80]

I made a huge point to discuss many points of design to mitigate cost so that a one time payment for an addon can make sense if the addon is attractive enough to users to help metadata; none of which has been addressed. 

Additionally, I also discussed a system by which the community can easily determine which kinds of updates they want. The system isnt any different then the system we have in place to picking posters, banners, etc. Note that I haven't edited my post. I OBVIOUSLY put some thought into both the financial parts and factors that help you navigate which metadata sources you prefer. 

 

Another possible way to manage the metadata sources could be to offer fragmented distribution by letting emby server admins host portions of the data and share it across a torrent like network.

[Deathsquirrel 741]

Well if you feel your idea is well-developed, throw it on the enhancement request forum.  See how much interest there is from other users.  It makes no sense to me but neither do lots of other enhancements, many of which have been implemented, so that's no impediment.

[ebr 14960]

I'm trying to understand how this would be different from the existing, community-driven metadata providers already out there.  Couldn't this same effort be applied to correcting whatever issues there are in those providers?

 

One hurdle with anything like this existing in Emby is that it would require us to break one of our primary privacy polices that we currently have - that is that we do not collect or know anything about the media you have or watch on your Emby system.  I know this would be an "opt-in" thing but still it would mean we could no longer make that statement and people don't always fully understand what they opt in to.