Emby's mechanic's of adding new episode

[CharleyVarrick 283]

Sorry if this question or clarification request might have been asked already.

 

I am trying to wrap my head around the inner working, or what happens in what order, when we add a recently (yesterday evening) aired tv episode in Emby.

For the record, I uses a tv rename utility to adhere to file naming convention, and also create normal folder name / structure. I have RTM enabled.

 

Case#1 Serie opener (aka s01e01 pilot)

Once s01e01 is added to show subfolder, within a minute or so, Emby creates nfo and some jpg for show/season/episode

At this same point in time, the new show/episode appears in Embyweb Home/Latest TV Show

Emby's  RTM feature (on its own) has scanned and detected new content.

In addition to this, Emby also, and again on its own, grabbed all available metadata from the provider to present the episode in its proper season and show context.

 

At that point in time (s01e01 aired last night), we cannot assume that all 6,12 or 24 future episodes are already accounted for on provider's side

They might only know about e01, or e01-03) and they will update further on weekly basis the next episodes).

They might also give it a temporary episode title (ie: episode 2) which they later update to actual/official title. Same goes for synopsis and a bunch of other info.

They might already know right off the bat there will be x number of episodes, but with the titles "TBA"

On the other hand, they might add next episodes one at a time during the whole season.

Emby will grab whatever and all it can find from provider when s01e01 is added to it.

 

The general idea of all of the above being:

The morning after a show premiere's, the data is likely quite incomplete on the provider's side.

Assuming you agree with the statement, my question (finally) is;

 

What will Emby do differently (or won't do) in a week when I feed it s01e02 (and 2 weeks later when I add s01e03, and so on)

To be clearer, I suspect the RTM scanning part of it will be pretty much the same.

But what about the part about new and/or updated metadata made necessary by adding s01e02 ?

The week before, it automatically grabbed metadata for it in advance, and a week later, I now give Emby the media in question.

I believe (hope) it needs to perform some kind of metadata refresh so it does not keep calling se01e02 TBA (or keep not showing synopsis, as there were none available the week before)

Related question: if at the s01e01 point in time, the provider didnt knew airdates for the future episodes, and either add them one at a time or in small bunch, how is Emby suppose to deal with that?

 

Sorry if this is on the long side, but at the present, I have no clue how Emby is dealing with in progress show.

Thanks in advance for explaining

 

 

 

 

Edited March 13, 2021 by CharleyVarrick

[rbjtech 5284]

The fault imo actually lies with the MetaData providers.  If they need to name an episode 'TBA' then the record should not be available to the API.

Emby will just see this as the episode title - and thus it will not automatically update it unless you have turned on periodic metadata replacement - it will not even correct it using 'missing metadata' as it's not missing.

The way I deal with it, is use an import tool which delays the import until it has a name ... 

 

 

Edited March 12, 2021 by rbjtech

[CharleyVarrick 283]

2 hours ago, rbjtech said:

The fault imo actually lies with the MetaData providers.  If they need to name an episode 'TBC' then the record should not be available to the API.

Emby will just see this as the episode title - and thus it will not automatically update it unless you have turned on periodic metadata replacement - it will not even correct it using 'missing metadata' as it's not missing.

The way I deal with it, is use an import tool which delays the import until it has a name ... 

 

 

TBA means To Be Announced, and I fully get this might happen when a new show is just begining.

The way I deal with this very specific scenario is manually editing metadata with the provider website title, as opposed to api un-updated "TBA"

 

My concern is much wider as in how can Emby keep up with those in-progress show in  which providers update their info "one epidode at a time".

How can this work out alright, apart from manually refreshing such in-progress show every week?

My question again is what does Emby stop doing once it holds a tv show pilot for instance?

Edited March 12, 2021 by CharleyVarrick

[rbjtech 5284]

"TBA means To Be Announced, and I fully get this might happen when a new show is just begining."

Why ?  If the metadata provider doesn't know the name of the episode, then why have it there in the first place ?  It is incomplete info.

If we followed this logic - every show would have a TBA for all the remaining episodes in each season ! - and if a new season was confirmed, then another set of TBA's for that yet unaired season as well...

So as I said, WAIT for the metadata, then import the show.  If you want to manually grab it from somewhere else, then fine, but for me, automation is key - I don't want to babysit my emby metadata unless I have to.

 

, 

 

[CharleyVarrick 283]

TBA as opposed to actual title is 1% of my question.Say providers has the proper title, but not the synopsis (say it gets updated a day or two after).

Or not the special guest (one time) actors.I prefer they have some info than offer none at all.

 

All I'm saying is I fully get that TVDB (or others) might not have 24 FULLY updated episode metadata the morning after the pilot aired.

My question (again) is how Emby is expected to grab info when said info is updated one episode at a time for weeks on end.

Edited March 12, 2021 by CharleyVarrick

[Happy2Play 9780]

To the overall question, I would say no as all metadata is a one and done to my knowledge.

I believe that is why Luke add the "After the initial import, automatically refresh metadata from the internet:" library option.  Otherwise Manual refreshes would need to be done.

Edited March 12, 2021 by Happy2Play

[CharleyVarrick 283]

By the way, same deal happens with movies.

When a movie comes out in bluray, tmdb volunteer contributors need to create a new entry from scratch and add info one bit at a time, it can require weeks, months or even years before they achieve a 100% score. So if Emby grabs partial info (say 25%) that is available 3 days after bluray hits the shelves, you will be stuck with 25% partial info displayed in Emby, unless you regularly update (refresh MD).

But a movie release happens only once, the much BIGGER challenge is weekly tv episodes. How can Emby keep up with those is beyond me.

[Happy2Play 9780]

Overall a one in done cover the majority of everything.  Of course things are updated but there is no need for anyone to spam getting new info unless there are major changes and that is still user perspective.

In theory since Emby gets a entire series metadata from TVDB with every update it would be a matter of reprocessing the entire series.  But that is a lot of wasted work the majority of the time.  But TVDB is the only provider Emby gets the entire series info in one zip file.  Don't know if that will change in TVDB v4 api when the forced user subscription change happens.

But once a item is imported/written there is nothing to keep up with.  It is now on you unless you enable automatic refreshes every 30/60/90.

Edited March 12, 2021 by Happy2Play
provider edit

[CharleyVarrick 283]

19 minutes ago, Happy2Play said:

To the overall question, I would say no as all metadata is a one and done to my knowledge.

I believe that is why Luke add the "After the initial import, automatically refresh metadata from the internet:" library option.  Otherwise Manual refreshes would need to be done.

I absolutely agree with you.

The thing I have doubt about is refreshing a whole, significant library tv collection every 30/60/90 days just to update s01e02 of an in-progress specific show

This is probably considered "hammering the providers", And if the providers "hangs up on Emby"  after 2 minutes, some shows will be up to date, but most will not. Those are done in alphabetical order if I'm not mistaken, so your A-F show will be perfect, but the G-Z will never get past provider connection time limit.

 

The library option feature "After the initial import, automatically refresh metadata from the internet" would need to put emphasis on current in progress show more than on show that might have ended years ago.

The actual process might be improved to update more recently added shows first, instead of starting alphabetically and getting timed out after same shows have been updated 300 times, while other shows never do.

Edited March 12, 2021 by CharleyVarrick

[Happy2Play 9780]

So is there truly a difference?  End series metadata does get updated, I have made dozens of update to Wrong data or person.  But true it isn't as often as the Current Season of a Continuing series,  Previous seasons are the same as Ends series.

Series keep going so this is still a endless update loop as that last weeks item could be updated next week.  Maybe get last 2/3 episode metadata when importing new item.  But to me it is after the fact as I would have probably already watched the item that did not have complete metadata at the time.

Another issues is there is usually a 24 hour delay to get any metadata added to provider sites.  So if Todays items do not have metadata you are at a lose for a minimum of 24 hours more.

[CharleyVarrick 283]

The  point i'm trying to make is:

updating Seinfield (1989-1988) automatically every 90 days is pointless. 

Updating Debris (2021) every 30 days is likely insufficient, as it premiered 10 days ago.

 

Don't go looking in your emby library to check the specifics, the whole idea is Emby needs to leave "vintage" shows alone as it probably got it all right first time around.

But it needs to improve at staying on top of the latest things.

This will never happen if it always perform update alphabetically.

[Happy2Play 9780]

6 minutes ago, CharleyVarrick said:

This will never happen if it always perform update alphabetically.

That is not the case and never has been.  The LastDateRefreshed is used.  So if a provider hung up on you well the date does not get changed and your server makes the call again.  Since this date can be different not every item gets updated at the same time.  But sure you can make a feature request to limit what Emby looks at on a automated refresh.

Edited March 12, 2021 by Happy2Play

[CharleyVarrick 283]

17 minutes ago, Happy2Play said:

That is not the case and never has been.  The LastDateRefreshed is used.  So if a provider hung up on you well the date does not get changed and your server makes the call again.  Since this date can be different not every item gets updated at the same time.  But sure you can make a feature request to limit what Emby looks at on a automated refresh.

Now were getting somehwere !

I recently asked here if Emby was keeping tabs on Last "successful" Date Refreshed and I believe EBR answered this was not in Emby's job description, or something to that effect.

That answer kind of took the Emby wind out of my sail.

When does Emby "makes the call again" ? At the next 90 days automatic refresh ?

 

I really suck at reading logs but I'm pretty sure I saw a repeated pattern of update being alphabetically performed, maybe not the same exact one over and over though.

 

Edited March 12, 2021 by CharleyVarrick

[Luke 42077]

There is a tvdb updates api that we use to get these kinds of updates regardless of your automatic interval option, however last I checked they had disabled it while they were doing development on their v4 api. When their v4 api is finalized we'll implement that again to whatever the new way of doing it is.

In the meantime the automatic refresh option can help with this.

[CharleyVarrick 283]

37 minutes ago, Happy2Play said:

The LastDateRefreshed is used.

BTW where is this visible ?

It has to be found somewhere so Emby knows which media have been updated and which didn't

[Luke 42077]

That's an internal value that's not displayed anywhere. You'd have to look directly in the database if you really wanted to.

[CharleyVarrick 283]

12 minutes ago, Luke said:

That's an internal value that's not displayed anywhere. You'd have to look directly in the database if you really wanted to.

Not sure I want to, if I have issues just looking at logs, you can imagine the rest.

I call this "not meant for human eyes" kind of data. Ha ha

 

Hopefully you agree about my point: 30 years old shows vs pilot that aired last week ago can't be treated equal when comes auto update time.

Later is more "urgent" than the former. And Emby need to act accordingly.

Thanks

 

Edited March 12, 2021 by CharleyVarrick

[CharleyVarrick 283]

Please explain

I added Back Roads s07e06 Agnes Water and 1770, QLD

TVDB website shows it as Back Roads s07e06 Agnes Water and 1770, QLD

https://thetvdb.com/series/back-roads/episodes/8219089

Emby shows it as  Back Roads s07e06 TBA

I have scanned library files of the show, refresh metdata of the show, of the season, and of the episode at least 6 times, yet I still see

tba as title, no synopsis

Edited March 13, 2021 by CharleyVarrick

[GrimReaper 4739]

10 minutes ago, CharleyVarrick said:

Please explain

I added Back Roads s07e06 Agnes Water and 1770, QLD

TVDB website shows it as Back Roads s07e06 Agnes Water and 1770, QLD

https://thetvdb.com/series/back-roads/episodes/8219089

Emby shows it as  Back Roads s07e06 TBA

I have scanned library files of the show, refresh metdata of the show, of the season, and of the episode at least 6 times, yet I still see

tba as title, no synopsis

Did you do Search for missing or Replace all metadata? Since both episode title and synopsis have TBA in respective fields, former option won't pupulate it, only latter one will. Or should I say "should“.

[Happy2Play 9780]

5 minutes ago, CharleyVarrick said:

Please explain

I added Back Roads s07e06 Agnes Water and 1770, QLD

TVDB website shows it as Back Roads s07e06 Agnes Water and 1770, QLD

https://thetvdb.com/series/back-roads/episodes/8219089

Emby shows it as  Back Roads s07e06 TBA

I have scanned library files of the show, refresh metdata of the show, of the season, and of the episode at least 6 times, yet I still see

tba as title, no synopsis

Not knowing exactly when someone updated TVDB, it take at least 24 hours for cache information to update from website entry to be available in their api data.   Same with images.

Couldn't find TVDB link

https://forums.sonarr.tv/t/how-long-do-tvdb-updates-take-to-come-through/24854/2

[Happy2Play 9780]

Don't have v4 api to see a difference, but v3 currently returns this for that episode.

Spoiler

{
  "data": {
    "id": 8219089,
    "airedSeason": 7,
    "airedSeasonID": 1899263,
    "airedEpisodeNumber": 6,
    "episodeName": "TBA",
    "firstAired": "2021-03-11",
    "guestStars": [],
    "directors": [],
    "writers": [],
    "overview": "TBA",
    "language": {
      "episodeName": "en",
      "overview": "en"
    },
    "productionCode": "",
    "showUrl": "",
    "lastUpdated": 1613727960,
    "dvdDiscid": "",
    "dvdSeason": null,
    "dvdEpisodeNumber": null,
    "dvdChapter": null,
    "absoluteNumber": null,
    "filename": "",
    "seriesId": 303849,
    "lastUpdatedBy": 16923,
    "airsAfterSeason": null,
    "airsBeforeSeason": null,
    "airsBeforeEpisode": null,
    "imdbId": "",
    "contentRating": null,
    "thumbAuthor": null,
    "thumbAdded": "",
    "thumbWidth": null,
    "thumbHeight": null,
    "siteRating": 0,
    "siteRatingCount": 0,
    "isMovie": 0
  }
}

 

Edited March 13, 2021 by Happy2Play

[CharleyVarrick 283]

@GrimReaper76tried search for missing and replace all, no joy

@Happy2Play This is my tv rename app, I hit refresh and it updates yesterday show

I am pretty sure it also use an api to get the data

Isn't Emby supposed to "take the fuss out of managing media"

Right now, its more fussy and not doing so great at managing. 

You might ask why I make a big deal of this, but take into account issue like this basically happening every other day

You are great, but I have a bad user experience with in progress shows day after day

Edited March 13, 2021 by CharleyVarrick

[Happy2Play 9780]

3 minutes ago, CharleyVarrick said:

@GrimReaper76tried search for missing and replace all, no joy

@Happy2Play This is my tv rename app, I hit refresh and it updates yesterday show

I am pretty sure it also use an api to get the data

Isn't Emby supposed to "take the fuss out of managing media"

Right now, its doing the opposite

You might ask why I make a big deal of this, but take into account issue like this basically happening every other day

My patience has limits.

I already posted what TVDB give us via the v3 api, there is nothing Emby can do.  I don't know what method Rename TV uses though.

[GrimReaper 4739]

1 minute ago, Happy2Play said:

I already posted what TVDB give us via the v3 api, there is nothing Emby can do.  I don't know what method Rename TV uses though.

Well, that IS kinda strange, looking at its homepage:

https://www.tweaking4all.com/home-theatre/rename-my-tv-series-v2/

It seems a bit dated, and only TVDB API v2 is mentioned. 

@CharleyVarrick you could, for test purposes, put TMDB on top of your scraper list in Library settings and then Replace all metadata on that episode, see what'll come out of it. 

[CharleyVarrick 283]

2 minutes ago, Happy2Play said:

I already posted what TVDB give us via the v3 api, there is nothing Emby can do.  I don't know what method Rename TV uses though.

All I can think of is it uses a better method.

Now I need to make a note to follow up

 

Sorry, I am being a pest but I pay $$ for Emby and I run into issues like this all the time

Instead of it working for me, its the other way around

 

Over and out