Emby Server 4.7.2.0 not reading folders like "john doe-tmdb-#####"

[pju 10]

I'm testing move from Emby Server 3.5.3 to latest release. I do not intend to use any backup/restore (even if possible).
The 3.5.3 server is .xml based also for movie/servies metadata.
The new server has 2 plugins added: "XML metadata" and "Migration Plugin" - thanks a lot to pünktchen!
All people metadata data are migrated into the new subfolder structure by the "Migration Plugin".
When creating a library, all fetchers are disabled since all movie/series data are in the .xml files. All my folders are of type "Mixed Content".
Also "Emby XML" metadata reader is ranked over Nfo.
And it seems works fine - at least no errors in the logfile. Maybe a problem with series.xml, see bottom).

The problem I have encountered is that folders with "-tmdb-#####" are not imported - or at least, what is read is not shown in "Cast & Crew" section.
While folders without this suffix are read and used. So the people metadata is correctly set up.
Example: Actor 'Claudia Galli' with TMDB ID 571547:
- if in folder "Claudia Galli-tmdb-571547": No data in "Cast & Crew".
- if folder is copied to "Claudia Galli: Data in "Cast & Crew".

I have done quite a bit of testing, but I cannot get a 100% grip on what causes the problem.
But, it seems that I have to enable fetchers at library creation to get "-tmdb-#####" folders imported. Which I'm not happy about, since I want to have control over my the data.
And it isn't the full explanation - because even with this setting, not all people gets data.

I really would appreciate help in finding out what is going on.

 

BTW: Another problem in activating fetchers is that I get wrong data. For instance the UK crime series "Blue Murder" is looked up as the below - even with a series.xml file. Isn't series.xml file used anymore?

 

[Luke 42078]

Quote

- if in folder "Claudia Galli-tmdb-571547": No data in "Cast & Crew".
- if folder is copied to "Claudia Galli: Data in "Cast & Crew".

Hi, the presence or lack thereof has nothing to do with whether or not the actor will show up in cast & crew.

What you want to look at instead is the metadata of Big Brother VIP, and check the actor list for that title.

[Luke 42078]

Quote

even with a series.xml file. Isn't series.xml file used anymore?

Hi, the xml format is legacy and has been for quite some time. I would suggest not using it anymore and switching to the server's default nfo format.

[pju 10]

Sorry, I think I haven't been able to explain my problem.

1. I'm doing a move from the old 3.5.3 platform to the new. And that includes usage of .nfo files and whatever comes with the move.
2. I have a lot of tailored metadata on the old platform, both movies/series and people. And of course I want to get this information along.
3. I have migrated my people metadata folders to the new subfolder standard.
4. I'm creating new libraries where I have to use the Emby XML plugin to read the legacy .xml files.
   And except for series.xml all data seems to have been imported into the new database.
   So I'm not trying to stay with series.xml - but I certainly want to get the information from the legacy .xml files.
5. My example with Claudia Gallo is not from the Blue Murders series. It is from a movie where both movie metadata and cast/crew is imported correct.
   But the corresponding people metadata does not show. Cast & Crew sections list all the people, but without pictures and description.
6. The problem with the Blue Murder series is that since series.xml is not used, data is in stead fetched from the Internet and apparently Big Brother VIP was the best match. But of course wrong.

So, I'm fighting to get in line with Emby standards but also trying to keep my metadata.

My primary question is: what prereqs must be present to make the metadata scan job pick up folders like "john doe-tmdb-#####"?

Edited June 7, 2022 by pju

[pju 10]

@LukePlease give me hand on this.
I have narrowed my problem down to adding new series.
The new series show up after a library scan as expected. With person metadata as the only exception. I only get the placeholder icon for each person.
I make people metadata show in 2 ways:
- Use the Identity lookup with the tmdb ID. This downloads a new set of .jpg/.nfo files to the already existing metadata folder. With the same content.
- Rename the person metadata folder by cutting of '-tmdb-######' and run the metadata refresh job. 
  Example: D:\Emby\metadata\people\d>ren "David Field-tmdb-57795" "David Field"
Everything seems fine when it comes to movies.
Also series initially present in Emby before running the migration plugin are fine.

I'm totally out of ideas to what I can do to make this work. So input/help would be highly appreciated.

[pünktchen 1409]

Ah, series! The reason will probably be that Emby is looking for -tvdb- but previous versions of Emby and the TVDB api did not provide that data. Set TMDB as your primary metadata downloader for tv shows also!

[pju 10]

Thanks for the input - but no. Still no people metadata.
Don't know which fetcher takes care of people metadata.
And it really puzzles me that there is a difference between pristine and subsequent library scans.

[Luke 42078]

I would expect that refreshing the metadata on a series or show will help resolve it, assuming you have internet metadata providers enabled on the tv library.

I'm guessing you might not want to do that, but upgrading from 3.5 is coming quite a long way forward. I would not expect it to be a perfect upgrade, and to be honest, it actually sounds like it's gone pretty well overall.

[pju 10]

I have tried the scan metadata folder job several times. No difference.
And yes, I have the fetchers enabled. At first I just used the defaults, but after pünktchen's advice, I enabled tmdb where possible.
I agree - it's a long jump, and it may no be perfect.
But, then again, I'm not doing an upgrade, it is a clean install and the people data are created by a plugin delivering data in .nfo format.
So, why can Emby see people with -tmdb-##### at first but not thereafter? Library scan goes well since the people names are there - but somehow the connection to the people database fails.
 

BTW: pünktchen mentioned in another topic that it in theory it could be OK to migrate people before creating libraries. Having tested, I can say it is not, no people gets metadata then. Guess it is the same issue I see.

Edited June 8, 2022 by pju

[pünktchen 1409]

3 hours ago, pju said:

I enabled tmdb where possible

But have you set it as the primary metadata downloader?! That works for me.

[Luke 42078]

Either of those two providers should be fine as the primary. I would try refreshing the metadata on a series. If you still have an issue after that, let's look at a specific example:

Thanks.

[pju 10]

9 hours ago, pünktchen said:

But have you set it as the primary metadata downloader?! That works for me.

Sorry, I was unclear. Yes, I moved TheMovieDb to the top of the list anywhere I found it.
My point was that TheMovieDb is not checked for for all fetchers in the default library template and I checked all missing. 

[pju 10]

6 hours ago, Luke said:

Either of those two providers should be fine as the primary. I would try refreshing the metadata on a series. If you still have an issue after that, let's look at a specific example:

Thanks.

I have executed Scan Metadata Folder.several time without seeing any difference. 

I have attached the Emby log file and some screen dumps, trying to describe my issue.
This is the order og my actions:

• Starting on a Emby Server with no libraries or people loaded.
• Created a library (Mixed Content) with TheMovieDb as primary and Emby Xml over Nfo.
• Executed Migration Plugin.
• People with metadata show up as expected both in movie and serie.
• Created another library (Mixed Content) with TheMovieDb as primary and Emby Xml over Nfo.
• No data is shown for series people. Movies OK.
• Renamed actor folder "Adam Godley-tmdb-23429"  to "Adam Godley".
• Executed Scan Metadata Folder.
• Actor  "Adam Godley" now show both picture and metadata.

Hope it's what you're looking for.

embyserver.zip series.xml

[Happy2Play 9780]

12 hours ago, pju said:

I have executed Scan Metadata Folder.several time without seeing any difference. 

I have attached the Emby log file and some screen dumps, trying to describe my issue.
This is the order og my actions:

• Starting on a Emby Server with no libraries or people loaded.
• Created a library (Mixed Content) with TheMovieDb as primary and Emby Xml over Nfo.
• Executed Migration Plugin.
• People with metadata show up as expected both in movie and serie.
• Created another library (Mixed Content) with TheMovieDb as primary and Emby Xml over Nfo.
• No data is shown for series people. Movies OK.
• Renamed actor folder "Adam Godley-tmdb-23429"  to "Adam Godley".
• Executed Scan Metadata Folder.
• Actor  "Adam Godley" now show both picture and metadata.

Hope it's what you're looking for.

embyserver.zip 1.61 MB · 0 downloads series.xml 5.06 kB · 0 downloads

Not that the xml metadata plugin reads it but there are no person provider ids in metadata so a by name look up will be used.

    <Person>
      <Name>Adam Godley</Name>
      <Type>Actor</Type>
      <Role>Tom Brown</Role>
    </Person>

 

Where an Emby nfo will provide all required person info to link to person-tmdb xxxxxxxxx folder.

  <actor>
    <name>Adam Godley</name>
    <role>Tom Brown</role>
    <type>Actor</type>
    <tmdbid>23429</tmdbid>
    <imdbid>nm0324134</imdbid>
    <tvdbid>324709</tvdbid>
  </actor>

[pünktchen 1409]

That is partially correct. Yes, because of the missing provider ids in the nfo on my system, Emby will first look for people folders without id. But because there isn't one, it will will then fetch the persons from the online metadata providers. If TMDB is in the first place, the person gets a correct id. Emby fetches the image, wants to store it to the people folder, sees there is already a folder with id and image inside and uses that existing image.

At least that is what is happening on my new Emby installation.

Maybe the difference lies between an existing series.xml and tvshow.nfo and the order how Emby executes things?!

Edited June 9, 2022 by pünktchen

[pünktchen 1409]

But it also fails for me with the series.xml from @pju and i think i understand what's happening.
The series.xml has IMDB and TVDB ids but no TMDB id. But because it has already a TVDB id, Emby skips the lookup at TheMovieDB and just grabs the data from TVDB.
TVDB doesn't return a proper id for the people folder and also doesn't have an image for Adam Godley for the show Fallet! That's why there isn't a person image in Emby at all.

[pju 10]

Thanks, both, for explaining. Now things start to make sense to me.

But, sounds to me that the subfolder structure is vulnerable to future changes in providers.

What happens if TMDB stop to exist - will all tmdb subfolders then be left dead in the water? Tmdb fetcher will most likely be removed/disabled in library. 

Will it help me if i rename the tvdb tag in series.xml?

[pünktchen 1409]

25 minutes ago, pju said:

But, sounds to me that the subfolder structure is vulnerable to future changes in providers.

What happens if TMDB stop to exist - will all tmdb subfolders then be left dead in the water? Tmdb fetcher will most likely be removed/disabled in library.

Yes, the current situation is not perfect but better than previous versions. If you now go on with writing nfo files in Emby, it will also write the provider ids for the people to movie, tvshow and episode nfo files. This way at least existing media and people should not be a problem for future re-installations of Emby.

[pünktchen 1409]

31 minutes ago, pju said:

Will it help me if i rename the tvdb tag in series.xml?

No. There's already an empty tmdb node in your series.xml

<TMDbId>
</TMDbId>

Just fill it with the proper tmdb id for the series. For your "Fallet" example:


 

<TMDbId>71624</TMDbId>

[pju 10]

So, if I understand you right:

Subfolders with "-tmdb-" are only used if series.xml contains a TMDB id - guess it doesn't mattter if it is valid - or?.
So I must go through my media folders and make sure my series.xml files contains a TMDB id.
And keep TMDB at top as fetcher. At least until my media metadata is moved to nfo.
Think there is a topic about how to do that.

Despite my troubles, I fully agree, the new design is definnitely an improvement.

[pünktchen 1409]

1 hour ago, pju said:

Subfolders with "-tmdb-" are only used if series.xml contains a TMDB id - guess it doesn't mattter if it is valid - or?.
So I must go through my media folders and make sure my series.xml files contains a TMDB id.

No, not needed because finally i've found the real problem with your data. Your series.xml has <LockData>True</LockData> and that prevents Emby from grabbing additional metadata from online sources. Even just for people! Do a search and replace with notepad++ for <LockData>True</LockData> to <LockData>False</LockData> for all your series.xml files but keep TMDB as active metadata downloader. The order of the metadata downloaders doesn't matter.

[pju 10]

Whooa - respect!  It would have taken quite a while before I had gone down that path!
A question, though: You talk about grabbing fron online sources - my aim is to use the migrated data already present on disk.

[pünktchen 1409]

Local data will always take precedence. Only data that is not available at all in local xml or nfo will be fetched from online sources.

[pju 10]

Okay, got it.
And tanks a lot for your great help!
Unfortunately i'm away for the most of a week - but I'll follow your advice as soon I get back. And let you know the outcome.

[Luke 42078]

On 6/10/2022 at 10:56 AM, pju said:

So, if I understand you right:

Subfolders with "-tmdb-" are only used if series.xml contains a TMDB id - guess it doesn't mattter if it is valid - or?.
So I must go through my media folders and make sure my series.xml files contains a TMDB id.
And keep TMDB at top as fetcher. At least until my media metadata is moved to nfo.
Think there is a topic about how to do that.

Despite my troubles, I fully agree, the new design is definnitely an improvement.

If I recall correctly I don't think we even read the person id's from series.xml. The xml format is legacy and hasn't been kept up to date with new features. You'd have to switch to nfo for that.