Jump to content


Photo

How to backup people images with beta going onwards?


  • Please log in to reply
163 replies to this topic

#41 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 131163 posts
  • Local time: 02:33 PM

Posted 18 January 2019 - 02:53 PM

I will try to look at this soon.



#42 d00zah OFFLINE  

d00zah

    Advanced Member

  • Members
  • 572 posts
  • Local time: 02:33 PM

Posted 18 January 2019 - 03:26 PM

@Luke, just for clarification... would this also apply to Artists/Album Artists in the music library?

 

I'm guessing yes, as I had both People / Artists in my metadata directory when I built a fresh v4.x database & it failed to import either.

 

Just finished manually adding images to ~50% of my ~1400 Album Artists. Mostly obscure, so no surprise they didn't scape.

 

TIA



#43 pünktchen OFFLINE  

pünktchen

    Advanced Member

  • Members
  • 2074 posts
  • Local time: 08:33 PM

Posted 22 January 2019 - 04:49 PM

@Happy2Play reading/writing person.xml and backup task for changed items only finished.

I've released the new version in Emby's plugin catalogue.



#44 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 131163 posts
  • Local time: 02:33 PM

Posted 22 January 2019 - 04:51 PM

Why not just add this to the xml metadata plugin?



#45 pünktchen OFFLINE  

pünktchen

    Advanced Member

  • Members
  • 2074 posts
  • Local time: 08:33 PM

Posted 22 January 2019 - 05:04 PM

Why not just add this to the xml metadata plugin?

The xml plugin is for metadata only, not for images?! Maybe later. Right now this is just for migration to your new library structure.

#46 pju OFFLINE  

pju

    Advanced Member

  • Members
  • 115 posts
  • Local time: 08:33 PM

Posted 23 January 2019 - 12:19 PM

@Happy2Play reading/writing person.xml and backup task for changed items only finished.

I've released the new version in Emby's plugin catalogue.

 

I could need a little help - seems like I've got something wrong.

I'm trying to import my old People data (~30000 actors) into my fresh Emby server.

And also copied a few videos in 3 libraries to the test server.

 

I have saved the path "D:\emby-test\metadata\People" and checked Enabled Logging.

Run Full Restore gives me:

2019-01-23 17:02:48.152 Info HttpServer: HTTP GET http://localhost:809.../RunFullRestore. UserAgent: Mozilla/5.0 (Windows NT 6.1; rv:64.0) Gecko/20100101 Firefox/64.0
2019-01-23 17:02:49.338 Info Plugin: [People Backup] Person: Laura Harris; Source: d:\emby-test\metadata\People\L\Laura Harris; Destination: C:\Users\mc\AppData\Roaming\Emby-Server\programdata\metadata\library\f7\f7d81fd9170340429dea0c5dfc1d06d9
2019-01-23 17:02:49.432 Info Plugin: [People Backup] Person: Rossif Sutherland; Source: d:\emby-test\metadata\People\R\Rossif Sutherland; Destination: C:\Users\mc\AppData\Roaming\Emby-Server\programdata\metadata\library\63\63763dceb300450d8860f12aa62a4be6
2019-01-23 17:02:49.525 Info Plugin: [People Backup] Person: Nahanni Johnstone; Source: d:\emby-test\metadata\People\N\Nahanni Johnstone; Destination: C:\Users\mc\AppData\Roaming\Emby-Server\programdata\metadata\library\18\181466f84e984b3bb7fe18947b7523e1
2019-01-23 17:02:49.588 Info Plugin: [People Backup] Restore of People successfully finished

Not quite the outcome I expected - and have no idea why these 3 are selected.

A backup gives a more expected result:

2019-01-23 17:13:27.280 Info HttpServer: HTTP GET http://localhost:809...p/RunFullBackup. UserAgent: Mozilla/5.0 (Windows NT 6.1; rv:64.0) Gecko/20100101 Firefox/64.0
2019-01-23 17:13:27.295 Info Plugin: [People Backup] Person: Laura Harris; Source: C:\Users\mc\AppData\Roaming\Emby-Server\programdata\metadata\library\f7\f7d81fd9170340429dea0c5dfc1d06d9; Destination: d:\emby-test\metadata\People\L\Laura Harris
2019-01-23 17:13:27.342 Info Plugin: [People Backup] Person: Rossif Sutherland; Source: C:\Users\mc\AppData\Roaming\Emby-Server\programdata\metadata\library\63\63763dceb300450d8860f12aa62a4be6; Destination: d:\emby-test\metadata\People\R\Rossif Sutherland
2019-01-23 17:13:27.373 Info Plugin: [People Backup] Person: Nahanni Johnstone; Source: C:\Users\mc\AppData\Roaming\Emby-Server\programdata\metadata\library\18\181466f84e984b3bb7fe18947b7523e1; Destination: d:\emby-test\metadata\People\N\Nahanni Johnstone
2019-01-23 17:13:27.420 Info Plugin: [People Backup] Backup of People successfully finished
2019-01-23 17:13:27.420 Info HttpServer: HTTP Response 200 to ::1. Time: 140ms. http://localhost:809...p/RunFullBackup

 



#47 pünktchen OFFLINE  

pünktchen

    Advanced Member

  • Members
  • 2074 posts
  • Local time: 08:33 PM

Posted 23 January 2019 - 12:50 PM

The plugin will only restore persons from your people folder if those persons already exist in Emby's database. It will not create new persons in the database.

So the question is: Has your initial library scan already finished?



#48 pju OFFLINE  

pju

    Advanced Member

  • Members
  • 115 posts
  • Local time: 08:33 PM

Posted 23 January 2019 - 01:08 PM

Yes, and for some reason exact these 3 actors have a picture and bio.
But most have only a dummy icon and the name.
Don't know what state they are in in respect to the database when it looks like that.
I have no fetchers enabled since all scraping is done by Metabrowser and "manual".
Metadata is read by Emby Xml plugin.



#49 pünktchen OFFLINE  

pünktchen

    Advanced Member

  • Members
  • 2074 posts
  • Local time: 08:33 PM

Posted 23 January 2019 - 02:49 PM

Yes, and for some reason exact these 3 actors have a picture and bio.
But most have only a dummy icon and the name.
Don't know what state they are in in respect to the database when it looks like that.
I have no fetchers enabled since all scraping is done by Metabrowser and "manual".
Metadata is read by Emby Xml plugin.

I will push an update with some extra logging right after querying the database.
Can you please send me one or two of your movie.xml where no people restore occurs?

#50 pju OFFLINE  

pju

    Advanced Member

  • Members
  • 115 posts
  • Local time: 08:33 PM

Posted 23 January 2019 - 03:08 PM

Sure. Just found out that one of my movies have an .nfo file - guess it's from a period where I had not deselected .nfo.

For some reason I'm not allowed to attach .xml files - just says "This upload failed" ???

So here in clear text, movie.xml corresponding to the attached .nfo file:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Title>
  <LocalTitle>The Big Easy - Kærlighed i kugleregn</LocalTitle>
  <OriginalTitle>The Big Easy</OriginalTitle>
  <SortTitle>Big Easy</SortTitle>
  <Set>
  </Set>
  <Added>20-10-2013 11:07:27</Added>
  <ProductionYear>1986</ProductionYear>
  <RunningTime>96</RunningTime>
  <IMDBrating>6.5</IMDBrating>
  <CriticRating>
  </CriticRating>
  <CriticRatingSummary>
  </CriticRatingSummary>
  <MPAARating>R</MPAARating>
  <MPAADescription>
  </MPAADescription>
  <CustomRating>
  </CustomRating>
  <Plot><![CDATA[Set in New Orleans. Remy McSwain, lieutenant in Homicide finds that he has two problems, the first of a series of gang killings and Ann Osborne, a beautiful attorney from the D.A.'s police ...]]></Plot>
  <Description><![CDATA[Det ellers så festlige New Orleans danner pludselig rammen om et makabert fund.
I et af byens springvand flyder en lokal gangster rundt.
Og han er ikke alene druknet  - han er brutalt myrdet.
Politibetjent McSwain(Dennis Quiad) og hans partner stempler hurtigt episoden som et gement mafiamord, men den smukke kvindelige stats-advokat Anne Osborne (Ellen Barkin) finder ikke makker-parrets  konklusion overbevisende.
Osborne bliver hurtigt involveret i efterforskningen og endnu hurtigere i den tiltrækkende McSwain.
Og når lidenskaben brænder, kan det til tider være svært at se klart for bare røg.]]></Description>
  <Type>WTV</Type>
  <AspectRatio>1.85:1</AspectRatio>
  <Watched>False</Watched>
  <Comment>
  </Comment>
  <LockData>True</LockData>
  <CollectionNumber>
  </CollectionNumber>
  <Top250>
  </Top250>
  <Website>
  </Website>
  <Trailer>
  </Trailer>
  <Budget>
  </Budget>
  <Revenue>
  </Revenue>
  <AllowRenaming>True</AllowRenaming>
  <IMDB>tt0092654</IMDB>
  <TMDbId>
  </TMDbId>
  <MyMoviesId>
  </MyMoviesId>
  <NetflixId>
  </NetflixId>
  <MovieMeterId>
  </MovieMeterId>
  <YahooIndiaId>
  </YahooIndiaId>
  <AdultDVDEmpireId>
  </AdultDVDEmpireId>
  <AlloCineId>
  </AlloCineId>
  <FilmAffinityId>
  </FilmAffinityId>
  <AmazonId>
  </AmazonId>
  <RottenTomatoesId>
  </RottenTomatoesId>
  <CineFactsId>
  </CineFactsId>
  <OFDbId>
  </OFDbId>
  <CDUniverseId>
  </CDUniverseId>
  <CSFDId>
  </CSFDId>
  <MoviePlayerId>
  </MoviePlayerId>
  <Persons>
    <Person>
      <Name>Dennis Quaid</Name>
      <Type>Actor</Type>
      <Role>Det. Remy McSwain</Role>
    </Person>
    <Person>
      <Name>Ellen Barkin</Name>
      <Type>Actor</Type>
      <Role>Anne Osborne</Role>
    </Person>
    <Person>
      <Name>Ned Beatty</Name>
      <Type>Actor</Type>
      <Role>Jack Kellom</Role>
    </Person>
    <Person>
      <Name>John Goodman</Name>
      <Type>Actor</Type>
      <Role>Det. Andre DeSoto</Role>
    </Person>
    <Person>
      <Name>Lisa Jane Persky</Name>
      <Type>Actor</Type>
      <Role>McCabe</Role>
    </Person>
    <Person>
      <Name>Ebbe Roe Smith</Name>
      <Type>Actor</Type>
      <Role>Ed Dodge</Role>
    </Person>
    <Person>
      <Name>Tom O'Brien</Name>
      <Type>Actor</Type>
      <Role>Bobby McSwain</Role>
    </Person>
    <Person>
      <Name>Charles Ludlam</Name>
      <Type>Actor</Type>
      <Role>Lamar Parmentel</Role>
    </Person>
    <Person>
      <Name>Grace Zabriskie</Name>
      <Type>Actor</Type>
      <Role>Mama</Role>
    </Person>
    <Person>
      <Name>Marc Lawrence</Name>
      <Type>Actor</Type>
      <Role>Vinnie 'The Cannon' DiMotti</Role>
    </Person>
    <Person>
      <Name>Solomon Burke</Name>
      <Type>Actor</Type>
      <Role>Daddy Mention</Role>
    </Person>
    <Person>
      <Name>Gailard Sartain</Name>
      <Type>Actor</Type>
      <Role>Chef Paul</Role>
    </Person>
    <Person>
      <Name>Jim McBride</Name>
      <Type>Director</Type>
      <Role>
      </Role>
    </Person>
    <Person>
      <Name>Daniel Petrie Jr.</Name>
      <Type>Writer</Type>
      <Role>
      </Role>
    </Person>
  </Persons>
  <Genres>
    <Genre>Crime</Genre>
    <Genre>Drama</Genre>
    <Genre>Romance</Genre>
    <Genre>Thriller</Genre>
  </Genres>
  <Studios>
    <Studio>Kings Road Entertainment</Studio>
  </Studios>
  <Countries>
    <Country>USA</Country>
  </Countries>
  <FilmingLocations>
    <FilmingLocation>Moss St. by Cabrini High School, New Orleans, Louisiana, USA</FilmingLocation>
    <FilmingLocation>Antoine's Restaurant - 713 St. Louis Street, French Quarter, New Orleans, Louisiana, USA</FilmingLocation>
    <FilmingLocation>French Quarter, New Orleans, Louisiana, USA</FilmingLocation>
    <FilmingLocation>Louis Street, New Orleans, Louisiana, USA</FilmingLocation>
    <FilmingLocation>New Orleans, Louisiana, USA</FilmingLocation>
    <FilmingLocation>Piazza d'Italia - 377 Poydras Street, New Orleans, Louisiana, USA</FilmingLocation>
    <FilmingLocation>Poydras Street, New Orleans, Louisiana, USA</FilmingLocation>
    <FilmingLocation>Tipitina's, New Orleans, Louisiana, USA</FilmingLocation>
  </FilmingLocations>
  <TagLine>Between them lies the fate of the state of New Orleans</TagLine>
  <Taglines>
    <Tagline>Between them lies the fate of the state of New Orleans</Tagline>
    <Tagline>A cop and a lady lawyer caught in an explosive truth of police corruption</Tagline>
    <Tagline>Police action at its best</Tagline>
  </Taglines>
</Title>

 

It is btw some of a puzzle to me when actors are being read from my movie.xml files. For some movies they do, for some, tjhey don't.

But that's outside this exact tracing.  More a question for Luke, I think.

Also attached a screenshot from Emby Cast & Crew.

Attached Files


Edited by pju, 23 January 2019 - 03:15 PM.


#51 pünktchen OFFLINE  

pünktchen

    Advanced Member

  • Members
  • 2074 posts
  • Local time: 08:33 PM

Posted 23 January 2019 - 03:22 PM

@pju new plugin version with some extra logging is online. It doesn't seem to auto update with Emby beta server...



#52 pju OFFLINE  

pju

    Advanced Member

  • Members
  • 115 posts
  • Local time: 08:33 PM

Posted 23 January 2019 - 03:33 PM

From the Log:

2019-01-23 20:28:21.647 Info HttpServer: HTTP GET http://localhost:809.../RunFullRestore. UserAgent: Mozilla/5.0 (Windows NT 6.1; rv:64.0) Gecko/20100101 Firefox/64.0
2019-01-23 20:28:21.663 Info Plugin: [People Backup] Person found in Database: Jim McBride
2019-01-23 20:28:21.710 Info Plugin: [People Backup] Person found in Database: Daniel Petrie, Jr.
2019-01-23 20:28:21.710 Info Plugin: [People Backup] Person found in Database: Dennis Quaid
2019-01-23 20:28:21.725 Info Plugin: [People Backup] Person found in Database: Ellen Barkin
2019-01-23 20:28:21.725 Info Plugin: [People Backup] Person found in Database: Ned Beatty
2019-01-23 20:28:21.741 Info Plugin: [People Backup] Person found in Database: John Goodman
2019-01-23 20:28:21.741 Info Plugin: [People Backup] Person found in Database: Lisa Jane Persky
2019-01-23 20:28:21.756 Info Plugin: [People Backup] Person found in Database: Ebbe Roe Smith
2019-01-23 20:28:21.756 Info Plugin: [People Backup] Person found in Database: Tom O'Brien
2019-01-23 20:28:21.772 Info Plugin: [People Backup] Person found in Database: Grace Zabriskie
2019-01-23 20:28:21.772 Info Plugin: [People Backup] Person found in Database: Gailard Sartain
2019-01-23 20:28:21.772 Info Plugin: [People Backup] Person found in Database: Gary Cole
2019-01-23 20:28:21.788 Info Plugin: [People Backup] Person found in Database: Laura Harris
2019-01-23 20:28:21.788 Info Plugin: [People Backup] Person: Laura Harris; Source: d:\emby-test\metadata\People\L\Laura Harris; Destination: C:\Users\mc\AppData\Roaming\Emby-Server\programdata\metadata\library\f7\f7d81fd9170340429dea0c5dfc1d06d9
2019-01-23 20:28:21.803 Info Plugin: [People Backup] Person found in Database: Rossif Sutherland
2019-01-23 20:28:21.803 Info Plugin: [People Backup] Person: Rossif Sutherland; Source: d:\emby-test\metadata\People\R\Rossif Sutherland; Destination: C:\Users\mc\AppData\Roaming\Emby-Server\programdata\metadata\library\63\63763dceb300450d8860f12aa62a4be6
2019-01-23 20:28:21.819 Info Plugin: [People Backup] Person found in Database: Nahanni Johnstone
2019-01-23 20:28:21.834 Info Plugin: [People Backup] Person: Nahanni Johnstone; Source: d:\emby-test\metadata\People\N\Nahanni Johnstone; Destination: C:\Users\mc\AppData\Roaming\Emby-Server\programdata\metadata\library\18\181466f84e984b3bb7fe18947b7523e1
2019-01-23 20:28:21.850 Info Plugin: [People Backup] Person found in Database: Charles Ludlam
2019-01-23 20:28:21.850 Info Plugin: [People Backup] Person found in Database: Marc Lawrence
2019-01-23 20:28:21.866 Info Plugin: [People Backup] Person found in Database: Solomon Burke
2019-01-23 20:28:21.866 Info Plugin: [People Backup] Restore of People successfully finished

 

And an example from the disk:

 

D:\>dir "d:\emby-test\metadata\People\D\Dennis Quaid"
Volume in drive D is D-data
Volume Serial Number is 606C-C4B5
Directory of d:\emby-test\metadata\People\D\Dennis Quaid
0-01-2019  22:28    <DIR>          .
0-01-2019  22:28    <DIR>          ..
0-12-2011  18:21            37.958 folder.jpg
9-01-2018  18:49             1.117 person.xml

 

The content:

 

d:\>type "d:\emby-test\metadata\People\D\Dennis Quaid\person.xml"
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Item>
  <TMDbId>6065</TMDbId>
  <IMDbId>nm0000598</IMDbId>
  <LocalTitle>Dennis Quaid</LocalTitle>
  <BirthDate>1954-04-09</BirthDate>
  <PlaceOfBirth>Houston, Texas, USA</PlaceOfBirth>
  <DeathDate>
  </DeathDate>
  <Overview>From Wikipedia, the free encyclopedia.
Dennis William Quaid (born April 9, 1954) is an American actor. He became known
during the 1980s after appearing in several successful films, including as Mike
Brody in Jaws 3-D (1983), Alex Gardner in Dreamscape (1984), Remy McSwain in The
 Big Easy (1987), Tuck Pendleton in Innerspace (1987), Jefferson "Jeff" Blue in
Undercover Blues (1993), Bowen in Dragonheart (1996), Joe Doe/William in Gang Re
lated (1997), Frank Towns in Flight of the Phoenix (2004), Jack Hall in The Day
After Tomorrow (2004) and Reverend Shaw Moore in Footloose (2011).
Description above from the Wikipedia article Dennis William Quaid , licensed und
er CC-BY-SA, full list of contributors on Wikipedia.</Overview>
  <Website>
  </Website>
  <Adult>False</Adult>
  <LockData>False</LockData>
</Item>
d:\>

 

 



#53 pju OFFLINE  

pju

    Advanced Member

  • Members
  • 115 posts
  • Local time: 08:33 PM

Posted 23 January 2019 - 03:36 PM

Running backup in stead of restore?



#54 pünktchen OFFLINE  

pünktchen

    Advanced Member

  • Members
  • 2074 posts
  • Local time: 08:33 PM

Posted 23 January 2019 - 03:44 PM

Ah okay, found the cause for missing images: i'm only searching for "poster.jpg" right now. Will include "folder.jpg" with an update.

Is your log from a test library with only one movie? Otherwise there should be thousand lines of "Person found in Database".


Edited by pünktchen, 23 January 2019 - 03:51 PM.


#55 pju OFFLINE  

pju

    Advanced Member

  • Members
  • 115 posts
  • Local time: 08:33 PM

Posted 23 January 2019 - 04:04 PM

:) I would have thought you should look for the .xml file. I have lots of actors with no picture at all.

And yes, I have only copied 6 movies to my VM.

Moreover, for some reason the library scan seems not to read all movie.xml until I have done some change to it.

Seems like Emby somhow see the movie.xml file as read without having fetched the content.

Guess, I'll have to ask Luke about the background for this.



#56 pünktchen OFFLINE  

pünktchen

    Advanced Member

  • Members
  • 2074 posts
  • Local time: 08:33 PM

Posted 23 January 2019 - 04:13 PM

:) I would have thought you should look for the .xml file. I have lots of actors with no picture at all.

It's either xml or image with the restore process. The backup has a condition to only include persons with an image right now.

New version online!



#57 pju OFFLINE  

pju

    Advanced Member

  • Members
  • 115 posts
  • Local time: 08:33 PM

Posted 23 January 2019 - 04:34 PM

Moving forward - much better. 

Am I correct in that you only log Source/Destination for images?

I'll just mention that I'll be somewhat in the dark if an image is needed for backup.

I have quite a bit of old and  non-commercial movies, where many of the actors do not have an image to be scraped.

 

But, as I understand it - I could in principle go on as before using Metabrowser (for as long as it lives as not maintained).
And the use your plugin to update the Emby database.
Changes to an actor should then be made in Emby and backed up to my /People directory… ?
Does the backup overwrite existing files?
Will it at the end of the day be possible to run it under the scheduler.
Of course would be the integration with Emby XML as Luke proposed.
But till now - great initiative!!


Edited by pju, 23 January 2019 - 04:42 PM.


#58 pünktchen OFFLINE  

pünktchen

    Advanced Member

  • Members
  • 2074 posts
  • Local time: 08:33 PM

Posted 23 January 2019 - 05:44 PM

You are right, i should also log and backup metadata only people.
There's already a backup task without any trigger that replaces all poster.jpg, fanart.jpg and person.xml with changes that you've made in Emby. The first task run is a full back of course.

#59 pünktchen OFFLINE  

pünktchen

    Advanced Member

  • Members
  • 2074 posts
  • Local time: 08:33 PM

Posted 23 January 2019 - 08:28 PM

You are right, i should also log and backup metadata only people.

Done. New version is online.



#60 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 131163 posts
  • Local time: 02:33 PM

Posted 23 January 2019 - 10:08 PM

I am still planning on having the server support the old path. I just haven't gotten to it yet. Thanks.


  • Abobader, Guy, Happy2Play and 2 others like this




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users