Backup / Restore Actors - Beta with local or web based restore.

[mickle026 400]

The original plugin has ceased development, a new plugin is in development futher on in this thread.  (re-using the thread as the plugin now does this and much more)

Please go to the latest post for the newest updates


Backup / Restore Actors - Beta with local or web based restore.

 

This is by no means meant to replace the other addon for backing up Actors, it is simply an alternative with a different goal.

 

I'm looking for a couple of testers (up to 50), I can supply my web based restore url for my actor database, so you can try restoring from mine and see if it adds any people to you collection.

IT IS NOT A WEB SCRAPER!!

It is ideally suited for an online collaboration, backup to your hdd/ssd drive , ftp them folder structure online some where, add to it via other peoples backups
use a single url for downloads between the collaborators.

You can use either a local disk to restore from, a web based server or a NAS.

With a NAS you have more than 1 option.  You can use it as a local backup/restore server.  You can enable http services in your NAS, open the ports and allow others to restore their images directly from your NAS via your ip address.  http://<yourip>:port/  , http://<your noip/dyndns>/, http://<your noip/dyndns>:port/

Backup to Local Drive (any that your machine can write to including smb)

Restore from Local Backup Location

Restore from Internet Location , ie a http:// location. 

The restore function will check your file/actor integrity and delete/repair corrupt files and associations even if it doesn't replace the image.
if you use the restore function with a valid path but no backup, the app will run the integrity check and fix corrupt actors and placeholders, without restoring any images.

Use / Share backups online to increase your Actors via collaboration

Extracts into an IMDB structured folder
Extracts into an TMDB structured folder
Extracts unknowns in to Actors Folder

Extracts Movie Titles into Folders with Actor Names
Extracts TV Shows into folders with Actor Names

Currently there is no stop button, or progress indicator other than minimal log output.
However if you do NOT navigate away from the page it will update when its done.

You must click "Save Settings" after changing anything to update the server.

There is no liability offered or accepted, use completely at your own risk.

Built with netstandard2.0

Its not been tested in any other environment than win 10 x64, but:

It should be Linux compatible and It should work on all systems Emby runs on, linux or windows based.

Still in development.  Currently has not tested on any other machine than mine.

Open to suggestions.

 

Edited May 23, 2022 by mickle026

[GrimReaper 3289]

Howdy, @mickle026

Before anything, thanks for the plugin, it did solve a "blank placeholder" image issue I had and which was irritating beyond belief. Kudos, and for that alone a big thank you.

Unfortunately, actual Backup/Restore process doesn't work for me (4.5.4.0 stable, Win10) for reasons unknown. Log throws no errors, I'll attach it still. 

Anyway, process was to do a Local Backup and that worked as intended, all folders created and images exported (as you'll see on later screenshots). While we're at it, Actors with dash " - " in the name DO NOT get exported properly - examples Joseph Gordon-Levitt, Gugu Mbatha-Raw and such - folders and files created contain NO DASH hence are unrecognizable for the plugin, as Log shows them named properly. Anyway, upon backup, I deleted 2 random images, Toni Collette and Joseph Gordon-Levitt. Restore/Search started afterwards and completed with 0 images restored. 

As seen in attached screenshots, folders for both of them were made during backup and images are available. However, both Emby and Log shows them as missing as neither got restored. 

I'm open for any further suggestion or tests to be made. 

Cheers

embyserver-63748673926.txt

[mickle026 400]

The backup should remove all puctuation.  So O'Neill becomes ONeill.  Restore would read the Actor as O'Neill from your Library and look for ONeill at the backup location.

I am currently reworking this plugin, because of an issue I found that can stop it completely.  However unless you run across this particualr problem (which is during export - actors with double quotes like "weird al-yannick") it should work just fine.

Can you restart your server and try again?  I have noticed that sometimes edits to people are in the library.db-wal (pending commit to the library.db) file and haven't updated to the library.db file, and will update on exit and restart, so they do not appear when the db is read as missing until a restart.

 

Let me know if it still won't update

Edited February 11, 2021 by mickle026

[GrimReaper 3289]

Restarted server (checked whether images have shown up - nope), run Restore again - same result, Finished with 0 images restored, Log shows same number of missing images, both of them present, 6592 and 6806.

[mickle026 400]

OK, Like I said I am currently looking at reworking that plugin, so will keep an eye out for that.

Are you willing to try a web restore if I give you an Url?

Edited February 11, 2021 by mickle026

[Qowy 0]

Hi,

I wanted to fix my broken Actor images (the blank ones, not the missing ones). But I get this error. Is there a new Version /Plugin somewhere?

2021-11-22 15:44:17.302 Error Server: Error processing request
	*** Error Report ***
	Version: 4.6.7.0
	Command line: C:\Users\Administrator\AppData\Roaming\Emby-Server\system\EmbyServer.dll -service
	Operating system: Microsoft Windows 10.0.14393
	Framework: .NET Core 3.1.21
	OS/Process: x64/x64
	Runtime: C:/Users/Administrator/AppData/Roaming/Emby-Server/system/System.Private.CoreLib.dll
	Processor count: 16
	Data path: C:\Users\Administrator\AppData\Roaming\Emby-Server\programdata
	Application path: C:\Users\Administrator\AppData\Roaming\Emby-Server\system
	System.MissingMethodException: System.MissingMethodException: Method not found: 'System.Collections.Generic.Dictionary`2<System.String,System.String> MediaBrowser.Controller.Entities.BaseItem.get_ProviderIds()'.
	   at MissingActor.BackupRestoreActorsService.Get(SearcMissingActors result)
	   at Emby.Server.Implementations.Services.ServiceController.Execute(HttpListenerHost appHost, Object requestDto, IRequest req)
	   at Emby.Server.Implementations.Services.ServiceHandler.ProcessRequestAsync(HttpListenerHost appHost, IRequest httpReq, IResponse httpRes, RestPath restPath, String responseContentType, CancellationToken cancellationToken)
	   at Emby.Server.Implementations.HttpServer.HttpListenerHost.RequestHandler(IRequest httpReq, ReadOnlyMemory`1 urlString, ReadOnlyMemory`1 localPath, CancellationToken cancellationToken)
	Source: BackupRestoreActors
	TargetSite: System.String Get(SearcMissingActors)

 

[mickle026 400]

14 minutes ago, Qowy said:

Hi,

I wanted to fix my broken Actor images (the blank ones, not the missing ones). But I get this error. Is there a new Version /Plugin somewhere?

The server has had quite a few updates since this plugin and this plugin is now pretty much obsolete.

I am coding a new plugin that has many features and this repair is a 'still to do' within it, so at the moment there isn't a new version but within a month or two there should be.

[TallBoiDez 8]

is there a new version of this plugin

[mickle026 400]

2 hours ago, TallBoiDez said:

is there a new version of this plugin

There will be but not yet, its not done yet.

[TallBoiDez 8]

40 minutes ago, mickle026 said:

There will be but not yet, its not done yet.

[TallBoiDez 8]

@mickle026 Can we get an update on the progress of this plugin?

[mickle026 400]

Hi,

 

Yes its currently much more than it was.  It will change still further, but this is where I am at.

 

Some MAJOR changes and additions -

Fetchers fetch:
People (Actors)
Artists/Album Artists/Backdrops/Logos
Movie Covers/Logos/Backdrops etc
Album Covers/Logos/Backdrops etc
Track(Song) Covers (Backdrops to be added)

Backup / Restore
Movies
Series
People
Artists
Albums

Fix
Allocates items named the same as the media to covers/backdrops etc that are in your media folders

Restore
Assigns from backups

 

 

Edited January 28, 2022 by mickle026

[TallBoiDez 8]

1 hour ago, mickle026 said:

Hi,

 

Yes its currently much more than it was.  It will change still further, but this is where I am at.

 

Some MAJOR changes and additions -

Fetchers fetch:
People (Actors)
Artists/Album Artists/Backdrops/Logos
Movie Covers/Logos/Backdrops etc
Album Covers/Logos/Backdrops etc
Track(Song) Covers (Backdrops to be added)

Backup / Restore
Movies
Series
People
Artists
Albums

Fix
Allocates items named the same as the media to covers/backdrops etc that are in your media folders

Restore
Assigns from backups

 

 

Wow, this looks amazing. Are we able to download and install this version yet?

[mickle026 400]

5 minutes ago, TallBoiDez said:

Wow, this looks amazing. Are we able to download and install this version yet?

I can upload it so you can play with it, but its still alpha (not even beta) and may produce errors or not work quite as expected , ie miss something , fail to restore something etc.
It isnt reporting back to the webpage properly yet.  I am currently converting the logging system to behave on the checkbox and to write my own logs so they are not mixed with the server, thats not finished, theres a lot not finished.  Not everything is set up to restore yet either.

Testing or playing with it would obviously be totally your own resposibility and should be done on a test server first.

 

Do you want me to upload it?

[TallBoiDez 8]

1 hour ago, mickle026 said:

I can upload it so you can play with it, but its still alpha (not even beta) and may produce errors or not work quite as expected , ie miss something , fail to restore something etc.
It isnt reporting back to the webpage properly yet.  I am currently converting the logging system to behave on the checkbox and to write my own logs so they are not mixed with the server, thats not finished, theres a lot not finished.  Not everything is set up to restore yet either.

Testing or playing with it would obviously be totally your own resposibility and should be done on a test server first.

 

Do you want me to upload it?

No, I'll wait till the plugin is more stable

[tr0nllam 34]

16 hours ago, mickle026 said:

I can upload it so you can play with it, but its still alpha (not even beta) and may produce errors or not work quite as expected , ie miss something , fail to restore something etc.
It isnt reporting back to the webpage properly yet.  I am currently converting the logging system to behave on the checkbox and to write my own logs so they are not mixed with the server, thats not finished, theres a lot not finished.  Not everything is set up to restore yet either.

Testing or playing with it would obviously be totally your own resposibility and should be done on a test server first.

 

Do you want me to upload it?

I wouldn't mind messing around with it, if you don't mind uploading it.

[mickle026 400]

On 29/01/2022 at 11:03, tr0nllam said:

I wouldn't mind messing around with it, if you don't mind uploading it.

Download removed, newer version below

 

You can use http://media-and-meta.com as a server, its my crappy website and not very fast but it works, it doesn't have a whole backup on it, it has about 3000 Artist, A lot of  Album Covers, About 4000 Track Covers, a few hundred Series and some People(Actors) but no Movies at all yet.

If you use track covers and get portrait download previews, this was fixed in the beta so will make it later to production server.

 

Edited February 2, 2022 by mickle026

[mickle026 400]

Fixed My Custom Log Files Rotation causing Internal Server Error (restore will work now)
Started to upload People/Actors to http://media-and-meta.com

Added a bit more php so Artists are now clickable, to see meta overviews on the site

 

 

Edited March 28, 2022 by mickle026

[TallBoiDez 8]

6 hours ago, mickle026 said:

Fixed My Custom Log Files Rotation causing Internal Server Error (restore will work now)
Started to upload People/Actors to http://media-and-meta.com

Added a bit more php so Artists are now clickable, to see meta overviews on the site

 

CustomProvider.zip 2.66 MB · 2 downloads

is this the download for the plugin?

[mickle026 400]

Its still in development, I wouldn't even say its in alpha yet , let alone beta but if you wish to try it, its in last post of mine above. (with the same caveats as a few posts of mine ago, I accept no responsibility).

I would like some feeback though as to what works , what errors, what you would like it to do that it doesn't etc.

What I do know:

Odd occasions spew out a ".jpg" on backup without a filename. (so a few images may not be backed up)
Web page (your interface) - some buttons do not respond after click although the function is running.
Web page (your interface) - The info supplied on some buttons might stop updating.

Where I am at in development:

Im currently converting the logging to use my own system. Not everything is done, some stuff still logs to the embyserver.txt file.  Some stuff ignores the checkbox for verbose logging on/off and just logs.
The webIF interface, I am trying to get that to allow you to navigate away and then come back to it and still update.  I can't figure out websockets (im not actually a programmer) so I am trying a few methods to prevent logging of webroutes in the embyserver.txt logfile, using a callback floods it.

What some people don't understand about emby is that there is a backend doing stuff, and you interact using a webpage.  That backend and webpage have to communicate.  Its two different systems talking two different languages and that doesn't blend that well.  Luke seems to have this down to a fine art, nailed it as such.  I am struggling with it.  Its the biggest problem I have.

Plugin coding is also not that easy, you master the code but do a typo , the plugin throws an error but doesn't exactly pinpoint it (it often just tells you it happened somewhere here in this few hundred lines of code) and you cannot run the code in the debugger so it'll stop where the error is and show you.

I am having to learn c#, html, javascript and php for my website as well as run a house, go to work, see my family etc so please be patient - it all takes time, and..... with a quite time demanding wife ..... well you know...... (I know some have waited a long time already).  What should take a few hours can take weeks.

Theres pretty much no instructions yet either.

Quick start Instructions

In Global Tab set a server.  You either have to have created one (using the backup tab) or use mine http://media-and-meta.com or use up to 4 servers.
In Global Tab set a backup location.  Backup your files.  This can take ages as everything is md5 hashed.
Folders are created

ArtistsDB
AlbumDB
SeriesDB
MoviesDB
PersonDB
TracksDB
etc

To create a webserver you can use either online or a NAS (or another pc), copy these to your location (or if your backup location is a NAS, turn on http access), The Fetcher in Emby in the library settings only fetches over http or https.  Part of my code fails with https at the moment, so use http in NAS or website address.
Online sites have a caveat as well, free or cheap webspace usually has an inode limt.  An inode is the index stub for the storage.  Whilst they offer unlimited space they cap the inode, so thousands / 100's of thousands or millions of files cannot be uploaded.
Go to your Emby library and Edit.  Turn of the fetchers.  This allows you to use Emby internal Edit Images/Search and fetchers.  It allows Emby to Look at your server when its adding images automatically and grab one.

If you add any media that requires an image you have in your backup, Emby will grab it from there if its enabled and set up.

One thing I am doing is adding images to tracks by adding them to an album folder named the same as the track. 
First I run Clean Tab > Clean Tracks.  This removes all track images and resets Emby to inherit the album cover.  This is needed because if Emby had Image Extractor Enabled in library settings, then the track will be assigned the embeded cover.  This in many cases is the album cover.  Then using the Fix Tab>Fix Tracks to assign the tracks to the media in Emby, for this you have to have the track cover as an image in the directory with the track, and named the same.  So "Call Me.mp3" has to have an image "Call Me.jpg" (or other extension). Then Backup Tab>Backup Tracks to backup the tracks to my storage.  Then using Restore Tab>Restore Tracks to restore to other tracks of the same artist and track name (regardless of filename), restoring track images will put every track in the whole db by the same artist to the same cover image.  So I can backup 1 Call Me track cover and put it on to potentially dozens of tracks.

Example

Blonde - Call Me, Album Mum Rocks! (I put the image in the folder)
I run Fix Tracks this sets the image in the database.
I run Backup Tracks . Backed up as "Blonde - Call Me"
I run Restore Tracks.

Restores to:

Blondie

Track "Call Me" on "Blonde - The Platinum Collection"
Track "Call Me" on "Various Artists - 101 80's Hits"
Track "Call Me" on "Various Artists - They Sold A Million"

 

etc, until every track that can be identified by the artist with the name "Call Me" has the individual track cover.

 

 

[TallBoiDez 8]

1 hour ago, mickle026 said:

Its still in development, I wouldn't even say its in alpha yet , let alone beta but if you wish to try it, its in last post of mine above. (with the same caveats as a few posts of mine ago, I accept no responsibility).

I would like some feeback though as to what works , what errors, what you would like it to do that it doesn't etc.

What I do know:

Odd occasions spew out a ".jpg" on backup without a filename. (so a few images may not be backed up)
Web page (your interface) - some buttons do not respond after click although the function is running.
Web page (your interface) - The info supplied on some buttons might stop updating.

Where I am at in development:

Im currently converting the logging to use my own system. Not everything is done, some stuff still logs to the embyserver.txt file.  Some stuff ignores the checkbox for verbose logging on/off and just logs.
The webIF interface, I am trying to get that to allow you to navigate away and then come back to it and still update.  I can't figure out websockets (im not actually a programmer) so I am trying a few methods to prevent logging of webroutes in the embyserver.txt logfile, using a callback floods it.

What some people don't understand about emby is that there is a backend doing stuff, and you interact using a webpage.  That backend and webpage have to communicate.  Its two different systems talking two different languages and that doesn't blend that well.  Luke seems to have this down to a fine art, nailed it as such.  I am struggling with it.  Its the biggest problem I have.

Plugin coding is also not that easy, you master the code but do a typo , the plugin throws an error but doesn't exactly pinpoint it (it often just tells you it happened somewhere here in this few hundred lines of code) and you cannot run the code in the debugger so it'll stop where the error is and show you.

I am having to learn c#, html, javascript and php for my website as well as run a house, go to work, see my family etc so please be patient - it all takes time, and..... with a quite time demanding wife ..... well you know...... (I know some have waited a long time already).  What should take a few hours can take weeks.

Theres pretty much no instructions yet either.

Quick start Instructions

In Global Tab set a server.  You either have to have created one (using the backup tab) or use mine http://media-and-meta.com or use up to 4 servers.
In Global Tab set a backup location.  Backup your files.  This can take ages as everything is md5 hashed.
Folders are created

ArtistsDB
AlbumDB
SeriesDB
MoviesDB
PersonDB
TracksDB
etc

To create a webserver you can use either online or a NAS (or another pc), copy these to your location (or if your backup location is a NAS, turn on http access), The Fetcher in Emby in the library settings only fetches over http or https.  Part of my code fails with https at the moment, so use http in NAS or website address.
Online sites have a caveat as well, free or cheap webspace usually has an inode limt.  An inode is the index stub for the storage.  Whilst they offer unlimited space they cap the inode, so thousands / 100's of thousands or millions of files cannot be uploaded.
Go to your Emby library and Edit.  Turn of the fetchers.  This allows you to use Emby internal Edit Images/Search and fetchers.  It allows Emby to Look at your server when its adding images automatically and grab one.

If you add any media that requires an image you have in your backup, Emby will grab it from there if its enabled and set up.

One thing I am doing is adding images to tracks by adding them to an album folder named the same as the track. 
First I run Clean Tab > Clean Tracks.  This removes all track images and resets Emby to inherit the album cover.  This is needed because if Emby had Image Extractor Enabled in library settings, then the track will be assigned the embeded cover.  This in many cases is the album cover.  Then using the Fix Tab>Fix Tracks to assign the tracks to the media in Emby, for this you have to have the track cover as an image in the directory with the track, and named the same.  So "Call Me.mp3" has to have an image "Call Me.jpg" (or other extension). Then Backup Tab>Backup Tracks to backup the tracks to my storage.  Then using Restore Tab>Restore Tracks to restore to other tracks of the same artist and track name (regardless of filename), restoring track images will put every track in the whole db by the same artist to the same cover image.  So I can backup 1 Call Me track cover and put it on to potentially dozens of tracks.

Example

Blonde - Call Me, Album Mum Rocks! (I put the image in the folder)
I run Fix Tracks this sets the image in the database.
I run Backup Tracks . Backed up as "Blonde - Call Me"
I run Restore Tracks.

Restores to:

Blondie

Track "Call Me" on "Blonde - The Platinum Collection"
Track "Call Me" on "Various Artists - 101 80's Hits"
Track "Call Me" on "Various Artists - They Sold A Million"

 

etc, until every track that can be identified by the artist with the name "Call Me" has the individual track cover.

 

 

I added it to my server and it doesn't show up in my plugins list

 

[mickle026 400]

6 minutes ago, TallBoiDez said:

I added it to my server and it doesn't show up in my plugins list

 

Did you restart the server?

[TallBoiDez 8]

24 minutes ago, mickle026 said:

Did you restart the server?

Yes, twice

 

Edited February 2, 2022 by TallBoiDez

[mickle026 400]

ok, next question - so I know where you are at as I cannot see what you are doing.

Did you unzip the archive and install the dll to your "Emby-Server\programdata\plugins\" directory?

What server version are you running?

 

I dont even see your download, there were 2 yesterday and it still says 2 a few posts back

Edited February 2, 2022 by mickle026

[TallBoiDez 8]

44 minutes ago, mickle026 said:

ok, next question - so I know where you are at as I cannot see what you are doing.

Did you unzip the archive and install the dll to your "Emby-Server\programdata\plugins\" directory?

What server version are you running?

 

I dont even see your download, there were 2 yesterday and it still says 2 a few posts back

Yes I unzipped it and put in plugin directory. I'm running emby version 4.6.7.0