Metadata Savers?

[Lumute 108]

Hi Guys,

 

I would like to better understand the purpose of the Metadata Savers, is this needed for anything other than keeping a backup of the metadata or having it together with the media so it can be used by other applications? all the metadata is already in the Emby internal database, correct?

 

I only use Emby & Kodi with the plugin and would like to keep my media folders as clean as possible. Is there any use to enable any of these Metadata severs?, for what I have read in the Kodi plugin it is not used by Kodi at all, correct?

 

Thanks!

Edited August 11, 2015 by Lumute

[Deathsquirrel 745]

If you never edit the downloaded metadata, no, no point to local copies really.  If you you do make changes and want those to surviveman application reinstall or want those changes reflected in other apps that may read that library in the future, then save local copies.

[Lumute 108]

Awesome! now, if I edit the Metadata through the Emby Metadata Manager, and I keep regular backups using the Backup plugin, that would allow me to restore after a re-install so those changes would survive, right?

 

Finally, if I ever need the metadata, turning on the saver and running a full library scan would generate it for all the media in the Library? or how could I force the generation of all those metadata files?

Edited August 11, 2015 by Lumute

[sh0rty 717]

 

 

Awesome! now, if I edit the Metadata through the Emby Metadata Manager, and I keep regular backups using the Backup plugin, that would allow me to restore after a re-install so those changes would survive, right?

 

Finally, if I ever need the metadata, turning on the saver and running a full library scan would generate it for all the media in the Library? or how could I force the generation of all those metadata files?

Yes you need to fully rescan the library. But it's worth it imo.

 

Gesendet von meinem HTC One_M8 mit Tapatalk

[ebr 16184]

The files don't take up much room and they save us having to go out and re-download all of that data from the providers if your server installation changes.  I highly recommend keeping the savers turned on.  It will speed up re-scans and alleviate unnecessary stress on the FREE providers we use.

[Lumute 108]

The files don't take up much room and they save us having to go out and re-download all of that data from the providers if your server installation changes.  I highly recommend keeping the savers turned on.  It will speed up re-scans and alleviate unnecessary stress on the FREE providers we use.

 

Thanks Ebr, I understand your point but wouldn't the regular backups take care of that? why would I ever need to re-download all that metadata if I have backups (taken with the Emby Backup Plugin), unless I actually wanted to re-scrape the library from scratch?

 

I'm not questioning your logic, just trying to understand how the system works and make sure the backups don't provide a false sense of data safety...

Edited August 11, 2015 by Lumute

[ebr 16184]

The Emby backup plug-in is a "Server configuration backup" plug-in.

 

It does not backup your metadata or your media.  It just backs up the server configuration files and user data.

[Lumute 108]

Oh, I really thought the internal library / database was part of the backups... That's bad then, of course the Media cannot be part of the backup but the internal Database / Library should. Redundancy which is what metadata savers provide is no substitution for a backup policy (again, not media but library data)...

 

In several occasions a Kodi MySQL db backup saved me from having to re-scrape my whole library and lose the watched states and all the time I spent customizing the metadata. Keeping metadata with the Media and doing nightly backups of Terabytes of data is just not practical and it is very easy to mess up the metadata and want to restore only that.

 

Is there any way I can backup the internal Emby database that contains all the Metadata?

[plazma 13]

Yes, volume shadow copy access + copy is what I would use ;-) for the copy/backup probably something that can do a delta copy (rsync for example) use it for live sql backups all the time.

[Lumute 108]

Yes, volume shadow copy access + copy is what I would use ;-) for the copy/backup probably something that can do a delta copy (rsync for example) use it for live sql backups all the time.

 

So you mean use the Metadata Savers and enable VSS on that drive? sure would do the job but kind of overkill and would also back up the Media which requires lots of extra storage and unnecessary, for Media redundancy is more than enough in my opinion as it is not dynamic data but Metadata is... I'm already synchronizing Media with a cheap cloud storage, but even with Metadata files there it does not solve the problem...

 

I'm wondering about the reasoning behind not including the internal Metadata Library in the backup plugin, I'm still new to Emby so I'm sure I'm missing something here...

Edited August 11, 2015 by Lumute

[ebr 16184]

The plug-in was designed to backup a server configuration for the purposes of moving it to another system.  It wasn't designed as a library backup solution and that is why it is named as it is.

 

When migrating to a new server, the library database information may not be valid because the environment may very well have changed.  The system is designed to be able to re-build this information from the media and metadata as it exists on the new system.

[plazma 13]

Nope not volume snapshot the entire contents of the drive, its possible to create a snapshot, then pull individual files out, a delta copy splits individual files into parts (checksums) and then only does the parts of the files that have changed, that's a delta copy (ignore the sneaky rsync for sale software that called its self detacopy, tut tut).

 

I do it all the time for live sql databases and pst files etc.

 

Even if you Pointed rsync at your entire collection, it would only take the time to do whats changed and even then just the part of the file that changed.

 

If you checkout Cygwin and then you can use rsync with a batch script or just get a compiled port (single exe) it works just as well on Linux as it does on windows.

 

Lots of examples online of the command line to use.

 

http://blogs.msdn.com/b/adioltean/archive/2006/09/18/761515.aspx

 

Ignore all the places that try and sell you rsync for windows, just install Cygwin, tell it to install rsync, its gpl.

 

Now you can call rsync.exe from bafch files (as well as bash in Cygwin) ;-) ive been using it for live backups of stuff for years, script together volume shadow drive letter access and rsync, a little ssh foo and you have a backup solution (on or offsite) that people pay $$$ for commercial software that's doing the same thing.

 

https://www.cygwin.com/

 

;-)

 

I do a daily offsite replication of tb's of data (as only a few mbs changes each day), the backup takes minutes to complete.

 

Rsync was on of those programs that made me get far more into Linux years ago, now I only rock Linux servers and if windows 10 is a sign of things to come its probably going to become my desktop os of choice once windows 7 becomes end of life.

 

But when it comes to meta forget about having clean folders, thanks to emby you never really look at the folders anyway, personally I save all my meta (using media centre master) with my files (emby and the rest of the network only has read only access), should I do a full emby reinstall, move on to something else, the first scans still pretty quick, no download required its all ready local. I don't use emby for meta collection, theres no need I just set the meta in stone as the file goes into storage, then nothing can get deleted, changed or messed up.

 

Not that emby has ever messsd my data, just been down that road with other software, setting it in stone like this means no extra overhead to see if meta has needlessly changed etc and means if the meta was correct the day I put it into storage, its going to always be local and right going forward.

Edited August 11, 2015 by plazma

[Lumute 108]

Thanks Ebr, I understand a little better now... So the Metadata Savers are necessary and the only way to actually maintain proper Metadata Backups is to manually backup those nfo/xml ourselves as the internal Emby database contains paths that may not be the same after a restore on a new server...

 

Thanks Plazma, I get what you mean now. I use rsync often for offsite data replication but to be honest I still think it is overkill for what wee need here, maybe just because of the specifics of my environment, other users may find that's the solution for them so your post is very helpful...

 

What I ended doing is just creating a Window Task that backs up all the nfo/xml files from the Library into a zip file every night using xcopy, just after the Configuration Backup done by the Plugin with a 30 day retention... that should be good enough to restore my Metadata in case of a mess up...

[nickysilver 16]

TODAY'S DILEMA. 

Ok... now I'm trying to add my TV library. It is on a HD, pegged into a hub, plugged into the shield.

When I added the movie library, emby saw the dusk and allowed me to browse folders.

When I try to add the TV library it sees the disc, but there are no folders. (I have no idea what the path is!) The disc is fine. I can play a video accessing the folder/files using VLC, so I know it's there and working.  (I realize I'm a pain and not the brightest!)

[GrimReaper 4740]

Will you be writing anything in that folder by Emby server (artwork, NFOs...)?

[nickysilver 16]

Yes.

[GrimReaper 4740]

Make a folder named "NVIDIA_SHIELD" and put your main TV shows folder in it, see if it makes a difference. 

[nickysilver 16]

That will take a while... I have moved other things onto that disc so I'll have to shut down the shield before I can remove it to add a folder.

[nickysilver 16]

Got it!!!!

 

(I do apologie for being a chore)