We, The Supporters, Want Metadata Backup/Restore

[Endymion 6]

There is plenty to say BUT lets keep it On Topic.

I wasn't sure where to post this, because at the moment, I'm dealing with this issue in the Asustor update. However, this ISSUE applies to all version. So, I feel it belongs in general.

 

With that being said... Lets talk about that HUGE GLOWING PINK ELEPHANT in the room.

 

I've seen this question asked over a year ago and from what I'm seeing now, it is still NOT being addressed. In FACT, I've asked 3 days ago and have NOT gotten a reply. I've seen others ask a similar question with in this time period and they have NOT gotten a reply. Ignoring the question dose Not make it go way. It just pisses people OFF (Poor Customer Service).

What does it take... 30 second... a minute to reply??? To say that there is NOT away at the moment and we are to lazy to address the issue. lol Maybe it's been asked so many times that y'all just ignore it now. If that is the case, that makes it even worst.

 

Seriously, do you NOT see an issue? It would solve so MANY problems, if you would just give us the ability to back up our metadata and restore it "PERIOD" Especially when it takes a week to rebuild the library... like myself. You could build all the newer (better) versions you want and if our metadata gets wiped or we need to transfer it over to the newer (better) version... guess what? We could do that. Just like we can with user/server settings (plug in). Of course, you have to be a supporter to do so.

 

The way I see it... in this market there is you (Emby) and that other guy (starts with a "P"). "P" is every where and Emby... not so much (working on it). "P" has the ability to "backup metadata", Emby does NOT. Emby gives us MORE control on how we want our sever/s to run, "P" does NOT. On my weighting scale, I'd rather more control. That is why I've continued to support Emby. However, Emby, seems to continue to NOT listen to its customers (supporters) or even realize that one option would make life easier on every one.

 

It's a dog eat dog world. If you continue to ignore the big issues... you will get eaten up (lose customers/supporters).
 

OK... maybe this belongs in Feature Request... please move if need be.

Edited September 23, 2017 by Endymion

[ebr 16177]

Hi.  We offer you the ability to store the metadata alongside the media.  This is a very good way to be sure your metadata is always where you need it and allows it to be backed up with normal backup solutions as well if you wish.

 

We have not concentrated on a full backup feature primarily because there are many pure backup solutions available that could accomplish this so we are concentrating on the media-oriented features of the system at this time.

[gstuartj 40]

Doing a full server backup is already very easy outside of the plugin. Just copy your Emby settings directory. Boom, done. When you need to restore, copy it back to the same location. Everything will be intact, including libraries, server settings, users, AND metadata. Or store metadata with your media.

 

I've seen this feature requested once or twice, but I think you may be mistaken about how "wanted" this functionality is in the Emby plugin itself. It's certainly nowhere near the top of my list of desired improvements since there are existing ways to do it.

Edited September 23, 2017 by gstuartj

[Endymion 6]

OK... if there is so many back up solutions... Then could you please pass that info along. And in those back up solutions... does it restore the metadata if wiped by a new update or new version? What steps need to be taken? This kind of info needs to be Pinned.

 

From my experience, even if you already have the metadata stored with the media, Emby still scans and replaces it. For an example (I've mentioned this before), I store my Movie metadata in there respective folders. All the other metadata is stored in Emby's default setting. The folders for TV shows gets way to cluttered with all those extra files. Anyway, I've done an Emby install with the metadata in my movies and Emby still scanned and replaced that info. At which point I had to go back and edit again.

 

I mean seriously... This is a MAJOR feature that is NEEDED (plugin or whatever). Especially when it takes a week to get all that info again. I am far from the only one asking about it. I'm seeing more ask about it now since some of these new updates. Hell, I'd be happy if I could just copy some folders over and it would work. BUT, that doesn't seem to work either

 

So, even a work around, to save us days of work would be helpful. NONE of us supporters should have to spend days/week rebuilding our library because of an updates or new version. To me and many others... it should be all hands on deck... stop what you are doing... and get this ISSUE fixed. This issue has been brought up over a year ago for sure. And I'm sure it's been brought up since then, I very rarely look in the forums, so I'm not sure. I do know I've seen it asked about over a year ago.

[gstuartj 40]

OK... if there is so many back up solutions... Then could you please pass that info along. And in those back up solutions... does it restore the metadata if wiped by a new update or new version?

 

I did. Just copy the entire Emby settings directory. Use whatever tool you like or do it manually. Yes, it will restore the metadata if wiped.

 

Sorry, I just don't agree that this is a critical priority issue.

Edited September 23, 2017 by gstuartj

[Luke 42078]

The problem with having the plugin do it is that backups will be massive. Why not just turn on metadata saving to media folders?

[Endymion 6]

I did. Just copy the entire Emby settings directory. Use whatever tool you like or do it manually. Yes, it will restore the metadata if wiped.

 

Sorry, I just don't agree that this is a critical priority issue.

 

I do not see a Settings folder, Can you be more specific? Is it called something else or am I missing something?

 

Try spending a week rebuilding your library and then come tell me you don't see the issue. Not trying to sound rude... even though it's coming out that way.

 

The problem with having the plugin do it is that backups will be massive. Why not just turn on metadata saving to media folders?

 

And that is my point. Can you imagine spending a week trying to get all that info again (thousands of movies and tv shows). I know it's massive. I have all that info stored now. I just wish to know how to get to work in new updates (versions) With Out it having scan again. And as I've mentioned (it maybe fixed now) the last time I did an Emby install, it still scanned and replaced the metadata I already had. I'm trying to avoid all that.

[afullmark 20]

This might help: https://emby.media/community/index.php?/topic/46226-backup-restore-migration-behaviour-for-metadata/

 

@@Luke Did an update to the backup wiki to clarify matters, as I too was a little miffed by how things worked having been used to plex take care of all this:

 

 

 

Use the Emby Backup Plugin

We recommend using the Emby Backup plugin, which is designed to make this process really painless by doing the work for you. Emby Backup requires Emby Premiere and can be found in our plugin catalog.

The Emby Backup plugin can help you backup and restore the following:

• Server configuration
• Users
• User data (playstates, favorites, etc).
• Installed plugins
• Plugin settings
• Playlists

This will not backup library contents and metadata. To keep a permanent copy of metadata, we suggest enabling saving of local metadata to media folders.

 

 

And this is from @@ebr:

 

If you don't store metadata with the media, then it isn't saved anywhere except the database.

 

If your concern is to backup the system to recover from a failure and be able to restore it right back on the same system, then any typical backup solution would be able to do that by simply backing up the entire Emby folder.

 

Our goal with this plug-in was to make an easy way for someone to backup and restore just the configuration of an Emby system for either migration to a new location or to restore a configuration to a known working one.

 

If you customize your metadata at all, we highly recommend storing that metadata alongside the media.

 

 

And @@Happy2Play said something interesting too:

 

Well the metadata folder stores your media metadata by unique ids and those ids change with every database rebuild so the media folder becomes useless at least for your library sub folder.

 

Collections are as long as structure does not change as it is recorded in collection metadata

 

Playlists have a user id in metadata that could change during reinstall.

 

 

So backing up the Emby folder is NOT enough.

 

But you are saying that Emby replaced all the .nfo files (and your edits) anyhow, which doesn't said right and either @@ebr or @@Luke would have to comment on that.

 

I do tend to agree with you. Not having to bother with .nfo and have Emby just have one backup plugin that takes care of everything would be best (and that's why plex has it, because it is easy and people like easy). 

Edited September 23, 2017 by afullmark

[Happy2Play 9780]

I don't see a need.  With metadata saved with media it doesn't take me a week to start over with 38TB of Movies and TV.  With specific settings it can be done in less than 24hr.  A new build without metadata is another story.

 

And @@Happy2Play said something interesting too:

 

 

 

So backing up the Emby folder is NOT enough.

 

But you are saying that Emby replaced all the .nfo files (and your edits) anyhow, which doesn't said right and either @@ebr or @@Luke would have to comment on that.

 

I do tend to agree with you. Not having to bother with .nfo and have Emby just have one backup plugin that takes care of everything would be best (and that's why plex has it, because it is easy and people like easy). 

 

If you are backing up the Emby Server folder, the yes it is enough as you are backing up the db files also.  A full backup covers anything catastrophic.  But a backup like this should be OS level not application level.

Edited September 23, 2017 by Happy2Play

[kanipek 230]

I don't want to open a can of worms here - that being said I see absolutely no advantage to not storing metadata locally (with your media files). In my opinion it is a far simpler solution - especially for those using Emby with Kodi which I do for one client. In case of failure, reload, upgrade - whatever the reason a restore is able to be done in just a few minutes and with a library scan completed afterwards you are done. 

 

This is assuming you are maintaining a current backup of those media files. It is not a one stop(step) and done process and that may be the OP point. But as @@Luke said the size (and therefore speed) of the backup/restore process would be significantly increased.

 

I think if the Emby creators/coders/maintainers do choose to include metadata backup as part of the plugin in it should be an option to include it or not include it.

 

Anyway that is my two cents for what it's worth.

[gstuartj 40]

 

And @@Happy2Play said something interesting too:

 

 

 

So backing up the Emby folder is NOT enough.

 

Backing up the Emby folder does backup the database. It's a full backup and works great. I do it frequently when building new VMs, etc. It's honestly the best option, IMO.

[afullmark 20]

I don't see a need.  With metadata saved with media it doesn't take me a week to start over with 38TB of Movies and TV.  With specific settings it can be done in less than 24hr.  A new build without metadata is another story.

 

 

If you are backing up the Emby Server folder, the yes it is enough as you are backing up the db files also.  A full backup covers anything catastrophic.  But a backup like this should be OS level not application level.

 

 

 

So what did you mean by:

 

Well the metadata folder stores your media metadata by unique ids and those ids change with every database rebuild so the media folder becomes useless at least for your library sub folder.

 

[gstuartj 40]

I don't want to open a can of worms here - that being said I see absolutely no advantage to not storing metadata locally (with your media files). In my opinion it is a far simpler solution - especially for those using Emby with Kodi which I do for one client. In case of failure, reload, upgrade - whatever the reason a restore is able to be done in just a few minutes and with a library scan completed afterwards you are done. 

 

In my case the storage on which I run Emby is far faster than my media storage. Does Emby still store copies of the metadata in the DB if storing with the media is enabled? If so I should turn that on. If not then it could create minor performance issues for me.

[afullmark 20]

Backing up the Emby folder does backup the database. It's a full backup and works great. I do it frequently when building new VMs, etc. It's honestly the best option, IMO.

 

 

I did wonder why @@Happy2Play said:

 

Well the metadata folder stores your media metadata by unique ids and those ids change with every database rebuild so the media folder becomes useless at least for your library sub folder.

 

 

And you are saying this is NOT true?

 

And @@ebr said:

 

If you customize your metadata at all, we highly recommend storing that metadata alongside the media.

 

 

So for fresh install or just rebuild you are saying that backing up Emby folder IS EONUGH and .nfo is not need. Even for custom genes and the likes? 

[gstuartj 40]

So what did you mean by:

 

He means that if just the metadata folder is restored without also restoring the DB then it won't work. But if you restore the entire Emby config folder (full restore) then it works fine.

[gstuartj 40]

> So for fresh install or just rebuild you are saying that backing up Emby folder IS EONUGH and .nfo is not need. Even for custom genes and the likes? 

 

Correct. Restoring the entire folder is like an exact clone of the server. Everything is exactly the same, including the entire database.

[Endymion 6]

I don't see a need.  With metadata saved with media it doesn't take me a week to start over with 38TB of Movies and TV.  With specific settings it can be done in less than 24hr.  A new build without metadata is another story.

 

 

If you are backing up the Emby Server folder, the yes it is enough as you are backing up the db files also.  A full backup covers anything catastrophic.  But a backup like this should be OS level not application level.

 

I have over 100TB (all in h264 format no full disk copies)... so yes it takes me a week to get all that info again. I just did that like 6 months ago.

 

Back to the point... I and others would like to know what we have to do to get our metadata to work (transferred) in new updates or in my case, at the moment, new version (Asustor update). I'm sure that info can be applied to others.

[gstuartj 40]

I have over 100TB (all in h264 format no full disk copies)... so yes it takes me a week to get all that info again. I just did that like 6 months ago.

 

Back to the point... I and others would like to know what we have to do to get our metadata to work (transferred) in new updates or in my case, at the moment, new version (Asustor update). I'm sure that info can be applied to others.

 

The directory you need to backup varies by platform. On Linux it's typically /var/lib/emby-server. I'm unfamiliar with the ASUStor platform, so hopefully someone else can chime in. Is there a way to backup entire app containers/jails/whatever they use? If so, that would work.

[afullmark 20]

 

> So for fresh install or just rebuild you are saying that backing up Emby folder IS EONUGH and .nfo is not need. Even for custom genes and the likes? 

 

Correct. Restoring the entire folder is like an exact clone of the server. Everything is exactly the same, including the entire database.

 

 

except @@Luke said this:

 

Unfortunately, in order to achieve dual compatibility with the .net framework and .net core, I've had to rework the configuration format a little bit. This means that all custom mappings are going to be lost. Apologies.

 

 

https://emby.media/community/index.php?/topic/50425-32292-genre-cleaner/?p=481713

 

So it would seem only .nfo is enough is this case not an embryo folder restore?

[Endymion 6]

The directory you need to backup varies by platform. On Linux it's typically /var/lib/emby-server. I'm unfamiliar with the ASUStor platform, so hopefully someone else can chime in. Is there a way to backup entire app containers/jails/whatever they use? If so, that would work.

 

Yea... it's about the same. But, just copying the metadata over doesn't work. As from what was posted in this thread, I see why. It has to do with unique ids. So, only a thought, I have to get that info transferred over, too. Or at least get the new up date to read all that info. lol I don't know... why can't this just be easy??? lol

[gstuartj 40]

Yea... it's about the same. But, just copying the metadata over doesn't work. As from what was posted in this thread, I see why. It has to do with unique ids. So, only a thought, I have to get that info transferred over, too. Or at least get the new up date to read all that info. lol I don't know... why can't this just be easy??? lol

If you have a /var/lib/emby-server folder then copy that. It's not just metadata. It contains the metadata folder, but it also contains the entire database, including those unique ids. That's what you need.

[gstuartj 40]

except @@Luke said this:

 

 

https://emby.media/community/index.php?/topic/50425-32292-genre-cleaner/?p=481713

 

So it would seem only .nfo is enough is this case not an embryo folder restore?

 

If you copy the directory Emby resides in and then restore it, it will be restored as an exact clone, period. You will not lose mappings from a full restore. That post is regarding changes to the genre cleaner config format, but if you're already using the new format and do a full restore it should work.

[Happy2Play 9780]

Was the library.db built with this metadata already in place?  As I can delete my library.db and allow it to rebuild and appears to use those same ids.  Every scenario is unique in its outcome.

 

 

But a OS or folder level backup is your best option.   As you can restore a earlier version when issues arise.

[CBers 7450]

This thread is boring!!

[Happy2Play 9780]

And as for the Title of this topic.  I as a "Supporter" would never use or need this option.