Backup / Restore/ Migration behaviour for metadata

[mtjj 31]

So, I migrated to a new server, using the backup / restore functionality, which worked perfectly.   All my media is on external drives which i connected to the new server and mapped to the same drive letters, and media plays fine. 

 

However, behaviour with regards to metadata is a bit odd, and I have a couple of questions I hope someone can help with:

• I've lost all the actor images. I went back to the old server and found the metadata folder "People"  - can I just copy this over to the new server? Alternatively is there a way to force a full refresh of people to queue all the image downloads? I tried running the scheduled job "Refresh people" but it completed in 26 seconds without updating any images.
• Prior to upgrading, for all foreign movies / TV series I'd laboriously corrected the names to the original script and added the English translations in (brackets). This mostly affected Chinese movies which are really inconsistently named on IMDB etc. I then unticked the "name" on the edit info page, so it wouldn't get overwritten. When I browse in the new server, the "name" field is ticked, and I am terrified that at some point all my updated names are going to get overwritten by a refresh. I tried changing this setting in the metadata manager at a folder level, but this didn't seem to cascade down to individual movies. Is there an easier way to do this than go through every single movie and untick the field again?
• I'd added custom genres to a lot of the foreign media to make browsing easier. This has now been lost   It's not a big deal as I had only done this partially and I guess I can go back and do it again, but for reference, what do I need to backup next time I migrate computers to stop this happening again?

Hope someone can help? I save data to nfo files, if that makes a difference.

 

[Luke 37099]

Yes, copy your old People folder in, then run the People refresh scheduled task.

 

For the next two, those both are saved to metadata, so if you enable saving nfo to media folders, the nfo will be right next to your video files.

[mtjj 31]

Thank you!

 

One question though, where is this option to save nfo to media folders? I might be being dim, but I can't see it anywhere in the metadata or nfo settings pages of the library settings. I found a screenshot here: https://emby.media/community/index.php?/topic/35486-saving-metadata-and-nfo-files/

 

... but it seems the UI has changed since then? I am on version 3.2.7.0

[Luke 37099]

It's just the per library setting of saving metadata to media folders. just click on a library in library setup and you can configure it for that particular library.

[mtjj 31]

Thank you!

[afullmark 20]

Yes, copy your old People folder in, then run the People refresh scheduled task.

 

For the next two, those both are saved to metadata, so if you enable saving nfo to media folders, the nfo will be right next to your video files.

 

 

Hello @@Luke I have a further question. To avoid mass genre listings in search results I have fixed custom genre categories to:

 

 

1. Action & Adventure
2. Animation
3. Comedy
4. Documentary
5. Drama
6. Horror
7. Music
8. Romance
9. Sci-Fi & Fantasy
10. Thriller
11. Gay
12. Classic
13. Crime
14. Western
15. Short
16. Kids & Family
17. Period Drama
18. Mystery 

So, if I use the back-up plugin, at present it will NOT store this information and the ONLY way to retrieve this information if and database restore is needed is if .nfo files are stored with every movie? The genres will be stored in the database anyhow so I was confused that the backup plugin didn't store custom genre data for movies etc. 

[ebr 14925]

Hello @@Luke I have a further question. To avoid mass genre listings in search results I have fixed custom genre categories to:

 

 

1. Action & Adventure
2. Animation
3. Comedy
4. Documentary
5. Drama
6. Horror
7. Music
8. Romance
9. Sci-Fi & Fantasy
10. Thriller
11. Gay
12. Classic
13. Crime
14. Western
15. Short
16. Kids & Family
17. Period Drama
18. Mystery 

So, if I use the back-up plugin, at present it will NOT store this information and the ONLY way to retrieve this information if and database restore is needed is if .nfo files are stored with every movie? The genres will be stored in the database anyhow so I was confused that the backup plugin didn't store custom genre data for movies etc. 

 

Correct.  The proper way to handle this is to store the metadata with the movies.  This makes them portable and keeps the data tied to the item.

 

The full name of what you are calling the backup plug-in is "Server Configuration Backup" because the plug-in is designed to back up your server configuration, not your media or metadata.

 

Thanks.

[afullmark 20]

Correct. The proper way to handle this is to store the metadata with the movies. This makes them portable and keeps the data tied to the item.

 

The full name of what you are calling the backup plug-in is "Server Configuration Backup" because the plug-in is designed to back up your server configuration, not your media or metadata.

 

Thanks.

So does the backup plugin not backup the database? I'm used to plex method whereby the database and settings are backed up not just settings.

[ebr 14925]

So does the backup plugin not backup the database? I'm used to plex method whereby the database and settings are backed up not just settings.

 

No but you could easily do that with any file-based backup program if you wish.

 

However, the information in the database is not portable.  That is, it is unique to the specific locations of your media so if those locations changed, the database information would be incorrect.

 

Again, the plug-in is designed to backup configuration, not data (although it is able to get some information like user watched states etc. in a portable way).

[afullmark 20]

No but you could easily do that with any file-based backup program if you wish.

 

However, the information in the database is not portable. That is, it is unique to the specific locations of your media so if those locations changed, the database information would be incorrect.

 

Again, the plug-in is designed to backup configuration, not data (although it is able to get some information like user watched states etc. in a portable way).

Is this correct if I needed to restore server from scratch:

 

1. Use backup plugin to back all settings and collections and playlists

2. Backup the metadata folder myself

3. Replace folders with those backed up

4. Restart server.

 

So, the above is okay if nothing in the file structure has changed?

 

I would have thought the collections and playlists would also not be portable if you say the metadata is not portable?

 

But in ALL instances you advise to have metadata stored alongside media whether file structure has changed or not rather than just backup default metadata folder?

Edited September 7, 2017 by afullmark

[Happy2Play 8296]

Well the metadata folder stores your media metadata by unique ids and those ids change with every database rebuild so the media folder becomes useless at least for your library sub folder.

 

Collections are as long as structure does not change as it is recorded in collection metadata

 

Playlists have a user id in metadata that could change during reinstall.

[ebr 14925]

If you don't store metadata with the media, then it isn't saved anywhere except the database.

 

If your concern is to backup the system to recover from a failure and be able to restore it right back on the same system, then any typical backup solution would be able to do that by simply backing up the entire Emby folder.

 

Our goal with this plug-in was to make an easy way for someone to backup and restore just the configuration of an Emby system for either migration to a new location or to restore a configuration to a known working one.

 

If you customize your metadata at all, we highly recommend storing that metadata alongside the media.

[afullmark 20]

If you don't store metadata with the media, then it isn't saved anywhere except the database.

 

If your concern is to backup the system to recover from a failure and be able to restore it right back on the same system, then any typical backup solution would be able to do that by simply backing up the entire Emby folder.

 

Our goal with this plug-in was to make an easy way for someone to backup and restore just the configuration of an Emby system for either migration to a new location or to restore a configuration to a known working one.

 

If you customize your metadata at all, we highly recommend storing that metadata alongside the media.

That makes is perfectly clear. Thank you.

[afullmark 20]

If you don't store metadata with the media, then it isn't saved anywhere except the database.

 

If your concern is to backup the system to recover from a failure and be able to restore it right back on the same system, then any typical backup solution would be able to do that by simply backing up the entire Emby folder.

 

Our goal with this plug-in was to make an easy way for someone to backup and restore just the configuration of an Emby system for either migration to a new location or to restore a configuration to a known working one.

 

If you customize your metadata at all, we highly recommend storing that metadata alongside the media.

Could a write up in the backup wiki state something to this effect; it would be really helpful for new chaps to emby like me and avoid much confusion. If you could also state in the wiki what is the preferred method for meta data backup. Basically a wiki that address those questions asked here.

[Luke 37099]

@@afullmark, @@ebr started adding it here:

 

https://github.com/MediaBrowser/Wiki/wiki/Backup

 

Thanks.

[RogerO 5]

I'm new to Emby, and I've been reading a lot of threads (and confusion) about backups.  It's very frustrating to read sarcastic posts from users (not mods) just saying "store your metadata with your media and that's it", which is at best misleading and generally false, and applies only to very simplistic situations.

There are pages like the Wiki page that Luke posted above, which gives some specific instructions for specific circumstances.  But I think the average new user really needs a simple easy-to-read page that gives a broad overview of what datasets exist, where they are stored, how to back them up, and how to restore them for various scenarios:

 

---------------------------------

 

Datasets and their storage locations:

 

* media (this should be obvious to everyone!)

* server config (server config, users, plugins, playlists etc)

* metatada (for metadata stored by Emby in media folders, and list what is included here)

* metatada (for metadata not stored by Emby in media folders, e.g. artists, people, etc)
* others

 

---------------------------------

Options for backing up each dataset, e.g.:

 

* standard PC or Linux backup tools

* Emby backup utilities

* others

 

(This section might just be a set of "backup options" columns to the right of the description of each dataset.)

 

---------------------------------

 

That backup section needs to be written in conjunction with something that describes how to restore each of those sets of data for each broad restore scenario, because how and what you backup partly depends on your restore scenarios:

* fixing a corruped existing install

* preparing a new Emby install

* restoring data (where feasible) to a new install of some other software 

* others

For each asterisked line in this description, you could then link to a different Wiki page that goes into more detail, without cluttering the broad overview.  

 

[Luke 37099]

Thanks, these are great suggestions that can help us improve the backup documentation.