Metadata best practices - legacy metadata and media file migration

[mhaubrich 2]

History: I've been using Emby since at least 2015 (back when it was MB), and in the "old days" I think the default/recommended practice was to store all of the .nfo, .jpg, etc. files (which is what I'm calling the metadata, hopefully using that term correctly) right in the media folders.  I think the current recommended practice is NOT do to that, but instead store these "metadata" files in a separate directory local to the Emby Server (???).

My setup: I currently have Emby Server running on a Windows 10 Pro VM (hyper-v) that is running on a Windows 2012r2 file server.  The VM that the Emby Server is running on is basically dedicated to Emby, and has been continuously upgraded since (I think) version 3.0.  The media files are all stored on the file server (Win2012r2) and shared with Emby via UNC paths.  Media are mostly music, movies, and photos, but I do have an HDHomerun and watch/record live TV using Emby.

My situation: I'm looking to retire the Windows 2012r2 server in the next 6 months so doing some planning.  My current plan is to move all of the media to another server (likely UnRaid) and perhaps shift to running Emby Server on a separate Ubuntu instance (not decided yet whether VM or bare metal, and I'm also looking into the option of running it in a container on the file server).

My question: What's the best practice for me to move my media files to the new file server, and how should I migrate/store metadata on the new Emby/file Server?  When I move media files to the new file server, should I also move all of the legacy metadata?  I haven't done much with ratings and honestly am not concerned about my "watch history", so I'm thinking of maybe starting with a "clean slate" and filtering out the .nfo, .jpg. etc. files when I move the files to the new file server.  My media folders are very cluttered with Emby's .jpg and other files/folders.  Also, this approach would give me the opportunity to do some re-organizing of my media files to better fit some of the Emby features that didn't exist in v3.0.  I think I'd also start with a clean slate on the Emby Server side and just start a new server (which, once I'm ready, I would move my Premier license over to) without migrating the old.  However, I've been using Emby so long that I'm afraid I haven't thought of something.  What are some negative side effects of not migrating my metadata?  Any recommendations from the community?  Thanks!

[Carlo 4561]

Hi, what I would suggest is taking a look at these two KB articles which show the current ideal way to have your media setup and what files/folders are supported.
Movie Naming
TV Naming

I do agree with your upgrade plan especially since you have an old system that has went through many upgrades. Starting with a clean library.db will give you all the benefits of optimizations that have been done over the years without the columns in the database no longer used.

If it were me I would get Emby up and running then do a selective restore from backup using Emby Backup Configuration Plugin of only Users & their watch states. There is no need to recreate users or loose the watch state.

Everything else you can setup by hand manually tweaked for the new environment.

Since you want to get a clean start I'd recommend coping over only the media, subs and BIF files but not the NFO or graphics.  That will help you get all new/fresh images.
I just had Emby update all my posters by running a script renaming the poster file and then doing a meta-data refresh with graphic replacement and couldn't believe how much nicer many posters are today then when I first pulled them down. My old posters often had lots of text on them and "muddy" looking where most today have the extraneous text removed and just look a lot better overall!

Above, I mentioned coping the BIF files over and that's because they can take a great deal of time to generate.  The BIF file generation has improved over the years and may now produce smaller files (good thing) then what you presently have to I would run a small test against 6 to 12 movies. Rename the present BIF files to BIFold or similar and have Emby regenerate them.  Now compare file size to see it would be worth allowing Emby to redo them as well for both space and bandwidth savings when playing them back.  If you find the new BIF files smaller then what I would do is still copy them all over with the media and subs but later remove them section/library by section/library and let Emby rebuild them during the nightly process.  This way you can still use the old ones, but over time replace them with more optimized version (best of both worlds).

As for the type of install if at all possible, I would do a bare-metal install or native OS install and not run Emby in a VM or Docker instance but that's just me. If running on Windows or Linux you can do this but if using some other type of hypervisor you may not be able to do this and need to use a VM.

Is this going to be an upgrade in place or will you be building this out on new hardware?
Will you have enough new space in the new environment to copy all media you presently have?

[mhaubrich 2]

Thanks very much for the detailed response.  Nice to get validation that I'm on the right track with my thoughts, and I'll definitely read over those KB articles before I start migrating files.  I don't think I've seen "BIF" files before so that was a new one (glad you mentioned).  I wonder why I'm not seeing those with my media files?  Perhaps being stored somewhere else?

New file server will be built on (mostly) new hardware. Still researching what that might be. I'm leaning toward using UnRAID, and if I were going to run Emby Server as a VM on that server, then probably using an Intel platform (thinking ASRock Rack W480D4U & i7-10700K) so I can take advantage of iGPU and QuickSync. If I have Emby Server on separate hardware, then I might lean toward an AMD solution for the file server (such as ASRock Rack X570D4U & Ryzen 7 5800x).  I have an unused second-hand mini PC with an i5-6500t (35w) that I could use as a dedicated Emby Server on either Ubuntu or Win10 (leaning Ubuntu so I don't have to mess around trying to run Emby Server as a "service" or re-login every time Windows reboots).  I'd love to hear your reasons for going bare-metal for Emby Server rather than a VM - might help me make the decision about hardware for the file server as well as the Emby Server.

Follow-up question: I've been thinking maybe it might be good to have Emby Server access the media folders on my file server as read-only, and I've seen other forum posts that suggest it should be fine (??) to have media files sourced from read-only directories.  But then I got to thinking about live TV and recordings.  Won't I need Emby Server to have some storage it's writing those files to?  I guess I could set up a separate directory on the file server for it to write those to, or I could use "local" storage.  What are your thoughts about having media files be sourced from read-only directories?

 

[Deathsquirrel 745]

I would only add that storing your images and NFO files with the media is still absolutely a good idea as it ensures that if anything goes wrong with your database, you can very quickly recover anything you'd customized.  I almost never use the default poster that Emby pulls, for example, so if I didn't store my images with my movie files I'd be in a world of hurt if my system ever needed rebuilt.

[mhaubrich 2]

That's an interesting point. I was thinking that there are various backup plugin tools to address the problem , but now that you mention I'm not sure whether or not they back up the images and nfos?  I'm planning to have a backup solution for the Emby Server (virtual?) machine, so hopefully that'll cover this scenario.  But it's important to consider how/where the jpegs and nfos are backed up, and I appreciate you mentioning it. 

[GrimReaper 4749]

34 minutes ago, mhaubrich said:

That's an interesting point. I was thinking that there are various backup plugin tools to address the problem , but now that you mention I'm not sure whether or not they back up the images and nfos?  I'm planning to have a backup solution for the Emby Server (virtual?) machine, so hopefully that'll cover this scenario.  But it's important to consider how/where the jpegs and nfos are backed up, and I appreciate you mentioning it. 

No plugin backs-up NFOs/artwork, you must opt for such in any library settings, there are 2 options you need to enable:

1) NFO metadata saver

2) Save artwotk into media folders

Note that it only applies to new and/or changed items, for other exiting ones Refresh Metadata is required in order to trigger writing.