Using VHDX Virtual Harddrive for Metadata

[denethor 90]

Last 2 days I was dealing with the in place upgrade of my Win8.1 HTPC  to  Win10. Actually nothing went wrong with upgrade but I did some mistakes while deleting my Windows.old folder to save some space. (Note: Win10 would delete that folder safely after 30 days of upgrade. There was a scheduled task for that) Anyway lets get back to topic.

 

I recovered my setup mostly but I decided to go clean install route to be in a safe side. But I hit to very big problem: Backup!

Backing up both Plex and EMBY meta data was really huge pain in the a**!

Gigabytes of data with billions of small files! It takes hrs to backup those files. As a bonus, during my back-up my 128GB Kingston Hyper-X thrown an error "Disk Full!" I was like "what hell!" And I noticed that file system was ExFAT So 24GB of data took 128GB of files space on disk! Hours of copying wasted and I had to format my USB stick to NTFS with 512 bytes sectors and start over. File copy still continuos...

 

In the mean time I decided to find a better solution for this kind of issue: Billions of small files with just a size of 30-40GB.. And I came up with the topic title and decided to share this with the community as a good practice.

I highly recommend you guys to use virtual hard drives with your metadata folders. This way you can backup your everything with just copying one VHD file in 5 minutes! Here is the quick how to:

 

- Open Disk Management. (Hit Win+X and select Disk management from pop-menu)

- From the action menu select "Create VHD"

- Set your location for VHD, set your desired size, VHDX as VHD disk format  and select Dynamically Expanding for disk type. Hit OK.

- Now you are back in main "Disk Management" window. Initialise Disk, Create Volume, assign drive latter and format it with NTFS.

- Now go to your EMBY dashboard. Select Metadata from the left hand menu and click to Advanced. Set your metadata path to your new virtual drive.

 

Follow below link to mount your VHD on every restart.

http://www.eightforums.com/tutorials/20342-vhd-auto-mount-startup-windows-8-a.html

Now you can copy/move just one file for backup or migrate your metadata for saving some space...I think same method can be used for both Kodi, Plex, Emby. This is just a initial setup for me. I will post the real life results. Performance etc...

 

 

[Deathsquirrel 745]

Or just write your metadata to the movie folders themselves.  Saves a hell of a lot of time and makes backup easy.  Throw in the server config backup utility and all you really have to do in a rebuild is allow the sysdtem to build its cache folders.

[JeremyFr79 228]

Or just write your metadata to the movie folders themselves.  Saves a hell of a lot of time and makes backup easy.  Throw in the server config backup utility and all you really have to do in a rebuild is allow the sysdtem to build its cache folders.

This doesn't work for everyone.  Some of us like myself are running large RAID array's optimized for LARGE file sizes where these "little" files would waste TONS of space.  Luckily I have a dedicated Emby server with it's own drives and RAID's and I have a RAID in it specifically for metadata and small files so I'm not watching drive room get eaten away by little files that use large stripes.

[denethor 90]

Either way once you need a backup or need to migrate somewhere else both solution will be failed. You have to backup/move those millions of files. Using single 64GB VHD file makes things more portable and thats the whole point.

By the way I have enabled compression within my VHD file this also adds more efficiency to drive space especially for the small files. You don't want enable compression on your RAID or system root drive but with VHD its more applicable.

 

In the mean time my tests are going well I don't have any performance issues with the VHDx. But I am using it on Mac Pro PCI-E Flash storage which is really fast. I will test mounting VHD from SMB share in coming days. AFAIK Windows 10 SMB 3.x have some performance advantages for samba shares and Synology DSM 5.x supports it.

 

Edit: AFAIK using movie folders for metadata not supported for Kodi Thumbnail folder and Plex.

Edited August 24, 2015 by denethor