schingeck 40 Posted May 19, 2015 Share Posted May 19, 2015 Hey guys, I wanted to ask you all a question on the best way to manage backing up data... I don't want to mirror the files or use up half of my hard drive space on backing up since it's generally easy to replace the files in the event they are deleted. What I would like to know is in the event a hard drive failed on this PC, is there a way to tell which files were affected on said hard drive (or movie/tv shows that are now missing in Emby) so I can go ahead and start putting them back onto the server on a new hard? I use storage pools in Windows but this is new to me so I'm not 100% sure if this feature is available or some software out there that can keep track of these things. Link to comment Share on other sites More sharing options...
JeremyFr79 228 Posted May 19, 2015 Share Posted May 19, 2015 Yes, Emby flag's missing files in your collection if a drive or folder disappears without being properly moved. I sadly know this from experience. Link to comment Share on other sites More sharing options...
schingeck 40 Posted May 19, 2015 Author Share Posted May 19, 2015 Wonderful to know, thank you. Link to comment Share on other sites More sharing options...
Koleckai Silvestri 1150 Posted May 19, 2015 Share Posted May 19, 2015 If you use a Raid 5 setup, you only need one drive to recover data from any single failed drive in the array. So in an array of 5 drives, you only use 20% storage for parity. When a single drive fails, you can rebuild its data on a new drive. Unfortunately, if two drives fail at the same time, the whole raid is lost. This will provide a bit of redundancy and safety without the cost of a full backup array. Link to comment Share on other sites More sharing options...
Spaceboy 2500 Posted May 19, 2015 Share Posted May 19, 2015 Yes, Emby flag's missing files in your collection if a drive or folder disappears without being properly moved. I sadly know this from experience.I don't think you can report on this though, can you? I had similar recently, fortunately I was able to restore 99% from a recent backup but I had to use the report function and sort by date added to find those videos that didn't make it into the back up Link to comment Share on other sites More sharing options...
schingeck 40 Posted May 19, 2015 Author Share Posted May 19, 2015 Would be nice if you could report on it... or idk if anyone uses Couchpotato or sonarr as a manager, if they also had that functionality? I guess Sonarr automatically does because it'll tell you what episodes are missing from your seasons, but movies, I don't know how that works in Couchpotato. Link to comment Share on other sites More sharing options...
JeremyFr79 228 Posted May 19, 2015 Share Posted May 19, 2015 If you use a Raid 5 setup, you only need one drive to recover data from any single failed drive in the array. So in an array of 5 drives, you only use 20% storage for parity. When a single drive fails, you can rebuild its data on a new drive. Unfortunately, if two drives fail at the same time, the whole raid is lost. This will provide a bit of redundancy and safety without the cost of a full backup array. Problem is with todays drive sizes, the time it take's to rebuild the array with a new drive it's almost a guarantee that another drive will die during the rebuild as that is the time of highest stress on the drives. This is where RAID6 comes into play but even that isn't much better. Having a hot spare doesn't help either as you still have to rebuild the array. Luckily in my case I didn't lose a 2nd drive during rebuild when one of my array's lost a drive but with 8x1TB disk's in the array it took over 6 hours to rebuild the array. RAID is never going to be as good as a full backup and is not intended to be. Personally I use RAID5 for the performance and don't count on the parity ever actually saving me. The data I care most about is backed up offsite in real time, the rest is "oh well" data to me. Link to comment Share on other sites More sharing options...
JeremyFr79 228 Posted May 19, 2015 Share Posted May 19, 2015 I don't think you can report on this though, can you? I had similar recently, fortunately I was able to restore 99% from a recent backup but I had to use the report function and sort by date added to find those videos that didn't make it into the back up You are correct I don't believe it creates a report of missing media as of yet, it's something I know I brought it up as a request at one time to the development team in a thread long ago. But at the same time it's not too hard to go through and look at which media has offline plastered across the poster in webclient and copy it down into a text file or something. Link to comment Share on other sites More sharing options...
schingeck 40 Posted July 16, 2015 Author Share Posted July 16, 2015 (edited) Well I ended up having a failure on one of the drives haha. And windows storage pools are HORRID! Using a simple resiliency they are almost useless because if 1 hard drive fails your entire collection is basically done with. Have to delete everything to take 1 drive out of the storage pool and then start over completely. I don't need backups and I don't need parity, I just need to know what was lost on the hard drive failure and replace it (It's easy to replace this data, nothing sensitive so why bother?). Anywho, I found a better solution for those of you that may be looking for a good one. I used StableBits software to connect all my drives to one, like a storage pool in windows but it doesn't break up a file across multiple drives. Instead, it seems to routes them to a specific drive in your pool once you move a file to the pool. You know exactly what files are contained on which drive, you can move the pool to any computer and use it, you can take a hard drive out and use it. Even better is you can use it in the pool BUT you can also move files to it outside of the pool. So you can still use it as a regular hard drive if you'd like. It's much better than using windows storage pools Edited July 16, 2015 by schingeck 2 Link to comment Share on other sites More sharing options...
xnappo 1593 Posted July 16, 2015 Share Posted July 16, 2015 I use the KISS method. With the costs of drives now, I just do plain 'ol nightly backups and use Emby for pooling functionality. Link to comment Share on other sites More sharing options...
ebr 14947 Posted July 16, 2015 Share Posted July 16, 2015 FlexRaid is another option which also gives you some limited failure recovery capability. 1 Link to comment Share on other sites More sharing options...
Sp3kt3r 13 Posted July 17, 2015 Share Posted July 17, 2015 (edited) I've been using flexraid since last September and so far I'm very happy with my setup. All my movies are centralised on my flexraid and I have backup running morning and night and so far no problem with emby. Edited July 17, 2015 by Sp3kt3r Link to comment Share on other sites More sharing options...
shaefurr 1337 Posted July 17, 2015 Share Posted July 17, 2015 (edited) I use flexraid and recommend that, I have 6 drives for data and 1 for parity, gives a little bit of security (I can lose any 1 drive at a time and recover). In the last year I lost the parity drive, didn't replace it right away and then lost a data drive so I lost 3TB of movies/shows. Point being, if you don't keep up on your raid and replace drives as they fail there's no point in doing it I learned the hard way putting it off a while when the parity drive failed. Mediacenter master lets you export an html list of your tv/movies. As long as you make a regular backup list you can print out the before/after lists and highlight the ones missing. Not the best method ever but it works. I'm sure someone out there with any coding could make a program to compare the 2 files and report which ones are missing. At first losing 3TB of media sucked, since it was basically everything current from the last year I had added. But as you said its easy to replace it just takes a while. Been a few months and im about 50% recovered, I still occasionally go to watch something to only find it missing Edited July 17, 2015 by shaefurr Link to comment Share on other sites More sharing options...
hamstercat 63 Posted July 17, 2015 Share Posted July 17, 2015 I'm gonna name-drop SnapRAID here, despite its name it's not a RAID system but rather works with snapshot. You have to run a sync whenever your files changes, and it can both recover and list files when there's a failure. It's open-source and also has (very) basic pooling abilities. Also, since it works with files rather than filesystems, it works nicely with any filesystem and hard drive recovery tools in case of a lot of failures. Link to comment Share on other sites More sharing options...
trusselo 220 Posted July 17, 2015 Share Posted July 17, 2015 Hey guys, I wanted to ask you all a question on the best way to manage backing up data... I don't want to mirror the files or use up half of my hard drive space on backing up since it's generally easy to replace the files in the event they are deleted. What I would like to know is in the event a hard drive failed on this PC, is there a way to tell which files were affected on said hard drive (or movie/tv shows that are now missing in Emby) so I can go ahead and start putting them back onto the server on a new hard? I use storage pools in Windows but this is new to me so I'm not 100% sure if this feature is available or some software out there that can keep track of these things. storage pools should let you create a pool with parity redundancy, that should take up half the space of a 1:1 backup. it is similar to raid 5. that way if any drive dies, you pop-in a new one and it rebuilds the lost data. Link to comment Share on other sites More sharing options...
Overseer 66 Posted July 18, 2015 Share Posted July 18, 2015 Anywho, I found a better solution for those of you that may be looking for a good one. I used StableBits software to connect all my drives to one, like a storage pool in windows but it doesn't break up a file across multiple drives. Instead, it seems to routes them to a specific drive in your pool once you move a file to the pool. You know exactly what files are contained on which drive, you can move the pool to any computer and use it, you can take a hard drive out and use it. Even better is you can use it in the pool BUT you can also move files to it outside of the pool. So you can still use it as a regular hard drive if you'd like. It's much better than using windows storage pools Yep, I use drivepool with my Win Server 2011. Pretty happy. Link to comment Share on other sites More sharing options...
xnappo 1593 Posted July 19, 2015 Share Posted July 19, 2015 Just from personal experience.... Remember parity is NOT a backup solution. [repeat x200000] Link to comment Share on other sites More sharing options...
Xzener 729 Posted July 19, 2015 Share Posted July 19, 2015 Yes... The dreaded "Offline" indication = Butterflies is my stomach. I've been able to recover HDDs in the past, but its not always possible. Just recently lost ALL my 3D Bluray rips. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now