Jump to content


Photo

Hard Drives Dead (What Next)

Home Theatre Windows Media Centre Samsung Smart TV

  • Please log in to reply
39 replies to this topic

#21 Steverido OFFLINE  

Steverido

    Advanced Member

  • Members
  • 217 posts
  • Local time: 11:19 AM

Posted 30 April 2015 - 09:43 AM

I run a windows 8.1 server with 

 

Emby

WMC

ServerWMC

Emby ServerWMC plugin

uTorrent

SickRage

Flexraid tRAID

 

I also have a lot of steam/origin games installed on it, and I play them through Moonlight game streaming. I would like for all my clients to be android/fanless, but until the nvidia shield console (and maybe not even then) is released android doesn't support all my audio/video formats. I don't want to transcode at all on my lan, because it's a waste and I have three family members pretty frequently transcoding from my server at remote locations. I currently run kodi on an intel g1610 (openelec) htpc and a nexus player. The g1610 is play everything, but the nexus player (and the mk808 b+ that I'm trying out) don't do passthrough audio or all video formats.

 

I have been happy with Flexraid tRAID. The main site is down right now, but the setup guide is here http://wiki.flexraid...ck-setup-guide/ and there is also an older discussion thread on the difference between tRAID (newer) and RAID-F (old). http://emby.media/community/index.php?/topic/2167-flexraid-transparent-raid-traid-vs-raid-over-file-system-raid-f/

 

tRAID is realtime parity protection, which isn't backup but gives you the ability to protect against as many drive failures as you have parity disks. The parity disk has to be the largest disk in your array, but they can all be the same size. I have a mismatch of drives

  • 6TB parity (HGST)
  • 6x 4TB (Seagate, HGST)
  • 2x 3TB (Seagate, HGST)

I also have my C: drive (SSD) partitioned to use half for C: (125 GB) and the other half for a tRAID landing disk. With the landing disk, I can write to the pool faster than my scratch (download) disk can send the data. It goes up to about 150 MB/s writes, but I've written a file from the desktop (SSD) just to see it go to 300 ;)

 

Whichever protection method you go with you'll eventually need to add more sata ports if you don't have enough available onboard. The ibm m1015 is a good choice along with the dell perc h210 or h300. They can all be flashed to LSI firmware which will be a very passive way of connecting additional drives to your windows install. Each connected drive will just be in a jbod config and allowed to set however you like in windows disk management, which is what you want for software RAID. Here's a guide for flashing the Dell cards https://techmattr.wo...ing-to-it-mode/ and the 1015 http://www.0x00.to/post/2013/04/07/Flash-IBM-ServeRAID-M1015-to-LSI9211-8i-with-UEFI-mainboard

I will certainly look at the information you have shared here. Thanks. Looks like some useful info.

 

I really like the NUC devices that Adrian has kindly shared. It is a shame that there is no room for a small PCI-E card then I could plug in my Music Production Soundcard and have an NUC in the living room and my studio with my current PC as the server and make the software and RAID changes which would make it a bit cheaper. I could lost the soundcard but it has got DAC's that are as good as Pro Tools so I can't It would introduce latency issues for me. 

 

Would you share your file and explain how to use it?  -_-



#22 Steverido OFFLINE  

Steverido

    Advanced Member

  • Members
  • 217 posts
  • Local time: 11:19 AM

Posted 30 April 2015 - 09:45 AM

After messing with mirroring and RAID for a year, I decided it was way too risky.  Storage is cheap enough now just to do full-on nightly backups.  Backups are also nice because you can easily retrieve accidentally deleted files.  I have mine set never to delete anything on backup, then when I feel like it, I manually do a backup with remove enabled.

How do you backup xnappo? Have you looked at Goodsync? It will save all amendments into its own folder just in case you wanted to reverse the last backup.  ;)



#23 dark_slayer OFFLINE  

dark_slayer

    Advanced Member

  • Developers
  • 695 posts
  • Local time: 11:19 AM

Posted 30 April 2015 - 10:40 AM

I will certainly look at the information you have shared here. Thanks. Looks like some useful info.

I really like the NUC devices that Adrian has kindly shared. It is a shame that there is no room for a small PCI-E card then I could plug in my Music Production Soundcard and have an NUC in the living room and my studio with my current PC as the server and make the software and RAID changes which would make it a bit cheaper. I could lost the soundcard but it has got DAC's that are as good as Pro Tools so I can't It would introduce latency issues for me.

Would you share your file and explain how to use it? -_-

Essentially if you already have a PC doing "server" duties it can easily become your dedicated server. Just move all the storage drives to it and create network shares in windows. I use open (no-password) shares on my LAN from w8.1 because I'm accessing them from android, Linux, and Windows. It's the easiest, and my router does what I'd call a good job of defending me against the outside world. Increasing complexity can improve security, but I like the minimally complex with good enough security approach better

For the software raid you have choices. There are a few that run over top of your windows file system like Snapraid, Flexraid RAID-F, and Flexraid tRAID. Only tRAID is real time, the other two create a snapshot at given points in time (scheduled) and update that snapshot per the schedule.

The three I mentioned use parity protection which allows a single drive to protect against the failure of all you're other drives with XoR calculations similar to the standard RAID4 configuration. Another like this is unRAID which is real time, but will not run on top of windows. It requires it's own OS to be running, so it's more like a NAS you set in the corner and R/W to on your LAN. Additional applications can be run on unraid, but they rely on the community to have created a plugin. Fortunately the community is very strong and has a lot of plugins already. There is also supposed to be an Emby plugin for unraid though I admit I've never used it or even entered that subforum. For your first time I think a windows server with either snapraid or Flexraid is the easiest because you keep all of your windows management familiarity and application support. Snapraid is free, but most consider Flexraid well worth the price (it has a free trial)

tRAID requires the least maintenance in my opinion, but all versions have some learning curve

There is no option where you don't have to learn anything other then buying a commercial NAS, even then (and with all solutions) it's worth knowing step by step what your plan is to recover data from a drive failure. Flexraid, snapraid, and unraid have guides and wikis available to walk you through data recovery

Edited by dark_slayer, 30 April 2015 - 10:41 AM.

  • Steverido likes this

#24 AdrianW OFFLINE  

AdrianW

    Advanced Member

  • Super Testers
  • 2493 posts
  • Local time: 07:19 PM
  • LocationPerth, Australia

Posted 30 April 2015 - 11:03 AM

Thanks for this Adrian, I was looking for something more compact and this looks just the job. I will review this and see what the specs are. :)  How does the remote fare with fast forward and rewind in both classic and PPC-HC? I found with MPC-HC jumps back to the start when I try to fast forward or jump. I saw a post on how to configure it but it didn't seem to change anything. If you have any information on this I would be grateful.

 

Of course you can have the server on your NUC or pC but the files located on your server elsewhere. Good plan.

 

FF/RW in MBC and MPC-HC and even plain vanilla WMC has never worked well (or at all) with any files other than standard wtv recording files. With MPC-HC I've setup three different skip amounts (IRC 10 secs, 30 secs and 1 minute) and have these mapped to FWD/RWD + Skip Fwd/Skip Bck + Left/Right. For jumping larger amounts I can always use the chapter listing.


  • Steverido likes this

#25 Steverido OFFLINE  

Steverido

    Advanced Member

  • Members
  • 217 posts
  • Local time: 11:19 AM

Posted 30 April 2015 - 02:31 PM

FF/RW in MBC and MPC-HC and even plain vanilla WMC has never worked well (or at all) with any files other than standard wtv recording files. With MPC-HC I've setup three different skip amounts (IRC 10 secs, 30 secs and 1 minute) and have these mapped to FWD/RWD + Skip Fwd/Skip Bck + Left/Right. For jumping larger amounts I can always use the chapter listing.

Do you have the MPC-HC software set to Global Media Keys? I can see that there are two other options. I am using a standard Windows Media Center Remote with Infra Red.



#26 Steverido OFFLINE  

Steverido

    Advanced Member

  • Members
  • 217 posts
  • Local time: 11:19 AM

Posted 30 April 2015 - 02:39 PM

Essentially if you already have a PC doing "server" duties it can easily become your dedicated server. Just move all the storage drives to it and create network shares in windows. I use open (no-password) shares on my LAN from w8.1 because I'm accessing them from android, Linux, and Windows. It's the easiest, and my router does what I'd call a good job of defending me against the outside world. Increasing complexity can improve security, but I like the minimally complex with good enough security approach better

For the software raid you have choices. There are a few that run over top of your windows file system like Snapraid, Flexraid RAID-F, and Flexraid tRAID. Only tRAID is real time, the other two create a snapshot at given points in time (scheduled) and update that snapshot per the schedule.

The three I mentioned use parity protection which allows a single drive to protect against the failure of all you're other drives with XoR calculations similar to the standard RAID4 configuration. Another like this is unRAID which is real time, but will not run on top of windows. It requires it's own OS to be running, so it's more like a NAS you set in the corner and R/W to on your LAN. Additional applications can be run on unraid, but they rely on the community to have created a plugin. Fortunately the community is very strong and has a lot of plugins already. There is also supposed to be an Emby plugin for unraid though I admit I've never used it or even entered that subforum. For your first time I think a windows server with either snapraid or Flexraid is the easiest because you keep all of your windows management familiarity and application support. Snapraid is free, but most consider Flexraid well worth the price (it has a free trial)

tRAID requires the least maintenance in my opinion, but all versions have some learning curve

There is no option where you don't have to learn anything other then buying a commercial NAS, even then (and with all solutions) it's worth knowing step by step what your plan is to recover data from a drive failure. Flexraid, snapraid, and unraid have guides and wikis available to walk you through data recovery

I agree with you, it is always a learning curve but well worth the trouble in the long term. I have looked at Flexraid already and it is quite interesting. If I would have had that setup or Snapraid then we would not have been having this conversation. Snapraid looks hard work though so I am going to look at Flexraid next. I have already decided not to buy an off the shelf NAS. I want control of it myself and be able to upgrade should I need. I noticed with Snapraid that you could add drives to it also as you fill them up which is great.



#27 xnappo OFFLINE  

xnappo

    Advanced Member

  • Developers
  • 4755 posts
  • Local time: 06:19 AM
  • LocationAustin, TX

Posted 01 May 2015 - 09:24 AM

How do you backup xnappo? Have you looked at Goodsync? It will save all amendments into its own folder just in case you wanted to reverse the last backup.  ;)

Using ancient EZBack-it-up .  Ten year old software, back fast and reliable.  I will look at GoodSync though - it would be nice to be able to set a 30 day expire on deletes or something like that.



#28 JeremyFr79 OFFLINE  

JeremyFr79

    Advanced Member

  • Members
  • 925 posts
  • Local time: 04:19 AM
  • LocationSeattle, WA

Posted 01 May 2015 - 10:59 AM

Ok, so my setup is probably a bit more than what most people have but here goes.

 

I have 2 dedicated file servers both running on actual server hardware (read NOT desktop parts) each has a 6.4TB hardware RAID 5, plus I have an addiontal 13TB split into 2 RAID 5's in a Fibre Channel Enclosure, I use crashplan for offsite backup, and I use Folder redirection with offline files for domain users.  I've had several drive failures and have been able to recover just fine but with any raid there's always the risk of a an absolute worst case scenario failure.  I don't backup my movies and tv do not backup offsite they are only protected by the RAID5 Arrays as I don't feel like uploading 20+TB to the internet and can be replaced easily enough (at least in my eyes), All the crucial stuff i.e. documents photo's etc go through Crashplan, it monitors real time for file changes, is fairly inexpensive (6 bux a month 1 pc unlimited storage) and fast.  I can access my backup from any device if needed which is nice to.  All of my servers do a nightly bare metal backup that also get's uploaded to crashplan as well.  All in all I have 5 servers for the house.

 

P.S. all my music uploads real time to Google Music for backup.


Edited by JeremyFr79, 01 May 2015 - 11:06 AM.


#29 shaefurr OFFLINE  

shaefurr

    Freelance Photoshop Designer

  • Super Testers
  • 2205 posts
  • Local time: 05:19 AM
  • LocationNew Mexico

Posted 01 May 2015 - 05:54 PM

I also lost a hard drive recently, which contained basically everything I added from the past year. I have the older flexraid Raid-F, but my parity drive died a while back and I didn't replace it and paid the price. Needless to say I'll be keeping my raid up to date and my parity drive working from now on. But I find flexraid to be a good choice, never had any problems with it at all.

 

I haven;t tried the TRAID yet but ill probably purchase that soon to replace my older Raid-F



#30 Steverido OFFLINE  

Steverido

    Advanced Member

  • Members
  • 217 posts
  • Local time: 11:19 AM

Posted 02 May 2015 - 06:43 AM

Ok, so my setup is probably a bit more than what most people have but here goes.

 

I have 2 dedicated file servers both running on actual server hardware (read NOT desktop parts) each has a 6.4TB hardware RAID 5, plus I have an addiontal 13TB split into 2 RAID 5's in a Fibre Channel Enclosure, I use crashplan for offsite backup, and I use Folder redirection with offline files for domain users.  I've had several drive failures and have been able to recover just fine but with any raid there's always the risk of a an absolute worst case scenario failure.  I don't backup my movies and tv do not backup offsite they are only protected by the RAID5 Arrays as I don't feel like uploading 20+TB to the internet and can be replaced easily enough (at least in my eyes), All the crucial stuff i.e. documents photo's etc go through Crashplan, it monitors real time for file changes, is fairly inexpensive (6 bux a month 1 pc unlimited storage) and fast.  I can access my backup from any device if needed which is nice to.  All of my servers do a nightly bare metal backup that also get's uploaded to crashplan as well.  All in all I have 5 servers for the house.

 

P.S. all my music uploads real time to Google Music for backup.

I looked at Crashplan and Backblaze and there is an interesting article where they compared the two and Crashplan came out on top just because of the amount of features.

 

The only problem for me is that I would prefer the storage company were in the UK and my research so far has come up with zilch. All of the great backup companies or in the US.  :(

 

Current research also seem to favor RAID 6 to RAID 5. I am not sure if I will go with RAID again. I think Flexiraid is the most likely.



#31 Steverido OFFLINE  

Steverido

    Advanced Member

  • Members
  • 217 posts
  • Local time: 11:19 AM

Posted 02 May 2015 - 06:49 AM

Using ancient EZBack-it-up .  Ten year old software, back fast and reliable.  I will look at GoodSync though - it would be nice to be able to set a 30 day expire on deletes or something like that.

I don't think it can do that. I believe that it only saves what has been deleted in the last sync within the gssync folder which is hidden in Windows unless you have Windows set to see hidden folders.

 

You might be able to request it as a future feature though  ;)



#32 Steverido OFFLINE  

Steverido

    Advanced Member

  • Members
  • 217 posts
  • Local time: 11:19 AM

Posted 02 May 2015 - 06:57 AM

Essentially if you already have a PC doing "server" duties it can easily become your dedicated server. Just move all the storage drives to it and create network shares in windows. I use open (no-password) shares on my LAN from w8.1 because I'm accessing them from android, Linux, and Windows. It's the easiest, and my router does what I'd call a good job of defending me against the outside world. Increasing complexity can improve security, but I like the minimally complex with good enough security approach better

For the software raid you have choices. There are a few that run over top of your windows file system like Snapraid, Flexraid RAID-F, and Flexraid tRAID. Only tRAID is real time, the other two create a snapshot at given points in time (scheduled) and update that snapshot per the schedule.

The three I mentioned use parity protection which allows a single drive to protect against the failure of all you're other drives with XoR calculations similar to the standard RAID4 configuration. Another like this is unRAID which is real time, but will not run on top of windows. It requires it's own OS to be running, so it's more like a NAS you set in the corner and R/W to on your LAN. Additional applications can be run on unraid, but they rely on the community to have created a plugin. Fortunately the community is very strong and has a lot of plugins already. There is also supposed to be an Emby plugin for unraid though I admit I've never used it or even entered that subforum. For your first time I think a windows server with either snapraid or Flexraid is the easiest because you keep all of your windows management familiarity and application support. Snapraid is free, but most consider Flexraid well worth the price (it has a free trial)

tRAID requires the least maintenance in my opinion, but all versions have some learning curve

There is no option where you don't have to learn anything other then buying a commercial NAS, even then (and with all solutions) it's worth knowing step by step what your plan is to recover data from a drive failure. Flexraid, snapraid, and unraid have guides and wikis available to walk you through data recovery

I was reading the old post about fRAID & tRAID and am I correct in understanding that tRAID is better if you are changing the data frequently? I saw others in the old thread saying that they had not switched to tRAID because they had film collections that infrequently changed.



#33 JeremyFr79 OFFLINE  

JeremyFr79

    Advanced Member

  • Members
  • 925 posts
  • Local time: 04:19 AM
  • LocationSeattle, WA

Posted 02 May 2015 - 06:28 PM

I looked at Crashplan and Backblaze and there is an interesting article where they compared the two and Crashplan came out on top just because of the amount of features.

 

The only problem for me is that I would prefer the storage company were in the UK and my research so far has come up with zilch. All of the great backup companies or in the US.  :(

 

Current research also seem to favor RAID 6 to RAID 5. I am not sure if I will go with RAID again. I think Flexiraid is the most likely.

The UK is a small enough area you really would be better off doing backup on another continent in case of major disaster to be honest.  Intercontinental pipes are big enough these day's doing a backup to the US shouldn't be any slower than local in the UK, only thing you'd see is a bit higher latency.  I love Crashplan been with them for years, as a matter of fact I work for one of the largest stock image/music/video websites in the world and we use Crashplan pro for personal backup's at work.  We actually have a dedicated Crashplan server onsite in the datacenter (where i actually work) that handles backups from people's PC's in the offices.  Pretty cool setup.



#34 Cerothen OFFLINE  

Cerothen

    Advanced Member

  • Members
  • 213 posts
  • Local time: 07:19 AM

Posted 05 May 2015 - 09:21 AM

I run a windows 8.1 server with 

 

Emby

WMC

ServerWMC

Emby ServerWMC plugin

uTorrent

SickRage

Flexraid tRAID

 

I also have a lot of steam/origin games installed on it, and I play them through Moonlight game streaming. I would like for all my clients to be android/fanless, but until the nvidia shield console (and maybe not even then) is released android doesn't support all my audio/video formats. I don't want to transcode at all on my lan, because it's a waste and I have three family members pretty frequently transcoding from my server at remote locations. I currently run kodi on an intel g1610 (openelec) htpc and a nexus player. The g1610 is play everything, but the nexus player (and the mk808 b+ that I'm trying out) don't do passthrough audio or all video formats.

 

I have been happy with Flexraid tRAID. The main site is down right now, but the setup guide is here http://wiki.flexraid...ck-setup-guide/ and there is also an older discussion thread on the difference between tRAID (newer) and RAID-F (old). http://emby.media/community/index.php?/topic/2167-flexraid-transparent-raid-traid-vs-raid-over-file-system-raid-f/

 

tRAID is realtime parity protection, which isn't backup but gives you the ability to protect against as many drive failures as you have parity disks. The parity disk has to be the largest disk in your array, but they can all be the same size. I have a mismatch of drives

  • 6TB parity (HGST)
  • 6x 4TB (Seagate, HGST)
  • 2x 3TB (Seagate, HGST)

I also have my C: drive (SSD) partitioned to use half for C: (125 GB) and the other half for a tRAID landing disk. With the landing disk, I can write to the pool faster than my scratch (download) disk can send the data. It goes up to about 150 MB/s writes, but I've written a file from the desktop (SSD) just to see it go to 300 ;)

 

Whichever protection method you go with you'll eventually need to add more sata ports if you don't have enough available onboard. The ibm m1015 is a good choice along with the dell perc h210 or h300. They can all be flashed to LSI firmware which will be a very passive way of connecting additional drives to your windows install. Each connected drive will just be in a jbod config and allowed to set however you like in windows disk management, which is what you want for software RAID. Here's a guide for flashing the Dell cards https://techmattr.wo...ing-to-it-mode/ and the 1015 http://www.0x00.to/post/2013/04/07/Flash-IBM-ServeRAID-M1015-to-LSI9211-8i-with-UEFI-mainboard

 

I was looking at Flexraid tRaid and haven't been able to form a comfort level with it yet. In my tests if I force a failure of a drive that had the data on it and the content was no longer accessible that was on that drive (based on 3 drives, 3 place holders and 2 parity drives) this test was done using Hyper V and eight 50GB virtual disks.

 

I couldn't find any information or reviews on tRaid that really provided me with enough information to trust my data with it. Any advise for testing it?



#35 WarrenH OFFLINE  

WarrenH

    Advanced Member

  • Members
  • 143 posts
  • Local time: 11:19 AM
  • LocationKent, UK

Posted 06 May 2015 - 10:47 AM

Your comments are interesting WarrenH. I have thought about a similar option where I could start up backup drives once a week and then shutdown again but it requires more manpower. I suppose if I were to build a PC just for backup then that would work but I really wanted to cut down on PC and especially the power. I use a great bit of software called Goodsync http://www.goodsync.com/ Once you have set up the folders (which is easy) you literally click analyse and sync and it does th job for you. You can either backup or sync and in either direction. It is amazing and worth the money. Updates are frequent too. Ideally I would like a physical backup at home and then one on the cloud. Once I have everything on the cloud keeping it backed up will be easy.

 

Although Backblaze is cheap it is in California and ideally I would like to find a company in the UK who do a similar deal but allow you to send the hard drive to them to backup rather than it taking 18 days over the web.

 

You say that you get 60 MB/S; is this a restriction of NAS? On my Windows Network I have up to 132 MB/s write speeds depending on the file size. If my memory serves me correctly then smaller files don't transfer at higher speeds do they? I now it is dependent on the size of the drive and at what speed it spins if your using mechanical.

 

Backing up to external drives is really slow though is it not?

 

Steve sorry the late response.

 

I use SmartSync Pro, does the same as Goodsync tho find it much simpler to use. I only do incremental synchronisation to my backup drives. I've over 12TB data, Cloud will be expensive and will probably take about 6 months to download and reinstate to a new system if I had to reinstate. Also, my internet provider will probably have a heart attack. My weekly 50GB backup will take a fair amount of cloud backup time and bandwidth. And - do I want to rely on someone else to look after my data, or an American company that the US government can just seize if they feel like it, especially considering the questionability over the legitimacy of ripped DVD's, music?

 

My 60mbps is what Windows shows me as the write speed to my QNap Nas - I'm using Cat 6 wired LAN so it's probably the speed of the QNap tho once again, adequate for my usage.

 

And finally, I use USB3 external HDD hub into which I slot my external backup HDD's. It's actually quite fast. A normal Sunday morning backup of around 50GB takes about 20 minutes.

 

Yes, I could automate all of this but if my house burnt down, got flooded, or thieves stole my equipment, I wouldn't have any duplicate backup copies so it's well worth the weekly 20 minutes intervention.


Edited by WarrenH, 06 May 2015 - 10:50 AM.


#36 dark_slayer OFFLINE  

dark_slayer

    Advanced Member

  • Developers
  • 695 posts
  • Local time: 11:19 AM

Posted 08 May 2015 - 12:23 AM

I was looking at Flexraid tRaid and haven't been able to form a comfort level with it yet. In my tests if I force a failure of a drive that had the data on it and the content was no longer accessible that was on that drive (based on 3 drives, 3 place holders and 2 parity drives) this test was done using Hyper V and eight 50GB virtual disks.

 

I couldn't find any information or reviews on tRaid that really provided me with enough information to trust my data with it. Any advise for testing it?

 

Sure thing. Why were you using place holders out of curiosity?

 

Looking through the wiki I can see how it's required for raid expansion in multi-ppu setups, but if you look through the forums you'll also see that recreating the array is sometimes recommended over any sort of raid expansion (especially if trying to swap a smaller drive for a larger drive since you have to do contraction then expansion). I also don't use the log raid from the wiki

 

If you can get comfortable running verify/sync to create your parity online (instead of waiting for the parity to create before using the array) then you will be less hesitant to re-create your configuration vs raid-expansion. Creating your parity online is a nice feature of t-raid IMO, though realize that the initial verify/sync (after initializing the array with the do-nothing option) will report [FAILURE] because it had to update your parity (which is what you want). You can run a normal online verify afterwards to see that it only fails verify/sync because blocks in the parity were updated

 

With raid-expansion out of the way, and hopefully me talking you out of placeholders, there are some useful ways to harness the flexibility of tRAID for restoration. I've used this without issue for test purposes, and I definitely recommend the same to anyone considering any sort of data protection method. What I used (which is recommended in the forums) to recover a dropped disk was the live data reconstruction which is enabled by default on new configurations

 

I tested it in a less sophisticated way by making use of some of my old smaller disks. I have 3x 2TB and a couple 1.5TB. I created the array with 3x ~20-30GB folders on a single 1.5TB and single 2TB drive with another 2TB drive as parity. To fail, I shutdown and pulled the 1.5TB then restarted. The array started with all 6 folders present (as if I hadn't pulled the disk at all). I got an email notification that a disk had dropped (remember to setup notifications) and I went to the web client to check things out. DRU01 (what I had named it) was shown as dropped, but all the data was present in the array (live data reconstruction). What's more interesting is that if you look at your windows disk manager after creating a tRAID array, each of your disks (DRU01, 02, etc . . . whatever you name them in tRAID) are presented twice. Once as the real physical disk, but again as a virtual disk. After the disk pull, the physical disk was missing but the virtual disk was still present (very cool).

 

What you can do in this situation instead of using the web client to restore, etc. is to simply give a drive letter to the virtual disk corresponding to the disk that tRAID has reported dropped. Then in windows explorer, copy the contents of that drive letter to your empty disk. To do this, I connected the extra 2TB drive, and rebooted once again. With the empty 2TB drive online and the virtual disk given a drive letter in windows explorer, I just initiated a copy/paste of all the contents to the empty drive. Then I deleted the tRAID config. I put the pulled data disk and new data disk online after a reboot and compared the data (exact copy)

 

From that I could tell it did it's job, and I could have re-created a raid config with the restored data. Since it was a test, I just moved onto creating my own array instead. It sounds a little bizarre, but once you setup a config and look through windows disk management you'll see what I mean. tRAID presents the virtual disk even after the physical disk is dropped by the array, which is good for as many disk failures as you have parity drives

 

Here is a screenshot to partly explain what I'm referring to with physical/virtual disks in windows disk management (DRU##, etc are virtual)

YllIKv8.jpg



#37 writersblock73 OFFLINE  

writersblock73

    Newbie

  • Members
  • 8 posts
  • Local time: 11:19 AM

Posted 09 May 2015 - 12:25 PM

My backups are pretty old-school.  I use external hard drives to save new encodes to, then unplug the drive and set it on a bookshelf.  Should I lose a production hard drive, I simply drag and drop from the appropriate external and let the files transfer overnight.  For all the more often I've faced having to replace a hard drive, it just didn't seem justifiable to build a server with automatic backup functionality.  I'm not Netflix.

 

It's not even that inconvenient should I lose a hard drive while the whole family's wanting to watch a movie.  The ONE occasion this has happened, we simply watched from the external where my back up was stored, and I used that same external to repopulate the replacement drive later.  Once done, back on the bookshelf went the external.

 

My encoding machine is not my server, and I've tried to keep my viewing devices as low-power as possible, so this system has yet to fail me.  Now that I'm thinking about it, Newegg's got some great prices on external USB 3.0's.  Might pay to back up my back ups!


  • xnappo likes this

#38 Cerothen OFFLINE  

Cerothen

    Advanced Member

  • Members
  • 213 posts
  • Local time: 07:19 AM

Posted 11 May 2015 - 11:24 AM

Sure thing. Why were you using place holders out of curiosity?

 

Looking through the wiki I can see how it's required for raid expansion in multi-ppu setups, but if you look through the forums you'll also see that recreating the array is sometimes recommended over any sort of raid expansion (especially if trying to swap a smaller drive for a larger drive since you have to do contraction then expansion). I also don't use the log raid from the wiki

 

If you can get comfortable running verify/sync to create your parity online (instead of waiting for the parity to create before using the array) then you will be less hesitant to re-create your configuration vs raid-expansion. Creating your parity online is a nice feature of t-raid IMO, though realize that the initial verify/sync (after initializing the array with the do-nothing option) will report [FAILURE] because it had to update your parity (which is what you want). You can run a normal online verify afterwards to see that it only fails verify/sync because blocks in the parity were updated

 

With raid-expansion out of the way, and hopefully me talking you out of placeholders, there are some useful ways to harness the flexibility of tRAID for restoration. I've used this without issue for test purposes, and I definitely recommend the same to anyone considering any sort of data protection method. What I used (which is recommended in the forums) to recover a dropped disk was the live data reconstruction which is enabled by default on new configurations

 

I tested it in a less sophisticated way by making use of some of my old smaller disks. I have 3x 2TB and a couple 1.5TB. I created the array with 3x ~20-30GB folders on a single 1.5TB and single 2TB drive with another 2TB drive as parity. To fail, I shutdown and pulled the 1.5TB then restarted. The array started with all 6 folders present (as if I hadn't pulled the disk at all). I got an email notification that a disk had dropped (remember to setup notifications) and I went to the web client to check things out. DRU01 (what I had named it) was shown as dropped, but all the data was present in the array (live data reconstruction). What's more interesting is that if you look at your windows disk manager after creating a tRAID array, each of your disks (DRU01, 02, etc . . . whatever you name them in tRAID) are presented twice. Once as the real physical disk, but again as a virtual disk. After the disk pull, the physical disk was missing but the virtual disk was still present (very cool).

 

What you can do in this situation instead of using the web client to restore, etc. is to simply give a drive letter to the virtual disk corresponding to the disk that tRAID has reported dropped. Then in windows explorer, copy the contents of that drive letter to your empty disk. To do this, I connected the extra 2TB drive, and rebooted once again. With the empty 2TB drive online and the virtual disk given a drive letter in windows explorer, I just initiated a copy/paste of all the contents to the empty drive. Then I deleted the tRAID config. I put the pulled data disk and new data disk online after a reboot and compared the data (exact copy)

 

From that I could tell it did it's job, and I could have re-created a raid config with the restored data. Since it was a test, I just moved onto creating my own array instead. It sounds a little bizarre, but once you setup a config and look through windows disk management you'll see what I mean. tRAID presents the virtual disk even after the physical disk is dropped by the array, which is good for as many disk failures as you have parity drives

 

Here is a screenshot to partly explain what I'm referring to with physical/virtual disks in windows disk management (DRU##, etc are virtual)

YllIKv8.jpg

 

Thanks for the input. I will rebuild a test environment later and see what I come up with. I had decided to use place holders since I had planned on having a reasonable amount of data using all the same disk size (about 50TB whenever I have all of them) and rebuilding the parity sounds awful since it would both take forever and it would leave the array vulnerable during a time of excessive disk use.

 

I also noted in your example that it sounds like you don't have drive pooling on is this the case?

 

Also in my tests I was trying things like building a small array of some virtual disks in a virtual machine then shutting down, removing one, and starting it back up. It did report the disk as missing and instructed me to mark it as failed. After I did that I tried to access files over the network (the VM had a dedicated NIC) and while I could see the files it would try to play but no image would appear and it would just make the occasional blip over the speakers. After that I tried to swap in another hard drive (again its virtual but the VM sees it as a real drive) but it just kept this loading bar going for hours rather than starting an actual task.

 

Most recently I tried it with actual disks (rather than virtual ones) and had basically the same issues.



#39 dark_slayer OFFLINE  

dark_slayer

    Advanced Member

  • Developers
  • 695 posts
  • Local time: 11:19 AM

Posted 11 May 2015 - 04:55 PM

I had pooling enabled

It sounds to me like you could be experiencing a quirk with placeholders unless you've tried it without them. I'm not sure what all use cases they are intended for and I don't see enough specifics in the wiki to feel comfortable with them personally.

#40 Steverido OFFLINE  

Steverido

    Advanced Member

  • Members
  • 217 posts
  • Local time: 11:19 AM

Posted 16 May 2015 - 10:24 AM

Sorted it all out now Guys.

 

I have decided that I am going to keep it really simple for a while and review in six months or so.

 

I have got a ridiculous amount of Redundancy built in to both PC's to ensure that I never have to go through the same again. I sent my Hard Drives back to Seagate and they informed me that the problem drive had a scratch on it. It was a mechanical issue with the Head failing and scratching it. However, they have informed me by email that they have recovered the data but I am waiting for them to confirm that it was all of it.

 

I can only recommend three backups minimum now if you want to avoid the damage it has cost me to recover from Seagate. Just over £1000. Ouch!!

 

My current setup is still the two PC's and the Media one I reckon that I might replace (when I move home) to one of the cool ultra Compact barebones PC like what Gigabyte or Asus produce. I can then have the Backups in the one PC.

 

My Main PC has the following in:

 

1  Samsung 240 GB SSD

1  6 TB WD Red

2  WD 5 TB Red

 

Media PC

 

500 GB Hybrid Drive

4 TB WD Red

3 TB Hitachi

 

I have an Orico Hard Drive Power Switch in each PC so that I can switch on/off the Backup Hard Drives when I like. http://www.amazon.co...r/dp/B005NVU1S2

 

I currently have five Backups plus whatever Seagate send back.  ;)  :P

 

To control the backup of this I use an amazing (mentioned before) piece of software called Goodsync. If you don't have one of the Backups with parity etc like Flexiraid or others mentioned, (and are creating your backups the old fashioned way) then this will serve you well.

 

See my screen capture as an example.

 

I make music and DJ Dance Music so you can imagine with Samples alone what it is like to try and keep a backup of all that without screwing it up and not to mention the time involved.  :o  :o  :o  :unsure:

 

So I am now looking to crack on with some production and not have to spend forever rebuilding or maintaining the PC's.

 

I just wanted to say thanks to all of you that shared your setups and kindly (spent a lot of time writing) gave all of the valuable and interesting advice which really did help me to make the choices that I have.

Perhaps at some point I will venture into the Flexraid T system but for now I don't want to have to learn something else.

 

Enjoy the weekend & thanks again.

 

Attached Files







Also tagged with one or more of these keywords: Home Theatre, Windows Media Centre, Samsung Smart TV

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users