Jump to content

Pooling drives with storage spaces


1971camaroguy

Recommended Posts

1971camaroguy

Had a question

 

I have my emby set-up on a dedicated computer running windows 10. Currently I have five 8 tb hard drives with various folders set-up on each on with emby mapped to folders on each of the drives.

 

I also have a similar computer set up with the same size and number of hard drives doing a 1:1 mirror with a program I use called goodsynch to prevent data loss.

 

Has anyone had any good luck with storage spaces on windows 10?  Right now I my media broken down by genre and type spread out across the drives. Some drives have more storage than others and I am basically wasting space. 

 

It would be a better allocation of resources to combine all of the drives into one storage pool, and would be easier to work with too. I figure I could make one big E: drive one each computer and let goodsynch manage backups.

 

I'm just really paranoid about something going wrong lol....but if I am able to do this it will make life easier with managing my media. If anything I could pool the main computer with emby and leave the backup on the other computer as individual drives. 

 

Just curious how others are managing their data storage, maybe there is another option that I am not aware of

 

 

Link to comment
Share on other sites

Gilgamesh_48

Although I have heard some good reports about storage spaces I have also heard several disaster stories where single drive failures caused loss of data as well as simple software fails where data was lost.

 

I use StableBit's Drive pool to pool over a dozen drives and I have never lost any data even when I have had a drive fail. I use the duplication feature to duplicate each file to a different drive so I am not at any risk in the event of a single drive failure.

 

It is well to note that duplication like DrivePool uses is not backup and if you actually delete a file both copies are gone.

 

I use Windows 10 but I do not trust Microsoft to correctly write software like pooling software.

 

DrivePool is not free but, like with many things, "Free is often worth exactly what you pay for it."

 

Edit: It should be noted that DrivePool stores everything in regular Windows readable format allowing data recovery even if DrivePool was to totally fail or the computer being used were to go belly up.

Edited by Gilgamesh_48
Link to comment
Share on other sites

1971camaroguy

Hi

 

I have 4 external hdd in a pool.

For me it is working great and i have no problems.

 

Thanks for the input, 

 

I have been brainstorming this a little and I think I am going to pool the drives on the main computer running emby and leave the drives on the backup computer alone for now. The only issue with doing it this way is the organization of the files on the backup will be more of a jumbled mess lol...I have to move the folders around where it will keep from running out of space on the drives that have filled up more.

Link to comment
Share on other sites

1971camaroguy

Although I have heard some good reports about storage spaces I have also heard several disaster stories where single drive failures caused loss of data as well as simple software fails where data was lost.

 

I use StableBit's Drive pool to pool over a dozen drives and I have never lost any data even when I have had a drive fail. I use the duplication feature to duplicate each file to a different drive so I am not at any risk in the event of a single drive failure.

 

It is well to note that duplication like DrivePool uses is not backup and if you actually delete a file both copies are gone.

 

I use Windows 10 but I do not trust Microsoft to correctly write software like pooling software.

 

DrivePool is not free but, like with many things, "Free is often worth exactly what you pay for it."

 

Thanks, I don't mind paying for good software, I used synchtoy (free) for a while but it was buggy and if a drive get's corrupt...it copied that corruption to your backup as well. Goodsynch was $49 for a lifetime subscription but well worth it.

 

I appreciate the heads up on Drivepool...I'm going to look into that

Link to comment
Share on other sites

1971camaroguy

Will drivepool backup across the network with drives on another computer? I am looking at that now

Link to comment
Share on other sites

1971camaroguy

Although I have heard some good reports about storage spaces I have also heard several disaster stories where single drive failures caused loss of data as well as simple software fails where data was lost.

 

I use StableBit's Drive pool to pool over a dozen drives and I have never lost any data even when I have had a drive fail. I use the duplication feature to duplicate each file to a different drive so I am not at any risk in the event of a single drive failure.

 

It is well to note that duplication like DrivePool uses is not backup and if you actually delete a file both copies are gone.

 

I use Windows 10 but I do not trust Microsoft to correctly write software like pooling software.

 

DrivePool is not free but, like with many things, "Free is often worth exactly what you pay for it."

 

Edit: It should be noted that DrivePool stores everything in regular Windows readable format allowing data recovery even if DrivePool was to totally fail or the computer being used were to go belly up.

 

 

If I am reading this right on the stablebit website.....I can add my drives to a pool I created and it doesn't affect the files on those discs? I don't have to erase and reformat my drives and start from scratch? 

 

5c3c942971611_add_to_pool.png

Link to comment
Share on other sites

Gilgamesh_48

If I am reading this right on the stablebit website.....I can add my drives to a pool I created and it doesn't affect the files on those discs? I don't have to erase and reformat my drives and start from scratch? 

 

5c3c942971611_add_to_pool.png

That is absolutely correct. You just plug a new drive in and call up DrivePool's dashboard and add the new drive to the pool. DrivePool then creates a new hidden directory with a long name like "Pool1xrt2vw..." which is used for files in the pool. Anything in that directory is in the pool and everything not in that directory is not in the pool. You can even save time getting files in the pool by directly copying files and directories from the drive into the hidden pool directory. Once they are in the pool DrivePool takes over and balancing and duplicating are handled in the background.

 

DrivePool never ever modifies your files. It just copies or duplicates them.

Link to comment
Share on other sites

Gilgamesh_48

Will drivepool backup across the network with drives on another computer? I am looking at that now

 

I "think" DrivePool can be made to do that BUT it is strongly recommended against. DrivePool is NOT backup software at all, it is pooling software.

Link to comment
Share on other sites

Fratopolis

I use a IBM ServeRaid M5015 on 8 3TB sas drives. It will also work with SATA as well if you are interested they are on eBay for about 40-50 bucks.

 

I set mine up as two sets of raid 5 and use Veeams free backup.

Link to comment
Share on other sites

jaspalgill

Hi, I have a couple of servers running the free Microsoft 2016 Hyper-V Server, which allows the creation of Storage Spaces.

 

I have a number of drives in pools, 4x4tb, 2x8tb, 4x2tb etc. - all with one drive redundancy.

 

Been like for a couple of years now with no data loss. Managing Server 2016 is a challenge....but the Storage Spaces have been fine. Just make sure you plan the sizes and number of disks carefully first.

 

Sent from my ONEPLUS A5000 using Tapatalk

Link to comment
Share on other sites

TylerV76

Another Stablebits recommendation here.

 

I had a computer at work using windows storage spaces and lost everything when one drive failed. Corrupted the entire pool and spent forever trying to recover the lost data.

 

I tried drivebender and wasnt a fan. Stablebits just worked out of the box and has been for a while now. When I moved my 6 emby drives into stablebits I didnt lose any data at all and duplication works perfectly.

 

The only issue Ive ever seen is watching an active dvr recording from a pooled drive doesnt play so nice. It reaches the point that is was recorded to and then stops. I had to move my dvr drive out of the pool for that reason.

 

 

Sent from my iPhone using Tapatalk

Link to comment
Share on other sites

Gilgamesh_48

Another Stablebits recommendation here.

 

I had a computer at work using windows storage spaces and lost everything when one drive failed. Corrupted the entire pool and spent forever trying to recover the lost data.

 

I tried drivebender and wasnt a fan. Stablebits just worked out of the box and has been for a while now. When I moved my 6 emby drives into stablebits I didnt lose any data at all and duplication works perfectly.

 

The only issue Ive ever seen is watching an active dvr recording from a pooled drive doesnt play so nice. It reaches the point that is was recorded to and then stops. I had to move my dvr drive out of the pool for that reason.

 

 

Sent from my iPhone using Tapatalk

 

Any pooled drive from any pooling provider should never be used for data that is extremely real time dynamic. Things like DVR drives, databases (Emby or Plex), receiving directories for bit-torrent files or any files that are updated and used in real time should never be within the pool, it is just unreliable and StableBit actually states that the Plex database will not work in the pool and I am quite sure that Emby's will not as well. You can use drives in the pool for that kind of thing but only the areas outside the pool directory.

 

Pooling should be for fairly stable data only as the data written into any pool is not completely stable during the writing process. This is particularly true for drives with some form of duplication turned on.

Link to comment
Share on other sites

TylerV76

Any pooled drive from any pooling provider should never be used for data that is extremely real time dynamic. Things like DVR drives, databases (Emby or Plex), receiving directories for bit-torrent files or any files that are updated and used in real time should never be within the pool, it is just unreliable and StableBit actually states that the Plex database will not work in the pool and I am quite sure that Emby's will not as well. You can use drives in the pool for that kind of thing but only the areas outside the pool directory.

 

Pooling should be for fairly stable data only as the data written into any pool is not completely stable during the writing process. This is particularly true for drives with some form of duplication turned on.

I only use the pool for storage at this point. All my cache, metadata etc is on a separate ssd and my DVR is on its own ssd. I tried it more so as a test knowing it was probably gonna be an issue.

 

 

Sent from my iPhone using Tapatalk

Link to comment
Share on other sites

ThePaladinTech

I'm using stablebit drive drivepool on 2 media servers... Mine running plex and emby. My brothers running plex. Works great! I tried storage spaces on my wmc server (win 8.1) and it seemed to cause issues, can't be sure but wmc would have difficulty playing recordings. Removing storage spaces seems to have resolved the issue.

 

Yes, you do not need to format your drives to use drivepool. It creates a new hidden folder on the drives in the pool to be the poooled drive. Both servers have had 12+ drives. Mine is down to six as i upgrade to 8tb drives

Link to comment
Share on other sites

ThePaladinTech

In my case the os, envy, and emby cache are on ssds. Emby data (movies, tv, and dvr recordings) are on the pool

Link to comment
Share on other sites

CBers

Any pooled drive from any pooling provider should never be used for data that is extremely real time dynamic. Things like DVR drives, databases (Emby or Plex), receiving directories for bit-torrent files or any files that are updated and used in real time should never be within the pool, it is just unreliable and StableBit actually states that the Plex database will not work in the pool and I am quite sure that Emby's will not as well. You can use drives in the pool for that kind of thing but only the areas outside the pool directory.

 

 

I have ben using DriveBender since it was first released in 2011 and I have never lost any data.

 

I have my DVR files written to the pool without any issues.

 

DriveBender has just released v3, which includes a built-in "firewall", which allows for only permitted applications to access files within the pool, plus other new options.

 

It's very stable and it's an install, configure and forget product.

  • Like 1
Link to comment
Share on other sites

VladTepz

+1 for Stablebits Drivepool

I've used all 3, plus a few Linux options. 

I started off with DriveBender, then moved to Storage Spaces after development of DriveBender stalled.
Fortunately DriveBender development has started back up recently and they just released a big update.
 

  • DriveBender is great, but has/had some issues for my use case. (Didn't support symlinks & weird issues with some software.)
     
  • Storage Spaces is decent unless you use their mirroring options.  ;)  I prefer not to use raid/fake raid as one drive failure can bring down everything. (depending on your setup)
     

I am now currently using Stablebits Drivepool. I've been using it for about a year & have 0 complaints/issues. In fact that it works so well, that 99% of the time I forget I even have it installed. (Until a thread like this pops up)
The same could not be said from my experiences with DriveBender or Storage Spaces... :(

Pair Stablebits Drivepool with the Stablebits Scanner, which has saved my pool from a failing hard drive, and it's a combo that's hard to beat. :)

Edited by VladTepz
  • Like 1
Link to comment
Share on other sites

Just wanted to add another +1 for DrivePool.

 

Been using it for years and over that time have lost 2 drives (on separate occasions so no data loss) and even pulled old 1TB drives out the pool to replace with 4Tb drives all whilst the server was still running!

 

It handles almost everything you can throw at it.

 

Only downside, taht's already been mentioned, is I was running a VirtualBox VM with the data being duplicated in real time and, whilst it coped, it was not happy about the constant file changes.

 

Also threw the Scanner in and it tells me when my drives are having a paddy!

Link to comment
Share on other sites

1971camaroguy

Ok everyone, I am trying out drivebender and it's scaring the piss out of me lol. I merged a almost full 8 TB drive I have called movie boxed sets into a F drive pool I created called Storage.

 

I ticked the box for the files to be merged into the pool....after the merge I am only seeing a portion of the size of the files even though the drive still shows full. Is there a background processes going on and I need to wait a bit for it to show up?

Link to comment
Share on other sites

1971camaroguy

I can go to the F drive and my box sets folder is there...when I hover over it and select properties it's showing 212 gb

 

When I go into that folder and highlight the genre folders within that and select properties I get 818 gb

 

But the drive itself still shows 7.95 TB in the drive bender interface

Link to comment
Share on other sites

CBers

@@1971camaroguy What version do you have installed, v3.1.0.0? Which OS, W10?

 

It's a while since I set up a pool in DriveBender, as had the same one for many years :)

 

The files should show up as soon as the initial merge tasks have completed.

 

What does Windows Explorer say is the size of your F: drive?

Link to comment
Share on other sites

1971camaroguy

It's the newest version on the website, yeah windows 10

 

I merged two 8tb  and a 3tb drives, one 8tb and the 3tb was almost full and the other 8tb had abouot 4.00 tb left

 

The F drive shows 17.2 TB and 4.65 free which is right where it should be

 

I can go down to the root of the folder where my movies are stored and it shows the full size...even though the folder those are dropped in shows significantly less

 

Here's what i did....on my main computer that runs emby I created a large pool and added most of my movie drives to it. Initially I messed up my boxed set drive and just reformatted it.

 

I was going to rely on my boxed set back up drive on my 2nd computer that had been merged on it's own with drivebender on a separate install 

 

My plan was to copy that entire folder across the network to the main computer missing the files...but I found out real quick that you can't see the virtual drive across the network

 

I was able to assign a drive to the drives on each computer and I am copying the files manually...so I think I have a handle on that now

 

Once I get my main computer drives pooled and all of the files where they need to be again I am going to start adding the other drives to the storage pool one at a time over the network. Forget doing a 1:1 backup and just pool all of my drives together and let drivebender handle the backup. 

 

Doing it this way I will increase my storage and ability to remove drives as needed 

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...