Not really hardware but

[mediacowboy 438]

I am thinking of switching my media server OS from Windows to Linux. My problem is I have 21.7 TB of media split between 6-8 tb hard drives pooled by drive bender. How hard would it be to make the switch?

[mastrmind11 717]

Yikes.  I think you're pretty much hosed if you're thinking it'd be a simple process.  From what I can tell after googling for a few minutes is that drive bender doesn't stripe or mirror the files across drives, it simply creates pointers to the files on each drive to simulate 1 giant drive.  This is good in that, if you're really wanting to get into doing this migration, you could technically manually remove the drives 1 by 1 and manually reconstruct the folder structure required for an EMby library.  This would obviously take quite a bit of time considering the size of your collection, but seems technically feasible. 

 

Here's a link about how to go about attempting this:  https://support.division-m.com/hc/en-us/articles/214025823-How-do-I-manually-remove-a-drive-from-the-pool-

 

However, keep in mind that if this does work, they'll all be separate mount points under linux.  I'm not certain, but most, if not all *nix flavored raid-like OS's are proprietary and will likely destroy your files when you initialize the array.  

[CBers 6766]

I am thinking of switching my media server OS from Windows to Linux.

 

Why?

 

 

 

My problem is I have 21.7 TB of media split between 6-8 tb hard drives pooled by drive bender. How hard would it be to make the switch?

 

As you know, you can take the drives out of DriveBender and assign them drive letters, so you would then need to make them available in Linux.

[mediacowboy 438]

 

 

Why?

Because it's windows 10 and it likes to do updates and reboots at the most inconvenient times.

[mediacowboy 438]

Yikes. I think you're pretty much hosed if you're thinking it'd be a simple process. From what I can tell after googling for a few minutes is that drive bender doesn't stripe or mirror the files across drives, it simply creates pointers to the files on each drive to simulate 1 giant drive. This is good in that, if you're really wanting to get into doing this migration, you could technically manually remove the drives 1 by 1 and manually reconstruct the folder structure required for an EMby library. This would obviously take quite a bit of time considering the size of your collection, but seems technically feasible.

 

Here's a link about how to go about attempting this: https://support.division-m.com/hc/en-us/articles/214025823-How-do-I-manually-remove-a-drive-from-the-pool-

 

However, keep in mind that if this does work, they'll all be separate mount points under linux. I'm not certain, but most, if not all *nix flavored raid-like OS's are proprietary and will likely destroy your files when you initialize the array.

That is my fear. Maybe in a few years when I build new I'll start off on Linux and move everything over.

[CBers 6766]

That is my fear. Maybe in a few years when I build new I'll start off on Linux and move everything over.

 

Maybe Anthony will have DriveBender running on Linux by then

[Tur0k 143]

Might be able to leverage cloud services ex:

1. Upload to a cloud backup tool (backblaze or carbonite).

2. Use an imaging tool like acronis to copy the system and application drives in the event you want to revert.

3. Rebuild system on Linux.

4. Then move it back to your rebuilt system.

 

 

Sent from my iPhone using Tapatalk

[Jdiesel 1113]

Might be able to leverage cloud services ex:

1. Upload to a cloud backup tool (backblaze or carbonite).

2. Use an imaging tool like acronis to copy the system and application drives in the event you want to revert.

3. Rebuild system on Linux.

4. Then move it back to your rebuilt system.

 

 

Sent from my iPhone using Tapatalk

 

I did something similar a year or so back. Granted I only had 16TB of data to move and my upload bandwidth was somewhat decent. Still took over two weeks non-stop if I remember correctly.

[mediacowboy 438]

I'm just not all that familiar with Linux and would want the pooling solution.

[mastrmind11 717]

I'm just not all that familiar with Linux and would want the pooling solution.

You can technically create a drive pool in Lunix using an LVM partition, but as pointed out earlier, you'd need your media backed up somewhere or risk losing it.

[mediacowboy 438]

Backup isn't an issue as I backup to a USB drive and then to a server off site

[CBers 6766]

You can create a Volume Group with as many physical drives you want in it, then create virtual drives underneath.

 

If you had a spare one or two, create the VG, create the volumes, mount them and them migrate disks across one at a time, repeating until you're done. You can always add more disks and extend LVs to accommodate the new disk space.

 

With separate mountpoints, you could have a /movies, /tv, /livetv etc.

[mediacowboy 438]

Well I did something extremely stupid today. IT's stupid because I am not a Linux pro by any means. I remembered that I still had all the old drives in my gaming rig that I used for emby when it was in that system. Then I found virtual box on my gaming system and the light bulb clicked. Instead of messing with production to learn all the bugs and hurdles I would face I spun up a Ubuntu 16.04.4 LTS vm and started to play. I am not anywhere close to switch this over to production but I now have at least a dashboard for all my containers. I installed sabnzbd, sonarr, radarr, and watchtower to watch for docker changes. They man plan now is to duplicate everything from my windows 10 machine to Ubuntu by using dockers. Play with it in the test environment. Then one weekend when I have time make a backup of the windows machine, clean the 8 tb drive's and start over with Ubuntu as the host and rebuild. 

 

As far as the hard drives go I am going to use LVM for volume groups. Instead of multiple mount points I will just have one big one so that I don't have to mess with it all that much. 

 

Well my brain hurts from wrapping my head around all this new information but I am off and running. If anyone is interested in this I will keep posting otherwise thanks everyone for your help.

[CBers 6766]

Well my brain hurts from wrapping my head around all this new information but I am off and running.

[mastrmind11 717]

nice.  well done.

[SkyBehind 23]

I'll have to look up drive bender to really comment on it, not it sounds like what you want is a drive pool. MergerFS sounds pretty much like what you want. A bunch of driver or together to mimic one large drive.

 

It does more than this though, it can balence data, write to drive with most free space first, etc etc.

 

For a little disaster mitigation, you can pair it with snapraid. It's a parity drive where you can rebuild one of the drives in your pool if need be.

 

Keep in mind you aren't tired into anything with either of these. I can yank a drive out now, throw it in another box, and access all the files that are on it.

 

If you're interested in some help, get at me and I'll help you walk through it. Or put another way, I'll help you at the Linux light, lol.

 

Oh, on a side note, while your doing all that, stuff it in docker containers, it's pretty easy, I can help with that as well. Offer is open :-)

 

Sent from my Nexus 6 using Tapatalk

[mediacowboy 438]

I'll have to look up drive bender to really comment on it, not it sounds like what you want is a drive pool. MergerFS sounds pretty much like what you want. A bunch of driver or together to mimic one large drive.

 

It does more than this though, it can balence data, write to drive with most free space first, etc etc.

 

For a little disaster mitigation, you can pair it with snapraid. It's a parity drive where you can rebuild one of the drives in your pool if need be.

 

Keep in mind you aren't tired into anything with either of these. I can yank a drive out now, throw it in another box, and access all the files that are on it.

 

If you're interested in some help, get at me and I'll help you walk through it. Or put another way, I'll help you at the Linux light, lol.

 

Oh, on a side note, while your doing all that, stuff it in docker containers, it's pretty easy, I can help with that as well. Offer is open :-)

 

Sent from my Nexus 6 using Tapatalk

 

I will defiantly have to take you up on help. 

 

I have been reading up on MergeFS and it does fit the bill. I do want disaster mitigation locally but don't want to have to give up one of my hotswap bays for a parity drive. 

 

The only issue I am having right now with docker is getting the linuxserver/letsencrypt to work. but that could be because my isp blocks 80 and 443.

[Diedrich 355]

Because it's windows 10 and it likes to do updates and reboots at the most inconvenient times.

You need to go into your settings. Mine never reboots unless I tell it to.

[SkyBehind 23]

I will defiantly have to take you up on help. 

 

I have been reading up on MergeFS and it does fit the bill. I do want disaster mitigation locally but don't want to have to give up one of my hotswap bays for a parity drive. 

 

The only issue I am having right now with docker is getting the linuxserver/letsencrypt to work. but that could be because my isp blocks 80 and 443.

 

So I guess you could use a usb drive for parity, it doesn't run constantly, it's only a snapshot, whenever you kick off the script/program.  

 

Where are you located, you're positive ports 80 and 443 are blocked?  Have you done a port forward on your router to forward ports 80 and 443 to your server?

[mediacowboy 438]

You need to go into your settings. Mine never reboots unless I tell it to.

Mine normally doesn't but when it does reboot, power glitch or i tell it to it doesn't come back up right. Plus I want a more stable system.

[mediacowboy 438]

So I guess you could use a usb drive for parity, it doesn't run constantly, it's only a snapshot, whenever you kick off the script/program.  

 

Where are you located, you're positive ports 80 and 443 are blocked?  Have you done a port forward on your router to forward ports 80 and 443 to your server?

In Texas. My ISP sucks but they are the only ones with the speed.

[SkyBehind 23]

In Texas. My ISP sucks but they are the only ones with the speed.

OK, so just making sure you port forwarded port 80 on your router.  If so, then the next thing is:

 

Are you wanting to access your server via a domain name you purchased?  You could set up a reverse proxy on a VPS.  If not, you can do something like a port redirect feature like this

[SkyBehind 23]

I can't believe I didn't think about linking this before, but here's some pretty easy instructions for setting up a media server with linux and docker

https://www.smarthomebeginner.com/docker-home-media-server-2018-basic/#Install_Docker_and_Docker_Compose

 

 

Here's basic instructions for Snapraid

https://zackreed.me/setting-up-snapraid-on-ubuntu/

 

 

Here's basic instructions for MergerFS

https://zackreed.me/mergerfs-another-good-option-to-pool-your-snapraid-disks/

[SkyBehind 23]

I have a question, how do you know port 80 is blocked.  I'm curious, because I've read about people's ISP's blocking ports, but I've never actually run into it.

 

 

 

EDIT: What's the result of a test like this http://canyouseeme.org/

Edited April 20, 2018 by SkyBehind

[mediacowboy 438]

I have a question, how do you know port 80 is blocked.  I'm curious, because I've read about people's ISP's blocking ports, but I've never actually run into it.

 

 

 

EDIT: What's the result of a test like this http://canyouseeme.org/

"The Internet Service is provided for personal and family use within a single residential household. You agree that you will not use, nor allow others to use, the Internet Service to operate any type of business or commercial enterprise or to use the Internet Service as an Internet service provider. You may set up one (1) Web page per e-mail account for personal use using the Internet Service, but you may not establish a web page using a server located at your home. You agree that you will not use, nor allow others to use, your home computer as a Web Server, FTP Server, file server or game server or to run any other server applications. Customer will not resell or redistribute, or allow others to resell or redistribute, access to the Internet Service in any manner, including by wireless means."

That is directly from my isp