Jump to content

Storage server upgrade


sooty234
Go to solution Solved by sooty234,

Recommended Posts

sooty234

My present chassis is only 16 bay, and it's filled. I'm looking to upgrade to this Supermicro Coraid.

https://www.theserverstore.com/Supermicro-Coraid-36-Bay-4U-SRX6300-W-X9DRH-iF_p_901.html

Anyone got experience with this or something similar? I see that the fans are controlled by the motherboard. Are these things loud? I'm expecting them to be, and if so, I'll replace them. Are the PSU fans also loud? Supermicro are known to be built well, which is what's attracting me to it. I also like the look of this thing. I could just add drives to my server box, but I wanted to keep the storage separate. Part of the decision is the rails for the Norcotek boxes I have are rubbish, and I would like to move away from the Norcotek stuff.

  • Like 1
Link to comment
Share on other sites

PenkethBoy

i have HP Servers and they are loud!

as its enterprise grade kit - its likely to be very loud - at startup especially - may calm down at bit once booted

replacing the fans is likely to be a challenge as they will not be standard fans and have none standard connectors - as in not the connectors you are used to with domestic fans for normal pc's - removing a fan or two when running is likely to cause it to run the fans flat out - as it will see it as a failure and default into running remaining fans flat out to protect the server even if the temps are fine. I experimented with some quieter fans - involved a soldering iron and wire strippers was successful - but a major PIA - in the end i bought a Li-Lian case which they dont make anymore that allows me to easily put 30+ drives in and takes normal fans etc etc

You might be able to replace the fans with better options but the management chip may not accept them as they may have additional sensors that it needs to see or some id it understands to recognise them etc

PSu fans might be noisy but i think any noise they make will be drowned out by the main fans

@CBers and @Spaceboy have super micro servers - IIRC they are off as they make too much noise!

 

If you have a separate room it can live in and you can shut the door etc - then you may be ok

  • Like 1
Link to comment
Share on other sites

sooty234

I think I can get around the fan connector issue. I was planning to replace them all, and use my fan controller. Unless you're saying there is some circuitry that connects the fans directly to the backplane and hard drive censors. I can probably modify that, if that's the case. But looking at the wiring, I don't think that's what's happening (and I'm referring to my second link). I won't be using a server mobo now, as I no longer need all of those expansion slots. So, regular fans to a controller. I think that will work. Or am I missing something?

s-l1600.thumb.jpg.f82d52be80b443ae14d2334a9a1822d3.jpg

Link to comment
Share on other sites

sooty234

Yeah, I think they're just regular connectors and fans, directly wired. So it shouldn't be an issue. Might even connect them to the fan controller and see how loud they are. This could be fun :) Now I just need to decide if I want to part with the money :D 

1765472043_s-l1600(2).jpg.84eedda56227ea9fc3ca30d97a0906a5.jpg

Link to comment
Share on other sites

PenkethBoy

ok without the server m/b then you should be ok assuming those fans have normal connectors - image looks like it

think you will end up removing that fan wall as i bet those fans are non standard sizes

let us know how you get on

Link to comment
Share on other sites

CBers
2 hours ago, PenkethBoy said:

 

@CBers and @Spaceboy have super micro servers - IIRC they are off as they make too much noise

My SuperMicro is off as I don't currently have a use for it. 

It is loud on boot, as are all enterprise servers, but it does quieten down after that. 

 

 

  • Like 1
Link to comment
Share on other sites

Spaceboy

yeah i bought a 2U one as a £100 experiment. took it for re-cycling a couple of weeks ago. far too loud for me to deal with. i did try to replace the fans but i was ending up throwing more money after bad and never going to achieve anything useful with it

  • Like 1
Link to comment
Share on other sites

sooty234
7 hours ago, PenkethBoy said:

ok without the server m/b then you should be ok assuming those fans have normal connectors - image looks like it

think you will end up removing that fan wall as i bet those fans are non standard sizes

let us know how you get on

The fan wall looks to be riveted in place, but each fan has it's own housing that is latched in place. The only issue I can see with fan replacement is the finger groove that gives access to the latch. A replacement fan may not be as thick, so that groove might butt up to fan frame. I guess I could cut out the groove with a dremel. It's a curious design. The sides of the case are vented, so there won't be much negative pressure to pull air through the drives. That likely has to do with the drive bays in the bottom 2U at the back. 

I think I'll ruminate on it. With shipping it's gonna cost $700.

Link to comment
Share on other sites

vdatanet
9 hours ago, sooty234 said:

I have better tools than this guys

But they have an Apple TV!!! 😄

2055111187_Capturadepantalla2020-11-16102305.thumb.jpg.b412c74bebddde5a211ee392c609647a.jpg

  • Haha 1
Link to comment
Share on other sites

  • 2 weeks later...
sooty234

Well, it arrived....but they didn't send the rails I ordered and they didn't include the SAS cables (as advertised). I wanted to get it on the rack over thanksgiving, but now that won't happen 😕

Link to comment
Share on other sites

sooty234

They are shipping the rails and cables out. A little something I discovered. There are 3, what look to be new, 3TB SAS drives tucked in the back of this thing. Nice! They'll be added to one of the pools. So with the new drive I bought, I get a nice bump of around 18TB. Silver linings :)

Link to comment
Share on other sites

sooty234

While I wait for the rails and SAS cables to arrive, I prepared the fans. It was very easy to do. You just need to cut a slot for the back of the replacement fans. Then you can connect them directly to the backplane. These fans, even though there are more of them, could possibly be more quiet than my existing Noctua 120mm fans. The drive cages in this thing are much more ventilated than my Norcotek. It uses caddies and not full drive trays. The frame has big cutouts between the drives, and the front bezels are really well perforated.

                               PXL_20201127_220537668.jpg.9178f10f3d53782a6c1040f869472f11.jpg            PXL_20201127_220453443.jpg.3650a23d867044a886a933dc39457e38.jpg                    

 

PXL_20201127_222232565.jpg.532c420aad8cb3ea2cf4decd998e1e2c.jpg

Link to comment
Share on other sites

sooty234

After trying a few difference fan configurations, I'm coming the conclusion that the backplanes simply won't allow much air flow through the drives to keep them as cool as I would like. There are cutouts in the backplane, and I can feel air being pulled through, but the drives still get to the mid 40s with my industrial Noctua fans mounted directly to the backplane frame and running at max (3000rpm). In summer, I think they might fry. The Supermicro backplane is vertical, while in the Norcotek case there are four separate backplanes and they are horizontal, allowing more air to flow. I would consider this a design flaw in the Supermicro. You'd have to ensure that the air being pulled in through the front is always cool, to maintain an optimal drive temperature. I think for now, I'll have to use the Norcotek and maybe one day I'll stuff the Supermicro in cool closet and turn it into a big NAS.

Edited by sooty234
Link to comment
Share on other sites

mastrmind11
3 hours ago, sooty234 said:

I'm feeling like a storinator is in my future. I'd forgotten about those....damn it! It's perfect for what I want. Lots of storage, and quiet. A little expensive, but I think in the end it'll be worth it.

https://www.45drives.com/products/storage/

holy crap, whats the power draw on something like that?

Link to comment
Share on other sites

sooty234
25 minutes ago, mastrmind11 said:

holy crap, whats the power draw on something like that?

Ha! Not that bad, apparently. It comes with an 850 watt PSU. Which is what I am already using, but obviously not as many drives as that can hold. Resting power of a hard drive (when it's not performing read write) isn't that high, just a few watts. I mean I've already got 25+ drives, which I'd like to consolidate in one case. So this is the ideal solution.

Link to comment
Share on other sites

mastrmind11
7 minutes ago, sooty234 said:

Ha! Not that bad, apparently. It comes with an 850 watt PSU. Which is what I am already using, but obviously not as many drives as that can hold. Resting power of a hard drive (when it's not performing read write) isn't that high, just a few watts. I mean I've already got 25+ drives, which I'd like to consolidate in one case. So this is the ideal solution.

very nice.  i have 10 but i'm pretty conservative with my hoarding.  cheers

Link to comment
Share on other sites

sooty234

Gentlemen! Today I had a fantastic idea (It's my first, don't judge).

As I mentioned previously, the only issue I had with the Supermicro was getting enough air through the drives to cool them. So.... I am going to make a quickly attachable/detachable front mounted fan wall with one of the front Norco bezels that I don't use.

Behold! (Just a mockup)

1229960006_PXL_20201202_0220596713.jpg.0a054c75ca8863fae5a7be2bf5c818b0.jpg

PXL_20201202_020630635.jpg.4ae5020fc6694b185c640ce2754347da.jpg

 

It even has a dust filter :)

Combined with the fans behind the backplane, I'll have a push pull thingy, and hopefully provide enough air flow :) 

 

 

  • Like 2
Link to comment
Share on other sites

On 29/11/2020 at 19:52, sooty234 said:

After trying a few difference fan configurations, I'm coming the conclusion that the backplanes simply won't allow much air flow through the drives to keep them as cool as I would like. There are cutouts in the backplane, and I can feel air being pulled through, but the drives still get to the mid 40s with my industrial Noctua fans mounted directly to the backplane frame and running at max (3000rpm). In summer, I think they might fry. The Supermicro backplane is vertical, while in the Norcotek case there are four separate backplanes and they are horizontal, allowing more air to flow. I would consider this a design flaw in the Supermicro. You'd have to ensure that the air being pulled in through the front is always cool, to maintain an optimal drive temperature. I think for now, I'll have to use the Norcotek and maybe one day I'll stuff the Supermicro in cool closet and turn it into a big NAS.

Wish I had seen this post sooner so I could have saved you some troubles.

I have a Supermicro SC847 36 bay chassis just like the one in your picture. It came with 7x FAN-0126L4 "screamers" as I call them that sound like a jet taking off when they spin up. 

I first replaced them with the Supermicro SQ fans, FAN-0074L4. Honestly, I could barely notice a difference in sound. They were still loud as hell. 

After I replaced all the fans with another 80x80 (I can't remember which ones now). Within 24hrs I started getting bombarded with overheating warnings from my drives as well as the rear CPU. There just wasn't enough static pressure being generated by these fans to pull the air through the drives. 

Finally, I put in 5x Noctua NF-F12 iPPC 3000pwm 120mm fans. I fit 2 in the front row (where there were 3 fans), and 3 in the back row (where there were 4). These provide enough static pressure to keep things cool, and they're relatively quiet. I wanted to mount them on a Noctua fan wall, I had ordered some, but they never arrived. I'm embarassed to say, but I actually zip-tied them to each other and then mounted them in place with another one. These have done the job at quieting things down and keeping the drives cool, but I had to replace the passive cpu coolers to active ones because that rear cpu was still sitting higher than I liked in temps.

Looking at the fans you chose, they look to be the 1700pwm version, I can almost guarantee you're going to run into heating issues when under a heavy load. I'd spend the extra $5 or so per fan to bump up to the industrial version for the higher pressure and more CFM.

Link to comment
Share on other sites

sooty234
21 hours ago, MRobi said:

Wish I had seen this post sooner so I could have saved you some troubles.

I have a Supermicro SC847 36 bay chassis just like the one in your picture. It came with 7x FAN-0126L4 "screamers" as I call them that sound like a jet taking off when they spin up. 

I first replaced them with the Supermicro SQ fans, FAN-0074L4. Honestly, I could barely notice a difference in sound. They were still loud as hell. 

After I replaced all the fans with another 80x80 (I can't remember which ones now). Within 24hrs I started getting bombarded with overheating warnings from my drives as well as the rear CPU. There just wasn't enough static pressure being generated by these fans to pull the air through the drives. 

Finally, I put in 5x Noctua NF-F12 iPPC 3000pwm 120mm fans. I fit 2 in the front row (where there were 3 fans), and 3 in the back row (where there were 4). These provide enough static pressure to keep things cool, and they're relatively quiet. I wanted to mount them on a Noctua fan wall, I had ordered some, but they never arrived. I'm embarassed to say, but I actually zip-tied them to each other and then mounted them in place with another one. These have done the job at quieting things down and keeping the drives cool, but I had to replace the passive cpu coolers to active ones because that rear cpu was still sitting higher than I liked in temps.

Looking at the fans you chose, they look to be the 1700pwm version, I can almost guarantee you're going to run into heating issues when under a heavy load. I'd spend the extra $5 or so per fan to bump up to the industrial version for the higher pressure and more CFM.

The fans you see in the picture are not the fans I have inside the chassis. Those are the same fans you've mentioned. Those fans are what I've been using for several years and are in my current chassis. They still can't pull as much air as I want. As I described, it's because of the backplane. The fans you see in the picture are new fans I had laying around. I'm going to mount them in front of the drives, outside of the chassis, to push air into the drives. With the industrial 3000rpm fans pulling air from behind. The chassis will be at the bottom of the rack, so I can mount the fans to the rack and be able to easily remove it. I'll probably clamp the fan frame/wall (haven't fully figured that out yet). I plan to take pictures of everything, so everyone can see what I did. One of problems is the air vents at the sides. Some of the air gets pulled through those, so you lose pressure. But that air needs to be pulled in for the back drives, and PSU. So getting air directly into the front drives is the only answer, other than keeping this thing in a fridge.

Link to comment
Share on other sites

  • 4 weeks later...
  • Solution
sooty234

OK, project finally finished. And the results are good.

I'll start by saying that the system always worked, but the noise of the main fans and what I later discovered, the PSU fan, was intolerable. So the goal was to reduce noise but maintain proper cooling.

The first thing that needed to be done was remove the existing fan wall(s) and all of the fans. A little fiddly, but not difficult. The Supermicro case (847) has rear drives, so there are two levels inside the case. The top one is where the motherboard sits, and it slides back to allow access below it. Air obviously needs to pass through the rear drive bays, too. But the air is pulled through the front and pushed out the back. There is plenty of space to work with, so I mounted two 80mm fans to a piece of aluminum angle, and screwed that to the available mounting posts on the bottom of the chassis. This will direct air flow to the back drives. The top of the fans align with the edge of the motherboard tray.

 

PXL_20210101_205224693(1).thumb.jpg.06a93e898fca88385e253caba8335331.jpg

 

Next, we need fans to pull air through the front drives. I am using Noctua NF-F12 iPPC 3000 PWM fans, three of them. And I mounted them to a Norco fan wall I was previously using. The height dimension of the fan wall is perfect, but the length is ~10mm shorter than the inside dimension of the Supermicro chassis. So I just lined it with some foam weather stripping. The chassis has side ventilation where the fan wall sits. So I used those to screw the fan wall in place. I drilled and tapped the fan wall to screw into.

PXL_20210101_205251292.thumb.jpg.8427ace13959811c3a71725627dcd173.jpg

 

PXL_20210101_205224693(2).thumb.jpg.c11a57f996fef2495239ce5649caa13b.jpg

 

It's secure and doesn't vibrate.

Once you have the power cables etc hooked up, you can slide the motherboard back into place and complete the rest of the connections.

 

PXL_20210101_230023372.thumb.jpg.b28148b6c504a241b6f42b740a50c150.jpg

 

NOTE: I am not using passively cooled Xeon processors. I am using a standard ATX board with an i7 that has active cooling (a fan). If you are using passively cooled Xeons, sealing off the side vents might not be a good idea. You could try, but monitor the CPU temps. Argus will monitor that, too.

You also need to seal off the side vents, to create more negative pressure. Just use duct tape.

 

PXL_20210101_230001295.thumb.jpg.0f124e77b2056dc5fb10db1d7da39d28.jpg

The 80mm fans are connected to the motherboard with a 'Y', and the front 120mm fans are connected to the Corsair Command Pro. I connected them this way because I wanted control of the fans. I wanted to connect all of the fans to the Command Pro and control the fans through that, but my motherboard BIOS only sees four fans, and on boot only four fans in a particular configuration will spin up. And this seemed the best way. I wanted to control the fans with a curve based on the hard drive temps. The Command Pro won't do this as it doesn't read the drives. So I found Argus Monitor. It monitors all parts of the system and each drive using the S.M.A.R.T data. With this I can specify which drives I can apply a fan control curve it is based on. I simply looked for the hottest drive(s) and applied the same curve to all three 120mm fans. And I run the 80mm fans at max. It's been running for several hours now, and the temps are on par with what I had in the Norco case. A little better actually, as the curve I made is running the fans at a slightly lower RPM (see below pics)

Now, as I mentioned in the beginning, the PSU fan is also noisy. I was originally going to replace the fans, but after a little research there is a replacement PSU that is quiet. The Supermicro PWS-920P-SQ 920W 80 Plus Platinum. Just replace, and all good.

 

 

93965508_Screenshot2021-01-10093630.png.8c62d9f9bdc165823e04ce4080325c40.png

640502588_Screenshot2021-01-10093723.png.baf54c6fca0936f66316cbfd38daa956.png

1440735287_Screenshot2021-01-10094501.thumb.png.614c1e71a6584038d842816e07610752.png

 

 

 

The end result is a very nice. I am content.

 

PXL_20210101_225949461.thumb.jpg.d2da67dde1bdbfb307e6840f634f5b8b.jpg

Edited by sooty234
  • Like 2
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...