Carlo 4560 Posted October 9, 2021 Posted October 9, 2021 The boxes I use have USB3 with UASP and eSATA. https://www.adorama.com/stusb3sata.html They cost a bit more and your eSata needs to support multiplex or you will need a PCI card that does (cheap).
MBSki 1114 Posted October 9, 2021 Author Posted October 9, 2021 Thanks @cayars. This one actually looks pretty good too. 15 bay 12Gb/s SAS Tower JBOD Enclosure - J1512. PC PitStop Data Storage Solutions - SAS Enclosures, DAS, NAS, iSCSI & FC SAN (pc-pitstop.com). I know it's $900, but I'd probably spend the same amount with a normal case, hot swap bays, PSU, and fans.
Carlo 4560 Posted October 9, 2021 Posted October 9, 2021 Honestly if you plan on needing more than the 8 bays that is a better way to go. SAS over USB3 any day of the week. 1
aspdend 177 Posted October 11, 2021 Posted October 11, 2021 On 09/10/2021 at 10:08, rbjtech said: Thanks - and yes, they are very expensive for what is just a shorter server case lol. So I bit the bullet and bought 2 x SC43550B's yesterday from servercase.co.uk (thanks for the link btw), I already have 3 x SC-500's - so I'm going to transplant those into one of these cases to give me 15 x front hot-swap drives. The other is my offsite backup server, that can use fixed HDD's, don't need hot-swap on that one. This server case is also 'short' (it's 550mm - it even has a fan wall) so fits nicely (just) in a 600mm wall mounted rack. HDD Fan's will be reversed so it blows air 'out' the front (as I do now) Just need to wait for the intel 12th Gen to arrive and then start the big upgrade lol .. You're welcome - I've used servercase and they were good for me...ounds like you are going to bve having fun when everything arrives, lucky you! 1
Dizzy49 99 Posted October 11, 2021 Posted October 11, 2021 I have two servers that I manage, one at my place and another at my parent's place. I have a Norco 4224 Case (24bay) Gigabyte Z270X with i7-7700k 64GB 1066Mhz ram 2x Adaptec 71605 Controllers 250GB Samsung Pro SSD (OS) 1TB Samsung Pro SSD (Download Landing Drive) 250GB Samsung Pro NVMe (PrimoCache) 250GB Samsung Pro NVMe (125GB for Drivepool Cache/SSD Swap, 125GB for Emby Cache) 4x 18TB Seagate Exos 3x 16GB Exos 3x 10TB IronWolf 3x 3TB WD Red 3x 4TB WD Gold I use Drivepool and Scanner as well. I'm recovering from a bad PSU that blew up half my equipment. I don't think I lost any data. If I did, I haven't figured out what's missing yet :D I use duplication for most of our important things, but I'd need about 5x more 18TB drives to duplicate all my videos. 1
rbjtech 5284 Posted October 11, 2021 Posted October 11, 2021 5 hours ago, Dizzy49 said: I'm recovering from a bad PSU that blew up half my equipment. I don't think I lost any data. If I did, I haven't figured out what's missing yet I use duplication for most of our important things, but I'd need about 5x more 18TB drives to duplicate all my videos. Ouch.. what brand PSU was it may I ask ? Was it undersized ? I have always used Corsair/Seasonic and 'touch wood' they have never been an issue - they do sit behind an APC UPS - so I would hope that protects them a bit more from mains spikes.
Dizzy49 99 Posted October 11, 2021 Posted October 11, 2021 11 hours ago, rbjtech said: Ouch.. what brand PSU was it may I ask ? Was it undersized ? I have always used Corsair/Seasonic and 'touch wood' they have never been an issue - they do sit behind an APC UPS - so I would hope that protects them a bit more from mains spikes. It was an EVGA SuperNova 850. Definitely not underpowered It was 9yrs old. I got it back to EVGA just under the 10yr deadline and they replaced it with no fuss. I have a pretty beefy SurgeX UPS so I didn't think about power issues. Doesn't help with the issues comes AFTER the UPS.
MBSki 1114 Posted March 5, 2023 Author Posted March 5, 2023 Hey All, reviving an old post here as I'm seriously considering a new PC build. the question I have is about the CPU. My current CPU is an Intel X Skylake (i9-7900x) which I got because it had a lot of PCIe lanes (44) and chipset lanes (24). I connect 8 SATA drives directly to the board along with 2 NVMe drives. the rest are connected to LSI x8 controllers. I also have an nVidia Quadro RTX 4000 in an x16 slot. So now I'm looking for a new CPU, ideally with PCIe 5.0 and I'm wondering, do I actually need all those PCIe and chipset lanes? I thought I needed them for all the drives I was connecting, but am I wrong about that? Either way, what CPU would everyone recommend?
rbjtech 5284 Posted March 5, 2023 Posted March 5, 2023 Intel 13th Gen - ditch/sell the RTX as the iGPU/QSV will match it for transcoding performance (believe it or not ..) Most standard ATX even mATX mobo's will have at least 6 x SATA3's - and 2 x NVME slots - and at least 3 x PCIe slots (16x,8x and 1x as a minimum). PCIe 5.0 is really not needed for 'storage' reasons - PCIe 4.0 is silly fast via NVME.
MBSki 1114 Posted March 5, 2023 Author Posted March 5, 2023 1 hour ago, rbjtech said: Intel 13th Gen - ditch/sell the RTX as the iGPU/QSV will match it for transcoding performance (believe it or not ..) Wow, that's incredible! It would be nice to free up some space in the box. Can't believe the CPU can handle the same as the Quadro. It works for HDR tone mapping too? 1 hour ago, rbjtech said: PCIe 5.0 is really not needed for 'storage' reasons - PCIe 4.0 is silly fast via NVME. Yea, I figured PCIe 5.0 might be overkill. I can save some money and stick with 4.0. What about the # of lanes on the CPU. Does it matter at all? I think the Intel it has 20 PCIe and 24 chipset lanes. Is that more than enough to connect a 30+ hard drives? Some drives directly connected to mobo (as many as mobo can support) and the rest on controller cards?
rbjtech 5284 Posted March 5, 2023 Posted March 5, 2023 (edited) 52 minutes ago, MBSki said: Wow, that's incredible! It would be nice to free up some space in the box. Can't believe the CPU can handle the same as the Quadro. It works for HDR tone mapping too? Yea, I figured PCIe 5.0 might be overkill. I can save some money and stick with 4.0. What about the # of lanes on the CPU. Does it matter at all? I think the Intel it has 20 PCIe and 24 chipset lanes. Is that more than enough to connect a 30+ hard drives? Some drives directly connected to mobo (as many as mobo can support) and the rest on controller cards? Yep - TM is done in the QSV hardware. Maybe you are getting confused with the # PCI lanes and the number of drives - they are not related. You could have 30 HDD's connected to a PCIe 3.0 1x PCI lane if you wanted to - that would give you a max of ~1GBit/sec (on the PCIe Bus) - about to 112 Mbytes per second. The issues start to arise if MULTIPLE HDD's all want to read/write at the same time - THEN and only then does the PCIe bandwidth become an issue. Your LSI controller is 8x (pcie 3.0) - that is 7.9GBit/sec - so 8 x the raw PCI capacity. 800-900Mbytes per second (if the controller could handle that). All over ONE physical PCIe connection and 8 x PCIe lanes ... You only need a large amount of PCIe lanes if you are talking huge servers with 4+ SAS controllers attached to huge NAS/SAN arrays - all working in parallel. For an emby server, you simply do not need this .. PCIe lane spec for the Z790 below (top end chipset for the 13th Gen CPU's) - lower end boards will have less PCI lanes on their chipsets, but CPU PCI lanes stay the same. https://www.intel.com/content/www/us/en/products/docs/chipsets/desktop-chipsets/z790-chipset-brief.html Edited March 5, 2023 by rbjtech 1
MBSki 1114 Posted March 5, 2023 Author Posted March 5, 2023 4 minutes ago, rbjtech said: Maybe you are getting confused with the # PCI lanes and the number of drives - they are not related. Yes, yes I was. And I believe I was duped by the previous PC builder I bought a pre-built from. So glad to hear I don't have to worry about the lanes. The options are endless now! 1
MBSki 1114 Posted March 5, 2023 Author Posted March 5, 2023 @rbjtech Any recommendation on which Intel 13th gen CPU to go with? Looking at the specs it seems like even an i5 is more than enough.
rbjtech 5284 Posted March 5, 2023 Posted March 5, 2023 1 hour ago, MBSki said: @rbjtech Any recommendation on which Intel 13th gen CPU to go with? Looking at the specs it seems like even an i5 is more than enough. Yep - it certainly is - any non-F iteration is going to be good but if it were me I would get the i5 13500 - as that has the strongest iGPU - the same as what they use in the i9 - a UHD770.
MBSki 1114 Posted March 5, 2023 Author Posted March 5, 2023 3 minutes ago, rbjtech said: Yep - it certainly is - any non-F iteration is going to be good but if it were me I would get the i5 13500 - as that has the strongest iGPU - the same as what they use in the i9 - a UHD770. Awesome, thanks!
Carlo 4560 Posted March 8, 2023 Posted March 8, 2023 On 10/3/2021 at 6:06 PM, cayars said: I've got a bit of everything with with 24 bay cases, 8 bay cases, 2 WD NAS, Synology NAS, dozen or so USB3 Externals. Some are located in my non attached garage for backup and some are primary. I get PMs asking what I have a lot since I didn't specifically mention it here. Besides the WD & Synology NAS boxes and the 8 bay External Encloses (all of these are on my backup server). I'm presently running Debian Linux using ZFS on a Dell R740XD that has 24 drive bays. I'm using 3 SAS cards in this server. 1 12MB SAS card is used for the internal drives. The other 2 SAS cards connect to the following external chasses: 2U 24 bay HP 3PAR M6710 3U 15 bay Dell EMC KTN-STL3 3U 18 bay Promise JX30 Storage Array 4U 24 bay Netapp DS4246 2U 12 bay Dell Compellent SC200 Storage Array That's 117 SAS Enterprise Drives I also have a Dell Fiber SAN and a 4 Node Ceph Cluster but those aren't used by Emby. My Emby disk allocation is about 1/6th of my total storage. I'm planning on changing things around a soon moving the Emby setup to a clustered environment of 3 tiered storage that does replication in real-time over a 100Mb backbone and then retire the current equipment to backup duties. 1
rbjtech 5284 Posted March 8, 2023 Posted March 8, 2023 4 minutes ago, cayars said: I get PMs asking what I have a lot since I didn't specifically mention it here. Besides the WD & Synology NAS boxes and the 8 bay External Encloses (all of these are on my backup server). I'm presently running Debian Linux using ZFS on a Dell R740XD that has 24 drive bays. I'm using 3 SAS cards in this server. 1 12MB SAS card is used for the internal drives. The other 2 SAS cards connect to the following external chasses: 2U 24 bay HP 3PAR M6710 3U 15 bay Dell EMC KTN-STL3 3U 18 bay Promise JX30 Storage Array 4U 24 bay Netapp DS4246 2U 12 bay Dell Compellent SC200 Storage Array That's 117 SAS Enterprise Drives I also have a Dell Fiber SAN and a 4 Node Ceph Cluster but those aren't used by Emby. My Emby disk allocation is about 1/6th of my total storage. I'm planning on changing things around a soon moving the Emby setup to a clustered environment of 3 tiered storage that does replication in real-time over a 100Mb backbone and then retire the current equipment to backup duties. I'd hate to see your electric bill ! 1
roaku 842 Posted March 8, 2023 Posted March 8, 2023 4 minutes ago, cayars said: I get PMs asking what I have a lot since I didn't specifically mention it here. Besides the WD & Synology NAS boxes and the 8 bay External Encloses (all of these are on my backup server). I'm presently running Debian Linux using ZFS on a Dell R740XD that has 24 drive bays. I'm using 3 SAS cards in this server. 1 12MB SAS card is used for the internal drives. The other 2 SAS cards connect to the following external chasses: 2U 24 bay HP 3PAR M6710 3U 15 bay Dell EMC KTN-STL3 3U 18 bay Promise JX30 Storage Array 4U 24 bay Netapp DS4246 2U 12 bay Dell Compellent SC200 Storage Array That's 117 SAS Enterprise Drives I also have a Dell Fiber SAN and a 4 Node Ceph Cluster but those aren't used by Emby. My Emby disk allocation is about 1/6th of my total storage. I'm planning on changing things around a soon moving the Emby setup to a clustered environment of 3 tiered storage that does replication in real-time over a 100Mb backbone and then retire the current equipment to backup duties. 2
MBSki 1114 Posted March 8, 2023 Author Posted March 8, 2023 5 minutes ago, cayars said: I get PMs asking what I have a lot since I didn't specifically mention it here. Besides the WD & Synology NAS boxes and the 8 bay External Encloses (all of these are on my backup server). I'm presently running Debian Linux using ZFS on a Dell R740XD that has 24 drive bays. I'm using 3 SAS cards in this server. 1 12MB SAS card is used for the internal drives. The other 2 SAS cards connect to the following external chasses: 2U 24 bay HP 3PAR M6710 3U 15 bay Dell EMC KTN-STL3 3U 18 bay Promise JX30 Storage Array 4U 24 bay Netapp DS4246 2U 12 bay Dell Compellent SC200 Storage Array That's 117 SAS Enterprise Drives I also have a Dell Fiber SAN and a 4 Node Ceph Cluster but those aren't used by Emby. My Emby disk allocation is about 1/6th of my total storage. I'm planning on changing things around a soon moving the Emby setup to a clustered environment of 3 tiered storage that does replication in real-time over a 100Mb backbone and then retire the current equipment to backup duties. Wow @cayars! How many users are you supporting? And how many TB do you have allocated to Emby for media storage?
Carlo 4560 Posted March 8, 2023 Posted March 8, 2023 (edited) 1 hour ago, rbjtech said: I'd hate to see your electric bill ! Pretty easy to calculate actually as I'm using 4 Circuit each 20 amp provisioned. Last month usage (monitored) averages 13.6, 15.2, 15.8, 14.2 amps or 58.8 amps of used power. 58.8amps x 120 volts = 7,056 Watts or 7KW an hour 7KWx24 hours=168KW per day 168KW x 365(year) / 12 (months) = 5,110KW or 5.11 MW (megawatts) If you pay 13.5 cents per KW that would be $689.85. If you have negotiated rates you might get it down to 11 cents per KW which would be $562.10 1 hour ago, MBSki said: Wow @cayars! How many users are you supporting? And how many TB do you have allocated to Emby for media storage? I can count my total users on both hands so it's nothing to do with number of users. I only share with family and a couple of close friends. I'm a media packrat with over a 1 million media files not counting music. Most of the vdevs are all 10TB, some 8TB and some 12TB so lets call it 10TB x 117 bays all filled. That's 1.17PB of raw storage. It's backup twice so you could say I have around 3.5PB of storage dedicated to media and storage. It's about 75% allocated. Keep in mind I do contracting for storage and cloud HCI (as well as hybrid clouds) as well as more and more for Kubernetes infrastructure and software designs so there is a reason I have so much storage and compute power. I can spin up Virtual Machines or Kubernetes containers in any size configuration I need to demo or testing purposes. I've got 4 Supermicro SuperServers each with 4 nodes that each have 2 Xeon CPUs (40 cores each)=4x4x2x40x2=2560 vCPUs plus 8 additional Dell Servers each dual XEON Gold or Platinum CPUs (1280 vCPUs) and I also have 16 Dell single CPU i7s that I can power up as needed (Kubernetes) (256 vCPUs). That's close to 4K vCPUs available, not to mention the various GPUs available. Edited March 8, 2023 by cayars
MBSki 1114 Posted March 8, 2023 Author Posted March 8, 2023 That's incredible @cayars. Definitely helps that it's part of your day job. 1
FIGHTPASS 1 Posted March 10, 2023 Posted March 10, 2023 On 3/8/2023 at 1:14 PM, cayars said: Pretty easy to calculate actually as I'm using 4 Circuit each 20 amp provisioned. Last month usage (monitored) averages 13.6, 15.2, 15.8, 14.2 amps or 58.8 amps of used power. 58.8amps x 120 volts = 7,056 Watts or 7KW an hour 7KWx24 hours=168KW per day 168KW x 365(year) / 12 (months) = 5,110KW or 5.11 MW (megawatts) If you pay 13.5 cents per KW that would be $689.85. If you have negotiated rates you might get it down to 11 cents per KW which would be $562.10 I can count my total users on both hands so it's nothing to do with number of users. I only share with family and a couple of close friends. I'm a media packrat with over a 1 million media files not counting music. Most of the vdevs are all 10TB, some 8TB and some 12TB so lets call it 10TB x 117 bays all filled. That's 1.17PB of raw storage. It's backup twice so you could say I have around 3.5PB of storage dedicated to media and storage. It's about 75% allocated. Keep in mind I do contracting for storage and cloud HCI (as well as hybrid clouds) as well as more and more for Kubernetes infrastructure and software designs so there is a reason I have so much storage and compute power. I can spin up Virtual Machines or Kubernetes containers in any size configuration I need to demo or testing purposes. I've got 4 Supermicro SuperServers each with 4 nodes that each have 2 Xeon CPUs (40 cores each)=4x4x2x40x2=2560 vCPUs plus 8 additional Dell Servers each dual XEON Gold or Platinum CPUs (1280 vCPUs) and I also have 16 Dell single CPU i7s that I can power up as needed (Kubernetes) (256 vCPUs). That's close to 4K vCPUs available, not to mention the various GPUs available. Do you own Netflix ?
Carlo 4560 Posted March 10, 2023 Posted March 10, 2023 3 hours ago, FIGHTPASS said: Do you own Netflix ? No, but I've been doing custom programming for them a few times a year for close to 10 years. Now you want to talk about an electric bill, think about how much Netflix or similar companies must be paying. 2
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now