Is this new hardware good enough?

[Rufftimo 3]

• There's several dozen subscribers to a library of 3,500 titles (15 TB)
• Existing system is a 10-year old Intel i5 WIndows-based system, it's a piece of junk
• Looking at buying:
  • 12th gen i5 or i7
  • Mid-range motherboard, 32 GB DDR4 3200 RAM
  • Quadro P5000 graphics card

My big remaining questions are storage-related:

• Should we just get a few high-capacity SSDs, plug them in to the motherboard and call it good?
• Are SSDs overkill, when lower-cost optical drives are sufficient?
• What are the advantages of a NAS setup over this more traditional configuration? When should NAS be considered?
• If the library is stored across several drives, how should they be configured? (they're currently in a RAID config)

My general question: Which of these components should be top-shelf, and which can we skimp on

[Dibbes 432]

Opinions differ, but from experience, SSD/NVME for the OS and Emby cache and database, but for the titles itself, 7200RPM spinning disks are more than enough. I'm running my library of 2 different Synology NAS boxes, with 8TB WD Red spinning disks running in RAID 5 and Btrfs. My server is an ageing i7 - 4th gen CPU with 32GB RAM and a nVidia 1660 Superfor encoding.

The advantages of a NAS is to have the storage and the server separated. If something goes belly-up, you don't lose everything at once. Also, and this again is my personal opinion, a NAS filesystem, optimised for exactly this, gives better performance than a generic server every could.

Edited November 24, 2021 by Dibbes

[ginjaninja 543]

how many concurrent users do you [estimate you] need to support?

do you have a feel for much transcoding you will be doing (in relation to content / client hardware capabilities)

what is your current storage config (storage os, raid level, drives)?

what backup do you have and can the service comfortably stomach the time to restore in the event of storage failure?

[rbjtech 4357]

I've just been through a similar scenario - build log here - 

https://emby.media/community/index.php?/topic/103658-alder-lake-build/

You don't need any discreet graphics card as the iGPU on the 12th Gen (UHD770) is a monster when it comes to transcoding.

I had it running 8 x 4K remux (~80+Mbit) transcodes (with HDR>SDR ToneMapping ) to 1080p @ 10Mbit/sec (throttle off - ie parallel) and it handled it fine.

You don't need massive amounts of RAM - but with the iGPU, the more RAM you have, the more you can assign to transcoding.  I have mine set to max 32Gbyte (lol) but each 4K session takes approx 1Gb.

Depending on other duties, I would say 32 Gbyte of main system memory is plenty.

Standard HDD's are fine for media duties - you bottleneck will be the controllers anyway.  Main emby system and cache, defo use an NVME as said above.

The key issue with a NAS is the network connecting the devices, at 1gig, it is simply a huge bottleneck vs DAS (Directly Attached SAS, SATA3 or USB3 Storage) at 10gig it starts to make more sense but is extremely costly - and is still not faster than any local PCIe bus (at more than 1x anyway).

[Rufftimo 3]

Wow, great answers, everyone! This is a goldmine of information for me, exactly what I was looking for.

Attached is the current usage, by type.

Seems our best path is DAS, not NAS, but it seems like that won't make a huge difference either way.

Also, I'll save $600 by skipping the discrete GPU altogether and just seeing how far the 12th Gen i7 can handle everything... amazing tip!

Thanks again!

[Dibbes 432]

2 hours ago, rbjtech said:

The key issue with a NAS is the network connecting the devices, at 1gig, it is simply a huge bottleneck vs DAS (Directly Attached SAS, SATA3 or USB3 Storage) at 10gig it starts to make more sense but is extremely costly - and is still not faster than any local PCIe bus (at more than 1x anyway).

My Synology boxes have four 1-gig ports and I have a Cisco switch with LAG support, so that helps... I actually do get 3.6gigs copying from one box to the other, so it's actually better than I expected when I configured this. Since most of the clients are on WiFi or coming in from the outside world, it's more than enough...

I do agree though that DAS is a lot faster, of course, but since I don't have the luxury of being able to put a rack anywhere in the apartment nor will my SO allow any really noisy server, NAS boxes it is

[rbjtech 4357]

24 minutes ago, Dibbes said:

My Synology boxes have four 1-gig ports and I have a Cisco switch with LAG support, so that helps... I actually do get 3.6gigs copying from one box to the other, so it's actually better than I expected when I configured this. Since most of the clients are on WiFi or coming in from the outside world, it's more than enough...

I do agree though that DAS is a lot faster, of course, but since I don't have the luxury of being able to put a rack anywhere in the apartment nor will my SO allow any really noisy server, NAS boxes it is

The multiple ports heavily depend on the protocol you are using - something like SMB Multichannel (v3) will utilise all of the links in parallel - but using LACP on it's own will not, you'll simply get 4 'paths' of 1Gig - great for 4 x 1 Gig connections(parallel), but you won't ever get 1 x 4Gig over any normal protocol.

But yea - for general usage, 1gig is more than enough anyway, I only use a bigger pipe for my backup server sync.

Edited November 24, 2021 by rbjtech

[Rufftimo 3]

OK, so the conclusion is to pick up three 10 TB drives and use them as DAS with the new motherboard in a RAID 5 configuration, using the motherboard's support of RAID 5.

Good? Or should we put those drives into a Synology NAS?

[rbjtech 4357]

My personal view is RAID is no longer desirable - especially for media storage - using software based disk pooling software such as Drivepool will allow you to utilise your disks much better without the need for proprietary / low flexibility / low/zero upgradeability RAID options. 

As an example - if you have a Backup of your media files (which you should have..) and have a typical number of users on your system, then why would you want media files duplicated ?  If you lost the disk, then you just restore them, it's certainly quicker than trying to rebuild a 10Tb RAID array using parity.  

Yes RAID 5 will give you a read performance increase - but for media, the question is do you need this - the answer is likely no, not at all.    For typical usage, Media files are read very slowly (ie at the rate you watch them - 2 hrs for a film) - so you don't need performance at all.

I would still buy the 3 x 10Tb disks, but use 2 of them as JBOD's - with disk pooling (duplicating just want you cannot restore and/or really important files) and use the other disk in a hot swap removable caddy (or use via USB3 etc) - which you can use as a full backup of your system.  Once you get past 10Gb of storage, then you'll need to buy a further disk to keep backing up the pool as potentially with no duplication, you will have 20Tb of storage ..

[Rufftimo 3]

JBOD = Just a Bunch of Disks, right?  (I didn't look up anything... )

But isn't DrivePool also proprietary? You're just saying skip the RAID and just pool the drives, whether we do it with DrivePool, unraid, etc., right? And if you have half a dozen users watching the same library at the same time (our situation), doesn't that increase the need for disk performance?

I've read and watched quite a few positive things about unraid as the OS for a shared media library like Emby, and we could easily do that with this new build and skip using Windows altogether... any big downside to avoiding Windows, from your perspective? (I created a new thread to hopefully answer the Windows vs. Unraid question, but so far there's been no responses)

Also, I ended up purchasing three 12tb disks. Right now the library is 15TB, and we don't expect it to grow much from there. So what's the smartest way to pool these disks to get optimal performance while having a backup? Or does the math not quite work with three, and I'd need a fourth 12 TB drive to have a full backup... ?

[RanmaCanada 362]

I personally have all my media on an UNRAID server, and I've only had 10 concurrent users at once, but no one complained.  I personally have a laptop as my Emby server (i3-8130u), as I just wanted something that had a screen, and was small.  Quicksync can handle tonnes of simultaneous encodes (my laptop, 17 1080p encodes).  Rbjtech is correct on how much of a freaking beast Adler Lake is, as the xe asics are fast, and better than NVENC.

As for a backup, if your library is "only" 15TB, you could potentially get a 16TB, or 2 10's for cold storage. Or backblaze as it's currently on sale?