Hardware Upgrade Experience (story)

[erikphilips 13]

I ended the current iteration of my Emby Setup as:
CPU: I5-4570
MB: ASRock Z97 Extreme 6
MEM: 4x 8gb @ 1333MHz
PSU: Seasonic SS-660XP 
Boot: Raid 1 2x Crucial BX500
Transcoding: Crucial BX1000
Store: LSI Megaraid 2970CV -> Raid 6 with 8x 4TB drives
OS: Windows 2012 R2 -> 2019
Hypervisor: HyperV  (Emby ran on a Windows VM just because could mostly, tried to keep the Host machine as clean of installs as possible)

It's been a fun work in progress to see this PC get upgraded and upgraded more (mostly just more ram and some SSDs).  It all started even before Emby existed and it was just the MediaBrowser and playing them via Windows Media Center.  Eventually I added Emby Plugin to Windows Media Center and used that until Emby Theatre was released.  But the time has come to rebuild some of those core components.

I decided that it would be more .. efficient/fun to upgrade my own work/gaming machine as well, so I replaced my Ryzen 3600 with a Ryzen 5900, and thus I had my new CPU.  I didn't do any research really into a good server motherboard, and I probably should have.  When I built my work machine, I looked long and hard at some of the best motherboards based on VRM components (apparently some manufactures were being really cheap and using bad VRM components) so to make it easy on myself I decided I'd just buy another ASRock x570 Taichi.  Worst case scenario I have a back up in one way or another.  I have way to may Chrome tabs open, Visual Studio's, Sql Server Mgmt Studio.. things these days just eat RAM, so upgraded my 2x 16gb 3200 to 2x 32gb 3600.. so now I have the RAM.

So I really didn't think to much in advance about the whole build. I got lazy and just assumed I could replace one type of component with another.  Sometimes it doesn't work out, and it definitely didn't work out for me.  Took out the mobo, put the new one in, start plugging everything in and realized, the new mobo needs another CPU connection from the PSU, and I don't have that cord at my current location.  Also the PSU is now 9-10 years old, almost past it's warranty, so purchased a Thermaltake Toughpower PF1 650W.  Received that after day or two, rewired everything again.

Then as I start to plug in the external devices... I have no video card as the 3600 doesn't have an IGP... sheesh.  Ok well if I'm gonna buy a video card, lets go for something with NVENC and do some GPU offloading I thought.  Trying to not make the same mistake twice about research, I looked into which cards support what encoding (https://developer.nvidia.com/video-encode-and-decode-gpu-support-matrix-new) and found that the consumer geforce cards are limited to 3 sessions.  The quadro cards however, are not, and so it appears that besides the B-Frame HEVC (which I honestly don't know what it means) the first card to really support most other encoding features is the P2000.  Off to ebay, buy a cheap P2000 and WAIT for that.  At this point I really should have looked at some of the other quadro cards.  From my very little search and some other documentation that is semi-relevant regarding encoding it would seem that the limit for cards is upwards of 250mb per transcode of the GPU (the p2000 has 5gb, so that 20 transcodes, way more than I'll need).  I could have gotten a bigger card for $30+ more.  There are driver hacks for Linux/Windows to remove the session limits, but I'd rather keep everything stock.

Now I'm ready to go.  Get everything plugged in and find out that in order to do SATA raid off the motherboard, I have to disable CSM.  It appears that disabling CSM prevents the LSI Megaraid card from working.  I'll probably do some additional testing in the future but for now, no Raid on the boot drive (oh well).  Start to install Windows Server 2022 and no network card!  WTF.  Apparently the Intel i211 card has no Windows Server drivers...  But there is a hack to make it work (https://blog.workinghardinit.work/2017/06/19/installing-intel-i211-i217v-i218v-i219v-drivers-windows-server-2016-eufi-boot/)?  Now I'm not really into going off the rails to make something work, just because it means if something crashes, I can't prove it wasn't some weird hack I did to get it work.  So.. do I buy a Windows server compatible PCIe network card and wait again? 

At this point I was ready to try something a little different.  A friend of mine suggested trying a different OS/Hypervisor.  I had used VMWare Workstation way way back in 2006-2008 when it went free, but I was totally unaware that ESXi became free.  I'm honestly amazed at this.  So started the install of ESXi and to surprise, literally everything worked.  No downloads, drivers, config files to mess with, it.. just.. worked.

Created a VM just for serving files, with PCIe passthrough of the LSI card, works flaw-less-ly.  

Created another VM for Emby and well it can't do passthrough of the P2000 because it's in use on the host (I could totally be wrong, but I don't know ESXi so..).  I bought a NVS 295 off amazon for super cheap, took out the P2000, put in the NVS 295, ESXi boots... no problem.  Yes!  Ok install the P2000, booted, ESXi allowed me to enable passthrough, installed nVidia Quadro desktop driver, checked Emby, it found the driver, and enabled nvenc.

Loaded up four chrome players on my work machine, playing the same 4k video (80mbps) and started adjusting the quality (1080p @ 10mbps, 1080p @ 6mbps, 720 @ 4mbps and 4k @ 40mbps).  CPU at 17%, GPU-ENC at 81%, GPU-DEC @ 100%, GPU Mem at 3.5gb.

Navigating through each of the players and changing the timecode causes a 100% spice in the GPU-ENC for just a second or so.  No dropped frames. Video response (start, stop, timecode change) is now almost instantaneous (way way faster then the previous build). 

In the end:
CPU: Ryzen 3600
MB: ASRock x570 Taichi
MEM: 2x 16gb @ 3200MHz
PSU: Thermaltake Toughpower PF1 650W 
Boot (ESXi): Crucial BX500
VMs: Crucial BX500
Transcoding: Crucial BX1000
Store: LSI Megaraid 2970CV -> Raid 6 with 8x 4TB drives
GPU offloading: P2000
OS:ESXi 7.0 U3
- Windows Server 2022 - File Share
- Windows Server 2022 - Emby

I hope this lasts me another 10 years!  And lesson learned... do more research.

 

 

Edited September 8, 2022 by erikphilips

[Luke 37069]

Thanks for sharing your experience !