Jump to content

NEW SERVER BUILD - GPU Transcoding


pir8radio

Recommended Posts

pir8radio

Well..........    After hearing the future GPU support in this thread https://emby.media/community/index.php?/topic/66959-gpu-transcode-multiple-gpus/ I have decided to build a new emby server with transcoding running on GPU's insead of raw CPU...  

 

In an attempt to please @ I have blindly ordered the following equipment should be here end of Dec lol....   :lol:

I'm going to start with one gpu, with future expansion to 4 if needed/when I have cash lol.

I'll post pics and stats as I get going...   Kind of excited!   Its my Christmas present 5c1fa08e8769a_333333333333333333333333.p

 

Case:  https://www.supermicro.com/products/system/2u/2027/sys-2027gr-trf.cfm

Motherboard:  https://www.supermicro.com/products/motherboard/Xeon/C600/X9DRG-HF.cfm

GPU's: https://www.pny.com/nvidia-quadro-p4000

 

Processor: 2x Intel Xeon E5-2670 V2 Deca (10) Core 2.5Ghz
Memory: 64GB DDR3 (4 x 16GB - DDR3 - REG PC3-10600R (1333MHZ) )
Hard Drives:  None
Controller: Onboard SATA RAID RAID 0, 1, 5, 10 (8x SATA2 and 2x SATA3 ports)
NIC: Integrated Onboard Dual 1GB Ethernet ports (2)

Secondary Chassis/ Motherboard specs:
Supermicro 2U 10x 2.5" Drive Bays 
Server Chassis/ Case: SYS-2027GR-TRF
Motherboard: X9DRG-HF
Integrated IPMI 2.0 Management
Backplane: BPN-SAS-218A 10-port 2U (GPU chassis) SAS 6Gbps direct-attached backplane, support up to 10x 2.5-inch SAS/SATA HDD/SSD
PCI-Expansions slots: 4x PCI-E 3.0 x16 slots (support GPU/Xeon Phi cards, opt.), 1x PCI-E 3.0 x8 (in x16) slot, 1x PCI-E 2.0 x4 (in x16) slot
Drive bay: 10x 2.5" Supermicro caddy
Power: 2x 1800watt Power Supply PWS-1K81P-1R Platinum
Rail Kit: Generic Supermicro 3rd party

 

post-1037-0-73349800-1545614535_thumb.jpgpost-1037-0-75191500-1545614545_thumb.jpg

Edited by pir8radio
  • Like 1
Link to comment
Share on other sites

Guest asrequested

Holy crap in a hat! Yeah, that thing is a beast! And as I understand it, those GPUs aren't limited, so it's just a matter of increasing the load until it just can't handle any more.

Edited by Doofus
  • Like 1
Link to comment
Share on other sites

mastrmind11

I'm still scratching my head why you're in here talking about GPU when u have dual xeon monsters.  lose those things and save yourself the cheddar if you're gonna add *multiple* gpus....

Link to comment
Share on other sites

pir8radio

I have a single Xeon today and to be honest it struggles with some 4K hevc stuff. Only getting like 22-25 FPS at times with one stream. It can tear up a bunch of 1080p transcode streams. But I’ve been adding more and more 4K stuff. I just want to build out something that is as future proof as possible. Plus I’ve been wanting to try out GPU transcoding. My server also runs a few websites. But yea. The server I ordered came with the dual cpu’s Already.

Edited by pir8radio
Link to comment
Share on other sites

mastrmind11

I have a single Xeon today and to be honest it struggles with some 4K hevc stuff. Only getting like 22-25 FPS at times with one stream. It can tear up a bunch of 1080p transcode streams. But I’ve been adding more and more 4K stuff. I just want to build out something that is as future proof as possible. Plus I’ve been wanting to try out GPU transcoding. My server also runs a few websites. But yea. The server I ordered came with the dual cpu’s Already.

oh yeah, hevc stuff is a different beast.  i haven't, and likely won't (at least in the near future) venture down that path simply because idgaf about 4k for the little time i have in front of the screen.  kids obviously don't care, and wife stares at her phone no matter what's playing unless it's one of the 30 iterations of housewives.  so :shrug:, i'm good for a while.  anyway, best of luck.  post your meg-a-watt charts once this thing is up and running :)

  • Like 1
Link to comment
Share on other sites

Guest asrequested

Quick question. I may be misreading something, but how are going to put a 4U GPU in a 2U case? They are 4.4". If the power connectors are at the back, you can just about squeeze one in a 3U. That's what I did with my GTX 1050. But for the GTX 1060, the power connectors were on top, and I needed to get a 4U case.

Edited by Doofus
Link to comment
Share on other sites

pir8radio

Quick question. I may be misreading something, but how are going to put a 4U GPU in a 2U case?

[emoji846] they go in sideways. It’s a GPU case. Takes up to 4 full size GPU’s even supplies power to them from motherboard power connectors. At My server colo site I only pay for 5U’ and my drive array eats 3U so I had to go with another 2U server or redo everything which I didn’t wanna do.

 

 

Sent from my iPhone using Tapatalk

Edited by pir8radio
Link to comment
Share on other sites

Guest asrequested

[emoji846] they go in sideways. It’s a GPU case. Takes up to 4 full size GPU’s even supplies power to them from motherboard power connectors. At My server colo site I only pay for 5U’ and my drive array eats 3U so I had to go with another 2U server or redo everything which I didn’t wanna do.

 

 

Sent from my iPhone using Tapatalk

 

Well, damn! That is very cool! That thing looks like it makes some noise? Gonna need a lot of air going through that for those GPUs. Now I can see how it's set up. Those 'pockets' on the sides. Nice!

Edited by Doofus
Link to comment
Share on other sites

pir8radio

Well, damn! That is very cool! That thing looks like it makes some noise? Gonna need a lot of air going through that for those GPUs. Now I can see how it's set up. Those 'pockets' on the sides. Nice!

 

These are the two best pictures I could find that shows GPU's installed...    Two on each side near the front of the server...  You cant actually access the HDMI ports on the GPU's they are there just for processing..

post-1037-0-73349800-1545614535_thumb.jpg

post-1037-0-75191500-1545614545_thumb.jpg

Link to comment
Share on other sites

bleomycin

This is really interesting, beautiful build. I'm currently running the 3.6 beta on a E5-2678 v3 (12 core). I never have any issues with transcodes for live streaming but I'm very interested in decreasing the transcoding time in preparation for offline sync to a device. Currently if I queue up a boatload of items for sync before I head off to a trip I need to allocate a good 12-24 hours to ensure everything trans codes in time. Actually transferring the completed files over goes fast enough, ~300-400Mbit to my ipad.

 

Would adding gpu's such as you are doing help in this situation? Is there a way to calculate how much faster they would be?

 

Thanks!

Link to comment
Share on other sites

pir8radio

Would adding gpu's such as you are doing help in this situation? Is there a way to calculate how much faster they would be?

 

Thanks!

 

from what I read online yes.  I was unable to find any values or stats...  Thats one thing I hope to do when done, I hope to supply some actual FPS transcode speeds based on different files, no one has been able to give me that yet. 

Link to comment
Share on other sites

bleomycin

from what I read online yes.  I was unable to find any values or stats...  Thats one thing I hope to do when done, I hope to supply some actual FPS transcode speeds based on different files, no one has been able to give me that yet. 

 

Ahhh, awesome. Glad you're going to do some testing, I look forward to the results!

Link to comment
Share on other sites

BAlGaInTl

This looks promising. I have a 1060 6GB laying around doing nothing at the moment. I probably wouldn't need more than 2 transcodes at a time.

 

What's the Linux/Docker compatibility like for doing GPU decoding?

 

If one were to go over 2... would it fall back to CPU for the 3rd?

Link to comment
Share on other sites

Hemaniaco

Sorry to disturb the theme buit I was exactly thinking about do a setup pretty similar.

 

At the end I decided to get a Workstaion and keep Windows as Im nobby as hell on Linux but before ordering and doing a mistake I wanna ask here

 

Device will run Emby server and Unifi Vide Server (15 cameras). 2 TV will be sucking 4k content thru a 10gigabit network

 

  • Chassis: 1 x Fujitsu Celsius R920 Tower - 800W (Grade B)
  • Processor(s): 2 x Intel Xeon E5-2670 V1 - 8-Core 2.60Ghz (20MB Cache, 8.00GTs, 115W)
  • Heatsink(s): 2 x Fujitsu Celsius M720,M730,R920,R930 Heatsink
  • Memory (RAM): 6 x 16GB - DDR3L 1066MHz (PC3L-8500R, 4RX4, ECC REG)
  • Hard Drive(s) & SSD(s): 1 x 128GB - SATA (6G) SSD - Major Brand
  • Graphics Card: 1 x nVidia Quadro K4000 - 3GB GDDR5 (DVI, 2*DisplayPort)
  • Optical Drive: 1 x DVD Drive - Full Height (5.25")
  • Black Power Cable(s): 1 x EU Plug to C13 (Kettle Lead) Power Cable
  • 4x10TB Seagate EXOS

 

 

I would really apreciate the opinion!   Best

Edited by Hemaniaco
Link to comment
Share on other sites

pir8radio

Sorry to disturb the theme buit I was exactly thinking about do a setup pretty similar.

 

At the end I decided to get a Workstaion and keep Windows as Im nobby as hell on Linux but before ordering and doing a mistake I wanna ask here

 

Device will run Emby server and Unifi Vide Server (15 cameras). 2 TV will be sucking 4k content thru a 10gigabit network

 

  • Chassis: 1 x Fujitsu Celsius R920 Tower - 800W (Grade B)
  • Processor(s): 2 x Intel Xeon E5-2670 V1 - 8-Core 2.60Ghz (20MB Cache, 8.00GTs, 115W)
  • Heatsink(s): 2 x Fujitsu Celsius M720,M730,R920,R930 Heatsink
  • Memory (RAM): 6 x 16GB - DDR3L 1066MHz (PC3L-8500R, 4RX4, ECC REG)
  • Hard Drive(s) & SSD(s): 1 x 128GB - SATA (6G) SSD - Major Brand
  • Graphics Card: 1 x nVidia Quadro K4000 - 3GB GDDR5 (DVI, 2*DisplayPort)
  • Optical Drive: 1 x DVD Drive - Full Height (5.25")
  • Black Power Cable(s): 1 x EU Plug to C13 (Kettle Lead) Power Cable
  • 4x10TB Seagate EXOS

 

 

I would really apreciate the opinion!   Best

 

I would not use a K4000  i would go with a P4000   Kepler has no HEVC or h265 support where as Pascal does (at least for 4:2:0).

My current server has the E5-2470 V2's in it and does not do HEVC transcoding very well. 

My current server is basically what you are proposing building above. If you don't run a bunch of concurrent streams, or a couple HEVC, that might be fine, not sure the requirements for the unifi software, my NVR runs 9 - 8MP 4k cameras recording 24/7 and doesn't use hardly any resources unless I open up live video views on that PC.

Edited by pir8radio
Link to comment
Share on other sites

  • 2 weeks later...
pir8radio

Well my new toys came in, and I could'nt resist a quick test.....   Just installed win 10 and some drivers..  nothing is really setup yet.   Took a 4096x2304 hevc file and tried transcoding to 720p at 3mbps with CPU (20 xeon cores with hyperthreading) and then with the GPU...  Results are interesting... But I'm sure I have a drive bottleneck right now...    Promising though...  

 

CPU:

frame= 2659 fps= 48 q=28.0 size=N/A time=00:02:15.07 bitrate=N/A dup=3 drop=0 speed=2.45x

post-1037-0-47419000-1547356944_thumb.png

 

 

 

GPU:

frame= 2674 fps=142 q=19.0 size=N/A time=00:01:51.29 bitrate=N/A dup=3 drop=0 speed=5.92x

post-1037-0-83106700-1547356990_thumb.png

Edited by pir8radio
Link to comment
Share on other sites

pir8radio

You're testing on both?

Oh sorry. I read it fast. I thought you said “are you testing the beta release”.

 

Only on beta.

 

 

Sent from my iPhone using Tapatalk

Link to comment
Share on other sites

pir8radio

Have you tried running multiple 4k transcodes?

 

Not yet, I'm still working on getting all of my software onto the new server...   Migrating my os from a mechanical drive to SSD raid 1.    I still have a lot to do hardware and software wise first, then ill start beating it up lol.

Link to comment
Share on other sites

legallink

Looking at the latest test, was there a reason you dropped the quality settings so drastically from one to the other?  If it's the same source material, I would assume you would want to keep things as uniform as possible?  Or is that just your standard settings?

 

Also, are you utilizing 20% of your CPU for 1 file transcoding?  I'm assuming this is CLI run and not through Emby just to do some testing?  How long did the whole file take?

 

Looks lovely though.  I feel sheepish with my 2400g sitting next to me, but oddly, it does the trick.

 

Relate/Unrelated, I assume your "Media" stats in your signature haven't been updated in a while.  How much data are you running now?  I don't have nearly as much data as you and I am running close to 16TB (predominantly movies, and mostly 1080p.  Just starting to get into 4k video.  Multiple versions of each file to reduce transcode requirements (typically 3 - 1 raw uncompressed, 1 mp4 compressed file at q:21 through handbrake, and 1 small file with 1 audio track for downloading onto mobile devices).

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...