What is the best solution for hardware acceleration if I am going to be buying new just for this purpose?

[risho 1]

So my cpu is getting pegged whenever I transcode and I was looking into hardware transcoding. I would be buying a new card strictly for this purpose, so not repurposing an old one. I would like to be able to support 2 simultaneous 4k transcodes ideally, though realistically I don't usually transcode more than 3 1080p videos at a time. One thing of note is that I watch primarily anime which uses advanced substation subtitles which my understanding it they have to be burned in. My understanding is also that transcoding weird subtitle formats is also either bottlenecked or computationally expensive for hardware acceleration. Is it still worth persuing hardware acceleration? Will I be able to hardware transcode mutliple 1080p videos with .ass subtitles?

 

Now getting to the hardware: my server is running a xeon without an igpu or quicksync, so I need to get a dedicated card. My understanding is that the quality of hardware accelerated encoding vs software encoding is noticably different. How much worse is hardware encoding? It's also my understanding that as you get newer models of gpus the quality drop is reduced. So what will offer me the best perfomance and quality for hardware transcoding? I'm assuming an nvidia card nvenc? Probably a 10 series? Does the 20 series offer higher quality or performance? What about the quadro? How does amd do? Vaapi/vdpau? I'm not looking to spend a bunch of money on a gtx 1080 that will be bottlenecked to 2 streams and never actually be utilized to it's full potential, but I'm more than willing to pay whatever amount it costs for a solution that is high quality and high performance. I do have an affinity for open source, so I would prefer an amd/non-proprietary solution given the choice, but not at the cost of quality/performance.

 

For what it's worth I'm running debian/proxmox

 

Also worth noting over 50 percent of my media is h264 10 bit color as is standard for a lot of anime (hi10p)

Edited September 2, 2019 by risho

[Peckmore 23]

From what I can gather (and hopefully people will correct me if I'm wrong!   ) NVIDIA is the way to go for GPU transcoding. There is a good breakdown of their GPU's and what each can support here:

 

https://developer.nvidia.com/video-encode-decode-gpu-support-matrix

 

Note that for the cards that state a "Max # of concurrent sessions", this is only a driver limit and there is a patch to bypass this:

 

https://github.com/keylase/nvidia-patch

 

From my own experience I think the Quadro P400-P1000 cards are the best balance of "bang for buck", unless you have any videos that are H.265 (HEVC) 4:4:4 which these cards won't decode (I don't know much that comes in this format, but I also don't know the Anime scene). You can find them reasonably priced on ebay if you keep an eye out, especially the P400 and P600/P620. For transcoding there is no difference between them, and any other GPU features you won't be making use of, so functionally they are identical.

 

Comparing the Quadro's to the GeForce cards, if you look at cards using the same chipset (using that first link), then generally I find the Quadro comes out on top as it is usually a smaller, quieter card. As an example, I have the Quadro P600, which is a single-slot, low-profile card (it also includes a full-height bracket). The equivalent GeForce is the 1050, which is full-height and double-slot, and also uses more power. So my personal opinion would be to go for the Quadro P400 or P600/P620, as they use small amounts of power, support almost everything, can be purchased reasonably cheap, and are small cards. And the 2 stream limit can be patched out. 

 

For some anecdotal evidence (and this is definitely only anecdotal!) here is a Reddit post of someone who had 8 streams encoding on a Quadro P400 using the patched driver:

 

https://www.reddit.com/r/PleX/comments/9fs3z7/quadro_p400_encoding_8_streams_in_linux_on_xeon/

 

Worst case, you could add another one of the cards (which is the benefit of them being single slot!) if you need more streams simultaneously.

 

Quality wise I haven't noticed any easily perceptible drop in quality, but I only use mine for transcoding when I stream externally and the video exceeds my internet upstream. I guess if your CPU is struggling to decode the videos at the moment, then a possible drop in quality could be a fair trade off versus an unplayable video? And apologies, but I don't have any experience with the 20 series GPU's to know whether they are any better quality than the 10 series.

 

For yet more anecdotal evidence, someone answered a similar question in the same Reddit comment thread I mentioned above:

 

https://www.reddit.com/r/PleX/comments/9fs3z7/quadro_p400_encoding_8_streams_in_linux_on_xeon/e5z3fmn/

 

"i've tested a p2000, a radeon rx580, a gtx 980 ti, and a gtx 980 and the quality between those and cpu software transcoding was near identical."

 

With all of the above said however, caveat emptor, as I have no experience with .ass subtitles so you may want to look into how that affects things! 

 

Hope this helps!