Ideal configuration for transcoding HEVC 4K

[TrickOTreat 0]

Hi,

 

 

I plan to create a new transcoding Emby server to serve, at least and simultaneously :

- 1 transcoding from HEVC media sources to a 4K TV (3840 x 2160 so) by using a little PC to play it on a Chromium browser ;

- 2 transcodings from HEVC media sources to 2 1080p monitor/TV with other little PC and with Chromium browser also.

 

My "little PC" are in production yet and can serve without any trouble 1080p video on their Chromium. They are composed from a Celeron G3900T CPU https://ark.intel.com/content/www/us/en/ark/products/90738/intel-celeron-processor-g3900t-2m-cache-2-60-ghz.html and 4Go DDR4 2400MHz. No GPU, the Intel HD integrated graphics seems to do the job well. As those CPU support 4K rendering, I suppose they will be sufficient enough to play 4K video on my TV/monitor at 60Hz (play not transcode).

 

For the moment I use a 4 vCPU / 2Go RAM Debian Virtual Machine on a Proxmox to hold my Emby Server. The Proxmox itself is on a Xeon 1230 v5 https://ark.intel.com/content/www/fr/fr/ark/products/88182/intel-xeon-processor-e3-1230-v5-8m-cache-3-40-ghz.html with 32Go DDR4 2133 MHz. But I'd like to separate the Emby Server VM and my NAS VM from other VM I have into.

 

My main problem, is which CPU would hold well those requirements ? I need to transcode from H265 medias to H264 because I encode my media in HEVC to spare disk space and because my players on Chromium does not support H265 natively (so as most/all browsers I think). I don't need many RAM on this second Proxmox server I plan to assemble, since I don't see any massive usage of RAM on my actual VM (NFS NAS and Emby Server).

 

So, I'm thinking about 8G RAM, that would be sufficient for the second server, and a 4 cores / 8 threads CPU, maybe ? But which one ? Or, will I have no choice to use a more overpowered CPU, as a 8c/16t ? But it seems way too big for the vision I have right now...

 

 

 

Thanks you for your advices !

[Luke 37056]

@ is this similar to your setup?

[Guest asrequested]

If you're transcoding 4k hevc, you'll either need GPU that supports that, or a better CPU. Xeons have a difficult time with that.

[TrickOTreat 0]

Does a budget and low power consumption CPU and a good GPU, supporting 4K HDR HEVC transcoding, will be enough, assuming an Emby Server VM will access the GPU by passthrough ?

 

For instance, I'm thinking about this kind of hardware :

 

Server :

- CPU : Intel G5500T https://ark.intel.com/content/www/us/en/ark/products/129947/intel-pentium-gold-g5500t-processor-4m-cache-3-20-ghz.html

- GPU : Nvidia  GTX1080 Ti

 

Emby Server VM :

- 2 vCPU

- GPU passthrough

 

I have not think about using a GPU for transcoding, I'm a bit confused on how it will work and which CPU/GPU combo will be good enough...

[Guest asrequested]

As long as the CPU can run the server, the GPU can transcode (depending on how many concurrent transcodes). That GPU is more than you need. How many simultaneous transcodes will you be doing?

[TrickOTreat 0]

I'd like to transcode one 4K HEVC to a TV and 2x1080p to monitors/TV at the same time. These are my maximum needs.

 

I've got a Radeon RX570 4GB, could I use it for my purpose ? How can I enable GPU transcoding in Emby Server ? I don't find how to do it on my actual VM 

[Guest asrequested]

It looks like that GPU supports HEVC 4k. How you enable it in a VM is something I don't know. There's a bunch of guys using VMs, maybe one of those can help?

[TrickOTreat 0]

I have deploy a new VM for testing in which I'm using GPU passthrough with my Radeon RX 570. Linux firmwares upgraded and my Debian VM can recognize the GPU :

 

$ lshw -c display | grep -iE "amd|radeon"

       produit: Ellesmere [Radeon RX 470/480]

       fabriquant: Advanced Micro Devices, Inc. [AMD/ATI]

       configuration: driver=amdgpu latency=0

 

Fun fact, it's shown as RX 470/480 instead of RX 570 but I don't believe it does matter.

 

I've upgraded my Emby Server to Emby Premiere too, added the key in the admin panel and set "Advanced" for transcoding hardware acceleration. But I can't choose any GPU nor select anything : my prefered hardware encoders/decoders sections are empty.

 

What am I doing wrong ? 

[Guest asrequested]

That will be a question for @@softworkz I think.

[Luke 37056]

Have you installed the latest video drivers?

[TrickOTreat 0]

It was probably a driver problem on Debian as on Win10 (VM too, GPU passthrough too), no problem, I can see hardware preferences.

 

So I've managed to test before installing my new Emby Server, and I've got some questions on hardware acceleration :

 

- I saw the GPU percentage usage growing during my test, so it's OK. But the 4 threads CPU were used too. Is it normal ?

- Does GPU in hardware acceleration do almost all the job for transcoding or CPU is quite important too ? I've got a tiny G4560 I could use in relation with the Radeon RX 570 but I'm afraid it would be too underpowered for that task...

- If not, if the CPU doesn't matter as far as Embu use the GPU, I will need to change the GPU by another more powerful if transcoding 4K HEVC will be too slow. Am I right ?

 

Thanks very much for all your help !

[Luke 37056]

It was probably a driver problem on Debian as on Win10 (VM too, GPU passthrough too), no problem, I can see hardware preferences.

 

So I've managed to test before installing my new Emby Server, and I've got some questions on hardware acceleration :

 

- I saw the GPU percentage usage growing during my test, so it's OK. But the 4 threads CPU were used too. Is it normal ?

- Does GPU in hardware acceleration do almost all the job for transcoding or CPU is quite important too ? I've got a tiny G4560 I could use in relation with the Radeon RX 570 but I'm afraid it would be too underpowered for that task...

- If not, if the CPU doesn't matter as far as Embu use the GPU, I will need to change the GPU by another more powerful if transcoding 4K HEVC will be too slow. Am I right ?

 

Thanks very much for all your help !

 

Hi there, @@softworkz may have some insight on this question, but we'd really have to look at some ffmpeg log examples in order to give a meaningful response. thanks.

[lightsout 144]

Remember that with an Nvidia card you are limited to only two streams transcoding at one time. Unless you get a Quadro. So that 1080ti would people be Overkill.

 

There is a driver hack you can do that takes that limitation away. I use it myself. Tested with a GTX 1060 and had 6 or 8 streams going. This was just in browser windows but they all looked good.

 

As far as hevc is the file size saving really all that good? Seems like you'd be better off just using a format that could didn't play.

[TrickOTreat 0]

Hi there !

 

I have done some tests and compile it in a zip you can find at : https://www.trickotreat.net/TrickOTest.zip (will be deleted in few weeks).

 

For the method of testing :

- Windows 10 + Emby Premiere

- VM with 4 vCPU + 6GB RAM + GPU Passhrough

- Server holding VM : CPU Intel G4560 2c/4t + 2400MHz RAM + AMD Radeon RX 570 4GB

 

I have done 3 tests with 3 different videos, all re-encoding with Handbrake in HEVC CRF18 / AAC FDK.

 

First test : No GPU usage, only CPU usage. It seems to be a subtitle encoding problem that prevent the GPU from transcoding. I may investigate on it to find a way to encode subtitles the same way every time.

Second & third test : Significant GPU usage but CPU is used too. Here, I can't evaluate how much the CPU is important and if it will be a point of failure if I want to transcode more 1080p and/or 4K (just 1x4K and 2x1080p will perfectly suit me !).

 

N.B. : The third test only produced one transcoding log file whereas the thow others produced two of them. I don't know why nor if it is important or relevant of something...

 

Well... With all of that, I can suppose :

1. I may transcoding 1x1080p fine with CPU and/or GPU usage ;

2. I might force the GPU transcoding if I fix the subtitle problem : I just have to find how to adjust Handbrake settings, I think ;

3. But I have no idea if the CPU usage will be a bottleneck even if the GPU is used. I expected to see high GPU usage an almost no CPU usage, but... No... Any idea ? 

 

Edit : @@lightsout - Yes, using H265 over H264 is a significant improvement for me in terme of disk usage. For instance a 30GB video in H264 can be reduced to 8GB in H265 with my Handbrake settings. So, it's a big money saving in the purchase of hard drives 

Edited April 14, 2019 by TrickOTreat

[lightsout 144]

Hi there !

 

I have done some tests and compile it in a zip you can find at : https://www.trickotreat.net/TrickOTest.zip (will be deleted in few weeks).

 

For the method of testing :

- Windows 10 + Emby Premiere

- VM with 4 vCPU + 6GB RAM + GPU Passhrough

- Server holding VM : CPU Intel G4560 2c/4t + 2400MHz RAM + AMD Radeon RX 570 4GB

 

I have done 3 tests with 3 different videos, all re-encoding with Handbrake in HEVC CRF18 / AAC FDK.

 

First test : No GPU usage, only CPU usage. It seems to be a subtitle encoding problem that prevent the GPU from transcoding. I may investigate on it to find a way to encode subtitles the same way every time.

Second & third test : Significant GPU usage but CPU is used too. Here, I can't evaluate how much the CPU is important and if it will be a point of failure if I want to transcode more 1080p and/or 4K (just 1x4K and 2x1080p will perfectly suit me !).

 

N.B. : The third test only produced one transcoding log file whereas the thow others produced two of them. I don't know why nor if it is important or relevant of something...

 

Well... With all of that, I can suppose :

1. I may transcoding 1x1080p fine with CPU and/or GPU usage ;

2. I might force the GPU transcoding if I fix the subtitle problem : I just have to find how to adjust Handbrake settings, I think ;

3. But I have no idea if the CPU usage will be a bottleneck even if the GPU is used. I expected to see high GPU usage an almost no CPU usage, but... No... Any idea ?

 

Edit : @@lightsout - Yes, using H265 over H264 is a significant improvement for me in terme of disk usage. For instance a 30GB video in H264 can be reduced to 8GB in H265 with my Handbrake settings. So, it's a big money saving in the purchase of hard drives

Is that true? 30 vs 8gb? Is that 30gb a remux.? Straight from a disk? Because if so it also could be reduced a good amount.

 

I did a comparison with the same setting on nvenc 264 vs 265 and the 265 file was larger? Had them both set at 7500kbps.

 

Maybe because 265 needs less bitrate?

 

 

Sent from my iPhone using Tapatalk

[Guest asrequested]

Is that true? 30 vs 8gb? Is that 30gb a remux.? Straight from a disk? Because if so it also could be reduced a good amount.

 

I did a comparison with the same setting on nvenc 264 vs 265 and the 265 file was larger? Had them both set at 7500kbps.

 

Maybe because 265 needs less bitrate?

 

 

Sent from my iPhone using Tapatalk

 

You did something wrong if the file is bigger. HEVC should be significantly smaller with almost no loss of detail. That's why 4k blueray discs are HEVC, too.

[Charlie117 56]

You did something wrong if the file is bigger. HEVC should be significantly smaller with almost no loss of detail. That's why 4k blueray discs are HEVC, too.

 

He specified a bitrate of 7500kbps for both the H264 and H265 encode so they should be roughly the same size really. He could reduce the bitrate of the H265 encode by 10-50% without losing too much detail compared to the H264 encode depending on the source material. 

 

As far as hevc is the file size saving really all that good? Seems like you'd be better off just using a format that could didn't play.

 

It really depends on the source material. With grain for example, the x264 encoder still outperforms the x265 encoder. But for very clean sources, you can achieve a nice bitrate reduction with x265 compared to x264 while keeping the quality the same.

[lightsout 144]

He specified a bitrate of 7500kbps for both the H264 and H265 encode so they should be roughly the same size really. He could reduce the bitrate of the H265 encode by 10-50% without losing too much detail compared to the H264 encode depending on the source material.

 

 

It really depends on the source material. With grain for example, the x264 encoder still outperforms the x265 encoder. But for very clean sources, you can achieve a nice bitrate reduction with x265 compared to x264 while keeping the quality the same.

I see. I haven't played around much with it. My source material is Mpeg2 cable recordings. So they aren't the greatest anyways. I have settled at 7500kpbs as when I've done some side by side comparison with the source it basically looked the same.

 

I also use nvenc so that factors in. The speed benefit is just too great for me to not use it. I don't notice much quality drop off.

 

As far as Blu Ray I just leave those original. The main reason I encode my TV recordings is because Emby handles 1080p vs 1080i much better.

[lightsout 144]

Ok to test I did my normal h264 @ 7.5mbps, and then did an h265 @ 4mbps. I couldn't tell them apart and the resulting video was a gig smaller for 265, not bad. The source on these is not the best so I am sure its not perfect but from what is an already just "ok" video it looks just fine.

[TrickOTreat 0]

Some more tests... It seems the GPU transcoding works well with the RX 570. I even can run 2x1080p transcoding with 2 vCPU from a G4560 and 2Go RAM on the VM which has the GPU.

 

Do you think I can rely on very few CPU threads as I use the GPU now ? Or will it be more judicious to migrate on a CPU which allows me to give more than 2 vCPU to the VM ? Like a Ryzen 5 2400G / 2600 ?

[Guest asrequested]

Some more tests... It seems the GPU transcoding works well with the RX 570. I even can run 2x1080p transcoding with 2 vCPU from a G4560 and 2Go RAM on the VM which has the GPU.

 

Do you think I can rely on very few CPU threads as I use the GPU now ? Or will it be more judicious to migrate on a CPU which allows me to give more than 2 vCPU to the VM ? Like a Ryzen 5 2400G / 2600 ?

 

That depends on what you want to do with the server. Adding plugins, using coverart etc. will increase the demand. My advice is never use cheap, if you don't have to.

[softworkz 3335]

It was probably a driver problem on Debian as on Win10 (VM too, GPU passthrough too), no problem, I can see hardware preferences.

 

So I've managed to test before installing my new Emby Server, and I've got some questions on hardware acceleration :

 

- I saw the GPU percentage usage growing during my test, so it's OK. But the 4 threads CPU were used too. Is it normal ?

 

- Does GPU in hardware acceleration do almost all the job for transcoding or CPU is quite important too ? I've got a tiny G4560 I could use in relation with the Radeon RX 570 but I'm afraid it would be too underpowered for that task...

- If not, if the CPU doesn't matter as far as Embu use the GPU, I will need to change the GPU by another more powerful if transcoding 4K HEVC will be too slow. Am I right ?

 

Thanks very much for all your help !

 

Yes. that can be normal, for a simple yet unexpected reason: Once you switch to hardware accelerated transcoding, this can end up in a situation where the video is being transcoded so fast that the CPU can't even keep up with that pace for audio transcoding. (audio is always done by the CPU. 

As a result, you might see even more system resources being used than with software transcoding. Unexpected situation, but it happens pretty often.

On the other side, you transcoding will be fully completed many times faster. But that doesn't by you much, so you should rather look into Emby's throttling feature to manage the use of your system resources more effectively.

 

PS: Some more background could be found here: https://emby.media/community/index.php?/topic/71573-h264-nvenc-provided-device-doesnt-support-required-nvenc-features/