Hardware acceleration for 4k content

[runtimesandbox 153]

I'm interested in peoples experiences with hardware encoding for 4k content. 

 

After some research my main concern is that a lot of people say that the quality is never as good as CPU transcoding, is the true and why is the case?

 

I am looking at getting an Nvidia card to transcode 4k down to 1080p / smaller for devices that can not playback 4k content or the high bit rate. NVENC seems to be the way to go for this (with an appropriate nvidia card). At the moment this is maxing out my CPU server and playback stutters.

 

Is it best to go down the graphics card / NVENC route or to upgrade the CPU to something like an i7 8800k? Can the latest intel processors encode 4k (multiple streams) / can emby take advantage of it?

 

 

Server is a Windows 2016 box

[PenkethBoy 2063]

why not use say "Handbrake" to transcode a lower resolution copy(s) that your client app support natively and make use of the multiple version option with emby - that way you only have to do the transcode once and the server will direct stream to your client(s) - and it costs you nothing

 

i have copies of 4k and 1080p movies in the same movie directory and emby can choose which to play on a client app or you can choose yourself when you press play - explained in the appropriate WIKI - how to name the files

 

As for why software transcoding is a better quality result is that QS or NVENC are "general" transcoders that have limited options that Intel and NVidia have made that you have little to no control over - with Handbrake or FFMpeg you have significantly more options and can get a result to your liking.

 

Yes a 8700k or a newish NVidia card will allow you to transcode on the fly - but they are a very expensive solution, NVidia is limited to 2 streams unless you get an even more expensive "pro" card.

 

try reading through this thread for extra info - opinions! etc  https://emby.media/community/index.php?/topic/10723-gpu-transcoding-intel-quicksync-and-nvidia-nvenc/

[runtimesandbox 153]

Reviving an old thread as I have started looking in to this again.. 

 

Has anyone had experience with the NVENC side of things? I am looking at getting a 1050ti as it can handle two 4k HVEC transcodes at a time and is fairly cheap.

 

What happens if 2 people are transcoding a 4k film and someone starts watching a 1080p film? Does Emby have the logic to use the CPU in certain circumstances for example if the GPU is unavailable? 

I noticed that using the built in intel GPU to do quick sync jobs had problems with some video files and would also take a lot longer to start playing that using the normal CPU transcoding. Would this likely be the same on an nvidia card?

 

I have a VM with 12 cores assigned which can just about handle one 4k transcode (transcoding back to 4k for a TV that couldn't handle the high bit rate file). This isn't a sustainable solution as it meant no other content would play properly at this time. 

Edited July 18, 2018 by spudd

[Luke 37238]

Yes we do have the logic to automatically switch to CPU.

[runtimesandbox 153]

Thanks for the response Luke. 

 

If more than one card is added to the system (for example 2 x 1050 ti) can Emby Utilize both cards so the total would be 4 4k transcodes? 

Is there anything else I should be looking out for?

[runtimesandbox 153]

So tonight i tried this on a nvidia 1080 with the latest drivers on a windows 10 PC. I enabled the nvenc like so:

 

 

Originally it still ran using only CPU (task manager showed very high cpu usage and no gpu usage) -surprisingly played back fine even when transcoding to 4k supported by the TV.

 

I updated to the latest graphics drivers and rebooted and it did start using the gpu however it stuttered, replayed frames every few seconds and kept going to a green screen. Am I missing something here?

I have hardware encoding ticked, task manager showed the GPU at about 25% for both decoding and encoding. 

 

Following on from this, it would be really nice if the "stats for nerds" could show what the server is using to transcode the file, either CPU or GPU.

Edited July 19, 2018 by spudd

[Guest asrequested]

Try disabling decoding

[Luke 37238]

Thanks for the response Luke. 

 

If more than one card is added to the system (for example 2 x 1050 ti) can Emby Utilize both cards so the total would be 4 4k transcodes? 

Is there anything else I should be looking out for?

 

It's pretty automatic. ffmpeg will decide which one to use, so yes i imagine it will be fine.

[Waldonnis 148]

So tonight i tried this on a nvidia 1080 with the latest drivers on a windows 10 PC. I enabled the nvenc like so:

 

Originally it still ran using only CPU (task manager showed very high cpu usage and no gpu usage) -surprisingly played back fine even when transcoding to 4k supported by the TV.

 

I updated to the latest graphics drivers and rebooted and it did start using the gpu however it stuttered, replayed frames every few seconds and kept going to a green screen. Am I missing something here?

I have hardware encoding ticked, task manager showed the GPU at about 25% for both decoding and encoding. 

 

Following on from this, it would be really nice if the "stats for nerds" could show what the server is using to transcode the file, either CPU or GPU.

 

An ffmpeg log would be very helpful to see what's going on.  I can think of a few reasons why that may be happening, but can't postulate without knowing what ffmpeg was being told to do and seeing any errors/warnings that might have been logged.

[runtimesandbox 153]

Attached the ffmpeg logs - two with debugging turned on and one without.

 

This is the GPU utilisation when both hardware decoding and encoding are enabled

 

CPU is roughly 20% 

 

This is the GPU and CPU usage when only hardware encoding is enabled

 

 

When using only hardware decoding there was no artifacting or green screens but it would buffer roughly ever 10 seconds

 

The emby dashboard states Transcoding (67 fps) - 62.5 Mbps ts h264 ac3

ffmpeg-transcode-decode-debug.txt

ffmpeg-transcode-decode-encode-debug.txt

ffmpeg-transcode-decode-encode-nodebug.txt

Edited July 19, 2018 by spudd

[Waldonnis 148]

Okay, looking at the log with both encoding and decoding enabled....it's as I feared.  It's trying to transcode to 2160p 8-bit h.264.  The card can definitely do it (upper limit on the encoder I think is 8k), but decoding 2160p h.264 is iffy on most playback devices and they can't handle that resolution with h.264.  Some are smart enough to say they can't do it, but others try and you get all kinds of weird behaviour because the decoder just can't keep up with the data or just aren't designed to process more than 1080p-sized frames (or can't downscale fast enough).  PCs (and HTPCs) can generally deal with 2160p h.264, but they're about the only consumer-level things that I know of that can.  To make matters worse, it's reducing the gamut without any tonemapping...read: it's doing 10->8bit without a proper conversion.  Even if it played properly, I wouldn't be surprised if it looked either too dark or washed out as a result.

 

Really, you probably do not want to transcode 2160p ("4k") HEVC "HDR" (HEVC Main10) files to anything but HEVC Main10...and even that is tricky right now as ffmpeg doesn't know how to carry the HDR metadata over from the source file (it has to be specified manually, which is a pain).  You can do HDR to 8bit (so-called "SDR"), but it requires a process called tonemapping, where the expanded colourspace of "10bit/HDR" is mapped onto the smaller 8bit colourspace.  Unfortunately, Emby doesn't appear to know how to do that yet.  Doing it "right" is a non-trivial process; doing it "well enough" is much easier, but there are still different methods that would need to be picked from.  Lastly, as noted above, most devices only support 2160p resolutions for specific codecs, often only HEVC and maybe VP9 as well, so it would be prudent to downscale for compatibility if h.264 is used.

 

Knowing all of that, and that Emby really doesn't "do" 4k transcoding well enough yet, you really don't want to touch the video stream at all for UHD HDR rips (audio transcodes are fine,sub overlay/burn-in/bitrate reduction are not)...and just manually transcode them yourself if you need lower bitrates (which takes forever using software encoding and can't be done well with something like Handbrake properly yet due to a back-end limitation, IIRC).  I'm not sure if ffmpeg's wrapper supports metadata inclusion with the hardware encoders (like nvenc) yet, so that may be out as an option too....although I think rigaya's nvencc encoder-wrapper supports it these days, so you could try that if it interests you.

 

I've actually written special scripts just to read HDR metadata from and re-encode UHDs because of how much of a PITA the HDR metadata is to retrieve and "pass along" to the x265 encoder.  It can clearly be automated, but encoding HEVC is significantly harder to do compared to h.264 and without a hardware encoder, it would crush a lot of machines if it was available to use in something like Emby.  Even with hardware encoding, injecting the metadata and signalling required for HDR playback isn't well supported by the hardware encoders' front-ends yet, so you'd have to find a way to inject the proper messages into the video stream after the fact (not easy) or you may not get "HDR mode" playback at all.

 

TLDR: Transcoding UHD HDR is hard and Emby doesn't know how, so avoid doing it     It's not an answer that I'm happy about, but until 4k transcoding with HEVC/x265 is implemented in Emby (or at least tonemapping and forced downscaling if using h.264), there isn't much that can be done about it.

[runtimesandbox 153]

Thanks for the excellent detailed response! Looks like i need to hold out for the updated Samsung Tizen Emby app so that the audio streams get transcoded and the video file left alone (https://emby.media/community/index.php?/topic/56264-transcode-of-hevc-to-un65ks8000/)

 

As an interim, using just the CPU seems to work and the transcoded files play back fine but it really does hammer the server (12 cores at 95%). 

[Luke 37238]

Well, even when it does have to transcode, we probably shouldn't transcode 4k to 4k. If we did 4k to 1080p it should perform much better.

[BurntTech 6]

I'm in the same boat of looking at solving 4k transcoding. I have movies that I want to watch in 4k but have unsupported audio for example on the xbox. The goal isn't to transcode but there currently some reasons requiring to do so. I fear the list will only continue like roku supported formats and other devices etc. If I was looking to be able to do 4k to 4k is there anyone that has a working config/hardware?

[Waldonnis 148]

I'm in the same boat of looking at solving 4k transcoding. I have movies that I want to watch in 4k but have unsupported audio for example on the xbox. The goal isn't to transcode but there currently some reasons requiring to do so. I fear the list will only continue like roku supported formats and other devices etc. If I was looking to be able to do 4k to 4k is there anyone that has a working config/hardware?

 

Transcoding only the audio track(s) should be fine.  It's only when resolution, HDR->non, or bitrate needs to be changed for the video stream that it gets hairy.  It's a tough problem to solve, since tonemapping isn't exactly an efficient process and there isn't a "one size fits all/best" way to do it for all content.  What makes it worse is that some HDR sources will look fine (or decent enough) when transcoded and gamut reduced to Rec.709 without tonemapping, so people think it's not a problem until they switch movies and the next one looks way too dark or washed out...then we're back to wondering why one movie worked while another one didn't.

 

Just to expand on why HDR->non is problematic...

 

My best analogy for why tonemapping is important (for HDR->non conversions) is attempting to convert a jpeg photo to a gif.  If the picture is simple and doesn't contain a wide range of shades and colours (think lone white table with studio lighting and single-colour backdrop) or is b&w, knocking it down to the 256-colours-max  palette of a gif won't be a problem and it'll look a lot like the jpg with maybe a tiny bit of banding despite having less colour depth (smaller palette).  If, however, the photo was of a sunset, the gif conversion will look very banded and bland...or may just look wrong entirely unless you carefully select which colours to keep in the palette.  The range of colours in the sunset alone would be so large that you just couldn't contain it within a smaller colour space, so you'd have to compromise somehow and even automated colour reduction methods may not work out well or would clip the very colours that make it look like a nice sunset.

 

Like photos, HDR movies are quite varied in how much of the available BT.2020 colourspace they use and how they use it.  Some movies only use tiny amounts of colour (usually highlights) that wouldn't be present in the smaller non-HDR colourspace (Rec.709) while most of the movie's colour range exists within Rec.709, so converting it may result in a video that seems a little flatter or slightly more drab, but still watchable.  Other movies, though, may stretch their entire colour range across BT.2020's coverage so that they use a lot more colours that are outside of Rec.709's range (like the sunset photo would compared to a 256-colour palette).  Just clipping the colours outside of Rec.709 would make the movie seem overly dark, washed out, or just weird looking.  Unless you map the larger colour range (HDR/BT.2020) so it fits within the smaller one (Rec.709) somehow, the results will always be hard to predict and could be terrible to watch compared to a properly graded BRD.  This mapping process is tonemapping.

 

That's pretty much why I rage about UHD->non transcodes.  To me, it's one of those "you think you want it, but you really don't" scenarios unless it can be done at least reasonably properly.

Edited August 1, 2018 by Waldonnis

[BurntTech 6]

I might need to chase down why playing a 4k movie that has reason for transcoding as unsupported audio chews through an Core i7 to 100% and gives about 12 fps video.

[Guest asrequested]

I might need to chase down why playing a 4k movie that has reason for transcoding as unsupported audio chews through an Core i7 to 100% and gives about 12 fps video.

Post the transcode log, and we can take a look.