GPU Transcoding (Intel QuickSync and nVidia NVENC)

[Waldonnis 148]

Got the PM, thank you!  Looks like the 970 hybrid encoder is reporting the same support, which seems odd at first, but makes sense to me (same code in ffmpeg; driver or hardware probing will probably yield more info, but it's beyond the scope of this).

 

I also tried a few tests locally on the hybrid encoder with my 970...same results that you're getting.  It seems that the nvenc hevc encoder is busted for several pixel formats (notably the two 10bit ones in the list, although I don't think converting another format to p010le is supported anyway) and returns the same error for me that you were getting.  Looks like a bug in ffmpeg (or a driver bug, not sure which).  I suppose it could be just unfinished or the cutting of yuv420p10le out of the original patch removed something important.  I did find a thread in Zeranoe's forum that has an ffmpeg binary that was compiled with the original patch.  It's dying with the same error, though, so either something changed in ffmpeg that created a regression or there's a driver bug.

 

I'll keep poking at it, since it's rather disappointing to see this happening.  My primary reasons for looking into Pascal cards are more related to hardware decoding than encoding, but it would be nice to have the hardware encoder working with ffmpeg for quick sample generation or preview encodings.

[Waldonnis 148]

Just a quick note after some further work on this:
 
It was time to do my weekly ffmpeg build anyway, so I played with it a little more.  It's not often that I run across issues with ffmpeg, so I totally forgot about the loglevel options before (and I was tired *grin*).  Running a test with loglevel 56, I got this (trimmed a bit):
 
 

$ ffmpeg -i yuv420p10le.mkv -vcodec "hevc_nvenc" -loglevel 56 bar.mkv
 
...(TRIMMED)...
 
[graph 0 input from stream 0:0 @ 0000000000377b60] w:3840 h:2160 pixfmt:yuv420p10le tb:1/1000 fr:30000/1001 sar:0/1 sws_param:flags=
2
Incompatible pixel format 'yuv420p10le' for codec 'hevc_nvenc', auto-selecting format 'p010le'
[format @ 0000000000514720] compat: called with args=[p010le]
[format @ 0000000000514720] Setting 'pix_fmts' to value 'p010le'
[auto-inserted scaler 0 @ 0000000000514b20] Setting 'flags' to value 'bicubic'
[auto-inserted scaler 0 @ 0000000000514b20] w:iw h:ih flags:'bicubic' interl:0
[format @ 0000000000514720] auto-inserting filter 'auto-inserted scaler 0' between the filter 'Parsed_null_0' and the filter 'format'
[AVFilterGraph @ 0000000000387860] query_formats: 4 queried, 2 merged, 1 already done, 0 delayed
[auto-inserted scaler 0 @ 0000000000584600] picking p010le out of 7 ref:yuv420p10le alpha:0
[auto-inserted scaler 0 @ 0000000000584600] w:3840 h:2160 fmt:yuv420p10le sar:0/1 -> w:3840 h:2160 fmt:p010le sar:0/1 flags:0x4
[hevc @ 00000000005848e0] nal_unit_type: 32(VPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 00000000005848e0] Decoding VPS
[hevc @ 00000000005848e0] Main 10 profile bitstream
[hevc @ 00000000005848e0] nal_unit_type: 33(SPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 00000000005848e0] Decoding SPS
[hevc @ 00000000005848e0] Main 10 profile bitstream
[hevc @ 00000000005848e0] Decoding VUI
[hevc @ 00000000005848e0] nal_unit_type: 34(PPS), nuh_layer_id: 0, temporal_id: 0
[hevc @ 00000000005848e0] Decoding PPS
[hevc @ 00000000005848e0] nal_unit_type: 39(SEI_PREFIX), nuh_layer_id: 0, temporal_id: 0
[hevc @ 00000000005848e0] Decoding SEI
[hevc @ 00000000005848e0] Skipped PREFIX SEI 5
[hevc_nvenc @ 000000000038c780] Loaded Nvenc version 7.1
[hevc_nvenc @ 000000000038c780] Nvenc initialized successfully
[hevc_nvenc @ 000000000038c780] 1 CUDA capable devices found
[hevc_nvenc @ 000000000038c780] [ GPU #0 - < GeForce GTX 970 > has Compute SM 5.2 ]
[hevc_nvenc @ 000000000038c780] 10 bit encode not supported
[hevc_nvenc @ 000000000038c780] No NVENC capable devices found
[hevc_nvenc @ 000000000038c780] Nvenc unloaded
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> hevc (hevc_nvenc))
Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[AVIOContext @ 000000000038d180] Statistics: 0 seeks, 0 writeouts
[AVIOContext @ 0000000000572fe0] Statistics: 32768 bytes read, 0 seeks

 As you can see, the increased logging showed a couple of real errors (the bit about the pixel format being incompatible and "10 bit encode is not supported") rather than the awful, generic error messages later in the output.  In my case, this would be accurate as my hardware can't encode or decode HEVC Main10, but Pascal should.  The mention of p010le concerned me a bit, mostly because that pixel format can only be used as an "input" (can't convert *to* p010le in NVidia's implementation, from what I've read), and even specifying the correct pixel format explicitly didn't change anything.
 
Can you try a manual transcode similar to mine and PM the output to me?  I'm not sure where the (correct, in my case) messages are coming from, but I suspect it's ffmpeg rather than the driver.  It also complains about yuv44416le not being supported despite being listed in the chroma sampling (pixel format) list, and even the NVidia header file hints at support for it.

 

Best I can guess, it's just unfinished work.  To be complete, I pulled a static 3.2.2 release build in case more recent changes broke it, but it too failed the same way.  I may compile NVidia's sample encoder just to see if that works, mostly out of curiosity.  It wouldn't be suitable for Emby's use, but it may at least eliminate the driver as a potential problem.

[Luke 42077]

Interesting that you had to bump the log level all the way up to see that.

[Waldonnis 148]

Interesting that you had to bump the log level all the way up to see that.

 

Agreed.  I probably didn't need to go quite as high, but better to see info that I can't use than miss something I wanted to see.  Some of that definitely didn't show at lower loglevels, though.  I've been looking over the nvenc bits in the ffmpeg code as well, but just to learn more about the implementation than to try to fix it...I have enough projects already 

[bxsteez 3]

So I upgraded to the latest stable server and the videos that were giving me errors are no longer giving me errors.  The playback is extremely choppy for some reason but it is transcoding and it is using the GPU.  So that is really good news!  Thought i'd report back on that.  I have not changed ffmpeg from the last time.

[Luke 42077]

Thanks for reporting back.

[jscoys 147]

Hello all! I activated NVENC with my GeForce TI 750 for several months now and it works pretty well BUT there is a little problem in the playback when seeking, and I think it's linked to the way Emby handles FFMPEG. I'm using the version Zeranoe version build from the 12/12. Here is what to do to reproduce the issue :

- launch from the beginning a movie and request a transco (put 240p for instance)

- let the video begins (few seconds) then seek to the middle of the movie

- watch. After a couple of minutes, 4 minutes exactly for me, the video will loop, so will return to the position you selected at the seek before

- the video will play correctly after that

 

So, we have a constant loop when seeking, happening one time.

 

The other problem is that the timeline is not synced after that loop, so when you seek after that it doesn't position you to the right time in the timeline.

 

Is another person having the problem?

 

 

Sent from my iPhone using Tapatalk

[Luke 42077]

And you don't see that with CPU transcoding?

[bxsteez 3]

So I upgraded to the latest stable server and the videos that were giving me errors are no longer giving me errors.  The playback is extremely choppy for some reason but it is transcoding and it is using the GPU.  So that is really good news!  Thought i'd report back on that.  I have not changed ffmpeg from the last time.

And today I updated to the latest 3.1.1.0 and it no longer works again. lol. 

[Luke 42077]

And today I updated to the latest 3.1.1.0 and it no longer works again. lol. 

 

In that case it probably has nothing to do with the server version and is something else entirely, and probably only experienced under certain conditions. You are associating it with the update because that seems to be the only thing different, but today's update was purely about resolving a couple small issues with 3.1.0. As always - how to report a problem.

[Luke 42077]

Hello all! I activated NVENC with my GeForce TI 750 for several months now and it works pretty well BUT there is a little problem in the playback when seeking, and I think it's linked to the way Emby handles FFMPEG. I'm using the version Zeranoe version build from the 12/12. Here is what to do to reproduce the issue :

- launch from the beginning a movie and request a transco (put 240p for instance)

- let the video begins (few seconds) then seek to the middle of the movie

- watch. After a couple of minutes, 4 minutes exactly for me, the video will loop, so will return to the position you selected at the seek before

- the video will play correctly after that

 

So, we have a constant loop when seeking, happening one time.

 

The other problem is that the timeline is not synced after that loop, so when you seek after that it doesn't position you to the right time in the timeline.

 

Is another person having the problem?

 

 

Sent from my iPhone using Tapatalk

 

What app are you seeing this in?

[jscoys 147]

What app are you seeing this in?

I tried with IOS app. Not happened with CPU transco. If you want I can try with Emby Theatre tonight after work and give you the transco log.

 

 

Sent from my iPad using Tapatalk

Edited December 22, 2016 by jscoys

[bradford 9]

It took me awhile to get the proprietary Nvidia drivers passed through to my LXC container, and then another while to build ffmpeg to support NVENC. I'm not happy it took half of my week off, but I am happy it works (I'll do a write-up later and link it in this thread). 

 

I have noticed a HUGE performance increase transcoding. I tested it transcoding a 23Mb 1080p source to 1Mb and 21Mb. The source is a fantastic rip of Enterprise, which has great film grain. What I immediately noticed was how poor the nvenc encoding quality is.

 

Has anyone else noticed a reduction in quality when they switched to nvenc?

 

Also, does the H264 setting in the transcoding options relate at all when using h264_nvenc?

[bradford 9]

And today I updated to the latest 3.1.1.0 and it no longer works again. lol. 

I built from the 3.1 release and nvenc is working for me. Did you build it yourself? What config settings did you use?

[CJ02 0]

just picked up a nvidia 1050 card to use with my emby server which is installed on ubuntu 16.04.

nvidia driver is installed via ppa (nvidia-375)

transcoding acceleration is set to nvidia (nvenc) in emby server 

however ffmpeg does not use nvenc for h264 and h265 videos when transcoding. 

 

the following is output from command ffmpeg -encoders | grep 264, there seems to be missing --enable-nvenc in the configuration, is there a different version of ffmpeg for nvenc to work? I am currently using the one comes with emby in /var/lib/emby-server/ffmpeg/20160215

 

ffmpeg version N-52954-g295de3e-static http://johnvansickle.com/ffmpeg/ Copyright © 2000-2016 the FFmpeg developers

  built with gcc 5.3.1 (Debian 5.3.1-8) 20160205

  configuration: --enable-gpl --enable-version3 --disable-shared --disable-debug --enable-runtime-cpudetect --enable-libmp3lame --enable-libx264 --enable-libx265 --enable-libwebp --enable-libspeex --enable-libvorbis --enable-libvpx --enable-libfreetype --enable-fontconfig --enable-libxvid --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libtheora --enable-libvo-amrwbenc --enable-gray --enable-libopenjpeg --enable-libopus --enable-libass --enable-gnutls --enable-libvidstab --enable-libsoxr --enable-frei0r --enable-libfribidi --disable-indev=sndio --disable-outdev=sndio --enable-librtmp --enable-libmfx --cc=gcc

  libavutil      55. 17.103 / 55. 17.103

  libavcodec     57. 24.102 / 57. 24.102

  libavformat    57. 25.100 / 57. 25.100

  libavdevice    57.  0.101 / 57.  0.101

  libavfilter     6. 31.100 /  6. 31.100

  libswscale      4.  0.100 /  4.  0.100

  libswresample   2.  0.101 /  2.  0.101

  libpostproc    54.  0.100 / 54.  0.100

 V..... libx264              libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (codec h264)

 V..... libx264rgb           libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 RGB (codec h264)

 

 V..... h264_qsv             H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (Intel Quick Sync Video acceleration) (codec h264)

[Waldonnis 148]

I have noticed a HUGE performance increase transcoding. I tested it transcoding a 23Mb 1080p source to 1Mb and 21Mb. The source is a fantastic rip of Enterprise, which has great film grain. What I immediately noticed was how poor the nvenc encoding quality is.

 

Has anyone else noticed a reduction in quality when they switched to nvenc?

 

I'm not sure what options Emby passes to ffmpeg in your case, but NVENC with a source with film grain is generally not that good to the discerning eye.  There are options that help, but whether or not you'd be happy with even that output is entirely subjective (I'm not, personally, but a lot of people don't even like seeing film grain and some don't even notice if it gets DNR'ed out).  The hardware encoders and ffmpeg interfaces to them in general are "good enough" for most people, but lack many of the analysis options that really help with level of detail preservation from some types of source material (lots of darker scenes, film grain, etc.).

 

One of my test films for encoding these days is (oddly enough) the newest American Godzilla movie simply because it's soooo dark throughout most of the movie that macroblocking can get really bad if the encoding settings aren't right.  I have several others that I use also for various reasons; high levels of motion, lots of grain, animated features, some high-contrast scenes, and so forth - each requiring a bit of tweaking to get to where I can accept the output.  It's really why I avoid on-the-fly transcoding in general - I'm just too picky and source material dictates my encoding settings, so a "one set of options fits all" just doesn't satisfy me.  Sadly, consumer video card hardware encoders aren't originally meant to do what we're doing with them anyway (although they're definitely useful if the output is okay to the viewer or for the task).  They tend to be designed with streaming/encoding speed and sources like live game play in mind (and are limited by how much they put into the silicon design-wise).

 

I'll yank a few recent logs out of the thread and peek at the options given to ffmpeg when using nvenc.  It's possible that there is room for improvement there since it's still so "new" and experimental in the grand scheme of things.  Any chance you can post the ffmpeg command line from your transcode log so I can see if there's anything that stands out in your case?

[bradford 9]

I'll yank a few recent logs out of the thread and peek at the options given to ffmpeg when using nvenc.  It's possible that there is room for improvement there since it's still so "new" and experimental in the grand scheme of things.  Any chance you can post the ffmpeg command line from your transcode log so I can see if there's anything that stands out in your case?

 

Sure, I've attached the log. I'm getting a respectable 112 fps from my passively cooled, 19 watt GT 710 for a 23Mbit source to a 21Mbit transcode. Can anyone confirm if the h264 settings in the transcoding settings even work? My testing showed no improvement when changing the quality level. I'd gladly lose 50% of my fps if I could get a transcode that looked good. But, previous to this, my 8 atom cores couldn't keep up with a single stream (might have been Proxmox cpu scheduling), so this is an improvement because it works, and FAST. 

 

I did a write-up on how I got GPU (NVENC) encoding working on a LXC-based install of emby-server using a custom-built ffmpeg. If anyone is interested, here it is for reference:

 

How To: GPU (NVENC) Accelerated FFMPEG in an LXC Container

 

enterprise-transcode.log.txt

Edited December 30, 2016 by bradford

[bradford 9]

just picked up a nvidia 1050 card to use with my emby server which is installed on ubuntu 16.04.

nvidia driver is installed via ppa (nvidia-375)

transcoding acceleration is set to nvidia (nvenc) in emby server 

however ffmpeg does not use nvenc for h264 and h265 videos when transcoding. 

 

the following is output from command ffmpeg -encoders | grep 264, there seems to be missing --enable-nvenc in the configuration, is there a different version of ffmpeg for nvenc to work? I am currently using the one comes with emby in /var/lib/emby-server/ffmpeg/20160215

 

ffmpeg version N-52954-g295de3e-static http://johnvansickle.com/ffmpeg/ Copyright © 2000-2016 the FFmpeg developers

  built with gcc 5.3.1 (Debian 5.3.1-8) 20160205

  configuration: --enable-gpl --enable-version3 --disable-shared --disable-debug --enable-runtime-cpudetect --enable-libmp3lame --enable-libx264 --enable-libx265 --enable-libwebp --enable-libspeex --enable-libvorbis --enable-libvpx --enable-libfreetype --enable-fontconfig --enable-libxvid --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libtheora --enable-libvo-amrwbenc --enable-gray --enable-libopenjpeg --enable-libopus --enable-libass --enable-gnutls --enable-libvidstab --enable-libsoxr --enable-frei0r --enable-libfribidi --disable-indev=sndio --disable-outdev=sndio --enable-librtmp --enable-libmfx --cc=gcc

  libavutil      55. 17.103 / 55. 17.103

  libavcodec     57. 24.102 / 57. 24.102

  libavformat    57. 25.100 / 57. 25.100

  libavdevice    57.  0.101 / 57.  0.101

  libavfilter     6. 31.100 /  6. 31.100

  libswscale      4.  0.100 /  4.  0.100

  libswresample   2.  0.101 /  2.  0.101

  libpostproc    54.  0.100 / 54.  0.100

 V..... libx264              libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (codec h264)

 V..... libx264rgb           libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 RGB (codec h264)

 

 V..... h264_qsv             H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (Intel Quick Sync Video acceleration) (codec h264)

 

I tried the static build suggested by the devs somewhere around here, and it looks like it should work with nvenc, however it would always segmentation fault when I tried to test it. I ended up building my own. I read somewhere around during my many hours or searching that NVENC can't be statically built, so maybe that's part of the problem... I don't know. 

 

Look at my signature for a guide on building ffmpeg for NVENC if you're feeling brave. 

Edited December 30, 2016 by bradford

[lorac 118]

I have a 1050 Nvidia card and it doesn't transcode at all just forces everything to direct play. If I switch to QSV there's no problems. Using the driver from 12.14 on Server 2016. I have tried various builds of Zeranoe but haven't seen any difference.

server-63618681322 - Copy.txt

[Luke 42077]

I have a 1050 Nvidia card and it doesn't transcode at all just forces everything to direct play. If I switch to QSV there's no problems. Using the driver from 12.14 on Server 2016. I have tried various builds of Zeranoe but haven't seen any difference.

 

Hi @@lorac, what issue are you trying to report? changing the hardware acceleration configuration will not affect whether transcoding happens or not.

[lorac 118]

@@Luke When using NVENC, transcoding doesn't work. If the player supports direct play / stream it will force that. So take Batman vs. Superman which is HEVC w/ DTS-HD. NVENC it won't play through the web but if I switch to QS it will. Bad Santa on the Roku 4 is another example. It has subs enabled by default which causes it to transcode if not disabled. The movie won't play if I switch to NVENC.

transcoding.7z

[Waldonnis 148]

Sure, I've attached the log. I'm getting a respectable 112 fps from my passively cooled, 19 watt GT 710 for a 23Mbit source to a 21Mbit transcode. Can anyone confirm if the h264 settings in the transcoding settings even work? My testing showed no improvement when changing the quality level. I'd gladly lose 50% of my fps if I could get a transcode that looked good. But, previous to this, my 8 atom cores couldn't keep up with a single stream (might have been Proxmox cpu scheduling), so this is an improvement because it works, and FAST. 

 

Thanks, and sorry for the delay.  I'm not seeing any explicit bitrate being set and even tried this on my end using a local sample clip (and on Windows - I haven't bothered recompiling lately on Linux, but it should be roughly the same minus any platform/driver differences) with similar output.  It seems that, in the absence of a "-b" argument, the NVENC encoder defaults to 2Mbit for both of us...which explains why it looks awful by comparison.  It seems that the -maxrate wasn't enough to tell the encoder to use a higher-than-default bitrate value and a -b:v is needed.

 

The max rate being set in the command line is ~10Mbit, so I added a bitrate argument for the video track ("-b:v 10000k") and it properly transcoded at a higher bitrate (my clip has no audio track anyway).  The resulting file compared to the source still looked like...well, bad...but it was a big improvement over the 2Mbit transcode.  Further adjusting the maxrate and bitrate values up to 21Mbit, the video looked even better.  It still suffered from some of the shortcomings of the hardware implementation, but it might be worth checking to see if it's good enough for your uses.

 

I just used a clip I've been using for HEVC testing - a high-motion car scene from BvS since it was handy, so it will probably do a bit better on less motion-y source material.  My usual grainy test clip wasn't really appropriate for this test for a few reasons (and the "grain" in BvS appears to have been added in post for an "archival feel" in this scene), so I would definitely recommend trying a manual transcode and double-checking the quality on your end with that same episode of Enterprise that you were trying before.  Oh, and transcoding speed was still very fast (~170 fps) as well, so it shouldn't take long to test out.

[pappaq 8]

Hey guys,

 

since the update to 3.1.2 the quality of all transcoding done via QVS is really bad. Over the android and iPhone app the server is a only transcoding to 1mbps and not to 5mbit, like it's set to. And even over the web app the quality is not good with 5mbits1080.

 

When QVS is disabled everything works normal as before.

 

My ffmpeg is the 20160410 version.

[CJ02 0]

I tried the static build suggested by the devs somewhere around here, and it looks like it should work with nvenc, however it would always segmentation fault when I tried to test it. I ended up building my own. I read somewhere around during my many hours or searching that NVENC can't be statically built, so maybe that's part of the problem... I don't know. 

 

Look at my signature for a guide on building ffmpeg for NVENC if you're feeling brave. 

 

I have had some time to try to build ffmpeg with nvenc support by using 3.1 release from git, there were some errors about libnppi.so.8.0 and libnppc.so.8.0 after ffmpeg was built, symlinking them from /var/local/cuda/lib64 solved it.

After configuring emby to use the newly built ffmpeg and nvenc acceleration, transcoding logs did show h264_nvenc and hevc_nvenc for both h264 and hevc content correctly. 

 

However, cpu usage in ubuntu was still between 50-80% during trancoding.  same as before when acceleration set to none with original ffmpeg (20160215). 

 

I have also tried git 3.2 release with the same result. 

[bxsteez 3]

I have been trying to get hardware acceleration working on a specific file for a while now and I've given up until I can build ffmpeg myself.  I'm pretty sure that is the issue.   I think it's my pixel format that is throwing the error.  I don't believe yuv420p10le is supported. 

 

Has anyone gotten this to work? It does play fine on the source computer but just not with ffmpeg.