Jump to content

GPU Transcoding (Intel QuickSync and nVidia NVENC)


witteschnitte

Recommended Posts

mjb2000

the changes i made are in the dev build i posted last night

 

Cheers Luke

 

I now have managed to get NVENC working for hardware encoding of h264 on certain nVidia graphics cards

 

I couldn't find a definitive list of cards but apparently these are Kepler and Maxwell architectures. So you'll have to check if your card supports NVENC technology (which is independent from CUDA technology). 

 

If you have a compatible card, before you try this out, make sure you update your drivers to the latest version from nVidia's website. (My initial false start with libnvenc was due to using out of date drivers).

 

For reference, my GTX660 does 1080p at around 270fps, dropping to around 210fps once ffmpeg is also encoding audio and resizing the video.

 

In order to try out libnvenc...

 

UPDATE: Look at the wiki for the latest instructions and links

  • Shutdown MediaBrowser Server
  • Update encoding.xml and set the h264 encoder to libnvenc
  • Download this ffmpeg.exe and replace it in your %APPDATA%\MediaBrowser-Server\config\[DATE] folder
  • Download my version of MediaBrowser.Api.dll and replace the existing one your %APPDATA%\MediaBrowser-Server\System

You'll notice you are replacing a MediaBrowser dll in this process. I built this DLL on dev version 3.0.5492.946, so I'm not sure if this will cause any issues in future versions (and remember this dll will get replaced if MB gets updated).

 

If a few people could test it out I will submit a pull request to Luke and we can see if we can get this added permanently.

 

NOTE to QuickSync users

 

The above ffmpeg includes both QSV the NVENC, so you can easily switch between the two.

My updated MediaBrowser.Api.dll also includes the required height and width rounding functions which are required for QuickSync compatibility, so this should help those who are waiting for the rounding to be released in the main dev build.

 

This has been possible thanks for the code I obtained here: https://github.com/Brainiarc7/ffmpeg_libnvenc

 

This uses the NVIDIA Video Codec SDK and CUDA Toolkit - Can someone review the license(s) and confirm that it is OK for ffmpeg.exe to be distributed as a binary, or what license details we might need to include? (I have no idea about this type of thing!).

Edited by mjb2000
  • Like 2
Link to comment
Share on other sites

Hi mjb2000,

 

Unfortunately it still does not Work for me on Version 3.0.5493.2287

 

C:\Users\Administrator\AppData\Roaming\MediaBrowser-Server\ffmpeg\20150110\ffmpeg.exe -fflags +genpts -i "http://localhost:8096/mediabrowser/videos/0defb4dea8ee4b724e85389d3a5b0aa9/stream?static=true&Throttle=true&mediaSourceId=0defb4dea8ee4b724e85389d3a5b0aa9&transcodingJobId=30bc1a078b3741c69b05f80c7e6520c4" -map_metadata -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 h264_qsv -preset 7 -b:v 5362000 -maxrate (5362000*1.2) -bufsize (5362000*2) -vsync vfr -level 4 -force_key_frames expr:gte(t,n_forced*6) -vf "scale=trunc(min(iw\,1920)/32)*32:trunc(min((iw/dar)\,1080)/32)*32:flags=fast_bilinear" -copyts -flags -global_header -codec:a:0 aac -strict experimental -ac 6 -ab 510000 -af "adelay=1,aresample=async=1" -hls_time 6 -start_number 0 -hls_list_size 0 -y "C:\Users\Administrator\AppData\Roaming\MediaBrowser-Server\transcoding-temp\streaming\c7aeafac20d4becf2c04e56e1363e7f7.m3u8"

ffmpeg version N-68994-g713e3bb Copyright © 2000-2015 the FFmpeg developers
  built on Jan 13 2015 13:06:15 with gcc 4.9.2 (Rev2, Built by MSYS2 project)
  configuration: --enable-libmfx --enable-iconv --arch=x86 --disable-debug --disable-shared --disable-doc --disable-w32threads --enable-gpl --enable-version3 --enable-runtime-cpudetect --enable-avfilter --enable-bzlib --enable-zlib --enable-decklink --enable-librtmp --enable-gnutls --enable-avisynth --enable-frei0r --enable-filter=frei0r --enable-libbluray --enable-libcaca --enable-libopenjpeg --enable-fontconfig --enable-libfreetype --enable-libass --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-libschroedinger --enable-libsoxr --enable-libtwolame --enable-libspeex --enable-libtheora --enable-libutvideo --enable-libvorbis --enable-libvo-aacenc --enable-libopus --enable-libvidstab --enable-libvpx --enable-libwavpack --enable-libxavs --enable-libx264 --enable-libx265 --enable-libxvid --enable-libzvbi
  libavutil      54. 16.100 / 54. 16.100
  libavcodec     56. 20.100 / 56. 20.100
  libavformat    56. 18.100 / 56. 18.100
  libavdevice    56.  3.100 / 56.  3.100
  libavfilter     5.  7.100 /  5.  7.100
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100
Input #0, matroska,webm, from 'http://localhost:8096/mediabrowser/videos/0defb4dea8ee4b724e85389d3a5b0aa9/stream?static=true&Throttle=true&mediaSourceId=0defb4dea8ee4b724e85389d3a5b0aa9&transcodingJobId=30bc1a078b3741c69b05f80c7e6520c4':
  Metadata:
    encoder         : libebml v0.7.9 + libmatroska v0.8.1
    creation_time   : 2010-01-18 14:26:10
  Duration: 02:37:49.47, start: 0.000000, bitrate: 7934 kb/s
    Stream #0:0: Video: h264 (High), yuv420p, 1280x534 [sAR 1:1 DAR 640:267], 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default)
    Stream #0:1(eng): Audio: dts (DTS), 48000 Hz, 5.1(side), fltp, 1536 kb/s (default)
    Metadata:
      title           : English  DTS 5.1 @ 1.5 Mbps
    Stream #0:2(eng): Audio: ac3, 48000 Hz, stereo, fltp, 192 kb/s
    Metadata:
      title           : English Comment AC3 2.0 @ 192kbps
[h264_qsv @ 04d33a20] MFXInit(): -3
Output #0, hls, to 'C:\Users\Administrator\AppData\Roaming\MediaBrowser-Server\transcoding-temp\streaming\c7aeafac20d4becf2c04e56e1363e7f7.m3u8':
    Stream #0:0: Video: h264, none, q=2-31, 128 kb/s, SAR 256:267 DAR 0:0, 23.98 fps (default)
    Metadata:
      encoder         : Lavc56.20.100 h264_qsv
    Stream #0:1: Audio: aac, 0 channels, 128 kb/s (default)
    Metadata:
      encoder         : Lavc56.20.100 aac
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_qsv))
  Stream #0:1 -> #0:1 (dts (dca) -> aac (native))
Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height

 

 

 

I updated the API.dll but your link to the latest ffmpeg is broken.

 

/Jacob

Edited by mylle
Link to comment
Share on other sites

mjb2000

your link to the latest ffmpeg is broken.

 

Whoops! - I occidentally left it as a draft, so it worked for me because I was logged in to GitHub. I have now fixed the link in the post above.

 

Unfortunately it still does not Work for me on Version 3.0.5493.2287

 

That is strange, everything I can see there looks to be correct. Can you try two things...

 

1) Enable debug logging in the transcoding settings and attach the results as a file (to save space for people reading the forum).

 

2) Run the following in a command windows (Start > All Programs > Accessories > Command Prompt)

CD %APPDATA%\MediaBrowser-Server\ffmpeg\20150110
ffmpeg -t 30 -i "D:\PathTo\MyVideo\MyVideo.mkv" -map 0:0 -c:v h264_qsv -b: 800000 -s 640x480 -report -loglevel debug QSV-Test.mkv

This should encode the first 30 seconds of the video.

 

Can you also post a copy of the encoding log that will be generated (The log file and QSV-Test.mkv output will be in the same folder as ffmpeg.exe)

Edited by mjb2000
Link to comment
Share on other sites

 

I couldn't find a definitive list of cards but apparently these are Kepler and Maxwell architectures. So you'll have to check if your card supports NVENC technology (which is independent from CUDA technology). 

 

 

 

I have done a bit of digging to try and find out what cards support NVENC and came across the below 

 

 

SUPPORTED NVIDIA GPUS FOR NVENC SDK

 

Source - https://developer.nvidia.com/nvidia-video-codec-sdk#gpulist

 

I hope this is of some help 

Edited by EdR883
  • Like 1
Link to comment
Share on other sites

mjb2000

I have done a bit of digging to try and find out what cards support NVENC and came across the below 

 

 

I hope this is of some help 

 

Cheers EdR

 

Have you got one of these cards? Have you been able to test it out? I've updated the Wiki with the details on how to get it working for both QuickSync and NVENC

Link to comment
Share on other sites

gandharva

This is also a good summary regarding NVENC: http://blog.medialooks.com/814EAo

 

Most relevant stuff:

 

 

Since version 340.52 of Nvidia's drivers CUDA is disabled and replaced with NVENC. It is still possible to use CUDA via the old drivers with MPlatform SDK, but we don't recommend doing so - it's best to upgrade to NVENC.

 

...

Supported H.264 profiles

The first generation of NVENC (Kepler-based GPUs) supports H.264 high-profile (YUV420, I/P/B frames, CAVLC/CABAC), H.264 SVC Temporal Encode VCE and Display Encode Mode (DEM). The second generation (Maxwell-based GPUs), also supports HiP444 profile (YUV4:4:4, predictive lossless encoding).

 

Nvidia models with NVENC support

The following product lines have NVENC on board:

  • NVIDIA GeForce
  • NVIDIA Quadro K4000 and above
  • NVIDIA Tesla K10 and above

According to NVENC specs there is a limit for number of simultaneous encoding threads. GeForce cards are limited by 2 streams and Quadro are limited by 6 streams. It means that you are unable to start 3 MWriter objects with NVENC encoding at the same time on a single machine if it has GeForce graphic card.

  • Like 1
Link to comment
Share on other sites

mjb2000

This is also a good summary regarding NVENC: http://blog.medialooks.com/814EAo

 

Thanks for joining us gandharva. Have you been able to get this working on your server? I'm looking for people to test this out so I can give Luke some updates to apply to the MediaBrowser source code.

  • Like 1
Link to comment
Share on other sites

gandharva

I have a GTX460 at home. Will check this in about 2 hours and report back. Thanks for all your work on hw transcoding!

Link to comment
Share on other sites

mjb2000

I have a GTX460 at home. Will check this in about 2 hours and report back. Thanks for all your work on hw transcoding!

 

Hmm - I'm not sure the GTX460 has NVENC. Although the link above mentions "GeForce" this refers only to the Kepler and Maxwell chipsets, and I think the GTX460 is a little too old for that.

 

Compiling NVENC in to ffmpeg required the use of the CUDA SDK, so perhaps there is a fallback to use CUDA instead of NVENC, so it may still be worth a try - let us know how you can on. Remember to update your drivers! :)

Link to comment
Share on other sites

Whoops! - I occidentally left it as a draft, so it worked for me because I was logged in to GitHub. I have now fixed the link in the post above.

 

 

That is strange, everything I can see there looks to be correct. Can you try two things...

 

1) Enable debug logging in the transcoding settings and attach the results as a file (to save space for people reading the forum).

 

2) Run the following in a command windows (Start > All Programs > Accessories > Command Prompt)

CD %APPDATA%\MediaBrowser-Server\ffmpeg\20150110
ffmpeg -i "D:\PathTo\MyVideo\MyVideo.mkv" -map 0:0 -c:v h264_qsv -b: 800000 -s 640x480 -report -loglevel debug QSV-Test.mkv

This should start encoding the video (press CTRL+C to cancel the encoding).

 

Can you also post a copy of the encoding log that will be generated (The log file and QSV-Test.mkv output will be in the same folder as ffmpeg.exe)

 

Here you go!

transcode-fe81547c-1da1-402a-a207-99c1b7853994.txt

ffmpeg-20150116-163930.log

Link to comment
Share on other sites

mjb2000

Here you go!

 

Thanks mylle

 

Looking at this Intel Quicksync doesn't get very far at all before it fails. The MFX Error -3 apparently means unsupported, so can you check a few things...

  • What CPU model number do you have?
  • Do you have the latest Intel display drivers installed (not old drivers or default drivers provided by Microsoft).
  • Do you have any other graphics cards in your computer (such as nVidia or ATI?)
  • What version of Windows are you using?
  • Are you using the computer directly, or through remote desktop / VNC
Link to comment
Share on other sites

Cheers EdR

 

Have you got one of these cards? Have you been able to test it out? I've updated the Wiki with the details on how to get it working for both QuickSync and NVENC

 

After some more digging I dont think my card is supported 

 

It is a GeForce 7900 GTX, I think it is to old judging by what I have found 

Link to comment
Share on other sites

mjb2000

That's interesting...

 

Windows 8+ and Server 2012+ deals with RemoteDesktop in a sepcial way which should mean that QuickSync is available in a remote session, even though you're not really attaching a monitor to the graphics card. My understanding is that QS will not work under Windows 7 RD, since it uses a completely virutal graphics adapter for the user running the remote session - VNC and TeamViewer connections would work since you are controlling a session that is using the physical hardware.

 

This reminds me of some issues I was having a while back with my J1900 chip. Using the latest drivers under Windows 8 (which is probably more similar to Server 2012 than it is to Windows 7) completely stopped QuickSync from working.

 

It's probably worth trying to run this from the physical console if possible to rule out RD (but I doubt RD is the problem on your OS).

 

I know it sounds counter intuitive, but can you also try using an older version of the Intel driver. Looking back at my issue log it seems a driver from early 2014 seemed to work, whilst late 2014 failed. Looking at the Intel site, I could only find late 2014 and early 2015 for your chip, so it might be tricky to track down.

 

I will also get in touch with the original developers of QuickSync for ffmpeg and see if they've made any more progress with ironing out these issues (which I understand to come from the new MFX API in later drivers. Although I should mention, if in the future we can take advantage of some of the new API features, we should be able to improve performance and encoding quality thanks to new look-ahead mechanisms :)

Link to comment
Share on other sites

scanner50
mjb2000, on 15 Jan 2015 - 04:46 AM, said:

Whoops! - I occidentally left it as a draft, so it worked for me because I was logged in to GitHub. I have now fixed the link in the post above.

 

 

That is strange, everything I can see there looks to be correct. Can you try two things...

 

1) Enable debug logging in the transcoding settings and attach the results as a file (to save space for people reading the forum).

 

2) Run the following in a command windows (Start > All Programs > Accessories > Command Prompt)

CD %APPDATA%\MediaBrowser-Server\ffmpeg\20150110
ffmpeg -i "D:\PathTo\MyVideo\MyVideo.mkv" -map 0:0 -c:v h264_qsv -b: 800000 -s 640x480 -report -loglevel debug QSV-Test.mkv

This should start encoding the video (press CTRL+C to cancel the encoding).

 

Can you also post a copy of the encoding log that will be generated (The log file and QSV-Test.mkv output will be in the same folder as ffmpeg.exe)

 

Hi mjb2000, phenomenal work you are doing with this. Unfortunately, I am also experiencing failures, even though my CPU supports QS. It's an i3-4020y in a Dell Venue 11 Pro. I followed the steps in your wiki and tried to transcode using a Roku streaming stick, and it got stuck at "retrieving" and then it kicked back into the menu to play the file. Tried it twice with same result. I also followed your instructions above to produce further logs and files. Here is a link for all the relevant files: http://1drv.ms/17LBY44

Edited by rainking430
Link to comment
Share on other sites

mjb2000

Hi mjb2000, phenomenal work you are doing with this. Unfortunately, I am also experiencing failures, even though my CPU supports QS. It's an i3-4020y in a Dell Venue 11 Pro. I followed the steps in your wiki and tried to transcode using a Roku streaming stick, and it got stuck at "retrieving" and then it kicked back into the menu to play the file. Tried it twice with same result. I also followed your instructions above to produce further logs and files. Here is a link for all the relevant files: http://1drv.ms/17LBY44

 

Cheers RainKing

 

The verdict for you is better :)

 

QS does seem to be working, I wonder it the reason you didn't get an output.mkv is because I said to CTRL+C - I probably should have said to press Q to quit ffmpeg. Try it without the "-loglevel debug" and you should see a status message indicating you are encoding correctly.

 

The reason MB might not be working is because of the factor of 2 roundung "/2)*2" - Update you MediaBrowser.Api.dll with my version (above) and you should start seeing factor or 32 rounding "/32)*32"

 

Let me know how you get on

Link to comment
Share on other sites

scanner50

The reason MB might not be working is because of the factor of 2 roundung "/2)*2" - Update you MediaBrowser.Api.dll with my version (above) and you should start seeing factor or 32 rounding "/32)*32"

 

Let me know how you get on

 

Haha, guess I didn't follow your wiki that closely after all. Unfortunately replacing it still didn't work and produced the same behavior on the Roku. I've replaced the files in that link I gave you with the latest log. Thanks for the help.

Link to comment
Share on other sites

mjb2000

Just seen you log file. It mentions "Device is busy" - This is not something I've seen before and don't know how to improve on the error handling unfortunately. Could the device actually be busy with something else? What other software are you running (Live broadcast, screen sharing, Chromecast streaming etc?)... Also the same questions I asked to mylle will apply:

  • What CPU model number do you have?
  • Do you have the latest Intel display drivers installed (not old drivers or default drivers provided by Microsoft).
  • Do you have any other graphics cards in your computer (such as nVidia or ATI?)
  • What version of Windows are you using?
  • Are you using the computer directly, or through remote desktop / VNC

For some more testing of QuickSync on your machine, I have found q264 to work reliably most of the time (but it can't be used as a replacement to ffmpeg unfortunately). Can you download and install q264 and try to encode your video using q264 and let me know how it goes.

 

From the documentation, you'll probably need to run a command such as:

q264 -i "D:\Path\To\MyVideo.mkv" -nFrames 1000 -v -o test.mp4

Let me know if this works for you - Keep an eye on your GPU usage - it should spike, whilst CPU usage shouldn't be too high.

Link to comment
Share on other sites

scanner50

Ok, for some reason q264 would not work, throwing an error that MSVCP120.dll is missing from my computer. So I went back and tried ffmpeg again with no deug logging and quitting using Q. It ran successfully for a few seconds and then stopped. Pressing Q did nothing so I had to close the window. The test video it created freezes 10 seconds in. I uploaded the appropriate files to my link above.

 

Oh and to answer your questions:

 

  • What CPU model number do you have? i3-4020Y
  • Do you have the latest Intel display drivers installed (not old drivers or default drivers provided by Microsoft). Not sure. I've tried to find them in the past and couldn't. It's a mobile processor so that may be why.
  • Do you have any other graphics cards in your computer (such as nVidia or ATI?) No, it's a hybrid device.
  • What version of Windows are you using? Windows 8.1.1 x64
  • Are you using the computer directly, or through remote desktop / VNC? Directly
Edited by rainking430
Link to comment
Share on other sites

mjb2000

It makes sense that the test video freezes 10 seconds in if that is roughly where the ffmpeg encode it stopped. (If you add -t 30 to the ffmpeg command line you can force it only to encode the first 30 seconds).

 

I can't quite work out what might be going. looking again at your logs it says Media SDK API version 1.10 which should indicate you have reasonably recent driver - you're right, obtaining drivers from Intel is difficult, they normally have to be obtained from the laptop manufacturer.

 

It does seem strange that it is working with the direct ffmpeg call but not as part of the MediaBrowser function. This probably is the limit of my ability to resolve this - as I've mentioned before I don't really understand how all this stuff works, I'm just going on bits I've read online and combined a couple of projects to get things working (to some extent).

 

Sorry it's not working for you at the moment, keep an eye on the thread and hopefully future updates or issues from other users will point us in the right direction as to what might be wrong.

Link to comment
Share on other sites

scanner50

Thanks for trying to help, anyway, really appreciate it. Here is one update which is kind of weird, I tried playing it through my browser, and sure enough it seems to be transcoding just fine with a little less CPU usage. But I tried a Windows 8.1 MB app and it says "the video failed to play" which is strange. It didn't do that before I made the QS changes. I will try the Roku device again when I have the chance.

 

Has me wondering: maybe apps have to be specifically updated with these tweaks in mind? Has anyone else had success with Roku devices and this tweak?

 

UPDATE: Nope, Roku still no worky.

Edited by rainking430
Link to comment
Share on other sites

mjb2000

It will be working in the browser since it uses libvpx for this (webm) not h264 video.

 

Just thinking, in case your issue is related to scaling or frame size, can you double check streaming a video that has a source of 1920x1080?

Link to comment
Share on other sites

scanner50

Yeah, tried that, same result. I realized I was using .ts files so reencoded them to .mkv with mkvmerge to see if that would help anything. Nope. Think I'll just give up on it for now and revisit it later.

 

Oh and in regard to:

 

It will be working in the browser since it uses libvpx for this (webm) not h264 video.

 

would ffmpeg trigger if webm is being used? I was trying it through IE11 and ffmpeg was definitely doing the transcoding.

Edited by rainking430
Link to comment
Share on other sites

mjb2000

Yep - ffmpeg will be used to convert whatever video you have as the source to a format suitable for streaming. Consider a very high bitrate h264 video with 4 languages of DTS audio and 2 subtitle tracks - IE11, or any web browser can't handle that (anyway, why send all the data if you're only going to listen to one audio channel at a time) - So MBB automatically chooses the correct codec that FFMPEG should encode to.

 

Interestingly, if your source is a h264 video and you are streaming to a device like a roku, as long as you have got enough bandwidth, MB will ask ffmpeg will simply "copy" the original datastream for the video, but may still do some audio encoding. However if bandwidth is insufficient, ffmpeg will need to re-encode the h264 video to a lower bitrate (possibly resizing it too). ffmpeg is always involved though, because at the very least it provides a mechanism for creating streamable files, either as an WEBM, MP4 or HLS data stream. It also means that you can skip ahead on your remote device without having to download all the content in advance (ffmpeg will start encoding from a later point in the source file).

 

There is a lot of clever stuff that is happening behind the scenes that @@Luke and friends have put in to make things as seamless as possible across all devices. Different sources, available bandwiths and playback devices will mean different ffmpeg settings, but it should always find the right method to get the video to you face :)

 

If you're interested, take a look at your transcode log files, at the top you will see the ffmpeg command line that is executed. Look for -c:v codecname and -c:a codecname and you'll see the codecs ffmpeg is converting to, as well as a bunch of other settings.

Edited by mjb2000
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...