GPU Transcoding (Intel QuickSync and nVidia NVENC)

[Luke 42077]

@@anthonws, what is the specific issue on the command line? Because a lot of users use VA API here, so if the command line looks OK then it's probably the Docker container that needs adjustment.

[Gerrit507 24]

The server checks the build for support and prints out the decoders and encoders that it finds. try restarting the server, then you'll see these tests near the beginning of the main server log.

 

With the ffmpeg git build it doesn't find any qsv decoders at all. With the version that comes with emby it only finds a 264_qsv decoder but no 265_qsv :/

serverlog-standard.txt

serverlog-git.txt

[Luke 42077]

Then I guess I stand corrected. I thought the builds on that website included all of those.

[anthonws 42]

@@anthonws, what is the specific issue on the command line? Because a lot of users use VA API here, so if the command line looks OK then it's probably the Docker container that needs adjustment.

 

First of all, my apologies for the n00biness

 

So, let me go step-by-step.

 

1. I'm using UnRAID 6.3.2 with i915 driver installed

2. I've installed Emby default docker image

3. Added to the extra parameters the following line: "--device /dev/dri:/dev/dri"

4. Here's the complete docker command line:

docker run -d --name="EmbyServer" --net="host" -e TZ="Europe/London" -e HOST_OS="unRAID" -e "TCP_PORT_8096"="8096" -e "APP_UID"="99" -e "APP_GID"="100" -v "/mnt":"/mnt":rw -v "/mnt/user/Apps/EmbyServer":"/config":rw --device /dev/dri:/dev/dri emby/embyserver
bf99fa49c78ae945248c0e340c3cb8491481a8e41bbe14436825fc5876bc2317

5. In the Server Dashboard I've set Transcoding to VAAPI and the VA API Device to the default "/dev/dri/card0"

6. FS Permissions are correctly set to allow any user/group access

7. When trying to play a movie and force transcoding, the movie does not play and I get the following error:

[AVHWDeviceContext @ 0x3979800] No VA display found for device: /dev/dri/card0.
[vaapi @ 0x30321e0] Failed to create a VAAPI device

So, my question was if the docker extra parameter is set correctly and/or if I need to change something else in UnRAID or Docker configuration to make it work.

 

Cheers,

anthonws.

[Gerrit507 24]

Then I guess I stand corrected. I thought the builds on that website included all of those.

Are there any other options I can try?

[Luke 42077]

Are there any other options I can try?

 

Well first, you need an ffmpeg build that supports the features you want to use.

[Gerrit507 24]

Well first, you need an ffmpeg build that supports the features you want to use.

@@Luke

 

Ok.. And where do I get that? It seems like hevc_qsv is under a license from Intel. Is the only way building it on my own?

 

EDIT: If I run ffmpeg -codecs it lists me hevc_qsv, so idk why emby isn't recognizing it...

 

EDIT2: Anyway why is emby using the h264 switch on hevc content, it should at least fallback to software decoding imho... Double checked that the ffmpeg version that comes with emby supports everything I need and it's still not recognizing the decoder...

 

EDIT3: Installed Windows 10. Exact same behavior. It plays fine and with hw-decoding  in the stock windows player.

Edited February 27, 2017 by Gerrit507

[Waldonnis 148]

I added the latest git version and it's not using qsv. Do I need to add something?

 

Also, why is the a ffmpeg and ffmpeg-10bit? Tried that also, but it seems to be 10bit only right?

 

Some of the underlying libraries used by ffmpeg require specific configuration options to support 10bit pixel formats.  It's possible to build an ffmpeg binary that supports 8 and 10bit, but some people don't bother doing so since it adds complexity to the build process or they have no need for it.

 

@@Luke

 

Ok.. And where do I get that? It seems like hevc_qsv is under a license from Intel. Is the only way building it on my own?

 

EDIT: If I run ffmpeg -codecs it lists me hevc_qsv, so idk why emby isn't recognizing it...

 

EDIT2: Anyway why is emby using the h264 switch on hevc content, it should at least fallback to software decoding imho... Double checked that the ffmpeg version that comes with emby supports everything I need and it's still not recognizing the decoder...

 

EDIT3: Installed Windows 10. Exact same behavior. It plays fine and with hw-decoding  in the stock windows player.

When you list the codec support with ffmpeg -codecs, does it show hevc_qsv in both decoders and encoders for hevc?  You can also try ffmpeg -encoders just to list the configured/detected encoders rather than the rest (Emby doesn't use hardware decoding, from what I can tell anyway).  Braswell should support HEVC Main (*not* Main10) hardware encoding, so it should find it okay assuming the system is reporting it properly (if it reports the presence of h264_qsv, hevc_qsv should be there as well).  There may be other limitations of Intel's or ffmpeg's encoder implementations (like resolution or pixel formats) that would break hardware encoding, but I don't know Braswell's specs well enough to comment on what they may be.

 

The reason it's specifying h264_qsv on the command line in the log is that it's trying to transcode the HEVC source to h.264.  I'm not seeing any resolution change, however, which may be why the encoder is choking.  This line in particular was odd, as it describes the output video stream:

Stream #0:0: Video: h264, none, q=2-31, 128 kb/s, SAR 1:1 DAR 0:0, 23.98 fps (default)

Normally, you'd see resolution and pixel format info there like this one from a different transcode log:

Stream #0:0: Video: h264 (h264_nvenc) (High) (H264 / 0x34363248), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 2000 kb/s, 23.98 fps, 1k tbn, 23.98 tbc

...so something's going wrong there, and I'm not even sure if Intel's implementation will handle 4k h.264 encoding (been too long since I tried, so I just don't remember the result). Since you mentioned browser playback, it's possible that the browser doesn't support HEVC playback, but has no resolution limitation reported, leading to this rather quirky transcode (hardly anything supports 4k h.264 playback, so I don't see this situation often).

 

Note that decoding and encoding are two different things.  Emby's hardware transcoding ffmpeg command lines that I've seen indicate that they use only software decoding, but use the hardware encoder for re-encoding the software-decoded stream to the codec supported by the client/app.  Also, not all clients/browsers support everything that the host hardware is capable of, so other limitations apply when video playback doesn't work as one would expect.

 

Unfortunately, I'm not sure where to grab a Linux ffmpeg binary that would support what you want, since I build my own and primarily boot to Windows for video stuff anyway.  Building your own is a pain, but may be worth it if you have need of any of the "non-free" encoders (it's why I build my own - I use libfdk_aac at times, but its license forbids redistribution).

[eTomm 6]

Hello @@Luke, I think I found some kind of bug. I have an mkv that with the latest nightly version of FFMPEG from Zeranoe doesn't play. Other videos play fines.

 

http://pastebin.com/nN5XmkVs

[Gerrit507 24]

 

Some of the underlying libraries used by ffmpeg require specific configuration options to support 10bit pixel formats.  It's possible to build an ffmpeg binary that supports 8 and 10bit, but some people don't bother doing so since it adds complexity to the build process or they have no need for it.

 

When you list the codec support with ffmpeg -codecs, does it show hevc_qsv in both decoders and encoders for hevc?  You can also try ffmpeg -encoders just to list the configured/detected encoders rather than the rest (Emby doesn't use hardware decoding, from what I can tell anyway).  Braswell should support HEVC Main (*not* Main10) hardware encoding, so it should find it okay assuming the system is reporting it properly (if it reports the presence of h264_qsv, hevc_qsv should be there as well).  There may be other limitations of Intel's or ffmpeg's encoder implementations (like resolution or pixel formats) that would break hardware encoding, but I don't know Braswell's specs well enough to comment on what they may be.

 

The reason it's specifying h264_qsv on the command line in the log is that it's trying to transcode the HEVC source to h.264.  I'm not seeing any resolution change, however, which may be why the encoder is choking.  This line in particular was odd, as it describes the output video stream:

Stream #0:0: Video: h264, none, q=2-31, 128 kb/s, SAR 1:1 DAR 0:0, 23.98 fps (default)

Normally, you'd see resolution and pixel format info there like this one from a different transcode log:

Stream #0:0: Video: h264 (h264_nvenc) (High) (H264 / 0x34363248), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 2000 kb/s, 23.98 fps, 1k tbn, 23.98 tbc

...so something's going wrong there, and I'm not even sure if Intel's implementation will handle 4k h.264 encoding (been too long since I tried, so I just don't remember the result). Since you mentioned browser playback, it's possible that the browser doesn't support HEVC playback, but has no resolution limitation reported, leading to this rather quirky transcode (hardly anything supports 4k h.264 playback, so I don't see this situation often).

 

Note that decoding and encoding are two different things.  Emby's hardware transcoding ffmpeg command lines that I've seen indicate that they use only software decoding, but use the hardware encoder for re-encoding the software-decoded stream to the codec supported by the client/app.  Also, not all clients/browsers support everything that the host hardware is capable of, so other limitations apply when video playback doesn't work as one would expect.

 

Unfortunately, I'm not sure where to grab a Linux ffmpeg binary that would support what you want, since I build my own and primarily boot to Windows for video stuff anyway.  Building your own is a pain, but may be worth it if you have need of any of the "non-free" encoders (it's why I build my own - I use libfdk_aac at times, but its license forbids redistribution).

 

wow, thank you for that comprehensive answer  

 

I basically want to achieve HEVC 8bit/10bit 4K to 1080p H264 transcoding, so all I need is hevc_qsv decoder. I've done it with the plex preview version and it worked, with 8bit at least, but they said that 10bit is not supported at the moment and then I found emby, which seems to be a better way to go if someone wants hw-decoding. I know that my N3150 is not capable of 10 bit HEVC but I get an Apollo Lake J3455 tomorrow The only problem is that, now with emby even 8 bit isn't working anymore, h264 4K works fine by the way.  Did I get that right, that the default ffmpeg of emby does not support 10bit and I would need to build one my own?

 

Anyway, I'll just give it a shot with the new board tomorrow and than I report my results here

[Waldonnis 148]

wow, thank you for that comprehensive answer

 

I basically want to achieve HEVC 8bit/10bit 4K to 1080p H264 transcoding, so all I need is hevc_qsv decoder. I've done it with the plex preview version and it worked, with 8bit at least, but they said that 10bit is not supported at the moment and then I found emby, which seems to be a better way to go if someone wants hw-decoding. I know that my N3150 is not capable of 10 bit HEVC but I get an Apollo Lake J3455 tomorrow The only problem is that, now with emby even 8 bit isn't working anymore, h264 4K works fine by the way.  Did I get that right, that the default ffmpeg of emby does not support 10bit and I would need to build one my own?

 

Anyway, I'll just give it a shot with the new board tomorrow and than I report my results here

 

Okay, I think I understand now and had your goal a bit backwards in my mind before.  Most people just want the encoding bit to be done on hardware, since it's the heaviest operation.  You probably still want to use the h264_qsv encoder to offload the encoding work, but given the limitations of the embedded/NUC-focused Intel CPU products, I can see why hardware decoding would be beneficial as well for something like HEVC (just making sure I understand the situation).

 

I'm not sure Emby's set up to do that, but we might be able to recommend an option to use hardware decoding to Luke/ebr; it's complicated and source codec dependent (not all codecs have corresponding hardware decoding support), but it may be doable; just need to nail down the options in a way that they can adapt to code and show a benefit for doing so.  I've run across some bugs in Intel's hardware decoding (driver-side) on some generations of CPUs (Sandy/Ivy Bridge in particular), so I've always been hesitant to recommend using it to anyone.  That being said, both your Braswell and Apollo Lake are newer and more likely to have any driver bugs addressed compared to the 4-5 year old chips - so you may not see any issues.  It's important to note as well that hardware decoding should be single-threaded, and I think ffmpeg still complains unless you specify that - just something to take into consideration when judging the benefits.

 

You can try some manual transcoding tests to make sure that the decoder is working well once you get an ffmpeg binary that works for this.  Still puzzling why it's not showing HEVC hardware support in the de/encoder lists if h.264 QSV is showing up, though, unless the driver/kernel just isn't reporting it.  Any chance you can PM me the output from the binaries you have for these commands:

ffmpeg -hwaccels
ffmpeg -codecs
ffmpeg -pix_fmts

I'm hoping to avoid having you build your own ffmpeg (I do it and it sucks unless you have it all scripted   ), so I want to see if it's even needed or if there's a problem elsewhere  The build configuration strings should give me a ton of info alone, but I want to double-check what the binaries are detecting as well hardware-wise..  The last command is just to check on pixel formats (they'll tell me more about if/how it's configured for 10bit support).  Luke can correct me here, but I don't think the "stock" ffmpeg that Emby includes would have vendor-specific hardware encoding support yet (nvenc and qsv, notably), and I seriously doubt that any distribution's ffmpeg packages include it either quite yet.

 

On a side note, vaapi may be an option since we're dealing with Linux.  I know that several have reported visual artifacts with it, but that was on the encoding side of things. Curious to see how the decoding side performs and if there are any artifact issues there as well.

[millsii 8]

I have recently changed over to use NVENC transcoding as it has provided an improvement with streaming to a high latency location (>300ms). This has been working quite well, but have noticed that the transcoding log does not seem to acknowledge the "h.264 preset." In the logs it always shows "-preset default." Is this normal behaviour for NVENC? Sample transcoding log attached.

NVENC transcode log.txt

[Waldonnis 148]

I have recently changed over to use NVENC transcoding as it has provided an improvement with streaming to a high latency location (>300ms). This has been working quite well, but have noticed that the transcoding log does not seem to acknowledge the "h.264 preset." In the logs it always shows "-preset default." Is this normal behaviour for NVENC? Sample transcoding log attached.

 

The NVENC encoders have their own preset values that don't always correspond to the values I see in that field on the web interface, so I'd assume it's just ignoring them.  Here's what the h264_nvenc encoder accepts for preset values:

  -preset            <int>        E..V.... Set the encoding preset (from 0 to 11) (default medium)
     default                      E..V....
     slow                         E..V.... hq 2 passes
     medium                       E..V.... hq 1 pass
     fast                         E..V.... hp 1 pass
     hp                           E..V....
     hq                           E..V....
     bd                           E..V....
     ll                           E..V.... low latency
     llhq                         E..V.... low latency hq
     llhp                         E..V.... low latency hp
     lossless                     E..V....
     losslesshp                   E..V....

The default's probably the best option out of that list for this type of transcoding, honestly, and there isn't much reason to override it.

 

Some observations on the presets listed:

 

For transcoding to files, I actually tried the 2-pass option once (slow), only to find out that it wasn't really 2-pass the way I was used to and didn't really have the same benefits as x264 2-pass encoding would (it was wacky).  The low-latency presets seemed to be more oriented towards video conferencing or live gaming streaming, since they looked to be trying to keep the frame sizes similar (nVidia app notes confirmed this observation; it can translate into a noticeable quality hit in some conditions).  I can't see a use for lossless for streaming or even normal file transcodes, so that wouldn't be worth using either.  "bd" is supposed to be more for Blu-Ray authoring, but it may actually work slightly better than "hq" or medium - I just haven't played with it enough to form an opinion yet.  The others (hp, hq) are mostly already covered by medium and fast, but the docs I last read hinted at some extra adaptive logic in medium/fast...I didn't look too deeply at that either, unfortunately, so I'm not sure what the differences really were.  I'll run some tests when I get some more free time, since you've got me curious now.

[millsii 8]

@@Waldonnis, thanks for the detailed response! The quality to me seems pretty good, so as you said the default preset does the required job.

 

As mentioned, I have moved to NVENC as it has made it possible to stream to my mate in South Korea (>300ms) for live TV. It seems to be due to NVENC (QSV does the same) having a stop start transfer profile in the router graph. ie large transfer spike, then nothing, then spike, then nothing etc. Whereas CPU transcoding produces a fairly consistent lower rate ongoing transfer. This consistent transfer rate (I presume of smaller files?) results in the player catching up and continually buffering.

 

Anyways, cheers again for the response! I really need to look more into this stuff myself.

[Andy777 21]

dsf

Are there any other options I can try?

 

Quicksync (qsv codecs) on Linux require the Intel Media Server Studio installed (ffmpeg is not enough). Media Server Studio requires patched custom kernel. Furthermore MSS only works with limited set of Core i and Xeon processors.

 

Result: ffmpeg qsv-codecs on Linux on your CPU doesn't work. It might fall back to software implementation.

 

BR,

 

Andy777

[Andy777 21]

 

On a side note, vaapi may be an option since we're dealing with Linux.  I know that several have reported visual artifacts with it, but that was on the encoding side of things. Curious to see how the decoding side performs and if there are any artifact issues there as well.

 

Vaapi decode (and deinterlace and render) is really good and has been operational for many years. Kodi (Linux on Intel) uses vaapi decode/deinterlace/render and the pq is really good. Emby vaapi-transcode also uses vaapi HW decode. It is the encode-side that has the problems. There is a lot of development going on currently to get the encode quality better (although it sill take many months until it lands on release distributions).

 

BR,

Andy777

[Andy777 21]

Well first, you need an ffmpeg build that supports the features you want to use.

 

Please Luke: Stop offering ffmpeg as a solution to qsv on Linux. Not a single version of ffmpeg help unless there is Media Server Studio installed and fully functional (= patched kernel, selected core i and xeon CPUs only).

 

BR,

Andy777

[witteschnitte 7]

@@Andy777

Its not a fault by the emby Team. quicksync on linux works very fine! but you have to know what you are doing. Its very nice that the emby Team includes this feature. if you do it right you can use your custom ffmpeg in linux with media Studio. if you dont know what to do or want it... please use windows!

[Gerrit507 24]

dsf

 

Quicksync (qsv codecs) on Linux require the Intel Media Server Studio installed (ffmpeg is not enough). Media Server Studio requires patched custom kernel. Furthermore MSS only works with limited set of Core i and Xeon processors.

 

Result: ffmpeg qsv-codecs on Linux on your CPU doesn't work. It might fall back to software implementation.

 

BR,

 

Andy777

 

If it runs easier on Windows, it's no big deal for me. I'm just used to let my Servers run on Linux, so that was my first attempt on emby.

 

Okay, I think I understand now and had your goal a bit backwards in my mind before.  Most people just want the encoding bit to be done on hardware, since it's the heaviest operation.  You probably still want to use the h264_qsv encoder to offload the encoding work, but given the limitations of the embedded/NUC-focused Intel CPU products, I can see why hardware decoding would be beneficial as well for something like HEVC (just making sure I understand the situation).

 

I'm not sure Emby's set up to do that, but we might be able to recommend an option to use hardware decoding to Luke/ebr; it's complicated and source codec dependent (not all codecs have corresponding hardware decoding support), but it may be doable; just need to nail down the options in a way that they can adapt to code and show a benefit for doing so.  I've run across some bugs in Intel's hardware decoding (driver-side) on some generations of CPUs (Sandy/Ivy Bridge in particular), so I've always been hesitant to recommend using it to anyone.  That being said, both your Braswell and Apollo Lake are newer and more likely to have any driver bugs addressed compared to the 4-5 year old chips - so you may not see any issues.  It's important to note as well that hardware decoding should be single-threaded, and I think ffmpeg still complains unless you specify that - just something to take into consideration when judging the benefits.

 

You can try some manual transcoding tests to make sure that the decoder is working well once you get an ffmpeg binary that works for this.  Still puzzling why it's not showing HEVC hardware support in the de/encoder lists if h.264 QSV is showing up, though, unless the driver/kernel just isn't reporting it.  Any chance you can PM me the output from the binaries you have for these commands:

ffmpeg -hwaccels
ffmpeg -codecs
ffmpeg -pix_fmts

I'm hoping to avoid having you build your own ffmpeg (I do it and it sucks unless you have it all scripted   ), so I want to see if it's even needed or if there's a problem elsewhere  The build configuration strings should give me a ton of info alone, but I want to double-check what the binaries are detecting as well hardware-wise..  The last command is just to check on pixel formats (they'll tell me more about if/how it's configured for 10bit support).  Luke can correct me here, but I don't think the "stock" ffmpeg that Emby includes would have vendor-specific hardware encoding support yet (nvenc and qsv, notably), and I seriously doubt that any distribution's ffmpeg packages include it either quite yet.

 

On a side note, vaapi may be an option since we're dealing with Linux.  I know that several have reported visual artifacts with it, but that was on the encoding side of things. Curious to see how the decoding side performs and if there are any artifact issues there as well.

 

I have the J3455 Board now and emby still doesn't recognize hevc_qsv. I'm running Windows 10 now. I attach the output of the three commands. Thank You

output.txt

[witteschnitte 7]

I also had Problems with my intel cpu. because the New Media Studio on linux doesnt Support it. So i Merge the New Media Studio with an old one that Supports my cpu and now quicksync runs with kernel 4.4 and my i7 4790T

Edited February 28, 2017 by witteschnitte

[Gerrit507 24]

I also had Problems with my intel cpu. because the New Media Studio on linux doesnt Support it. So i Merge the New Media Studio with an old one that Supports my cpu and now quicksync runs with kernel 4.4 and my i7 4790T

My problem is that it doesn't even work on Windows. if I play the same 4K hevc 10bit movie with the Windows 10 stock player, I have about 8-15% CPU usage, so it's definitely hw-decoding it. The issue has to be on embys side somehow, because ffmpeg also reports that it's supported...

[Andy777 21]

@@Andy777

Its not a fault by the emby Team. quicksync on linux works very fine! but you have to know what you are doing. Its very nice that the emby Team includes this feature. if you do it right you can use your custom ffmpeg in linux with media Studio. if you dont know what to do or want it... please use windows!

 

Yes I know that it works well if everything is setup correctly (have had it working myself), but the point is that it requirements are very different from windows media SDK quicksync, and that changing the ffmpeg will not make the different requirements go away.

[chvb 1]

 

First of all, my apologies for the n00biness

 

So, let me go step-by-step.

 

1. I'm using UnRAID 6.3.2 with i915 driver installed

2. I've installed Emby default docker image

3. Added to the extra parameters the following line: "--device /dev/dri:/dev/dri"

4. Here's the complete docker command line:

docker run -d --name="EmbyServer" --net="host" -e TZ="Europe/London" -e HOST_OS="unRAID" -e "TCP_PORT_8096"="8096" -e "APP_UID"="99" -e "APP_GID"="100" -v "/mnt":"/mnt":rw -v "/mnt/user/Apps/EmbyServer":"/config":rw --device /dev/dri:/dev/dri emby/embyserver
bf99fa49c78ae945248c0e340c3cb8491481a8e41bbe14436825fc5876bc2317

5. In the Server Dashboard I've set Transcoding to VAAPI and the VA API Device to the default "/dev/dri/card0"

6. FS Permissions are correctly set to allow any user/group access

7. When trying to play a movie and force transcoding, the movie does not play and I get the following error:

[AVHWDeviceContext @ 0x3979800] No VA display found for device: /dev/dri/card0.
[vaapi @ 0x30321e0] Failed to create a VAAPI device

So, my question was if the docker extra parameter is set correctly and/or if I need to change something else in UnRAID or Docker configuration to make it work.

 

Cheers,

anthonws.

 

 

Same situation for me.

Plex Docker is working fine out of the box with Hardware Encoding.

 

It would be so nice, if the Emby Docker runs also with Hardware encoding!

[Gerrit507 24]

I'm still looking for a solution to get hevc_qsv working. I'm open to any OS because I setup a seperate machine anyway.

[millsii 8]

Is there any way we can get transcoding to utilize GPU + CPU as a fallback.  For instance.. the first two ffmpeg transcodes get fired off gpu, and are stored as running... any subsequent request is then fired off with CPU... until one GPU ffmpeg terminates... etc.

 

A setting to limit simultaneous gpu transcodes might not be a bad idea.

 

@@Luke, was there any further thought on this? NVENC has been working quite well for us *touch wood* for the past week or so, but would be good to have the fall back to CPU for those times when we want to have a 3rd stream going, if it is possible?