GPU Transcoding (Intel QuickSync and nVidia NVENC)

[smathers3 0]

 

With release of 3.0.6050, how are you validating if encoding HW is present? I'm using a GeForce GT 720 (Kepler based NVENC) in another application on same box and it works fine. Emby still doesnt try to use it when I start transcoding (log shows ffmpeg using libx264 vs. nvenc_264) Using latest drivers.

 

[Luke 37095]

With release of 3.0.6050, how are you validating if encoding HW is present? I'm using a GeForce GT 720 (Kepler based NVENC) in another application on same box and it works fine. Emby still doesnt try to use it when I start transcoding (log shows ffmpeg using libx264 vs. nvenc_264) Using latest drivers.

 

I would bump up to 3.0.6070. As far as how we validate, we are checking if your ffmpeg build has the necessary encoding libraries that allow it to utilize your gpu.

[dark_slayer 103]

However, my video card is used only at 2%-4%. Is it your case?

That should be normal, that is one of the big draws of nvenc for use with things like game stream... it only lowers graphics by a few percent

[jscoys 143]

That should be normal, that is one of the big draws of nvenc for use with things like game stream... it only lowers graphics by a few percent

Ok the fact that I have to disable throttling force my CPU to be at 35% average for decoding the whole movie, in addition to the nvenc used at 2-4%. I use Zeranoe build and I really don't understand why my CPU still at this high level. If I play a game on my computer with two ffmpeg decoding process it's impossible to continue playing...

 

Do I have to wait for the throttling available with NVENC decoding? Or maybe it's my Zeranoe build which is deficient? What is your opinion?

 

 

Sent from my iPhone using Tapatalk

[Luke 37095]

Your build is fine. ffmpeg does not have a native means to throttle, we came up with a trick to get it to pause, but I'm sure there are situations where it won't work and this is probably one of them. Keep in mind the throttling only kicks in when the transcoding position gets really far ahead and that's not always going to happen anyway.

[jscoys 143]

Your build is fine. ffmpeg does not have a native means to throttle, we came up with a trick to get it to pause, but I'm sure there are situations where it won't work and this is probably one of them. Keep in mind the throttling only kicks in when the transcoding position gets really far ahead and that's not always going to happen anyway.

Yeah I know but this is a very useful feature when you've got two ffmpeg process at the same time. Anyway give us info when you would like to test a new trick to pause compatible with NVENC decoding [emoji3] would be very happy to test!

 

 

Sent from my iPhone using Tapatalk

[Luke 37095]

When you have multiple simultaneous instances your system is probably being taxed to a degree that throttling is unlikely to happen anyway, because it's less likely that any of the processes will be able to jump out ahead far enough to take advantage of it. So I really would suggest to just not worry about it and enjoy your movies.

[chvb 1]

I also use the nvenc feature.

The gpu transcoding is a really nice thing.

thanks for this feature emby team.

[Luke 37095]

I also use the nvenc feature.

The gpu transcoding is a really nice thing.

thanks for this feature emby team.

 

We're glad you're enjoying it

[Luke 37095]

Great news guys. The latest beta server, 3.0.119 adds support for VA API hardware transcoding, the Video Acceleration API.

 

If you're running on hardware that supports that, please try it out and report your experience. Thanks!

[schamane 18]

Hi,

 

just registered because of that topic.

 

I am on Ubuntu 16.04 with and Intel N3700 which supports Vaapi which I already use with Kodi, but transcoding seems not to work with emby

 

I tried quicksync in the past, but to be honest its a mess to get in running, especially in a "up to date" environment

 

The ffmpeg of Ubuntu doesnt come with vaapi support, so i downloaded one from here      http://johnvansickle.com/ffmpeg/ and linked it as suggested.

 

now i get the needed result

  

 V..... h264_vaapi           H.264/AVC (VAAPI) (codec h264)

 V..... hevc_vaapi           H.265/HEVC (VAAPI) (codec hevc)

 V..... mjpeg_vaapi          MJPEG (VAAPI) (codec mjpeg)

 

and i setup transcoding in Emby to vaapi

 

and 

 /dev/dri/card0 

is also existent, but its still not working

 

when I lower the data rate, that it has to decode, it stops immediately and also when I just play with full Network capacity it still takes 100% CPU 

 

any hints what I can try?

 

Looking forward for a fast solution, if this works like a charm you will have sold another lifetime

 

 

EDIT

OK,

found something, but no clue howto to fix it yet

 

libva info: Trying to open /home/john/.custom.software/src/static/lib/dri/i965_drv_video.so

libva info: va_openDriver() returns -1

[AVHWDeviceContext @ 0x3c3c640] Failed to initialise VAAPI connection: -1 (unknown libva error).

[vaapi @ 0x31171e0] Failed to create a VAAPI device

 

 

I have no john on my system

 

so i guess ffmpeg is build stange or  wrong from  john van sickle?

 

when i just link 

 

/home/john/.custom.software/src/static/lib/dri/i965_drv_video.so -> /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so

 

 will result in a coredump

 

 

EDIT2

this is getting a self-talk

 

fixed the above error with an new ffmpeg (ubuntu really sucks at this point)

 

 

when i do  your ffmpeg comman as root now its working (with 50% cpu rate )))) ) 

Input stream #0:0 frame changed from size:1920x800 fmt:yuv420p to size:1920x800 fmt:vaapi_vld

frame=   31 fps=0.0 q=-0.0 size=     445kB time=00:00:01.44 bitrate=2530.3kbits/s speed=2.76x

frame=   77 fps= 74 q=-0.0 size=     928kB time=00:00:03.33 bitrate=2278.4kbits/s speed=3.22x

frame=  127 fps= 83 q=-0.0 size=    1622kB time=00:00:05.49 bitrate=2417.0kbits/s speed=3.58x

frame=  173 fps= 85 q=-0.0 size=    2477kB time=00:00:07.36 bitrate=2753.2kbits/s speed= 3.6x

frame=  221 fps= 86 q=-0.0 size=    2972kB time=00:00:09.36 bitrate=2601.1kbits/s speed=3.66x

 

but as user emby

 

 [AVHWDeviceContext @ 0x258cd40] No VA display found for device: /dev/dri/card0.

[vaapi @ 0x63b0e8] Failed to create a VAAPI device

 

so added user emby to group video 

 

libva info: va_getDriverName() returns 1

libva error: va_getDriverName() failed with operation failed,driver_name=i965

[AVHWDeviceContext @ 0x2386ce0] Failed to initialise VAAPI connection: 1 (operation failed).

[vaapi @ 0x63b0e8] Failed to create a VAAPI device

 

hopefully edit3 will follow soon with permissions solved

 

EDIT3

 

Yes, I got it working now

 

the issue was not depending on permissions anymore

 

/dev/dri/card0 was just bloyked already via kodi

 

so changing it to 

 

  /dev/dri/renderD128

 

did the trick.

 

now i can transcode on the fly with 50% cpu usage at max!!! 

 

the limiting factor is IO now ))

 

had 4 transcode running at the same time and only waiting cause of IO 

 

Awsome guys, will immediately purchase a lifetime now

 

If someone interested in a howto i can sort it up a bit I guess

 

and a hint as well, the transcoding infos in the Board are really poor to be honest all depending on ffmpeg (of course it is), but get this all up and running (especially on Linux) isn`t really an easy task

Edited August 25, 2016 by schamane

[MSattler 387]

@@Luke,

 

The latest beta, 3.1.117, if I select QuickSync transcoding, when I select to use the built in ffmpeg, I get an error saying it cannot find it.  Although it really does since it works just fine.

 

Thanks!

[Luke 37095]

@@Luke,

 

The latest beta, 3.1.117, if I select QuickSync transcoding, when I select to use the built in ffmpeg, I get an error saying it cannot find it.  Although it really does since it works just fine.

 

Thanks!

 

Thanks. Can you let me know if you see that on 3.1.120? I did some work on the config page so I don't think it's an issue anymore. Thanks.

[MSattler 387]

Thanks. Can you let me know if you see that on 3.1.120? I did some work on the config page so I don't think it's an issue anymore. Thanks.

 

Got family members using it right now but will check tonight!

[MSattler 387]

All,

 

Is there a performance difference between vaapi and Quick Sync from Intel?  Obviously only Quick Sync can do h265 on the latest processors, but all in all are the results in vaapi the same as in Quick Sync?

 

Thanks!

[MSattler 387]

Got family members using it right now but will check tonight!

 

@@Luke,

 

It is still there.  Weird thing is when I go to the screen, it has Custom selected, with the default Emby FFMpeg directory/location listed.  If I change it to "Use System Installed" it then gives me an error, when I go back Custom is selected.

 

I think I see now what you are trying to do.  Would "Use system installed version" only be used for Linux installs?  If so maybe have these options:

 

1.)  "Use system installed version (Linux)"

2.)  "Use Default Emby Installation version (Windows)"

3.)  "Use your own custom version of Ffmpeg"

 

It looks like the options just confused me since I didn't think I was choosing custom, since I'm using the default version of ffmpeg.

 

Thanks!

[Luke 37095]

@@Luke,

 

It is still there.  Weird thing is when I go to the screen, it has Custom selected, with the default Emby FFMpeg directory/location listed.  If I change it to "Use System Installed" it then gives me an error, when I go back Custom is selected.

 

I think I see now what you are trying to do.  Would "Use system installed version" only be used for Linux installs?  If so maybe have these options:

 

1.)  "Use system installed version (Linux)"

2.)  "Use Default Emby Installation version (Windows)"

3.)  "Use your own custom version of Ffmpeg"

 

It looks like the options just confused me since I didn't think I was choosing custom, since I'm using the default version of ffmpeg.

 

Thanks!

 

System installed could be Windows if the path is added to the PATH environment variable. It's not very common but it does happen. 

[MSattler 387]

System installed could be Windows if the path is added to the PATH environment variable. It's not very common but it does happen. 

 

Agreed, but the language/options should be clarified a bit as I think people will miss understand it as it is now.

[dark_slayer 103]

Agreed, but the language/options should be clarified a bit as I think people will miss understand it as it is now.

Agree 95/5 rule

[thelamer 0]

It looks like the latest version of emby is passing something intel quicksync does not like in linux via the CLI. 

 

I have a custom compiled version of FFmpeg that was working fine until the latest update to "Version 3.0.6300.0". 

 

Here are two logs one transcoding log from before the update: 

 

http://pastebin.com/CxpCa9d5

 

And one After: 

 

http://pastebin.com/pm6Jey94

 

FFmpeg segfaults now without an explanation, it still works when passing libx264 instead of h264_qs (disable quicksync in settings) which is how I have it set now to stay online. 

 

As the old transcode commands still work running them manually and the new flags work fine with libx264 and the same FFmpeg binary I think the software might be passing some new flags that specifically quicksync does not like.

 

Is anyone else having these problems? 

 

 

Debian Jessie

Kernel 3.14.5

FFmpeg version N-80111-gd1520a6

 

 

Edit: 

 

Also wanted to add that by running a simple no nonsense encode command like: 

/var/lib/emby-server/ffmpeg/20160215/ffmpeg -i file:"/storage/media/Movies/The Matrix 1999.mkv" -c:v h264_qsv -preset:v faster /storage/out.mp4

The qsv encoding works fine for the file that segfaults. 

Edited August 29, 2016 by thelamer

[Luke 37095]

It looks like the latest version of emby is passing something intel quicksync does not like in linux via the CLI. 

 

I have a custom compiled version of FFmpeg that was working fine until the latest update to "Version 3.0.6300.0". 

 

Here are two logs one transcoding log from before the update: 

 

http://pastebin.com/CxpCa9d5

 

And one After: 

 

http://pastebin.com/pm6Jey94

 

FFmpeg segfaults now without an explanation, it still works when passing libx264 instead of h264_qs (disable quicksync in settings) which is how I have it set now to stay online. 

 

As the old transcode commands still work running them manually and the new flags work fine with libx264 and the same FFmpeg binary I think the software might be passing some new flags that specifically quicksync does not like.

 

Is anyone else having these problems? 

 

 

Debian Jessie

Kernel 3.14.5

FFmpeg version N-80111-gd1520a6

 

Thanks. can you play with the raw command lines to try to find the culprit?

[thelamer 0]

Thanks. can you play with the raw command lines to try to find the culprit?

Yeah it looks like it is the addition of this parameter "-c:v h264_qsv"

 

IE: 

 

before: (no works) 

 

 

/var/lib/emby-server/ffmpeg/20160215/ffmpeg \
-c:v h264_qsv  \
-i file:"/storage/media/Movies/the nice guys 2016.mkv" \
-map_metadata -1 \
-threads 1 \
-map 0:0 \
-map 0:1 \
-map -0:s \
-codec:v:0 h264_qsv \
-preset 7 \
-look_ahead 0 \
-b:v 1360000 \
-maxrate 1360000 \
-bufsize 2720000 \
-vsync -1 \
-profile:v high \
-level 4.1 \
-force_key_frames "expr:gte(t,n_forced*3)" \
-vf "scale=trunc(min(max(iw\,ih*dar)\,1280)/2)*2:trunc(ow/dar/2)*2" \
-flags \
-global_header \
-copyts \
-codec:a:0 aac \
-strict experimental \
-ac 6 \
-ab 640000 \
-af "adelay=1,aresample=async=1" \
-max_delay 5000000 \
-avoid_negative_ts disabled \
-start_at_zero \
-hls_time 3 \
-start_number 0 \
-hls_list_size 0 \-y "/var/lib/emby-server/transcoding-temp/8ca534279f40d4620bbc28a860720b7e.m3u8"

After: (works) 

/var/lib/emby-server/ffmpeg/20160215/ffmpeg \
-i file:"/storage/media/Movies/the nice guys 2016.mkv" \
-map_metadata -1 \
-threads 1 \
-map 0:0 \
-map 0:1 \
-map -0:s \
-codec:v:0 h264_qsv \
-preset 7 \
-look_ahead 0 \
-b:v 1360000 \
-maxrate 1360000 \
-bufsize 2720000 \
-vsync -1 \
-profile:v high \
-level 4.1 \
-force_key_frames "expr:gte(t,n_forced*3)" \
-vf "scale=trunc(min(max(iw\,ih*dar)\,1280)/2)*2:trunc(ow/dar/2)*2" \
-flags \
-global_header \
-copyts \
-codec:a:0 aac \
-strict experimental \
-ac 6 \
-ab 640000 \
-af "adelay=1,aresample=async=1" \
-max_delay 5000000 \
-avoid_negative_ts disabled \
-start_at_zero \
-hls_time 3 \
-start_number 0 \
-hls_list_size 0 \-y "/var/lib/emby-server/transcoding-temp/8ca534279f40d4620bbc28a860720b7e.m3u8"

[Luke 37095]

The first qsv is for decoding. The second one is for encoding. The last time I removed it from decoding the reaction to that was not good, so my hands a little tied on that. Maybe there is some characteristic about this particular video though that we can check for to exclude it from decoding if we know the QuickSync decoder is going to have trouble with it.

[thelamer 0]

The first qsv is for decoding. The second one is for encoding. The last time I removed it from decoding the reaction to that was not good, so my hands a little tied on that. Maybe there is some characteristic about this particular video though that we can check for to exclude it from decoding if we know the QuickSync decoder is going to have trouble with it.

I tested it on a loop of about 2500 movie files and it segfaulted on every one. 

 

Could just be my build of FFmpeg, will re-up to head and report back tomorrow. If all else fails I can comment and do a custom build so no big deal https://github.com/MediaBrowser/Emby/blob/master/MediaBrowser.MediaEncoding/Encoder/BaseEncoder.cs#L388 . 

 

Thanks for your time though. 

[Luke 37095]

A setting might not be a bad idea. I don't necessarily think it's a problem with your ffmpeg build process but maybe a defect in the version you built from.