GPU Transcoding (Intel QuickSync and nVidia NVENC)

[Frazas 2]

Hi @@Frazas

 

Can you post the transcode log with the qsv ? It would also be good to know if you had throttling enabled

 

 

Thanks

 

Thanks for getting back to me!

 

Yes, I had throttling enabled.

 

See below links or the logs:

 

With Quick-sync: http://pastebin.com/fXvSRiqf

 

Without Quick-sync: http://pastebin.com/MpfhqwGM

[Luke 37309]

So i tried to use quicksync on CentOS 7.

 

I got this log

 

/var/lib/emby-server/ffmpeg/20150917/ffmpeg -fflags +genpts -i file:"/XXX/XXX.mkv" -map 0:0 -map 0:1 -map -0:s -codec:v:0 h264_qsv -force_key_frames expr:gte(t,n_forced*5) -vf "scale=min(iw\,1280):trunc(ow/dar/2)*2" -pix_fmt yuv420p -preset 7 -look_ahead 0 -b:v 1872000 -vsync vfr -profile:v high -level 4.1 -map_metadata -1 -threads 0 -codec:a:0 aac -strict experimental -ac 2 -ab 128000 -af "aresample=async=1" -y "/var/lib/emby-server/transcoding-temp/4b033821280a2e23959f98d29806633e.ts"

 

 

ffmpeg version N-49867-gd36eac6-static http://johnvansickle.com/ffmpeg/  Copyright © 2000-2015 the FFmpeg developers

  built with gcc 4.9.3 (Debian 4.9.3-4)

  configuration: --enable-gpl --enable-version3 --disable-shared --disable-debug --enable-runtime-cpudetect --enable-libmp3lame --enable-libx264 --enable-libx265 --enable-libwebp --enable-libspeex --enable-libvorbis --enable-libvpx --enable-libfreetype --enable-fontconfig --enable-libxvid --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libtheora --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-gray --enable-libopenjpeg --enable-libopus --enable-libass --enable-gnutls --enable-libvidstab --enable-libsoxr --enable-frei0r --enable-libfribidi --cc=gcc-4.9

  libavutil      55.  2.100 / 55.  2.100

  libavcodec     57.  2.100 / 57.  2.100

  libavformat    57.  2.100 / 57.  2.100

  libavdevice    57.  0.100 / 57.  0.100

  libavfilter     6.  4.100 /  6.  4.100

  libswscale      4.  0.100 /  4.  0.100

  libswresample   2.  0.100 /  2.  0.100

  libpostproc    54.  0.100 / 54.  0.100

Unrecognized option 'look_ahead'.

Error splitting the argument list: Option not found

 

So it doesn´t work for me.

 

Do you have any ideas? I don´t know why libmfx isn´t in the ffmpeg enabled.

TVheadend Needs this lib to use quicksync.

 

Hi, the feature is not supported yet for your distribution.

[legallink 187]

Think you have to provide a log to see whats happening.

 

For me, quicksync is working but I sometimes run into trouble that the old ffmpeg instance is not closed correctly and a new one is created. (Happens while seeking inside the media or just selecting another bitrate quality)

 Sorry for the late reply.  Here are my logs from this morning's attempt at using quicksync.  https://www.dropbox.com/sh/bpolfw21j6mp071/AABnJDqgapyGKmDHqmqSJMvfa?dl=0

 

I did not replace my ffmpeg install...do I need to do that before turning it on?  I have checked with OBS software, and quicksync is available as an option, and I have tested it with OBS, so I know the hardware is correct.

 

Update, I now get a transcode log, but the most significant portion is:

Error initializing an internal MFX session

I take this to mean there is possibly a problem (as was pointed out in another thread) with the Intel Media SDK.  I am running older hardware (i5 2500k), which was the beginning of quick sync so that may add complications.

 

I tried installing older as well as the newer version of the video driver, as suggested in a previous thread, but no difference or change was to be had.

Edited December 2, 2015 by legallink

[Tranquil 93]

You may try the free tool Handbrake to see if it is working for you.

I also had problems with an I5-2500K on my old board which had no OnBoard Video support. If you are using another graphic device and your board is not supporting the CPUs graphic device you are maybe lost in that case.

[legallink 187]

Handbrake works like a beast with quick sync (~500fps on a conversion).I

 

 have a Z series board from ASROCK, so I don't think board support is the issue. 

[supermau 37]

I am planing on building a new linux server exclusively for Emby, on what linux distribution is Quick Sync enabled and supported?

[Dan64 30]

I am planing on building a new linux server exclusively for Emby, on what linux distribution is Quick Sync enabled and supported?

 

I have built a (headless) Linux server using ASRock Q1900-ITX, this board has J1900 Intel CPU (2.42 Ghz, Quad-core, 22nm). I'm using Debian 9 and I have installed ffmpeg 2.8.2. As maybe you know Linux run faster than Windows, and even if my ffmpeg does not support h264_qsv (I don't have a build with quick sync enabled) I have to say that the transcoding perfomances are very good and Emby works perfectly.

 

To give you an idea these are the transcoding performaces on my server:

Q1900-ITX Transcoding speed on Debian 9.0:
  
1080p, hevc: CPU 60%, 40fps
1080p, h264: CPU 55%, 49fps
 720p, hevc: CPU 52%, 60fps
 720p, h264: CPU 32%, 57fps
 

As you can see the CPU usage range between 32% - 60% and the transcoding speed is between 40fps - 60fps well above the minimum 25fps required to have a smooth playback. I read that on Windows the same board is able to reach a max speed of 23fps at 97% of CPU. So my advise is that if you want to build a Linux server this board has enough power to be used as Emby server while probably using Windows will be necessary to use a Quick Sync enabled version of ffmpeg to have acceptable perfomances.

Edited December 3, 2015 by Dan64

[Deihmos 147]

@@Dan64 Do you see the FPS in the transcoding log? I have an i3 cpu from 3 years ago and the cpu never goes pass 25% with intel quicsync. The fps reads 138 if I am reading it correctly and the full transcodes takes just a few minutes. WIthout QS it reads 85 fps. What did you get under Windows? I think one great benefit to QS is it uses less energy.

 

 

frame= 5867 fps=137 q=-0.0 size=N/A time=00:04:05.46 bitrate=N/A    

frame= 5940 fps=138 q=-0.0 size=N/A time=00:04:08.27 bitrate=N/A    

frame= 6009 fps=138 q=-0.0 size=N/A time=00:04:11.28 bitrate=N/A    

Edited December 3, 2015 by Deihmos

[Dan64 30]

I'm not using the J1900 Intel CPU on Windows but on Debian Linux. I have taken the encoding speed from Emby's transcoding log and the CPU usage from the "top" program.

 

The ASRock Q1900-ITX is a 10watt board, cannot be compared with a i3 board that usually need 60watt to stay in "idle". That's the reason why I was really surprised bo be able to do a realtime trascoding using a so low powered board. 

 

I read the Windows perfomance in this post : http://emby.media/community/index.php?/topic/10723-gpu-transcoding-intel-quicksync-and-nvidia-nvenc/&do=findComment&comment=137245.

Edited December 3, 2015 by Dan64

[Deihmos 147]

That's over a year ago.

[dark_slayer 103]

a i3 board that usually need 60watt to stay in "idle"

Nah, try 18-20. The only ones that suck that much extra power are Asus super deluxe models with a bunch of non standard junk included with the Intel chipsets. The i3 idle is down around 3-4W plus about 10 for the board with a few extra for RAM and peripherals

[Deihmos 147]

Nah, try 18-20. The only ones that suck that much extra power are Asus super deluxe models with a bunch of non standard junk included with the Intel chipsets. The i3 idle is down around 3-4W plus about 10 for the board with a few extra for RAM and peripherals

 

My i3-4130, 8GB ram, 120GB SSD system idles at 7-8 watts. My i3 nuc idles at 4-5 watts.

Edited December 3, 2015 by Deihmos

[Dan64 30]

My i3-4130, 8GB ram, 120GB SSD system idles at 7-8 watts. My i3 nuc idles at 4-5 watts.

 

Frankly speaking I don't want to spent too much time discussing this topic since it's outside the subject of this board. But in any case I will try to be more specific. Intel introduced the i3 CPU in 2010, the 1st generation of i3 has a declared TDP of 75w, the 2nd generation a declared TDP of 65w, the 3rd generation a declared TDP of 55w, and so on...

 

You can found all the specs here: https://en.wikipedia.org/wiki/List_of_Intel_Core_i3_microprocessors

 

The J1900 has a declared TDP of 10w but a lower typical power consumption of 8.13w, by contrast the i3-4130 has a declared TDP of 54w and a typical lower power consumption of 43.88w. You can find a comparison here: http://cpuboss.com/cpus/Intel-Core-i3-4130-vs-Intel-Celeron-J1900

 

The J1900 has a typical power consumption near to the ARM CPUs but with more computation power, in effect in Linux can transcode in real time a h264/hevc stream without using GPU optimizations. In the next future will be probably even less power hungry CPUs having enough power to do this transcoding process...

Edited December 4, 2015 by Dan64

[Deihmos 147]

The i3 Nuc and i5 Nuc TDP is 15 watts. I could see myself using a J1900 as a player but that's about it. 

[dark_slayer 103]

Frankly speaking I don't want to spent too much time discussing this topic since it's outside the subject of this board. But in any case I will try to be more specific. Intel introduced the i3 CPU in 2010, the 1st generation of i3 has a declared TDP of 75w, the 2nd generation a declared TDP of 65w, the 3rd generation a declared TDP of 55w, and so on...

 

You can found all the specs here: https://en.wikipedia.org/wiki/List_of_Intel_Core_i3_microprocessors

 

The J1900 has a declared TDP of 10w but a lower typical power consumption of 8.13w, by contrast the i3-4130 has a declared TDP of 54w and a typical lower power consumption of 43.88w. You can find a comparison here: http://cpuboss.com/cpus/Intel-Core-i3-4130-vs-Intel-Celeron-J1900

 

The J1900 has a typical power consumption near to the ARM CPUs but with more computation power, in effect in Linux can transcode in real time a h264/hevc stream without using GPU optimizations. In the next future will be probably even less power hungry CPUs having enough power to do this transcoding process...

TDP has nothing to do with idle (which I'm sure you know)

 

The numbers I gave two posts up are from my own board and an i3-2125. A DIY haswell or newer can do a little better. The NUC and j1900 can do better as well. They do better because there are less ac/dc conversions in newer chipsets since Intel adds more functions to its chipsets. All that being said, even the older sandy bridge chipset had the isle numbers I quoted and would never idle at 60W unless you had one of the performance/deluxe boards that added extra features at the cost of power consumption

[Della Dog 19]

Update, I now get a transcode log, but the most significant portion is:

Error initializing an internal MFX session

...

 

Same result with i7 3770k:

 [h264_qsv @ 000000222a91f2e0] Error initializing an internal MFX session

 

Auto outputs:

frame= 358 fps=116 q=22.0 size=N/A time=00:00:14.50 bitrate=N/A

frame= 435 fps=121 q=23.0 size=N/A time=00:00:17.64 bitrate=N/A

frame= 499 fps=122 q=23.0 size=N/A time=00:00:20.16 bitrate=N/A 

Edited December 4, 2015 by Della Dog

[witteschnitte 7]

@@Della Dog

 

to check if intel quick sync is running please check if the intel media sdk examples are working.

without that i got the same error on thheadend.

please check also if you had symlink your intel media sdk Drivers from /opt/intel/mediasdk/lib64 to usr/lib and maybe usr/lib64

 

with that quick sync in tvheadend working for me on centos7

[supermau 37]

@@witteschnitte

 

I want to try to build ffmpeg with quick sync in Ubuntu, do you have it working in your setup? did you compile it? any pointers?

 

Thank you

Edited December 5, 2015 by supermau

[Deihmos 147]

I thought Intel didn't support quicksync with Linux. I guess that changed.

Edited December 5, 2015 by Deihmos

[babgvant 143]

If you're having trouble with QS + ffmpeg on an older system, try this build (includes 32/64 builds). The difference is that ffmpeg inits the MSDK with version 1.1.

 

You do not need to replace the version in Emby (although it should be safe to do so). To test it directly use:

 

ffmpeg.exe -c:v h264_qsv -i "pathtoinputfile" -look_ahead 0 -c:v h264_qsv -y "pathtooutputfile"

[babgvant 143]

I thought Intel didn't support quicksync with Linux. I guess that changed.

 

It did. On demand transcoding farms run Linux. QS is perfectly suited to that application.

[witteschnitte 7]

intel quick sync is supported on linux using the intel media sdk. you have to follow the instructions to install the sdk. so use centos7 oder a generic kernel 3.14.5. this you can build in the media sdk setup.

 

to use quick sync you have to compile ffmpeg with libmfx.

i have quick sync working with tvheadend but not with emby at this moment.

 

but first be sure that the samples from the sdk are working because ffmpeg and the libs can be a tricky part.

 

just ask me for more help on linux

[witteschnitte 7]

the ffmpeg postet above is a windows version and can not be used in Linux

[Dan64 30]

@@witteschnitte

 

I want to try to build ffmpeg with quick sync in Ubuntu, do you have it working in your setup? did you compile it? any pointers?

 

Thank you

 

Before to decide to build ffmpeg with QSV support in Ubuntu, please consider the following issues:

 

1) You "must" patch your kernel, the patches provided by intel are for version 3.14.5 of linux kernel, this left you with the only reasonable option to install Ubuntu Server 12.04.5

 

2) Consider that on low powerd hardware libx264 with preset superfast (with ultrafast it's even faster) has perfomance similars to h264_qsv. To give you an idea on my small ASRock Q1900-ITX I'm able to transcode a 1080p video at 49 fps (cpu 55%), while using QSV it's possible transcode at 60 fps (cpu 95%). You can find more info in this long post: https://trac.ffmpeg.org/ticket/2591   

 

Having said that, if you still want try to compile ffmpeg with QSV support, you "must" execute the following steps:

1. Software installation on Ubuntu Server 12.04.5
a) install build tools:
apt-get install cmake make gcc pkg-config patch yasm g++ autoconf automake build-essential cifs-utils texi2html
 install libraries: 
apt-get install libpthread-stubs0-dev libpciaccess-dev libass-dev libfreetype6-dev libgpac-dev libsdl1.2-dev libtheora-dev libtool libva-dev libvdpau-dev libvorbis-dev
libx11-dev libxext-dev libxfixes-dev zlib1g-dev libx264-dev libmp3lame-dev libfaac-dev librtmp-dev libvo-aacenc-dev libx264-dev libvdpau-va-gl1 libva-intel-vaapi-driver

2. Media Server Studio Essentials 2015r6 (Community free edition)
wget http://registrationcenter.cps.intel.com/irc_nas/7719/mediaserverstudioessentials2015r6.tar.gz
tar xf mediaserverstudioessentials2015r6.tar.gz
cd MediaServerStudioEssentials2015R6
tar xf SDK2015Production16.4.2.1.tar.gz
cd SDK2015Production16.4.2.1/Generic/
tar xf intel-linux-media-ocl_generic_16.4.2.1-39163_64bit.tar.gz
./install_media.sh

3. libdrm
cd /opt/intel/mediasdk/opensource/libdrm/2.4.54-39163/
tar xf libdrm-2.4.54.tar.bz2
cd libdrm-2.4.54
./configure
make -j 8
make install

4. libva
cd /opt/intel/mediasdk/opensource/libva/1.3.1-staging.20130628.39163
tar xf libva-1.3.1.tar.bz2
cd libva-1.3.1
./configure --enable-static
make
make install

5. libmfx
cd /opt/intel/mediasdk/opensource/mfx_dispatch/
mkdir build
cd build
cmake –D__ARCH:STRING=intel64 ../
make -j 8
ln -s /opt/intel/mediasdk/include/ /usr/local/include/mfx

6. libs
cd /usr/local/lib
ln -sf /opt/intel/mediasdk/opensource/mfx_dispatch/build/__lib/libmfx.a
ln -sf /opt/intel/mediasdk/opensource/mfx_dispatch/build/__lib/libdispatch_trace.a
ln -sf /opt/intel/mediasdk/opensource/mfx_dispatch/build/__lib/libdispatch_shared.a
# libva.a, libva-drm.a, libva-tpi.a should be already in /usr/local/lib if not add the symlinks 

7. create file /usr/lib/pkgconfig/libmfx.pc:
prefix=/usr/local
exec_prefix=${prefix}
libdir=${prefix}/lib
includedir=${prefix}/include
Name: libmfx
Description: Intel Media SDK Dispatched static library
Version: 2015
Requires:
Requires.private:
Conflicts:
Libs: -L${libdir} -lmfx -ldispatch_shared -lva -lva-drm -lsupc++ -lstdc++ -ldl
Libs.private:
Cflags: -I${includedir} -I/opt/intel/mediasdk/opensource/mfx_dispatch/include

8. add to /etc/environment (shoud be already set by the sdk):
LD_LIBRARY_PATH="/usr/local/lib:/usr/lib64"
LIBVA_DRIVER_NAME=iHD
LIBVA_DRIVERS_PATH=/opt/intel/mediasdk/lib64

9. linux-kernel patch
wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.14.5.tar.xz
tar xf linux-3.14.5.tar.xz
cd linux-3.14.5
cp
/opt/intel/mediasdk/opensource/patches/kmd/3.14.5/intel-kernel-patches.tar.bz2
.
tar xf intel-kernel-patches.tar.bz2
for i in intel-kernel-patches/*.patch; do patch -p1 < $i; done
make olddefconfig
make -j 8
make modules_install
make install
reboot

10. compile a static build of ffmpeg
wget http://ffmpeg.org/releases/ffmpeg-snapshot.tar.bz2
tar xf ffmpeg-snapshot.tar.bz2
cd ffmpeg
./configure --disable-shared --enable-static --enable-gpl --enable-nonfree --enable-fontconfig --enable-libfaac --enable-libfreetype --enable-libmp3lame --enable-librtmp --enable-libvo-aacenc --enable-libx264 --enable-libx265 --enable-version3 --enable-ffplay --disable-doc --disable-ffserver --enable-pthreads --enable-filters --enable-libvorbis --enable-libtheora --enable-runtime-cpudetect --enable-libass --enable-bzlib --enable-zlib --enable-vdpau --enable-vaapi --enable-avfilter --disable-stripping --enable-swscale --enable-libmfx
make -j 8
make install 

That's all, good luck !

Edited December 6, 2015 by Dan64

[witteschnitte 7]

I dont know why the ffmpeg for linux isnt updated. luke said it will be released for other systems than Windows. but no linux build have a working ffmpeg. can you provide a working one?