FFmpeg only using 75% of CPU when Transcoding

[TehEpikDuckeh 6]

Transcoding only uses 75% of the CPU, only 7 out of 8 threads.

 

Processor: i7-2600 @ 3.8 GHz

 

Transcoding log:

Application version: 3.2.36.2

http://192.168.0.3:40330/emby/videos/418d80970834ea0588c1bb0697edb255/hls1/main/0.ts?DeviceId=0f646ede5a6cee1566fe5433befcb07c53005e86&MediaSourceId=418d80970834ea0588c1bb0697edb255&VideoCodec=h264&AudioCodec=mp3,aac&AudioStreamIndex=1&VideoBitrate=59616000&AudioBitrate=384000&PlaySessionId=7bc5ffbd11c4477fa037e48c7502b667&api_key=99bd3ef4dc7841fbb78343b4dded2dce&CopyTimestamps=false&TranscodingMaxAudioChannels=2&EnableSubtitlesInManifest=false&Tag=f6d58280eee7c62ff832b22128ed19cd&RequireAvc=true&RequireNonAnamorphic=false&SegmentContainer=ts&MinSegments=1&BreakOnNonKeyFrames=True&h264-profile=high,main,baseline,constrainedbaseline&h264-level=51&h264-deinterlace=true

{"Protocol":"File","Id":"418d80970834ea0588c1bb0697edb255","Path":"/media/HDD/TV Shows/The Man in the High Castle/Season 2/S02E01 - The Tiger's Cave.mkv","Type":"Default","Container":"mkv,webm","Name":"1080P/HEVC/EAC3","IsRemote":false,"ETag":"f6d58280eee7c62ff832b22128ed19cd","RunTimeTicks":34244149248,"ReadAtNativeFramerate":false,"IgnoreDts":false,"IgnoreIndex":false,"GenPtsInput":false,"SupportsTranscoding":true,"SupportsDirectStream":true,"SupportsDirectPlay":true,"IsInfiniteStream":false,"RequiresOpening":false,"RequiresClosing":false,"SupportsProbing":true,"RequiresLooping":false,"VideoType":"VideoFile","MediaStreams":[{"Codec":"hevc","TimeBase":"1/1000","CodecTimeBase":"1/25","IsInterlaced":false,"BitRate":2264283,"RefFrames":1,"IsDefault":true,"IsForced":false,"Height":1080,"Width":1920,"AverageFrameRate":25,"RealFrameRate":25,"Profile":"Main","Type":"Video","AspectRatio":"16:9","Index":0,"IsExternal":false,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"PixelFormat":"yuv420p","Level":120,"IsAnamorphic":false},{"Codec":"eac3","TimeBase":"1/1000","CodecTimeBase":"1/48000","DisplayTitle":"Dolby Digital+ 5.1 Default","IsInterlaced":false,"ChannelLayout":"5.1","BitRate":384000,"Channels":6,"SampleRate":48000,"IsDefault":true,"IsForced":false,"Type":"Audio","Index":1,"IsExternal":false,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Level":0,"IsAnamorphic":false},{"Codec":"ass","TimeBase":"1/1000","CodecTimeBase":"0/1","DisplayTitle":"Und","IsInterlaced":false,"IsDefault":false,"IsForced":false,"Type":"Subtitle","Index":2,"IsExternal":false,"IsTextSubtitleStream":true,"SupportsExternalStream":true,"Level":0,"IsAnamorphic":false},{"Codec":"srt","Language":"eng","DisplayTitle":"Eng","IsInterlaced":false,"IsDefault":false,"IsForced":false,"Type":"Subtitle","Index":3,"IsExternal":true,"IsTextSubtitleStream":true,"SupportsExternalStream":true,"Path":"/media/HDD/TV Shows/The Man in the High Castle/Season 2/S02E01 - The Tiger's Cave.eng.srt","IsAnamorphic":true}],"Formats":[],"Bitrate":2648283,"RequiredHttpHeaders":{}}

User policy for Mark: EnablePlaybackRemuxing: True EnableVideoPlaybackTranscoding: True EnableAudioPlaybackTranscoding: True

/var/lib/emby-server/ffmpeg/ffmpeg/ffmpeg -f matroska,webm -i file:"/media/HDD/TV Shows/The Man in the High Castle/Season 2/S02E01 - The Tiger's Cave.mkv" -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 libx264 -vf "scale=trunc(min(max(iw\,ih*dar)\,1920)/2)*2:trunc(ow/dar/2)*2" -pix_fmt yuv420p -preset veryslow -crf 1 -maxrate 4528566 -bufsize 9057132 -profile:v high -level 4.1 -x264opts:0 subme=0:me_range=4:rc_lookahead=10:me=dia:no_chroma_me:8x8dct=0:partitions=none -force_key_frames "expr:if(isnan(prev_forced_t),eq(t,t),gte(t,prev_forced_t+3))" -copyts -vsync -1 -codec:a:0 libmp3lame -ac 2 -ab 384000  -f segment -max_delay 5000000 -avoid_negative_ts disabled -map_metadata -1 -map_chapters -1 -start_at_zero -segment_time 3  -individual_header_trailer 0 -segment_format mpegts -segment_list_type m3u8 -segment_start_number 0 -segment_list "/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a8.m3u8" -y "/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a8%d.ts"

ffmpeg version N-88289-g783535a Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 4.9.2 (Debian 4.9.2-10)
  configuration: --enable-encoder=hevc_nvenc --enable-nonfree --enable-libx264 --enable-libx265 --enable-libfreetype --enable-libass --enable-gpl --enable-libvpx --enable-libtheora --enable-libopus --enable-libmp3lame --enable-libfdk-aac --enable-version3 --disable-w32threads --enable-runtime-cpudetect --enable-avisynth --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libgsm --enable-libspeex --enable-libxvid --enable-zlib
  libavutil      56.  0.100 / 56.  0.100
  libavcodec     58.  1.100 / 58.  1.100
  libavformat    58.  0.102 / 58.  0.102
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  0.101 /  7.  0.101
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.100 /  3.  0.100
  libpostproc    55.  0.100 / 55.  0.100
Input #0, matroska,webm, from 'file:/media/HDD/TV Shows/The Man in the High Castle/Season 2/S02E01 - The Tiger's Cave.mkv':
  Metadata:
    ENCODER         : Lavf57.25.100
  Duration: 00:57:04.42, start: 0.000000, bitrate: 2264 kb/s
    Stream #0:0: Video: hevc (Main), yuv420p(tv), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 1k tbn, 25 tbc (default)
    Metadata:
      DURATION        : 00:57:04.411000000
    Stream #0:1: Audio: eac3, 48000 Hz, 5.1(side), fltp, 384 kb/s (default)
    Metadata:
      ENCODER         : Lavc57.24.102 eac3
      DURATION        : 00:57:04.415000000
    Stream #0:2: Subtitle: ass
    Metadata:
      ENCODER         : Lavc57.24.102 ass
      DURATION        : 00:54:12.606000000
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (eac3 (native) -> mp3 (libmp3lame))
Press [q] to stop, [?] for help
[libx264 @ 0x3b6c700] subme=0 + direct=temporal is not supported
[libx264 @ 0x3b6c700] using SAR=1/1
[libx264 @ 0x3b6c700] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0x3b6c700] profile Main, level 4.1
[libx264 @ 0x3b6c700] 264 - core 148 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=4 deblock=1:0:0 analyse=0x1:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=4 chroma_me=0 trellis=2 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 lookahead_threads=8 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=8 b_pyramid=2 b_adapt=2 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=10 rc=crf mbtree=1 crf=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 vbv_maxrate=4528 vbv_bufsize=9057 crf_max=0.0 nal_hrd=none filler=0 ip_ratio=1.40 aq=1:1.00
[segment @ 0x3ba5f60] Opening '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a80.ts' for writing
Output #0, segment, to '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a8%d.ts':
  Metadata:
    encoder         : Lavf58.0.102
    Stream #0:0: Video: h264 (libx264), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 25 fps, 90k tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc58.1.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 4528000/0/0 buffer size: 9057000 vbv_delay: -1
    Stream #0:1: Audio: mp3 (libmp3lame), 48000 Hz, stereo, fltp, 384 kb/s (default)
    Metadata:
      encoder         : Lavc58.1.100 libmp3lame
frame=   43 fps=0.0 q=0.0 size=N/A time=00:00:01.94 bitrate=N/A dup=3 drop=0 speed=2.86x    
frame=   73 fps= 61 q=7.0 size=N/A time=00:00:03.14 bitrate=N/A dup=4 drop=0 speed=2.63x    
frame=  102 fps= 60 q=9.0 size=N/A time=00:00:04.29 bitrate=N/A dup=5 drop=0 speed=2.51x    
[segment @ 0x3ba5f60] Opening '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a8.m3u8.tmp' for writing
[segment @ 0x3ba5f60] Opening '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a81.ts' for writing
frame=  135 fps= 61 q=6.0 size=N/A time=00:00:05.68 bitrate=N/A dup=6 drop=0 speed=2.57x    
frame=  163 fps= 58 q=6.0 size=N/A time=00:00:06.74 bitrate=N/A dup=8 drop=0 speed=2.41x    
frame=  180 fps= 54 q=12.0 size=N/A time=00:00:07.46 bitrate=N/A dup=8 drop=0 speed=2.24x    
frame=  193 fps= 50 q=11.0 size=N/A time=00:00:07.94 bitrate=N/A dup=9 drop=0 speed=2.05x    
[segment @ 0x3ba5f60] Opening '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a8.m3u8.tmp' for writing
[segment @ 0x3ba5f60] Opening '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a82.ts' for writing
frame=  204 fps= 42 q=16.0 size=N/A time=00:00:08.42 bitrate=N/A dup=9 drop=0 speed=1.74x    
frame=  225 fps= 42 q=24.0 size=N/A time=00:00:09.26 bitrate=N/A dup=10 drop=0 speed=1.73x    
frame=  241 fps= 41 q=26.0 size=N/A time=00:00:09.86 bitrate=N/A dup=11 drop=0 speed=1.68x    
frame=  257 fps= 40 q=25.0 size=N/A time=00:00:10.53 bitrate=N/A dup=11 drop=0 speed=1.64x    
frame=  273 fps= 39 q=25.0 size=N/A time=00:00:11.16 bitrate=N/A dup=12 drop=0 speed=1.58x    
[segment @ 0x3ba5f60] Opening '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a8.m3u8.tmp' for writing
[segment @ 0x3ba5f60] Opening '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a83.ts' for writing
frame=  287 fps= 38 q=25.0 size=N/A time=00:00:11.68 bitrate=N/A dup=13 drop=0 speed=1.53x    
frame=  303 fps= 37 q=25.0 size=N/A time=00:00:12.40 bitrate=N/A dup=13 drop=0 speed=1.51x    
frame=  322 fps= 37 q=26.0 size=N/A time=00:00:13.10 bitrate=N/A dup=14 drop=0 speed= 1.5x    
frame=  335 fps= 36 q=27.0 size=N/A time=00:00:13.60 bitrate=N/A dup=15 drop=0 speed=1.46x    
[segment @ 0x3ba5f60] Opening '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a8.m3u8.tmp' for writing
[segment @ 0x3ba5f60] Opening '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a84.ts' for writing
[segment @ 0x3ba5f60] Opening '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a8.m3u8.tmp' for writing
frame=  344 fps= 33 q=-1.0 Lsize=N/A time=00:00:14.04 bitrate=N/A dup=15 drop=0 speed=1.33x    
video:5397kB audio:549kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libx264 @ 0x3b6c700] frame I:5     Avg QP: 6.97  size:117107
[libx264 @ 0x3b6c700] frame P:139   Avg QP:13.32  size: 28762
[libx264 @ 0x3b6c700] frame B:200   Avg QP:11.58  size:  4710
[libx264 @ 0x3b6c700] consecutive B-frames: 19.5% 20.9% 15.7%  5.8%  7.3%  3.5%  2.0%  7.0% 18.3%
[libx264 @ 0x3b6c700] mb I  I16..4: 55.2%  0.0% 44.8%
[libx264 @ 0x3b6c700] mb P  I16..4: 10.4%  0.0%  0.0%  P16..4: 46.4%  0.0%  0.0%  0.0%  0.0%    skip:43.2%
[libx264 @ 0x3b6c700] mb B  I16..4:  2.2%  0.0%  0.0%  B16..8:  8.8%  0.0%  0.0%  direct: 5.2%  skip:83.9%  L0:36.9% L1:42.6% BI:20.5%
[libx264 @ 0x3b6c700] coded y,uvDC,uvAC intra: 38.1% 21.6% 9.8% inter: 12.0% 7.7% 0.3%
[libx264 @ 0x3b6c700] i16 v,h,dc,p: 56% 20% 19%  5%
[libx264 @ 0x3b6c700] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 24% 13%  6%  9%  5% 10%  3%  9%
[libx264 @ 0x3b6c700] i8c dc,h,v,p: 69% 13% 15%  2%
[libx264 @ 0x3b6c700] Weighted P-Frames: Y:16.5% UV:5.0%
[libx264 @ 0x3b6c700] ref P L0: 82.7%  0.0% 13.1%  3.5%  0.7%  0.0%
[libx264 @ 0x3b6c700] ref B L0: 91.7%  7.5%  0.8%
[libx264 @ 0x3b6c700] ref B L1: 99.5%  0.5%
[libx264 @ 0x3b6c700] kb/s:3212.50

[Jdiesel 1113]

Is there an issue? Do you expect it to use more or less of your CPU?

[TehEpikDuckeh 6]

It should max out the CPU, it shouldn't be throttling but it seems to be.

[Luke 37007]

If you really feel there is an issue here then I think you will have to take it to the ffmpeg issue tracker.

[TehEpikDuckeh 6]

@@Luke, does it max out your CPU when transcoding?

[mastrmind11 717]

@@Luke, does it max out your CPU when transcoding?

I have a xeon e3 and it doesn't max me out.

[TehEpikDuckeh 6]

What's your max CPU usage when transcoding?

[mastrmind11 717]

3-4 cores @ 75% avgish load  It's ffmpeg, not Emby.  Emby is simply sending a command line arg...

[TehEpikDuckeh 6]

Right, I just wanted to see if other people were having the same issue and if there were any quick fixes.

[Andy777 21]

Right, I just wanted to see if other people were having the same issue and if there were any quick fixes.

 

All algorithms cannot be parallelized with perfect linear scalability (although plain video work is easier). Remember that 4 out of 8 logical cores in your i7 are hyperthreaded and therefore "slower" (simplification).

 

The stream will also need to be demuxed first (separate job) before fed to the decoder(s). Audio is also transcode, so that might be faster, and  thus waiting for video transcoding to catch up (if the video thread count stays constant). In the end, transcoded audio and video need to be in sync and remuxed into the container (once again, the remuxing is another job).

 

So simplified: sometimes the threads need to wait the other threads.

 

If there is no hard upper limit coded in the libx264/ffmpeg/libmp3lame, you could TRY to overload your system by forcing more threads that your cpu can simultaneously handle. You would then get 100% usage, but not necessarily any benefit in transcoding speed (due to extra context switching/cache trashing in the cpu, and additional thread cross communication).

 

Just for general information: in your example the video is encoded by libx264 and audio by libmp3lame, which are both developed outside the ffmpeg project. They are included (optionally) to the ffmpeg as external libraries when somebody builds/compiles ffmpeg. So the right place to ask more specific questions would be the x264 project and libmp3lame project.

 

BR

Andy777

Edited November 8, 2017 by Andy777

[TehEpikDuckeh 6]

Thanks for the information. Something I just tried was altering the arguments for FFmpeg and testing in Terminal. What I found out is that when removing either "-maxrate 3616002" and/or "-bufsize 7232004" the CPU went to 90-95% utilization. But, the FPS dropped from 120 fps to about 80 fps. Now, this is from forcing Emby to essentially transcode by changing the bitrate of the stream to 4 MB/s. Changing that to 6 MB/s sees a decrease to about 70% CPU utilization but decreased FPS. I'm assuming that's due to encoding to a higher bitrate, but wouldn't the opposite -- being closer to the source bitrate -- be true as well?