Jump to content


Photo

FFmpeg only using 75% of CPU when Transcoding

Debian 8 Emby Server Beta FFmpeg

  • Please log in to reply
10 replies to this topic

#1 TehEpikDuckeh OFFLINE  

TehEpikDuckeh

    Advanced Member

  • Members
  • 94 posts
  • Local time: 12:54 PM

Posted 06 November 2017 - 10:49 PM

Transcoding only uses 75% of the CPU, only 7 out of 8 threads.

 

Processor: i7-2600 @ 3.8 GHz

 

Transcoding log:

Application version: 3.2.36.2

http://192.168.0.3:40330/emby/videos/418d80970834ea0588c1bb0697edb255/hls1/main/0.ts?DeviceId=0f646ede5a6cee1566fe5433befcb07c53005e86&MediaSourceId=418d80970834ea0588c1bb0697edb255&VideoCodec=h264&AudioCodec=mp3,aac&AudioStreamIndex=1&VideoBitrate=59616000&AudioBitrate=384000&PlaySessionId=7bc5ffbd11c4477fa037e48c7502b667&api_key=99bd3ef4dc7841fbb78343b4dded2dce&CopyTimestamps=false&TranscodingMaxAudioChannels=2&EnableSubtitlesInManifest=false&Tag=f6d58280eee7c62ff832b22128ed19cd&RequireAvc=true&RequireNonAnamorphic=false&SegmentContainer=ts&MinSegments=1&BreakOnNonKeyFrames=True&h264-profile=high,main,baseline,constrainedbaseline&h264-level=51&h264-deinterlace=true

{"Protocol":"File","Id":"418d80970834ea0588c1bb0697edb255","Path":"/media/HDD/TV Shows/The Man in the High Castle/Season 2/S02E01 - The Tiger's Cave.mkv","Type":"Default","Container":"mkv,webm","Name":"1080P/HEVC/EAC3","IsRemote":false,"ETag":"f6d58280eee7c62ff832b22128ed19cd","RunTimeTicks":34244149248,"ReadAtNativeFramerate":false,"IgnoreDts":false,"IgnoreIndex":false,"GenPtsInput":false,"SupportsTranscoding":true,"SupportsDirectStream":true,"SupportsDirectPlay":true,"IsInfiniteStream":false,"RequiresOpening":false,"RequiresClosing":false,"SupportsProbing":true,"RequiresLooping":false,"VideoType":"VideoFile","MediaStreams":[{"Codec":"hevc","TimeBase":"1/1000","CodecTimeBase":"1/25","IsInterlaced":false,"BitRate":2264283,"RefFrames":1,"IsDefault":true,"IsForced":false,"Height":1080,"Width":1920,"AverageFrameRate":25,"RealFrameRate":25,"Profile":"Main","Type":"Video","AspectRatio":"16:9","Index":0,"IsExternal":false,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"PixelFormat":"yuv420p","Level":120,"IsAnamorphic":false},{"Codec":"eac3","TimeBase":"1/1000","CodecTimeBase":"1/48000","DisplayTitle":"Dolby Digital+ 5.1 Default","IsInterlaced":false,"ChannelLayout":"5.1","BitRate":384000,"Channels":6,"SampleRate":48000,"IsDefault":true,"IsForced":false,"Type":"Audio","Index":1,"IsExternal":false,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Level":0,"IsAnamorphic":false},{"Codec":"ass","TimeBase":"1/1000","CodecTimeBase":"0/1","DisplayTitle":"Und","IsInterlaced":false,"IsDefault":false,"IsForced":false,"Type":"Subtitle","Index":2,"IsExternal":false,"IsTextSubtitleStream":true,"SupportsExternalStream":true,"Level":0,"IsAnamorphic":false},{"Codec":"srt","Language":"eng","DisplayTitle":"Eng","IsInterlaced":false,"IsDefault":false,"IsForced":false,"Type":"Subtitle","Index":3,"IsExternal":true,"IsTextSubtitleStream":true,"SupportsExternalStream":true,"Path":"/media/HDD/TV Shows/The Man in the High Castle/Season 2/S02E01 - The Tiger's Cave.eng.srt","IsAnamorphic":true}],"Formats":[],"Bitrate":2648283,"RequiredHttpHeaders":{}}

User policy for Mark: EnablePlaybackRemuxing: True EnableVideoPlaybackTranscoding: True EnableAudioPlaybackTranscoding: True

/var/lib/emby-server/ffmpeg/ffmpeg/ffmpeg -f matroska,webm -i file:"/media/HDD/TV Shows/The Man in the High Castle/Season 2/S02E01 - The Tiger's Cave.mkv" -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 libx264 -vf "scale=trunc(min(max(iw\,ih*dar)\,1920)/2)*2:trunc(ow/dar/2)*2" -pix_fmt yuv420p -preset veryslow -crf 1 -maxrate 4528566 -bufsize 9057132 -profile:v high -level 4.1 -x264opts:0 subme=0:me_range=4:rc_lookahead=10:me=dia:no_chroma_me:8x8dct=0:partitions=none -force_key_frames "expr:if(isnan(prev_forced_t),eq(t,t),gte(t,prev_forced_t+3))" -copyts -vsync -1 -codec:a:0 libmp3lame -ac 2 -ab 384000  -f segment -max_delay 5000000 -avoid_negative_ts disabled -map_metadata -1 -map_chapters -1 -start_at_zero -segment_time 3  -individual_header_trailer 0 -segment_format mpegts -segment_list_type m3u8 -segment_start_number 0 -segment_list "/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a8.m3u8" -y "/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a8%d.ts"

ffmpeg version N-88289-g783535a Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 4.9.2 (Debian 4.9.2-10)
  configuration: --enable-encoder=hevc_nvenc --enable-nonfree --enable-libx264 --enable-libx265 --enable-libfreetype --enable-libass --enable-gpl --enable-libvpx --enable-libtheora --enable-libopus --enable-libmp3lame --enable-libfdk-aac --enable-version3 --disable-w32threads --enable-runtime-cpudetect --enable-avisynth --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libgsm --enable-libspeex --enable-libxvid --enable-zlib
  libavutil      56.  0.100 / 56.  0.100
  libavcodec     58.  1.100 / 58.  1.100
  libavformat    58.  0.102 / 58.  0.102
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7.  0.101 /  7.  0.101
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.100 /  3.  0.100
  libpostproc    55.  0.100 / 55.  0.100
Input #0, matroska,webm, from 'file:/media/HDD/TV Shows/The Man in the High Castle/Season 2/S02E01 - The Tiger's Cave.mkv':
  Metadata:
    ENCODER         : Lavf57.25.100
  Duration: 00:57:04.42, start: 0.000000, bitrate: 2264 kb/s
    Stream #0:0: Video: hevc (Main), yuv420p(tv), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 1k tbn, 25 tbc (default)
    Metadata:
      DURATION        : 00:57:04.411000000
    Stream #0:1: Audio: eac3, 48000 Hz, 5.1(side), fltp, 384 kb/s (default)
    Metadata:
      ENCODER         : Lavc57.24.102 eac3
      DURATION        : 00:57:04.415000000
    Stream #0:2: Subtitle: ass
    Metadata:
      ENCODER         : Lavc57.24.102 ass
      DURATION        : 00:54:12.606000000
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (eac3 (native) -> mp3 (libmp3lame))
Press [q] to stop, [?] for help
[libx264 @ 0x3b6c700] subme=0 + direct=temporal is not supported
[libx264 @ 0x3b6c700] using SAR=1/1
[libx264 @ 0x3b6c700] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0x3b6c700] profile Main, level 4.1
[libx264 @ 0x3b6c700] 264 - core 148 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=4 deblock=1:0:0 analyse=0x1:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=4 chroma_me=0 trellis=2 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 lookahead_threads=8 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=8 b_pyramid=2 b_adapt=2 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=10 rc=crf mbtree=1 crf=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 vbv_maxrate=4528 vbv_bufsize=9057 crf_max=0.0 nal_hrd=none filler=0 ip_ratio=1.40 aq=1:1.00
[segment @ 0x3ba5f60] Opening '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a80.ts' for writing
Output #0, segment, to '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a8%d.ts':
  Metadata:
    encoder         : Lavf58.0.102
    Stream #0:0: Video: h264 (libx264), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 25 fps, 90k tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc58.1.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 4528000/0/0 buffer size: 9057000 vbv_delay: -1
    Stream #0:1: Audio: mp3 (libmp3lame), 48000 Hz, stereo, fltp, 384 kb/s (default)
    Metadata:
      encoder         : Lavc58.1.100 libmp3lame
frame=   43 fps=0.0 q=0.0 size=N/A time=00:00:01.94 bitrate=N/A dup=3 drop=0 speed=2.86x    
frame=   73 fps= 61 q=7.0 size=N/A time=00:00:03.14 bitrate=N/A dup=4 drop=0 speed=2.63x    
frame=  102 fps= 60 q=9.0 size=N/A time=00:00:04.29 bitrate=N/A dup=5 drop=0 speed=2.51x    
[segment @ 0x3ba5f60] Opening '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a8.m3u8.tmp' for writing
[segment @ 0x3ba5f60] Opening '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a81.ts' for writing
frame=  135 fps= 61 q=6.0 size=N/A time=00:00:05.68 bitrate=N/A dup=6 drop=0 speed=2.57x    
frame=  163 fps= 58 q=6.0 size=N/A time=00:00:06.74 bitrate=N/A dup=8 drop=0 speed=2.41x    
frame=  180 fps= 54 q=12.0 size=N/A time=00:00:07.46 bitrate=N/A dup=8 drop=0 speed=2.24x    
frame=  193 fps= 50 q=11.0 size=N/A time=00:00:07.94 bitrate=N/A dup=9 drop=0 speed=2.05x    
[segment @ 0x3ba5f60] Opening '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a8.m3u8.tmp' for writing
[segment @ 0x3ba5f60] Opening '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a82.ts' for writing
frame=  204 fps= 42 q=16.0 size=N/A time=00:00:08.42 bitrate=N/A dup=9 drop=0 speed=1.74x    
frame=  225 fps= 42 q=24.0 size=N/A time=00:00:09.26 bitrate=N/A dup=10 drop=0 speed=1.73x    
frame=  241 fps= 41 q=26.0 size=N/A time=00:00:09.86 bitrate=N/A dup=11 drop=0 speed=1.68x    
frame=  257 fps= 40 q=25.0 size=N/A time=00:00:10.53 bitrate=N/A dup=11 drop=0 speed=1.64x    
frame=  273 fps= 39 q=25.0 size=N/A time=00:00:11.16 bitrate=N/A dup=12 drop=0 speed=1.58x    
[segment @ 0x3ba5f60] Opening '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a8.m3u8.tmp' for writing
[segment @ 0x3ba5f60] Opening '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a83.ts' for writing
frame=  287 fps= 38 q=25.0 size=N/A time=00:00:11.68 bitrate=N/A dup=13 drop=0 speed=1.53x    
frame=  303 fps= 37 q=25.0 size=N/A time=00:00:12.40 bitrate=N/A dup=13 drop=0 speed=1.51x    
frame=  322 fps= 37 q=26.0 size=N/A time=00:00:13.10 bitrate=N/A dup=14 drop=0 speed= 1.5x    
frame=  335 fps= 36 q=27.0 size=N/A time=00:00:13.60 bitrate=N/A dup=15 drop=0 speed=1.46x    
[segment @ 0x3ba5f60] Opening '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a8.m3u8.tmp' for writing
[segment @ 0x3ba5f60] Opening '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a84.ts' for writing
[segment @ 0x3ba5f60] Opening '/media/HDD/Temp/transcoding-temp/7272513b7e30642ec1ad89cf3cced6a8.m3u8.tmp' for writing
frame=  344 fps= 33 q=-1.0 Lsize=N/A time=00:00:14.04 bitrate=N/A dup=15 drop=0 speed=1.33x    
video:5397kB audio:549kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libx264 @ 0x3b6c700] frame I:5     Avg QP: 6.97  size:117107
[libx264 @ 0x3b6c700] frame P:139   Avg QP:13.32  size: 28762
[libx264 @ 0x3b6c700] frame B:200   Avg QP:11.58  size:  4710
[libx264 @ 0x3b6c700] consecutive B-frames: 19.5% 20.9% 15.7%  5.8%  7.3%  3.5%  2.0%  7.0% 18.3%
[libx264 @ 0x3b6c700] mb I  I16..4: 55.2%  0.0% 44.8%
[libx264 @ 0x3b6c700] mb P  I16..4: 10.4%  0.0%  0.0%  P16..4: 46.4%  0.0%  0.0%  0.0%  0.0%    skip:43.2%
[libx264 @ 0x3b6c700] mb B  I16..4:  2.2%  0.0%  0.0%  B16..8:  8.8%  0.0%  0.0%  direct: 5.2%  skip:83.9%  L0:36.9% L1:42.6% BI:20.5%
[libx264 @ 0x3b6c700] coded y,uvDC,uvAC intra: 38.1% 21.6% 9.8% inter: 12.0% 7.7% 0.3%
[libx264 @ 0x3b6c700] i16 v,h,dc,p: 56% 20% 19%  5%
[libx264 @ 0x3b6c700] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 24% 13%  6%  9%  5% 10%  3%  9%
[libx264 @ 0x3b6c700] i8c dc,h,v,p: 69% 13% 15%  2%
[libx264 @ 0x3b6c700] Weighted P-Frames: Y:16.5% UV:5.0%
[libx264 @ 0x3b6c700] ref P L0: 82.7%  0.0% 13.1%  3.5%  0.7%  0.0%
[libx264 @ 0x3b6c700] ref B L0: 91.7%  7.5%  0.8%
[libx264 @ 0x3b6c700] ref B L1: 99.5%  0.5%
[libx264 @ 0x3b6c700] kb/s:3212.50


#2 Jdiesel OFFLINE  

Jdiesel

    Advanced Member

  • Members
  • 2625 posts
  • Local time: 02:54 PM
  • LocationRegina, SK

Posted 06 November 2017 - 10:58 PM

Is there an issue? Do you expect it to use more or less of your CPU?



#3 TehEpikDuckeh OFFLINE  

TehEpikDuckeh

    Advanced Member

  • Members
  • 94 posts
  • Local time: 12:54 PM

Posted 07 November 2017 - 03:15 AM

It should max out the CPU, it shouldn't be throttling but it seems to be.



#4 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 128632 posts
  • Local time: 04:54 PM

Posted 07 November 2017 - 01:16 PM

If you really feel there is an issue here then I think you will have to take it to the ffmpeg issue tracker.

#5 TehEpikDuckeh OFFLINE  

TehEpikDuckeh

    Advanced Member

  • Members
  • 94 posts
  • Local time: 12:54 PM

Posted 07 November 2017 - 06:49 PM

@Luke, does it max out your CPU when transcoding?



#6 mastrmind11 OFFLINE  

mastrmind11

    Advanced Member

  • Members
  • 2667 posts
  • Local time: 04:54 PM
  • LocationLong Island, NY

Posted 07 November 2017 - 06:53 PM

@Luke, does it max out your CPU when transcoding?

I have a xeon e3 and it doesn't max me out.



#7 TehEpikDuckeh OFFLINE  

TehEpikDuckeh

    Advanced Member

  • Members
  • 94 posts
  • Local time: 12:54 PM

Posted 07 November 2017 - 06:54 PM

What's your max CPU usage when transcoding?



#8 mastrmind11 OFFLINE  

mastrmind11

    Advanced Member

  • Members
  • 2667 posts
  • Local time: 04:54 PM
  • LocationLong Island, NY

Posted 07 November 2017 - 06:56 PM

3-4 cores @ 75% avgish load  It's ffmpeg, not Emby.  Emby is simply sending a command line arg...



#9 TehEpikDuckeh OFFLINE  

TehEpikDuckeh

    Advanced Member

  • Members
  • 94 posts
  • Local time: 12:54 PM

Posted 07 November 2017 - 06:57 PM

Right, I just wanted to see if other people were having the same issue and if there were any quick fixes.



#10 Andy777 OFFLINE  

Andy777

    Advanced Member

  • Members
  • 117 posts
  • Local time: 10:54 PM

Posted 08 November 2017 - 11:53 AM

Right, I just wanted to see if other people were having the same issue and if there were any quick fixes.

 

All algorithms cannot be parallelized with perfect linear scalability (although plain video work is easier). Remember that 4 out of 8 logical cores in your i7 are hyperthreaded and therefore "slower" (simplification).

 

The stream will also need to be demuxed first (separate job) before fed to the decoder(s). Audio is also transcode, so that might be faster, and  thus waiting for video transcoding to catch up (if the video thread count stays constant). In the end, transcoded audio and video need to be in sync and remuxed into the container (once again, the remuxing is another job).

 

So simplified: sometimes the threads need to wait the other threads.

 

If there is no hard upper limit coded in the libx264/ffmpeg/libmp3lame, you could TRY to overload your system by forcing more threads that your cpu can simultaneously handle. You would then get 100% usage, but not necessarily any benefit in transcoding speed (due to extra context switching/cache trashing in the cpu, and additional thread cross communication).

 

Just for general information: in your example the video is encoded by libx264 and audio by libmp3lame, which are both developed outside the ffmpeg project. They are included (optionally) to the ffmpeg as external libraries when somebody builds/compiles ffmpeg. So the right place to ask more specific questions would be the x264 project and libmp3lame project.

 

BR

Andy777


Edited by Andy777, 08 November 2017 - 11:54 AM.

  • thefirstofthe300 likes this

#11 TehEpikDuckeh OFFLINE  

TehEpikDuckeh

    Advanced Member

  • Members
  • 94 posts
  • Local time: 12:54 PM

Posted 09 November 2017 - 12:41 AM

Thanks for the information. Something I just tried was altering the arguments for FFmpeg and testing in Terminal. What I found out is that when removing either "-maxrate 3616002" and/or "-bufsize 7232004" the CPU went to 90-95% utilization. But, the FPS dropped from 120 fps to about 80 fps. Now, this is from forcing Emby to essentially transcode by changing the bitrate of the stream to 4 MB/s. Changing that to 6 MB/s sees a decrease to about 70% CPU utilization but decreased FPS. I'm assuming that's due to encoding to a higher bitrate, but wouldn't the opposite -- being closer to the source bitrate -- be true as well?







Also tagged with one or more of these keywords: Debian 8, Emby Server Beta, FFmpeg

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users