Identify bottlenecks during transcoding

[B3nZ3n 0]

Hello,

 

I successfully installed emby server on my raspberry pi 2 and the server runs smoothly.

 

Unfortunately I encounter somme "lags" when playing a media via the web client.

 

• CPU seems to have some margin (load average ~3 on quadcore cpu)
• network seems ok (ethernet on both ends) since playing the media once the transcoding is finished is OK (also, playing a very low res media is also laggy)
• media is located on the sd card and temp transcode folder is also on the sd card 
• Sample from transcoding logs:

frame= 2154 fps= 15 q=0.0 size=   10069kB time=00:01:26.31 bitrate= 955.6kbits/s
frame= 2161 fps= 15 q=0.0 size=   10095kB time=00:01:26.63 bitrate= 954.5kbits/s
frame= 2167 fps= 15 q=0.0 size=   10143kB time=00:01:26.82 bitrate= 957.0kbits/s
frame= 2173 fps= 15 q=0.0 size=   10173kB time=00:01:27.16 bitrate= 956.1kbits/s

My question is: how would you investigate the bottleneck? How should I interprate the transcoding logs?

Does anyone was able to have a smooth transcode using a raspberrypi 2?

 

 

 

Many thanks

Edited April 15, 2015 by B3nZ3n

[B3nZ3n 0]

CPU status on raspberry pi 2:

 

 

Ethernet status on the client (web client on chrome win 8):

[B3nZ3n 0]

Note that some files can be transcoded smoothly... 

Edited April 15, 2015 by B3nZ3n

[thefirstofthe300 291]

A raspberry pi simply isn't going to have the transcoding power to be able to transcode one codec to another on the fly. As you can see, ffmpeg is only able to transcode 15 frames/second (about half the speed of a normal video). I would suggest installing Emby on a more powerful server if you need on demand transcoding or if you can anticipate what you are going to watch, using folder sync as a kind of "pre-transcoder."

 

What you are probably seeing when files are transcoding smoothly is a simple direct copy in which the pi doesn't actually transcode any of the streams to another codec, it just changes the container format.

[B3nZ3n 0]

Thanks for your reply,

 

I had issues with my usb cable wich was not able to provide enough voltage to my pi. Now that is fixed I can clearly see that the CPU spikes at 100% when transcoding.

 

The bottleneck is clearly the CPU

 

 

Thanks for the suggestion, but I do not want to power another machine to transcode on the fly..  As for the sd media, it seems to work, I ll keep it the way it is right now.

 

Can you tell me if, in the following log, transcode occurs or not?

Input #0, matroska,webm, from 'file:/media/2/TV/Kaamelott/Livre II/Épisodes/Kaamelott S02E080  Les Parchemins Magiques.mkv':
  Metadata:
    creation_time   : 2015-02-16 11:28:57
    ENCODER         : Lavf55.12.0
  Duration: 00:03:45.05, start: 0.000000, bitrate: 1773 kb/s
    Chapter #0:0: start 0.000000, end 225.032000
    Metadata:
      title           : Chapter 30
    Stream #0:0: Video: h264 (High), yuv420p(tv, smpte170m/bt470bg/bt709), 720x410, SAR 1:1 DAR 72:41, 25 fps, 25 tbr, 1k tbn, 50 tbc (default)
    Metadata:
      BPS             : 1578722
      BPS-eng         : 1578722
      DURATION        : 00:03:45.040000000
      DURATION-eng    : 00:03:45.040000000
      NUMBER_OF_FRAMES: 5626
      NUMBER_OF_FRAMES-eng: 5626
      NUMBER_OF_BYTES : 44409473
      NUMBER_OF_BYTES-eng: 44409473
      _STATISTICS_WRITING_APP: mkvmerge v7.6.0 ('Garden of Dreams') 64bit built on Feb  8 2015 13:04:34
      _STATISTICS_WRITING_APP-eng: mkvmerge v7.6.0 ('Garden of Dreams') 64bit built on Feb  8 2015 13:04:34
      _STATISTICS_WRITING_DATE_UTC: 2015-02-16 11:28:57
      _STATISTICS_WRITING_DATE_UTC-eng: 2015-02-16 11:28:57
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
    Stream #0:1(fre): Audio: aac (LC), 48000 Hz, stereo, fltp (default)
    Metadata:
      title           : Stereo
      LANGUAGE        : fre
      BPS             : 192440
      BPS-eng         : 192440
      DURATION        : 00:03:45.045000000
      DURATION-eng    : 00:03:45.045000000
      NUMBER_OF_FRAMES: 10549
      NUMBER_OF_FRAMES-eng: 10549
      NUMBER_OF_BYTES : 5413482
      NUMBER_OF_BYTES-eng: 5413482
      _STATISTICS_WRITING_APP: mkvmerge v7.6.0 ('Garden of Dreams') 64bit built on Feb  8 2015 13:04:34
      _STATISTICS_WRITING_APP-eng: mkvmerge v7.6.0 ('Garden of Dreams') 64bit built on Feb  8 2015 13:04:34
      _STATISTICS_WRITING_DATE_UTC: 2015-02-16 11:28:57
      _STATISTICS_WRITING_DATE_UTC-eng: 2015-02-16 11:28:57
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
[libx264 @ 0x20defc0] using SAR=819/820
[libx264 @ 0x20defc0] using cpu capabilities: ARMv6 NEON
[libx264 @ 0x20defc0] profile High, level 4.0
Output #0, hls, to '/home/pi/emby/ProgramData-Server/transcoding-temp/30a1f7c61353cd65b2eef725a3463054.m3u8':
  Metadata:
    encoder         : Lavf56.25.101
    Chapter #0:0: start 0.000000, end 225.032000
    Stream #0:0: Video: h264 (libx264), yuv420p, 640x364 [SAR 819:820 DAR 72:41], q=-1--1, max. 362 kb/s, 25 fps, 90k tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc56.26.100 libx264
    Stream #0:1: Audio: aac, 48000 Hz, 5.0, fltp, 510 kb/s (default)
    Metadata:
      encoder         : Lavc56.26.100 aac
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
frame=    7 fps=0.0 q=0.0 size=N/A time=00:00:00.56 bitrate=N/A
frame=   16 fps= 16 q=20.0 size=N/A time=00:00:00.98 bitrate=N/A
frame=   26 fps= 17 q=20.0 size=N/A time=00:00:01.33 bitrate=N/A
...
frame= 1791 fps= 18 q=28.0 size=N/A time=00:01:11.96 bitrate=N/A
frame= 1800 fps= 18 q=22.0 size=N/A time=00:01:12.32 bitrate=N/A
frame= 1808 fps= 17 q=-1.0 Lsize=N/A time=00:01:12.71 bitrate=N/A
video:3164kB audio:4493kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libx264 @ 0x20defc0] frame I:25    Avg QP:30.51  size:  9807
[libx264 @ 0x20defc0] frame P:735   Avg QP:30.91  size:  3073
[libx264 @ 0x20defc0] frame B:1048  Avg QP:31.05  size:   703
[libx264 @ 0x20defc0] consecutive B-frames: 13.7% 20.2% 20.2% 45.8%
[libx264 @ 0x20defc0] mb I  I16..4: 30.8% 50.9% 18.3%
[libx264 @ 0x20defc0] mb P  I16..4:  4.0%  5.4%  0.9%  P16..4: 45.7%  0.0%  0.0%  0.0%  0.0%    skip:43.9%
[libx264 @ 0x20defc0] mb B  I16..4:  0.2%  0.2%  0.0%  B16..8: 11.8%  0.0%  0.0%  direct: 6.8%  skip:81.0%  L0:27.3% L1:58.4% BI:14.2%
[libx264 @ 0x20defc0] 8x8 transform intra:52.1% inter:37.2%
[libx264 @ 0x20defc0] coded y,uvDC,uvAC intra: 48.4% 36.5% 10.8% inter: 8.9% 5.1% 0.1%
[libx264 @ 0x20defc0] i16 v,h,dc,p: 54% 20% 19%  8%
[libx264 @ 0x20defc0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19% 15% 26%  5%  7%  6%  6%  7%  8%
[libx264 @ 0x20defc0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 23% 19% 17%  5%  9%  6%  7%  6%  8%
[libx264 @ 0x20defc0] i8c dc,h,v,p: 57% 17% 21%  6%
[libx264 @ 0x20defc0] Weighted P-Frames: Y:4.5% UV:2.6%
[libx264 @ 0x20defc0] kb/s:358.45

[thefirstofthe300 291]

Yes that is transcoding. It is dropping the resolution from 720x410 to 640x364 and changing the aac audio too. As you can see, the FPS is hovering around 18 so unless your video is only an 18 FPS video (a standard video is somewhere around 29.x), you are going to get buffering.

[gsnerf 26]

Another tip for your use case: don't use the web client or pre-transcode all your video files to a format your usual browser can direct play. A video codec usually supported is h264, audio codec should be aac (but I'm not sure on that front, might need to be mp3). If you can use a client that can use the system installed codecs and install codecs for all your video formats.

 

Using one of these paths _should_ result in only needing direct play functionality, so only the container format is rewritten which is rather quickly done on the fly.

Edited April 18, 2015 by gsnerf

[B3nZ3n 0]

Thanks for your replies,

 

gsnerf, which "client" do you suggest? I tried VLC by opening the newtwork feed and providing the "download" link (ex: http://xxxxx.noip.me:8096/Items/4c5bf4c1fbdb6939cdf5fd16e96e159c/Download?api_key=xxxxx

It works well. Is there any way to easily to "direct play" a whole set of files (eg all episodes of a show)? 

[gsnerf 26]

I'd say try Kodi with the emby plugin.