Transcoding crippling the CPU?

[gunnarniels 0]

I'm running a contianerized emby (https://github.com/dperson/emby) on my CentOS 7 server, which doesn't have the most up to date hardware, but should be more than enough for transcoding purposes (i7 950 - 32GB RAM). I fired up emby on my roku and started watching an HD mp4 of about 1.5 hours, and it brought the machine to an absolute crawl. I'm seeing tons of ffmpeg processes and "MediaBrowser" processes using up more than one core at 100%. It's enough to actually impact my ssh session.

Something seems really wrong here. Could anyone advise? 

[Luke 42077]

Hi there, we're sorry to hear about this. The best thing to do is please provide the information requested in how to report a problem. Thanks !

[gunnarniels 0]

Wow, thank you for the quick reply! I spun up a new container with only this file. Attached are the transcoding and server logs. It takes a very long time to "retrieve" on the roku before it starts playing.

ffmpeg-transcode-9d64a84d-9ff6-4ab4-b61c-9b3e3694961e.txt

server-63614816152.txt

[Luke 42077]

have you tried increasing the in-app bitrate setting on the roku to try and avoid transcoding in the first place?

[gunnarniels 0]

have you tried increasing the in-app bitrate setting on the roku to try and avoid transcoding in the first place?

I have not; I will give that a try, but that's kind of a workaround and not really addressing the root cause. I'm running Emby specifically because I like the serverside transcoding if it's necessary.

 

EDIT: I was using the default video quality setting of 3.2 Mbps. Obviously you can't know the quality of my wireless network, but in theory, I should certainly be able to pull 30 Mbps over 2.5GHz wireless on a LAN?

Edited November 15, 2016 by gunnarniels

[lamero 1]

Try to disable CPU throttling

[gunnarniels 0]

Try to disable CPU throttling

Is this Emby side? Server side? I have no idea what you're suggesting.

[Luke 42077]

I don't necessarily consider this a workaround, because the bitrate setting not only affects direct play, but it also affects what transcoding methods are going to be utilized. At 3mbps if the video stream has a high enough bitrate, you're going to end up with a full video transcode and ffmpeg is designed to run at full blast until it completes.

 

If you raise the bitrate setting, not only do you get more direct play but you also open up the possibility of more video/audio stream copy. For example, suppose a video file requires conversion due to incompatible audio or subitles, we might be able to avoid the video conversion and just convert the audio/subs which by comparison is very cheap.

 

If you really want to improve the performance of the full video transcode, you probably want to look at our hardware transcoding options, but I would always consider that secondary to avoiding the transcoding in the first place.

[Luke 42077]

Just for further emphasis on the bitrate setting, we are working on an all-new app for Roku and it will have an automatic bitrate setting just like most of our other apps do. So that means you'll get a higher bitrate out of the box without having to do anything and this will also mean less transcoding. So what I'm asking you to do right now is simply what the new app will eventually do for you.

[gunnarniels 0]

First, thank you for you immediate attention and commitment to the project; this is exactly why I choose Emby over alternatives and am interested in working towards a better community solution.

Correct me if I am wrong, but I'm either going to be transcoding, or I'm not. Fiddling with client bitrates may or may not trigger this serverside, but my primary complaint is that when I _am_ transcoding, emby server completely overtakes my hardware, and this is very much unexpected. I just don't think a full video transcode initiated by emby for a single client should be enough to completely arrest an i7 950. Is this unexpected behavior, or am I underestimating how heavy this workload actually is? Do I need to do something additional to enable some kind of hardware acceleration?

As an aside, I'm a dev (many years in C#/JS/whatever), tried to ping you guys in freenode #emby but no response. Where do you primarily congregate? I would love to contribute.

Edited November 16, 2016 by gunnarniels

[Luke 42077]

That's why we have a throttling feature where once the transcode gets far enough ahead of the user playback position it will pause ffmpeg. You can also customize the transcoding thread count to 1 because by default it will let ffmpeg choose how many threads it uses. But ffmpeg does have any options to control how fast it goes, it is designed to go as fast as possible and will use all resources that you offer it. That's why your best bet is to avoid the transcoding and then when it can't be avoided, you can look at configuration such as the thread count and the experimental gpu transcoding settings.