One small step for ffmpeg... one giant leap for Emby! (Looking for C developer(s) to help with transcode throttling)

[Maudite 7]

Also, another question for you Luke.  To fully diagnose this, is there a way I can install previous versions (windows based server) so that I could find the versions that did not give me the startup framerate issue?  Then we could really dig into what changes/config may be causing this.

[Luke 37090]

yea the current stable release from the website.

[Maudite 7]

That is the one I am on now, and going back to it I can now switch quality to fix framerate again. I was thinking a few months older still, so perhaps I can find at what version it changed from initial startup playing smooth, to initial startup having poor framerate.

 

If I need to use GIT to switch between different versions, I would be open to that to really dig into where this changed.

[Luke 37090]

well the last stable release doesn't have transcoding throttling so now we're really unrelated to this topic.

[BAS 225]

the transcoding will finish. if for some reason it's not, then it needs to be reported and looked at. it could be completely unrelated. 

 

Well I'm back on developer, the transcoding finishes if its 1 user playing, maybe 2... its the fact that if 2 more users need to transcode and the other 2 havent finished, thats where I get to 100%. I will be home this evening hopefully so I'm switching this back to developer and making sure I have logging on. When primetime viewing hours hit this evening we shall see how it does.

[AgileHumor 123]

delete me, pii cleanup 

Edited April 17, 2015 by AgileHumor

[dark_slayer 103]

the issues of stranded ffmpeg processes related to throttling are resolved. i think the issue is more that your perception of the final result is based on your experience during early testing stages, and that's just not an accurate assessment. there could still be ffmpeg processes stranded but most likely it is due to the roku thumbnail plugin as there is work that needs to be done there.

Hi @@Luke

 

Resolved on dev or beta server?

 

I experienced this with fire TV running android app and casting from web to chromecast (manually closed ffmpeg in task manager 12hrs later). Never owned a roku

[BAS 225]

Yep for my user scenario and limited bandwidth it still wasn't quite right with Version 3.0.5567.23223 but I noticed a toggle under playback/transcoding for enable throttling, I unchecked it. Thanks for the toggle.

[MrWebsmith 598]

yea appreciate the toggle for the throttling.. thanks luke

[Luke 37090]

you although you guys are just telling yourselves what you think is a problem even though you have no evidence of it.

[BAS 225]

Would love for this toggle to still function... as of now I have shut down every external user I have as to prevent problems for my internal users.

[MrWebsmith 598]

agreed, something is still prevent my server from "getting ahead" like it used to on the transcode. ill post whatever logs are requested for analysis.

 

builds used: current dev and current beta both exhibit this

[CBers 6771]

agreed, something is still prevent my server from "getting ahead" like it used to on the transcode. ill post whatever logs are requested for analysis.

 

builds used: current dev and current beta both exhibit this

 

It's because ffmpeg has been amended to process, sleep, wake, process, sleep, wake, process . . . . . . . . . . . 

 

I posted about an issue when ffmpeg wakes, as playback can momentarily pause whilst ffmpeg starts processing again.

 

.

Edited June 1, 2015 by CBers

[techywarrior 688]

Not sure how that is all setup right now but it should at least process far enough ahead, or wake early enough, to prevent the pauses. Hopefully it's just a tweak to the parameters being used. Maybe they are a little too aggressive in trying to reduce CPU load. We could still get a huge benefit if an adjustment is made to keep it 1 minute ahead or something (or 1 minute more then now, whatever is needed)

[jabbera 23]

I've said it before and I'll say it again: there should be a setting that uses all (or a certain number of cores) idle CPU time for transcoding when there is something that isn't behind. I know people who don't have dedicated emby boxes don't want this, but those of us who have dedicated servers and host many users are in dire need of this.

Edited June 1, 2015 by jabbera

[BAS 225]

I just want to be able to open this to my external users again so whatever sleep/throttling option is available for everyone else and seems to be the focus that's all fine and great but I don't think its to much to ask to have an option to disable the sleep/throttling method and just have it get the copies done for my users. I have 4 internal users, 7 external, I have shut down access to my external users as the toggle to turn off throttling method no longer functions and I was tired of hearing how playback from external/transcoding users is horrible and they can't reach my server cause it goes down.

[dark_slayer 103]

7 people hitting the server for a transcode at the same time is not going to be smooth with or without throttling unless you have very solid upstream bandwidth from your ISP and a xeon

 

I know you keep saying it was working before but at best that was luck and not consistent or there were less transcoding users back then

 

I hope the option returns for you, but I think you'll still end up with some support calls

[MrWebsmith 598]

well i have a personally different reason for wanting it back... with 1 external user and a dedicated emby server box... why not let the transcode move ahead until done... get to the point where ffmeg is done and just streaming it for playback... why do i want to artificially hold up that process in my circumstance/useage if the cpu is available?

[BAS 225]

7 people hitting the server for a transcode at the same time is not going to be smooth with or without throttling unless you have very solid upstream bandwidth from your ISP and a xeon

 

I know you keep saying it was working before but at best that was luck and not consistent or there were less transcoding users back then

 

I hope the option returns for you, but I think you'll still end up with some support calls

 

Welp it was working just fine and has been for the exact reason that these users don't hit play all at the same time, and when they do there isnt a ffmpeg process running at the time cause it burned thru and finished the previous transcode from another user so there is no ffmpeg idling tasks involved taking cpu... and even when i do have multiple people hit play at the same time, it splits the ffmpeg cpu just fine, but still attempts to get them done. It's been working well for over a year until these changes went in, then we finally got a toggle to disable throttling method to go back to this way, now that toggle doesnt work.

 

I don't want to throttle and I don't think what I'm asking for is to much. This wasn't luck, its just how my users and I use my server and it worked correctly for well over a year, now they all don't have access because of these throttling methods and I will consider alternatives to Emby for my external family or wait for some response that this toggle functions again. 

[dark_slayer 103]

I think the way it's working right now is fantastic, and I don't see any evidence attached showing that it's failing

 

The current method transcodes at full steam to fill two minutes of playback in the temp server area. Then ffmpeg goes away completely

 

If you have seven people not pressing play all at the same time this should be working great with the method it's using now

 

When throttling was first introduced, anyone hitting pause would leave the server pegged with an ffmpeg process hovering around 14% CPU utilization

 

That's not an issue anymore. If you had a hundred users with transcoding paused your ffmpeg usage would be 0

 

MrWebsmith, just think of how much energy you'll save over the life of your emby server by not transcoding the credits for everything everyone watches (unless they want to watch them) as well as what people pause and come back later deciding not to watch. Eco-friendly ;-)

 

Anyway, jokes aside, like I said before I hope the option returns for you

Edited June 2, 2015 by dark_slayer

[ebr 14918]

I think the way it's working right now is fantastic, and I don't see any evidence attached showing that it's failing

 

The current method transcodes at full steam to fill two minutes of playback in the temp server area. Then ffmpeg goes away completely

 

If you have seven people not pressing play all at the same time this should be working great with the method it's using now

 

I think there is enough real evidence to believe what BAS says is happening is happening.  I can definitely see why it would be the case.

 

If you had 7 people who all hit play 15 minutes apart - if the transcode was going full-steam with no throttling your server would likely only ever have exactly one ffmpeg transcode going at any given time (because they would finish before the next person hit play).

 

if, however, the transcodes are always handled in a JIT fashion and you have this same playback pattern, you are almost guaranteed to have seven ffmpeg transcodes all going simultaneously at some point in time and the server probably cannot handle that so they all experience trouble.

[CBers 6771]

 

The current method transcodes at full steam to fill two minutes of playback in the temp server area. Then ffmpeg goes away completely

 

That's the problem.

 

When the ffmpeg process wakes, there is a momentary pause in playback.

 

See my post here.

 

.

Edited June 2, 2015 by CBers

[dark_slayer 103]

Not necessarily, because for example, my server will hit ffmpeg for about fifteen seconds then rest for a minute and 45

[CBers 6771]

Not necessarily, because for example, my server will hit ffmpeg for about fifteen seconds then rest for a minute and 45

 

Did you even read my post ??

 

Just cos yours work, doesn't mean others doesn't.

[MSattler 387]

7 people hitting the server for a transcode at the same time is not going to be smooth with or without throttling unless you have very solid upstream bandwidth from your ISP and a xeon

 

I know you keep saying it was working before but at best that was luck and not consistent or there were less transcoding users back then

 

I hope the option returns for you, but I think you'll still end up with some support calls

It is not just CPU util that can run into a problem. I believe what was discussed previously is you want 2GB of memory per stream. So for 7 users, plus memory for the OS you would want to be between 16-17GB of memory.

Additional as you transcode each of these you could have network/nic utilization issues. And of course the temp files for the transcodings all live on the same drive too.